Power efficiency optimization in throughput-based workloads
阅读说明:本技术 基于吞吐量的工作负载中的功率效率优化 (Power efficiency optimization in throughput-based workloads ) 是由 李奥纳多·德·保拉·罗莎·皮加 塞缪尔·纳夫齐格 伊凡·马托西维奇 因德拉尼·保罗 于 2018-06-19 设计创作,主要内容包括:一种功率管理算法框架提出:1)针对基于吞吐量的工作负载的服务质量(QoS)度量;2)启发法,用以区分吞吐量与对延迟敏感的工作负载;3)一种算法,所述算法结合了所述启发法和所述QoS度量来确定目标频率,以最小化空闲时间并提高功率效率,而不会降低性能。一种管理算法框架能够优化服务器级的基于吞吐量的工作负载中的功率效率,同时仍然为对延迟敏感的工作负载提供所需的性能。通过识别其中一个或多个处理核可以较低的频率(并因此以较低的功率)运行而不会产生明显的负面性能影响的工作负载来实现功率节省。(A power management algorithm framework proposes: 1) a quality of service (QoS) metric for throughput-based workloads; 2) heuristics to distinguish throughput from latency sensitive workloads; 3) an algorithm that combines the heuristics and the QoS metrics to determine a target frequency to minimize idle time and improve power efficiency without degrading performance. A management algorithm framework is able to optimize power efficiency in server-level throughput-based workloads, while still providing the required performance for delay-sensitive workloads. Power savings are achieved by identifying workloads where one or more processing cores may be running at a lower frequency (and therefore at a lower power) without significant negative performance impact.)
1. A method for optimizing power efficiency in a computing device, the method comprising:
determining, by the computing device, processing core activity deviation data for one or more processing cores of the computing device based on processing core activity data from the one or more processing cores; and
adjusting, by the computing device, a frequency of at least one of the one or more processing cores in the computing device based on the processing core activity deviation data.
2. The method of claim 1, further comprising:
determining, by the computing device, whether the processing core activity deviation data indicates a homogenous workload for the one or more processing cores; and
reducing, by the computing device, a frequency of the at least one of the one or more processing cores in response to determining that the processing core activity deviation data indicates the homogenous workload of the one or more processing cores.
3. The method of claim 1, wherein determining the processing core activity deviation data comprises:
determining, by the computing device, a number of the one or more processing cores that are in an active state;
determining, by the computing device, an expected number of the one or more processing cores to be in an active state; and
determining, by the computing device, an activity deviation between the number of the one or more processing cores that are in an active state and the expected number of the one or more processing cores that are to be in an active state.
4. The method of claim 3, wherein determining the number of the one or more processing cores that are in an active state is based on a processing core activity level of the one or more processing cores.
5. The method of claim 4, wherein determining the expected number of the one or more processing cores to be in an active state is based on a binomial distribution that takes into account the number of the one or more processing cores and the processing core activity levels of the one or more processing cores.
6. The method of claim 3, wherein a processing core is determined to be in an active state when a percentage of active cycles in the processing core in a previous interval is greater than a threshold.
7. The method of claim 6, wherein the previous interval is based on a moving average window.
8. The method of claim 2, wherein determining the processing core activity deviation data comprises determining how much a current quality of service (QoS) of the one or more processing cores deviates from a target QoS.
9. A system on a chip, comprising:
a plurality of processing cores; and
processing core frequency adjustment logic coupled to the plurality of processing cores, the processing core frequency adjustment logic configured to:
determining processing core activity deviation data for the plurality of processing cores based on processing core activity data from the plurality of processing cores; and
adjusting a frequency of at least one of the plurality of processing cores based on the determined processing core activity deviation data.
10. The system-on-chip of claim 9, wherein the processing core frequency adjustment logic is further configured to:
determining whether the processing core activity deviation data indicates a homogenous workload for the one or more processing cores; and
reducing a frequency of the at least one of the one or more processing cores in response to determining that the processing core activity deviation data indicates the homogenous workload of the one or more processing cores.
11. The system-on-chip of claim 9, wherein the processing core frequency adjustment logic is configured to determine the processing core activity deviation data by:
determining a number of the one or more processing cores that are in an active state;
determining an expected number of the one or more processing cores to be in an active state; and
determining an activity deviation between the number of the one or more processing cores that are in an active state and the expected number of the one or more processing cores that are to be in an active state.
12. The system-on-chip of claim 11, wherein determining the number of the one or more processing cores that are in an active state is based on a processing core activity level of the one or more processing cores.
13. The system-on-chip of claim 12, wherein determining the expected number of the one or more processing cores to be in an active state is based on a binomial distribution that takes into account the number of the one or more processing cores and the processing core activity levels of the one or more processing cores.
14. The system-on-chip of claim 11, wherein the processing core is determined to be in the active state when a percentage of active cycles in the processing core in a previous interval is greater than a threshold.
15. The system-on-chip as recited in claim 14, wherein the previous interval is based on a moving average window.
16. The system-on-chip of claim 10, wherein the processing core frequency adjustment logic is configured to determine the processing core activity deviation data by determining how much a current quality of service (QoS) of the one or more processing cores deviates from a target QoS.
17. A server, comprising:
a plurality of processing cores;
a network interface; and
processing core frequency adjustment logic coupled to the plurality of processing cores and the network interface, the processing core frequency adjustment logic configured to:
determining processing core activity deviation data for the plurality of processing cores based on processing core activity data from the plurality of processing cores; and
adjusting a frequency of at least one of the plurality of processing cores based on the determined processing core activity deviation data.
18. The server of claim 17, wherein the processing core frequency adjustment logic is further configured to:
determining whether the processing core activity deviation data indicates a homogenous workload for the one or more processing cores; and
reducing a frequency of the at least one of the one or more processing cores in response to determining that the processing core activity deviation data indicates the homogenous workload of the one or more processing cores.
19. The server of claim 18, wherein the processing core frequency adjustment logic is configured to determine the processing core activity deviation data by:
determining a number of the one or more processing cores that are in an active state;
determining an expected number of the one or more processing cores to be in an active state; and
determining an activity deviation between the number of the one or more processing cores that are in an active state and the expected number of the one or more processing cores that are to be in an active state.
20. The server of claim 19, wherein a processing core is determined to be in an active state when a percentage of active cycles in the processing core in a previous interval is greater than a threshold, wherein the previous interval is based on a moving average window.
Background
With various load balancing techniques, many server workloads exhibit stable homogenous load behavior at a fixed workload. These workloads are typically not delay critical and there are no data dependent threads, i.e., the time to process a single request is insignificant as long as the average throughput remains constant. System-on-chip (SoC) power management algorithms need to identify these throughput-based workloads so that frequency can be reduced to minimize idle time without degrading performance, thereby achieving optimal power efficiency. Existing power management algorithms either look at the behavior of individual instruction processors (e.g., a single processor) or incorporate temporary solutions for modeling the interaction between the SoC and the workload, making it unable to distinguish between throughput and latency critical workloads and resulting in sub-optimal operating points.
For example, a set of local optimization algorithms is unlikely to achieve a globally optimal operating state for a particular quality of service (QoS). Instead, this solution would result in poor performance and power efficiency. Thus, as socs become more complex, it becomes increasingly important to know that socs and workloads interact and manage power in a scalable manner while optimizing for global QoS targets.
Drawings
The present disclosure will become more readily understood from the following description taken in conjunction with the accompanying drawings, wherein like reference numerals designate like elements, and in which:
FIG. 1 is a functional block diagram illustrating an example computing device (e.g., a server) that includes processing core frequency adjustment logic that may adjust the frequency of one or more processing cores in accordance with one example set forth in the present disclosure;
FIG. 2 is a more detailed functional block diagram of the processing core frequency adjustment logic of FIG. 1 according to one example set forth in the present disclosure;
FIG. 3 is a schematic diagram indicating a CPU activity diagram for a single processing core in accordance with one example set forth in the present disclosure; and
fig. 4 is a flow diagram of an example method for measuring QoS for executing a workload in accordance with one example set forth in the disclosure.
Detailed Description
Briefly, methods and apparatus provide power efficiency optimization in throughput-based workloads. In one example, a method for optimizing power efficiency in a computing device is described. For example, the computing device may be a server. The computing device determines processing core activity deviation data for a processing core (e.g., CPU, GPU, etc.) based on processing core activity data from the processing core in the computing device. The computing device adjusts a frequency of at least one of the processing cores in the computing device based on the processing core activity deviation data.
The computing device determines whether the processing core activity deviation data indicates a homogenous workload for the processing core. In response to determining that the processing core activity deviation data indicates a homogenous workload for the processing cores, the computing device decreases the frequency of at least one of the processing cores.
In determining the processing core activity deviation data, the computing device determines a number of processing cores that are in an active state and an expected number of processing cores that are to be in an active state. The computing device then determines an activity deviation between the number of processing cores that are in the active state and an expected number of processing cores that will be in the active state.
Determining the number of processing cores that are in an active state is based on the processing core activity level of the processing core. Determining the expected number of processing cores to be in an active state is based on a binomial distribution that takes into account the number of processing cores and the processing core activity levels of the processing cores. Determining that the processing core is in the active state when a percentage of active cycles in the processing core in a previous interval is greater than a threshold, wherein the previous interval is based on a moving average window. When determining the processing core activity deviation data, the computing device determines how much the current QoS of the processing core deviates from the target QoS.
In one example, a system on a chip or SoC includes a plurality of processing cores and processing core frequency adjustment logic coupled to the plurality of processing cores. The processing core frequency adjustment logic determines processing core activity deviation data for the plurality of processing cores based on the processing core activity data from the plurality of processing cores and adjusts a frequency of at least one of the plurality of processing cores based on the determined processing core activity deviation data.
In another example, a server includes a plurality of processing cores, a network interface, and processing core frequency adjustment logic coupled to the plurality of processing cores and the network interface. The processing core frequency adjustment logic determines processing core activity deviation data for the plurality of processing cores based on the processing core activity data from the plurality of processing cores and adjusts a frequency of at least one of the plurality of processing cores based on the determined processing core activity deviation data.
The present disclosure describes a power management algorithm framework that proposes: 1) QoS metrics for throughput-based workloads; 2) heuristics to distinguish throughput from latency sensitive workloads; 3) an algorithm that combines heuristics and QoS metrics to determine a target frequency to minimize idle time and improve power efficiency without degrading performance.
The present disclosure provides a management algorithm framework that is able to optimize power efficiency in server-level throughput-based workloads, while still providing the desired performance for latency-sensitive workloads. The present disclosure is directed to achieving power savings by identifying workloads where one or more processing cores may be running at a lower frequency (and therefore at a lower power) without significant negative performance impact. The present disclosure addresses at least two of the following issues to make this technology practical: 1) how to identify workloads whose performance has low frequency sensitivity; 2) how much the frequency can be safely reduced before the performance is unduly degraded.
The present disclosure proposes a solution, in particular for: (1) differentiating workloads of a homogenous, stable class with independent threads, each thread executing a fixed amount of work from workloads with data-dependent threads, wherein data-dependent threads are latency critical; and 2) optimizing power efficiency by reducing idle time in throughput-oriented workloads.
For example, a particular QoS algorithm has a particular "target Central Processing Unit (CPU) idle percentage" (between 0% and 100%) as a parameter, and modulates the CPU frequency to achieve the QoS target. If the CPU idle time is greater than the threshold, the frequency is reduced and vice versa. For throughput-based loads, ideally, the system would like to reduce the idle time to zero, thereby eliminating all slack time provided by the idle time. However, in practice, this is not possible, as the system will need to take into account the granularity of changes in workload behavior and changes in power state. Thus, the actual target idle time is left as an adjustable configuration parameter to optimize for the balance between power savings and performance degradation.
The QoS metric for system idle time can be calculated using the following formula:
where N is the total number of processing cores in the system, the idle period for core i is the sum of the non-C0 (i.e., CC1 and CC6 cycles) for the ith core, and the total period is the total number of CPU cycles (including active and idle) at the current CPU frequency during the Synchronous Management Unit (SMU) sample time (lms).
The algorithm makes some assumptions about the application behavior.
1) The workload is based entirely on throughput and the delay in processing a single request is of no consequence to performance as long as the average throughput remains constant. Thus, the QoS algorithm will not turn on for any workload that similarly has alternating periods of active and idle CPU time, but where the active periods are delay critical. For example, if an idle period is spent waiting on input/output (I/O) and there is a data dependency between the computation and the I/O operation.
2) The algorithm does not necessarily suggest separate per-core control. Instead, it assumes a homogenous load for which the same frequency is appropriate for all processing cores. For server workloads (usually with load balancing), this is a realistic assumption. However, the algorithm should not be turned on for client workloads with, for example, a small number of threads, which would otherwise result in a single thread performance degradation.
3) The load level is stable enough to change slowly with respect to the feedback response time of the QoS algorithm. For example, if the load suddenly increases with the frequency set to target a low idle time and a much lower load, the system will get lower performance until a feedback response is received.
Since the QoS algorithm requires specific conditions to work properly, if it is turned on for an application that does not satisfy these conditions, there is a risk of degrading performance, and it is therefore necessary to adopt a conservative heuristic when it is turned on.
Kernel activity distribution-heuristic to differentiate throughput to handle delay-sensitive workloads
A further observation with respect to throughput-based homogeneous and stable loads is that the distribution of the number of processing cores active at a given time closely follows a binomial distribution. The number of active cores is defined as those whose percentage of active cycles in the previous short interval is above a high threshold (e.g., 90%). This is because the following characteristics can be observed for such a load.
1) Since the load is stable over time, the probability of a single processing core being in an active state at different times T1 and T2 is approximately equal. For example, at a load of 60%, any given core is active for approximately 60% of the time.
2) Since the load is homogenous and balanced, the probability is the same for all processing cores.
3) The probability that any two processing cores are in an active state at a given point in time is irrelevant.
Thus, as shown below, when N processing cores are observed at a given point in time, the expected number of active processing cores is given by the binomial distribution and probability A of N trials, which is equal to the activity level.
For example, if the activity level is 60% and the workload is running on 4 processing cores, then the expectation that exactly 3 processing cores are active for a period of time is equal to
I.e., about 34.6% of the time.By measuring the actual distribution of the number of active processing cores and comparing it to the theoretical binomial distribution (where N is the number of processing cores and a is the average activity level measured over the measurement time interval), the degree to which an application deviates from the type for which idle QoS is intended can be estimated. Specifically, this measurement will capture the following.
1) Deviations from a stable homogenous and balanced load can result in a bias in the distribution towards a particular processing core.
2) Deviation from a load based entirely on throughput, in which the individual work units are not correlated.
Only in case the workload is stable and homogenous, property 2) will add value with respect to the measurements. For example, consider a workload where a single thread, fully constrained by the CPU, runs without CPU affinity, and is therefore scheduled across N processing cores in a round-robin fashion. The workload appears stable and homogenous, with each processing core active for 1/N of the time. However, enabling idle QoS may degrade performance because it is not throughput-based, but each time slice executed on the CPU depends on the previous time slice.
This situation was successfully identified by the binomial distribution heuristic, since the distribution would show that only one processing core is active 100% of the time (instead of a binomial distribution where a is 1/N). Although this is a simple example, similar reasoning shows that for any workload that provides a range of dependent computations, reducing the CPU frequency lengthens the critical path and reduces performance, so the distribution will deviate from binomial because the time that a single processing core is active is relevant.
QoS on/off algorithm
To avoid performance degradation, the algorithm should be turned on only for workloads having the characteristics described in the previous section. The method is based on the following assumptions: some of the observed regularity for homogenous (load balanced) throughput-based applications is not observed for any other application type. Specifically, the algorithm will turn on QoS when the following is observed.
1) Each processing core has a stable average load over a period of tens of seconds scale.
2) All processing cores are busy with approximately the same average load.
3) The distribution of processing core activity is approximately a binomial distribution.
For a stable load level, it needs to be longTime scales, since in practice too much variation is observed for shorter time scales. For example, consider benchmark test programs from measuring server power and performance characteristics (as provided by the Standard Performance Evaluation Corporation (SPEC))
Benchmark), because the length of the active interval varies between a few milliseconds and a few hundred milliseconds, an interval of only one or two seconds will still have many random variations in how much active time and idle time it contains.This makes the technique effective for workloads that exhibit a steady load level on a scale of a few minutes. Typical server benchmarking (as provided by SPEC)
Or SERTTMBenchmark) are the cases that measure steady throttle load levels in a few minutes and have a longer warm-up interval at each given level before the measurement begins.To detect a stable average load, a moving average of activity may be used, but given the time resolution (lms) of the samples, this would require a large amount of memory. Thus, the calculation of common averages within a 1s scale may also be used, and a moving average window of these averages may be used. The size of the base averaging interval (1s), the moving average window (tens of seconds), the time threshold for stabilizing the workload (tens of seconds), and the allowed variation (within a single processing core, between processing cores, and the difference from the binomial distribution) are all adjustable parameters. Similarly, the algorithm will not be used when these conditions no longer apply and the load variation within and between processing cores (calculated in the same way) exceeds a given threshold.
Turning to the drawings, one example of the presently disclosed
As shown in fig. 1,
As shown in FIG. 1, the processing core
In some embodiments, some or all of the functionality of
The
Fig. 2 is a more detailed functional block diagram of the processing core
The processing core
The processing core workload
Processing core
To illustrate operation of an example algorithm as disclosed herein, FIG. 3 shows running a benchmark test program (e.g., as provided by SPEC)
Benchmark) with a workload at a load level of 50%. Although fig. 3 shows only one processing core for clarity, the load is homogenous and may look similar for other processing cores. Each point in fig. 3 shows the percentage of active (i.e., C0) cycles in a 1ms sample and plots the total period of two seconds. From fig. 3, it is evident that this workload keeps the CPU fully busy while processing the request, so periods of 100% C0 activity alternate with idle periods. In this case, approximately 50% of the time is idle, which reflects the load level at this stage in the benchmarking test. Benchmark programs (e.g. as provided by SPEC)Benchmark) controls the load by measuring the maximum throughput of the system and then adjusts the request rate between 0% and 100% in order to measure the way power consumption varies with load.For throughput-based loads like this, if the processing core utilization is significantly below 100%, the processing core can be reduced while maintaining the same throughput, thereby reducing the percentage of idle time. This is the basis of the idle time QoS algorithm.
Fig. 4 provides a
As shown in FIG. 4, at
The foregoing detailed description and examples described herein have been presented for purposes of illustration and description only and are not intended to be limiting. For example, the described operations may be performed in any suitable manner. It is therefore contemplated that the embodiments herein cover any and all modifications, variations or equivalents that fall within the scope of the basic underlying principles disclosed above and claimed herein. Further, while the above describes the processor executing hardware in the form of code, state machines, or dedicated logic producing the same results, other configurations are also contemplated.
- 上一篇:一种医用注射器针头装配设备
- 下一篇:一种降低耗电的方法和设备