Abstract
Cloud data centers try to achieve two primary but competing goals for latency-sensitive services: meeting latency service-level objective (SLO) and reducing energy utilization. In this paper, we perform a joint optimization of both goals, such that given a tail latency constraint and request rate, the number of active servers and their corresponding core frequencies which cause the least energy overhead can be derived. Both the server sleep mode and setup delay are taken into account. Based on the assumption that requests arrive following a Poisson process, we provide a numerical method to compute the optimal number of active servers and a closed-form expression for their corresponding core frequency. We propose how the optimization technique can be used for real, bursty, non-Poisson data center traffic.
![](https://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs42979-019-0030-z/MediaObjects/42979_2019_30_Fig1_HTML.png)
![](https://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs42979-019-0030-z/MediaObjects/42979_2019_30_Fig2_HTML.png)
![](https://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs42979-019-0030-z/MediaObjects/42979_2019_30_Fig3_HTML.png)
![](https://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs42979-019-0030-z/MediaObjects/42979_2019_30_Fig4_HTML.png)
![](https://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs42979-019-0030-z/MediaObjects/42979_2019_30_Fig5_HTML.png)
![](https://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs42979-019-0030-z/MediaObjects/42979_2019_30_Fig6_HTML.png)
![](https://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs42979-019-0030-z/MediaObjects/42979_2019_30_Fig7_HTML.png)
![](https://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs42979-019-0030-z/MediaObjects/42979_2019_30_Fig8_HTML.png)
![](https://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs42979-019-0030-z/MediaObjects/42979_2019_30_Fig9_HTML.png)
Similar content being viewed by others
Change history
28 September 2023
A Correction to this paper has been published: https://doi.org/10.1007/s42979-023-02168-3
Notes
The values of \(s_b\), \(s_c\), b, and c may be different under these different models, we do not differentiate them in the model equation here. However, their practical values will be used, respectively, in the numerical results.
References
Barroso LA, Hölzle U. The case for energy-proportional computing. Computer. 2007;40(12):33–7.
Beloglazov A, Abawajy J, Buyya R. Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Future Gener Comput Syst. 2012;28(5):755–68.
Dean J, Barroso LA. The tail at scale. Commun ACM. 2013;56(2):74–80.
Gandhi A, Harchol-Balter M, Das R, Lefurgy C Optimal power allocation in server farms. In: ACM SIGMETRICS performance evaluation review. 2009; vol. 37, pp. 157–168. ACM
Gandhi A, Harchol-Balter M, Raghunathan R, Kozuch MA. Autoscale: dynamic, robust capacity management for multi-tier data centers. ACM Trans Comput Syst (TOCS). 2012;30(4):14.
Gelenbe E, Lent R. Optimising server energy consumption and response time. Theor Appl Informatics. 2012;24(4):257–70.
Gelenbe E, Lent R. Trade-offs between energy and quality of service. In: 2012 sustainable internet and ICT for sustainability (SustainIT), pp. 1–5. IEEE. 2012.
Gelenbe E, Lent R, Douratsos M. Choosing a local or remote cloud. In: 2012 Second Symposium on Network Cloud Computing and Applications, pp. 25–30. IEEE. 2012.
Grandl R, Kandula S, Rao S, Akella A, Kulkarni J. Graphene: packing and dependency-aware scheduling for data-parallel clusters. In: Proc. of the USENIX OSDI. 2016.
Kapoor R, Porter G, Tewari M, Voelker G.M, Vahdat A. Chronos: predictable low latency for data center applications. In: Proceedings of the Third ACM Symposium on Cloud Computing, p. 9. ACM. 2012.
Kasture H, Bartolini DB, Beckmann N, Sanchez DR. Fast analytical power management for latency-critical systems. In: Proceedings of the 48th International Symposium on Microarchitecture, pp. 598–610. ACM. 2015.
Kasture H, Bartolini DB, Beckmann N, Sanchez DR. Fast analytical power management for latency-critical systems. In: Proceedings of the 48th International Symposium on Microarchitecture-MICRO-48. 2015. https://doi.org/10.1145/2830772.2830797.
Kliazovich D, Bouvry P, Khan SU. Dens: data center energy-efficient network-aware scheduling. Cluster Comput. 2013;16(1):65–75.
Lin M, Wierman A, Andrew LL, Thereska E. Dynamic right-sizing for power-proportional data centers. IEEE/ACM Trans Netw (TON). 2013;21(5):1378–91.
Lo D, Cheng L, Govindaraju R, Barroso L.A, Kozyrakis C. Towards energy proportionality for large-scale latency-critical workloads. In: ACM SIGARCH Computer Architecture News, vol. 42, pp. 301–312. IEEE Press. 2014.
Lo D, Cheng L, Govindaraju R, Ranganathan P, Kozyrakis CH. Improving resource efficiency at scale. In: ACM SIGARCH computer architecture news, vol. 43, pp. 450–462. ACM. 2015.
Pinckney N, Fojtik M, Giridhar B, Sylvester D, Blaauw D. Shortstop: an on-chip fast supply boosting technique. In: VLSI Circuits (VLSIC), 2013 Symposium on pp. C290–C291. IEEE. 2013.
Prekas G, Primorac M, Belay A, Kozyrakis C, Bugnion E. Energy proportionality and workload consolidation for latency-critical applications. In: Proceedings of the Sixth ACM symposium on cloud computing, pp. 342–355. ACM. 2015.
Sarood O, Langer A, Gupta A, Kale L. Maximizing throughput of overprovisioned hpc data centers under a strict power budget. In: Proceedings of the international conference for high performance computing, networking, storage and analysis, pp. 807–818. IEEE Press. 2014.
Sarood O, Miller P, Totoni E, Kale LV. cool load balancing for high performance computing data centers. IEEE Trans Comput. 2012;61(12):1752–64.
Srikantaiah S, Kansal A, Zhao F. Energy aware consolidation for cloud computing. In: Proceedings of the 2008 conference on Power aware computing and systems, vol. 10, pp. 1–5. San Diego, California. 2008.
Takagi H. A Foundation of performance evaluation: vacation and priority systems. Amsterdam: Elsevier; 1991.
Vamanan B, Sohail H.B, Hasan J, Vijaykumar T. Timetrader: exploiting latency tail to save datacenter energy for online search. In: Proceedings of the 48th international symposium on microarchitecture, pp. 585–597. ACM. 2015.
Wierman A, Andrew LL, Tang A. Power-aware speed scaling in processor sharing systems. In: INFOCOM 2009, IEEE, pp. 2007–2015. IEEE. 2009.
Wilkes J. More google cluster data. Google research blog. 2011.
Yang H, Chen Q, Riaz M, Luan Z, Tang L, Mars J. Powerchief: intelligent power allocation for multi-stage applications to improve responsiveness on power constrained cmp. In: Proceedings of the 44th Annual International Symposium on Computer Architecture, pp. 133–146. ACM. 2017.
Zhu H, Erez M. Dirigent: enforcing qos for latency-critical tasks on shared multicore systems. ACM SIGPLAN Notices. 2016;51(4):33–47.
Acknowledgements
This research was supported by NSF Grant CNS-1528087.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is part of the topical collection “Modelling methods in Computer Systems, Networks and Bioinformatics” guest edited by Erol Gelenbe.
Appendices
Appendix A: Proof of Proposition 1
The pdf of response time is given by:
based on which the tail performance is calculated through \(P(T>x)=\int _x^\infty p(t)dt\). Moreover, the distribution function is continuous, since
Appendix B: Proof of Proposition 2
If \(\mu -\lambda >\alpha \), we have \(e^{-(\mu -\lambda )D_0}-e^{-\alpha D_0}<0\), and thus, \(\frac{\alpha }{\mu -\lambda -\alpha }[e^{-(\mu -\lambda )D_0}-e^{-\alpha D_0}]<0\); if \(\mu -\lambda <\alpha \), there is \( e^{-(\mu -\lambda )D_0}-e^{-\alpha D_0}>0\), and \(\frac{\alpha }{\mu -\lambda -\alpha }[e^{-(\mu -\lambda )D_0}-e^{-\alpha D_0}]<0\). As a result, a sufficient condition to satisfy the QoS (tail delay requirement) constraint is \(e^{-\alpha D_0}-\epsilon < 0\), which equals \(\alpha >\frac{1}{D_0}\ln \frac{1}{\epsilon }\).
Before simplifying the delay constraint, assume that \(\varGamma =\frac{\alpha D_0}{e^{-\alpha D_0}-\epsilon }<0\), which can be transformed into \(\varGamma e^{\epsilon \varGamma }=e^{\alpha D_0+\epsilon \varGamma }(\alpha D_0+\epsilon \varGamma )\). Based on this equation, we have:
where \(\alpha D_0+\epsilon \varGamma \ge -1\) in the first case, and \(\alpha D_0+\epsilon \varGamma <-1\) otherwise.
-
(1)
If \(\mu -\lambda >\alpha \):
$$\begin{aligned}&P(T>D_0)\le \epsilon \end{aligned}$$(19)$$\begin{aligned}&\Longleftrightarrow \frac{\mu -\lambda }{\mu -\lambda -\alpha }e^{-\alpha D_0}-\frac{\alpha }{\mu -\lambda -\alpha }e^{-(\mu -\lambda )D_0}\le \epsilon \end{aligned}$$(20)$$\begin{aligned}&\Longleftrightarrow e^{-\alpha D_0}-\epsilon +\frac{\epsilon \alpha }{\mu -\lambda }\le \frac{\alpha }{\mu -\lambda }e^{-(\mu -\lambda )D_0} \end{aligned}$$(21)$$\begin{aligned}&\Longleftrightarrow e^{(\mu -\lambda )D_0}\left[(\mu -\lambda )(e^{-\alpha D_0}-\epsilon )+\epsilon \alpha \right]\le \alpha \end{aligned}$$(22)$$\begin{aligned}&\Longleftrightarrow e^{(\mu -\lambda )D_0} \left[(\mu -\lambda )D_0+\frac{\epsilon \alpha D_0}{e^{-\alpha D_0}-\epsilon }\right]\ge \frac{\alpha D_0}{e^{-\alpha D_0}-\epsilon } \end{aligned}$$(23)$$\begin{aligned}&\Longleftrightarrow e^{[(\mu -\lambda )D_0+\epsilon \varGamma ]}[(\mu -\lambda )D_0+\epsilon \varGamma ]\ge \varGamma e^{\epsilon \varGamma }. \end{aligned}$$(24)
-
When \(\alpha \ge \frac{1}{D_0}(-\mathbf {W}_{-1}[-\frac{\epsilon }{e}]-1)\), we have \((\mu -\lambda )D_0+\epsilon \varGamma >\alpha D_0+\epsilon \varGamma \ge -1\), and thus, \(P(T>D_0)\le \epsilon \) equals to \((\mu -\lambda )D_0+\epsilon \varGamma \ge \mathbf {W}_{0}[\varGamma e^{\epsilon \varGamma }]=\alpha D_0+\epsilon \varGamma \), which is \(\mu \ge \lambda +\alpha \).
-
When \(\alpha < \frac{1}{D_0}(-\mathbf {W}_{-1}[-\frac{\epsilon }{e}]-1)\), which means \(\alpha D_0+\epsilon \varGamma < -1\), the constraint equals \((\mu -\lambda )D_0+\epsilon \varGamma \ge \mathbf {W}_{0}[\varGamma e^{\epsilon \varGamma }]\) or \((\mu -\lambda )D_0+\epsilon \varGamma \le \mathbf {W}_{-1}[\varGamma e^{\epsilon \varGamma }]\). For the first case, we have \( \mu \ge \lambda +\frac{1}{D_0}(\mathbf {W}_{0}[\varGamma e^{\epsilon \varGamma }]-\epsilon \varGamma )>\lambda +\frac{1}{D_0}(\mathbf {W}_{-1}[\varGamma e^{\epsilon \varGamma }]-\epsilon \varGamma )=\lambda +\alpha \). For the second case, there is \(\mu \le \lambda +\frac{1}{D_0}(\mathbf {W}_{-1}[\varGamma e^{\epsilon \varGamma }]-\epsilon \varGamma )=\lambda +\alpha \), which contradicts with the condition \(\mu -\lambda >\alpha. \)
Therefore, the delay constraint equals: \(\mu >\lambda +\alpha \) when \(\alpha \ge \frac{1}{D_0}(-\mathbf {W}_{-1}[-\frac{\epsilon }{e}]-1)\), and \(\mu \ge \lambda +\frac{1}{D_0}(\mathbf {W}_{0}[\varGamma e^{\epsilon \varGamma }]-\epsilon \varGamma )\) when \(\alpha < \frac{1}{D_0}(-\mathbf {W}_{-1}[-\frac{\epsilon }{e}]-1)\).
-
(2)
Similarly, if \(\mu -\lambda <\alpha \), we can obtain that
$$\begin{aligned}&P(T>D_0)\le \epsilon \end{aligned}$$(25)$$\begin{aligned}&\Longleftrightarrow e^{[(\mu -\lambda )D_0+\epsilon \varGamma ]}[(\mu -\lambda )D_0+\epsilon \varGamma ]\le \varGamma e^{\epsilon \varGamma }. \end{aligned}$$(26)
-
When \(\alpha \ge \frac{1}{D_0}(-\mathbf {W}_{-1}[-\frac{\epsilon }{e}]-1)\), the constraint equals \(\mathbf {W}_{-1}[\varGamma e^{\epsilon \varGamma }] \le (\mu -\lambda )D_0+\epsilon \varGamma \le \mathbf {W}_{0}[\varGamma e^{\epsilon \varGamma }]\), which is \(\lambda +\frac{1}{D_0}(\mathbf {W}_{-1}[\varGamma e^{\epsilon \varGamma }]-\epsilon \varGamma ) \le \mu \le \lambda +\alpha \).
-
When \(\alpha < \frac{1}{D_0}(-\mathbf {W}_{-1}[-\frac{\epsilon }{e}]-1)\), \((\mu -\lambda )D_0+\epsilon \varGamma<\alpha D_0+\epsilon \varGamma <-1\), and thus, \((\mu -\lambda )D_0+\epsilon \varGamma \ge \mathbf {W}_{-1}[\varGamma e^{\epsilon \varGamma }]=\alpha D_0+\epsilon \varGamma \), which is \(\mu -\lambda \ge \alpha \). This contradicts with the condition \(\mu -\lambda <\alpha \).
Therefore, the delay constraint equals: \(\lambda +\frac{1}{D_0}(\mathbf {W}_{-1}[\varGamma e^{\epsilon \varGamma }]-\epsilon \varGamma ) \le \mu <\lambda +\alpha \) when \(\alpha \ge \frac{1}{D_0}(-\mathbf {W}_{-1}[-\frac{\epsilon }{e}]-1)\):
-
3.
If \(\mu -\lambda =\alpha \),
$$\begin{aligned}&P(T>D_0)\le \epsilon \end{aligned}$$(27)$$\begin{aligned}&\Longleftrightarrow \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!(1+\alpha D_0)e^{-\alpha D_0}\le \epsilon \end{aligned}$$(28)$$\begin{aligned}&\Longleftrightarrow \!\!\!\!\!-(1+\alpha D_0)e^{-(1+\alpha D_0)}\ge -\frac{\epsilon }{e} \end{aligned}$$(29)$$\begin{aligned}&\Longleftrightarrow \!\!\!\!\!\!\!\!\!\!\!\!\!\!-(1+\alpha D_0)\le \mathbf {W}_{-1}\left[-\frac{\epsilon }{e}\right] \end{aligned}$$(30)$$\begin{aligned}&\Longleftrightarrow \alpha \ge \frac{1}{D_0}\left(-\mathbf {W}_{-1}\left[-\frac{\epsilon }{e}\right]-1 \right)>0. \end{aligned}$$(31)
Combining the above three cases, the delay constraint equals to:
Here, \(\frac{1}{D_0}\ln \frac{1}{\epsilon } < \alpha \) is the sufficient condition to satisfy the delay constraint. Since \(\ln \epsilon <0\), there is \(-\frac{\epsilon }{e}>(\ln \epsilon -1)e^{\ln \epsilon -1}\), and thus, \(\mathbf {W}_{-1}[-\frac{\epsilon }{e}]<\ln \epsilon -1\). As a result, it is readily to prove that \(\frac{\ln \frac{1}{\epsilon }}{D_0}<\frac{1}{D_0}(-\mathbf {W}_{-1}[-\frac{\epsilon }{e}]-1)\).
Appendix C: Proof of Proposition 5
We prove Proposition 4 under the quadratic power consumption model here. The proof for the linear power consumption model, which is much easier, can be obtained similarly.
Given the number of servers in use K, the proof is divided into two steps. (1) First, given the routing variable \(\mathbf {p}\), the service of each server is independent with each other, and thus, we focus on solving the optimal frequency of each server with arrival rate \(p_i\lambda \), which is also the proof of Proposition 3. (2) Second, substituting the optimal frequency into Eq. (2), the power consumption of each server is a function of its routing parameter \(p_i\), which is denoted as \(P(p_i)\). Our objective is to prove that \(P(p_i)\) is a convex function, and thus, it satisfies that, for any \([p_1, p_2, ...,p_K]\) with \(\sum _{i=1}^{K}p_i=1\), \(P(p_1) + P(p_2) + ... +P(p_K) \ge KP(\frac{1}{K})\).
Step 1 The power consumption of single server with quadratic model can be simplified as:
which is a convex function, and has the extreme point
The constraint on the frequency is \(\max \{s_b, (p_i\lambda +X)l\}\le s_i \le s_c\), where \(X = \frac{1}{D_0}(\mathbf {W}_{-1}[\varGamma e^{\epsilon \varGamma }]-\epsilon \varGamma )\) when \(\alpha \ge \frac{1}{D_0}(-\mathbf {W}_{-1}[-\frac{\epsilon }{e}]\!-\!1)\), and \(X = \frac{1}{D_0}(\mathbf {W}_{0}[\varGamma e^{\epsilon \varGamma }]-\epsilon \varGamma )\) when \(\frac{1}{D_0}\ln \frac{1}{\epsilon }<\alpha < \frac{1}{D_0} (-\mathbf {W}_{-1}[-\frac{\epsilon }{e}]-1)\).
-
1.
When \(s_b\ge (p_i\lambda +X)l\), the constraint on the frequency is \(s_b \le s_i \le s_c\), the optimal frequency that minimizes the power consumption is:
$$\begin{aligned} s^*=\left\{ \begin{array}{ll} s_c, \quad&{} \hbox {if }s_e > s_c; \\ s_e, \quad&{} \hbox {if }s_e \le s_c. \end{array} \right. \end{aligned}$$ -
2.
When \(s_b < (p_i\lambda +X)l\), the constraint on the frequency is \((p_i\lambda +X)l\le s_i \le s_c\), and the optimal frequency that minimizes the power consumption is:
$$\begin{aligned} s^*=\left\{ \begin{array}{ll} s_c, \quad&{} \hbox {if }s_e > s_c; \\ (p_i\lambda +X)l, \quad&{} \hbox {if }s_e < (p_i\lambda +X)l;\\ s_e, \quad&{} \hbox {if }(p_i\lambda +X)l \le s_e \le s_c. \end{array} \right. \end{aligned}$$
Combining the upper two cases, given \(p_i\), the optimal frequency that minimizes the power consumption \(P(s_i, p_i)\) in Eq. (32) under the constraint \(\max \{s_b, (p_i\lambda +X)l\}\le s_i \le s_c\) is:
Step 2: Substituting the optimal frequency into Eq. (32), the power consumption of each server \(P(p_i)\) is given in Eq. (34):
(1) When \(s_e =\sqrt{(b-P_s){k_2}^2+{s_b}^2} > s_c \) \(P(p_i) = P_s+\frac{p_i\lambda l}{s_c}\left[ b-P_s+\frac{(s_c-s_b)^2}{k^2_2}\right] \) is a linear function of \(p_i\), and thus, it is convex.
(2) When \(s_e =\sqrt{(b-P_s){k_2}^2+{s_b}^2} \le s_c\), \(P(p_i)\) is a continuous function composed of two segments, with the junction point \(p_i = (\frac{s_e}{l} - X)/\lambda \). For the left segment with \(p_i \le (\frac{s_e}{l} - X)/\lambda \), \(P_l(p_i)=P_s+\frac{p_i\lambda l}{s_e}\left[ b-P_s+\frac{(s_e-s_b)^2}{k^2_2}\right] \) is a linear function of \(p_i\). For the right segment with \(p_i > (\frac{s_e}{l} - X)/\lambda \), it is readily to prove that \(P_r(p_i)=P_s+\frac{p_i\lambda l}{p_i\lambda l+Xl}\left[ b-P_s+\frac{(p_i\lambda l+Xl-s_b)^2}{k^2_2}\right] \) is an increasing convex function of \(p_i\). Moreover, there is:
As a result, \(P(p_i)\) is also convex when \(s_e \le s_c\).
Combining the upper two cases, \(P(p_i)\) is a convex function, and thus, it satisfies that, for any \([p_1, p_2, ...,p_K]\) with \(\sum _{i=1}^{K}p_i=1\), there is always \(P(p_1) + P(p_2) + ... +P(p_K) \ge KP(\frac{1}{K})\). This means that given the number of servers in use K, the optimal routing policy that minimizes the total power consumption is to evenly distribute the load among the K severs.
Rights and permissions
About this article
Cite this article
Leh, G.S., Wu, J., Shukla, S. et al. Model-Driven Joint Optimization of Power and Latency Guarantee in Data center Applications. SN COMPUT. SCI. 1, 34 (2020). https://doi.org/10.1007/s42979-019-0030-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s42979-019-0030-z