A Survey on Green-Energy-Aware Power Management for Datacenters

Abstract

Megawatt-scale datacenters have emerged to meet the increasing demand for IT applications and services. The hunger for power brings large electricity bills to datacenter operators and causes significant impacts to the environment. To reduce costs and environmental impacts, modern datacenters, such as those of Google and Apple, are beginning to integrate renewable or green energy sources into their power supply. This article investigates the green-energy-aware power management problem for these datacenters and surveys and classifies works that explicitly consider renewable energy and/or carbon emission. Our aim is to give a full view of this problem. Hence, we first provide some basic knowledge on datacenters (including datacenter components, power infrastructure, power load estimation, and energy sources' operations), the electrical grid (including dynamic pricing, power outages, and emission factor), and the carbon market (including cap-and-trade and carbon tax). Then, we categorize existing research works according to their basic approaches used, including workload scheduling, virtual machine management, and energy capacity planning. Each category's discussion includes the description of the shared core idea, qualitative analysis, and quantitative analysis among works of this category.

References

1. Zahra Abbasi, Georgios Varsamopoulos, and Sandeep K. S. Gupta. 2012. TACOMA: Server and workload management in internet data centers considering cooling-computing power trade-off and energy proportionality. ACM Transactions on Architecture and Code Optimization 9, 2 (2012), 11
2. Aerostar. 2008. Wind Turbines. Retrieved June 2013 from http://www.aerostarwind.com/power.html.
3. Sherif Akoush, Ripduman Sohan, Andrew Rice, Andrew W. Moore, and Andy Hopper. 2011. Free lunch: Exploiting renewable energy for computing. In Proceedings of HotOS. 17.
4. Baris Aksanli, Jagannathan Venkatesh, Liuyi Zhang, and Tajana Rosing. 2011. Utilizing green energy prediction to schedule mixed batch and service jobs in data centers. ACM SIGOPS Operating Systems Review 45, 3 (2012), 53--57.
5. Apple. 2013. Apple and the Environment. Retrieved June 2013 from http://www.apple.com/environment/ renewable-energy/.
6. Apple. 2014. Apple and the Environment. Retrieved January 2014 from http://www.apple.com/environment/ renewable-energy/.
7. Martin Arlitt, Cullen Bash, Sergey Blagodurov, Yuan Chen, Tom Christian, Daniel Gmach, Chris Hyser, Niru Kumari, Zhenhua Liu, Manish Marwah, et al. 2012. Towards the design and operation of net-zero energy data centers. In IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm). IEEE, 552--561.
8. M. F. Bari, Raouf Boutaba, Rafael Esteves, L. Z. Granville, Maxim Podlesny, M. G. Rabbani, Qi Zhang, and M. F. Zhani. 2012. Data center network virtualization: A survey. IEEE Communications Surveys & Tutorials 15, 2 (2013), 909--928.
9. Luiz André Barroso. 2005. The price of performance. Queue 3, 7 (2005), 48--53.
10. Luiz Andr Barroso and Urs Holzle. 2007. The case for energy-proportional computing. IEEE Computer 40, 12 (2007), 33--37.
11. Cullen Bash and George Forman. 2007. Cool job allocation: Measuring the power savings of placing jobs at cooling-efficient locations in the data center. In USENIX Annual Technical Conference. 140.
12. Anton Beloglazov, Rajkumar Buyya, Young Choon Lee, Albert Zomaya, et al. 2011. A taxonomy and survey of energy-efficient data centers and cloud computing systems. Advances in Computers 82, 2 (2011), 47--111.
13. Aruna Prem Bianzino, Claude Chaudet, Dario Rossi, and J.-L. Rougier. 2012. A survey of green networking research. IEEE Communications Surveys & Tutorials 14, 1 (2012), 3--20.
14. Kashif Bilal, Saif Ur Rehman Malik, Osman Khalid, Abdul Hameed, Enrique Alvarez, Vidura Wijaysekara, Rizwana Irfan, Sarjan Shrestha, Debjyoti Dwivedy, Mazhar Ali, et al. 2013. A taxonomy and survey on Green Data Center Networks. Future Generation Computer Systems 36, (2014), 189--208.
15. Eilyan Bitar, Pramod P. Khargonekar, and Kameshwar Poolla. 2011. Systems and control opportunities in the integration of renewable energy into the smart grid. In Proc. of IFAC World Congress. 4927--4932.
16. Michael Brown and Jose Renau. 2011. Rerack: Power simulation for data centers with renewable energy generation. ACM SIGMETRICS Performance Evaluation Review 39, 3 (2011), 77--81.
17. CAISO. 2013. Renewables Watch. Retrieved August 2013 from http://www.caiso.com/green/renewableswatch. html.
18. Carbon Tax Center. 2013. Homepage. Retrieved June 2013 from http://www.carbontax.org/.
19. Jichuan Chang, Justin Meza, Parthasarathy Ranganathan, Amip Shah, Rocky Shih, and Cullen Bash. 2012. Totally green: Evaluating and designing servers for lifecycle environmental impact. ACM SIGARCH Computer Architecture News. ACM, 25--36.
20. Changbing Chen, Bingsheng He, and Xueyan Tang. 2012. Green-aware workload scheduling in geographically distributed data centers. In CloudCom. 82--89.
21. Gong Chen, Wenbo He, Jie Liu, Suman Nath, Leonidas Rigas, Lin Xiao, and Feng Zhao. 2008. Energy-aware server provisioning and load dispatching for connection-intensive internet services. In NSDI. 337--350.
22. David Chiu, Christopher Stewart, and Bart McManus. 2012. Electric grid balancing through low cost workload migration. ACM SIGMETRICS Performance Evaluation Review 40, 3 (2012), 48--52.
23. CVX. 2012. CVX: Matlab Software for Disciplined Convex Programming. Retrieved December 2013 from http://cvxr.com/cvx/.
24. Nan Deng, Christopher Stewart, Daniel Gmach, and Martin Arlitt. 2012a. Policy and mechanism for carbon-aware cloud applications. In IEEE Network Operations and Management Symposium (NOMS). IEEE, 590--594.
25. Nan Deng, Christopher Stewart, Daniel Gmach, Martin Arlitt, and Jaimie Kelley. 2012b. Adaptive green hosting. In Proceedings of the 9th International Conference on Autonomic Computing. ACM, 135--144.
26. Wei Deng, Fangming Liu, Hai Jin, Chuan Wu, and Xue Liu. 2013. MultiGreen: Cost-minimizing multi-source datacenter power supply with online control. In Proceedings of the Fourth International Conference on Future Energy Systems. ACM, 149--160.
27. Joseph Doyle, Donal O'Mahony, and Robert Shorten. 2011. Server selection for carbon emission control. In Proceedings of the 2nd ACM SIGCOMM Workshop on Green Networking. ACM, 1--6.
28. Edison Electric Institute. 2011. Different Regions Use Different Fuels. Retrieved June 2013 from http://www. getenergyactive.org/fuel/state.htm.
29. EIA. 2013. Levelized Cost of New Generation Resources. Retrieved June 2013 from http://www.eia.gov/ forecasts/aeo/electricity_generation.cfm.
30. ENERCON. 2010. ENERCON Wind Energy Converters—Product Overview. Retrieved June 2013 from http://www.enercon.de/p/downloads/EN_Productoverview_0710.pdf.
31. EPA. 2012. Cap and Trade. (2012). Retrieved June 2013 from http://www.epa.gov/captrade/index.html.
32. EPRI. 2010. Addressing Solar Photovoltaic Operations and Maintenance Challenges. Retrieved June 2013 from http://www.smartgridnews.com/.
33. ERCOT. 2013. Homepage. Retrieved August 2013 from http://www.ercot.com/.
34. Xiaobo Fan, Wolf-Dietrich Weber, and Luiz Andre Barroso. 2007. Power provisioning for a warehouse-sized computer. In ACM SIGARCH Computer Architecture News. ACM, 13--23.
35. FERC. 2013. Federal Eerngy Regulatory Commsssion, Electric Power Markets: National Overview. Retrieved December 2013 from http://www.ferc.gov/market-oversight/mkt-electric/overview.asp.
36. Lakshmi Ganesh, Tudor Marian, Hakim Weatherspoon, and Ken Birman. 2013. Integrated approach to data center power management. IEEE Transactions on Computers 62, 6 (2013), 1086--1096.
37. Peter Xiang Gao, Andrew R. Curtis, Bernard Wong, and Srinivasan Keshav. 2012. It's not easy being green. ACM SIGCOMM Computer Communication Review 42, 4 (2012), 211--222.
38. Mahdi Ghamkhari and Hamed Mohsenian Rad. 2012a. Energy and performance management of green data centers: A profit maximization approach. IEEE Transactions on Smart Grid 4, 2 (2013), 1017--1025.
39. Mahdi Ghamkhari and Hamed Mohsenian Rad. 2012b. Optimal integration of renewable energy resources in data centers with behind-the-meter renewable generator. In IEEE International Conference on Communications (ICC). IEEE, 3340--3344.
40. Mahdi Ghamkhari and Hamed Mohsenian Rad. 2013. Profit maximization and power management of green data centers supporting multiple SLAs. In International Conference on Computing, Networking and Communications (ICNC). IEEE, 465--469.
41. Daniel Gmach, Yuan Chen, Amip Shah, Jerry Rolia, Cullen Bash, Tom Christian, and Ratnesh Sharma. 2010a. Profiling sustainability of data centers. In IEEE International Symposium on Sustainable Systems and Technology (ISSST). IEEE, 1--6.
42. Daniel Gmach, Jerry Rolia, Cullen Bash, Yuan Chen, Tom Christian, Amip Shah, Ratnesh Sharma, and Zhikui Wang. 2010b. Capacity planning and power management to exploit sustainable energy. In International Conference on Network and Service Management (CNSM). IEEE, 96--103.
43. Íñigo Goiri, Ryan Beauchea, Kien Le, Thu D. Nguyen, Md E Haque, Jordi Guitart, Jordi Torres, and Ricardo Bianchini. 2011. GreenSlot: Scheduling energy consumption in green datacenters. In Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, 20.
44. Íñigo Goiri, William Katsak, Kien Le, Thu D. Nguyen, and Ricardo Bianchini. 2013. Parasol and GreenSwitch: Managing datacenters powered by renewable energy. ACM SIGARCH Computer Architecture News. ACM, 51--64.
45. Íñigo Goiri, Kien Le, Thu D. Nguyen, Jordi Guitart, Jordi Torres, and Ricardo Bianchini. 2012. GreenHadoop: Leveraging green energy in data-processing frameworks. In Proceedings of the 7th ACM European Conference on Computer Systems. ACM, 57--70.
46. Google. 2013. ClusterData2011_1. Retrieved August 2013 from https://code.google.com/p/googleclusterdata/ wiki/ClusterData2011_1.
47. Google. 2013. Renewable Energy. Retrieved June 2013 from http://www.google.com/green/energy/.
48. Sriram Govindan, Anand Sivasubramaniam, and Bhuvan Urgaonkar. 2011. Benefits and limitations of tapping into stored energy for datacenters. In Annual International Symposium on Computer Architecture (ISCA). IEEE, 341--351.
49. Sriram Govindan, Di Wang, L. Y. Chen, Anand Sivasubramaniam, and Bhuvan Urgaonkar. 2010. Modeling and Analysis of Availability of Datacenter Power Infrastructure. Technical Report. CSE-10-006, The Pennsylvania State University.
50. Kira Grogg. 2005. Harvesting the Wind: The Physics of Wind Turbines. Liberal, Arts Scholarly Repository (LASR).
51. Gurobi Optimization. 2013. Homepage. Retrieved December 2013 from http://www.gurobi.com/.
52. James Hamilton. 2010. James Hamilton's Blog-Energy Proportional Datacenter Networks. Retrieved June 2013 from http://perspectives.mvdirona.com/2010/08/01/EnergyProportionalDatacenterNetworks.aspx.
53. Jian He, Xiang Deng, Dan Wu, Yonggang Wen, and Di Wu. 2012. Socially-responsible load scheduling algorithms for sustainable data centers over smart grid. In Third International Conference on Smart Grid Communications (SmartGridComm). IEEE, 406--411.
54. Andy Hopper and Andrew Rice. 2008. Computing for the future of the planet. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 366, 1881 (2008), 3685--3697.
55. Tibor Horvath, Tarek Abdelzaher, Kevin Skadron, and Xue Liu. 2007. Dynamic voltage scaling in multitier web servers with end-to-end delay control. IEEE Transactions on Computers 56, 4 (2007), 447--458.
56. David Irwin, Navin Sharma, and Prashant Shenoy. 2011. Towards continuous policy-driven demand response in data centers. In Proceedings of the 2nd ACM SIGCOMM Workshop on Green Networking. ACM, 19--24.
57. John Johnston. 2013. Is Google Developing a Floating Data Center Powered with Wave Power? Retrieved January 2014 from http://www.the9billion.com/2013/10/28/is-google-developing-a-floating-data-center- powered-with-wave-power/.
58. Fanxin Kong, Chuansheng Dong, Xue Liu, and Haibo Zeng. 2014a. Blowing hard is not all we want: Quantity vs quality of wind power in the smart grid. In Proceedings IEEE INFOCOM. IEEE, 2813--2821.
59. Fanxin Kong, Xue Liu, and Lei Rao. 2014b. Optimal energy source selection and capacity planning for green datacenters. In The 2014 ACM International Conference on Measurement and Modeling of Computer Systems. ACM, 575--576.
60. Andrew Krioukov, Sara Alspaugh, Prashanth Mohan, Stephen Dawson-Haggerty, David E. Culler, and Randy H. Katz. 2012. Design and Evaluation of an Energy Agile Computing Cluster. Technical Report. EECS, University of California, Berkeley.
61. Andrew Krioukov, Christoph Goebel, Sara Alspaugh, Yanpei Chen, David Culler, and Y Katz. 2011. Integrating renewable energy using data analytics systems: Challenges and opportunities. IEEE Data Eng. Bull 34, 1 (2011), 1.
62. Patrick Kurp. 2008. Green computing. Communication of the ACM 51, 10 (2008), 11--13.
63. Kien Le, Ricardo Bianchini, Margaret Martonosi, and Thu D. Nguyen. 2009. Cost-and energy-aware load distribution across data centers. In Proceedings of HotPower. 3--14.
64. Kien Le, Ricardo Bianchini, Thu D. Nguyen, Ozlem Bilgir, and Margaret Martonosi. 2010. Capping the brown energy consumption of internet services at low cost. In International Green Computing Conference. IEEE, 3--14.
65. Kien Le, Ozlem Bilgir, Ricardo Bianchini, Margaret Martonosi, and Thu D. Nguyen. 2008. Managing the Cost, Energy Consumption, and Carbon Footprint of Internet Services. In ACM SIGMETRICS Performance Evaluation Review 38, 1 (2010), 357--358.
66. Chao Li, Amer Qouneh, and Tao Li. 2011. Characterizing and analyzing renewable energy driven data centers. ACM SIGMETRICS Performance Evaluation Review 39, 1 (2011), 323--324.
67. Chao Li, Amer Qouneh, and Tao Li. 2012. Iswitch: Coordinating and optimizing renewable energy powered server clusters. In Annual International Symposium on Computer Architecture (ISCA). IEEE, 512--523.
68. Yang Li, David Chiu, Changbin Liu, Linh T. X. Phan, Tanveer Gill, Sanchit Aggarwal, Zhuoyao Zhang, Boon Thau Loo, David Maier, and Bart McManus. 2013. Towards dynamic pricing-based collaborative optimizations for green data centers. In International Conference on Data Engineering Workshops (ICDEW). IEEE, 272--278.
69. Zhenhua Liu, Yuan Chen, Cullen Bash, Adam Wierman, Daniel Gmach, Zhikui Wang, Manish Marwah, and Chris Hyser. 2012. Renewable and cooling aware workload management for sustainable data centers. In ACM SIGMETRICS Performance Evaluation Review. ACM, 175--186.
70. Zhenhua Liu, Minghong Lin, Adam Wierman, Steven H. Low, and Lachlan L. H. Andrew. 2011a. Geographical load balancing with renewables. ACM SIGMETRICS Performance Evaluation Review 39, 3 (2011), 62--66.
71. Zhenhua Liu, Minghong Lin, Adam Wierman, Steven H. Low, and Lachlan L. H. Andrew. 2011b. Greening geographical load balancing. In Proceedings of the ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems. ACM, 233--244.
72. MathWorks. 2014. Documentation Center-lingrog. (2014). Retrieved January 2014 from http://www. mathworks.com/help/optim/ug/linprog.html.
73. David Meisner, Brian T. Gold, and Thomas F. Wenisch. 2009. PowerNap: Eliminating server idle power. ACM SIGARCH Computer Architecture News 37, 1 (2009), 205--216.
74. Rich Miller. 2008. Microsoft To Use Solar Panels in New Data Center. Retrieved June 2013 from http://www.datacenterknowledge.com/archives/2008/09/24/microsoft-uses-solar-panels-in-new-data-center/.
75. Rich Miller. 2009. Scottish Server Farms Team on Tidal Power. (2009). Retrieved January 2014 from http://www.datacenterknowledge.com/.
76. Mygreenhosting. 2011. Green Data Center. Retrieved January 2014 from http://www.mygreenhosting. com/green-data-center.php.
77. Ripal Nathuji and Karsten Schwan. 2007. VirtualPower: Coordinated power management in virtualized enterprise systems. In ACM SIGOPS Operating Systems Review. ACM, 265--278.
78. Michael J. Neely. 2010. Stochastic network optimization with application to communication and queueing systems. Synthesis Lectures on Communication Networks 3, 1 (2010), 1--211.
79. NERSC. 2013. National Energy Research Scientific Computing Center. Retrieved December 2013 from http://www.nersc.gov.
80. Northern Arizona Wind & Sun. 2012. Cycles vs lifespan. Retrieved December 2013 from http://www. solar-electric.com/deep-cycle-battery-faq.html.
81. NREL. 2013a. Measurement and Instrumentation Data Center (MIDC). Retrieved December 2013 from http://www.nrel.gov/midc/.
82. NREL. 2013b. National Renewable Energy Laboratory, Wind Integration Dataset. Retrieved December 2013 from http://www.nrel.gov/.
83. NREL. 2013c. Research Cell Efficiency Records. Retrieved June 2013 from http://www.nrel.gov/ncpv/.
84. NREL. 2013d. Solar & Filled Meteorological Fields. Retrieved June 2013 from http://rredc.nrel.gov/ solar/old_data/nsrdb/1991-2010/NCDCStationData/.
85. NYISO. 2013. Market & Operational Data. Retrieved December 2013 from http://www.nyiso.com/.
86. Bala Pitchaikani. 2011. Strategies for the Containerized Data Center. Retrieved December 2013 from http://www.datacenterknowledge.com/.
87. Asfandyar Qureshi, Rick Weber, Hari Balakrishnan, John Guttag, and Bruce Maggs. 2009. Cutting the electric bill for internet-scale systems. ACM SIGCOMM Computer Communication Review 39, 4 (2009), 123--134.
88. Ashikur Rahman, Xue Liu, and Fanxin Kong. 2013. A survey on geographic load balancing based data center power management in the smart grid environment. IEEE Communications Surveys & Tutorials 16, 1 (2014), 214--233.
89. Lei Rao, Xue Liu, Le Xie, and Wenyu Liu. 2010. Minimizing electricity cost: Optimization of distributed internet data centers in a multi-electricity-market environment. In Proceedings IEEE INFOCOM. IEEE, 1--9.
90. Charles Reiss, Alexey Tumanov, Gregory R. Ganger, Randy H. Katz, and Michael A. Kozuch. 2012. Towards understanding heterogeneous clouds at scale: Google trace analysis. Intel Science and Technology Center for Cloud Computing, Tech. Rep. 2012.
91. Chuangang Ren, Di Wang, Bhuvan Urgaonkar, and Anand Sivasubramaniam. 2012. Carbon-aware energy capacity planning for datacenters. In International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS). IEEE, 391--400.
92. Anand Seetharam, Manikandan Somasundaram, Don Towsley, Jim Kurose, and Prashant Shenoy. 2010. Shipping to streaming: Is this shift green? In Proceedings of the first ACM SIGCOMM Workshop on Green Networking. ACM, 61--68.
93. Navin Sharma, Sean Barker, David Irwin, and Prashant Shenoy. 2011a. Blink: Managing server clusters on intermittent power. In ACM SIGPLAN Notices. ACM, 185--198.
94. Navin Sharma, Pranshu Sharma, David Irwin, and Prashant Shenoy. 2011b. Predicting solar generation from weather forecasts using machine learning. In IEEE International Conference on Smart Grid Communications (SmartGridComm). IEEE, 528--533.
95. Rahul Singh, David Irwin, Prashant Shenoy, and KK Ramakrishnan. 2013. Yank: Enabling green data centers to pull the plug. In NSDI. 143--155.
96. Christopher Stewart and Kai Shen. 2009. Some joules are more precious than others: Managing renewable energy in the datacenter. In Proceedings of the Workshop on Power Aware Computing and Systems.
97. Ralph Stuart. 2006. Command and Control Regulation. Retrieved June 2013 from http://www.eoearth.org/ article/Command_and_control_regulation.
98. TheGreenAge. 2013. The Effect of Temperature on Solar PV Panels. Retrieved June 2013 from http://www. thegreenage.co.uk/greenhome/self-generation/solar-photovoltaic-cells/effect-of-temperature-solarpv.
99. Toronto Hydro. 2013. About Time-of-Use Pricing. Retrieved August 2013 from http://www.torontohydro. com/sites/electricsystem/business/yourbilloverview/Pages/TOURates.aspx.
100. Dylan Tweney. 2013. Amazon Website Goes Down for 40 Minutes, Costing the Company $5 Million. Retrieved June 2013 from http://venturebeat.com/2013/08/19/amazon-website-down/.
101. Guido Urdaneta, Guillaume Pierre, and Maarten Van Steen. 2009. Wikipedia workload analysis for decentralized hosting. Computer Networks.
102. Rahul Urgaonkar, Bhuvan Urgaonkar, Michael J. Neely, and Anand Sivasubramaniam. 2011. Optimal power cost management using stored energy in data centers. In Proceedings of the ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems. ACM, 221--232.
103. Ward Van Heddeghem, Willem Vereecken, Didier Colle, Mario Pickavet, and Piet Demeester. 2012. Distributed computing for carbon footprint reduction by exploiting low-footprint energy availability. Future Generation Computer Systems.
104. Di Wang, Chuangang Ren, Anand Sivasubramaniam, Bhuvan Urgaonkar, and Hosam Fathy. 2012. Energy storage in datacenters: What, where, and how much? ACM SIGMETRICS Performance Evaluation Review 40, 1 (2012), 187--198.
105. Wind Power. 2013. Wind Turbine Power Output Variation with Steady Wind Speed. Retrieved June 2013 from http://www.wind-power-program.com/turbine_characteristics.htm.
106. Ryan Wiser and Mark Bolinger. 2012. 2011 Wind Technologies Market Report. Retrieved June 2013 from http://www1.eere.energy.gov.
107. Yanwei Zhang, Yefu Wang, and Xiaorui Wang. 2011. GreenWare: Greening cloud-scale data centers to maximize the use of renewable energy. In Middleware. Springer, 143--164.
108. Yanwei Zhang, Yefu Wang, and Xiaorui Wang. 2012. TEStore: Exploiting thermal and energy storage to cut the electricity bill for datacenter cooling. In Proceedings of the 8th International Conference on Network and Service Management. 19--27.
109. Rongliang Zhou, Zhikui Wang, Alan McReynolds, Cullen E. Bash, Thomas W. Christian, and Rocky Shih. 2012. Optimization and control of cooling microgrids for data centers. In IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm). IEEE, 338--343.

Index Terms

• Computer systems organization

• General and reference

  • Document types

    • Surveys and overviews

• Networks

  • Network services

• Software and its engineering

  • Software organization and properties

    • Contextual software domains

      • Operating systems

        • Process management