skip to main content
research-article

RAPL in Action: Experiences in Using RAPL for Power Measurements

Published: 22 March 2018 Publication History

Abstract

To improve energy efficiency and comply with the power budgets, it is important to be able to measure the power consumption of cloud computing servers. Intel’s Running Average Power Limit (RAPL) interface is a powerful tool for this purpose. RAPL provides power limiting features and accurate energy readings for CPUs and DRAM, which are easily accessible through different interfaces on large distributed computing systems. Since its introduction, RAPL has been used extensively in power measurement and modeling. However, the advantages and disadvantages of RAPL have not been well investigated yet. To fill this gap, we conduct a series of experiments to disclose the underlying strengths and weaknesses of the RAPL interface by using both customized microbenchmarks and three well-known application level benchmarks: Stream, Stress-ng, and ParFullCMS. Moreover, to make the analysis as realistic as possible, we leverage two production-level power measurement datasets from the Taito, a supercomputing cluster of the Finnish Center of Scientific Computing and also replicate our experiments on Amazon EC2. Our results illustrate different aspects of RAPL and document the findings through comprehensive analysis. Our observations reveal that RAPL readings are highly correlated with plug power, promisingly accurate enough, and have negligible performance overhead. Experimental results suggest RAPL can be a very useful tool to measure and monitor the energy consumption of servers without deploying any complex power meters. We also show that there are still some open issues, such as driver support, non-atomicity of register updates, and unpredictable timings that might weaken the usability of RAPL in certain scenarios. For such scenarios, we pinpoint solutions and workarounds.

References

[1]
VMSTAT. Retrieved from http://www.linuxcommand.org/man_pages/vmstat8.html.
[2]
David Abdurachmanov, Peter Elmer, Giulio Eulisse, Robert Knight, Tapio Niemi, Jukka K. Nurminen, Filip Nyback, Goncalo Pestana, Zhonghong Ou, and Kashif Nizam Khan. 2014. Techniques and tools for measuring energy efficiency of scientific software applications. CoRR abs/1410.3440 (2014). Retrieved from http://arxiv.org/abs/1410.3440.
[3]
Xi Chen, Chi Xu, Robert P. Dick, and Zhuoqing Morley Mao. 2010. Performance and power modeling in a multi-programmed multi-core environment. In Proceedings of the 47th Design Automation Conference (DAC’10). ACM, New York, NY, 813--818.
[4]
CSC. 2017. Taito supercluster. Retrieved from https://research.csc.fi/taito-supercluster.
[5]
Howard David, Eugene Gorbatov, Ulf R. Hanebutte, Rahul Khanna, and Christian Le. 2010. RAPL: Memory power estimation and capping. In Proceedings of the 16th ACM/IEEE International Symposium on Low Power Electronics and Design (ISLPED’10). ACM, New York, NY, 189--194.
[6]
Spencer Desrochers, Chad Paradis, and Vincent M. Weaver. 2016. A validation of DRAM RAPL power measurements. In Proceedings of the 2nd International Symposium on Memory Systems (MEMSYS’16). ACM, New York, NY, 455--470.
[7]
Mohammed El Mehdi Diouri, Manuel F. Dolz, Olivier Glück, Laurent Lefèvre, Pedro Alonso, Sandra Catalán, Rafael Mayo, and Enrique S. Quintana-Ortí. 2014. Assessing power monitoring approaches for energy and power analysis of computers. Sustain. Comput.: Informat. Syst. 4, 2 (2014), 68--82.
[8]
Jack Dongarra, Hatem Ltaief, Piotr Luszczek, and Vincent M. Weaver. 2012. Energy footprint of advanced dense numerical linear algebra using tile algorithms on multicore architectures. In Proceedings of the 2012 2nd International Conference on Cloud and Green Computing (CGC’12). IEEE, 274--281.
[9]
Amazon EC2. Instance Types. Retrieved from https://aws.amazon.com/ec2/instance-types/.
[10]
S. Agostinelli et al. 2003. Geant4- A simulation toolkit. Nucl. Instrum. Methods Phys. Res. Sec. A: Accel. Spectrom. Detect. Assoc. Equip. 506, 3 (2003), 250--303.
[11]
Daniel Hackenberg, Thomas Ilsche, Robert Schone, Daniel Molka, Maik Schmidt, and Wolfgang E Nagel. 2013. Power measurement techniques on standard compute nodes: A quantitative comparison. In Proceedings of the 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS’13). IEEE, 194--204.
[12]
Daniel Hackenberg, Robert Schöne, Thomas Ilsche, Daniel Molka, Joseph Schuchart, and Robin Geyer. 2015. An energy efficiency feature survey of the intel haswell processor. In Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’15).
[13]
D. Hackenberg, R. Schöne, T. Ilsche, D. Molka, J. Schuchart, and R. Geyer. 2015. An energy efficiency feature survey of the intel haswell processor. In Proceedings of the 2015 IEEE International Parallel and Distributed Processing Symposium Workshop. 896--904.
[14]
Marcus Hähnel, Björn Döbel, Marcus Völp, and Hermann Härtig. 2012. Measuring energy consumption for short code paths using RAPL. ACM SIGMETRICS Perform. Eval. Rev. 40, 3 (2012), 13--17.
[15]
Mikael Hirki. 2015. Energy and Performance Profiling of Scientific Computing. Master’s thesis. Aalto University.
[16]
Mikael Hirki. 2017. RAPL Testing and Instruction Decoder Benchmarks. Retrieved from https://github.com/mhirki/idq-bench2.
[17]
Mikael Hirki, Zhonghong Ou, Kashif Nizam Khan, Jukka K Nurminen, and Tapio Niemi. 2016. Empirical study of the power consumption of the x86-64 instruction decoder. In Proceedings of the USENIX Workshop on Cool Topics on Sustainable Data Centers (CoolDC’16). USENIX Association.
[18]
Song Huang, Michael Lang, Scott Pakin, and Song Fu. 2015. Measurement and characterization of haswell power and energy consumption. In Proceedings of the 3rd International Workshop on Energy Efficient Supercomputing (E2SC’15). ACM, New York, NY.
[19]
T. Ilsche, D. Hackenberg, S. Graul, R. Schöne, and J. Schuchart. 2015. Power measurements for compute nodes: Improving sampling rates, granularity and accuracy. In Proceedings of the 2015 6th International Green and Sustainable Computing Conference (IGSC’15). 1--8.
[20]
Intel Corporation 2015. Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 3, System Programming Guide. Intel Corporation.
[21]
R. Kavanagh, D. Armstrong, and K. Djemame. 2016. Accuracy of energy model calibration with IPMI. In Proceedings of the 2016 IEEE 9th International Conference on Cloud Computing (CLOUD’16). 648--655.
[22]
Jaimie Kelley, Christopher Stewart, Devesh Tiwari, and Saurabh Gupta. 2016. Adaptive power profiling for many-core HPC architectures. In Proceedings of the International Conference on Autonomic Computing.
[23]
K. N. Khan, F. Nybäck, Z. Ou, J. K. Nurminen, T. Niemi, G. Eulisse, P. Elmer, and D. Abdurachmanov. 2015. Energy profiling using IgProf. In Proceedings of the 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 1115--1118.
[24]
Kashif Nizam Khan, Zhonghong Ou, Mikael Hirki, Jukka K. Nurminen, and Tapio Niemi. 2016. How much power does your server consume? Estimating wall socket power using RAPL measurements. Comput. Sci. Res. Dev. 31, 4 (2016), 207--214.
[25]
R. Khanna, F. Zuhayri, M. Nachimuthu, C. Le, and M. J. Kumar. 2011. Unified extensible firmware interface: An innovative approach to DRAM power control. In Proceedings of the 2011 International Conference on Energy Aware Computing. 1--6.
[26]
Gary Lawson, Masha Sosonkina, and Yuzhong Shen. 2015. Towards modeling energy consumption of Xeon Phi. arXiv:1505.06539 (2015).
[27]
H. Liu. 2011. A measurement study of server utilization in public clouds. In Proceedings of the 2011 IEEE 9th International Conference on Dependable, Autonomic and Secure Computing. 435--442.
[28]
Ioannis Manousakis, Foivos S. Zakkak, Polyvios Pratikakis, and Dimitrios S. Nikolopoulos. 2015. TProf: An energy profiler for task-parallel programs. Sustainable Computing: Informatics and Systems 5 (2015), 1--13. http://www.sciencedirect.com/science/article/pii/S2210537914000390.
[29]
John D. McCalpin. 1995. Memory bandwidth and machine balance in current high performance computers. IEEE Comput. Soc. Tech. Committee Comput. Architect. (TCCA) Newsletter (Dec. 1995), 19--25.
[30]
John C. McCullough, Yuvraj Agarwal, Jaideep Chandrashekar, Sathyanarayan Kuppuswamy, Alex C. Snoeren, and Rajesh K. Gupta. 2011. Evaluating the effectiveness of model-based power characterization. In Proceedings of the 2011 USENIX Conference on USENIX Annual Technical Conference (USENIXATC’11). USENIX Association, Berkeley, CA, 12--12. Retrieved from
[31]
C. Möbius, W. Dargie, and A. Schill. 2014. Power consumption estimation models for processors, virtual machines, and servers. IEEE Trans. Parallel Distrib. Syst. 25, 6 (June 2014), 1600--1614.
[32]
Chad M. Paradis. 2015. Detailed Low-cost Energy and Power Monitoring of Computing Systems. Master’s thesis. University of Maine.
[33]
Tapasya Patki, David K. Lowenthal, Barry Rountree, Martin Schulz, and Bronis R. de Supinski. 2013. Exploring hardware overprovisioning in power-constrained, high performance computing. In Proceedings of the 27th International ACM Conference on International Conference on Supercomputing (ICS’13). ACM, New York, NY, 173--182.
[34]
Plugwise. 2017. Energy management systems of the 21st century. Retrieved from https://www.plugwise.com/.
[35]
Harald Servat, Germán Llort, Judit Giménez, and Jesús Labarta. 2016. Detailed and simultaneous power and performance analysis. Concurrency and Computation: Practice and Experience 28, 2 (2016), 252--273.
[36]
A. Skrenes and C. Williamson. 2016. Experimental calibration and validation of a speed scaling simulator. In Proceedings of the 2016 IEEE 24th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS’16). 105--114.
[37]
Balaji Subramaniam and Wu-chun Feng. 2015. On the energy proportionality of scale-out workloads. CoRR abs/1501.02729 (2015). Retrieved from
[38]
Balaji Subramaniam and Wu-chun Feng. 2013. Towards energy-proportional computing for enterprise-class server workloads. In Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE’13). ACM, New York, NY, 15--26.
[39]
Vincent Weaver. 2015. rapl-read.c. Retrieved from http://web.eece.maine.edu/ vweaver/projects/rapl/rapl-read.c.
[40]
V. M. Weaver, M. Johnson, K. Kasichayanula, J. Ralph, P. Luszczek, D. Terpstra, and S. Moore. 2012. Measuring energy and power with PAPI. In Proceedings of the 2012 41st International Conference on Parallel Processing Workshops. 262--268.
[41]
Vincent M. Weaver, Matt Johnson, Kiran Kasichayanula, James Ralph, Piotr Luszczek, Dan Terpstra, and Shirley Moore. 2012. Measuring energy and power with PAPI. In Proceedings of the 2012 41st International Conference on Parallel Processing Workshops (ICPPW’12). IEEE Computer Society, Washington, DC, 262--268.
[42]
Simon N. Wood. 2017. Generalized Additive Models: An Introduction with R. CRC Press.
[43]
Yan Zhai, Xiao Zhang, Stephane Eranian, Lingjia Tang, and Jason Mars. 2014. HaPPy: Hyperthread-aware power profiling dynamically. In Proceedings of the 2014 USENIX Annual Technical Conference (USENIXATC’14). USENIX Association, Philadelphia, PA, 211--217.
[44]
Huazhe Zhang and Henry Hoffmann. 2015. A quantitative evaluation of the RAPL power control system. In Proceedings of the 10th International Workshop on Feedback Computing.
[45]
Huazhe Zhang and Henry Hoffmann. 2016. Maximizing performance under a power cap: A comparison of hardware, software, and hybrid techniques. In Proceedings of the 21st International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS’16). ACM, New York, NY, 545--559.

Cited By

View all
  • (2025)Energy Implications of Mitigating Side-Channel Attacks on Branch PredictionComputers10.3390/computers1402007114:2(71)Online publication date: 16-Feb-2025
  • (2025)WaterWise: Co-optimizing Carbon- and Water-Footprint Toward Environmentally Sustainable Cloud ComputingProceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming10.1145/3710848.3710891(297-311)Online publication date: 28-Feb-2025
  • (2025)Parallelization strategies for high-performance and energy-efficient epidemic spread simulationsSimulation Modelling Practice and Theory10.1016/j.simpat.2024.103059140(103059)Online publication date: Apr-2025
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Performance Evaluation of Computing Systems
ACM Transactions on Modeling and Performance Evaluation of Computing Systems  Volume 3, Issue 2
Special Issue on ICPE 2017 and Regular Papers
June 2018
114 pages
ISSN:2376-3639
EISSN:2376-3647
DOI:10.1145/3199681
  • Editors:
  • Sem Borst,
  • Carey Williamson
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 March 2018
Accepted: 01 January 2018
Revised: 01 October 2017
Received: 01 June 2017
Published in TOMPECS Volume 3, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. DRAM power
  2. RAPL
  3. RAPL accuracy
  4. RAPL validation
  5. power modeling

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • Central Universities and National Natural Science Foundation of China

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)621
  • Downloads (Last 6 weeks)86
Reflects downloads up to 08 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Energy Implications of Mitigating Side-Channel Attacks on Branch PredictionComputers10.3390/computers1402007114:2(71)Online publication date: 16-Feb-2025
  • (2025)WaterWise: Co-optimizing Carbon- and Water-Footprint Toward Environmentally Sustainable Cloud ComputingProceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming10.1145/3710848.3710891(297-311)Online publication date: 28-Feb-2025
  • (2025)Parallelization strategies for high-performance and energy-efficient epidemic spread simulationsSimulation Modelling Practice and Theory10.1016/j.simpat.2024.103059140(103059)Online publication date: Apr-2025
  • (2025)EdgeInsight: An Agent-Driven Framework for System Health Monitoring in Edge Computing PlatformsProceedings of International Conference on Theoretical and Applied Computing10.1007/978-981-97-6957-5_19(225-242)Online publication date: 14-Feb-2025
  • (2025)A Power Monitoring Framework of a Post-quantum Cryptography Web ServerAlgorithms and Architectures for Parallel Processing10.1007/978-981-96-1548-3_15(219-232)Online publication date: 17-Feb-2025
  • (2025)Exploring RAPL as a Power Capping Leverage for Power-Constrained InfrastructuresAlgorithms and Architectures for Parallel Processing10.1007/978-981-96-1542-1_19(323-333)Online publication date: 15-Feb-2025
  • (2024)Quality-Aware Signal Processing Mechanism of PPG Signal for Long-Term Heart Rate MonitoringSensors10.3390/s2412390124:12(3901)Online publication date: 16-Jun-2024
  • (2024)On the Energy Behaviors of the Bellman–Ford and Dijkstra Algorithms: A Detailed Empirical StudyJournal of Sensor and Actuator Networks10.3390/jsan1305006713:5(67)Online publication date: 12-Oct-2024
  • (2024)Offloaded Data Processing Energy Efficiency EvaluationInformatica10.15388/24-INFOR567(649-669)Online publication date: 24-Jul-2024
  • (2024)EnergAt: Fine-Grained Energy Attribution for Multi-TenancyACM SIGEnergy Energy Informatics Review10.1145/3698365.36983694:3(18-25)Online publication date: 1-Jul-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media