research-article

Perfctr-Xen: a framework for performance counter virtualization

Authors:

Ruslan Nikolaev,

Godmar BackAuthors Info & Claims

VEE '11: Proceedings of the 7th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

Pages 15 - 26

https://doi.org/10.1145/1952682.1952687

Published: 09 March 2011 Publication History

Abstract

Virtualization is a powerful technique used for variety of application domains, including emerging cloud environments that provide access to virtual machines as a service. Because of the interaction of virtual machines with multiple underlying software and hardware layers, the analysis of the performance of applications running in virtualized environments has been difficult. Moreover, performance analysis tools commonly used in native environments were not available in virtualized environments, a gap which our work closes.

This paper discusses the challenges of performance monitoring inherent to virtualized environments and introduces a technique to virtualize access to low-level performance counters on a per-thread basis. The technique was implemented in perfctr-xen, a framework for the Xen hypervisor that provides an infrastructure for higher-level profilers. This framework supports both accumulative event counts and interrupt-driven event sampling. It is light-weight, providing direct user mode access to logical counter values. perfctr-xen supports multiple modes of virtualization, including paravirtualization and hardware-assisted virtualization. perfctr-xen applies guest kernel-hypervisor coordination techniques to reduce virtualization overhead. We present experimental results based on microbenchmarks and SPEC CPU2006 macrobenchmarks that show the accuracy and usability of the obtained measurements when compared to native execution.

References

[1]

AMD CodeAnalyst. http://developer.amd.com/cpu/codeanalyst/, 2011.

[2]

Performance counters for Linux. http://lwn.net/Articles/310176/, 2008.

[3]

Perf_counter direct access support. http://lwn.net/Articles/323891/, 2009.

[4]

Vmkperf utility for VMWare ESX 4.0, 2011.

[5]

VTune amplifier profiler. http://software.intel.com/en-us/articles/intel-vtune-amplifier-xe/, 2011.

[6]

L. Adhianto, S. Banerjee, M. Fagan, M. Krentel, G. Marin, J. Mellor-Crummey, and N. R. Tallent. HPCToolkit: Tools for performance analysis of optimized parallel programs. Concurrency and Computation: Practice and Experience, 22 (6): 685--701, 2010.

Digital Library

[7]

G. Back and D. S. Nikolopoulos. Application-specific system customization on many-core platforms: The VT-ASOS framework. In STMCS: Second Workshop on Software Tools for Multi-Core Systems (STMCS), San Jose, CA, Mar. 2007.

[8]

P. Barham, B. Dragovic, K. Fraser, and et al. Xen and the art of virtualization. In SOSP '03: Proceedings of the nineteenth ACM Symposium on Operating Systems Principles, pages 164--177, New York, NY, USA, 2003. ISBN 1-58113-757-5.

Digital Library

[9]

S. Bratanov, R. Belenov, and N. Manovich. Virtual machines: a whole new world for performance analysis. SIGOPS Oper. Syst. Rev., 43: 46--55, April 2009.

Digital Library

[10]

S. Browne, C. Deane, G. Ho, and P. Mucci. PAPI: A portable interface to hardware performance counters. In Proceedings of Department of Defense HPCMP Users Group Conference, June 1999.

[11]

E. Bugnion, S. Devine, K. Govil, and M. Rosenblum. Disco: running commodity operating systems on scalable multiprocessors. ACM Trans. Comput. Syst., 15 (4): 412--447, 1997. ISSN 0734-2071. 10.1145/265924.265930.

Digital Library

[12]

W. Cohen. Multiple architecture characterization of the build process with OProfile, Red Hat. http://people.redhat.com/wcohen/wwc2003/, 2003.

[13]

R. Creasy. The origin of the VM/370 time-sharing system. Softw. World (UK), 13 (1): 4--10, 1982. ISSN 0038-0652.

[14]

P. Drongowski, L. Yu, F. Swehosky, S. Suthikulpanit, and R. Richter. Incorporating instruction-based sampling into AMD CodeAnalyst. In IEEE International Symposium on Performance Analysis of Systems Software (ISPASS), pages 119--120, Mar. 2010. 10.1109/ISPASS.2010.5452049.

[15]

J. Du, N. Sehrawat, and W. Zwaenepoel. Performance profiling in a virtualized environment. In Proceedings of the 2nd USENIX conference on Hot topics in cloud computing, HotCloud'10, Berkeley, CA, USA, 2010. USENIX Association.

Digital Library

[16]

J. Du, N. Sehrawat, and W. Zwaenepoel. Performance profiling of virtual machines. In Proceedings of the 7th ACM SIGPLAN/SIGOPS international conference on Virtual Execution Environments, VEE '11, Newport Beach, CA, USA, 2011.

Digital Library

[17]

S. Eranian. Perfmon2: A flexible performance monitoring interface for linux. In Ottawa Linux Symposium, pages 269--288, Ottawa, Canada, 2006.

[18]

I. Habib. Virtualization with KVM. phLinux Journal, 2008 (166): 8, 2008. ISSN 1075--3583.

Digital Library

[19]

S. T. King, G. W. Dunlap, and P. M. Chen. Operating system support for virtual machines. In ATEC '03: Proceedings of the annual conference on USENIX Annual Technical Conference, pages 71--84, Berkeley, CA, USA, 2003. USENIX Association.

Digital Library

[20]

A. Kivity. KVM: the Linux virtual machine monitor. In OLS '07: The 2007 Ottawa Linux Symposium, pages 225--230, July 2007.

[21]

A. Menon, J. R. Santos, Y. Turner, G. J. Janakiraman, and W. Zwaenepoel. Diagnosing performance overheads in the Xen virtual machine environment. In VEE '05: Proceedings of the 1st ACM/USENIX International Conference on Virtual Execution Environments, pages 13--23, New York, NY, USA, 2005. ISBN 1-59593-047-7. 10.1145/1064979.1064984.

Digital Library

[22]

M. Pettersson. Perfctr library. http://user.it.uu.se/ mikpe/linux/perfctr/, 2011.

[23]

J. S. Robin and C. E. Irvine. Analysis of the intel pentium's ability to support a secure virtual machine monitor. In 9th USENIX Security Symposium, pages 129--144, 2000.

Digital Library

[24]

M. Rosenblum. The reincarnation of virtual machines. Queue, 2 (5): 34--40, 2004.

Digital Library

[25]

C. Sapuntzakis, D. Brumley, R. Chandra, N. Zeldovich, J. Chow, M. S. Lam, and M. Rosenblum. Virtual appliances for deploying and maintaining software. In LISA '03: Proceedings of the 17th USENIX conference on System administration, pages 181--194, Berkeley, CA, USA, 2003. USENIX Association.

Digital Library

[26]

S. Shende and A. D. Malony. TAU: The TAU parallel performance system. International Journal of High Performance Computing Applications, 20 (2): 287--311, 2006.

Digital Library

[27]

D. Zaparanuks, M. Jovic, and M. Hauswirth. Accuracy of performance counter measurements. In IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS 2009)., pages 23--32, Apr. 2009.

Cited By

Nam JSeo JShin S(2018)ProbiusProceedings of the Symposium on SDN Research10.1145/3185467.3185495(1-13)Online publication date: 28-Mar-2018
https://dl.acm.org/doi/10.1145/3185467.3185495
Teabe BTchana AHagimont D(2018)Enforcing CPU allocation in a heterogeneous IaaSFuture Generation Computer Systems10.1016/j.future.2015.05.01353:C(1-12)Online publication date: 30-Dec-2018
https://dl.acm.org/doi/10.1016/j.future.2015.05.013
Benedict S(2018)Performance issues and performance analysis tools for HPC cloud applicationsComputing10.1007/s00607-012-0213-095:2(89-108)Online publication date: 31-Dec-2018
https://dl.acm.org/doi/10.1007/s00607-012-0213-0
Show More Cited By

Index Terms

Perfctr-Xen: a framework for performance counter virtualization

Recommendations

Perfctr-Xen: a framework for performance counter virtualization
VEE '11

Virtualization is a powerful technique used for variety of application domains, including emerging cloud environments that provide access to virtual machines as a service. Because of the interaction of virtual machines with multiple underlying software ...
Xen and the art of virtualization
SOSP '03

Numerous systems have been designed which use virtualization to subdivide the ample resources of a modern computer. Some require specialized hardware, or cannot support commodity operating systems. Some target 100% binary compatibility at the expense of ...
Hybrid CPU Management for Adapting to the Diversity of Virtual Machines

As an important cornerstone for clouds, virtualization plays a vital role in building this emerging infrastructure. Virtual machines (VMs) with a variety of workloads may run simultaneously on a physical machine in the cloud platform. The scheduling ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

VEE '11: Proceedings of the 7th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

March 2011

250 pages

ISBN:9781450306874

DOI:10.1145/1952682

General Chair:
Erez Petrank
The Technion, Israel
,
Program Chair:
Doug Lea
SUNY Oswego, USA

ACM SIGPLAN Notices Volume 46, Issue 7
VEE '11
July 2011
231 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/2007477
Issue’s Table of Contents

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 March 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

VEE '11

Sponsor:

VEE '11: ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments

March 9 - 11, 2011

California, Newport Beach, USA

Acceptance Rates

Overall Acceptance Rate 80 of 235 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
1,046
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Nam JSeo JShin S(2018)ProbiusProceedings of the Symposium on SDN Research10.1145/3185467.3185495(1-13)Online publication date: 28-Mar-2018
https://dl.acm.org/doi/10.1145/3185467.3185495
Teabe BTchana AHagimont D(2018)Enforcing CPU allocation in a heterogeneous IaaSFuture Generation Computer Systems10.1016/j.future.2015.05.01353:C(1-12)Online publication date: 30-Dec-2018
https://dl.acm.org/doi/10.1016/j.future.2015.05.013
Benedict S(2018)Performance issues and performance analysis tools for HPC cloud applicationsComputing10.1007/s00607-012-0213-095:2(89-108)Online publication date: 31-Dec-2018
https://dl.acm.org/doi/10.1007/s00607-012-0213-0
Gan QWu SJin HWang K(2017)vScope: A Fine-Grained Approach to Schedule vCPUs in NUMA Systems2017 IEEE 19th International Conference on High Performance Computing and Communications; IEEE 15th International Conference on Smart City; IEEE 3rd International Conference on Data Science and Systems (HPCC/SmartCity/DSS)10.1109/HPCC-SmartCity-DSS.2017.69(531-538)Online publication date: Dec-2017
https://doi.org/10.1109/HPCC-SmartCity-DSS.2017.69
Tchana ABui BTeabe BNitu VHagimont D(2016)Mitigating performance unpredictability in the IaaS using the Kyoto principleProceedings of the 17th International Middleware Conference10.1145/2988336.2988342(1-10)Online publication date: 28-Nov-2016
https://dl.acm.org/doi/10.1145/2988336.2988342
Teabe BTchana AHagimont DCadar CPietzuch PKeeton KRodrigues R(2016)Application-specific quantum for multi-core platform schedulerProceedings of the Eleventh European Conference on Computer Systems10.1145/2901318.2901340(1-14)Online publication date: 18-Apr-2016
https://dl.acm.org/doi/10.1145/2901318.2901340
Wu SSun HZhou LGan QJin H(2016)vProbe: Scheduling Virtual Machines on NUMA Systems2016 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER.2016.60(70-79)Online publication date: Sep-2016
https://doi.org/10.1109/CLUSTER.2016.60
Bui VTeabe BTchana AHagimont DTchana AHagimont DPalma N(2015)KyotoProceedings of the International workshop on Virtualization Technologies10.1145/2835075.2835077(1-6)Online publication date: 7-Dec-2015
https://dl.acm.org/doi/10.1145/2835075.2835077
Wang SZhang WWang TYe CHuang T(2015)VMonProceedings of the 2015 IEEE 39th Annual Computer Software and Applications Conference - Volume 0210.1109/COMPSAC.2015.14(399-408)Online publication date: 1-Jul-2015
https://dl.acm.org/doi/10.1109/COMPSAC.2015.14
Liu MLi TYew PZhai AKeckler S(2014)Optimizing virtual machine consolidation performance on NUMA server architecture for cloud workloadsProceeding of the 41st annual international symposium on Computer architecuture10.5555/2665671.2665720(325-336)Online publication date: 14-Jun-2014
https://dl.acm.org/doi/10.5555/2665671.2665720
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten