research-article

Memory buddies: exploiting page sharing for smart colocation in virtualized data centers

Authors:

Gabriel Tarasuk-Levin,

Prashant Shenoy,

Peter Desnoyers,

Emmanuel Cecchet,

Mark D. CornerAuthors Info & Claims

VEE '09: Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

Pages 31 - 40

https://doi.org/10.1145/1508293.1508299

Published: 11 March 2009 Publication History

Abstract

Many data center virtualization solutions, such as VMware ESX, employ content-based page sharing to consolidate the resources of multiple servers. Page sharing identifies virtual machine memory pages with identical content and consolidates them into a single shared page. This technique, implemented at the host level, applies only between VMs placed on a given physical host. In a multi-server data center, opportunities for sharing may be lost because the VMs holding identical pages are resident on different hosts. In order to obtain the full benefit of content-based page sharing it is necessary to place virtual machines such that VMs with similar memory content are located on the same hosts.

In this paper we present Memory Buddies, a memory sharing-aware placement system for virtual machines. This system includes a memory fingerprinting system to efficiently determine the sharing potential among a set of VMs, and compute more efficient placements. In addition it makes use of live migration to optimize VM placement as workloads change. We have implemented a prototype Memory Buddies system with VMware ESX Server and present experimental results on our testbed, as well as an analysis of an extensive memory trace study. Evaluation of our prototype using a mix of enterprise and e-commerce applications demonstrates an increase of data center capacity (i.e. number of VMs supported) of 17%, while imposing low overhead and scaling to as many as a thousand servers.

References

[1]

B. Bloom. Space/time trade-offs in hash coding with allowable errors. Communications of the ACM, 13(7):422--426, July 1970.

Digital Library

[2]

Andrei Broder and Michael Mitzenmacher. Network applications of Bloom filters: A survey. Internet Mathematics, 1(4):485--509, 2003.

[3]

Edouard Bugnion, Scott Devine, and Mendel Rosenblum. DISCO: Running Commodity Operating Systems on Scalable Multiprocessors. In SOSP, pages 143--156, 1997.

Digital Library

[4]

Emmanuel Cecchet, Anupam Chanda, Sameh Elnikety, Julie Marguerite, and Willy Zwaenepoel. Performance Comparison of Middleware Architectures for Generating Dynamic Web Content. In 4th ACM/IFIP/USENIX International Middleware Conference, June 2003.

Digital Library

[5]

C. Clark, K. Fraser, S. Hand, J. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfiel. Live migration of virtual machines. In Proceedings of Usenix Symposium on Network Systems Design and Implementation (NSDI), May 2005.

Digital Library

[6]

Laura Grit, David Irwin, Aydan Yumerefendi, and Jeff Chase. Virtual machine hosting for networked clusters: Building the foundations for autonomic orchestration. In Workshop on Virtualization Technology in Distributed Computing (VTDC), November 2006.

Digital Library

[7]

Deke Guo, Jie Wu, Honghui Chen, and Xueshan Luo. Theory and Network Applications of Dynamic Bloom Filters. In INFOCOM, 2006.

[8]

Diwaker Gupta, Sangmin Lee, Michael Vrable, Stefan Savage, Alex C. Snoeren, George Varghese, Geoffrey M. Voelker, and Amin Vahdat. Difference engine: Harnessing memory redundancy in virtual machines. In Usenix OSDI, December 2008.

Digital Library

[9]

Paul Hsieh. Hash functions. http://www.azillionmonkeys.com/qed/ hash.html.

[10]

Navendu Jain, Michael Dahlin, and Renu Tewari. Using Bloom Filters to Refine Web Search Results. In WebDB, pages 25--30, 2005.

[11]

Jacob Kloster, Jesper Kristensen, and Arne Mejlholm. On the Feasibility of Memory Sharing: Content-Based Page Sharing in the Xen Virtual Machine Monitor. Master's thesis, Department of Computer Science, Aalborg University, June 2006.

[12]

M. Kozuch and M. Satyanarayanan. Internet suspend and resume. In Proceedings of the Fourth IEEE Workshop on Mobile Computing Systems and Applications, Calicoon, NY, June 2002.

Digital Library

[13]

Purushottam Kulkarni, Prashant J. Shenoy, and Weibo Gong. Scalable Techniques for Memory-efficients CDN Simulations. In WWW, 2003.

Digital Library

[14]

libvirt. The Virtualization API. http://libvirt.org.

[15]

Xucheng Luo, Zhiguang Qin, Ji Geng, and Jiaqing Luo. IAC: Interest-Aware Caching for Unstructured P2P. In SKG, page 58, 2006.

Digital Library

[16]

David Mosberger and Tai Jin. httperf: A tool for measuring web server performance. In First Workshop on Internet Server Performance, pages 59--67. ACM, June 1998.

Digital Library

[17]

E. Nahum. Deconstructing specweb, 2002.

[18]

Michael Nelson, Beng-Hong Lim, and Greg Hutchins. Fast Transparent Migration for Virtual Machines. In USENIX Annual Technical Conference, 2005.

Digital Library

[19]

M. Powell and B. Miller. Process migration in DEMOS/MP. Operating Systems Review, 17(5):110--119, 1983.

Digital Library

[20]

Apache Open For Business Project. http://ofbiz.apache.org.

[21]

Paul Ruth, Junghwan Rhee, Dongyan Xu, Rick Kennell, and Sebastien Goasguen. Autonomic Live Adaptation of Virtual Computational Environments in a Multi-Domain Infrastructure. In IEEE International Conference on Autonomic Computing (ICAC), June 2006.

Digital Library

[22]

Constantine P. Sapuntzakis, Ramesh Chandra, Ben Pfaff, Jim Chow, Monica S. Lam, and Mendel Rosenblum. Optimizing the migration of virtual computers. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation, December 2002.

Digital Library

[23]

W. Smith. TPC--W: Benchmarking An Ecommerce Solution. http://www.tpc.org/information/other/techarticles.asp.

[24]

The standard performance evaluation corporation (spec). http://www.spec.org.

[25]

A. Sundararaj, A. Gupta, and P. Dinda. Increasing Application Performance in Virtual Environments through Run-time Inference and Adaptation. In Fourteenth International Symposium on High Performance Distributed Computing (HPDC), July 2005.

Digital Library

[26]

M. M. Theimer, K. A. L., and D. R. Cheriton. Preemptable Remote Execution Facilities for the V-System. pages 2--12, December 1985.

[27]

C. Waldspurger. Memory Resource Management in VMWare ESX Server. In Proceedings of the Fifth Symposium on Operating System Design and Implementation (OSDI'02), December 2002.

Digital Library

[28]

A. Whitaker, M. Shaw, and S. D. Gribble. Scale and Performance in the Denali Isolation Kernel. In Proceedings of the Fifth Symposium on Operating System Design and Implementation (OSDI'02), December 2002.

Digital Library

[29]

VMware Whitepaper. Drs performance and best practices.

[30]

Timothy Wood, Ludmila Cherkasova, Kivanc Ozonat, and Prashant Shenoy. Profiling and modeling resource usage of virtualized applications. In International Middleware Conference, 2008.

Digital Library

[31]

Timothy Wood, Prashant J. Shenoy, Arun Venkataramani, and Mazin S. Yousif. Black-box and Gray-box Strategies for Virtual Machine Migration. In Networked Systems Design and Implementation (NSDI '07), 2007.

Digital Library

Cited By

Asanuma KYamada H(2024)DBMS-Assisted Live Migration of Virtual MachinesIEEE Transactions on Computers10.1109/TC.2023.332994373:2(380-393)Online publication date: Feb-2024
https://doi.org/10.1109/TC.2023.3329943
Bai YZhang XZhang Y(2024)Meunik: Rethinking Virtual Machine Memory Resource Management for Unikernel-Based VMs2024 IEEE/ACM Symposium on Edge Computing (SEC)10.1109/SEC62691.2024.00030(292-305)Online publication date: 4-Dec-2024
https://doi.org/10.1109/SEC62691.2024.00030
Tiwari SBhatt C(2023)A Comprehensive Study on Cloud Computing: Architecture, Load Balancing, Task Scheduling and Meta-Heuristic OptimizationIntelligent Cyber Physical Systems and Internet of Things10.1007/978-3-031-18497-0_11(137-162)Online publication date: 4-Feb-2023
https://doi.org/10.1007/978-3-031-18497-0_11
Show More Cited By

Index Terms

Memory buddies: exploiting page sharing for smart colocation in virtualized data centers
1. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        Memory management
        Main memory
    2. Software system structures
      1. Distributed systems organizing principles

Recommendations

Memory buddies: exploiting page sharing for smart colocation in virtualized data centers

Many data center virtualization solutions, such as VMware ESX, employ content-based page sharing to consolidate the resources of multiple servers. Page sharing identifies virtual machine memory pages with identical content and consolidates them into a ...
Proactively Breaking Large Pages to Improve Memory Overcommitment Performance in VMware ESXi
VEE '15: Proceedings of the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments

VMware ESXi leverages hardware support for MMU virtualization available in modern Intel/AMD CPUs. To optimize address translation performance when running on such CPUs, ESXi preferably uses host large pages (2MB in x86-64 systems) to back VM's guest ...
Performance Analysis for Pareto-Optimal Green Consolidation Based on Virtual Machines Live Migration

Huge energy requirement of cloud data centers is prime concern. Dynamic Virtual Machine VM consolidation based on VM live migration to switched-off or put some of the under-loaded host Physical Machines PMs into a low power consumption mode can ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

VEE '09: Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

March 2009

148 pages

ISBN:9781605583754

DOI:10.1145/1508293

General Chair:
Antony Hosking
Purdue University, USA
,
Program Chairs:
David Bacon
IBM Research, USA
,
Orran Krieger
VMware, USA

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 March 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

VEE '09

Sponsor:

VEE '09: ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments

March 11 - 13, 2009

DC, Washington, USA

Acceptance Rates

Overall Acceptance Rate 80 of 235 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

117
Total Citations
View Citations
911
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Asanuma KYamada H(2024)DBMS-Assisted Live Migration of Virtual MachinesIEEE Transactions on Computers10.1109/TC.2023.332994373:2(380-393)Online publication date: Feb-2024
https://doi.org/10.1109/TC.2023.3329943
Bai YZhang XZhang Y(2024)Meunik: Rethinking Virtual Machine Memory Resource Management for Unikernel-Based VMs2024 IEEE/ACM Symposium on Edge Computing (SEC)10.1109/SEC62691.2024.00030(292-305)Online publication date: 4-Dec-2024
https://doi.org/10.1109/SEC62691.2024.00030
Tiwari SBhatt C(2023)A Comprehensive Study on Cloud Computing: Architecture, Load Balancing, Task Scheduling and Meta-Heuristic OptimizationIntelligent Cyber Physical Systems and Internet of Things10.1007/978-3-031-18497-0_11(137-162)Online publication date: 4-Feb-2023
https://doi.org/10.1007/978-3-031-18497-0_11
Rampersaud SGrosu D(2021)An Approximation Algorithm for Sharing-Aware Virtual Machine Revenue MaximizationIEEE Transactions on Services Computing10.1109/TSC.2017.278672814:1(1-15)Online publication date: 1-Jan-2021
https://doi.org/10.1109/TSC.2017.2786728
Zhang YNiu KWu WLi KZhou Y(2021)Speeding Up VM Startup by Cooperative VM Image CachingIEEE Transactions on Cloud Computing10.1109/TCC.2018.27915099:1(360-371)Online publication date: 1-Jan-2021
https://doi.org/10.1109/TCC.2018.2791509
Kherbache VMadelaine EHermenier F(2020)Scheduling Live Migration of Virtual MachinesIEEE Transactions on Cloud Computing10.1109/TCC.2017.27542798:1(282-296)Online publication date: 1-Jan-2020
https://doi.org/10.1109/TCC.2017.2754279
Kumar T SMishra DPanda BDeshmukh N(2019)CoWLightProceedings of the 8th International Workshop on Hardware and Architectural Support for Security and Privacy10.1145/3337167.3337170(1-8)Online publication date: 23-Jun-2019
https://dl.acm.org/doi/10.1145/3337167.3337170
Ala'anzy MOthman M(2019)Load Balancing and Server Consolidation in Cloud Computing Environments: A Meta-StudyIEEE Access10.1109/ACCESS.2019.29444207(141868-141887)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2944420
Kim SChoi Y(2019)Constraint-aware VM placement in heterogeneous computing clustersCluster Computing10.1007/s10586-019-02966-6Online publication date: 1-Aug-2019
https://doi.org/10.1007/s10586-019-02966-6
Hassan MBabiker ABaker MHamad M(2018)SLA Management For Virtual Machine Live Migration Using Machine Learning with Modified Kernel and Statistical ApproachEngineering, Technology & Applied Science Research10.48084/etasr.16928:1(2459-2463)Online publication date: 20-Feb-2018
https://doi.org/10.48084/etasr.1692
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten