skip to main content
research-article

Memory buddies: exploiting page sharing for smart colocation in virtualized data centers

Published: 31 July 2009 Publication History

Abstract

Many data center virtualization solutions, such as VMware ESX, employ content-based page sharing to consolidate the resources of multiple servers. Page sharing identifies virtual machine memory pages with identical content and consolidates them into a single shared page. This technique, implemented at the host level, applies only between VMs placed on a given physical host. In a multiserver data center, opportunities for sharing may be lost because the VMs holding identical pages are resident on different hosts. In order to obtain the full benefit of content-based page sharing it is necessary to place virtual machines such that VMs with similar memory content are located on the same hosts.
In this paper we present Memory Buddies, a memory sharingaware placement system for virtual machines. This system includes a memory fingerprinting system to efficiently determine the sharing potential among a set of VMs, and compute more efficient placements. In addition it makes use of live migration to optimize VM placement as workloads change.We have implemented a prototype Memory Buddies system with VMware ESX Server and present experimental results on our testbed, as well as an analysis of an extensive memory trace study. Evaluation of our prototype using a mix of enterprise and e-commerce applications demonstrates an increase of data center capacity (i.e. number of VMs supported) of 17%, while imposing low overhead and scaling to as many as a thousand servers.

References

[1]
B. Bloom. Space/time trade-offs in hash coding with allowable errors. Communications of the ACM, 13(7):422--426, July 1970.
[2]
Andrei Broder and Michael Mitzenmacher. Network applications of Bloom filters: A survey. Internet Mathematics, 1(4):485--509, 2003.
[3]
Edouard Bugnion, Scott Devine, and Mendel Rosenblum. DISCO: Running Commodity Operating Systems on Scalable Multiprocessors. In SOSP, pages 143--156, 1997.
[4]
Emmanuel Cecchet, Anupam Chanda, Sameh Elnikety, Julie Marguerite, and Willy Zwaenepoel. Performance Comparison of Middleware Architectures for Generating Dynamic Web Content. In 4th ACM/IFIP/USENIX International Middleware Conference, June 2003.
[5]
C. Clark, K. Fraser, S. Hand, J. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfiel. Live migration of virtual machines. In Proceedings of Usenix Symposium on Network Systems Design and Implementation (NSDI), May 2005.
[6]
Laura Grit, David Irwin, Aydan Yumerefendi, and Jeff Chase. Virtual machine hosting for networked clusters: Building the foundations for autonomic orchestration. In Workshop on Virtualization Technology in Distributed Computing (VTDC), November 2006.
[7]
Deke Guo, Jie Wu, Honghui Chen, and Xueshan Luo. Theory and Network Applications of Dynamic Bloom Filters. In INFOCOM, 2006.
[8]
Diwaker Gupta, Sangmin Lee, Michael Vrable, Stefan Savage, Alex C. Snoeren, George Varghese, Geoffrey M. Voelker, and Amin Vahdat. Difference engine: Harnessing memory redundancy in virtual machines. In Usenix OSDI, December 2008.
[9]
Paul Hsieh. Hash functions. http://www.azillionmonkeys.com/qed/hash.html.
[10]
Navendu Jain, Michael Dahlin, and Renu Tewari. Using Bloom Filters to Refine Web Search Results. In WebDB, pages 25--30, 2005.
[11]
Jacob Kloster, Jesper Kristensen, and Arne Mejlholm. On the Feasibility of Memory Sharing: Content-Based Page Sharing in the Xen Virtual Machine Monitor. Master's thesis, Department of Computer Science, Aalborg University, June 2006.
[12]
M. Kozuch and M. Satyanarayanan. Internet suspend and resume. In Proceedings of the Fourth IEEE Workshop on Mobile Computing Systems and Applications, Calicoon, NY, June 2002.
[13]
Purushottam Kulkarni, Prashant J. Shenoy, andWeibo Gong. Scalable Techniques for Memory-efficients CDN Simulations. In WWW, 2003.
[14]
libvirt. The Virtualization API. http://libvirt.org.
[15]
Xucheng Luo, Zhiguang Qin, Ji Geng, and Jiaqing Luo. IAC: Interest-Aware Caching for Unstructured P2P. In SKG, page 58, 2006.
[16]
David Mosberger and Tai Jin. httperf: A tool for measuring web server performance. In First Workshop on Internet Server Performance, pages 59--67. ACM, June 1998.
[17]
E. Nahum. Deconstructing specweb, 2002.
[18]
Michael Nelson, Beng-Hong Lim, and Greg Hutchins. Fast Transparent Migration for Virtual Machines. In USENIX Annual Technical Conference, 2005.
[19]
M. Powell and B. Miller. Process migration in DEMOS/MP. Operating Systems Review, 17(5):110--119, 1983.
[20]
Apache Open For Business Project. http://ofbiz.apache.org.
[21]
Paul Ruth, Junghwan Rhee, Dongyan Xu, Rick Kennell, and Sebastien Goasguen. Autonomic Live Adaptation of Virtual Computational Environments in a Multi-Domain Infrastructure. In IEEE International Conference on Autonomic Computing (ICAC), June 2006.
[22]
Constantine P. Sapuntzakis, Ramesh Chandra, Ben Pfaff, Jim Chow, Monica S. Lam, and Mendel Rosenblum. Optimizing the migration of virtual computers. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation, December 2002.
[23]
W. Smith. TPC-W: Benchmarking An Ecommerce Solution. http://www.tpc.org/information/other/techarticles.asp.
[24]
The standard performance evaluation corporation (spec). http://www.spec.org.
[25]
A. Sundararaj, A. Gupta, and P. Dinda. Increasing Application Performance in Virtual Environments through Run-time Inference and Adaptation. In Fourteenth International Symposium on High Performance Distributed Computing (HPDC), July 2005.
[26]
M.M. Theimer, K.A.L., and D.R. Cheriton. Preemptable Remote Execution Facilities for the V-System. pages 2--12, December 1985.
[27]
C. Waldspurger. Memory Resource Management in VMWare ESX Server. In Proceedings of the Fifth Symposium on Operating System Design and Implementation (OSDI'02), December 2002.
[28]
A. Whitaker, M. Shaw, and S.D. Gribble. Scale and Performance in the Denali Isolation Kernel. In Proceedings of the Fifth Symposium on Operating System Design and Implementation (OSDI'02), December 2002.
[29]
VMware Whitepaper. Drs performance and best practices.
[30]
Timothy Wood, Ludmila Cherkasova, Kivanc Ozonat, and Prashant Shenoy. Profiling and modeling resource usage of virtualized applications. In International Middleware Conference, 2008.
[31]
Timothy Wood, Prashant J. Shenoy, Arun Venkataramani, and Mazin S. Yousif. Black-box and Gray-box Strategies for Virtual Machine Migration. In Networked Systems Design and Implementation (NSDI '07), 2007.

Cited By

View all
  • (2023)Joint Optimization of Memory Sharing and Communication Distance for Virtual Machine Instantiation in Cloudlet NetworksElectronics10.3390/electronics1220420512:20(4205)Online publication date: 10-Oct-2023
  • (2023)Securing Container-based Clouds with Syscall-aware SchedulingProceedings of the 2023 ACM Asia Conference on Computer and Communications Security10.1145/3579856.3582835(812-826)Online publication date: 10-Jul-2023
  • (2023)Boosting Cache Performance by Access Time MeasurementsACM Transactions on Storage10.1145/357277819:1(1-29)Online publication date: 17-Feb-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGOPS Operating Systems Review
ACM SIGOPS Operating Systems Review  Volume 43, Issue 3
July 2009
109 pages
ISSN:0163-5980
DOI:10.1145/1618525
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 July 2009
Published in SIGOPS Volume 43, Issue 3

Check for updates

Author Tags

  1. consolidation
  2. page sharing
  3. virtualization

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)25
  • Downloads (Last 6 weeks)4
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Joint Optimization of Memory Sharing and Communication Distance for Virtual Machine Instantiation in Cloudlet NetworksElectronics10.3390/electronics1220420512:20(4205)Online publication date: 10-Oct-2023
  • (2023)Securing Container-based Clouds with Syscall-aware SchedulingProceedings of the 2023 ACM Asia Conference on Computer and Communications Security10.1145/3579856.3582835(812-826)Online publication date: 10-Jul-2023
  • (2023)Boosting Cache Performance by Access Time MeasurementsACM Transactions on Storage10.1145/357277819:1(1-29)Online publication date: 17-Feb-2023
  • (2023)Nephele: Extending Virtualization Environments for Cloning Unikernel-based VMsProceedings of the Eighteenth European Conference on Computer Systems10.1145/3552326.3587454(574-589)Online publication date: 8-May-2023
  • (2023)Making Dynamic Page Coalescing Effective on Virtualized CloudsProceedings of the Eighteenth European Conference on Computer Systems10.1145/3552326.3567487(298-313)Online publication date: 8-May-2023
  • (2023)User-guided Page Merging for Memory Deduplication in Serverless Systems2023 IEEE International Conference on Big Data (BigData)10.1109/BigData59044.2023.10386487(159-169)Online publication date: 15-Dec-2023
  • (2021)Intelligent container reallocation at Microsoft 365Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3468264.3473936(1438-1443)Online publication date: 20-Aug-2021
  • (2020)Exploiting Virtual Machine Commonality for Improved Resource Allocation in Edge NetworksJournal of Sensor and Actuator Networks10.3390/jsan90400589:4(58)Online publication date: 13-Dec-2020
  • (2020)Hierarchical Orchestration of Disaggregated MemoryIEEE Transactions on Computers10.1109/TC.2020.296852569:6(844-855)Online publication date: 1-Jun-2020
  • (2019)Coordinate Memory Deduplication and Partition for Improving Performance in Cloud ComputingIEEE Transactions on Cloud Computing10.1109/TCC.2015.25117387:2(357-368)Online publication date: 1-Apr-2019
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media