skip to main content
article

Memory resource management in VMware ESX server

Published:31 December 2002Publication History
Skip Abstract Section

Abstract

VMware ESX Server is a thin software layer designed to multiplex hardware resources efficiently among virtual machines running unmodified commodity operating systems. This paper introduces several novel ESX Server mechanisms and policies for managing memory. A ballooning technique reclaims the pages considered least valuable by the operating system running in a virtual machine. An idle memory tax achieves efficient memory utilization while maintaining performance isolation guarantees. Content-based page sharing and hot I/O page remapping exploit transparent page remapping to eliminate redundancy and reduce copying overheads. These techniques are combined to efficiently support virtual machine workloads that overcommit memory.

References

  1. Andrea C. Arpaci-Dusseau and Remzi H. Arpaci-Dusseau. "Information and Control in Gray-Box Systems," Proc. Symposium on Operating System Principles, October 2001.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Ozalp Babaoglu and William Joy. "Converting a Swap-Based System to do Paging in an Architecture Lacking Page-Reference Bits," Proc. Symposium on Operating System Principles, December 1981.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Edouard Bugnion, Scott Devine, Kinshuk Govil, and Mendel Rosenblum. "Disco: Running Commodity Operating Systems on Scalable Multiprocessors," ACM Transactions on Computer Systems, 15(4), November 1997.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Pei Cao, Edward W. Felten, and Kai Li. "Implementation and Performance of Application-Controlled File Caching," Proc. Symposium on Operating System Design and Implementation, November 1994.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Jeffrey S. Chase, Darrell C. Anderson, Prachi N. Thakar, and Amin M. Vahdat. "Managing Energy and Server Resources in Hosting Centers," Proc. Symposium on Operating System Principles, October 2001.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. R. J. Creasy. "The Origin of the VM/370 Time-Sharing System," IBM Journal of Research and Development, 25(5), September 1981.]]Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Bryan Ford, Mike Hibler, Jay Lepreau, Patrick Tullman, Godmar Back, and Stephen Clawson. "Microkernels Meet Recursive Virtual Machines," Proc. Symposium on Operating System Design and Implementation, October 1996.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Robert P. Goldberg. "Survey of Virtual Machine Research," IEEE Computer, 7(6), June 1974.]]Google ScholarGoogle Scholar
  9. Kinshuk Govil, Dan Teodosiu, Yongqiang Huang, and Mendel Rosenblum. "Cellular Disco: Resource Management Using Virtual Clusters on Shared-Memory Multiprocessors," Proc. Symposium on Operating System Principles, December 1999.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Peter H. Gum. "System/370 Extended Architecture: Facilities for Virtual Machines," IBM Journal of Research and Development', 27(6), November 1983.]]Google ScholarGoogle Scholar
  11. Steven M. Hand. "Self-Paging in the Nemesis Operating System," Proc. Symposium on Operating Systems Design and Implementation, February 1999.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Kieran Harty and David R. Cheriton. "Application-Controlled Physical Memory using External Page-Cache Management," Proc. Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, October 1992.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Intel Corporation. IA-32 Intel Architecture Software Developer's Manual. Volumes I, II, and III, 2001.]]Google ScholarGoogle Scholar
  14. Bob Jenkins. "Algorithm Alley," Dr. Dobbs Journal, September 1997. Source code available from http://burtleburtle.net/bob/hash/]]Google ScholarGoogle Scholar
  15. Ted Kaehler. "Virtual Memory for an Object-Oriented Language," Byte, August 1981.]]Google ScholarGoogle Scholar
  16. Minkyong Kim and Brian Noble. "Mobile Network Estimation," Proc. Seventh Annual International Conference on Mobile Computing and Networking, July 2001.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Jochen Liedtke, Hermann Hartig, and Michael Hohmuth. "OS-Controlled Cache Predictability for Real-Time Systems," Proc. Third IEEE Real-Time Technology and Applications Symposium, June 1997.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Marshall K. McKusick, Keith Bostic, Michael J. Karels, and John S. Quaterman. The Design and Implementation of the 4.4 BSD Operating System, Addison-Wesley, 1996.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Theodore H. Romer, Dennis Lee, Brian N. Bershad and J. Bradley Chen. "Dynamic Page Mapping Policies for Cache Conflict Resolution on Standard Hardware," Proc. Symposium on Operating System Design and Implementation, November 1994.]]Google ScholarGoogle Scholar
  20. L. H. Seawright and R. A. McKinnon. "VM/370: A Study of Multiplicity and Usefulness," IBM Systems Journal, 18(1), 1979.]]Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Timothy Sherwood, Brad Calder and Joel S. Emer. "Reducing Cache Misses Using Hardware and Software Page Placement," Proc. International Conference on Supercomputing, June 1999.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. David A. Solomon and Mark Russinovich. Inside Microsoft Windows 2000, Third Ed., Microsoft Press, 2001.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Jeremy Sugerman, Ganesh Venkitachalam, and Beng-Hong Lim. "Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor," Proc. Usenix Annual Technical Conference, June 2001.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. David G. Sullivan, Robert Haas, and Margo I. Seltzer. "Tickets and Currencies Revisited: Extensions to Multi-Resource Lottery Scheduling," Proc. Seventh Workshop on Hot Topics in Operating Systems', March 1999.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. David G. Sullivan and Margo I. Seltzer. "Isolation with Flexibility: A Resource Management Framework for Central Servers," Proc. Usenix Annual Technical Conference, June 2000.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Andrew S. Tanenbaum. Modern Operating Systems, Prentice Hall, 1992.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. R. Tremaine, P. Franaszek, J. Robinson, C. Schulz, T. Smith, M. Wazlowski, and P. Bland. "IBM Memory Expansion Technology (MXT)," IBM Journal of Research and Development', 45(2), March 2001.]]Google ScholarGoogle Scholar
  28. Andrew Tridgell. "dbench" benchmark. Available from ftp://samba.org/pub/tridge/dbench/, September 2001.]]Google ScholarGoogle Scholar
  29. VMware, Inc. VMware ESX Server User's Manual Version 1.5, Palo Alto, CA, April 2002.]]Google ScholarGoogle Scholar
  30. Carl A. Waldspurger and William E. Weihl. "Lottery Scheduling: Flexible Proportional-Share Resource Management," Proc. Symposium on Operating System Design and Implementation, November 1994.]] Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Carl A. Waldspurger. Lottery and Stride Scheduling: Flexible Proportional-Share Resource Management, Ph.D. thesis, Technical Report MIT/LCS/TR-667, September 1995.]] Google ScholarGoogle Scholar
  32. Carl A. Waldspurger and William E. Weihl. "An Object-Oriented Framework for Modular Resource Management," Proc. Fifth Workshop on Object-Orientation in Operating Systems, October 1996.]] Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Memory resource management in VMware ESX server

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM SIGOPS Operating Systems Review
        ACM SIGOPS Operating Systems Review  Volume 36, Issue SI
        OSDI '02: Proceedings of the 5th Symposium on Operating Systems Design and Implementation
        Winter 2002
        398 pages
        ISSN:0163-5980
        DOI:10.1145/844128
        Issue’s Table of Contents

        Copyright © 2002 Copyright is held by the owner/author(s)

        Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 31 December 2002

        Check for updates

        Qualifiers

        • article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader