Optimizing the migration of virtual computers

ABSTRACT

This paper shows how to quickly move the state of a running computer across a network, including the state in its disks, memory, CPU registers, and I/O devices. We call this state a capsule. Capsule state is hardware state, so it includes the entire operating system as well as applications and running processes.We have chosen to move x86 computer states because x86 computers are common, cheap, run the software we use, and have tools for migration. Unfortunately, x86 capsules can be large, containing hundreds of megabytes of memory and gigabytes of disk data. We have developed techniques to reduce the amount of data sent over the network: copy-on-write disks track just the updates to capsule disks, "ballooning" zeros unused memory, demand paging fetches only needed blocks, and hashing avoids sending blocks that already exist at the remote end. We demonstrate these optimizations in a prototype system that uses VMware GSX Server virtual machine monitor to create and run x86 capsules. The system targets networks as slow as 384 kbps.Our experimental results suggest that efficient capsule migration can improve user mobility and system management. Software updates or installations on a set of machines can be accomplished simply by distributing a capsule with the new changes. Assuming the presence of a prior capsule, the amount of traffic incurred is commensurate with the size of the update or installation package itself. Capsule migration makes it possible for machines to start running an application within 20 minutes on a 384 kbps link, without having to first install the application or even the underlying operating system. Furthermore, users' capsules can be migrated during a commute between home and work in even less time.

REFERENCES

• 1: FIPS 180-1. Announcement of weakness in the secure hash standard. Technical report, National Institute of Standards and Technology (NIST), April 1994.
• 2: Andrea C. Arpaci-Dusseau , Remzi H. Arpaci-Dusseau, Information and control in gray-box systems, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 3: A. A. Awadallah and M. Rosenblum. The vMatrix: A network of virtual machine monitors for dynamic content distribution. In Seventh International Workshop on Web Content Caching and Distribution, August 2002.
• 4: Edouard Bugnion , Scott Devine , Kinshuk Govil , Mendel Rosenblum, Disco: running commodity operating systems on scalable multiprocessors, ACM Transactions on Computer Systems (TOCS), v.15 n.4, p.412-447, Nov. 1997 [doi>10.1145/265924.265930]
• 5: Florent Chabaud , Antoine Joux, Differential Collisions in SHA-0, Proceedings of the 18th Annual International Cryptology Conference on Advances in Cryptology, p.56-71, August 23-27, 1998
• 6: Peter M. Chen , Brian D. Noble, When Virtual Is Better Than Real, Proceedings of the Eighth Workshop on Hot Topics in Operating Systems, p.133, May 20-22, 2001
• 7: http://www.connectix.com/.
• 8: P. Deutsch. Zlib compressed data format specification version 3.3, May 1996.
• 9: R. P. Goldberg. Survey of virtual machine research. Computer, 7(6):34--45, June 1974.
• 10: A. Grimshaw , A. Ferrari , F. Knabe , M. Humphrey, Legion: An Operating System for Wide-Area Computing, University of Virginia, Charlottesville, VA, 1999
• 11: IA-32 Intel architecture software developer's manual volumes 1--3. http://developer.intel.com/design/pentium4/manuals/.
• 12: IBM Virtual Machine/370 Planning Guide. IBM Corporation, 1972.
• 13: Anthony D. Joseph , Joshua A. Tauber , M. Frans Kaashoek, Mobile Computing with the Rover Toolkit, IEEE Transactions on Computers, v.46 n.3, p.337-352, March 1997 [doi>10.1109/12.580429 ]
• 14: Eric Jul , Henry Levy , Norman Hutchinson , Andrew Black, Fine-grained mobility in the Emerald system, ACM Transactions on Computer Systems (TOCS), v.6 n.1, p.109-133, Feb. 1988 [doi>10.1145/35037.42182]
• 15: Michael Kozuch , M. Satyanarayanan, Internet Suspend/Resume, Proceedings of the Fourth IEEE Workshop on Mobile Computing Systems and Applications, p.40, June 20-21, 2002
• 16: M. Litzkow, M. Livny, and M. Mutka. Condor - a hunter of idle workstations. In Proceedings of the 8th International Conference on Distributed Computing Systems, pages 104--111, June 1988.
• 17: U. Manber. Finding similar files in a large file system. In Proceedings of the USENIX Winter 1994 Technical Conference, pages 1--10, 17--21 1994.
• 18: J. Moore and J. Chase. Cluster on demand. Technical report, Duke University, May 2002.
• 19: Athicha Muthitacharoen , Benjie Chen , David Mazières, A low-bandwidth network file system, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 20: M. Muuss. The story of T-TCP.
• 21: J. Plank, M. Beck, G. Kingsley, and K. Li. Libckpt: Transparent checkpointing under Unix. In Proceedings of the USENIX Winter 1995 Technical Conference, pages 213--224, January 1995.
• 22: F. Rauch, C. Kurmann, and T. Stricker. Partition repositories for partition cloning---OS independent software maintenance in large clusters of PCs. In Proceedings of the IEEE International Conference on Cluster Computing 2000, pages 233--242, 2000.
• 23: Luigi Rizzo, Dummynet: a simple approach to the evaluation of network protocols, ACM SIGCOMM Computer Communication Review, v.27 n.1, p.31-41, Jan. 1997 [doi>10.1145/251007.251012]
• 24: Brian Keith Schmidt , Monica S. Lam, Supporting ubiquitous computing with stateless consoles and computation caches, 2000
• 25: Georg Stellner, CoCheck: Checkpointing and Process Migration for MPI, Proceedings of the 10th International Parallel Processing Symposium, p.526-531, April 15-19, 1996
• 26: Marvin M. Theimer , Keith A. Lantz , David R. Cheriton, Preemptable remote execution facilities for the V-system, Proceedings of the tenth ACM symposium on Operating systems principles, p.2-12, December 1985, Orcas Island, Washington, United States
• 27: A. Tridgell. Efficient Algorithms for Sorting and Synchronization. PhD thesis, Australian National University, April 2000.
• 28: "GSX server", white paper. http://www.vmware.com/pdf/gsx_whitepaper.pdf.
• 29: Carl A. Waldspurger, Memory resource management in VMware ESX server, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060307]
• 30: A. Whitaker, M. Shaw, and S. Gribble. Denali: Lightweight virtual machines for distributed and networked applications. Technical report, University of Washington, February 2001.
• 31: Brian White , Jay Lepreau , Leigh Stoller , Robert Ricci , Shashi Guruprasad , Mac Newbold , Mike Hibler , Chad Barb , Abhijeet Joglekar, An integrated experimental environment for distributed systems and networks, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060313]
• 32: Wintel architecture specifications. http://www.microsoft.com/hwdev/specs/.