Optimizing the migration of virtual computers

Abstract

This paper shows how to quickly move the state of a running computer across a network, including the state in its disks, memory, CPU registers, and I/O devices. We call this state a capsule. Capsule state is hardware state, so it includes the entire operating system as well as applications and running processes.We have chosen to move x86 computer states because x86 computers are common, cheap, run the software we use, and have tools for migration. Unfortunately, x86 capsules can be large, containing hundreds of megabytes of memory and gigabytes of disk data. We have developed techniques to reduce the amount of data sent over the network: copy-on-write disks track just the updates to capsule disks, "ballooning" zeros unused memory, demand paging fetches only needed blocks, and hashing avoids sending blocks that already exist at the remote end. We demonstrate these optimizations in a prototype system that uses VMware GSX Server virtual, machine monitor to create and run x86 capsules. The system targets networks as slow as 384 kbps.Our experimental results suggest that efficient capsule migration can improve user mobility and system management. Software updates or installations on a set of machines can be accomplished simply by distributing a capsule with the new changes. Assuming the presence of a prior capsule, the amount of traffic incurred is commensurate with the size of the update or installation package itself. Capsule migration makes it possible for machines to start running an application within 20 minutes on a 384 kbps link, without having to first install the application or even the underlying operating system. Furthermore, users' capsules can be migrated during a commute between home and work in even less time.

References

1. FIPS 180-1. Announcement of weakness in the secure hash standard. Technical report, National Institute of Standards and Technology (NIST), April 1994.]]
2. A. C. Arpaci-Dusseau and R. H. Arpaci-Dusseau. Information and control in gray-box systems. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP '01), pages 43--59, October 2001.]]
3. A. A. Awadallah and M. Rosenblum. The vMatrix: A network of virtual machine monitors for dynamic content distribution. In Seventh International Workshop on Web Content Caching and Distribution, August 2002.]]
4. E. Bugnion, S. Devine, and M. Rosenblum. Disco: Running commodity operating systems on scalable multiprocessors. ACM Transactions on Computer Systems, 15(4):412--447, November 1997.]]
5. F. Chabaud and A. Joux. Differential collisions in SHA-0. In Proceedings of CRYPTO '98, 18th Annual International Cryptology Conference, pages 56--71, August 1998.]]
6. P. M. Chen and B. D. Noble. When virtual is better than real. In Proceedings of the 8th IEEE Workshop on Hot Topics on Operating Systems, May 2001.]]
7. http://www.connectix.com/.]]
8. P. Deutsch. Zlib compressed data format specification version 3.3, May 1996.]]
9. R. P. Goldberg. Survey of virtual machine research. Computer, 7(6):34--45, June 1974.]]
10. A. Grimshaw, A. Ferrari, F. Knabe, and M. Humphrey. Legion: An operating system for wide-area computing. Technical Report CS99-12, Dept. of Computer Science, University of Virginia, March 1999.]]
11. IA-32 Intel architecture software developer's manual volumes 1--3. http://developer.intel.com/design/pentium4/manuals/.]]
12. IBM Virtual Machine/370 Planning Guide. IBM Corporation, 1972.]]
13. A. Joseph, J. Tauber, and M. Kaashoek. Mobile computing with the Rover toolkit. IEEE Transactions on Computers, 46(3):337--352, March 1997.]]
14. E. Jul, H. Levy, N. Hutchinson, and A. Black. Finegrained mobility in the Emerald system. ACM Transaction on Computer Systems, 6(1):109--133, February 1988.]]
15. M. Kozuch and M. Satyanarayanan. Internet suspend/resume. In Proceedings of the Workshop on Mobile Computing Systems and Applications, pages 40--46, June 2002.]]
16. M. Litzkow, M. Livny, and M. Mutka. Condor - a hunter of idle workstations. In Proceedings of the 8th International Conference on Distributed Computing Systems, pages 104--111, June 1988.]]
17. U. Manber. Finding similar files in a large file system. In Proceedings of the USENIX Winter 1994 Technical Conference, pages 1--10, 17--21 1994.]]
18. J. Moore and J. Chase. Cluster on demand. Technical report, Duke University, May 2002.]]
19. A. Muthitacharoen, B. Chen, and D. Mazières. A low-bandwidth network file system. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP '01), pages 174--187, October 2001.]]
20. M. Muuss. The story of T-TCP.]]
21. J. Plank, M. Beck, G. Kingsley, and K. Li. Libckpt: Transparent checkpointing under Unix. In Proceedings of the USENIX Winter 1995 Technical Conference, pages 213--224, January 1995.]]
22. F. Rauch, C. Kurmann, and T. Stricker. Partition repositories for partition cloning---OS independent software maintenance in large clusters of PCs. In Proceedings of the IEEE International Conference on Cluster Computing 2000, pages 233--242, 2000.]]
23. L. Rizzo. Dummynet: a simple approach to the evaluation of network protocols. ACM Computer Communication Review, 27(1):31--41, Jan. 1997.]]
24. B. K. Schmidt. Supporting Ubiquitous Computing with Stateless Consoles and Computation Caches. PhD thesis, Computer Science Department, Stanford University, August 2000.]]
25. G. Stellner. CoCheck: Checkpointing and process migration for MPI. In Proceedings of the l0th International Parallel Processing Symposium, pages 526--531, April 1996.]]
26. M. M. Theimer, K. A. Lantz, and D. R. Cheriton. Preemptable remote execution facilities for the Vsystem. In Proc. l0th Symposium on Operating Systems Principles, pages 10--12, December 1985.]]
27. A. Tridgell. Efficient Algorithms for Sorting and Synchronization. PhD thesis, Australian National University, April 2000.]]
28. "GSX server", white paper. http://www.vmware.com/pdf/gsx_whitepaper.pdf.]]
29. C. A. Waldspurger. Memory resource management in VMware ESX server. In Proceedings of the Fifth Symposium on Operating Systems Design and Implementation, December 2002.]]
30. A. Whitaker, M. Shaw, and S. Gribble. Denali: Lightweight virtual machines for distributed and networked applications. Technical report, University of Washington, February 2001.]]
31. B. White et al. An integrated experimental environment for distributed systems and networks. In Proceedings of the Fifth Symposium on Operating Systems Design and Implementation, December 2002.]]
32. Wintel architecture specifications. http://www.microsoft.com/hwdev/specs/.]]

Index Terms

• Social and professional topics

  • Professional topics

    • Management of computing and information systems

      • File systems management

      • Implementation management

      • System management

• Software and its engineering

  • Software creation and management

    • Software development process management

      • Software development methods

  • Software organization and properties

    • Contextual software domains

      • Operating systems

        • Memory management

          • Distributed memory