Mortar: filling the gaps in data center memory

ABSTRACT

Data center servers are typically overprovisioned, leaving spare memory and CPU capacity idle to handle unpredictable workload bursts by the virtual machines running on them. While this allows for fast hotspot mitigation, it is also wasteful. Unfortunately, making use of spare capacity without impacting active applications is particularly difficult for memory since it typically must be allocated in coarse chunks over long timescales. In this work we propose re- purposing the poorly utilized memory in a data center to store a volatile data store that is managed by the hypervisor. We present two uses for our Mortar framework: as a cache for prefetching disk blocks, and as an application-level distributed cache that follows the memcached protocol. Both prototypes use the framework to ask the hypervisor to store useful, but recoverable data within its free memory pool. This allows the hypervisor to control eviction policies and prioritize access to the cache. We demonstrate the benefits of our prototypes using realistic web applications and disk benchmarks, as well as memory traces gathered from live servers in our university's IT department. By expanding and contracting the data store size based on the free memory available, Mortar improves average response time of a web application by up to 35% compared to a fixed size memcached deployment, and improves overall video streaming performance by 45% through prefetching.

References

1. G. Ananthanarayanan, A. Ghodsi, S. Shenker, and I. Stoica. Disk-locality in datacenter computing considered irrelevant. In Proceedings of the 13th USENIX conference on Hot topics in operating systems, HotOS'13, pages 12--12, Berkeley, CA, USA, 2011. USENIX Association.
2. A. C. Arpaci-Dusseau and R. H. Arpaci-Dusseau. Information and control in gray-box systems. In Proceedings of the eighteenth ACM symposium on Operating systems principles, pages 43--56, Banff, Alberta, Canada, 2001. ACM.
3. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In Proceedings of the ACM Symposium on Operating Systems Principles, 2003.
4. L. A. Barroso and U. Holzle. The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines. 2009.
5. N. C. Burnett, J. Bent, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. Exploiting Gray-Box knowledge of Buffer-Cache management. In Proceedings of the annual conference on USENIX Annual Technical Conference, pages 29--44, 2002.
6. A. R. Butt, C. Gniady, and Y. C. Hu. The performance impact of kernel prefetching on buffer cache replacement algorithms. In Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems , SIGMETRICS '05, pages 157--168, New York, NY, USA, 2005.
7. X. Ding, S. Jiang, F. Chen, K. Davis, and X. Zhang. Diskseen: exploiting disk layout and access history to enhance i/o prefetch. In USENIX Annual Technical Conference, pages 20:1--20:14, 2007.
8. FileBench. http://sourceforge.net/projects/filebench/.
9. B. S. Gill and D. S. Modha. SARC: sequential prefetching in adaptive replacement cache. In Proceedings of the USENIX Annual Technical Conference. Berkeley, CA, USA, 2005.
10. D. Gupta, S. Lee, M. Vrable, S. Savage, A. C. Snoeren, G. Varghese, G. M. Voelker, and A. Vahdat. Difference engine: Harnessing memory redundancy in virtual machines. USENIX, 2008.
11. J. Hwang and T. Wood. Adaptive performance-aware distributed memory caching. USENIX Internation Conference on Autonomic Computing, 2013.
12. P. R. Jelenkovic and A. Radovanovic. Optimizing lru caching for variable document sizes. Combinatorics, Probability & Computing, 13(4--5):627--643, 2004.
13. S. T. Jones, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. Geiger: monitoring the buffer cache in a virtual machine environment. In Proceedings of the 12th international conference on architectural support for programming languages and operating systems, pages 14--24, 2006.
14. Y. Joo, J. Ryu, S. Park, and K. Shin. FAST: quick application launch on solid-state drives. In Proceedings of the 9th USENIX conference on File and stroage technologies, pages 19--19. USENIX Association,2011.
15. P. Lu and K. Shen. Virtual machine memory access tracing with hypervisor exclusive cache. In Proceedings of the USENIX Annual Technical Conference, ATC'07, pages 3:1--3:15, Berkeley, CA, USA, 2007.
16. D. Magenheimer, C. Mason, D. McCracken, and K. Hackel. Transcendent memory and linux. Oracle Corp., 2009.
17. M. Masmano, I. Ripoll, A. Crespo, and J. Real. Tlsf: A new dynamic memory allocator for real-time systems. ECRTS, 2004.
18. G. Milos, D. G. Murray, S. Hand, and M. A. Fetterman. Satori: Enlightened page sharing. USENIX, 2009.
19. R. Nishtala, H. Fugal, S. Grimm, M. Kwiatkowski, H. Lee, H. C. Li, R. McElroy, M. Paleczny, D. Peek, P. Saab, D. Stafford, T. Tung, and V. Venkataramani. Scaling memcache at facebook. USENIX Symposium on Networked Systems Design and Implementation, 2013.
20. R. Owens and W. Wang. Non-interactive os fingerprinting through memory de-duplication technique in virtual machines. In Proceedings of the 30th IEEE International Performance Computing and Communications Conference, PCCC '11, pages 1--8. IEEE Computer Society, 2011.
21. W. Sobel, S. Subramanyam, A. Sucharitakul, J. Nguyen, H. Wong, A. Klepchukov, S. Patil, O. Fox, and D. Patterson. Cloudstone: Multi-platform, multi-language benchmark and measurement tools for web 2.0, 2008.
22. S. Srinath, O. Mutlu, H. Kim, and Y. N. Patt. Feedback directed prefetching: Improving the performance and bandwidth-efficiency of hardware prefetchers. In Proceedings of the 2007 IEEE 13th International Symposium on High Performance Computer Architecture, pages 63--74, 2007.
23. C. Stewart, A. Chakrabarti, and R. Griffith. Zoolander: Efficiently meeting very strict, low-latency slos. USENIX ICAC, 2013.
24. VMware. Resource management with vmware drs. Technical Re- source Center, 2006.
25. C. A. Waldspurger. Memory resource management in vmware esx server. OSDI, 2002.
26. A. Warfield, R. Ross, K. Fraser, C. Limpach, and S. Hand. Parallax: managing storage for a million machines. In Proceedings of the 10th conference on Hot Topics in Operating Systems - Volume 10, HOTOS'05, pages 4--4, Berkeley, CA, USA, 2005.
27. D. Williams, H. Jamjoom, Y.-H. Liu, and H. Weatherspoon. Over- driver: handling memory overload in an oversubscribed cloud. In 7th International Conference on Virtual Execution Environments, pages 205--216. ACM, 2011.
28. W. Zhao and Z. Wang. Dynamic memory balancing for virtual machines. VEE, 2009.
29. T. Zhu, A. Gandhi, M. Harchol-Balter, and M. A. Kozuch. Saving cache by using less cache. HotCloud, 2012.

Index Terms

• Software and its engineering

  • Software organization and properties

    • Contextual software domains

      • Operating systems

        • Memory management

          • Distributed memory