Dynamic heterogeneity and the need for multicore virtualization

Abstract

As the computing industry enters the multicore era, exponential growth in the number of transistors on a chip continues to present challenges and opportunities for computer architects and system designers. We examine one emerging issue in particular: that of dynamic heterogeneity, which can arise, even among physically homogeneous cores, from changing reliability, power, or thermal conditions, different cache and TLB contents, or changing resource configurations. This heterogeneity results in a constantly varying pool of hardware resources, which greatly complicates software's traditional task of assigning computation to cores. In part to address dynamic heterogeneity, we argue that hardware should take a more active role in the management of its computation resources. We propose hardware techniques to virtualize the cores of a multicore processor, allowing hardware to flexibly reassign the virtual processors that are exposed, even to a single operating system, to any subset of the physical cores. We show that multicore virtualization operates with minimal overhead, and that it enables several novel resource management applications for improving both performance and reliability.

References

1. N. Aggarwal, P. Ranganathan, N.P. Jouppi, and J.E. Smith. Configurable isolation: building high availability systems with commodity multi-core processors. In Proc. of 34th ISCA, 2007.
2. W. Armstrong, R. Arndt, D. Boutcher, R. Kovacs, D. Larson, K. Lucke, N. Nayar, and R. Swanberg. Advanced virtualization capabilities of POWER5 systems. IBMJ. Res. & Dev., 49(4/5), 2005.
3. D. Bernick, B. Bruckert, P.D. Vigna, D. Garcia, R. Jardine, J. Klecka, and J. Smullen. Nonstop advanced architecture. In Proc. of 2005 DSN, 2005.
4. S. Borkar, T. Karnik, J. Tschanz, A. Keshavarzi, and V. De. Parameter variations and impact on circuits and microarchitecture. In Proc. of 40th DAC, 2003.
5. K. Bowman, S. Duvall, and J. Meindl. Impact of die-to-die and within-die parameter fluctuations on the maximum clock frequency distribution for gigascale integration. J. of Solid-State Circuits, 37(2):183--190, Feb 2002.
6. D. Brooks and M. Martonosi. Dynamic thermal management for high-performance microprocessors. In Proc. of 7th HPCA, 2001.
7. K. Chakraborty, P.M. Wells, and G.S. Sohi. Computation spreading: Employing hardware migration to specialize CMP cores on-the-fly. In Proc. of 12th ASPLOS, 2006.
8. K. Chakraborty, P.M. Wells, and G.S. Sohi. A case for an over-provisioned multicore system: Energy efficient processing of multithreaded programs. Technical Report CS-TR-2007-1607, University of Wisconsin-Madison, Aug 2007.
9. C. Constantinescu. Trends and challenges in VLSI circuit reliability. IEEE Micro, 23(4):14--19, 2003.
10. J. Dorsey, S. Searles, M. Ciraula, S. Johnson, N. Bujanos, D. Wu, M. Braganza, S. Meyers, E. Fang, and R. Kumar. An integrated quad-core Opteron processor. pages 102--103, Feb. 2007.
11. D. Ernst, N.S. Kim, S. Das, S. Pant, R. Rao, T. Pham, C. Ziesler, D. Blaauw, T. Austin, K. Flautner, and T. Mudge. Razor: A lowpower pipeline based on circuit-level timing speculation. In Proc. of 36th MICRO, 2003.
12. K. Govil, D. Teodosiu, Y. Huang, and M. Rosenblum. Cellular Disco: Resource management using virtual clusters on shared-memory multiprocessors. In Proc. of 16th SOSP, 1999.
13. M. Gschwind, P. Hofstee, B. Flachs, M. Hopkins, Y. Watanabe, and T. Yamazaki. A novel SIMD architecture for the Cell heterogeneous chip-multiprocessor. In Proc. of 17th Hot Chips, 2005.
14. S.H. Gunther, F. Binns, D.M. Carmean, and J.C. Hall. Managing the impact of increasing microprocessor power consumption. Intel Tech. J., Q1, 2001.
15. S.N. Hamilton and A. Orailoglu. Transient and intermittent fault recovery without rollback. In Proc. of 13th Defect and Fault-Tolerance in VLSI Sys., 1998.
16. S. Harizopoulos and A. Ailamaki. STEPS towards cache-resident transaction processing. In Proc. of 30th VLDB, 2004.
17. E. Ípek, M. Kirman, N. Kirman, and J. F. Martínez. Core fusion: accommodating software diversity in chip multiprocessors. In Proc. of 34th ISCA, 2007.
18. R. Kumar, D.M. Tullsen, P. Ranganathan, N.P. Jouppi, and K.I. Farkas. Single-ISA heterogeneous multi-core architectures for multithreaded workload performance. In Proc. of 31st ISCA, 2004.
19. C. LaFrieda, E. Ípek, J.F. Martínez, and R. Manohar. Utilizing dynamically coupled cores to form a resilient chip multiprocessor. In Proc. of 2007 DSN, 2007.
20. J.R. Larus and M. Parkes. Using cohort-scheduling to enhance server performance. In Proceedings of the General Track USENIX Annual Technical Conference, 2002.
21. J. Laudon. Performance/watt: the new server focus. Comp. Arch. News, 33(4):5--13, 2005.
22. T. Li, L.K. John, A. Sivasubramaniam, N. Vijaykrishnan, and J. Rubio. Understanding and improving operating system effects in control flow prediction. In Proc. of 10th ASPLOS, 2002.
23. P. Magnusson et al. Simics: A full system simulation platform. IEEE Comp., 35(2):50--58, Feb 2002.
24. D. McEvoy. The architecture of tandem's nonstop system. In Proc. of ACM 1981 Conf., 1981.
25. D. Nellans, R. Balasubramonian, and E. Brunvand. A case for increased operating system support in chip multi-processors. In Proc. of 2nd IBM Watson P=ac2, 2005.
26. J.K. Ousterhout. Scheduling techniques for concurrent systems. In Distributed Computing Systems, 1982.
27. M.D. Powell, M. Gomaa, and T.N. Vijaykumar. Heat-and-run: leveraging SMT and CMP to manage power density through the operating system. In Proc. of 11th ASPLOS, 2004.
28. K. Sankaralingam, R. Nagarajan, H. Liu, C. Kim, J. Huh, D. Burger, S.W. Keckler, and C.R. Moore. Exploiting ilp, tlp, and dlp with the polymorphous trips architecture. In Proc. of 30th ISCA, 2003.
29. Semiconductor Industry Association. International technology roadmap for semiconductors: Executive summary, 2005.
30. J.W. Sheaffer, D.P. Luebke, and K. Skadron. The visual vulnerability spectrum: characterizing architectural vulnerability for graphics hardware. In Proc. of 21st Eurographics GH, 2006.
31. P. Shivakumar, M. Kistler, S.W. Keckler, D. Burger, and L. Alvisi. Modeling the effect of technology trends on the soft error rate of combinational logic. In Proc. of 2002 DSN, 2002.
32. K. Skadron, M.R. Stan, W. Huang, S. Velusamy, K. Sankaranarayanan, and D. Tarjan. Temperature-aware microarchitecture. In Proc. of 30th ISCA, 2003.
33. T.J. Slegel et al. IBM's S/390 G5 microprocessor design. IEEE Micro, 19(2):12--23, 1999.
34. J.C. Smolens, B.T. Gold, B. Falsafi, and J.C. Hoe. Reunion: Complexity-effective multicore redundancy. In Proc. of 39th MICRO, 2006.
35. G.S. Sohi, S.E. Breach, and T.N. Vijaykumar. Multiscalar processors. In Proc. of 22nd ISCA, pages 414--425, 1995.
36. D.J. Sorin, M.M.K. Martin, M.D. Hill, and D.A. Wood. Safetynet: improving the availability of shared memory multiprocessors with global checkpoint/recovery. In Proc. of 29th ISCA, 2002.
37. Sun Microsystems, Inc. Sun fire high-end and midrange systems dynamic reconfiguration user's guide. Viewed 12/19/2007.
38. V. Uhlig, J. LeVasseur, E. Skoglund, and U. Dannowski. Towards scalable multiprocessor virtual machines. In Proc. of 3rd Virt. Mach. Research and Tech. Symp., 2004.
39. VMware. ESX Server -- best practices using VMware virtual SMP. Viewed 5/03/2006.
40. C.A. Waldspurger. Memory resource management in VMware ESX Server. In Proc. of 5th Symposium on OSDI, 2002.
41. P.M. Wells, K. Chakraborty, and G.S. Sohi. Hardware support for spin management in overcommitted virtual machines. In Proc. of 15th PACT, 2006.
42. P.M. Wells, K. Chakraborty, and G.S. Sohi. Adapting to intermittent faults in multicore systems. In Proc. of 13th ASPLOS, pages 255--264, 2008.
43. P.M. Wells, K. Chakraborty, and G.S. Sohi. Mixed-mode multicore reliability. In Proc. of 14th ASPLOS, 2009.
44. M. Welsh, D. Culler, and E. Brewer. SEDA: an architecture for well-conditioned, scalable internet services. In Proceedings of the 18th Symposium on Operating Systems Principles, 2001.
45. K. Wonyoung, G. Meeta, W. Gu-Yeon, and B. David. System level analysis of fast, per-core DVFS using on-chip switching regulators. In Proc. of 14th HPCA, February 2008.

Index Terms

• General and reference

  • Cross-computing tools and techniques

    • Performance

• Software and its engineering

  • Software organization and properties

    • Contextual software domains

      • Operating systems

        • Communications management

        • Process management

          • Concurrency control

    • Extra-functional properties

      • Software performance