QoS policies and architecture for cache/memory in CMP platforms

ABSTRACT

As we enter the era of CMP platforms with multiple threads/cores on the die, the diversity of the simultaneous workloads running on them is expected to increase. The rapid deployment of virtualization as a means to consolidate workloads on to a single platform is a prime example of this trend. In such scenarios, the quality of service (QoS) that each individual workload gets from the platform can widely vary depending on the behavior of the simultaneously running workloads. While the number of cores assigned to each workload can be controlled, there is no hardware or software support in today's platforms to control allocation of platform resources such as cache space and memory bandwidth to individual workloads. In this paper, we propose a QoS-enabled memory architecture for CMP platforms that addresses this problem. The QoS-enabled memory architecture enables more cache resources (i.e. space) and memory resources (i.e. bandwidth) for high priority applications based on guidance from the operating environment. The architecture also allows dynamic resource reassignment during run-time to further optimize the performance of the high priority application with minimal degradation to low priority. To achieve these goals, we will describe the hardware/software support required in the platform as well as the operating environment (O/S and virtual machine monitor). Our evaluation framework consists of detailed platform simulation models and a QoS-enabled version of Linux. Based on evaluation experiments, we show the effectiveness of a QoS-enabled architecture and summarize key findings/trade-offs.

REFERENCES

• 1: Azul Systems. Azul Compute Appliance. http://www.azulsystems.com/products/cpools_cappliance.html
• 2: Paul Barham , Boris Dragovic , Keir Fraser , Steven Hand , Tim Harris , Alex Ho , Rolf Neugebauer , Ian Pratt , Andrew Warfield, Xen and the art of virtualization, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 3: Dhruba Chandra , Fei Guo , Seongbeom Kim , Yan Solihin, Predicting Inter-Thread Cache Contention on a Chip Multi-Processor Architecture, Proceedings of the 11th International Symposium on High-Performance Computer Architecture, p.340-351, February 12-16, 2005 [doi>10.1109/HPCA.2005.27]
• 4: T. Deshane, D. Dimatos, et al. Performance Isolation of a Misbehaving Virtual Machine with Xen, VMware and Solaris Containers. http://people.clarkson.edu/~jnm/publications/isolationOfMisbehavingVMs.pdf.
• 5: Lisa R. Hsu , Steven K. Reinhardt , Ravishankar Iyer , Srihari Makineni, Communist, utilitarian, and capitalist cache policies on CMPs: caches as a shared resource, Proceedings of the 15th international conference on Parallel architectures and compilation techniques, September 16-20, 2006, Seattle, Washington, USA [doi>10.1145/1152154.1152161]
• 6: R. P. Goldberg. Survey of virtual machine research. IEEE Transactions on Computers, 1974.
• 7: Intel Corporation. Intel Dual-Core Processors-The First Multi-core Revolution. http://www.intel.com/technology/computing/dual-core/.
• 8: R. Iyer. On Modeling and Analyzing Cache Performance using CASPER. In Proc. of 11th International Symposium on Modeling, Analysis and Simulation of Computer & Telecom Systems, Oct 2003.
• 9: Ravi Iyer, CQoS: a framework for enabling QoS in shared caches of CMP platforms, Proceedings of the 18th annual international conference on Supercomputing, June 26-July 01, 2004, Malo, France [doi>10.1145/1006209.1006246]
• 10: Seongbeom Kim , Dhruba Chandra , Yan Solihin, Fair Cache Sharing and Partitioning in a Chip Multiprocessor Architecture, Proceedings of the 13th International Conference on Parallel Architectures and Compilation Techniques, p.111-122, September 29-October 03, 2004 [doi>10.1109/PACT.2004.15]
• 11: P. Kongetira, K. Aingaran, and K. Olukotun. Niagara: A 32-Way Multithreaded Sparc Processor.In Proc. of Annual International Symposium on Microarchitecture(MICRO), Mar 2005.
• 12: K. Krewell. Best Servers of 2004: Multicore is Norm. Microprocessor Report, www.mpronline.com, Jan 2005.
• 13: Rakesh Kumar , Dean M. Tullsen , Norman P. Jouppi , Parthasarathy Ranganathan, Heterogeneous Chip Multiprocessors, Computer, v.38 n.11, p.32-38, November 2005 [doi>10.1109/MC.2005.379]
• 14: J. Laudon. Performance/Watt: The New Server Focus. In 1st Workshop on Design, Architecture and Simulation of CMP (dasCMP), Nov 2005.
• 15: K. Lee, T. Lin and C. Jen. An Efficient Quality-Aware Memory Controller for Multimedia Platform SoC. IEEE Trans. On Circuits and Systems for Video Technology, May 2005.
• 16: Chitra Natarajan , Bruce Christenson , Fayé Briggs, A study of performance impact of memory controller features in multi-processor server environment, Proceedings of the 3rd workshop on Memory performance issues: in conjunction with the 31st international symposium on computer architecture, p.80-87, June 20-20, 2004, Munich, Germany [doi>10.1145/1054943.1054954]
• 17: Kyle J. Nesbit , Nidhi Aggarwal , James Laudon , James E. Smith, Fair Queuing Memory Systems, Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture, p.208-222, December 09-13, 2006 [doi>10.1109/MICRO.2006.24]
• 18: Kunle Olukotun , Basem A. Nayfeh , Lance Hammond , Ken Wilson , Kunyung Chang, The case for a single-chip multiprocessor, Proceedings of the seventh international conference on Architectural support for programming languages and operating systems, p.2-11, October 01-04, 1996, Cambridge, Massachusetts, United States
• 19: Moinuddin K. Qureshi , Yale N. Patt, Utility-Based Cache Partitioning: A Low-Overhead, High-Performance, Runtime Mechanism to Partition Shared Caches, Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture, p.423-432, December 09-13, 2006 [doi>10.1109/MICRO.2006.49]
• 20: Nauman Rafique , Won-Taek Lim , Mithuna Thottethodi, Architectural support for operating system-driven CMP cache management, Proceedings of the 15th international conference on Parallel architectures and compilation techniques, September 16-20, 2006, Seattle, Washington, USA [doi>10.1145/1152154.1152160]
• 21: Parthasarathy Ranganathan , Norman Jouppi, Enterprise IT Trends and Implications for Architecture Research, Proceedings of the 11th International Symposium on High-Performance Computer Architecture, p.253-256, February 12-16, 2005 [doi>10.1109/HPCA.2005.14]
• 22: Scott Rixner , William J. Dally , Ujval J. Kapasi , Peter Mattson , John D. Owens, Memory access scheduling, Proceedings of the 27th annual international symposium on Computer architecture, p.128-138, June 2000, Vancouver, British Columbia, Canada
• 23: Mendel Rosenblum , Tal Garfinkel, Virtual Machine Monitors: Current Technology and Future Trends, Computer, v.38 n.5, p.39-47, May 2005 [doi>10.1109/MC.2005.176]
• 24: L. Sha , R. Rajkumar , J. P. Lehoczky, Priority Inheritance Protocols: An Approach to Real-Time Synchronization, IEEE Transactions on Computers, v.39 n.9, p.1175-1185, September 1990 [doi>10.1109/12.57058 ]
• 25: SPECint, http://www.spec.org/cpu2000/SPECint
• 26: SPECjbb2005, http://www.spec.org/jbb2005
• 27: Harold S. Stone , John Turek , Joel L. Wolf, Optimal Partitioning of Cache Memory, IEEE Transactions on Computers, v.41 n.9, p.1054-1068, September 1992 [doi>10.1109/12.165388 ]
• 28: G. Edward Suh , Srinivas Devadas , Larry Rudolph, A New Memory Monitoring Scheme for Memory-Aware Scheduling and Partitioning, Proceedings of the 8th International Symposium on High-Performance Computer Architecture, p.117, February 02-06, 2002
• 29: "Test TCP (TTCP) Benchmarking Tool", http://www.pcausa.com
• 30: "TPC-C Design Document", http://www.tpc.org/tpcc/
• 31: Rich Uhlig , Gil Neiger , Dion Rodgers , Amy L. Santoni , Fernando C. M. Martins , Andrew V. Anderson , Steven M. Bennett , Alain Kagi , Felix H. Leung , Larry Smith, Intel Virtualization Technology, Computer, v.38 n.5, p.48-56, May 2005 [doi>10.1109/MC.2005.163]
• 32: R. Uhlig, R. Fishtein, et. al. SoftSDV: A Presilicon Software Development Environment for the IA-64 Architecture. Intel Technology Journal. (http://www.intel.com/technology/itjf)
• 33: Thomas Y. Yeh , Glenn Reinman, Fast and fair: data-stream quality of service, Proceedings of the 2005 international conference on Compilers, architectures and synthesis for embedded systems, September 24-27, 2005, San Francisco, California, USA [doi>10.1145/1086297.1086328]
• 34: L. Zhao, J. Moses, R. Iyer, et al. Architectural Evaluation of Large-Scale CMP Platforms using ManySim. In Intel's Design & Test Technology Conference (DTTC), Aug 2006.
• 35: H. Zhang. Service Disciplines for Guaranteed Performance Service in Packet-switching Networks. In Proc. of IEEE, Oct. 1995.
• 36: Zhichun Zhu , Zhao Zhang, A Performance Comparison of DRAM Memory System Optimizations for SMT Processors, Proceedings of the 11th International Symposium on High-Performance Computer Architecture, p.213-224, February 12-16, 2005 [doi>10.1109/HPCA.2005.2]

INDEX TERMS

Primary Classification:
  B. Hardware
  B.3 MEMORY STRUCTURES
      B.3.2 Design Styles
          Subjects: Cache memories

General Terms:
Algorithms, Design, Experimentation, Management, Measurement, Performance

Keywords:
CMP, QoS, cache/memory, performance, quality of service, resource sharing priniciples, service level agreements