NCID: a non-inclusive cache, inclusive directory architecture for flexible and efficient cache hierarchies

ABSTRACT

Chip-multiprocessor (CMP) architectures employ multi-level cache hierarchies with private L2 caches per core and a shared L3 cache like Intel's Nehalem processor and AMD's Barcelona processor. When designing a multi-level cache hierarchy, one of the key design choices is the inclusion policy: inclusive, non-inclusive or exclusive. Either choice has its benefits and drawbacks. An inclusive cache hierarchy (like Nehalem's L3) has the benefit of allowing incoming snoops to be filtered at the L3 cache, but suffers from (a) reduced space efficiency due to replication between the L2 and L3 caches and (b) reduced flexibility since it cannot bypass the L3 cache for transient or low priority data. In an inclusive L2/L3 cache hierarchy, it also becomes difficult to flexibly chop L3 cache size (or increase L2 cache size) for different product instantiations because the inclusion can start to affect performance (due to significant back-invalidates). In this paper, we present a novel approach to addressing the drawbacks of inclusive caches, while retaining its positive features of snoop filtering. We present NCID: a non-inclusive cache, inclusive directory architecture that allows data in the L3 to be non-inclusive or exclusive, but retains tag inclusion in the directory to support complete snoop filtering. We then describe and evaluate a range of NCID-based architecture options and policies. Our evaluation shows that NCID enables a flexible and efficient cache hierarchy for future CMP platforms and has the potential to improve performance significantly for several important server benchmarks.

References

1. J. L. Baer and W .H. Wang. "On the inclusion properties for multi-level cache hierarchies," Proceedings of the 15th Annual International Symposium on Computer Architecture, page 73--80, 1988.
2. L. Barroso, K. Gharachorloo, R. McNamara, A. Nowatzyk, S. Qadeer, B. Sano, S. Smith, R. Stets, and B. Verghese. Piranha: A Scalable Architecture Based on Single-Chip Multiprocessing. Proceedings of the 27th Annual International Symposium on Computer Architecture, pages 282--293, June 2000.
3. Bradford M. Beckmann, Michael R. Marty, and David A. Wood, "ASR: Adaptive Selective Replication for CMP Caches," 39th International Symposium on Microarchitecture (MICRO), December 2006.
4. Bradford M. Beckmann and David A. Wood, "Managing Wire Delay in Large Chip-Multiprocessor Caches," 37th International Symposium on Microarchitecture (MICRO), December 2004.
5. L. Cheng, N. Muralimanohar, K. Ramani, R. Balasubramonian, and J. Carter. Interconnect-Aware Coherence Protocols for Chip Multiprocessors. In Proceedings of ISCA-33, June 2006.
6. Z. Chishti, M. D. Powell, and T. N. Vijaykumar. "Optimizing replication, communication and capacity allocation in CMPs," In the 32nd ISCA, pages 357--368, June 2005.
7. S. Ghai, J. Joyner, and L. John. Investigating the Effectiveness of a Third Level Cache. Technical Report TR-980501-01, Laboratory for Computer Architecture, The University of Texas at Austin, May 1998.
8. Intel® Microarchitecture (Nehalem), http://www.intel.com/technology/architecture-silicon/next-gen/
9. R. Iyer, "CQoS: A Framework for Enabling QoS in Shared Caches of CMP Platforms," 18th Annual International Conference on Supercomputing (ICS'04), July 2004.
10. R. Iyer, L. Zhao, et al., "QoS Policies and Architecture for Cache/Memory in CMP Platforms", the ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), June 2007
11. N. P. Jouppi and Steven J. E. Wilton, "Tradeoffs in Two-Level On-chip Caching," Proceedings of the 21st annual international symposium on Computer architecture 1994 , Chicago, Illinois, United States
12. C. Kim, D.C. Burger, and S.W. Keckler, "NUCA: A Non-Uniform Cache Access Architecture for Wire-Delay Dominated On-Chip Caches," IEEE Micro Special Issue (Top Picks in Computer Architecture), Nov/Dec 2003.
13. M. M. K. Martin, M. D. Hill, and D. A. Wood. "Token coherence: Decoupling performance and correctness," In the 30th ISCA, pages 182--193, June 2003.
14. Michael R. Marty and Mark D. Hill, "Virtual Hierarchies," IEEE Micro Special Issue: Micro's Top Picks from Microarchitecture Conferences, January-February 2008.
15. N. Muralimanohar, R. Balasubramonian, "Interconnect Design Considerations in Large NUCA caches," Proceedings of the 34th annual international symposium on Computer architecture 2007, San Diego, California, USA.
16. S. Przybylski, M. Horowitz, and J. Hennessy, "Characteristics of performance-optimal multi-level cache hierarchies," In Proceedings of the 16th Annual international Symposium on Computer Architecture (Jerusalem, Israel). ISCA '89. ACM, New York, NY, 114--121. DOI= http://doi.acm.org/10.1145/74925.74939
17. M. K. Qureshi, A. Jaleel, Y. N. Patt, S. C. Steely Jr., and J. Emer. "Adaptive Insertion Policies for High-Performance Caching", in the International Symposium on Computer Architecture (ISCA), 2007
18. Sap America Inc., "SAP Standard Benchmarks," http://www.sap.com/solutions/benchmark/index.epx
19. A. J. Smith, "Cache Memories," ACM Computing Surveys, Vol.14, No.3, September 1982.
20. SPECjbb2005, http://www.spec.org/jbb2005/
21. E. Speight, H. Shafi, L. Zhang, and R. Rajamony, "Adaptive Mechanisms and Policies for Managing Cache Hierarchies in Chip Multiprocessors," Proceedings of the 32nd annual international symposium on Computer Architecture 2005.
22. The TPC-C Benchmark, http://www.tpc.org/tpcc/
23. The TPC-E Benchmark, http://www.tpc.org/tpce/
24. B. Waldecker, "AMD Quad Core Processor Overview", http://www.amd.com/us-en/Processors/TechnicalResources/0,,30_182,00.html
25. M. Zhang and K. Asanovic. "Victim replication: Maximizing capacity while hiding wire delay in tiled CMPs", In the 32nd ISCA, pages 336--345, June 2005.
26. L. Zhao, R. Iyer, J. Moses, R. Illikkal, S. Makineni and D. Newell, "Exploring Large-scale CMP Architectures using ManySim", IEEE Micro, July/August 2000

Index Terms

• Hardware

  • Integrated circuits

    • Semiconductor memory

      • Dynamic memory