Reducing Latency in an SRAM/DRAM Cache Hierarchy via a Novel Tag-Cache Architecture

ABSTRACT

Memory speed has become a major performance bottleneck as more and more cores are integrated on a multi-core chip. The widening latency gap between high speed cores and memory has led to the evolution of multi-level SRAM/DRAM cache hierarchies that exploit the latency benefits of smaller caches (e.g. private L1 and L2 SRAM caches) and the capacity benefits of larger caches (e.g. shared L3 SRAM and shared L4 DRAM cache). The main problem of employing large L3/L4 caches is their high tag lookup latency. To solve this problem, we introduce the novel concept of small and low latency SRAM/DRAM Tag-Cache structures that can quickly determine whether an access to the large L3/L4 caches will be a hit or a miss. The performance of the proposed Tag-Cache architecture depends upon the Tag-Cache hit rate and to improve it we propose a novel Tag-Cache insertion policy and a DRAM row buffer mapping policy that reduce the latency of memory requests. For a 16-core system, this improves the average harmonic mean instruction per cycle throughput of latency sensitive applications by 13.3% compared to state-of-the-art.

References

1. S. Borkar and A. A. Chien, "The Future of Microprocessors", Communications of the ACM, vol. 54, no. 5, pp. 67--77, May 2011.
2. R. X. Arroyo, R. J. Harrington, S. P. Hartman, and T. Nguyen, "IBM POWER7 Systems", IBM Journal of Research and Development, vol. 55, no. 3, pp. 2:1--2:13, 2011.
3. F. Hameed, L. Bauer, and J. Henkel, "Simultaneously Optimizing DRAM Cache Hit Latency and Miss Rate via Novel Set Mapping Policies", in International Conference on Compilers, Architecture, and Synthesis for Embedded Systems (CASES), 2013.
4. G. Loh and M. Hill, "Efficiently Enabling Conventional Block Sizes for Very Large Die-stacked DRAM Caches", in 44th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2011, pp. 454--464.
5. G. Loh and M. Hill, "Supporting Very Large DRAM Caches with Compound Access Scheduling and MissMaps", in IEEE Micro Magazine, Special Issue on Top Picks in Computer Architecture Conferences, 2012.
6. M. Qureshi and G. Loh, "Fundamental Latency Trade-offs in Architecting DRAM Caches", in 45th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2012, pp. 235--246.
7. L. Zhao, R. Iyer, R. Illikkal, and D. Newell, "Exploring DRAM Cache Architecture for CMP Server Platforms", in 25th International Symposium on Computer Design (ICCD), 2007, pp. 55--62.
8. B. Black, M. Annavaram, N. Brekelbaum et al., "Die-Stacking (3D) Microarchitecture", in 39th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2006, pp. 469--479.
9. Y. Deng and W. Maly, "Interconnect Characteristics of 2.5-D System Integration Scheme", in International Symposium on Physical Design (ISPD), 2001, pp. 171--175.
10. F. Hameed, L. Bauer, and J. Henkel, "Adaptive Cache Management for a Combined SRAM and DRAM Cache Hierarchy for Multi-Cores", in 15th conference on Design, Automation and Test in Europe (DATE), 2013, pp. 77--82.
11. S. Thoziyoor, J. Muralimanohart, R. and Ahn, and N. Jouppi, "CACTI 5.1 HPL 2008/20, HP Labs", 2008.
12. G. Loh, S. Subramaniam, and Y. Xie, "Zesto: A Cycle-Level Simulator for Highly Detailed Microarchitecture Exploration", in International Symposium on Performance Analysis of Systems and Software (ISPASS), 2009.
13. "Standard Performance Evaluation Corporation", http://www.spec.org.
14. S. Rixner, W. Dally, U. Kapasi et al., "Memory Access Scheduling", in 32nd International Symposium on Computer Architecture (ISCA), 2000, pp. 128--138.

• Hardware

  • Integrated circuits

    • Semiconductor memory