Banshee: Bandwidth-Efficient DRAM Caching via Software/Hardware Cooperation!

Size: px

Start display at page:

Download "Banshee: Bandwidth-Efficient DRAM Caching via Software/Hardware Cooperation!"

Ezra Hawkins
5 years ago
Views:

1 Banshee: Bandwidth-Efficient DRAM Caching via Software/Hardware Cooperation! Xiangyao Yu 1, Christopher Hughes 2, Nadathur Satish 2, Onur Mutlu 3, Srinivas Devadas 1 1 MIT 2 Intel Labs 3 ETH Zürich 1

2 High-Bandwidth In-Package In-package DRAM has - 5X higher bandwidth than off-package DRAM Core! In-Package! Off-Package! - Similar latency as offpackage DRAM - Limited capacity (up to 16 GB) SRAM! Hierarchy! 16 GB 384 GB > 400 GB/s In-package DRAM can be Memory Controller! 90 GB/s used as a cache On-Chip In-Package 2 * Numbers from Intel Knights Landing

3 Bandwidth Inefficiency in Existing DRAM Designs! Drawback 1: Metadata traffic (e.g., tags, LRU bits, frequency 6 Hit Metadata counters, etc.) Bytes per Instruction Unison Alloy BEAR Tagless DRAM Traffic Breakdown 3

4 Bandwidth Inefficiency in Existing DRAM Designs! Drawback 1: Metadata traffic (e.g., tags, LRU bits, frequency 6 Hit Metadata Replacement Coarse-Granularity counters, etc.) Drawback 2: replacement traffic - Especially for coarse-granularity (e.g., page-granularity) DRAM Bytes per Instruction Fine-Granularity cache designs 0 Unison Alloy BEAR Tagless DRAM Traffic Breakdown 4

5 Banshee Improves DRAM Bandwidth Efficiency! Idea 1: Page-table-based contents tracking with efficient 6 Hit Metadata Replacement translation lookaside buffer (TLB) coherence - Track contents of DRAM cache using page tables and TLBs - Lightweight TLB coherence mechanism Bytes per Instruction This paper 0 Unison Alloy BEAR Tagless Banshee DRAM Traffic Breakdown 5

6 Banshee Improves DRAM Bandwidth Efficiency! Idea 1: Page-table-based contents tracking with efficient 6 Hit Metadata Replacement translation lookaside buffer (TLB) coherence Idea 2: Bandwidth-aware frequency-based replacement (FBR) policy - Replacement traffic reduction: Limit Bytes per Instruction This paper the rate of DRAM cache replacement 0 - Metadata traffic reduction: Access Unison Alloy BEAR Tagless Banshee metadata for a sampled fraction of memory accesses DRAM Traffic Breakdown 6

7 Page-Table-Based DRAM Contents Tracking! Track DRAM cache contents using the the virtual memory mechanism Advantage - Zero overhead for tag storage and lookup Software Page Table Entry PPN Page Table Hardware Core! TLB Entry VPN PPN Translation Lookaside Buffer (TLB) SRAM Hierarchy! Disadvantage - TLB coherence overhead - replacement overhead Memory Controller In-Package! Off-Package! 7 * assuming 4-way set associativity DRAM cache

8 Idea 1: Efficient TLB Coherence! Software Hardware Track DRAM cache contents using page tables and TLBs Page Table Entry PPN Mapping Page Table Core! TLB Entry VPN PPN d! (1 bit)! Translation Lookaside Buffer (TLB) SRAM Hierarchy! Way! (2 bits)! Mapping Memory Controller In-Package! Off-Package! 8 * Assuming 4-way set-associative DRAM cache

9 Idea 1: Efficient TLB Coherence! Software Hardware Track DRAM cache contents using page tables and TLBs Maintain latest mapping for recently remapped pages in the Tag Buffer Page Table Entry PPN Mapping Page Table Core! TLB Entry VPN PPN d! (1 bit)! Translation Lookaside Buffer (TLB) SRAM Hierarchy! Way! (2 bits)! Mapping Tag Buffer PPN! V! Mapping! Memory Controller In-Package! Off-Package! 9 * Assuming 4-way set-associative DRAM cache

10 Idea 1: Efficient TLB Coherence! Software Hardware Track DRAM cache contents using page tables and TLBs Maintain latest mapping for recently remapped pages in the Tag Buffer Enforce TLB coherence lazily when the Tag Buffer is full to amortize the cost Page Table Entry PPN Mapping Page Table Reverse Mapping (Find all PTEs that map to a given PPN)! Core! TLB Entry VPN PPN d! (1 bit)! Translation Lookaside Buffer (TLB) SRAM Hierarchy! Tag Buffer Way! (2 bits)! Mapping PPN! V! Mapping! Memory Controller In-Package! Off-Package! 10 * Assuming 4-way set-associative DRAM cache

11 Idea 2: Bandwidth-Aware Replacement! DRAM cache replacement incurs significant DRAM traffic - replacement traffic - Metadata traffic (e.g., frequency counter lookups/updates) Memory Controller! Misses (64 B) Hits (64 B) In-! Package! Off- Package! 11

12 Idea 2: Bandwidth-Aware Replacement! DRAM cache replacement incurs significant DRAM traffic - replacement traffic - Metadata traffic (e.g., frequency counter lookups/updates) Memory Controller! Misses (64 B) Hits (64 B) In-! Package! Off- Package! Replacements (4096 B) 12

13 Idea 2: Bandwidth-Aware Replacement! DRAM cache replacement incurs significant DRAM traffic - replacement traffic - Metadata traffic (e.g., frequency counter lookups/updates) Memory Controller! Misses (64 B) Hits (64 B) In-! Package! Off- Package! Frequency Counter Accesses Replacements (4096 B) 13

14 Idea 2: Bandwidth-Aware Replacement! DRAM cache replacement incurs significant DRAM traffic Limit cache replacement rate - Replace only when the incoming page s frequency counter is greater than the victim pages s counter by a threshold Memory Controller! Hits (64 B) In-! Package! Frequency Counter Accesses Misses (64 B) Limited Replacements Off- Package! 14

15 Idea 2: Bandwidth-Aware Replacement! DRAM cache replacement incurs significant DRAM traffic Limit cache replacement rate Misses (64 B) Reduce metadata traffic - Access frequency counters for a randomly sampled fraction Memory Controller! Hits (64 B) In-! Package! Off- Package! of memory accesses Sampled Frequency Counter Accesses Limited Replacements 15

16 Banshee Extensions! Supporting large pages (e.g., 2MB) - A large page is cached either in its entirety or not at all Supporting multi-socket processors - Coherent DRAM caches - Partitioned DRAM caches 16

17 Performance Evaluation! ZSim simulator [1] 16 cores (4-issue, out-of-order, 2.7 GHz) In-package DRAM (1 GB, 84 GB/s) Off-package DRAM (21 GB/s) Tag Buffer - One Tag Buffer per memory controller (MC) entries, 5 KB in size [1] Sanchez, Daniel, and Christos Kozyrakis. "ZSim: fast and accurate microarchitectural simulation of thousand-core systems." ISCA,

18 Speedup (Normalized to off-package DRAM only)! Perfect In-package DRAM Normalized Speedup % within perfect DRAM cache 15% improvement Unison Alloy BEAR TaglessBanshee Only Banshee improves performance by 15% on average over the best-previous (i.e., BEAR) latency-optimized DRAM cache design 18

19 DRAM Bandwidth Efficiency! Bytes per Instruction Hit Metadata Replacement Unison Alloy 36% in-package DRAM traffic reduction BEAR Tagless Banshee In-Package DRAM Traffic Breakdown Banshee reduces 36% in-package DRAM traffic over the best-previous design 19

20 DRAM Bandwidth Efficiency! Hit Metadata Replacement Bytes per Instruction % in-package DRAM traffic reduction Bytes per Instruction % off-package DRAM traffic reduction 0 0 Unison Alloy BEAR Tagless Banshee In-Package DRAM Traffic Unison Alloy BEAR TaglessBanshee Off-Package DRAM Traffic Breakdown Banshee reduces 36% in-package DRAM traffic over the best-previous design Banshee reduces 3% off-package DRAM traffic over the best-previous design 20

21 Effect of Replacement Traffic Reduction! Normalized Speedup Limiting Replacement Rate Sampling Frequency Counters 0 Banshee LRU Banshee FBR (No Sample) Banshee Limiting replacement rate and sampling frequency counters are both important for bandwidth efficiency in Banshee 21

22 More Analysis in the Paper! Performance with large (2 MB) pages Balancing in- and off-package DRAM bandwidth Overhead for page table update and TLB coherence Storing tags in SRAM Sweep DRAM cache latency and bandwidth Sampling coefficient DRAM cache associativity 22

23 Summary! Need to optimize for bandwidth efficiency to fully exploit the performance of in-package DRAM Idea 1: Improving page-table-based DRAM cache designs with efficient Translation Lookaside Buffer (TLB) coherence Idea 2: Bandwidth-aware frequency-based replacement (FBR) policy Banshee improves performance by 15% and reduces in-package DRAM traffic by 36% over the best-previous latency-optimized DRAM cache design 23

24 Banshee: Bandwidth-Efficient DRAM Caching via Software/Hardware Cooperation! Xiangyao Yu 1, Christopher Hughes 2, Nadathur Satish 2, Onur Mutlu 3, Srinivas Devadas 1 1 MIT 2 Intel Labs 3 ETH Zürich 24

25 Backup Slides! 25

26 Summary of Operational Characteristics of Different State-of-the-Art DRAM Designs! 26

27 Tag Buffer Organization! 27

28 DRAM Layout! In-Package Data Row Layout 4KB Page! 4KB Page! 4KB Page! 4KB Page! 4KB Page! 4KB Page! 4-way set-associative Metadata Row Layout Row Buffer! Metadata for One Set 4 cached pages + 5 candidate pages 28 Metadata for One Page Tag Freq Cntr V D

29 Speedup Normalized to No! 29

30 In-Package DRAM Traffic Breakdown! 30

31 Off-Package DRAM Traffic! 31

32 Sensitivity to Page Table Update Cost! 32

33 Sensitivity to DRAM Latency and Bandwidth! 33

Banshee: Bandwidth-Efficient DRAM Caching via Software/Hardware Cooperation

Banshee: Bandwidth-Efficient DRAM Caching via Software/Hardware Cooperation Xiangyao Yu 1 Christopher J. Hughes 2 Nadathur Satish 2 Onur Mutlu 3 Srinivas Devadas 1 1 MIT 2 Intel Labs 3 ETH Zürich ABSTRACT