ECE232: Hardware Organization and Design

Transcription

1 ECE232: Hardware Organization and Design Lecture 23: Associative Caches Adapted from Computer Organization and Design, Patterson & Hennessy, UCB

2 Overview Last time: Direct mapped cache Pretty simple to understand Every memory block goes in only one place in the cache Somewhat limiting May cause a lot of the cache to be unused Idea! Why not be more flexible: data can go into more than one place Associative caches ECE232: Associative Caches 2

3 Cache addressing Memory How do you know if something is in the cache? (Q1) If it is in the cache, how to find it? (Q2) CPU To Processor From Processor Cache Block X Block X Block Y Traditional Memory Given an address, provide the data (has address decoder) Associative Memory AKA Content Addressable Memory Each line contain the address (or part of it) and the data Full/MSBs of Address Tag Data ECE232: Associative Caches 3

4 LSBs of Address Cache Organization Fully-associative: any memory location can be stored anywhere in the cache Cache location and memory address are unrelated Direct-mapped: each memory location maps onto exactly one cache entry Some of the memory address bit are used to index the cache MSBs of Address Tag Data N-way set-associative: each memory location can go into one of N sets ECE232: Associative Caches 4

5 Direct mapped cache (assume 1 byte/block) Cache Block 0 can be occupied by data from Memory blocks 0, 4, 8, 12 Cache Block 1 can be occupied by data from Memory blocks 1, 5, 9, 13 Cache Block 2 can be occupied by data from Memory blocks 2, 6, 10, 14 Cache Block 3 can be occupied by data from Memory blocks 3, 7, 11, Block Index Memory Cache Index 4-Block Direct Mapped Cache ECE232: Associative Caches 5

6 Fully Associative Cache Block Index Memory Memory block address tag offset 1 word tag ECE232: Associative Caches 6

7 address data Fully Associative Cache: Block=1 Byte CPU Main Memory Tag Data ECE232: Associative Caches 7

8 Two-way Set Associative Cache Two direct-mapped caches operate in parallel Cache Index selects a set from the cache (set includes 2 blocks) The two tags in the set are compared in parallel Data is selected based on the tag result Valid Cache Tag Cache Data Cache Block 0 : : : Cache Index Cache Data Cache Block 0 : Cache Tag Valid : : Tag Compare Sel1 1 Mux 0 Sel0 Compare Tag Set Hit OR Cache Block ECE232: Associative Caches 8

9 4-way Set Associative Cache Allow block anywhere in a set Advantages: Better hit rate Address Disadvantage: Index V Tag Data V Tag Data V Tag Data V Tag Data More tag bits More hardware Higher access time to-1 multiplexor Hit Data A Four-Way Set-Associative Cache, Block size = 4 bytes ECE232: Associative Caches 9

10 Set Associative Cache - addressing TAG INDEX/Set # OFFSET Tag to check if have correct block anywhere in set Index to select a set in cache Byte offset ECE232: Associative Caches 10

11 Associative Caches Fully associative Allow a given block to go in any cache entry Requires all entries to be searched at once Comparator per entry (expensive) n-way set associative Each set contains n entries Block number determines which set (Block number) modulo (#Sets in cache) Search all entries in a given set at once n comparators (less expensive) ECE232: Associative Caches 11

12 Spectrum of Associativity For a cache with 8 entries ECE232: Associative Caches 12

13 How Much Associativity Increased associativity decreases miss rate But with diminishing returns Simulation of a system with 64KB D-cache, 16-word blocks, SPEC way: 10.3% 2-way: 8.6% 4-way: 8.3% 8-way: 8.1% ECE232: Associative Caches 13

14 Set Associative Cache Organization ECE232: Associative Caches 14

15 Types of Cache Misses (for 3 organizations) Compulsory (cold start): location has never been accessed - first access to a block not in the cache Capacity: since the cache cannot contain all the blocks of a program, some blocks will be replaced and later retrieved Conflict: when too many blocks try to load into the same set, some blocks will be replaced and later retrieved ECE232: Associative Caches 15

16 Cache Design Decisions For a given cache size Block (Line) size Number of Blocks (Lines) How is the cache organized Write policy Replacement Strategy Increase cache size More Blocks (Lines) More lines == Higher hit rate Slower Memory As many as practical ECE232: Associative Caches 16

17 Summary Today: Associative caches Provide more choices for block storage More expensive in terms of hardware Require comparators for tags Many caches are set associative Remember: Direct mapped = 1 way set associative Full associative = N way set associate (N is total blocks in cache) ECE232: Associative Caches 17