Caches. Hakim Weatherspoon CS 3410, Spring 2012 Computer Science Cornell University. See P&H 5.1, 5.2 (except writes)

Size: px

Start display at page:

Download "Caches. Hakim Weatherspoon CS 3410, Spring 2012 Computer Science Cornell University. See P&H 5.1, 5.2 (except writes)"

Cornelius McDowell
6 years ago
Views:

1 Caches akim Weatherspoon CS 341, Spring 212 Computer Science Cornell University See P& 5.1, 5.2 (except writes)

2 ctrl ctrl ctrl inst imm B A B D D Big Picture: emory emory: big & slow vs Caches: small & fast compute jump/branch targets memory register file alu PC +4 new pc Instruction Fetch control extend detect hazard Instruction Decode forward unit Execute d in addr d out memory emory Write- Back IF/ID ID/EX EX/E E/WB 2

3 Goals for Today: caches Examples of caches: Direct apped Fully Associative N-way set associative Performance and comparison it ratio (conversly, miss ratio) Average memory access time (AAT) Cache size 3

4 Cache Performance Average emory Access Time (AAT) Cache Performance (very simplified): L1 (SRA): 512 x 64 byte cache lines, direct mapped Data cost: 3 cycle per word access Lookup cost: 2 cycle em (DRA): 4GB Data cost: 5 cycle per word, plus 3 cycle per consecutive word Performance depends on: Access time for hit, miss penalty, hit rate 4

5 isses Cache misses: classification The line is being referenced for the first time Cold (aka Compulsory) iss The line was in the cache, but has been evicted 5

6 Avoiding isses Q: ow to avoid Cold isses Unavoidable? The data was never in the cache Prefetching! Other isses Buy more SRA Use a more flexible cache design 6

7 Bigger cache doesn t always help em access trace:, 16, 1, 17, 2, 18, 3, 19, 4, it rate with four direct-mapped 2-byte cache lines? With eight 2-byte cache lines? With four 4-byte cache lines?

8 isses Cache misses: classification The line is being referenced for the first time Cold (aka Compulsory) iss The line was in the cache, but has been evicted because some other access with the same index Conflict iss because the cache is too small i.e. the working set of program is larger than the cache Capacity iss 8

9 Avoiding isses Q: ow to avoid Cold isses Unavoidable? The data was never in the cache Prefetching! Capacity isses Buy more SRA Conflict isses Use a more flexible cache design 9

10 Three common designs A given data block can be placed in any cache line Fully Associative in exactly one cache line Direct apped in a small set of cache lines Set Associative 1

11 Comparison: Direct apped Using byte addresses in this example! Addr Bus = 5 bits Processor LB $1 [ 1 ] LB $3 [ 1 ] LB $3 [ 4 ] LB $2 [ ] Cache 4 cache lines 2 word block 2 bit tag field 2 bit index field 1 bit block offset field tag data 1 2 isses: its: emory

12 Comparison: Direct apped Using byte addresses in this example! Addr Bus = 5 bits Processor LB $1 [ 1 ] LB $3 [ 1 ] LB $3 [ 4 ] LB $2 [ ] Cache 4 cache lines 2 word block 2 bit tag field 2 bit index field 1 bit block offset field tag data isses: 8 its: emory

13 Comparison: Fully Associative Using byte addresses in this example! Addr Bus = 5 bits Processor LB $1 [ 1 ] LB $3 [ 1 ] LB $3 [ 4 ] LB $2 [ ] Cache 4 cache lines 2 word block 4 bit tag field 1 bit block offset field isses: its: tag data emory

14 Comparison: Fully Associative Using byte addresses in this example! Addr Bus = 5 bits Processor LB $1 [ 1 ] LB $3 [ 1 ] LB $3 [ 4 ] LB $2 [ ] Cache 4 cache lines 2 word block 4 bit tag field 1 bit block offset field tag isses: 3 its: 8 data emory

15 Comparison: 2 Way Set Assoc Using byte addresses in this example! Addr Bus = 5 bits Processor LB $1 [ 1 ] LB $3 [ 1 ] LB $3 [ 4 ] LB $2 [ ] Cache tag data 2 sets 2 word block 3 bit tag field 1 bit set index field 1 bit block offset field isses: its: emory

16 Comparison: 2 Way Set Assoc Using byte addresses in this example! Addr Bus = 5 bits Processor LB $1 [ 1 ] LB $3 [ 1 ] LB $3 [ 4 ] LB $2 [ ] Cache tag data 2 sets 2 word block 3 bit tag field 1 bit set index field 1 bit block offset field isses: 4 its: emory

17 Cache Size 17

18 Direct apped Cache (Reading) Tag Index Offset V Tag Block = hit? word select data 32bits 18

19 Direct apped Cache Size Tag Index Offset n bit index, m bit offset Q: ow big is cache (data only)? Q: ow much SRA needed (data + overhead)? 19

20 Direct apped Cache Size Tag Index Offset n bit index, m bit offset Q: ow big is cache (data only)? Q: ow much SRA needed (data + overhead)? Cache of size 2 n blocks Block size of 2 m bytes Tag field: 32 (n + m) Valid bit: 1 Bits in cache: 2 n x (block size + tag size + valid bit size) = 2 n (2 m bytes x 8 bits-per-byte + (32-n-m) + 1) 2

21 Fully Associative Cache (Reading) Tag Offset V Tag Block = = = = line select 64bytes word select hit? data 32bits 21

22 Fully Associative Cache Size Tag Offset m bit offset, 2 n cache lines Q: ow big is cache (data only)? Q: ow much SRA needed (data + overhead)? 22

23 Fully Associative Cache Size Tag Offset m bit offset, 2 n cache lines Q: ow big is cache (data only)? Q: ow much SRA needed (data + overhead)? Cache of size 2 n blocks Block size of 2 m bytes Tag field: 32 m Valid bit: 1 Bits in cache: 2 n x (block size + tag size + valid bit size) = 2 n (2 m bytes x 8 bits-per-byte + (32-m) + 1) 23

24 Fully-associative reduces conflict misses... assuming good eviction strategy em access trace:, 16, 1, 17, 2, 18, 3, 19, 4, 2, it rate with four fully-associative 2-byte cache lines?

25 but large block size can still reduce hit rate vector add trace:, 1, 2, 1, 11, 21, 2, 22, it rate with four fully-associative 2-byte cache lines? With two fully-associative 4-byte cache lines? 25

26 isses Cache misses: classification Cold (aka Compulsory) The line is being referenced for the first time Capacity The line was evicted because the cache was too small i.e. the working set of program is larger than the cache Conflict The line was evicted because of another access whose index conflicted 26

27 Direct apped + Smaller + Less + Less + Faster + Less + Very Lots Low Common Cache Tradeoffs Tag Size SRA Overhead Controller Logic Speed Price Scalability # of conflict misses it rate Pathological Cases? Fully Associative Larger ore ore Slower ore Not Very Zero + igh +? 27

28 Administrivia Prelim2 today, Thursday, arch 29 th at 7:3pm Location is Phillips 11 and prelim2 starts at 7:3pm Project2 due next onday, April 2 nd 28

29 Summary Caching assumptions small working set: 9/1 rule can predict future: spatial & temporal locality Benefits big & fast memory built from (big & slow) + (small & fast) Tradeoffs: associativity, line size, hit cost, miss penalty, hit rate Fully Associative higher hit cost, higher hit rate Larger block size lower hit cost, higher miss penalty Next up: other designs; writing to caches 29

Caches. Hakim Weatherspoon CS 3410, Spring 2012 Computer Science Cornell University. See P&H 5.1, 5.2 (except writes)

Caches. Hakim Weatherspoon CS 3410, Spring 2012 Computer Science Cornell University. See P&H 5.1, 5.2 (except writes) s akim Weatherspoon CS, Spring Computer Science Cornell University See P&.,. (except writes) Big Picture: : big & slow vs s: small & fast compute jump/branch targets memory PC + new pc Instruction Fetch