CS 61C: Great Ideas in Computer Architecture (Machine Structures) Caches Part 2

Transcription

1 3//5 CS 6C: Great Ideas in Computer Architecture (Machine Structures) Caches Part Instructors: Krste Asanovic & Vladimir Stojanovic hfp://insteecsberkeleyedu/~cs6c/ Parallel Requests Assigned to computer eg, Search Katz Parallel Threads Assigned to core eg, Lookup, Ads So$ware Parallel InstrucXons > one Xme eg, 5 pipelined instrucxons Parallel > data one Xme eg, Add of 4 pairs of words Hardware descripxons All one Xme Programming Languages You Are Here! Harness Parallelism & Achieve High Performance Hardware Warehouse Scale Computer Core Memory Input/Output InstrucXon Unit(s) Main Memory Computer (Cache) Core Core FuncXonal Unit(s) A +B A +B A +B A 3 +B 3 Smart Phone Today s Lecture Logic Gates Caches Review Principle of Locality Temporal Locality and SpaXal Locality Hierarchy of Memories (speed/size/cost per bit) to Exploit Locality Cache copy of data in lower level of memory hierarchy Direct Mapped to find block in cache using field and Valid bit for Hit Cache design choice: Write- Through vs Write- Back 3 Review: Adding Cache to Computer Processor Control path PC Registers ArithmeXc & Logic Unit (ALU) Enable? Read/Write Cache Address Write Read Processor- Memory Interface Memory Program Bytes Input Output I/O- Memory Interfaces 4 Processor Address Fields used by Cache Controller Block Offset: Byte address within block Set : Selects which set : Remaining porxon of processor address Processor Address (3- bits total) Set Block offset Size of = log (number of sets) Size of = Address size Size of log (number of bytes/block) 5 Review: Direct- Mapped Cache One word blocks, cache size = K words (or 4KB) Hit Valid bit ensures something useful in cache for this index Compare with upper part of Address to see if a Hit Valid Block offset 3 Comparator Read data from cache instead of memory if a Hit 6

2 3//5 Cache Terms Hit rate: fracxon of accesses that hit in the cache Miss rate: Hit rate Miss penalty: Xme to replace a block from lower level in memory hierarchy to cache Hit Xme: Xme to access cache memory (including tag comparison) Average Memory Access Time (AMAT) Average Memory Access Time (AMAT) is the average to access memory considering both hits and misses in the cache AMAT = Time for a hit + Miss rate x Miss penalty AbbreviaXon: $ = cache (A Berkeley innovaxon!) 7 8 Clickers/Peer instrucxon AMAT = Time for a hit + Miss rate x Miss penalty Given a psec clock, a miss penalty of 5 clock cycles, a miss rate of misses per instrucxon and a cache hit Xme of clock cycle, what is AMAT? A: psec Example: Direct- Mapped $ with 4 Single- Word Lines, Worst- Case Reference String Consider the main memory address reference string Start with an empty cache - all blocks B: 4 psec C: 6 psec! Example: Direct- Mapped $ with 4 Single- Word Lines, Worst- Case Reference String Consider the main memory address reference string Start with an empty cache - all blocks miss 4 miss miss 4 miss 4 4 Mem() Mem() Mem(4) Mem() AlternaXve Block Placement Schemes miss 4 miss 4 miss 4 miss 4 Mem(4) Mem() Mem(4) Mem() 8 requests, 8 misses Ping- pong effect due to conflict misses - two memory locaxons that map into the same cache block DM placement: mem block in 8 block cache: only one cache block where mem block can be found ( modulo 8) = 4 SA placement: four sets x - ways (8 cache blocks), memory block in set ( mod 4) = ; either element of the set FA placement: mem block can appear in any cache blocks

3 3//5 Example: - Way Set AssociaXve $ Cache Way Set V (4 words = sets x ways per set) Q: Is it there? Compare all the cache tags in the set to the high order 3 memory address bits to tell if the memory block is in the cache xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx Main Memory One word blocks Two low order bits define the byte in the word (3b words) Q: How do we find it? Use next low order memory address bit to determine which cache set (ie, modulo the number of sets in the cache) 3 Example: 4 Word - Way SA $ Same Reference String Consider the main memory word reference string Start with an empty cache - all blocks Example: 4- Word - Way SA $ Same Reference String Consider the main memory address reference string Start with an empty cache - all blocks Different OrganizaXons of an Eight- Block Cache miss 4 miss hit 4 hit Mem() Mem() Mem() Mem() 8 requests, misses Mem(4) Mem(4) Mem(4) Solves the ping- pong effect in a direct- mapped cache due to conflict misses since now two memory locaxons that map into the same cache set can co- exist! Total size of $ in blocks is equal to number of sets x associabvity For fixed $ size, increasing associaxvity decreases number of sets while increasing number of elements per set With eight blocks, an 8- way set- associaxve $ is same as a fully associaxve $ 5 6 Four- Way Set- AssociaXve Cache 8 = 56 sets each with four ways (each with one block) Byte offset V V V V Way Way Way Way Range of Set- AssociaXve Caches For a fixed- size cache, each increase by a factor of two in associaxvity doubles the number of blocks per set (ie, the number or ways) and halves the number of sets decreases the size of the index by bit and increases the size of the tag by bit Word offset Byte offset 3 4x select Hit 7 8 3

4 3//5 Range of Set- AssociaXve Caches For a fixed- size cache, each increase by a factor of two in associaxvity doubles the number of blocks per set (ie, the number or ways) and halves the number of sets decreases the size of the index by bit and increases the size of the tag by bit Used for tag compare Decreasing associaxvity Direct mapped (only one way) Smaller tags, only a single comparator Selects the set Increasing associaxvity Selects the word in the block Word offset Byte offset Fully associaxve (only one set) is all the bits except block and byte offset Costs of Set- AssociaXve Caches N- way set- associaxve cache costs N comparators (delay and area) MUX delay (set selecxon) before data is available available azer set selecxon (and Hit/Miss decision) DM $: block is available before the Hit/Miss decision In Set- AssociaXve, not possible to just assume a hit and conxnue and recover later if it was a miss When miss occurs, which way s block selected for replacement? Least Recently Used (LRU): one that has been unused the longest (principle of temporal locality) Must track when each way s block was used relaxve to other blocks in the set For - way SA $, one bit per set set to when a block is referenced; reset the other way s bit (ie, last used ) 9 Cache Replacement Policies Random Replacement Hardware randomly selects a cache evict Least- Recently Used Hardware keeps track of access history Replace the entry that has not been used for the longest Xme For - way set- associaxve cache, need one bit for LRU replacement Example of a Simple Pseudo LRU ImplementaXon Assume 64 Fully AssociaXve entries Hardware replacement pointer points to one cache entry Whenever access is made to the entry the pointer points to: Move the pointer to the next entry Otherwise: do not move the pointer (example of not- most- recently used replacement policy) Benefits of Set- AssociaXve Caches Choice of DM $ or SA $ depends on the cost of a miss versus the cost of implementaxon Replacement Pointer Entry Entry : Entry 63 Largest gains are in going from direct mapped to - way (%+ reducxon in miss rate) Administrivia Project - due Sunday March 5 th, :59PM Use pinned Piazza threads! We ll penalize those who ask, but don t search! Guerilla secxons starxng this weekend OpXonal secxons, focus on lecture/exam material, not projects Vote for Xme slot on Piazza poll Understanding Cache Misses: The 3Cs Compulsory (cold start or process migraxon, st reference): First access to block impossible to avoid; small effect for long running programs SoluXon: increase block size (increases miss penalty; very large blocks could increase miss rate) Capacity: Cache cannot contain all blocks accessed by the program SoluXon: increase cache size (may increase access Xme) Conflict (collision): MulBple memory locabons mapped to the same cache locabon SoluBon : increase cache size SoluBon : increase associabvity (may increase access Bme) 3 4 4

5 3//5 How to Calculate 3C s using Cache Simulator Compulsory: set cache size to infinity and fully associaxve, and count number of misses Capacity: Change cache size from infinity, usually in powers of, and count misses for each reducxon in size 6 MB, 8 MB, 4 MB, 8 KB, 64 KB, 6 KB 3 Conflict: Change from fully associaxve to n- way set associaxve while counxng misses Fully associaxve, 6- way, 8- way, 4- way, - way, - way 5 3Cs Analysis Three sources of misses (SPEC integer and floaxng- point benchmarks) Compulsory misses 6%; not visible Capacity misses, funcxon of cache size Conflict porxon depends on associaxvity and cache size 6 Improving Cache Performance AMAT = Time for a hit + Miss rate x Miss penalty Reduce the Xme to hit in the cache Eg, Smaller cache Reduce the miss rate Eg, Bigger cache Reduce the miss penalty Eg, Use mulxple cache levels Impact of Larger Cache on AMAT? ) Reduces misses (what kind(s)?) ) Longer Access Xme (Hit Xme): smaller is faster Increase in hit Xme will likely add another stage to the pipeline At some point, increase in hit Xme for a larger cache may overcome the improvement in hit rate, yielding a decrease in performance Computer architects expend considerable effort opxmizing organizaxon of cache hierarchy big impact on performance and power! 7 8 Clickers: Impact of longer cache blocks on misses? For fixed total cache capacity and associaxvity, what is effect of longer blocks on each type of miss: A: Decrease, B: Unchanged, C: Increase Compulsory? Capacity? Conflict? Clickers: Impact of longer blocks on AMAT For fixed total cache capacity and associaxvity, what is effect of longer blocks on each component of AMAT: A: Decrease, B: Unchanged, C: Increase Hit Time? Miss Rate? Miss Penalty? 9 3 5

6 3//5 Clickers/Peer InstrucXon: For fixed capacity and fixed block size, how does increasing associaxvity effect AMAT? 3 Cache Design Space Several interacxng dimensions Cache size Block size AssociaXvity Replacement policy Write- through vs write- back Write allocaxon OpXmal choice is a compromise Depends on access characterisxcs Workload Use (I- cache, D- cache) Depends on technology / cost Simplicity ozen wins Bad Good Cache Size Factor A Less AssociaDvity Block Size Factor B More 3 And, In Conclusion Name of the Game: Reduce AMAT Reduce Hit Time Reduce Miss Rate Reduce Miss Penalty Balance cache parameters (Capacity, associaxvity, block size) 33 6