CS 153 Design of Operating Systems

Transcription

1 CS 53 Design of Operating Systems Spring 8 Lectre 9: Locality and Cache Instrctor: Chengy Song Slide contribtions from Nael Ab-Ghazaleh, Harsha Madhyvasta and Zhiyn Qian Some slides modified from originals by Dave O hallaron

2 Recap: Memory Hierarchy L: Registers CPU registers hold words retrieved from L cache Smaller, faster, costlier per byte Larger, slower, cheaper per byte L4: L3: L2: L: L cache (SRAM) L2 cache (SRAM) Main memory (DRAM) Local secondary storage (local disks) L cache holds cache lines retrieved from L2 cache L2 cache holds cache lines retrieved from main memory Main memory holds disk blocks retrieved from local disks Local disks hold files retrieved from disks on remote network servers L5: Remote secondary storage (tapes, distribted file systems, Web servers) CS 53 Lectre 9 Locality and Cache 2

3 Locality Principle of Locality: Programs tend to se data and instrctions with addresses near or eqal to those they have sed recently Temporal locality: Recently referenced items are likely to be referenced again in the near ftre Spatial locality: Items with nearby addresses tend to be referenced close together in time CS 53 Lectre 9 Locality and Cache 3

4 Locality Example sm = ; for (i = ; i < n; i++) sm += a[i]; retrn sm; Data references Reference array elements in sccession (stride- reference pattern). Reference variable sm each iteration. Instrction references Reference instrctions in seqence. Cycle throgh loop repeatedly. Spatial locality Temporal locality Spatial locality Temporal locality CS 53 Lectre 9 Locality and Cache 4

5 Qalitative Estimates of Locality Claim: Being able to look at code and get a qalitative sense of its locality is a key skill for a professional programmer. Qestion: Does this fnction have good locality with respect to array a? int sm_array_rows(int a[m][n]) { int i, j, sm = ; for (i = ; i < M; i++) for (j = ; j < N; j++) sm += a[i][j]; retrn sm; } CS 53 Lectre 9 Locality and Cache 5

6 Locality Example Qestion: Does this fnction have good locality with respect to array a? int sm_array_cols(int a[m][n]) { int i, j, sm = ; } for (j = ; j < N; j++) for (i = ; i < M; i++) sm += a[i][j]; retrn sm; CS 53 Lectre 9 Locality and Cache 6

7 Locality Example Qestion: Can yo permte the loops so that the fnction scans the 3-d array a with a stride- reference pattern (and ths has good spatial locality)? int sm_array_3d(int a[m][n][n]) { int i, j, k, sm = ; } for (i = ; i < M; i++) for (j = ; j < N; j++) for (k = ; k < N; k++) sm += a[k][i][j]; retrn sm; CS 53 Lectre 9 Locality and Cache 7

8 Memory hierarchy Cache: A smaller, faster storage device that acts as a staging area for a sbset of the data in a larger, slower device. Fndamental idea of a memory hierarchy: For each layer, faster, smaller device caches larger, slower device Why do memory hierarchies work? Becase of locality!» Hit fast memory mch more freqently even thogh its smaller Ths, the storage at level k+ can be slower (bt larger and cheaper!) Big Idea: The memory hierarchy creates a large pool of storage that costs as mch as the cheap storage near the bottom, bt that serves data to programs at the rate of the fast storage near the top. CS 53 Lectre 9 Locality and Cache 8

9 General Cache Concepts Cache Smaller, faster, more expensive memory caches a sbset of the blocks 4 Data is copied in block-sized transfer nits Memory Larger, slower, cheaper memory viewed as partitioned into blocks CS 53 Lectre 9 Locality and Cache 9

10 General Cache Concepts: Hit Cache Reqest: Data in block b is needed Block b is in cache: Hit! Memory CS 53 Lectre 9 Locality and Cache

11 General Cache Concepts: Miss Cache Reqest: Data in block b is needed Block b is not in cache: Miss! 2 Reqest: 2 Block b is fetched from memory Memory Block b is stored in cache Placement policy: determines where b goes Replacement policy: determines which block gets evicted (victim) CS 53 Lectre 9 Locality and Cache

12 General Caching Concepts: Types of Cache Misses Cold (complsory) miss Cold misses occr becase the cache is empty. Conflict miss Most caches limit blocks at level k+ to a small sbset (sometimes a singleton) of the block positions at level k.» E.g. Block i at level k+ mst be placed in block (i mod 4) at level k. Conflict misses occr when the level k cache is large enogh, bt mltiple data objects all map to the same level k block.» E.g. Referencing blocks, 8,, 8,, 8,... wold miss every time. Capacity miss Occrs when the set of active cache blocks (working set) is larger than the cache. CS 53 Lectre 9 Locality and Cache 2

13 Cache replacement policy Cache replacement policy: determine which data to remove when we need a victim Does it matter? Yes! Cache filling is expensive» Especially when closer to the bottom of the hierarchy Getting the nmber down, can improve the performance of the system significantly CS 53 Lectre 9 Locality and Cache 3

14 Considerations Cache replacement spport has to be simple They happen all the time, we cannot make that part slow Bt it can be complicated/expensive when a miss occrs why? Reason : if we are sccessfl, this will be rare Reason 2: when it happens we are paying the cost of loading» Loading from lower layer is very relatively slower: can afford to do some extra comptation» Worth it if we can save some ftre miss What makes a good cache replacement policy? CS 53 Lectre 9 Locality and Cache 4

15 Locality to the Resce Recall that cache works becase of locality Temporal and spatial Work at different scales: for cache, at a line level, for VM, at page level, and even at larger scales All caching schemes depend on locality What happens if a program does not have locality? High cost of cache miss is acceptable, if infreqent Processes sally reference data in localized patterns, making caching practical CS 53 Lectre 9 Locality and Cache 5

16 Evicting the Best Data Goal is to redce the cache miss rate The best data to evict is the one never toched again Will never have a cache miss on it Never is a long time, so picking the data closest to never is the next best thing Evicting the data that won t be sed for the longest period of time minimizes the nmber of cache misses Proved by Belady We ll srvey varios replacement algorithms in next lectre CS 53 Lectre 9 Locality and Cache 6

17 Simple Memory System Example Addressing 4-bit virtal addresses 2-bit physical address Page size = 64 bytes VPN Virtal Page Nmber VPO Virtal Page Offset PPN Physical Page Nmber PPO Physical Page Offset 7

18 Simple Memory System Page Table Only show first 6 entries (ot of 256) VPN PPN Valid VPN PPN Valid A B 4 C 5 6 D 2D 6 E 7 F D CS 53 Lectre 9 Locality and Cache 8

19 Simple Memory System TLB 6 entries 4-way associative TLBT TLBI VPN VPO Set Tag PPN Valid Tag PPN Valid Tag PPN Valid Tag PPN Valid 3 9 D D 2 4 A D A

20 Simple Memory System Cache 6 lines, 4-byte block size Physically addressed Direct mapped CT CI CO PPN PPO Idx Tag Valid B B B2 B3 Idx Tag Valid B B B2 B A D 2 B A 2D 93 5 DA 3B 3 36 B B D 8F 9 C 2 5 D F D D C2 DF 3 E F B 2 D3

21 Address Translation Example # Virtal Address: x3d4 TLBT TLBI VPN VPO xf x3 x3 Y N xd VPN TLBI TLBT TLB Hit? Page Falt? PPN: Physical Address CT CI CO PPN PPO x5 xd Y x36 CO CI CT Hit? Byte: CS 53 Lectre 9 Locality and Cache 2

22 Address Translation Example #2 Virtal Address: xb8f TLBT TLBI VPN VPO x2e 2 xb N Y TBD VPN TLBI TLBT TLB Hit? Page Falt? PPN: Physical Address CT CI CO PPN PPO CO CI CT Hit? Byte: CS 53 Lectre 9 Locality and Cache 22

23 Address Translation Example #3 Virtal Address: x2 TLBT TLBI VPN VPO x x N N x28 VPN TLBI TLBT TLB Hit? Page Falt? PPN: Physical Address CT CI CO PPN PPO x8 x28 N Mem CO CI CT Hit? Byte: CS 53 Lectre 9 Locality and Cache 23

24 Smmary so far Basic concepts of cache Cache hit Cache miss Well-written programs exhibit a property called locality, this is the fondation of caching Cache replacement policy is important CS 53 Lectre 9 Locality and Cache 24

25 Next time Virtal memory Preparation Read Modle 2 & 22 CS 53 Lectre 9 Locality and Cache 25