Caches I. CSE 351 Spring Instructor: Ruth Anderson

Transcription

1 L16: Cches I Cches I CSE 351 Spring 2017 Instructor: Ruth Anderson Teching Assistnts: Dyln Johnson Kevin Bi Linxing Preston Jing Cody Ohlsen Yufng Sun Joshu Curtis

2 L16: Cches I Administrivi Homework 3, due next Fridy (5/5) Midterm, Mondy (5/8) Lb 3, due Thursdy (5/11) 2

3 L16: Cches I Rodmp C: cr *c = mlloc(sizeof(cr)); c->miles = 100; c->gls = 17; flot mpg = get_mpg(c); free(c); Assembly lnguge: Mchine code: get_mpg: pushq movq... popq ret %rbp %rsp, %rbp %rbp Jv: Cr c = new Cr(); c.setmiles(100); c.setgls(17); flot mpg = c.getmpg(); OS: Memory & dt Integers & flots x86 ssembly Procedures & stcks Executbles Arrys & structs Memory & cches Processes Virtul memory Memory lloction Jv vs. C Computer system: 3

4 L16: Cches I How does execution time grow with SIZE? int rry[size]; int sum = 0; for (int i = 0; i < ; i++) { for (int j = 0; j < SIZE; j++) { sum += rry[j]; } } Time Plot SIZE 4

5 Time L16: Cches I Actul Dt SIZE 5

6 L16: Cches I Mking memory ccesses fst! Cche bsics Principle of loclity Memory hierrchies Cche orgniztion Progrm optimiztions tht consider cches 6

7 Performnce L16: Cches I Processor-Memory Gp first Intel CPU with cche on chip 1998 Pentium III hs two cche levels on chip Moore s Lw Processor-Memory Performnce Gp (grows 50%/yer) µproc 55%/yer (2X/1.5yr) DRAM 7%/yer (2X/10yrs) Yer 7

8 L16: Cches I Problem: Processor-Memory Bottleneck Processor performnce doubled bout every 18 months CPU Reg Bus ltency / bndwidth evolved much slower Min Memory Core 2 Duo: Cn process t lest 256 Bytes/cycle Core 2 Duo: Bndwidth 2 Bytes/cycle Ltency cycles (30-60ns) cycle: single mchine step (fixed-time) Problem: lots of witing on memory 8

9 L16: Cches I Problem: Processor-Memory Bottleneck Processor performnce doubled bout every 18 months CPU Reg Cche Bus ltency / bndwidth evolved much slower Min Memory Core 2 Duo: Cn process t lest 256 Bytes/cycle Core 2 Duo: Bndwidth 2 Bytes/cycle Ltency cycles (30-60ns) cycle: single mchine step (fixed-time) Solution: cches 9

10 L16: Cches I Cche Pronuncition: csh We bbrevite this s $ English: A hidden storge spce for provisions, wepons, nd/or tresures Computer: Memory with short ccess time used for the storge of frequently or recently used instructions (i-cche/i$) or dt (d-cche/d$) More generlly: Used to optimize dt trnsfers between ny system elements with different chrcteristics (network interfce cche, I/O cche, etc.) 10

11 L16: Cches I Generl Cche Mechnics Cche Smller, fster, more expensive memory. Cches subset of the blocks (.k.. lines) Dt is copied in block-sized trnsfer units Memory Lrger, slower, cheper memory. Viewed s prtitioned into blocks or lines 11

12 L16: Cches I Generl Cche Concepts: Hit Request: 14 Dt in block b is needed Cche Block b is in cche: Hit! Dt is returned to CPU Memory

13 L16: Cches I Generl Cche Concepts: Miss Cche Request: Dt in block b is needed Block b is not in cche: Miss! 12 Request: 12 Block b is fetched from memory Memory Block b is stored in cche Plcement policy: determines where b goes Replcement policy: determines which block gets evicted (victim) Dt is returned to CPU 13

14 L16: Cches I Why Cches Work Loclity: Progrms tend to use dt nd instructions with ddresses ner or equl to those they hve used recently 14

15 L16: Cches I Why Cches Work Loclity: Progrms tend to use dt nd instructions with ddresses ner or equl to those they hve used recently Temporl loclity: Recently referenced items re likely to be referenced gin in the ner future block 15

16 L16: Cches I Why Cches Work Loclity: Progrms tend to use dt nd instructions with ddresses ner or equl to those they hve used recently Temporl loclity: Recently referenced items re likely to be referenced gin in the ner future Sptil loclity: Items with nerby ddresses tend to be referenced close together in time block block How do cches tke dvntge of this? 16

17 L16: Cches I Exmple: Any Loclity? sum = 0; for (i = 0; i < n; i++) { sum += [i]; } return sum; Dt: Temporl: Sptil: Instructions: Temporl: Sptil: 17

18 L16: Cches I Exmple: Any Loclity? sum = 0; for (i = 0; i < n; i++) { sum += [i]; } return sum; Dt: Temporl: Sptil: Instructions: Temporl: Sptil: sum referenced in ech itertion rry [] ccessed in stride-1 pttern cycle through loop repetedly reference instructions in sequence 18

19 L16: Cches I Loclity Exmple #1 int sum_rry_rows(int [M][N]) { int i, j, sum = 0; for (i = 0; i < M; i++) for (j = 0; j < N; j++) sum += [i][j]; } return sum; 19

20 L16: Cches I Loclity Exmple #1 int sum_rry_rows(int [M][N]) { int i, j, sum = 0; } for (i = 0; i < M; i++) for (j = 0; j < N; j++) sum += [i][j]; return sum; Lyout in Memory [3] [3] Note: 76 is just one possible strting ddress of rry [3] M = 3, N=4 [3] [3] [3] Access Pttern: stride =? 1) 2) 3) 4) [3] 5) 6) 7) 8) [3] 9) 10) 11) 12) [3] 20

21 L16: Cches I Loclity Exmple #2 int sum_rry_cols(int [M][N]) { int i, j, sum = 0; for (j = 0; j < N; j++) for (i = 0; i < M; i++) sum += [i][j]; } return sum; 21

22 L16: Cches I Loclity Exmple #2 int sum_rry_cols(int [M][N]) { int i, j, sum = 0; } for (j = 0; j < N; j++) for (i = 0; i < M; i++) sum += [i][j]; return sum; Lyout in Memory [3] [3] [3] M = 3, N=4 [3] [3] [3] Access Pttern: stride =? 1) 2) 3) 4) 5) 6) 7) 8) 9) 10) [3] 11) [3] 12) [3] 22

23 L16: Cches I Loclity Exmple #3 int sum_rry_3d(int [M][N][L]) { int i, j, k, sum = 0; } for (i = 0; i < N; i++) for (j = 0; j < L; j++) for (k = 0; k < M; k++) sum += [k][i][j]; return sum; Wht is wrong with this code? How cn it be fixed? [3] [3] [3] [3] [3] [3] [3] [3] [3] m = 0 m = 2 m = 1 23

24 L16: Cches I Loclity Exmple #3 int sum_rry_3d(int [M][N][L]) { int i, j, k, sum = 0; } for (i = 0; i < N; i++) for (j = 0; j < L; j++) for (k = 0; k < M; k++) sum += [k][i][j]; return sum; Wht is wrong with this code? How cn it be fixed? Lyout in Memory (M =?, N = 3, L = 4) [3] [3] [3] [3] [3] [3]

25 L16: Cches I Cche Performnce Metrics Huge difference between cche hit nd cche miss Could be 100x speed difference between ccessing cche nd min memory (mesured in clock cycles) Miss Rte (MR) Frction of memory references not found in cche (misses / ccesses) = 1 - Hit Rte Hit Time (HT) Time to deliver block in the cche to the processor Includes time to determine whether the block is in the cche Miss Penlty (MP) Additionl time required becuse of miss 25

26 L16: Cches I Cche Performnce Two things hurt the performnce of cche: Miss rte nd miss penlty Averge Memory Access Time (AMAT): verge time to ccess memory considering both hits nd misses AMAT = Hit time + Miss rte Miss penlty (bbrevited AMAT = HT + MR MP) AMAT= HR*HT + MR(HT + MP) = HT(HR+MR) + MR*MP = HT(1) + MR*MP 99% hit rte twice s good s 97% hit rte! Assume HT of 1 clock cycle nd MP of 100 clock cycles 97%: AMAT = 99%: AMAT = 26

27 L16: Cches I Question ps= pico second = Processor specs: 200 ps clock, MP of 50 clock cycles, MR of 0.02 misses/instruction, nd HT of 1 clock cycle AMAT = Which improvement would be best for AMAT? A. 190 ps clock B. Miss penlty of 40 clock cycles C. MR of misses/instruction 27

28 L16: Cches I Cn we hve more thn one cche? Why would we wnt to do tht? Avoid going to memory! Typicl performnce numbers: Miss Rte L1 MR = 3-10% L2 MR = Quite smll (e.g. < 1%), depending on prmeters, etc. Hit Time L1 HT = 4 clock cycles L2 HT = 10 clock cycles Miss Penlty P = cycles for missing in L2 & going to min memory Trend: incresing! 28

29 L16: Cches I Memory Hierrchies Some fundmentl nd enduring properties of hrdwre nd softwre systems: Fster storge technologies lmost lwys cost more per byte nd hve lower cpcity The gps between memory technology speeds re widening True for: registers cche, cche DRAM, DRAM disk, etc. Well-written progrms tend to exhibit good loclity These properties complement ech other beutifully They suggest n pproch for orgnizing memory nd storge systems known s memory hierrchy 29

30 L16: Cches I An Exmple Memory Hierrchy <1 ns 5-10 s registers Smller, fster, costlier per byte 5-10 ns 1 ns on-chip L1 cche (SRAM) off-chip L2 cche (SRAM) 1-2 min Lrger, slower, cheper per byte 100 ns 150,000 ns 10,000,000 ns (10 ms) SSD Disk min memory (DRAM) locl secondry storge (locl disks) min 31 dys 66 months = 5.5 yers ms remote secondry storge (distributed file systems, web servers) 1-15 yers 30

31 L16: Cches I An Exmple Memory Hierrchy Smller, fster, costlier per byte registers on-chip L1 cche (SRAM) off-chip L2 cche (SRAM) CPU registers hold words retrieved from L1 cche L1 cche holds cche lines retrieved from L2 cche L2 cche holds cche lines retrieved from min memory Lrger, slower, cheper per byte min memory (DRAM) locl secondry storge (locl disks) remote secondry storge (distributed file systems, web servers) Min memory holds disk blocks retrieved from locl disks Locl disks hold files retrieved from disks on remote network servers 31

32 L16: Cches I An Exmple Memory Hierrchy Smller, fster, costlier per byte registers on-chip L1 cche (SRAM) off-chip L2 cche (SRAM) explicitly progrm-controlled (e.g. refer to exctly %rx, %rbx) progrm sees memory ; hrdwre mnges cching trnsprently Lrger, slower, cheper per byte min memory (DRAM) locl secondry storge (locl disks) remote secondry storge (distributed file systems, web servers) 32

33 L16: Cches I Memory Hierrchies Fundmentl ide of memory hierrchy: For ech level k, the fster, smller device t level k serves s cche for the lrger, slower device t level k+1 Why do memory hierrchies work? Becuse of loclity, progrms tend to ccess the dt t level k more often thn they ccess the dt t level k+1 Thus, the storge t level k+1 cn be slower, nd thus lrger nd cheper per bit Big Ide: The memory hierrchy cretes lrge pool of storge tht costs s much s the chep storge ner the bottom, but tht serves dt to progrms t the rte of the fst storge ner the top 33

34 L16: Cches I Intel Core i7 Cche Hierrchy Processor pckge Core 0 Regs Core 3 Regs Block size: 64 bytes for ll cches L1 d-cche L1 i-cche L1 d-cche L1 i-cche L1 i-cche nd d-cche: 32 KiB, 8-wy, Access: 4 cycles L2 unified cche L2 unified cche L2 unified cche: 256 KiB, 8-wy, Access: 11 cycles L3 unified cche (shred by ll cores) L3 unified cche: 8 MiB, 16-wy, Access: cycles Min memory 34

35 L16: Cches I Summry Memory Hierrchy Successively higher levels contin most used dt from lower levels Exploits temporl nd sptil loclity Cches re intermedite storge levels used to optimize dt trnsfers between ny system elements with different chrcteristics Cche Performnce Idel cse: found in cche (hit) Bd cse: not found in cche (miss), serch in next level Averge Memory Access Time (AMAT) = HT + MR MP Hurt by Miss Rte nd Miss Penlty 35