Lecture-22 (Cache Coherence Protocols) CS422-Spring

Transcription

1 Lecture-22 (Cache Coherence Protocols) CS422-Spring 2018

2 Single Core Core 0 Private L1 Cache Bus (Packet Scheduling) Private L2 DRAM CS422: Spring 2018 Biswabandan Panda, CSE@IITK 2

3 Multicore Core 0 Core 1 Core k Private L1 Cache Private L1 Cache Private L1 Cache Private L2 Cache Private L2 Cache Private L2 Cache Interconnect (Packet Scheduling) I Shared Last Level Cache (L3) DRAM CS422: Spring 2018 Biswabandan Panda, CSE@IITK 3

4 Cache Coherence P1 Cache Bus LLC/DRAM sum=0 CS422: Spring 2018 Biswabandan Panda, 4

5 Cache Coherence rd &sum P1 sum =0 V DRAM sum=0 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 5

6 Cache Coherence P1 rd &sum sum=0 V sum=0 V DRAM sum=0 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 6

7 Cache Coherence wr sum (sum = 3) P1 sum=0 3 V D sum=0 V DRAM sum=0 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 7

8 Cache Coherence P1 wr &sum (sum = 7) sum=3 D sum=0 7 V D DRAM sum=0 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 8

9 Cache Coherence Err! Voila rd &sum P1 print sum=3 print sum=7 sum=3 D sum=7 D DRAM sum=0 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 9

10 The Problem If multiple cores cache the same block, how do they ensure they all see a consistent state? Solution: 1. Write Propagation: All writes eventually become visible to other cores. 2. Write Serialization: All cores see write to a cache line in same order. Need a protocol to ensure (1) and (2) called cache coherence protocol CS422: Spring 2018 Biswabandan Panda, CSE@IITK 10

11 Non-solutions Keeping caches coherent is software s responsibility (+) Makes microarchitect s life (my) easier (-) Makes average programmer s life (yours) much harder CS422: Spring 2018 Biswabandan Panda, CSE@IITK 11

12 Non-solutions All caches are shared between all processors + No need for coherence -- Shared cache becomes the bandwidth bottleneck -- Scalability : 100 monkeys 1 banana CS422: Spring 2018 Biswabandan Panda, CSE@IITK 12

13 Who Is Responsible? (1) ISA - What if the ISA provided a cache flush/invalidate instruction? Such as - FLUSH-LOCAL, FLUSH-GLOBAL, FLUSH-CACHE (2) Hardware - Invalidate all other copies of block say A when a core writes to it Simplifies yours job CS422: Spring 2018 Biswabandan Panda, CSE@IITK 13

14 Invalidate vs Update Invalidate Write to a cache line, and simultaneously broadcast invalidation of address to all others Other cores clear/invalidate their cache lines Update Write to a cache line, and simultaneously broadcast written data to all others Other cores update their caches if data was present CS422: Spring 2018 Biswabandan Panda, CSE@IITK 14

15 MSI Coherence Protocol Extend single valid bit per line to three states: M(odified): only one cache, memory not updated. S(hared): one or more caches, and memory copy is up-to-date I(nvalid): not present Read - Read request on bus, transitions to S Write - ReadEx request, transitions to M state CS422: Spring 2018 Biswabandan Panda, CSE@IITK 15

16 State Transitions Core Side PrRd/ PrWr/ PrRd/ M PrWr/BusRdX S PrWr/BusRdX I PrRd/BusRd CS422: Spring 2018 Biswabandan Panda, CSE@IITK 16

17 Bus Side BusRd/ M BusRd/Flush S BusRdX/Flush I BusRd/ BusRdX/ BusRdX/ CS422: Spring 2018 Biswabandan Panda, CSE@IITK 17

18 MSI State Bits Tag Data 31 0 P1 Cache Bus Snooper Snooper Snooper DRAM X=1 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 18

19 MSI P1 rd &X BusRd Snooper Snooper Snooper DRAM X=1 CS422: Spring 2018 Biswabandan Panda, 19

20 MSI P1 X=1 S Snooper Snooper Snooper DRAM X=1 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 20

21 MSI P1 wr &X (X=2) X=1 2 S M Snooper Snooper Snooper DRAM X=1 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 21

22 MSI P1 X=2 M Snooper Snooper Snooper rd &X BusRd DRAM X=1 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 22

23 MSI P1 X=2 M S X=2 S Snooper Flush Snooper Snooper DRAM X=1 2 Cancel DRAM read CS422: Spring 2018 Biswabandan Panda, CSE@IITK 23

24 MSI P1 X=2 S I X=2 3 S Snooper Snooper Snooper wr &X X=3 M BusRdX LLC X=2 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 24

25 MSI P1 rd &X BusRd X=2 3 I S X=3 M Snooper Snooper Snooper Flush S DRAM X=2 3 Update DRAM as well CS422: Spring 2018 Biswabandan Panda, CSE@IITK 25

26 MSI P1 X=3 S X=3 S rd &X Snooper Snooper Snooper DRAM X=3 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 26

27 MSI Proc Action State P1 State State Bus Action Data From R BusRd Mem W BusRdX Mem R BusRd P1 cache W BusRdX Mem R BusRd cache R R BusRd Mem CS422: Spring 2018 Biswabandan Panda, CSE@IITK 27

28 MSI Proc Action State P1 State State Bus Action Data From R1 S - - BusRd Mem W1 M - - BusRdX Mem R3 S - S BusRd P1 cache W3 I - M BusRdX Mem R1 S - S BusRd cache R3 S - S - - R2 S S S BusRd Mem CS422: Spring 2018 Biswabandan Panda, CSE@IITK 28

29 1. Write Propagation: All writes eventually become visible to other cores. In MSI - (Through Invalidation and flush on subsequent BusRds) 2. Write Serialization: All cores see write to a cache line in same order. In MSI Writes (BusRdX) that go to the bus appear in bus order (and handled by snoopers in bus order!) CS422: Spring 2018 Biswabandan Panda, CSE@IITK 29

30 Issues Rd, Wr sequence incurs 2 transactions even when no one sharing (e.g., serial program!) BusRd (I->S) followed by BusRdX (S->M) In general, penalizing serial programs is unacceptable CS422: Spring 2018 Biswabandan Panda, CSE@IITK 30

31 Solution - MESI Add Exclusive state: Invalid modified (dirty) shared (two or more caches may have copies) Exclusive: (only this cache has clean copy, same value as in memory) How to decide I E or I S? Need to check whether someone else has copy a separate signal on bus: wired-or line asserted in response to BusRd. Call it C = copy exists CS422: Spring 2018 Biswabandan Panda, CSE@IITK 31

32 State Transitions Processor level M E PrWr/BusRdX PrRd/BusRd(!C) PrWr/BusUpgr S PrRd/BusRd(C) I PrRd/- PrWr/- PrRd/- PrWr/- PrRd/- CS422: Spring 2018 Biswabandan Panda, CSE@IITK 32

33 Bus Level M BusRdX/Flush E BusRd/Flush BusRd/FlushOpt BusRdX/FlushOpt BusRd/FlushOpt S BusRdX/FlushOpt BusUpgr/- I BusRd/- BusRdX/- BusUpgr/- CS422: Spring 2018 Biswabandan Panda, CSE@IITK 33

34 MESI: An Example P1 Cache Bus Snooper Snooper Snooper Mem Ctrl Main Memory X=1 CS422: Spring 2018 Biswabandan Panda, 34

35 P1 rd &X BusRd Snooper Snooper Snooper Mem Ctrl X=1 CS422: Spring 2018 Biswabandan Panda, 35

36 P1 X=1 E Snooper Snooper Snooper Mem Ctrl X=1 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 36

37 P1 wr &X (X=2) X=1 2 E M Snooper Snooper Snooper Mem Ctrl X=1 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 37

38 P1 rd &X X=2 M Snooper Snooper Snooper BusRd Mem Ctrl X=1 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 38

39 P1 X=2 M S X=2 S Snooper Snooper Snooper Flush Mem Ctrl X=1 2 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 39

40 P1 X=2 S I X=2 3 S M wr &X X=3 Snooper Snooper Snooper BusUpgr Mem Ctrl X=2 Note: BusUpgr instead of BusRdX. Mem Ctrl knows it can ignore the request CS422: Spring 2018 Biswabandan Panda, CSE@IITK 40

41 P1 rd &X X=3 X=3 S I X=3 S M Snooper Snooper Snooper BusRead Mem Ctrl X=3 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 41

42 P1 rd &X X=3 S X=3 S Snooper Snooper Snooper Mem Ctrl X=3 CS422: Spring 2018 Biswabandan Panda, CSE@IITK 42

43 P1 rd &X X=3 S X=3 S X=3 S Snooper Snooper Snooper FlushOpt BusRd Mem Ctrl X=3 Referred to as Cache-to-cache transfer in Illinois MESI protocol CS422: Spring 2018 Biswabandan Panda, CSE@IITK 43

44 Proc Action State P1 State State Bus Action Data From R1 E - - BusRd Mem W1 M R3 S - S BusRd/Flush P1 s cache W3 I - M BusUpgr - R1 S - S BusRd/Flush s cache R3 S - S - - R2 S S S BusRd/Flush Opt P1/ s cache CS422: Spring 2018 Biswabandan Panda, CSE@IITK 44

45 Coherence misses: misses due to accesses to blocks that were invalidated due to coherence events True sharing: misses that occur when multiple threads share the same data item (byte/word) False sharing: misses that occur when multiple threads share different data items located in a single cache block CS422: Spring 2018 Biswabandan Panda, CSE@IITK 45

46 Parameters True Sharing Misses False Sharing Misses Larger cache size increased increased Larger block/line size Larger associativity decreased unclear increased unclear CS422: Spring 2018 Biswabandan Panda, 46

47 Directory Based P P P P P $ $ $ $ $ Interconnection Network LLC C(k) C(k+1) C(k+j) 1 modified bit for each cache block in LLC/ memory 1 presence bit for each core, each cache block in LLC/ memory CS422: Spring 2018 Biswabandan Panda, CSE@IITK 47