Memory Hierarchy in a Multiprocessor

Transcription

1 EEC 581 Computer Architecture Multiprocessor and Coherence Department of Electrical Engineering and Computer Science Cleveland State University Hierarchy in a Multiprocessor Shared cache Fully-connected shared memory (Dancehall) Bus-based shared memory Distributed shared memory 2 1

2 Coherency Closest cache level is private Multiple copies of cache line can be present across different processor nodes Local updates Lead to incoherent state roblem exhibits in both write-through and writeback caches Bus-based globally visible oint-to-point interconnect visible only to communicated processor nodes 3 Example (Writeback ) Rd? X= -100 Rd? X= -100 X=

3 Example (Write-through ) Rd? X= -100 X= Defining Coherence An M is coherent if the results of any execution of a program can be reconstructed by a hypothetical serial order Implicit definition of coherence Write propagation Writes are visible to other processes Write serialization All writes to the same location are seen in the same order by all processes (to all locations called write atomicity) E.g., w1 followed by w2 seen by a read from 1, will be seen in the same order by all reads by other processors i 6 3

4 Sounds Easy? A=0 B= T1 A=1 B=2 T2 A=1 A=1 B=2 B=2 T3 A=1 A=1 B=2 B=2 A=1 B=2 T3 A=1 A=1 B=2 B=2 B=2 B=2 A=1 A=1 See A s update before B s See B s update before A s 7 Bus Snooping based on Write-Through All the writes will be shown as a transaction on the shared bus to memory Two protocols Update-based rotocol Invalidation-based rotocol 8 4

5 Bus Snooping (Update-based rotocol on Write-Through cache) X= 505 Bus transaction Bus snoop Each processor s cache controller constantly snoops on the bus Update local copies upon snoop hit 9 Bus Snooping (Invalidation-based rotocol on Write-Through cache) Load X X= 505 Bus transaction Bus snoop Each processor s cache controller constantly snoops on the bus Invalidate local copies upon snoop hit 10 5

6 A Simple Snoopy Coherence rotocol for a WT, No Write-Allocate rrd / --- rwr / BusWr Valid rrd / BusRd BusWr / --- Invalid Observed / Transaction rwr / BusWr 11 rocessor-initiated Transaction Bus-snooper-initiated Transaction Coherence rotocols for WB caches A cache has an exclusive copy of a line if It is the only cache having a valid copy may or may not have it Modified (dirty) cache line The cache having the line is the owner of the line, because it must supply the block 12 6

7 Coherence rotocol (Update-based rotocol on Writeback cache) Store X update update Bus transaction Update data for all processor nodes who share the same data For a processor node keeps updating the memory location, a lot of traffic will be incurred 13 Coherence rotocol (Update-based rotocol on Writeback cache) Load X Store X X= Hit! X= X= update update Bus transaction Update data for all processor nodes who share the same data For a processor node keeps updating the memory location, a lot of traffic will be incurred 14 7

8 Coherence rotocol (Invalidation-based rotocol on Writeback cache) Store X X= -100 X= -100 invalidate invalidate Bus transaction Invalidate the data copies for the sharing processor nodes Reduced traffic when a processor node keeps updating the same memory location 15 Coherence rotocol (Invalidation-based rotocol on Writeback cache) Load X X= 505 Miss! X= 505 Snoop hit Bus transaction Bus snoop Invalidate the data copies for the sharing processor nodes Reduced traffic when a processor node keeps updating the same memory location 16 8

9 Coherence rotocol (Invalidation-based rotocol on Writeback cache) Store X Store X Store X X= X= 505 Bus transaction Bus snoop Invalidate the data copies for the sharing processor nodes Reduced traffic when a processor node keeps updating the same memory location 17 MSI Writeback Invalidation rotocol Modified Dirty Only this cache has a valid copy Shared is consistent One or more caches have a valid copy Invalid Writeback protocol: A cache line can be written multiple times before the memory is updated. 18 9

10 MSI Writeback Invalidation rotocol Two types of request from the processor rrd rwr Three types of bus transactions post by cache controller BusRd rrd misses the cache or another cache supplies the line BusRd exclusive (Read-to-own) rwr is issued to a line which is not in the Modified state BusWB Writeback due to replacement rocessor does not directly involve in initiating this operation 19 MSI Example X=10 S BusRd Bus MEMORY X=10 rocessor Action State in 1 State in 2 State in 3 Bus Transaction Data Supplier 1 reads X S BusRd 20 10

11 MSI Example X=10 S X=10 S BusRd Bus MEMORY X=10 rocessor Action State in 1 State in 2 State in 3 Bus Transaction Data Supplier 1 reads X S BusRd 3 reads X S --- S BusRd 21 MSI Example X= SI X=10 X=-25 SM BusRdX Bus MEMORY X=10 rocessor Action State in 1 State in 2 State in 3 Bus Transaction Data Supplier 1 reads X 3 reads X S S S BusRd BusRd 3 writes X I --- M BusRdX Does not come from memory if having BusUpgrade 22 11

12 MSI Example X=-25 IS X=-25 MS BusRd Bus MEMORY X=10 X=-25 rocessor Action State in 1 State in 2 State in 3 Bus Transaction Data Supplier 1 reads X 3 reads X S S S BusRd BusRd 3 writes X I --- M BusRdX 1 reads X S --- S BusRd 3 23 MSI Example X=-25 S X=-25 S X=-25 MS BusRd Bus MEMORY X=10 X=-25 rocessor Action State in 1 State in 2 State in 3 Bus Transaction Data Supplier 1 reads X 3 reads X S S S BusRd BusRd 3 writes X I --- M BusRdX 1 reads X S --- S BusRd 3 2 reads X S S S BusRd 24 12

13 MESI Writeback Invalidation rotocol To reduce two types of unnecessary bus transactions BusRdX that snoops and converts the block from S to M when only you are the sole owner of the block BusRd that gets the line in S state when there is no sharers (that lead to the overhead above) Introduce the Exclusive state One can write to the copy without generating BusRdX Illinois rotocol: roposed by amarcos and atel in 1984 Employed in Intel, owerc, MIS 25 MESI Writeback Invalidation rotocol rocessor Request (Illinois rotocol) rwr / --- rrd / --- rrd, rwr / --- Exclusive Modified rrd / BusRd (not-s) rwr / BusRdX rwr / BusRdX Invalid Shared S: Shared Signal rocessor-initiated rrd / BusRd (S) 26 rrd /

14 MESI Writeback Invalidation rotocol Bus Transactions (Illinois rotocol) Whenever possible, Illinois protocol performs -to- transfer rather than having memory to supply the data Use a Selection algorithm if there are multiple suppliers (Alternative: add an O state or force update memory) Most of the MESI implementations simply write to memory Exclusive BusRd / Flush Or ---) Modified BusRdX / --- BusRd / Flush BusRdX / Flush Invalid BusRdX / Flush* Shared BusRd / Flush* 27 Bus-snooper-initiated Flush*: Flush for data supplier; no action for other sharers MESI Writeback Invalidation rotocol (Illinois rotocol) rwr / --- rrd / --- rrd, rwr / --- rrd / BusRd (not-s) Exclusive BusRd / Flush (or ---) rwr / BusRdX BusRdX / --- BusRdX / Flush Modified BusRd / Flush rwr / BusRdX Invalid BusRdX / Flush* S: Shared Signal rocessor-initiated rrd / 28 BusRd (S) Bus-snooper-initiated Flush*: Flush for data supplier; no action for other sharers Shared BusRd / Flush* rrd /

15 MOESI rotocol Add one additional state Owner state Similar to Shared state The O state processor will be responsible for supplying data (copy in memory may be stale) Employed by Sun UltraSparc AMD Opteron In dual-core Opteron, cache-to-cache transfer is done through a system request interface (SRI) running at full CU speed 29 CU0 L2 CU1 L2 System Request Interface Mem Controller Crossbar Hyper- Transport Implication on Multi-Level s How to guarantee coherence in a multi-level cache hierarchy Snoop all cache levels? Intel s 8870 chipset has a snoop filter for quad-core Maintaining inclusion property Ensure data in the outer level must be present in the inner level Only snoop the outermost level (e.g. L2) L2 needs to know L1 has write hits Use Write-Through cache Use Write-back but maintain another modified-but-stale bit in L

16 Inclusion roperty Not so easy Replacement: Different bus observes different access activities, e.g. L2 may replace a line frequently accessed in L1 Split L1 caches: Imagine all caches are direct-mapped. Different cache line sizes 31 Inclusion roperty Use specific cache configurations E.g., DM L1 + bigger DM or set-associative L2 with the same cache line size Explicitly propagate L2 action to L1 L2 replacement will flush the corresponding L1 line Observed BusRdX bus transaction will invalidate the corresponding L1 line To avoid excess traffic, L2 maintains an Inclusion bit for filtering (to indicate in L1 or not) 32 16

17 Directory-based Coherence rotocol Modified bit resence bits, one for each node Snooping-based protocol Directory N transactions for an N-node M All caches need to watch every memory request from each processor Not a scalable solution for maintaining coherence in large shared memory systems Directory protocol Directory-based control of who has what; HW overheads to keep the directory (~ # lines * # processors) 33 Directory-based Coherence rotocol C(k) C(k+1) C(k+j) 1 modified bit for each cache block in memory 1 presence bit for each processor, each cache block in memory 34 17

18 Directory-based Coherence rotocol (Limited Dir) modified bit for each cache block in memory resence encoding is NULL or not Encoded resent bits (lg 2 N), each cache line can reside in 2 processors in this example 35 Distributed Directory Coherence rotocol Directory Directory Directory Directory Directory Directory Centralized directory is less scalable (contention) Distributed shared memory (DSM) for a large M system Interconnection network is no longer a shared bus Maintain cache coherence (CC-NUMA) Each address has a home 36 18

19 Distributed Directory Coherence rotocol Snoop bus Snoop bus Directory Directory Stanford DASH (4 CUs in each cluster, total 16 clusters) Invalidation-based cache coherence Directory keeps one of the 3 status of a cache block at its home node Uncached Shared (unmodified state) Dirty 37 DASH Hierarchy Snoop bus Snoop bus Directory Directory rocessor Level Local Cluster Level Home Cluster Level (address is at home) If dirty, needs to get it from remote node which owns it Remote Cluster Level 38 19

20 Directory Coherence rotocol: Read Miss Miss Z (read) Z Z Home of Z Go to Home Node Z Data Z is shared (clean) 39 Directory Coherence rotocol: Read Miss Miss Z (read) Z Z Go to Data Home Request Node Respond with Owner Info Z Data Z Data is Clean, Z is Dirty Shared by 3 nodes 40 20

21 Directory Coherence rotocol: Write Miss Miss Z (write) Z Z Z Invalidate Go to Home Node ACK ACK Respond w/ sharers Z Write Z can proceed in