An#Aside# Course#thus#far:# We#have#focused#on#parallelism:#how#to#divide#up#work#into# parallel#tasks#so#that#they#can#be#done#faster.

Transcription

1 An#Aside# Course#thus#far:# We#have#focused#on#parallelism:#how#to#divide#up#work#into# parallel#tasks#so#that#they#can#be#done#faster.# Rest#of#the#term:# We#will#focused#on#concurrency:#how#to#correctly#and# efficiently#manage#access#to#shared#resources.## work' requests' resources' parallelism. resource' concurrency.

2 Memory#Coherence# Coherence"#by#Danial#J.#Sorin,#Mark#D.#Hill,#and#David#A.#Wood#####

3 The#Shared#Memory#AbstracLon# x'=='1' load'x' x'='1' x:#1# Memory' rogrammer s#intuilon:#memory#is#coherent.##

4 The#roblem#of#Coherence# load'x' $' $' $' $' x:#0# Memory'

5 The#roblem#of#Coherence# load'x' load'x' x#=#0# x#=#0# x:#0# Memory'

6 The#roblem#of#Coherence# load'x' x'='1' x#=#0# x#=#1# x:#0# Memory' Coherence#problems#exist#because#the#abstracLon#of#shared# address#space#is#implemented#by#both#a#global#storage#(main#

7 Cache#Hierarchy#of#Intel#Core#i7#CU# Shared L3 Cache (One bank per core) L3: (per chip) 8 MB, inclusive 16-way set associative 32B / clock per bank cycle latency Ring Interconnect L2 Cache L2 Cache L2 Cache L2 Cache L2: (private per core) 256 KB 8-way set associative, write back 32B / clock, 12 cycle latency Up to 16 outstanding misses L1 Data Cache Core L1 Data Cache Core L1 Data Cache Core L1 Data Cache Core L1: (private per core) 32 KB 8-way set associative, write back 2 x 16B loads + 1 x 16B store per clock 4-6 cycle latency Up to 10 outstanding misses

8 DefiniLon#of#Coherence# core#1#and#2# core#3# core#4# core#3# Dividing#a#memory#locaLon s#lifelme#into#epochs,#where# 1. Every#epoch#has#a#either#a#single#writer#or#mulLple#readers# (write'serializa4on:'ex:#1#writes#1#to#x#and#2#write#2#to#x,# the#values#are#observed#in#the#same#order).# Lme# 2. The#value#of#the#memory#locaLon#propagate#from#the#end#of# one#epoch#to#the#beginning#of#the#next#epoch## (value'propaga4on:#the#new#value#eventually#gets#to#other# processors.).# A#cache#coherence#protocol#maintains#these#two#invariants.# (The#granularity#of#coherence#is#a#cache#line#size.)#

9 Memory#Coherence#vs#Memory#Consistency# Coherence#only#defines#the#behavior#of#reads#and# writes#to#the#same#memory'loca4on.### It#does#not#define#a#global#ordering#among#reads#and# writes#to#different#memory#localons#from#two#different# processors.### You#cannot#put#the#two#memory#operaLons#on#a#single# Lmeline#and#have#both#processor s#observalons#agree# with#the#lmeline).# Memory#consistency#(later)#defines#the#behaviro#of# reads#and#writes#to#different'memory'loca4ons.#

10 ImplemenLng#Coherence# We#will#assume:# private#caches## memory#upon#eviclon)# upon#a#memory#access#(load#or#store),#the#memory# gets#brought#into#the#cache.# We#will#focus#on#hardware#cache#coherence#protocol:# snooping# directory#

11 Cache#line:# ImplemenLng#Coherence# Line state Tag Data (64 bytes on Intel Core i7) Dirty bit Cache#coherence#protocol#uses#the#state#bits#to#maintain#a#state# machine#per#cache#line.## ossible#cache#line#states:# Modified'(M):#the#block#is#valid,#exclusive,#owned#by#this# cache,#and#potenlally#dirty.# Invalid'(I):'the#block#is#invalid.# Exclusive'(E):'the#block#is#valid,#exclusive,#and#clean## (an#oplmizalon;#not#strictly#necessary).# (There#is#also#"Owned"#and#"Forward"#states,#but#we#will#ignore#those.)#

12 Snooping#rotocol#

13 System#Model#for#Snooping#Cache# cache'' controller' L1'L2' cache' cache'' controller' L1'L2' cache' Interconnect' LLC/memory' controller' lastflevel'' cache'(llc)' Broadcast#request#on# interconnect,#and#all#the# controllers#colleclvely# do#the#right#thing.# Mul4core' processor'chip' Main'memory'

14 System#Model#for#Snooping#Cache# load#/#store# cache'' controller' L1'L2' cache' cache'' controller' L1'L2' cache' Interconnect' LLC/memory' controller' lastflevel'' cache'(llc)' Mul4core' processor'chip' Main'memory'

15 System#Model#for#Snooping#Cache# examine#cache#line# state#and#send#out# request# cache'' controller' L1'L2' cache' cache'' controller' L1'L2' cache' Interconnect' LLC/memory' controller' lastflevel'' cache'(llc)' Mul4core' processor'chip' Main'memory'

16 System#Model#for#Snooping#Cache# cache'' controller' L1'L2' cache' maintain#line#state#from# the#cache's#point#of#view:# M#means#some#cache# owns#it;#otherwise#not.# Interconnect' cache'' controller' L1'L2' cache' LLC/memory' controller' lastflevel'' cache'(llc)' Mul4core' processor'chip' Main'memory'

17 Basic#MSI#InvalidaLon#rotocol# Key#tasks#of#protocol# obtain#exclusive#access#for#a#write# localng#most#recent#copy#of#data#on#a#cache#miss# Cache#line#state:#M,#S,#I# rocessor#events:# rrd#(load)# rwr#(store)# Interconnect#events:# BusRd:#obtain#a#copy#with#no#intent#to#modify# BusRdX:#obtain#a#copy#with#intent#to#modify# BusWB:#write#the#line#back#to#memory#

18 MSI#State#TransiLon#Diagram# rrd / -- rwr / -- M (Modified) A/B:#if#acLon#A#is#observed#by#cache# controller,#aclon#b#is#taken.# :#inilated#by#other#processors# :#inilated#by#this#processor# flush#:#write#dirty#line#back#to#memory# rwr / BusRdX BusRd / flush rwr / BusRdX S (Shared) BusRdX / flush rrd / BusRd rrd / -- BusRdX / -- BusRd / -- I (Invalid)

19 Cache#Controller## MSI#State#TransiLon#Diagram# rrd / -- rwr / -- M (Modified) A/B:#if#acLon#A#is#observed#by#cache# controller,#aclon#b#is#taken.# :#inilated#by#other#processors# :#inilated#by#this#processor# flush#:#write#dirty#line#back#to#memory# rwr / BusRdX BusRd / flush rwr / BusRdX S (Shared) BusRdX / flush rrd / BusRd rrd / -- BusRdX / -- BusRd / -- I (Invalid)

20 LLC#/#Memory#Controller## MSI#State#TransiLon#Diagram# M' BusRdX# BusRd#/#Flush# S'or'I' It#also#sends#the#data#if#it's#cache#line#state#is#in#S#or#I.#

21 Does#MSI#SaLsfy#Coherence?# Write#serializaLon:# two#writes#from#different#processors#are#serialized# by#the#order#their#requests#arrive#to#interconnect# (crucial:'we#assume#atomic#transaclon).# readers#at#a#lme.# Value#propagaLon:# achieved#via#a#combinalon#of#invalidalon#on# BusRd#/#BusRdX#from#another#processor.#

22 MESI#InvalidaLon#rotocol# MSI#requires#two#bus#transacLons#for#the# common#case#of#reading#data#then#subsequently# wrilng#to#it:# BusRd#to#move#from#I#to#S# BusRdX#to#move#from#S#to#M# Adding#an#E#state:# upgrading#from#e#to#m#does#not#require#any#bus# transaclon## cache#line#is#clean#(not#modified#yet),#but#this#cache# has#the#only#copy:# LLC#explicitly#disLnguish#between#I#and#S#(conservaLve#or# we'd#require#addilonal#transaclon#upon#s#line#eviclon).#

23 MESI#State#TransiLon#Diagram# MESI state transition diagram rrd / -- rwr / -- M (Modified) rwr / -- rwr / BusRdX E (Exclusive) BusRd / flush rwr / BusRdX rrd / -- BusRd / -- BusRdX / flush S (Shared) rrd / BusRd (no other cache asserts shared) rrd / BusRd (another cache asserts shared) rrd / -- BusRd / -- BusRdX / -- BusRdX / -- I (Invalid) CMU , Spring 2014

24 Who#should#supply#data#on#a#cache#miss#when# line#is#in#the#e#or#s#state#of#another#cache?# Can#get#it#either#from#LLC#/#Memory#control#or# from#another#cache#controller.# If#from#another#cache#controller,#which#cache# controller#should#provide#it?# (why#there#is#"mesif"#and#"moeis")# commonly#used#to#reduce#latency#of#access# and#memory#bandwidth.#

25 False#sharing# Two#processors#write#to#different#memory# localons#that#map#to#the#same#cache#line.# Each#processor#gets#it#in#M#state#whenever#it# between#the#caches#of#the#wrilng#processors.# An#arLfact#of#the#fact#that#we#maintain# coherence#with#cache#line#granularity.# Your#program#performance#will#suck.#