Modern High-Performance Locking

Transcription

1 Modern High-Performance Locking Nir Shavit Slides based in part on The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

2 Locks (Mutual Exclusion) public interface Lock { public void lock(); public void unlock(); } Art of Multiprocessor Programming 2

3 Locks (Mutual Exclusion) public interface Lock { public void lock(); acquire lock public void unlock(); } Art of Multiprocessor Programming 3

4 Locks (Mutual Exclusion) public interface Lock { public void lock(); public void unlock(); } acquire lock release lock Art of Multiprocessor Programming 4

5 Mutual Exclusion Properties Mutual Exclusion At most one thread holds the lock (has completed lock() and not completed unlock()) at any time Art of Multiprocessor Programming 5

6 Mutual Exclusion Properties Freedom from Deadlock If a thread calls lock() or unlock() and never returns, then other threads must complete invocations of lock() and unlock() infinitely often. Art of Multiprocessor Programming 6

7 Mutual Exclusion Properties Freedom from Starvation Every call to lock() or unlock() eventually returns. Art of Multiprocessor Programming 7

8 Locking and Amdahl s Law Coarse Grained Speedup= 1 1 p + p n Fine Grained 25% Shared Parallel fraction 25% Shared 75% Unshared 75% Unshared Concurrent Program Concurrent Program

9 Locking and Amdahl s Law Honk! Honk! Honk! Amdahls Law: only 2.9x speedup with 8 processors Coarse Grained Fine Grained 25% Shared 25% Shared 75% Unshared 75% Unshared Concurrent Program Concurrent Program

10 Locking and Amdahl s Law Honk! Honk! Honk! Only 4x speedup with processors Coarse Grained Fine Grained 25% Shared 25% Shared 75% Unshared 75% Unshared Concurrent Program Concurrent Program

11 Locking and Amdahl s Law Honk! Honk! Honk! Why fine-grained low overhead locking maters Coarse Grained Fine Grained 25% Shared 25% Shared 75% Unshared 75% Unshared Concurrent Program Concurrent Program

12 What Should you do if you can t get a lock? Keep trying spin or busy-wait Good if delays are short Give up the processor Good if delays are long Always good on uniprocessor Art of Multiprocessor Programming 12 (1)

13 What Should you do if you can t get a lock? Keep trying spin or busy-wait Good if delays are short Give up the processor Good if delays are long Always good on uniprocessor our focus Art of Multiprocessor Programming 13

14 Basic Spin-Lock CS. spin lock critical section Resets lock upon exit Art of Multiprocessor Programming 14

15 Basic Spin-Lock lock introduces sequential bottleneck CS. spin lock critical section Resets lock upon exit Art of Multiprocessor Programming 15

16 Basic Spin-Lock lock suffers from contention CS. spin lock critical section Resets lock upon exit Art of Multiprocessor Programming 16

17 Basic Spin-Lock lock suffers from contention CS. spin lock critical section Resets lock upon exit These are distinct phenomena Art of Multiprocessor Programming 17

18 Basic Spin-Lock lock suffers from contention CS. spin lock critical section Resets lock upon exit Seq Bottleneck à no parallelism Art of Multiprocessor Programming 18

19 Basic Spin-Lock lock suffers from contention CS. spin lock critical section Resets lock upon exit Contention à overloaded communication medium Art of Multiprocessor Programming 19

20 Mutual Exclusion What do we want to optimize? Bus bandwidth used by spinning threads Release/Acquire latency Acquire latency for idle lock Art of Multiprocessor Programming 20

21 Review: Test-and-Set public class AtomicBoolean { boolean value; public synchronized boolean getandset(boolean newvalue) { boolean prior = value; value = newvalue; return prior; } } Art of Multiprocessor Programming 21 (5)

22 Review: Test-and-Set public class AtomicBoolean { boolean value; public synchronized boolean getandset(boolean newvalue) { boolean prior = value; value = newvalue; return prior; } } Package java.util.concurrent.atomic Art of Multiprocessor Programming 22

23 Review: Test-and-Set public class AtomicBoolean { boolean value; public synchronized boolean getandset(boolean newvalue) { boolean prior = value; value = newvalue; return prior; } } Swap old and new values. Art of Multiprocessor Programming 23

24 Review: Test-and-Set AtomicBoolean lock = new AtomicBoolean(false) boolean prior = lock.getandset(true) Art of Multiprocessor Programming 24

25 Review: Test-and-Set AtomicBoolean lock = new AtomicBoolean(false) boolean prior = lock.getandset(true) Swapping in true is called test-and-set or TAS. Both Swap and TAS available in hardware. Art of Multiprocessor Programming 25 (5)

26 Test-and-Set Locks Locking Lock is free: value is false Lock is taken: value is true Acquire lock by calling TAS If result is false, you win If result is true, you lose Release lock by writing false Art of Multiprocessor Programming 26

27 Simple TASLock TAS invalidates cache lines Spinners Miss in cache Go to bus Thread wants to release lock delayed behind spinners Art of Multiprocessor Programming 27

28 Test-and-Test-and-Set Locks Lurking stage Wait until lock looks free Spin while read returns true (lock taken) Pouncing state As soon as lock looks available Read returns false (lock free) Call TAS to acquire lock If TAS loses, back to lurking Art of Multiprocessor Programming 28

29 Test-and-test-and-set Lock class TTASlock { AtomicBoolean state = new AtomicBoolean(false); void lock() { while (true) { while (state.get()) {} if (!state.getandset(true)) return; } } Art of Multiprocessor Programming 29

30 Test-and-test-and-set Lock class TTASlock { AtomicBoolean state = new AtomicBoolean(false); void lock() { while (true) { while (state.get()) {} if (!state.getandset(true)) return; } } Wait until lock looks free Art of Multiprocessor Programming 30

31 Test-and-test-and-set Lock class TTASlock { AtomicBoolean state = new AtomicBoolean(false); void lock() { while (true) { while (state.get()) {} if (!state.getandset(true)) return; } } Then try to acquire it Art of Multiprocessor Programming 31

32 Test-and-test-and-set Wait until lock looks free Spin on local cache No bus use while lock busy Problem: when lock is released Invalidation storm Art of Multiprocessor Programming 32

33 Problems Everyone misses Reads satisfied sequentially Everyone does TAS Invalidates others caches Eventually quiesces after lock acquired Quiescence time often linear in number of cores Art of Multiprocessor Programming 33

34 Solution: Introduce Delay If the lock looks free - but I fail to get it There must be contention - better to back off than to collide again time r 2 d r 1 d d spin lock Art of Multiprocessor Programming 34

35 Dynamic Example: Exponential Backoff time 4d 2d d spin lock If I fail to get lock Wait random duration before retry Each subsequent failure doubles expected wait Art of Multiprocessor Programming 35

36 Exponential Backoff Lock public class Backoff implements lock { public void lock() { int delay = MIN_DELAY; while (true) { while (state.get()) {} if (!state.getandset(true)) return; sleep(random() % delay); if (delay < MAX_DELAY) delay = 2 * delay; }}} Art of Multiprocessor Programming 36

37 Exponential Backoff Lock public class Backoff implements lock { public void lock() { int delay = MIN_DELAY; while (true) { while (state.get()) {} if (!state.getandset(true)) return; sleep(random() % delay); if (delay < MAX_DELAY) delay = 2 * delay; }}} Fix minimum delay Art of Multiprocessor Programming 37

38 Exponential Backoff Lock public class Backoff implements lock { public void lock() { int delay = MIN_DELAY; while (true) { while (state.get()) {} if (!state.getandset(true)) return; sleep(random() % delay); if (delay < MAX_DELAY) delay = 2 * delay; }}} Wait until lock looks free Art of Multiprocessor Programming 38

39 Exponential Backoff Lock public class Backoff implements lock { public void lock() { int delay = MIN_DELAY; while (true) { while (state.get()) {} if (!state.getandset(true)) return; sleep(random() % delay); if (delay < MAX_DELAY) delay = 2 * delay; }}} If we win, return Art of Multiprocessor Programming 39

40 Exponential Backoff Lock public class Backoff implements lock { public void lock() { int delay = MIN_DELAY; while (true) { while (state.get()) {} if (!state.getandset(true)) return; sleep(random() % delay); if (delay < MAX_DELAY) delay = 2 * delay; }}} Back off for random duration Art of Multiprocessor Programming 40

41 Exponential Backoff Lock public class Backoff implements lock { public void lock() { int delay = MIN_DELAY; while (true) { while (state.get()) {} if (!state.getandset(true)) return; sleep(random() % delay); if (delay < MAX_DELAY) delay = 2 * delay; }}} Double max delay, within reason Art of Multiprocessor Programming 41

42 Actual Data on 40-Core Machine Art of Multiprocessor Programming 42

43 Backoff: Other Issues Good Easy to implement Beats TTAS lock Bad Must choose parameters carefully Not portable across platforms Art of Multiprocessor Programming 43

44 Idea Avoid useless invalidations By keeping a queue of threads Each thread Notifies next in line Without bothering the others Art of Multiprocessor Programming 44

45 CLH Lock First Come First Served order Small, constant-size overhead per thread Art of Multiprocessor Programming 45

46 Initially idle tail false Art of Multiprocessor Programming 46

47 Initially idle tail false Queue tail Art of Multiprocessor Programming 47

48 Initially idle tail false Lock is free Art of Multiprocessor Programming 48

49 Initially idle tail false Art of Multiprocessor Programming 49

50 Purple Wants the Lock acquiring tail false Art of Multiprocessor Programming 50

51 Purple Wants the Lock acquiring tail false true Art of Multiprocessor Programming 51

52 Purple Wants the Lock acquiring Swap tail false true Art of Multiprocessor Programming 52

53 Purple Has the Lock acquired tail false true Art of Multiprocessor Programming 53

54 Red Wants the Lock acquired acquiring tail false true true Art of Multiprocessor Programming 54

55 Red Wants the Lock acquired acquiring tail false Swap true true Art of Multiprocessor Programming 55

56 Red Wants the Lock acquired acquiring tail false true true Art of Multiprocessor Programming 56

57 Red Wants the Lock acquired acquiring tail false true true Art of Multiprocessor Programming 57

58 Red Wants the Lock acquired acquiring Implicit Linked list tail false true true Art of Multiprocessor Programming 58

59 Red Wants the Lock acquired acquiring tail false true true Art of Multiprocessor Programming 59

60 Red Wants the Lock acquired acquiring tail false true true true Actually, it spins on cached copy Art of Multiprocessor Programming 60

61 Purple Releases release acquiring false Bingo! tail false false true Art of Multiprocessor Programming 61

62 Purple Releases released acquired tail true Art of Multiprocessor Programming 62

63 CLH Queue Lock class Qnode { AtomicBoolean locked = } new AtomicBoolean(true); Art of Multiprocessor Programming 63

64 CLH Queue Lock class Qnode { AtomicBoolean locked = } new AtomicBoolean(true); Not released yet Art of Multiprocessor Programming 64

65 CLH Queue Lock class CLHLock implements Lock { AtomicReference<Qnode> tail; ThreadLocal<Qnode> mynode = new Qnode(); public void lock() { Qnode pred = tail.getandset(mynode); while (pred.locked) {} }} Art of Multiprocessor Programming 65

66 CLH Queue Lock class CLHLock implements Lock { AtomicReference<Qnode> tail; ThreadLocal<Qnode> mynode = new Qnode(); public void lock() { Qnode pred = tail.getandset(mynode); while (pred.locked) {} }} Queue tail Art of Multiprocessor Programming 66

67 CLH Queue Lock class CLHLock implements Lock { AtomicReference<Qnode> tail; ThreadLocal<Qnode> mynode = new Qnode(); public void lock() { Qnode pred = tail.getandset(mynode); while (pred.locked) {} }} Thread-local Qnode Art of Multiprocessor Programming 67

68 CLH Queue Lock class CLHLock implements Lock { AtomicReference<Qnode> tail; ThreadLocal<Qnode> mynode = new Qnode(); public void lock() { Qnode pred = tail.getandset(mynode); while (pred.locked) {} }} Swap in my node Art of Multiprocessor Programming 68

69 CLH Queue Lock class CLHLock implements Lock { AtomicReference<Qnode> tail; ThreadLocal<Qnode> mynode = new Qnode(); public void lock() { Qnode pred = tail.getandset(mynode); while (pred.locked) {} }} Spin until predecessor releases lock Art of Multiprocessor Programming 69

70 CLH Queue Lock Class CLHLock implements Lock { } } public void unlock() { mynode.locked.set(false); mynode = pred; Art of Multiprocessor Programming 70

71 CLH Queue Lock Class CLHLock implements Lock { } } public void unlock() { mynode.locked.set(false); mynode = pred; Notify successor Art of Multiprocessor Programming 71

72 CLH Queue Lock Class CLHLock implements Lock { } } public void unlock() { mynode.locked.set(false); mynode = pred; Recycle predecessor s node Art of Multiprocessor Programming 72

73 CLH Queue Lock Class CLHLock implements Lock { } } public void unlock() { mynode.locked.set(false); mynode = pred; (Here we don t actually reuse mynode. Can see how it s done in Art of Multiprocessor Programming book) Art of Multiprocessor Programming 73

74 CLH Lock Good Lock release affects predecessor only Small, constant-sized space Bad Doesn t work for uncached NUMA architectures Art of Multiprocessor Programming 74

75 NUMA and cc-numa Architectures Acronym: Non-Uniform Memory Architecture ccnuma = cache coherent NUMA Illusion: Flat shared memory Truth: No caches (sometimes) Some memory regions faster than others Art of Multiprocessor Programming 75

76 NUMA Machines Spinning on local memory is fast Art of Multiprocessor Programming 76

77 NUMA Machines Spinning on remote memory is slow Art of Multiprocessor Programming 77

78 CLH Lock Each thread spins on predecessor s memory Could be far away Art of Multiprocessor Programming 78

79 MCS Lock FCFS order Spin on local memory only Small, Constant-size overhead Art of Multiprocessor Programming 79

80 Initially idle tail false Art of Multiprocessor Programming 80

81 Acquiring acquiring (allocate Qnode) tail false true Art of Multiprocessor Programming 81

82 Acquiring acquired tail swap false true Art of Multiprocessor Programming 82

83 Acquiring acquired tail false true Art of Multiprocessor Programming 83

84 Acquired acquired tail false true Art of Multiprocessor Programming 84

85 Acquiring acquired acquiring tail swap false true Art of Multiprocessor Programming 85

86 Acquiring acquired acquiring tail false true Art of Multiprocessor Programming 86

87 Acquiring acquired acquiring tail false true Art of Multiprocessor Programming 87

88 Acquiring acquired acquiring tail false true Art of Multiprocessor Programming 88

89 Acquiring acquired acquiring tail true false true Art of Multiprocessor Programming 89

90 Acquiring acquired acquiring tail true Yes! false true Art of Multiprocessor Programming 90

91 MCS Queue Lock class Qnode { volatile boolean locked = false; volatile qnode next = null; } Art of Multiprocessor Programming 91

92 MCS Queue Lock class MCSLock implements Lock { AtomicReference tail; public void lock() { Qnode qnode = new Qnode(); Qnode pred = tail.getandset(qnode); if (pred!= null) { qnode.locked = true; pred.next = qnode; while (qnode.locked) {} }}} Art of Multiprocessor Programming 92

93 MCS Queue Lock class MCSLock implements Lock { AtomicReference tail; public void lock() { Qnode qnode = new Qnode(); Qnode pred = tail.getandset(qnode); if (pred!= null) { qnode.locked = true; pred.next = qnode; while (qnode.locked) {} }}} Make a QNode Art of Multiprocessor Programming 93

94 MCS Queue Lock class MCSLock implements Lock { AtomicReference tail; public void lock() { Qnode qnode = new Qnode(); Qnode pred = tail.getandset(qnode); if (pred!= null) { qnode.locked = true; pred.next = qnode; while (qnode.locked) {} }}} add my Node to the tail of queue Art of Multiprocessor Programming 94

95 MCS Queue Lock class MCSLock implements Lock { AtomicReference tail; public void lock() { Qnode qnode = new Qnode(); Qnode pred = tail.getandset(qnode); if (pred!= null) { qnode.locked = true; pred.next = qnode; while (qnode.locked) {} }}} Fix if queue was non-empty Art of Multiprocessor Programming 95

96 MCS Queue Lock class MCSLock implements Lock { AtomicReference tail; public void lock() { Qnode qnode = new Qnode(); Qnode pred = tail.getandset(qnode); if (pred!= null) { qnode.locked = true; pred.next = qnode; while (qnode.locked) {} }}} Wait until unlocked Art of Multiprocessor Programming 96

97 Purple Release releasing swap false false Art of Multiprocessor Programming 97

98 Purple Release releasing I don t see a successor. But by looking at the queue, I see another thread is active swap false false Art of Multiprocessor Programming 98

99 Purple Release releasing I don t see a successor. But by looking at the queue, I see another thread is active swap false false I have to release that thread so must wait for it to identify its node Art of Multiprocessor Programming 99

100 Purple Release releasing prepare to spin true false Art of Multiprocessor Programming 100

101 Purple Release releasing spinning true false Art of Multiprocessor Programming 101

102 Purple Release releasing spinning false true false Art of Multiprocessor Programming 102

103 Purple Release releasing Acquired lock false true false Art of Multiprocessor Programming 103

104 MCS Queue Unlock class MCSLock implements Lock { AtomicReference tail; public void unlock() { if (qnode.next == null) { if (tail.cas(qnode, null) return; while (qnode.next == null) {} } qnode.next.locked = false; }} Art of Multiprocessor Programming 104

105 MCS Queue Lock class MCSLock implements Lock { AtomicReference tail; public void unlock() { if (qnode.next == null) { if (tail.cas(qnode, null) return; while (qnode.next == null) {} } qnode.next.locked = false; }} Missing successor? Art of Multiprocessor Programming 105

106 MCS Queue Lock class MCSLock implements Lock { If really no successor, AtomicReference tail; public void unlock() { if (qnode.next == null) { if (tail.cas(qnode, null) return; while (qnode.next == null) {} } qnode.next.locked = false; }} return Art of Multiprocessor Programming 106

107 MCS Queue Lock class MCSLock implements Lock { Otherwise wait for AtomicReference tail; public void unlock() { if (qnode.next == null) { if (tail.cas(qnode, null) return; while (qnode.next == null) {} } qnode.next.locked = false; }} successor to catch up Art of Multiprocessor Programming 107

108 MCS Queue Lock class MCSLock implements Lock { AtomicReference queue; public void unlock() { if (qnode.next == null) { if (tail.cas(qnode, null) return; while (qnode.next == null) {} } qnode.next.locked = false; }} Pass lock to successor Art of Multiprocessor Programming 108

109 Abortable Locks What if you want to give up waiting for a lock? For example Timeout Database transaction aborted by user Art of Multiprocessor Programming 109

110 Back-off Lock Aborting is trivial Just return from lock() call Extra benefit: No cleaning up Wait-free Immediate return Art of Multiprocessor Programming 110

111 Queue Locks Can t just quit Thread in line behind will starve Need a graceful way out Art of Multiprocessor Programming 111

112 Queue Locks spinning spinning spinning true true true Art of Multiprocessor Programming 112

113 Queue Locks locked spinning spinning false true true Art of Multiprocessor Programming 113

114 Queue Locks locked spinning false true Art of Multiprocessor Programming 114

115 Queue Locks locked false Art of Multiprocessor Programming 115

116 Queue Locks spinning spinning spinning true true true Art of Multiprocessor Programming 116

117 Queue Locks spinning spinning true true true Art of Multiprocessor Programming 117

118 Queue Locks locked spinning false true true Art of Multiprocessor Programming 118

119 Queue Locks spinning false true Art of Multiprocessor Programming 119

120 Queue Locks pwned false true Art of Multiprocessor Programming 120

121 Abortable CLH Lock When a thread gives up Removing node in a wait-free way is hard Idea: let successor deal with it. Art of Multiprocessor Programming 121

122 Initially idle Pointer to predecessor (or null) tail A Art of Multiprocessor Programming 122

123 Initially idle Distinguished available node means lock is free tail A Art of Multiprocessor Programming 123

124 acquiring Acquiring tail A Art of Multiprocessor Programming 124

125 acquiring Acquiring Null predecessor means lock not available and not aborted A Art of Multiprocessor Programming 125

126 acquiring Acquiring Swap A Art of Multiprocessor Programming 126

127 acquiring Acquiring A Art of Multiprocessor Programming 127

128 Acquired locked Reference to AVAILABLE means lock is free. A Art of Multiprocessor Programming 128

129 Normal Case locked spinning spinning Null means lock is not free & request not aborted Art of Multiprocessor Programming 129

130 One Thread Aborts locked Timed out spinning Art of Multiprocessor Programming 130

131 Successor Notices locked Timed out spinning Non-Null means predecessor aborted Art of Multiprocessor Programming 131

132 Recycle Predecessor s Node locked spinning Art of Multiprocessor Programming 132

133 Spin on Earlier Node locked spinning Art of Multiprocessor Programming 133

134 Spin on Earlier Node released spinning A The lock is now mine Art of Multiprocessor Programming 134

135 Time-out Lock public class TOLock implements Lock { static Qnode AVAILABLE = new Qnode(); AtomicReference<Qnode> tail; ThreadLocal<Qnode> mynode; Art of Multiprocessor Programming 135

136 Time-out Lock public class TOLock implements Lock { static Qnode AVAILABLE = new Qnode(); AtomicReference<Qnode> tail; ThreadLocal<Qnode> mynode; AVAILABLE node signifies free lock Art of Multiprocessor Programming 136

137 Time-out Lock public class TOLock implements Lock { static Qnode AVAILABLE = new Qnode(); AtomicReference<Qnode> tail; ThreadLocal<Qnode> mynode; Tail of the queue Art of Multiprocessor Programming 137

138 Time-out Lock public class TOLock implements Lock { static Qnode AVAILABLE = new Qnode(); AtomicReference<Qnode> tail; ThreadLocal<Qnode> mynode; Remember my node Art of Multiprocessor Programming 138

139 Time-out Lock public boolean lock(long timeout) { Qnode qnode = new Qnode(); mynode.set(qnode); qnode.prev = null; Qnode mypred = tail.getandset(qnode); if (mypred== null } mypred.prev == AVAILABLE) { return true; Art of Multiprocessor Programming 139

140 Time-out Lock public boolean lock(long timeout) { Qnode qnode = new Qnode(); mynode.set(qnode); qnode.prev = null; Qnode mypred = tail.getandset(qnode); if (mypred == null } mypred.prev == AVAILABLE) { return true; Create & initialize node Art of Multiprocessor Programming 140

141 Time-out Lock public boolean lock(long timeout) { Qnode qnode = new Qnode(); mynode.set(qnode); qnode.prev = null; Qnode mypred = tail.getandset(qnode); if (mypred == null } mypred.prev == AVAILABLE) { return true; Swap with tail Art of Multiprocessor Programming 141

142 Time-out Lock public boolean lock(long timeout) { Qnode qnode = new Qnode(); mynode.set(qnode); qnode.prev = null; Qnode mypred = tail.getandset(qnode); if (mypred == null }... mypred.prev == AVAILABLE) { return true; If predecessor absent or released, we are done Art of Multiprocessor Programming 142

143 locked Time-out Lock spinning spinning long start = now(); while (now()- start < timeout) { } Qnode predpred = mypred.prev; if (predpred == AVAILABLE) { return true; } else if (predpred!= null) { mypred = predpred; } Art of Multiprocessor Programming 143

144 Time-out Lock long start = now(); while (now()- start < timeout) { } Qnode predpred = mypred.prev; if (predpred == AVAILABLE) { return true; } else if (predpred!= null) { mypred = predpred; } Keep trying for a while Art of Multiprocessor Programming 144

145 Time-out Lock long start = now(); while (now()- start < timeout) { } Qnode predpred = mypred.prev; if (predpred == AVAILABLE) { return true; } else if (predpred!= null) { mypred = predpred; } Spin on predecessor s prev field Art of Multiprocessor Programming 145

146 Time-out Lock long start = now(); while (now()- start < timeout) { } Qnode predpred = mypred.prev; if (predpred == AVAILABLE) { return true; } else if (predpred!= null) { mypred = predpred; } Predecessor released lock Art of Multiprocessor Programming 146

147 Time-out Lock long start = now(); while (now()- start < timeout) { } Qnode predpred = mypred.prev; if (predpred == AVAILABLE) { return true; } else if (predpred!= null) { mypred = predpred; } Predecessor aborted, advance one Art of Multiprocessor Programming 147

148 Time-out Lock if (!tail.compareandset(qnode, mypred)) qnode.prev = mypred; } } return false; What do I do when I time out? Art of Multiprocessor Programming 148

149 Time-out Lock if (!tail.compareandset(qnode, mypred)) qnode.prev = mypred; } } return false; Do I have a successor? If CAS succeeds: no successor, tail just set to my pred, simply return false Art of Multiprocessor Programming 149

150 Time-out Lock if (!tail.compareandset(qnode, mypred)) qnode.prev = mypred; } } return false; If CAS fails, I do have a successor. Tell it about mypred Art of Multiprocessor Programming 150

151 Time-Out Unlock public void unlock() { } Qnode qnode = mynode.get(); if (!tail.compareandset(qnode, null)) qnode.prev = AVAILABLE; Art of Multiprocessor Programming 151

152 Time-out Unlock public void unlock() { } Qnode qnode = mynode.get(); if (!tail.compareandset(qnode, null)) qnode.prev = AVAILABLE; If CAS failed: successor exists, notify it can enter Art of Multiprocessor Programming 152

153 Timing-out Lock public void unlock() { } Qnode qnode = mynode.get(); if (!tail.compareandset(qnode, null)) qnode.prev = AVAILABLE; CAS successful: set tail to null, no clean up since no successor waiting Art of Multiprocessor Programming 153

154 Fairness and NUMA Locks MCS lock mechanics are aware of NUMA Lock Fairness is FCFS Is this a good fit with NUMA and Cache-Coherent NUMA machines?

155 Lock Data Access in NUMA Machine Node 1 CS MCS lock various memory locations Node 2

156 Who s the Unfairest of Them All? locality crucial to NUMA performance Big gains if threads from same node/ cluster obtain lock consecutively Unfairness pays

157 Hierarchical Backoff Lock (HBO) Back off less for thread from same node time 4d 2d d CS Unfairness is key to performance Global T&T&S lock time 4d 2d d

158 Hierarchical Backoff Lock (HBO) Advantages: Simple, improves locality Disadvantages: Requires platform specific tuning Unstable Unfair Continuous invalidations on shared global lock word

159 Hierarchical CLH Lock (HCLH) Each thread spins on cached copy of predecessor s node Local Tail Thread at local head splices local queue into global queue Local CLH queue Global Tail CAS() CAS() Local Tail Local CLH queue CS CAS()

160 Hierarchical CLH Lock (HCLH) 7e+06 6e+06 hbo hclh Throughput 5e+06 4e+06 3e+06 2e+06 1e+06 HCLH HBO Number of Threads Threads access 4 cache lines in CS

161 Hierarchical CLH Lock (HCLH) Advantages: Improved locality Local spinning Fair Disadvantages: Complex code implies long common path Splicing into both local and global requires CAS Hard to get long local sequences

162

163 Lock Cohorting General technique for converting almost any lock into a NUMA lock Allows combining different lock types But need these locks to have certain properties (will discuss shortly)

164 Lock Cohorting Non-empty cohort empty cohort Acquire local lock and proceed to critical section Local Lock On release: if nonempty cohort of waiting threads, release only local lock; leave mark Thread that acquired local lock can now acquire global lock Local Lock CS Global Lock On release: since cohort is empty must release global lock to avoid deadlock

165 Thread Obliviousness A lock is thread-oblivious if After being acquired by one thread, Can be released by another Art of Multiprocessor Programming 165

166 Cohort Detection A lock provides cohort detection if It can tell whether any thread is trying to acquire it Art of Multiprocessor Programming 166

167 Lock Cohorting Two levels of locking Global lock: thread oblivious Thread acquiring the lock can be different than one releasing it Local lock: cohort detection Thread releasing can detect if some thread is waiting to acquire it

168 Two new states: acquire local and acquire global. Do we own global lock? Lock Cohorting: C-BO-MCS In Lock, cohort Lock Local MCS lock tail detection by checking successor pointer CAS() False False True Global backoff lock Bound number of Local consecutive MCS lock acquires to control tail unfairness time 4d 2d d CS CAS() False False True BO Lock is thread oblivious by definition

169 How to add cohort detection Lock Cohorting: property to BO lock? C-BO-BO Lock 4d 2d d Global backoff lock CS time 4d 2d d 4d 2d d As noted BO Lock is thread oblivious

170 Write successorexists Lock Cohorting: C-BO-BO Lock field before attempting to acquire local lock. successorexists reset on lock release. 4d 2d d Release might overwrite another successor s Global write but we don t backoff care why? lock CS time 4d 2d d 4d 2d d

171 C-BO-BO Aborting thread is a resets Time-Out successorexists field before leaving local lock. Spinning threads set it to true. NUMA Lock 4d 2d d BO locks trivially abortable Global backoff lock If releasing thread finds successorexists time false, 4d it releases global lock 2d d CS 4d 2d d

172 Time-Out NUMA Lock Local time-out queue Local Time-Out locks have cohort detection property why? Not enough swap() Local time-out queue time 4d 2d d CS swap()

173 Lock Cohorting Advantages: Great locality Low contention on shared lock Practically no tuning Has whatever properties you want: Can be more or less fair, abortable just choose the appropriate type of locks Disadvantages: Must tune fairness parameters

174 Lock Cohorting Throughput 7e+06 6e+06 5e+06 4e+06 3e+06 C-BO-MCS C-BO-BO hbo hclh fc-mcs nmcs nclh nbo nticket HCLH 2e+06 1e+06 HBO Number of Threads

175 Time-Out (Abortable) Lock Cohorting A-BO-CLH (time-out lock + BO) Throughput in CR-nCRs per sec 4e e+06 3e e+06 2e e+06 1e A-BO-BO a-clh a-hbo a-bo-bo a-bo-clh (CAS) a-bo-clh Number of Threads Abortable CLH (our time-out lock) and HBO

176 One Lock To Rule Them All? TTAS+Backoff, CLH, MCS, ToLock Each better than others in some way There is no one solution Lock we pick really depends on: the application the hardware which properties are important Art of Multiprocessor Programming 176

177 Yeahy! Amdahl s Law Works Fine grained parallelism gives great performance benefit Coarse Grained Fine Grained 25% Shared 25% Shared 75% Unshared 75% Unshared

178 But Can we always draw the right conclusions from Amdahl s law? Claim: sometimes the overhead of finegrained synchronization is so high that it is better to have a single thread do all the work sequentially in order to avoid it

179 object Oyama et. al Mutex object lock CAS() Head Improves cache behavior and reduces contention on object Not great since every request involves CAS Release lock CAS() a b c d Apply a,b,c, and d to object Combine lock requests return responses

180 Flat Combining Have single lock holder collect and perform requests of all others Without using CAS operations to coordinate requests With (non-naïve) combining of requests (if cost of k batched operations is less than that of k operations in sequence à we win)

181 Flat-Combining Most requests do not involve a CAS, in fact, not even a memory barrier object counter 54 object lock Collect Again try requests to collect requests Head CAS() Publication list Deq() 53 Deq() null 54 Enq(d) null 12 Enq(d) 54 Apply requests to object Deq() Deq() Enq(d)

182 Flat-Combining Pub-List object counter 54 Cleanup Every combiner increments counter and updates record s time stamp when returning response Traverse and remove from list records with old time stamp Cleanup requires no CAS, only reads and writes Head Publication list Deq() 53 null 12 Enq(d) 54 Enq(d) 54 Enq(d) If thread reappears must add itself to pub list

183 Fine-Grained Lock-free FIFO Queue Head CAS() Tail CAS() a CAS() b c d P: Dequeue() => a Q: Enqueue(d)

184 Flat-Combining FIFO Queue counter object lock 54 Head CAS() Publication list Head Tail null 54 Enq(b) 12 Enq(b) 54 Deq() Enq(a) Enq(b) Sequential FIFO Queue

185 Flat-Combining FIFO Queue OK, but can do better combining: collect all items into a fat node, enqueue Fat in Node one step easy sequentially but object cannot counter lock be done in concurrent alg without CAS 54 Head Tail Head CAS() Publication list Deq() 54 Enq(b) 12 Enq(b) 54 a Deq() Enq(a) Enq(b) Sequential Fat Node FIFO Queue

186 Linearizable FIFO Queue Flat Combining MS queue, Oyama Combining tree

187 Benefit s of Flat Combining Flat Combining in Red

188 Linearizable Stack Flat Combining A Bug Elimination Stack Treiber Lockfree Stack

189 Concurrent Priority Queue (Chapter 15) k deletemin opera.ons take O(k log n) deletemin() traverses CASing un.l you manage to mark a node, then use skiplist remove your marked node 1

190 Flat-Combining Priority Queue counter object lock 54 Head CAS() Publica.on list cvccc Deq() 54 Enq(b) 12 Enq(b) 54 Deq() Enq(a) Enq(b)

191 Flat Combining Priority Queue k deletemin opera.ons take O(k+log n) Traverse Collect skiplist k deletemin towards requests kth key, removing all nodes below your path Remove traverse to find kth key, collect values to be returned

192 Flat Combining Skiplist based queue Whats this? Priority Queue lock-based SkipQueue lock-free SkipQueue

193 Priority Queue Flat combining with sequen.al pairing heap plugged in

194 Priority Queue on Intel Flat combining with sequen.al pairing heap plugged in

195 Don t be Afraid of the Big Bad Lock Fine grained parallelism comes with an overhead not always worth the effort. Sometimes using a single global lock is a win. Art of Multiprocessor Programming 195

196 Thanks! Art of Multiprocessor Programming 196

197 This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License. You are free: to Share to copy, distribute and transmit the work to Remix to adapt the work Under the following conditions: Attribution. You must attribute the work to The Art of Multiprocessor Programming (but not in any way that suggests that the authors endorse you or your use of the work). Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights. Art of Multiprocessor Programming 197