Programming Paradigms for Concurrency Lecture 6 Synchronization of Concurrent Objects

Transcription

1 Programming Paradigms for Concurrency Lecture 6 Synchronization of Concurrent Objects Based on companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Modified by Thomas Wies New York University

2 Last Two Lectures: Synchronization Primitives CS. spin lock critical section Resets lock upon exit 2

3 Today: Concurrent Objects Adding threads should not lower throughput Contention effects Mostly fixed by Queue locks 3

4 Today: Concurrent Objects Adding threads should not lower throughput Contention effects Mostly fixed by Queue locks Should increase throughput Not possible if inherently sequential Surprising things are parallelizable 4

5 Coarse-Grained Synchronization Each method locks the object Avoid contention using queue locks 5

6 Coarse-Grained Synchronization Each method locks the object Avoid contention using queue locks Easy to reason about In simple cases 6

7 Coarse-Grained Synchronization Each method locks the object Avoid contention using queue locks Easy to reason about In simple cases So, are we done? 7

8 Coarse-Grained Synchronization Sequential bottleneck Threads stand in line 8

9 Coarse-Grained Synchronization Sequential bottleneck Threads stand in line Adding more threads Does not improve throughput Struggle to keep it from getting worse 9

10 Coarse-Grained Synchronization Sequential bottleneck Threads stand in line Adding more threads Does not improve throughput Struggle to keep it from getting worse So why even use a multiprocessor? Well, some apps inherently parallel 10

11 This Lecture Introduce four patterns Bag of tricks Methods that work more than once 11

12 This Lecture Introduce four patterns Bag of tricks Methods that work more than once For highly-concurrent objects Concurrent access More threads, more throughput 12

13 First: Fine-Grained Synchronization Instead of using a single lock Split object into Independently-synchronized components Methods conflict when they access The same component At the same time 13

14 Second: Optimistic Synchronization Search without locking 14

15 Second: Optimistic Synchronization Search without locking If you find it, lock and check OK: we are done Oops: start over 15

16 Second: Optimistic Synchronization Search without locking If you find it, lock and check OK: we are done Oops: start over Evaluation Usually cheaper than locking, but Mistakes are expensive 16

17 Third: Lazy Synchronization Postpone hard work Removing components is tricky Logical removal Mark component to be deleted Physical removal Do what needs to be done 17

18 Fourth: Lock-Free Synchronization Don t use locks at all Use compareandset() & relatives 18

19 Fourth: Lock-Free Synchronization Don t use locks at all Use compareandset() & relatives Advantages No Scheduler Assumptions/Support 19

20 Fourth: Lock-Free Synchronization Don t use locks at all Use compareandset() & relatives Advantages No Scheduler Assumptions/Support Disadvantages Complex Sometimes high overhead 20

21 Linked List Illustrate these patterns Using a list-based Set Common application Building block for other apps 21

22 Set Interface Unordered collection of items 22

23 Set Interface Unordered collection of items No duplicates 23

24 Set Interface Unordered collection of items No duplicates Methods add(x) put x in set remove(x) take x out of set contains(x) tests if x in set 24

25 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } 25

26 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } Add item to set 26

27 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(tt x); } Remove item from set 27

28 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } Is item in set? 28

29 List Node public class Node { public T item; public int key; public Node next; } 29

30 List Node public class Node { public T item; public int key; public Node next; } item of interest 30

31 List Node public class Node { public T item; public int key; public Node next; } Usually hash code 31

32 List Node public class Node { public T item; public int key; public Node next; } Reference to next node 32

33 The List-Based Set - a b c + Sorted with Sentinel nodes (min & max possible keys) 33

34 Reasoning about Concurrent Objects Invariant Property that always holds 34

35 Reasoning about Concurrent Objects Invariant Property that always holds Established because True when object is created Truth preserved by each method Each step of each method 35

36 Specifically Invariants preserved by add() remove() contains() 36

37 Specifically Invariants preserved by add() remove() contains() Most steps are trivial Usually one step tricky Often linearization point 37

38 Interference Invariants make sense only if methods considered are the only modifiers 38

39 Interference Invariants make sense only if methods considered are the only modifiers Language encapsulation helps List nodes not visible outside class 39

40 Interference Freedom from interference needed even for removed nodes Some algorithms traverse removed nodes Careful with malloc() & free()! Garbage collection helps here 40

41 Abstract Data Types Concrete representation: a b Abstract Type: {a, b} 41

42 Abstract Data Types Meaning of representation given by abstraction map S( a b ) ) = {a,b} 42

43 Rep Invariant Which concrete values meaningful? Sorted? Duplicates? Rep invariant Characterizes legal concrete reps Preserved by methods Relied on by methods 43

44 Blame Game Rep invariant is a contract Suppose add() leaves behind 2 copies of x remove() removes only 1 Which is incorrect? 44

45 Blame Game Suppose add() leaves behind 2 copies of x remove() removes only 1 45

46 Blame Game Suppose add() leaves behind 2 copies of x remove() removes only 1 Which is incorrect? If rep invariant says no duplicates add() is incorrect Otherwise remove() is incorrect 46

47 Rep Invariant (partly) Sentinel nodes tail reachable from head Sorted No duplicates 47

48 Abstraction Map S(head) = { x there exists a such that } a reachable from head and a.item = x 48

49 Sequential List Based Set add() a c d remove() a b c 49

50 Sequential List Based Set add() a c d remove() b a b c 50

51 Coarse-Grained Locking a b d 51

52 Coarse-Grained Locking a b d c 52

53 Coarse-Grained Locking a b d honk! honk! c Simple but hotspot + bottleneck 53

54 Coarse-Grained Locking Easy, same as synchronized methods One lock to rule them all 54

55 Coarse-Grained Locking Easy, same as synchronized methods One lock to rule them all Simple, clearly correct Deserves respect! Works poorly with contention Queue locks help But bottleneck still an issue 55

56 Fine-grained Locking Requires careful thought Do not meddle in the affairs of wizards, for they are subtle and quick to anger 56

57 Fine-grained Locking Requires careful thought Do not meddle in the affairs of wizards, for they are subtle and quick to anger Split object into pieces Each piece has own lock Methods that work on disjoint pieces need not exclude each other 57

58 Hand-over-Hand locking a b c 58

59 Hand-over-Hand locking a b c 59

60 Hand-over-Hand locking a b c 60

61 Hand-over-Hand locking a b c 61

62 Hand-over-Hand locking a b c 62

63 Removing a Node a b c d remove(b) 63

64 Removing a Node a b c d remove(b) 64

65 Removing a Node a b c d remove(b) 65

66 Removing a Node a b c d remove(b) 66

67 Removing a Node a b c d remove(b) 67

68 Removing a Node a c d remove(b) Why hold 2 locks? 68

69 Concurrent Removes a b c d remove(b) remove(c) 69

70 Concurrent Removes a b c d remove(b) remove(c) 70

71 Concurrent Removes a b c d remove(b) remove(c) 71

72 Concurrent Removes a b c d remove(b) remove(c) 72

73 Concurrent Removes a b c d remove(b) remove(c) 73

74 Concurrent Removes a b c d remove(b) remove(c) 74

75 Concurrent Removes a b c d remove(b) remove(c) 75

76 Concurrent Removes a b c d remove(b) remove(c) 76

77 Concurrent Removes a b c d remove(b) remove(c) 77

78 Uh, Oh a c d remove(b) remove(c) 78

79 Uh, Oh Bad news, c not removed a c d remove(b) remove(c) 79

80 Problem To delete node c Swing node b s next field to d a b c Problem is, Someone deleting b concurrently could direct a pointer a b c to c 80

81 Insight If a node is locked, no one can delete node s successor If a thread locks node to be deleted and its predecessor, then it works 81

82 Hand-Over-Hand Again a b c d remove(b) 82

83 Hand-Over-Hand Again a b c d remove(b) 83

84 Hand-Over-Hand Again a b c d remove(b) 84

85 Hand-Over-Hand Again a b c d remove(b) Found it! 85

86 Hand-Over-Hand Again a b c d remove(b) Found it! 86

87 Hand-Over-Hand Again a c d remove(b) 87

88 Removing a Node a b c d remove(b) remove(c) 88

89 Removing a Node a b c d remove(b) remove(c) 89

90 Removing a Node a b c d remove(b) remove(c) 90

91 Removing a Node a b c d remove(b) remove(c) 91

92 Removing a Node a b c d remove(b) remove(c) 92

93 Removing a Node a b c d remove(b) remove(c) 93

94 Removing a Node a b c d remove(b) remove(c) 94

95 Removing a Node a b c d remove(b) remove(c) 95

96 Removing a Node a b c d Must acquire Lock for b remove(c) 96

97 Removing a Node a b c d Cannot acquire lock for b remove(c) 97

98 Removing a Node a b c d Wait! remove(c) 98

99 Removing a Node a b d Proceed to remove(b) 99

100 Removing a Node a b d remove(b) 100

101 Removing a Node a b d remove(b) 101

102 Removing a Node a d remove(b) 102

103 Removing a Node a d 103

104 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; try { } finally { curr.unlock(); pred.unlock(); }} 104

105 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; try { } finally { curr.unlock(); pred.unlock(); }} Key used to order node 105

106 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; try { } finally { currnode.unlock(); prednode.unlock(); }} Predecessor and current nodes 106

107 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; try { } finally { curr.unlock(); pred.unlock(); }} Make sure locks released 107

108 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; try { } finally { curr.unlock(); pred.unlock(); }} Everything else 108

109 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } 109

110 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } lock pred == head 110

111 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } Lock current 111

112 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } Traversing list 112

113 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; 113

114 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } Search key range pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; 114

115 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; At start of each loop: curr and pred locked 115

116 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; If item found, remove node 116

117 Remove: searching Unlock predecessor while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; 117

118 Remove: searching Only one node locked! while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; 118

119 Remove: searching while (curr.key <= key) { demote current if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; 119

120 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = currnode; curr = curr.next; curr.lock(); } return false; Find and lock new current 120

121 Remove: searching while (curr.key <= key) { if Lock (item invariant == curr.item) restored { pred.next = curr.next; return true; } pred.unlock(); pred = currnode; curr = curr.next; curr.lock(); } return false; 121

122 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Otherwise, not present 122

123 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; pred reachable from head curr is pred.next So curr.item is in the set 124

124 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linearization point if item is present 125

125 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Node locked, so no other thread can remove it. 126

126 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Item not present 127

127 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; pred reachable from head curr.lock(); curr is pred.next } pred.key < key return false; key < curr.key 128

128 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linearization point 129

129 Adding Nodes To add node e Must lock predecessor Must lock successor Neither can be deleted (Is successor lock actually required?) 130

130 Same Abstraction Map S(head) = { x there exists a such that } a reachable from head and a.item = x 131

131 Rep Invariant Easy to check that tail always reachable from head Nodes sorted, no duplicates 132

132 Drawbacks Better than coarse-grained lock Threads can traverse in parallel Still not ideal Long chain of acquire/release Inefficient 133

133 Optimistic Synchronization Find nodes without locking Lock nodes Check that everything is OK 134

134 Optimistic: Traverse without Locking a b d e add(c) Aha! 135

135 Optimistic: Lock and Load a b d e add(c) 136

136 Optimistic: Lock and Load c a b d e add(c) 137

137 What could go wrong? a b d e add(c) Aha! 138

138 What could go wrong? a b d e add(c) 139

139 What could go wrong? a b d e remove(b) 140

140 What could go wrong? a b d e remove(b) 141

141 What could go wrong? a b d e add(c) 142

142 What could go wrong? c a b d e add(c) 143

143 What could go wrong? a d e add(c) Uh-oh 144

144 Validate Part 1 a b d e add(c) Yes, b still reachable from head 145

145 What Else Could Go Wrong? a b d e add(c) Aha! 146

146 What Else Coould Go Wrong? a b d e add(c) add(b ) 147

147 What Else Coould Go Wrong? a b d e add(c) b add(b ) 148

148 What Else Could Go Wrong? a b d e add(c) b 149

149 What Else Could Go Wrong? c a b d e add(c) 150

150 Validate Part 2 (while holding locks) a b d e add(c) Yes, b still points to d 151

151 Optimistic: Linearization Point a b d e add(c) c 152

152 Same Abstraction Map S(head) = { x there exists a such that } a reachable from head and a.item = x 153

153 Invariants Careful: we may traverse deleted nodes But we establish properties by Validation After we lock target nodes 154

154 Correctness If nodes b and c both locked node b still accessible node c still successor to b Then neither will be deleted OK to delete and return true 155

155 Unsuccessful Remove a b d e remove(c) Aha! 156

156 Validate (1) a b d e remove(c) Yes, b still reachable from head 157

157 Validate (2) a b d e remove(c) Yes, b still points to d 158

158 OK Computer a b d e remove(c) return false 159

159 Correctness If nodes b and d both locked node b still accessible node d still successor to b Then neither will be deleted no thread can add c after b OK to return false 160

160 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } 161

161 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } Predecessor & return false; } current nodes 162

162 private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Begin at the beginning 163

163 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Search range of keys 164

164 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Predecessor reachable 165

165 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Is current node next? 166

166 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Otherwise move on 167

167 Validation Predecessor not reachable private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } 168

168 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } 169

169 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } Search key 170

170 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } Retry on synchronization conflict 171

171 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } Examine predecessor and current nodes 172

172 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; Search by key } 173

173 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } Stop if we find item 174

174 Remove: searching public boolean remove(item item) { int key Move = along item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } 175

175 On Exit from Loop If item is present curr holds item pred just before curr If item is absent curr has first higher key pred just before curr Assuming no synchronization problems 176

176 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr)) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} 177

177 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr)) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Always unlock 178

178 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr)) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Lock both nodes 179

179 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr)) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Check for synchronization conflicts 180

180 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr)) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} target found, remove node 181

181 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} target not found 182

182 Optimistic List Limited hot-spots Targets of add(), remove(), contains() No contention on traversals Moreover Traversals are wait-free Food for thought 183

183 So Far, So Good Much less lock acquisition/release Performance Concurrency Problems Need to traverse list twice contains() method acquires locks 184

184 Evaluation Optimistic is effective if cost of scanning twice without locks is less than cost of scanning once with locks Drawback contains() acquires locks 90% of calls in many apps 185

185 Lazy List Like optimistic, except Scan once contains(x) never locks Key insight Removing nodes causes trouble Do it lazily 186

186 Lazy List remove() Scans list (as before) Locks predecessor & current (as before) Logical delete Marks current node as removed (new!) Physical delete Redirects predecessor s next (as before) 187

187 Lazy Removal a b c d 188

188 Lazy Removal a b c d Present in list 189

189 Lazy Removal a b c d Logically deleted 190

190 Lazy Removal a b c d Physically deleted 191

191 Lazy Removal a b d Physically deleted 192

192 Lazy List All Methods Scan through locked and marked nodes Removing a node doesn t slow down other method calls Must still lock pred and curr nodes. 193

193 Validation No need to rescan list! Check that pred is not marked Check that curr is not marked Check that pred points to curr 194

194 Business as Usual a b c 195

195 Business as Usual a b c 196

196 Business as Usual a b c 197

197 Business as Usual a b c remove(b) 198

198 Business as Usual a b c a not marked 199

199 Business as Usual a b c a still points to b 200

200 Business as Usual a b c Logical delete 201

201 Business as Usual a b c physical delete 202

202 Business as Usual a b c 203

203 New Abstraction Map S(head) = { x there exists node a such that } a reachable from head and a.item = x and a is unmarked 204

204 Invariant If not marked, then item in the set and reachable from head and if not yet traversed, it is reachable from pred 205

205 Validation private boolean validate(node pred, Node curr) { return!pred.marked &&!curr.marked && pred.next == curr); } 206

206 List Validate Method private boolean validate(node pred, Node curr) { return!pred.marked &&!curr.marked && pred.next == curr); } Predecessor not Logically removed 207

207 List Validate Method private boolean validate(node pred, Node curr) { return!pred.marked &&!curr.marked && pred.next == curr); } Current not Logically removed 208

208 List Validate Method private boolean validate(node pred, Node curr) { return!pred.marked &&!curr.marked && pred.next == curr); } Predecessor still Points to current 209

209 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Art of Multiprocessor Programming 210

210 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Validate as before Art of Multiprocessor Programming 211

211 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Key found Art of Multiprocessor Programming 212

212 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Logical remove Art of Multiprocessor Programming 213

213 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} physical remove Art of Multiprocessor Programming 214

214 Contains public boolean contains(item item) { int key = item.hashcode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key &&!curr.marked; } 215

215 Contains public boolean contains(item item) { int key = item.hashcode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key &&!curr.marked; } Start at the head 216

216 Contains public boolean contains(item item) { int key = item.hashcode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key &&!curr.marked; } Search key range 217

217 Contains public boolean contains(item item) { int key = item.hashcode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key &&!curr.marked; } Traverse without locking (nodes may have been removed) 218

218 Contains public boolean contains(item item) { int key = item.hashcode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key &&!curr.marked; } Present and undeleted? 219

219 Summary: Wait-free Contains a 0 b 0 d c 10 e 0 Use Mark bit + list ordering 1. Not marked in the set 2. Marked or missing not in the set 220

220 Lazy List a 0 b 0 d c 10 e 0 Lazy add() and remove() + Wait-free contains() 221

221 Evaluation Good: contains() doesn t lock In fact, its wait-free! Good because typically high % contains() Uncontended calls don t re-traverse Bad Contended add() and remove() calls do retraverse Traffic jam if one thread delays 222

222 Traffic Jam Any concurrent data structure based on mutual exclusion has a weakness If one thread Enters critical section And eats the big muffin Cache miss, page fault, descheduled Everyone else using that lock is stuck! Need to trust the scheduler. 223

223 Reminder: Lock-Free Data No matter what Structures Guarantees minimal progress in any execution i.e. Some thread will always complete a method call Even if others halt at malicious times Implies that implementation can t use locks 224

224 Lock-free Lists Next logical step Wait-free contains() lock-free add() and remove() Use only compareandset() What could go wrong? 225

225 Lock-free Lists Logical Removal a 0 b 0 c 10 e 0 Use CAS to verify pointer is correct Physical Removal Not enough! 226

226 Problem Logical Removal a 0 b 0 c 10 e 0 Physical Removal d 0 Node added 227

227 The Solution: Combine Bit and Pointer Logical Removal = Set Mark Bit a 0 b 0 c 10 e 0 Mark-Bit and Pointer are CASed together (AtomicMarkableReference) Physical Removal CAS d 0 Fail CAS: Node not added after logical Removal 228

228 Solution Use AtomicMarkableReference Atomically Swing reference and Update flag Remove in two steps Set mark bit in next field Redirect predecessor s pointer 229

229 Marking a Node AtomicMarkableReference class Java.util.concurrent.atomic package Reference address F mark bit 230

230 Extracting Reference & Mark public Object get(boolean[] marked); 231

231 Extracting Reference & Mark public Object get(boolean[] marked); Returns reference Returns mark at array index 0! 232

232 Extracting Mark Only public boolean ismarked(); Value of mark 233

233 Changing State public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); 234

234 Changing State If this is the current reference public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); And this is the current mark 235

235 Changing State then change to this new reference public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); and this new mark 236

236 Changing State public boolean attemptmark( Object expectedref, boolean updatemark); 237

237 Changing State public boolean attemptmark( Object expectedref, boolean updatemark); If this is the current reference 238

238 Changing State public boolean attemptmark( Object expectedref, boolean updatemark);.. then change to this new mark. 239

239 Removing a Node a b CAS c d remove c 240

240 Removing a Node failed a CAS b CAS c d remove b remove c 241

241 Removing a Node a b c d remove b remove c 242

242 Removing a Node a d remove b remove c 243

243 Traversing the List Q: what do you do when you find a logically deleted node in your path? A: finish the job. CAS the predecessor s next field Proceed (repeat as needed) 244

244 Lock-Free Traversal (only Add and Remove) pred pred curr curr CAS a b c d Uh-oh 245

245 The Window Class class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; } } 246

246 The Window Class class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; } } A container for pred and current values 247

247 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; 248

248 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Find returns window 249

249 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Extract pred and curr 250

250 The Find Method Window window = find(item); At some instant, item or pred curr succ 251

251 The Find Method Window window = find(item); At some instant, item not in list pred curr= null succ 252

252 Remove public boolean remove(t item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset(succ, succ, false true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; }}} 253

253 Remove public boolean remove(t item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset (succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; }}} Keep trying 254

254 Remove public boolean remove(t item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset (succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; }}} Find neighbors 255

255 Remove public boolean remove(t item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset(succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; }}} She s not there 256

256 Remove public boolean remove(t item) { Boolean snip; Try to mark node as deleted while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset(succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; }}} 257

257 Remove public boolean remove(t item) { Boolean If it doesn t snip; work, while (true) { just retry, if it Window window = find(head, key); Node does, pred = job window.pred, curr = window.curr; essentially if (curr.key done!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset(succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; }}} 258

258 Remove public boolean remove(t item) { Boolean snip; while (true) { a Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; Try to advance reference } else { Node (if we succ don t = curr.next.getreference(); succeed, someone else did or will). snip = curr.next.compareandset(succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; }}} 259

259 Add public boolean add(t item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareandset(curr, node, false, false)) { return true; } }}} 260

260 Add public boolean add(t item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareandset(curr, node, false, false)) {return true;} }}} Item already there. 261

261 Add public boolean add(t item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareandset(curr, node, false, false)) {return true;} }}} create new node 262

262 Add public boolean add(t item) { boolean splice; Install new node, while (true) { Window window = find(head, key); else retry loop Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareandset(curr, node, false, false)) {return true;} }}} 263

263 Wait-free Contains public boolean contains(t item) { boolean marked; int key = item.hashcode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key &&!marked[0]) } 264

264 Wait-free Contains public boolean contains(t item) { boolean marked; Only diff is that we int key = item.hashcode(); get and check Node curr = this.head; marked while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key &&!marked[0]) } 265

265 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} 266

266 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { } }} succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; If list changes while traversed, start over 267

267 Lock-free Find public Window find(node head, int key) { Node pred = null, Start curr looking = null, from succ = head null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} 268

268 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { Move down the list pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} 269

269 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); } }} pred = curr; curr = succ; Get ref to successor and current deleted bit 270

270 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; Try to remove curr = succ; deleted nodes in } }} path code details soon 271

271 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { } }} If curr key that is greater or equal, return pred and curr succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; 272

272 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); Otherwise advance window and } }} while (marked[0]) { loop again } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; 273

273 Lock-free Find retry: while (true) { while (marked[0]) { snip = pred.next.compareandset(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } 274

274 Lock-free Find Try to snip out node retry: while (true) { while (marked[0]) { snip = pred.next.compareandset(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } 275

275 Lock-free Find if predecessor s next field changed, retry whole traversal retry: while (true) { while (marked[0]) { snip = pred.next.compareandset(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } 276

276 Lock-free Find Otherwise move on to check if next node deleted retry: while (true) { while (marked[0]) { snip = pred.next.compareandset(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } 277

277 Performance Different list-based set implementations 16-node machine Vary percentage of contains() calls 278

278 High Contains Ratio Lock-free Lazy list Coarse Grained Fine Lock-coupling 279

279 Low Contains Ratio Lock-free Lazy list Coarse Grained Fine Lock-coupling 280

280 As Contains Ratio Increases Lock-free Lazy list Coarse Grained Fine Lock-coupling % Contains() 281

281 Summary Coarse-grained locking Fine-grained locking Optimistic synchronization Lazy synchronization Lock-free synchronization 282

282 To Lock or Not to Lock Locking vs. Non-blocking: Extremist views on both sides The answer: nobler to compromise Example: Lazy list combines blocking add() and remove()and a wait-free contains() Remember: Blocking/non-blocking is a property of a method 283

283 This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License. You are free: to Share to copy, distribute and transmit the work to Remix to adapt the work Under the following conditions: Attribution. You must attribute the work to The Art of Multiprocessor Programming (but not in any way that suggests that the authors endorse you or your use of the work). Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights. 284