Linked Lists: Locking, Lock- Free, and Beyond. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

Transcription

1 Linked Lists: Locking, Lock- Free, and Beyond Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

2 Last Lecture: Spin-Locks CS. spin lock critical section Resets lock upon exit Art of Multiprocessor Programming 2

3 Today: Concurrent Objects Adding threads should not lower throughput Contention effects Mostly fixed by Queue locks Art of Multiprocessor Programming 3

4 Today: Concurrent Objects Adding threads should not lower throughput Contention effects Mostly fixed by Queue locks Should increase throughput Not possible if inherently sequential Surprising things are parallelizable Art of Multiprocessor Programming 4

5 Coarse-Grained Synchronization Each method locks the object Avoid contention using queue locks Art of Multiprocessor Programming 5

6 Coarse-Grained Synchronization Each method locks the object Avoid contention using queue locks Easy to reason about In simple cases Art of Multiprocessor Programming 6

7 Coarse-Grained Synchronization Each method locks the object Avoid contention using queue locks Easy to reason about In simple cases So, are we done? Art of Multiprocessor Programming 7

8 Coarse-Grained Synchronization Sequential bottleneck Threads stand in line Art of Multiprocessor Programming 8

9 Coarse-Grained Synchronization Sequential bottleneck Threads stand in line Adding more threads Does not improve throughput Struggle to keep it from getting worse Art of Multiprocessor Programming 9

10 Coarse-Grained Synchronization Sequential bottleneck Threads stand in line Adding more threads Does not improve throughput Struggle to keep it from getting worse So why even use a multiprocessor? Well, some apps inherently parallel Art of Multiprocessor Programming 10

11 This Lecture Introduce four patterns Bag of tricks Methods that work more than once Art of Multiprocessor Programming 11

12 This Lecture Introduce four patterns Bag of tricks Methods that work more than once For highly-concurrent objects Concurrent access More threads, more throughput Art of Multiprocessor Programming 12

13 First: Fine-Grained Synchronization Instead of using a single lock Split object into Independently-synchronized components Methods conflict when they access The same component At the same time Art of Multiprocessor Programming 13

14 Second: Optimistic Synchronization Search without locking If you find it, lock and check OK: we are done Oops: start over Evaluation Usually cheaper than locking, but Mistakes are expensive Art of Multiprocessor Programming 14

15 Third: Lazy Synchronization Postpone hard work Removing components is tricky Logical removal Mark component to be deleted Physical removal Do what needs to be done Art of Multiprocessor Programming 15

16 Fourth: Lock-Free Synchronization Don t use locks at all Use compareandset() & relatives Advantages No Scheduler Assumptions/Support Disadvantages Complex Sometimes high overhead Art of Multiprocessor Programming 16

17 Linked List Illustrate these patterns Using a list-based Set Common application Building block for other apps Art of Multiprocessor Programming 17

18 Set Interface Unordered collection of items No duplicates Methods add(x) put x in set remove(x) take x out of set contains(x) tests if x in set Art of Multiprocessor Programming 18

19 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } Art of Multiprocessor Programming 19

20 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } Add item to set Art of Multiprocessor Programming 20

21 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(tt x); } Remove item from set Art of Multiprocessor Programming 21

22 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } Is item in set? Art of Multiprocessor Programming 22

23 List Node public class Node { public T item; public int key; public Node next; } Art of Multiprocessor Programming 23

24 List Node public class Node { public T item; public int key; public Node next; } item of interest Art of Multiprocessor Programming 24

25 List Node public class Node { public T item; public int key; public Node next; } Usually hash code Art of Multiprocessor Programming 25

26 List Node public class Node { public T item; public int key; public Node next; } Reference to next node Art of Multiprocessor Programming 26

27 The List-Based Set - a b c + Sorted with Sentinel nodes (min & max possible keys) Art of Multiprocessor Programming 27

28 Reasoning about Concurrent Objects Invariant Property that always holds Established because True when object is created Truth preserved by each method Each step of each method Art of Multiprocessor Programming 28

29 Specifically Invariants preserved by add() remove() contains() Most steps are trivial Usually one step tricky Often linearization point Art of Multiprocessor Programming 29

30 Interference Invariants make sense only if methods considered are the only modifiers Art of Multiprocessor Programming 30

31 Interference Invariants make sense only if methods considered are the only modifiers Language encapsulation helps List nodes not visible outside class Art of Multiprocessor Programming 31

32 Interference Freedom from interference needed even for removed nodes Some algorithms traverse removed nodes Careful with malloc() & free()! Garbage collection helps here Art of Multiprocessor Programming 32

33 Abstract Data Types Concrete representation: a b Abstract Type: {a, b} Art of Multiprocessor Programming 33

34 Abstract Data Types Meaning of rep given by abstraction map S( a b ) = {a,b} Art of Multiprocessor Programming 34

35 Rep Invariant Which concrete values meaningful? Sorted? Duplicates? Rep invariant Characterizes legal concrete reps Preserved by methods Relied on by methods Art of Multiprocessor Programming 35

36 Blame Game Rep invariant is a contract Suppose add() leaves behind 2 copies of x remove() removes only 1 Which is incorrect? Art of Multiprocessor Programming 36

37 Blame Game Suppose add() leaves behind 2 copies of x remove() removes only 1 Which is incorrect? If rep invariant says no duplicates add() is incorrect Otherwise remove() is incorrect Art of Multiprocessor Programming 37

38 Rep Invariant (partly) Sentinel nodes tail reachable from head Sorted No duplicates Art of Multiprocessor Programming 38

39 Abstraction Map S(head) = { x there exists a such that } a reachable from head and a.item = x Art of Multiprocessor Programming 39

40 Sequential List Based Set Add() a c d Remove() a b c Art of Multiprocessor Programming 40

41 Sequential List Based Set Add() a c d Remove() b a b c Art of Multiprocessor Programming 41

42 Coarse-Grained Locking a b d Art of Multiprocessor Programming 42

43 Coarse-Grained Locking a b d c Art of Multiprocessor Programming 43

44 Coarse-Grained Locking a b d honk! honk! c Simple but hotspot + bottleneck Art of Multiprocessor Programming 44

45 Coarse-Grained Locking Easy, same as synchronized methods One lock to rule them all Art of Multiprocessor Programming 45

46 Coarse-Grained Locking Easy, same as synchronized methods One lock to rule them all Simple, clearly correct Deserves respect! Works poorly with contention Queue locks help But bottleneck still an issue Art of Multiprocessor Programming 46

47 Fine-grained Locking Requires careful thought Do not meddle in the affairs of wizards, for they are subtle and quick to anger Art of Multiprocessor Programming 47

48 Fine-grained Locking Requires careful thought Do not meddle in the affairs of wizards, for they are subtle and quick to anger Split object into pieces Each piece has own lock Methods that work on disjoint pieces need not exclude each other Art of Multiprocessor Programming 48

49 Hand-over-Hand locking a b c Art of Multiprocessor Programming 49

50 Hand-over-Hand locking a b c Art of Multiprocessor Programming 50

51 Hand-over-Hand locking a b c Art of Multiprocessor Programming 51

52 Hand-over-Hand locking a b c Art of Multiprocessor Programming 52

53 Hand-over-Hand locking a b c Art of Multiprocessor Programming 53

54 Removing a Node a b c d remove(b) Art of Multiprocessor Programming 54

55 Removing a Node a b c d remove(b) Art of Multiprocessor Programming 55

56 Removing a Node a b c d remove(b) Art of Multiprocessor Programming 56

57 Removing a Node a b c d remove(b) Art of Multiprocessor Programming 57

58 Removing a Node a b c d remove(b) Art of Multiprocessor Programming 58

59 Removing a Node a c d remove(b) Why hold 2 locks? Art of Multiprocessor Programming 59

60 Concurrent Removes a b c d remove(b) remove(c) Art of Multiprocessor Programming 60

61 Concurrent Removes a b c d remove(b) remove(c) Art of Multiprocessor Programming 61

62 Concurrent Removes a b c d remove(b) remove(c) Art of Multiprocessor Programming 62

63 Concurrent Removes a b c d remove(b) remove(c) Art of Multiprocessor Programming 63

64 Concurrent Removes a b c d remove(b) remove(c) Art of Multiprocessor Programming 64

65 Concurrent Removes a b c d remove(b) remove(c) Art of Multiprocessor Programming 65

66 Concurrent Removes a b c d remove(b) remove(c) Art of Multiprocessor Programming 66

67 Concurrent Removes a b c d remove(b) remove(c) Art of Multiprocessor Programming 67

68 Concurrent Removes a b c d remove(b) remove(c) Art of Multiprocessor Programming 68

69 Concurrent Removes a b c d remove(b) remove(c) Art of Multiprocessor Programming 69

70 Uh, Oh a c d remove(b) remove(c) Art of Multiprocessor Programming 70

71 Uh, Oh Bad news, c not removed a c d remove(b) remove(c) Art of Multiprocessor Programming 71

72 Problem To delete node c Swing node b s next field to d a b c Problem is, Someone deleting b concurrently could direct a pointer a b c to c Art of Multiprocessor Programming 72

73 Insight If a node is locked No one can delete node s successor If a thread locks Node to be deleted And its predecessor Then it works Art of Multiprocessor Programming 73

74 Hand-Over-Hand Again a b c d remove(b) Art of Multiprocessor Programming 74

75 Hand-Over-Hand Again a b c d remove(b) Art of Multiprocessor Programming 75

76 Hand-Over-Hand Again a b c d remove(b) Art of Multiprocessor Programming 76

77 Hand-Over-Hand Again a b c d remove(b) Found it! Art of Multiprocessor Programming 77

78 Hand-Over-Hand Again a b c d remove(b) Found it! Art of Multiprocessor Programming 78

79 Hand-Over-Hand Again a c d remove(b) Art of Multiprocessor Programming 79

80 Removing a Node a b c d remove(b) remove(c) Art of Multiprocessor Programming 80

81 Removing a Node a b c d remove(b) remove(c) Art of Multiprocessor Programming 81

82 Removing a Node a b c d remove(b) remove(c) Art of Multiprocessor Programming 82

83 Removing a Node a b c d remove(b) remove(c) Art of Multiprocessor Programming 83

84 Removing a Node a b c d remove(b) remove(c) Art of Multiprocessor Programming 84

85 Removing a Node a b c d remove(b) remove(c) Art of Multiprocessor Programming 85

86 Removing a Node a b c d remove(b) remove(c) Art of Multiprocessor Programming 86

87 Removing a Node a b c d remove(b) remove(c) Art of Multiprocessor Programming 87

88 Removing a Node a b c d Must acquire Lock for b remove(c) Art of Multiprocessor Programming 88

89 Removing a Node a b c d Cannot acquire lock for b remove(c) Art of Multiprocessor Programming 89

90 Removing a Node a b c d Wait! remove(c) Art of Multiprocessor Programming 90

91 Removing a Node a b d Proceed to remove (b) Art of Multiprocessor Programming 91

92 Removing a Node a b d remove(b) Art of Multiprocessor Programming 92

93 Removing a Node a b d remove(b) Art of Multiprocessor Programming 93

94 Removing a Node a d remove(b) Art of Multiprocessor Programming 94

95 Removing a Node a d Art of Multiprocessor Programming 95

96 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; try { } finally { curr.unlock(); pred.unlock(); }} Art of Multiprocessor Programming 96

97 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; try { } finally { curr.unlock(); pred.unlock(); }} Key used to order node Art of Multiprocessor Programming 97

98 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; try { } finally { currnode.unlock(); prednode.unlock(); }} Predecessor and current nodes Art of Multiprocessor Programming 98

99 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; try { } finally { curr.unlock(); pred.unlock(); }} Make sure locks released Art of Multiprocessor Programming 99

100 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; try { } finally { curr.unlock(); pred.unlock(); }} Everything else Art of Multiprocessor Programming 100

101 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } Art of Multiprocessor Programming 101

102 Remove method lock pred == head try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } Art of Multiprocessor Programming 102

103 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } Lock current Art of Multiprocessor Programming 103

104 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } Traversing list Art of Multiprocessor Programming 104

105 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Art of Multiprocessor Programming 105

106 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } Search key range pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Art of Multiprocessor Programming 106

107 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; At start of each loop: curr and pred locked Art of Multiprocessor Programming 107

108 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; If item found, remove node Art of Multiprocessor Programming 108

109 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; If node found, remove it Art of Multiprocessor Programming 109

110 Remove: searching Unlock predecessor while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Art of Multiprocessor Programming 110

111 Remove: searching Only one node locked! while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Art of Multiprocessor Programming 111

112 Remove: searching while (curr.key <= key) { demote current if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Art of Multiprocessor Programming 112

113 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = currnode; curr = curr.next; curr.lock(); } return false; Find and lock new current Art of Multiprocessor Programming 113

114 Remove: searching while (curr.key <= key) { Lock if (item invariant == curr.item) restored { pred.next = curr.next; return true; } pred.unlock(); pred = currnode; curr = curr.next; curr.lock(); } return false; Art of Multiprocessor Programming 114

115 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Otherwise, not present Art of Multiprocessor Programming 115

116 Why does this work? To remove node e Must lock e Must lock e s predecessor Therefore, if you lock a node It can t be removed And neither can its successor Art of Multiprocessor Programming 116

117 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; pred reachable from head curr is pred.next So curr.item is in the set Art of Multiprocessor Programming 117

118 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linearization point if item is present Art of Multiprocessor Programming 118

119 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Node locked, so no other thread can remove it. Art of Multiprocessor Programming 119

120 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Item not present Art of Multiprocessor Programming 120

121 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; pred reachable from head curr.lock(); curr is pred.next } pred.key < key return false; key < curr.key Art of Multiprocessor Programming 121

122 Why remove() is linearizable while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Linearization point Art of Multiprocessor Programming 122

123 Adding Nodes To add node e Must lock predecessor Must lock successor Neither can be deleted (Is successor lock actually required?) Art of Multiprocessor Programming 123

124 Same Abstraction Map S(head) = { x there exists a such that } a reachable from head and a.item = x Art of Multiprocessor Programming 124

125 Rep Invariant Easy to check that tail always reachable from head Nodes sorted, no duplicates Art of Multiprocessor Programming 125

126 Drawbacks Better than coarse-grained lock Threads can traverse in parallel Still not ideal Long chain of acquire/release Inefficient Art of Multiprocessor Programming 126

127 Optimistic Synchronization Find nodes without locking Lock nodes Check that everything is OK Art of Multiprocessor Programming 127

128 Optimistic: Traverse without Locking a b d e add(c) Aha! Art of Multiprocessor Programming 128

129 Optimistic: Lock and Load a b d e add(c) Art of Multiprocessor Programming 129

130 Optimistic: Lock and Load c a b d e add(c) Art of Multiprocessor Programming 130

131 What could go wrong? a b d e add(c) Aha! Art of Multiprocessor Programming 131

132 What could go wrong? a b d e add(c) Art of Multiprocessor Programming 132

133 What could go wrong? a b d e remove(b) Art of Multiprocessor Programming 133

134 What could go wrong? a b d e remove(b) Art of Multiprocessor Programming 134

135 What could go wrong? a b d e add(c) Art of Multiprocessor Programming 135

136 What could go wrong? c a b d e add(c) Art of Multiprocessor Programming 136

137 What could go wrong? a d e add(c) Uh-oh Art of Multiprocessor Programming 137

138 Validate Part 1 a b d e add(c) Yes, b still reachable from head Art of Multiprocessor Programming 138

139 What Else Could Go Wrong? a b d e add(c) Aha! Art of Multiprocessor Programming 139

140 What Else Coould Go Wrong? a b d e add(c) add(b ) Art of Multiprocessor Programming 140

141 What Else Coould Go Wrong? a b d e add(c) b add(b ) Art of Multiprocessor Programming 141

142 What Else Could Go Wrong? a b d e add(c) b Art of Multiprocessor Programming 142

143 What Else Could Go Wrong? c a b d e add(c) Art of Multiprocessor Programming 143

144 Validate Part 2 (while holding locks) a b d e add(c) Yes, b still points to d Art of Multiprocessor Programming 144

145 Optimistic: Linearization Point a b d e add(c) c Art of Multiprocessor Programming 145

146 Same Abstraction Map S(head) = { x there exists a such that } a reachable from head and a.item = x Art of Multiprocessor Programming 146

147 Invariants Careful: we may traverse deleted nodes But we establish properties by Validation After we lock target nodes Art of Multiprocessor Programming 147

148 Correctness If Nodes b and c both locked Node b still accessible Node c still successor to b Then Neither will be deleted OK to delete and return true Art of Multiprocessor Programming 148

149 Unsuccessful Remove a b d e remove(c) Aha! Art of Multiprocessor Programming 149

150 Validate (1) a b d e remove(c) Yes, b still reachable from head Art of Multiprocessor Programming 150

151 Validate (2) a b d e remove(c) Yes, b still points to d Art of Multiprocessor Programming 151

152 OK Computer a b d e remove(c) return false Art of Multiprocessor Programming 152

153 Correctness If Nodes b and d both locked Node b still accessible Node d still successor to b Then Neither will be deleted No thread can add c after b OK to return false Art of Multiprocessor Programming 153

154 Validation private boolean validate(node pred, Node curry) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Art of Multiprocessor Programming 154

155 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } Predecessor & return false; current nodes } Art of Multiprocessor Programming 155

156 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Begin at the beginning Art of Multiprocessor Programming 156

157 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Search range of keys Art of Multiprocessor Programming 157

158 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Predecessor reachable Art of Multiprocessor Programming 158

159 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Is current node next? Art of Multiprocessor Programming 159

160 Validation private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Otherwise move on Art of Multiprocessor Programming 160

161 Validation Predecessor not reachable private boolean validate(node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Art of Multiprocessor Programming 161

162 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } Art of Multiprocessor Programming 162

163 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } Search key Art of Multiprocessor Programming 163

164 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } Retry on synchronization conflict Art of Multiprocessor Programming 164

165 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } Examine predecessor and current nodes Art of Multiprocessor Programming 165

166 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; Search by key } Art of Multiprocessor Programming 166

167 Remove: searching public boolean remove(item item) { int key = item.hashcode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } Stop if we find item Art of Multiprocessor Programming 167

168 Remove: searching public boolean remove(item item) { int key Move = item.hashcode(); along retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (item == curr.item) break; pred = curr; curr = curr.next; } Art of Multiprocessor Programming 168

169 On Exit from Loop If item is present curr holds item pred just before curr If item is absent curr has first higher key pred just before curr Assuming no synchronization problems Art of Multiprocessor Programming 169

170 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Art of Multiprocessor Programming 170

171 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Always unlock Art of Multiprocessor Programming 171

172 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Lock both nodes Art of Multiprocessor Programming 172

173 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Check for synchronization conflicts Art of Multiprocessor Programming 173

174 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} target found, remove node Art of Multiprocessor Programming 174

175 Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.item == item) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} target not found Art of Multiprocessor Programming 175

176 Optimistic List Limited hot-spots Targets of add(), remove(), contains() No contention on traversals Moreover Traversals are wait-free Food for thought Art of Multiprocessor Programming 176

177 So Far, So Good Much less lock acquisition/release Performance Concurrency Problems Need to traverse list twice contains() method acquires locks Art of Multiprocessor Programming 177

178 Evaluation Optimistic is effective if cost of scanning twice without locks is less than cost of scanning once with locks Drawback contains() acquires locks 90% of calls in many apps Art of Multiprocessor Programming 178

179 Lazy List Like optimistic, except Scan once contains(x) never locks Key insight Removing nodes causes trouble Do it lazily Art of Multiprocessor Programming 179

180 Lazy List remove() Scans list (as before) Locks predecessor & current (as before) Logical delete Marks current node as removed (new!) Physical delete Redirects predecessor s next (as before) Art of Multiprocessor Programming 180

181 Lazy Removal a b c d Art of Multiprocessor Programming 181

182 Lazy Removal a b c d Present in list Art of Multiprocessor Programming 182

183 Lazy Removal a b c d Logically deleted Art of Multiprocessor Programming 183

184 Lazy Removal a b c d Physically deleted Art of Multiprocessor Programming 184

185 Lazy Removal a b d Physically deleted Art of Multiprocessor Programming 185

186 Lazy List All Methods Scan through locked and marked nodes Removing a node doesn t slow down other method calls Must still lock pred and curr nodes. Art of Multiprocessor Programming 186

187 Validation No need to rescan list! Check that pred is not marked Check that curr is not marked Check that pred points to curr Art of Multiprocessor Programming 187

188 Business as Usual a b c Art of Multiprocessor Programming 188

189 Business as Usual a b c Art of Multiprocessor Programming 189

190 Business as Usual a b c Art of Multiprocessor Programming 190

191 Business as Usual a b c remove(b) Art of Multiprocessor Programming 191

192 Business as Usual a b c a not marked Art of Multiprocessor Programming 192

193 Business as Usual a b c a still points to b Art of Multiprocessor Programming 193

194 Business as Usual a b c Logical delete Art of Multiprocessor Programming 194

195 Business as Usual a b c physical delete Art of Multiprocessor Programming 195

196 Business as Usual a b c Art of Multiprocessor Programming 196

197 New Abstraction Map S(head) = { x there exists node a such that } a reachable from head and a.item = x and a is unmarked Art of Multiprocessor Programming 197

198 Invariant If not marked then item in the set and reachable from head and if not yet traversed it is reachable from pred Art of Multiprocessor Programming 198

199 Validation private boolean validate(node pred, Node curr) { return!pred.marked &&!curr.marked && pred.next == curr); } Art of Multiprocessor Programming 199

200 List Validate Method private boolean validate(node pred, Node curr) { return!pred.marked &&!curr.marked && pred.next == curr); } Predecessor not Logically removed Art of Multiprocessor Programming 200

201 List Validate Method private boolean validate(node pred, Node curr) { return!pred.marked &&!curr.marked && pred.next == curr); } Current not Logically removed Art of Multiprocessor Programming 201

202 List Validate Method private boolean validate(node pred, Node curr) { return!pred.marked &&!curr.marked && pred.next == curr); } Predecessor still Points to current Art of Multiprocessor Programming 202

203 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Art of Multiprocessor Programming 203

204 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Validate as before Art of Multiprocessor Programming 204

205 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Key found Art of Multiprocessor Programming 205

206 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Logical remove Art of Multiprocessor Programming 206

207 Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} physical remove Art of Multiprocessor Programming 207

208 Contains public boolean contains(item item) { int key = item.hashcode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key &&!curr.marked; } Art of Multiprocessor Programming 208

209 Contains public boolean contains(item item) { int key = item.hashcode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key &&!curr.marked; } Start at the head Art of Multiprocessor Programming 209

210 Contains public boolean contains(item item) { int key = item.hashcode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key &&!curr.marked; } Search key range Art of Multiprocessor Programming 210

211 Contains public boolean contains(item item) { int key = item.hashcode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key &&!curr.marked; } Traverse without locking (nodes may have been removed) Art of Multiprocessor Programming 211

212 Contains public boolean contains(item item) { int key = item.hashcode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key &&!curr.marked; } Present and undeleted? Art of Multiprocessor Programming 212

213 Summary: Wait-free Contains a 0 b 0 d c 10 e 0 Use Mark bit + list ordering 1. Not marked à in the set 2. Marked or missing à not in the set Art of Multiprocessor Programming 213

214 Lazy List a 0 b 0 d c 10 e 0 Lazy add() and remove() + Wait-free contains() Art of Multiprocessor Programming 214

215 Evaluation Good: contains() doesn t lock In fact, its wait-free! Good because typically high % contains() Uncontended calls don t re-traverse Bad Contended add() and remove() calls do re-traverse Traffic jam if one thread delays Art of Multiprocessor Programming 215

216 Traffic Jam Any concurrent data structure based on mutual exclusion has a weakness If one thread Enters critical section And eats the big muffin Cache miss, page fault, descheduled Everyone else using that lock is stuck! Need to trust the scheduler. Art of Multiprocessor Programming 216

217 Reminder: Lock-Free Data No matter what Structures Guarantees minimal progress in any execution i.e. Some thread will always complete a method call Even if others halt at malicious times Implies that implementation can t use locks Art of Multiprocessor Programming 217

218 Lock-free Lists Next logical step Wait-free contains() lock-free add() and remove() Use only compareandset() What could go wrong? Art of Multiprocessor Programming 218

219 Lock-free Lists Logical Removal a 0 b 0 c 10 e 0 Use CAS to verify pointer is correct Physical Removal Not enough! Art of Multiprocessor Programming 219

220 Problem Logical Removal a 0 b 0 c 10 e 0 Physical Removal d 0 Node added Art of Multiprocessor Programming 220

221 The Solution: Combine Bit and Pointer Logical Removal = Set Mark Bit a 0 b 0 c 10 e 0 Physical Removal CAS Mark-Bit and Pointer are CASed together (AtomicMarkableReference) d 0 Fail CAS: Node not added after logical Removal Art of Multiprocessor Programming 221

222 Solution Use AtomicMarkableReference Atomically Swing reference and Update flag Remove in two steps Set mark bit in next field Redirect predecessor s pointer Art of Multiprocessor Programming 222

223 Marking a Node class package Reference address F mark bit Art of Multiprocessor Programming 223

224 Extracting Reference & Mark Public Object get(boolean[] marked); Art of Multiprocessor Programming 224

225 Extracting Reference & Mark Public Object get(boolean[] marked); Returns reference Returns mark at array index 0! Art of Multiprocessor Programming 225

226 Extracting Reference Only public boolean ismarked(); Value of mark Art of Multiprocessor Programming 226

227 Changing State Public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); Art of Multiprocessor Programming 227

228 Changing State If this is the current reference Public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); And this is the current mark Art of Multiprocessor Programming 228

229 Changing State then change to this new reference Public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); and this new mark Art of Multiprocessor Programming 229

230 Changing State public boolean attemptmark( Object expectedref, boolean updatemark); Art of Multiprocessor Programming 230

231 Changing State public boolean attemptmark( Object expectedref, boolean updatemark); If this is the current reference Art of Multiprocessor Programming 231

232 Changing State public boolean attemptmark( Object expectedref, boolean updatemark);.. then change to this new mark. Art of Multiprocessor Programming 232

233 Removing a Node a b CAS c d remove c Art of Multiprocessor Programming 233

234 Removing a Node failed a CAS b CAS c d remove b remove c Art of Multiprocessor Programming 234

235 Removing a Node a b c d remove b remove c Art of Multiprocessor Programming 235

236 Removing a Node a d remove b remove c Art of Multiprocessor Programming 236

237 Traversing the List Q: what do you do when you find a logically deleted node in your path? A: finish the job. CAS the predecessor s next field Proceed (repeat as needed) Art of Multiprocessor Programming 237

238 Lock-Free Traversal (only Add and Remove) pred curr pred curr CAS a b c d Uh-oh Art of Multiprocessor Programming 238

239 The Window Class class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; } } Art of Multiprocessor Programming 239

240 The Window Class class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; } } A container for pred and current values Art of Multiprocessor Programming 240

241 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Art of Multiprocessor Programming 241

242 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Find returns window Art of Multiprocessor Programming 242

243 Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Extract pred and curr Art of Multiprocessor Programming 243

244 The Find Method Window window = find(item); At some instant, item or pred curr succ Art of Multiprocessor Programming Herlihy-Shavit

245 The Find Method Window window = find(item); At some instant, item not in list pred curr= null succ Art of Multiprocessor Programming Herlihy-Shavit

246 Remove public boolean remove(t item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset(succ, succ, false true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; }}} Art of Multiprocessor Programming 246

247 Remove public boolean remove(t item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset (succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); Keep trying return true; Art of Multiprocessor Programming 247 }}}

248 Remove public boolean remove(t item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset (succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; Find neighbors Art of Multiprocessor Programming 248 }}}

249 Remove public boolean remove(t item) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset(succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; She s not there }}} Art of Multiprocessor Programming 249

250 Remove public boolean remove(t item) { Boolean Try snip; to mark node as deleted while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset(succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; }}} Art of Multiprocessor Programming 250

251 Remove public boolean remove(t item) { Boolean If it snip; doesn t work, while (true) just retry, { Window window = find(head, key); Node if it pred does, = window.pred, job curr = window.curr; essentially if (curr.key done!= key) { return false; } else { Node succ = curr.next.getreference(); snip = curr.next.compareandset(succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; }}} Art of Multiprocessor Programming 251

252 Remove public boolean remove(t item) { Boolean snip; while (true) { a Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key!= key) { return false; Try to advance reference } else { Node (if we succ don t = curr.next.getreference(); succeed, someone else did or will). snip = curr.next.compareandset(succ, succ, false, true); if (!snip) continue; pred.next.compareandset(curr, succ, false, false); return true; }}} Art of Multiprocessor Programming 252

253 Add public boolean add(t item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareandset(curr, node, false, false)) {return true;} }}} Art of Multiprocessor Programming 253

254 Add public boolean add(t item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareandset(curr, node, false, false)) {return true;} }}} Item already there. Art of Multiprocessor Programming 254

255 Add public boolean add(t item) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareandset(curr, node, false, false)) {return true;} }}} create new node Art of Multiprocessor Programming 255

256 Add public boolean add(t item) { boolean splice; Install new node, while (true) { else retry loop Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(item); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareandset(curr, node, false, false)) {return true;} }}} Art of Multiprocessor Programming 256

257 Wait-free Contains public boolean contains(t item) { boolean marked; int key = item.hashcode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key &&!marked[0]) } Art of Multiprocessor Programming 257

258 Wait-free Contains public boolean contains(t item) { boolean marked; Only diff is that we int key = item.hashcode(); get and check Node curr = this.head; marked while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key &&!marked[0]) } Art of Multiprocessor Programming 258

259 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Art of Multiprocessor Programming 259

260 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); }} while (true) { succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } If list changes while traversed, start over Art of Multiprocessor Programming 260

261 Lock-free Find public Window find(node head, int key) { Node pred = null, Start curr looking = null, succ from = head null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Art of Multiprocessor Programming 261

262 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { Move down the list pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Art of Multiprocessor Programming 262

263 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); }} } pred = curr; curr = succ; Get ref to successor and current deleted bit Art of Multiprocessor Programming 263

264 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} path code details soon Try to remove deleted nodes in Art of Multiprocessor Programming 264

265 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); If while curr (true) key that { is greater or equal, succ = return curr.next.get(marked); pred and curr while (marked[0]) { } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Art of Multiprocessor Programming 265

266 Lock-free Find public Window find(node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getreference(); while (true) { succ = curr.next.get(marked); Otherwise while (marked[0]) advance { window and loop again } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Art of Multiprocessor Programming 266

267 Lock-free Find retry: while (true) { while (marked[0]) { snip = pred.next.compareandset(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } Art of Multiprocessor Programming 267

268 Lock-free Find Try to snip out node retry: while (true) { while (marked[0]) { snip = pred.next.compareandset(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } Art of Multiprocessor Programming 268

269 Lock-free Find if predecessor s next field changed must retry whole traversal retry: while (true) { while (marked[0]) { snip = pred.next.compareandset(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } Art of Multiprocessor Programming 269

270 Lock-free Find Otherwise move on to check if next node deleted retry: while (true) { while (marked[0]) { snip = pred.next.compareandset(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } Art of Multiprocessor Programming 270

271 Performance On 16 node shared memory machine Benchmark throughput of Java List-based Set algs. Vary % of Contains() method Calls. Art of Multiprocessor Programming 271

272 High Contains Ratio Lock-free Lazy list Coarse Grained Fine Lock-coupling Art of Multiprocessor Programming 272

273 Low Contains Ratio Lock-free Lazy list Coarse Grained Fine Lock-coupling Art of Multiprocessor Programming 273

274 As Contains Ratio Increases Lock-free Lazy list Coarse Grained Fine Lock-coupling % Contains() Art of Multiprocessor Programming 274

275 Summary Coarse-grained locking Fine-grained locking Optimistic synchronization Lock-free synchronization Art of Multiprocessor Programming 275

276 To Lock or Not to Lock Locking vs. Non-blocking: Extremist views on both sides The answer: nobler to compromise, combine locking and non-blocking Example: Lazy list combines blocking add() and remove() and a wait-free contains() Remember: Blocking/non-blocking is a property of a method Art of Multiprocessor Programming 276

277 This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License. You are free: to Share to copy, distribute and transmit the work to Remix to adapt the work Under the following conditions: Attribution. You must attribute the work to The Art of Multiprocessor Programming (but not in any way that suggests that the authors endorse you or your use of the work). Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights. Art of Multiprocessor Programming 277