Lock-free algorithms for Kotlin coroutines. It is all about scalability Presented at SPTCC 2017 /Roman JetBrains

Transcription

1 Lock-free algorithms for Kotlin coroutines It is all about scalability resented at TCC 2017 /Roman JetBrains

2 peaker: Roman Elizarov 16+ years experience reviously developed high-perf trading Devexperts Teach concurrent & distributed t. etersburg ITMO University Chief Northeastern European Region of ACM ICC Now work on JetBrains

3 Agenda Kotlin coroutines overview & motivation for lockfree algorithms Lock-free doubly linked list Lock-free multi-word compare-and-swap Combining them to get more complex atomic operations (without TM)

4 Kotlin basic facts Kotlin is a JVM language developed by JetBrains General purpose and statically-typed Object-oriented and functional paradigms Open source under Apache 2.0 Reached version 1.0 in 2016 Compatibility commitment Now at version 1.1 Officially supported by Google on Android

5 Kotlin is Modern Concise afe Extensible ragmatic Fun to work with!

6 Kotlin is pragmatic and easy to learn

7 Coroutines Asynchronous programming made easy

8 How do we write code that waits for something most of the time?

9 Blocking threads Kotlin fun postitem(item: Item) { val token = requesttoken() val post = submitost(token, item) processost(post) }

10 Callbacks Kotlin fun postitem(item: Item) { requesttoken { token -> submitost(token, item) { post -> processost(post) } } }

11 Futures/romises/Rx Kotlin fun postitem(item: Item) { requesttoken().thencompose { token -> submitost(token, item) }.thenaccept { post -> processost(post) } }

12 Coroutines Kotlin fun postitem(item: Item) { launch(commonool) { val token = requesttoken() val post = submitost(token, item) processost(post) } }

13 C & Actor models A style of programming for modern systems Lots of concurrent tasks / jobs Waiting most of the time Communicating all the time hare data by communicating

14 Kotlin coroutines primitives Jobs/Deferreds (futures) join/await Channels send & receive synchronous & buffered channels elect/alternatives Atomically wait on multiple events Cancellation arent-child hierarchies

15 Implementation challenges Coroutines are like light-weight threads All the low-level scheduling & communication mechanisms have to scale to lots of coroutines

16 Lock-free algorithms

17 Building blocks ingle-word CA (that s all we have on JVM) Automatic memory management (GC) ractical lock-free algorithms Lock-Free and ractical Doubly Linked List-Based Deques Using ingle-word Compare-and-wap by undell and Tsigas A ractical Multi-Word Compare-and-wap Operation by Timothy L. Harris, Keir Fraser and Ian A. ratt.

18 Doubly linked list next links form logical list contents prev links are auxiliary H N 1 N T sentinel sentinel Use same node in practice

19 Insert ushright (like in queue)

20 Doubly linked list (insert 0) 2 2 N create & init H N 1 N 1 T

21 Doubly linked list (insert 1) 2 N H N 1 N CA T Retry insert on CA failure

22 Doubly linked list (insert 2) finish insert 2 N H N 1 N CA T Ignore CA failure

23 Remove opleft (like in queue)

24 Doubly linked list (remove 1) 2 Mark removed node s next link H N 1 1 N CA T Retry remove on CA failure Use wrapper object for mark in practice Don t use AtomicMarkableReference Cache wrappers in pointed-to nodes

25 Doubly linked list (remove 2) finish remove Mark removed node s prev link H N CA 1 N T Retry marking on CA failure

26 Doubly linked list (remove 3) help remove fixup next links H CA N 1 N T

27 Doubly linked list (remove 4) correct prev fixup prev links H N 1 N T CA

28 tate transitions

29 Node states correct prev Init next: Ok prev: Ok prev.next: -- next.prev: -- Insert 1 next: Ok prev: Ok prev.next: me next.prev: -- Insert 2 next: Ok prev: Ok prev.next: me next.prev: me Remove 1 next: Rem prev: Ok prev.next: me next.prev: me Remove 2 next: Rem prev: Rem prev.next: me next.prev: me Remove 3 next: Rem prev: Rem prev.next: ++ next.prev: me Remove 4 next: Rem prev: Rem prev.next: ++ next.prev: ++ help remove correct prev

30 Helping

31 Concurrent insert

32 Concurrent insert (0) I2 I3 2 N H N 1 N T 3 N

33 Concurrent insert (1) I2 I3 2 N H N 1 N CA ok T CA fail 3 N

34 Concurrent insert (2) I2 I3 2 N H N 1 N T 3 N detect wrong prev (t.prev.next!= t)

35 Concurrent insert (3) I2 I3 2 N correct prev H N 1 N T 3 N

36 Concurrent insert (4) I2 I3 2 N H N 1 N T 3 N reinit & repeat

37 Concurrent remove

38 Concurrent remove (0) R1 R2 H N 1 N 2 N T

39 Concurrent remove (1) R1 R2 R1 H N 1 N 2 N T

40 Concurrent remove (2) R1 R2 R1 H N 1 N 2 N T Finds already removed R2

41 Concurrent remove (3) R1 R2 R1 help remove H N 1 N 2 N T mark prev R2

42 Concurrent remove (4) R1 R2 R1 H N 1 N 2 N T Retry with corrected next R2

43 Concurrent remove (5) R1 R2 R1 help remove H N 1 N 2 N T R2

44 Concurrent remove (6) R1 R2 R1 H N 1 N 2 N T correct prev R2

45 Concurrent remove & insert When remove wins

46 Concurrent remove & insert (0) R1 I2 2 N create & init H N 1 N T R1

47 Concurrent remove & insert (1) R1 I2 2 N H N remove first 1 N T R1

48 Concurrent remove & insert (2) R1 I2 2 N CA fail H N 1 N T R1

49 Concurrent remove & insert (3) R1 I2 2 N H N 1 N T R1 detect wrong prev (t.prev.next -- removed) do correct prev

50 Concurrent remove & insert (4) R1 I2 2 N fixup next H N mark prev 1 N T R1

51 Concurrent remove & insert (5) R1 I2 2 N H N 1 N T update prev R1

52 Concurrent remove & insert (6) R1 I2 2 N reinit & repeat H N 1 N T R1

53 Concurrent remove & insert When insert wins

54 Concurrent remove & insert (0) R1 I2 2 N create & init H N 1 N T R1

55 Concurrent remove & insert (1) R1 I2 2 N H N 1 N CA T R1

56 Concurrent remove & insert (2) R1 I2 2 N H N 1 N T will succeed marking on remove retry R1

57 Concurrent remove & insert (3) R1 I2 help remove 2 N H N 1 N T mark prev R1

58 Concurrent remove & insert (4) R1 I2 correct prev 2 N H N 1 N T R1 Remove is over!

59 Concurrent remove & insert (5) R1 I2 2 N H N 1 N T correct prev R1

60 Takeaways A kind of algo you need a paper for Hard to improve w/o writing another paper Good news: stress tests uncover most impl bugs Bad news: when stress test fails, you up to long hours More bad news: hard to find bugs that violate lockfreedomness of algorithm

61 ummary: what we can do Insert items (at the end of the queue) Remove items (at the front of the queue) Traverse the list Remove items at arbitrary locations In O(1)

62 Linearizability Insert last Linearizes at CA of next Remove first / arbitrary uccess at CA of next Fail at read of head.next

63 More about algorithm undell & Tsigas algo supports deque operations Can ushleft & opright opleft is simple read head.next & remove But cannot linearize them all at cas points ushleft, ushright, opright - Ok opleft linearizes at head.next read (!!!)

64 ummary of impl notes Use GC (drop all memory management details) Merge head & tail into a single sentinel node Empty list is just one object (prev & next onto itself) One item += one object Q N 1 N Reuse remove mark objects One-element lists reuse of ptrs to sentinel all the time Encapsulate!

65

66 Mods More complex atomic operations

67 Basic mods (1) Insert item conditionally on prev tail value 2 N check & bailout before CA H N 1 N T

68 Basic mods (2) Remove head conditionally on prev head value check & bailout before CA H N 1 N T R1

69 ractical use-case: synchronous channels 1 2 val channel = Channel<Int>() // coroutine #1 for (x in 1..5) { channel.send(x * x) } 3 // coroutine #2 repeat(5) { println(channel.receive()) }

70 enders wait Incoming receivers More senders ender #1 H ender #2 ender #3 T Receiver removes first if it is a sender node ender inserts last if it is not a receiver node

71 Receivers wait Incoming senders More receivers Receiver #1 H Receiver #2 Receiver #3 T ender removes first if it is a receiver node Receiver inserts last if it is not a sender node

72 end function sketch 1 fun send(element: T) { while (true) { // try to add sender, unless prev is receiver 2 if (enqueueend(element)) break // try to remove first receiver 3 val receiver = removefirstreceiver() if (receiver!= null) { 4 receiver.resume(element) // resume receiver break } } }

73 Channel use-case recap Uses insert/remove ops conditional on tail/head node Can abort (cancel) wait to receive/send at any time by using remove Full removal -- no garbage is left retty efficient in practice One item lists one garbage object

74 Multi-word compare and swap (CAN) Build even bigger atomic operations

75 Use-case: select expression val channel1 = Channel<Int>() val channel2 = Channel<Int>() select { channel1.onreceive { e ->... } channel2.onreceive { e ->... } }

76 Impl summary: register (1) elect status: N 1. Not selected 2. elected Channel1 Queue Channel2 Queue

77 Impl summary: register (2) elect status: N Add node to channel1 queue if not selected (N) yet Channel1 Queue N1 Channel2 Queue

78 Impl summary: register (3) elect status: N Add node to channel2 queue if not selected (N) yet Channel1 Queue N1 Channel2 Queue N2

79 Impl summary: wait elect status: N Channel1 Queue N1 Channel2 Queue N2

80 Impl summary: select (resume) elect status: Make selected and remove node from queue Channel1 Queue N1 Channel2 Queue

81 Impl summary: clean up rest elect status: Remove non-selected waiters from queue Channel1 Queue Channel2 Queue

82 Double-Compare ingle-wap (DC) Building block for CAN

83 DC spec in pseudo-code A B 1 2 fun <A,B> dcss( a: Ref<A>, expecta: A, updatea: A, b: Ref<B>, expectb: B) = atomic { 3 if (a.value == expecta && b.value == expectb) { 4 a.value = updatea } }

84 DC: init descriptor expecta A B expectb updatea DC Descriptor (a, expecta, updatea, b, expectb)

85 DC: prepare expecta A B expectb updatea CA ptr to descriptor if a.value == expecta DC Descriptor (a, expecta, updatea, b, expectb)

86 DC: read b.value expecta A B expectb updatea CA ptr to descriptor if a.value == expecta DC Descriptor (a, expecta, updatea, b, expectb)

87 DC: complete (when success) expecta A B expectb updatea CA to updated value if a still points to descriptor DC Descriptor (a, expecta, updatea, b, expectb)

88 DC: complete (alternative) expecta A B!expectB updatea DC Descriptor (a, expecta, updatea, b, expectb)

89 DC: complete (when fail) expecta A B!expectB updatea CA to original value if a still points to descriptor DC Descriptor (a, expecta, updatea, b, expectb)

90 DC: tates Any other thread encountering descriptor helps complete 1 2 Init prep ok A:??? (desc created) A: desc A was expecta success 3 A: updatea B was expectb prep fail fail 4 5 A:??? A was!expecta A: expecta B was!expectb Originator cannot learn what was the outcome one tread Lock-free algorithm without loops!

91 Caveats A & B locations must be totally ordered or risk stack-overflow while helping One way to look at it: Restricted DC (RDC)

92 DC Mod: learn outcome A B fun <A,B> dcssmod( a: Ref<A>, expecta: A, updatea: A, b: Ref<B>, expectb: B): Boolean = atomic { if (a.value == expecta && b.value == expectb) { a.value = updatea true } else false }

93 DC Mod: init descriptor expecta A B expectb updatea DC Descriptor (a, expecta, updatea, b, expectb) Outcome: UNDECIDED Consensus

94 DC Mod: prepare expecta A B expectb updatea DC Descriptor (a, expecta, updatea, b, expectb) Outcome: UNDECIDED

95 DC Mod: read b.value expecta A B expectb updatea DC Descriptor (a, expecta, updatea, b, expectb) Outcome: UNDECIDED

96 DC Mod: reach consensus expecta A B expectb updatea DC Descriptor (a, expecta, updatea, b, expectb) Outcome: UCCE CA(UNDECIDED, DECIION)

97 DC Mod: complete expecta A B expectb updatea DC Descriptor (a, expecta, updatea, b, expectb) Outcome: UCCE

98 DC Mod: tates 1 2 Init A:??? prep ok A: desc Outcome: UND Outcome: UND A was expecta (desc created) success 3 A: desc Outcome: UCC B was expectb prep fail fail 5 A:??? Outcome: FAIL A was!expecta 6 A: desc Outcome: FAIL 4 7 A: updatea A: expecta one tread till no loops!

99 Compare-And-wap N-words (CAN) The ultimate atomic update

100 CAN spec in pseudo-code For two words, for simplicity A B 1 2 fun <A,B> cas2( a: Ref<A>, expecta: A, updatea: A, b: Ref<B>, expectb: B, updateb: B): Boolean = 3 atomic { if (a.value == expecta && b.value == expectb) { } 4 5 a.value = updatea b.value = updateb true } else false

101 CAN: init descriptor expecta A B expectb updatea updateb DC Descriptor (a, expecta, updatea, b, expectb, updateb) Outcome: UNDECIDED

102 CAN: prepare (1) expecta A B expectb updatea CA updateb DC Descriptor (a, expecta, updatea, b, expectb, updateb) Outcome: UNDECIDED

103 CAN: prepare (2) expecta A B expectb updatea DC updateb DC Descriptor (a, expecta, updatea, b, expectb, updateb) Outcome: UNDECIDED Use DC to update B if Outcome == UNDECIDED

104 CAN: decide expecta A B expectb updatea updateb DC Descriptor (a, expecta, updatea, b, expectb, updateb) Outcome: UCCE CA outcome

105 CAN: complete (1) expecta A B expectb updatea CA updateb DC Descriptor (a, expecta, updatea, b, expectb, updateb) Outcome: UCCE

106 CAN: complete (2) expecta A B expectb updatea CA updateb DC Descriptor (a, expecta, updatea, b, expectb, updateb) Outcome: UCCE

107 CAN: tates A!= expecta A:??? B:??? O: UND 7 8 A:??? B:??? O: FAIL one tread B!= expectb A: desc B:??? O: UND A: desc B:??? O: FAIL DC 9 A: desc B: desc O: UND A: expecta B:??? O: FAIL 4 A: desc B: desc O: UCC revents from going back in this M 5 6 A: updatea B: desc O: UCC Init A: updatea B: updateb O: UCC descriptor is known to other (helping) threads

108 Using it in practice All the little things that matter

109 It is easy to combine multiple operations with DC/CAN that linearize on a CA with a descriptor parameters that are known in advance

110 Trivial example: Treiber stack expect TO 1 2 CA update 3 New node

111 Let s go deeper Into unpublished territory

112 Doubly linked list: insert last

113 Doubly linked list: insert (0) 2 N CA here H N 1 N T Operation Descriptor A ref:??? expecta: entinel updatea: Node #2 Outcome: UNDECIDED DC here is needed (always!)??? can fill in A before CA & update on retry We know expected value for CA in advance We know updated value for CA in advance

114 Doubly linked list: insert (1) 2 N H N 1 N T Operation Descriptor A ref:??? expecta: entinel updatea: Node #2 Outcome: UNDECIDED DC Descriptor affected node: #1 operation ref

115 Doubly linked list: insert (2) Helpers are a bound to stumble upon the same descriptor Competing inserts will complete (help) us first 2 N H N 1 N T CA can only succeed on last node Operation Descriptor A ref:??? expecta: entinel updatea: Node #2 Outcome: UNDECIDED DC Descriptor affected node: #1 operation ref

116 Doubly linked list: insert (3) 2 N H N 1 N T desc is updated after successful DC Operation Descriptor A ref: Node #1 expecta: entinel updatea: Node #2 Outcome: UNDECIDED DC Descriptor affected node: #1 operation ref

117 Doubly linked list: insert (4) 2 N H N 1 N T Operation Descriptor tays pointed until operation outcome is decided A ref: Node #1 expecta: entinel updatea: Node #2 Outcome: UNDECIDED

118 Doubly linked list: remove first

119 Doubly linked list: remove (0) R1 CA here H N 1 N 2 N T Operation Descriptor A ref:??? expecta:??? updatea: Rem[???] Outcome: UNDECIDED Both not known in advance Deterministic f(expecta)

120 Doubly linked list: remove (1) R1 H N 1 N 2 N T Operation Descriptor A ref:??? expecta:??? updatea: Rem[???] Outcome: UNDECIDED DC Descriptor affected node: #1 old value: #2 operation ref

121 Doubly linked list: remove (2) Cannot change w/o removal of #1 We don t support ushleft!!! R1 It locks what node we are to remove H N 1 N 2 N T Operation Descriptor A ref:??? expecta:??? updatea: Rem[???] Outcome: UNDECIDED DC Descriptor affected node: #1 old value: #2 operation ref

122 Doubly linked list: remove (3) R1 H N 1 N 2 N T desc is updated after successful DC Operation Descriptor A ref: Node #1 expecta: Node #2 updatea: Rem[#2] Outcome: UNDECIDED DC Descriptor affected node: #1 old value: #2 operation ref

123 Doubly linked list: remove (4) R1 H N 1 N 2 N T Operation Descriptor tays pointed until operation outcome is decided A ref: Node #1 expecta: Node #2 updatea: Rem[#2] Outcome: UNDECIDED

124 Closing notes All we care about is CA that linearizes operation ubsequent updates are helper moves Invoke regular help/correct functions erfect algorithm to combine with optional Hardware Transactional Memory (HTM)

125 Let s enjoy what we ve accomplished

126 References Kotlin language Kotlin coroutines support library

127 Thank you Any questions? me to elizarov at gmail relizarov