Programming Language Seminar Concurrency 2: Lock-free algorithms

Transcription

1 Programming Language Seminar Concurrency 2: Lock-free algorithms Peter Sestoft Friday

2 Outline for today Compare-and-swap instruction Atomic test-then-set operation Implemented directly in x86 and other hardware Treiber stack Michael & Scott non-blocking queue Correctness: Seq. consistency, linearizability Liveness: Non-blocking, lock-free Queue benchmarks Miniproject 2 2

3 Unsafe number range [lower,upper] public class NumberRange {! // INVARIANT: lower <= upper! private final AtomicInteger lower = new AtomicInteger(0);! private final AtomicInteger upper = new AtomicInteger(0);! public void setlower(int i) {! if (i > upper.get())! throw new IllegalArgumentException("can't set lower");! lower.set(i);!! public void setupper(int i) {! if (i < lower.get())! throw new IllegalArgumentException("can't set upper");! upper.set(i);!!! Test-then-set not atomic Unsafe, may violate invariant Goetz Listing

4 p 67 Immutable integer pairs private class IntPair {! // INVARIANT: lower <= upper! final int lower, upper;! public IntPair(int lower, int upper) {! this.lower = lower;! this.upper = upper;!!! Immutable, and safely publishable 5

5 Safe number range rep. public class CasNumberRange {! private final AtomicReference<IntPair> values! = new AtomicReference<IntPair>(new IntPair(0, 0));! public int getlower() { return values.get().lower;! public void setlower(int i) {! while (true) {! IntPair oldv = values.get();! if (i > oldv.upper)! throw new IllegalArgumentException("Can't set lower");! IntPair newv = new IntPair(i, oldv.upper);! if (values.compareandset(oldv, newv))! return;!!! Set if nobody else changed it Atomic replacement of one pair by another But may create many pairs before success... Goetz Listing

6 Compare-and-swap (CAS) Atomic test-then-set sequence (IBM 1970) Java AtomicReference<T> val.compareandset(t oldval, T newval): If val holds oldval, set it to newval and return true.net/cli System.Threading.Interlocked CompareExchange<T>(ref T loc, T newval, T oldval): If loc holds oldval, set it to newval and return true Implemented directly in x86 hardware See machine code in Interlocked.cs Optimistic concurrency Try to update, if it fails, retry (maybe) Just like database systems In contrast to pessimistic synchronized/lock 7

7 CAS has visibility effects Java's AtomicReference.compareAndSet etc have the same visibility effects as volatile: "The memory effects for accesses and updates of atomics generally follow the rules for volatiles" (java.util.concurrent.atomic package documentation) Also in C#/.NET/CLI, Ecma-335, I : "... atomic operations in the System.Threading.Interlocked class... perform implicit acquire/release operations" 8

8 Treiber's lock-free stack (1986) class ConcurrentStack <E> {! private static class Node <E> {! public final E item;! public Node<E> next;! public Node(E item) {! this.item = item;!!! Goetz Listing 15.6 AtomicReference<Node<E>> top = new AtomicReference<Node<E>>();!...!! top

9 Treiber's stack operations public void push(e item) {! Node<E> newhead = new Node<E>(item);! Node<E> oldhead;! do {! oldhead = top.get();! newhead.next = oldhead;! while (!top.compareandset(oldhead, newhead));!! Set top to new if not changed public E pop() {! Node<E> oldhead, newhead;! do {! oldhead = top.get();! if (oldhead == null)! return null;! newhead = oldhead.next;! while (!top.compareandset(oldhead, newhead));! return oldhead.item;!! Set top to next if not changed 10

10 CAS versus mutual exclusion (locks) Optimistic versus pessimistic concurrency Pro CAS Modern CAS is quite fast, cycles (CAS is used to implement lock acquisition itself) A failed CAS, unlike failed lock acquisition, requires no context switch, see Java Precisely p. 67 Therefore fast when contention is moderate Con CAS May fail arbitrarily many times Therefore slow when contention is very high Not all hardware implements CAS 11

11 Lock-based queue with sentinel private static class Node { final int item; volatile Node next; class SentinelLockQueue { private Node head, tail; public SentinelLockQueue() { head = tail = new Node(-444, null);... Invariants: Allocate sentinel node If empty: head==tail and head.next==null If non-empty: head!=tail, head.next is first item, tail points to last item File TestQueues.java 12

12 Lock-based queue operations public synchronized boolean put(int item) { Node node = new Node(item, null); tail.next = node; tail = node; return true; Enqueue at tail public synchronized int get() { if (head.next == null) return -999; Node first = head; head = first.next; return head.item; Dequeue from second node, becomes new sentinel Important property: Enqueue (put) updates tail but not head Dequeue (get) updates head but not tail 13

13 Michael-Scott lock-free queue with sentinel private static class Node { final int item; final AtomicReference<Node> next; class MSNonBlockingQueue1 { private final AtomicReference<Node> head, tail; public MSNonblockingQueue() { Node dummy = new Node(-444, null); head = new AtomicReference<Node>(dummy); tail = new AtomicReference<Node>(dummy); If non-empty: head.next is first item, tail points to last item ("quiescent state") or the second-last item ("intermediate state") 14

14 Michael-Scott queue operations Two-step enqueue at tail Two-step dequeue at head 15 Herlihy & Shavit ch 10

15 Michael-Scott enqueue (put) public boolean put(int item) { Node node = new Node(item, null); while (true) { Node last = tail.get(); Node next = last.next.get(); if (last == tail.get()) { if (next == null) { if (last.next.compareandset(next, node)) { tail.compareandset(last, node); return true; 2 else { tail.compareandset(last, next); Needed? 1 Quiescent, try add Success, try move tail Intermediate, try move tail "help another enqueuer" 16

16 Intermediate state and "help" Goetz et al. 17

17 Michael-Scott dequeue (get) public int get() { while (true) { Node first = head.get(), last = tail.get(), next = first.next.get(); if (first == head.get()) { Needed? if (first == last) { if (next == null) return -999; tail.compareandset(last, next); else { int result = next.item; 1 if (head.compareandset(first, next)) { return result; 2 Intermediate, try move tail (*) Try move head In Java or C#, but not C, (1) could go after (2) 18

18 Queue is empty A dequeues a B enqueues b (*) Why must dequeue mess with the tail? Scenario: B has set a.next=b but not yet tail=b A reads a's item A sets head=a.next Now tail item not reachable from head while (true) {... if (first == last) { if (next == null) return -999; tail.compareandset(last, next); else... Herlihy & Shavit 19

19 But... creates a lot of AtomicReference objects private static class Node { final int item; final AtomicReference<Node> next; OLD Must be CAS'able public Node(int item, Node next) { this.item = item; this.next = new AtomicReference<Node>(next); One AR per Node private static class Node { final int item; volatile Node next;... NEW Better, no AtomicReference object needed Instead, use an "updater" AtomicReferenceFieldUpdater<Node, Node> nextupdater = AtomicReferenceFieldUpdater.newUpdater(Node.class, Node.class, "next"); 20

20 Michael-Scott enqueue, using "updater" for.next public boolean put(int item) { Node node = new Node(item, null); while (true) { Node last = tail.get(); Node next = last.next; if (last == tail.get()) { if (next == null) { if (nextupdater.compareandset(last, next, node)) { tail.compareandset(last, node); return true; else { tail.compareandset(last, next); 21

21 CAS in Java versus.net.net uses static CAS methods in Interlocked Allows non-atomic access by mistake, bad Java's AtomicReference<T> seems safer Because must access the field through that class But, for efficiency, Java allows standard field access through AtomicReferenceFieldUpdater This is at least as bad as the.net design And gives poor tool support (IDE, refactoring,...) 22

22 Queue benchmarks Queue implementations Lock-based Lock-based, sentinel node Lock-free, sentinel node, AtomicReference Lock-free, sentinel node, Atomic...FieldUpdater Platforms Hotspot 64 bit Java 1.7.0_b147, Windows 7, Xeon W3505, 2.53GHz, 2 cores, 2009Q1 Hotspot 64 bit Java 1.6.0_37, MacOS, Core 2 Duo, 2.66GHz, 2 cores, 2008Q1 Icedtea Java 1.7.0_b21, Linux, Xeon E5320, 1.86GHz, 4/8 cores, 2006Q4 Hotspot 64 bit Java 1.7.0_25-b15, Linux, AMD Opteron 6386 SE, 32 cores, 2012Q4 Measurements probably flawed: the client threads do no useful work, only en/dequeue Nevertheless, big differences between machines 23

23 Java 1.7, Xeon W3505, 2 cores 6 5 Time as function of number of concurrent threads 4 3 LockQueue MSNonblockingQueue MSNonblockingQueueRefl SentinelLockQueue

24 Java 1.6, Core 2 Duo, 2 cores LockQueue MSNonblockingQueue MSNonblockingQueueRefl SentinelLockQueue

25 Java 1.7, Xeon E5320, 4/8 cores LockQueue MSNonblockingQueue MSNonblockingQueueRefl SentinelLockQueue

26 Java 1.7, AMD Opteron, 32 cores Very slow?! LockQueue MSNonblockingQueue MSNonblockingQueueRefl SentinelLockQueue

27 Gross performance bug in Goetz's Listing 15.7 Initial sentinel node is stored in a queue field class LinkedQueue<E> { private final Node<E> dummy = new Node<E>(null, null); private final AtomicReference<Node<E>> head = new AtomicReference<Node<E>>(dummy); private final AtomicReference<Node<E>> tail = new AtomicReference<Node<E>>(dummy);... This makes the initial sentinel live forever! Worse: It keeps all nodes ever allocated live! Because it links to them, directly or indirectly HUGE spaceleak, BAD performance Very bad 28

28 Obvious fix (after 8 hours) Don't bind the sentinel to a field: class LinkedQueue<E> { private final AtomicReference<Node<E>> head, tail; public LinkedQueue() { Node<E> dummy = new Node<E>(null, null); head = new AtomicReference<Node<E>>(dummy); tail = new AtomicReference<Node<E>>(dummy);... Now the sentinel dies as soon as the first item is dequeued No space leak, fine performance Polite sent to Mr. Goetz... no reply 29

29 Non-blocking, lock-free, wait-free Non-blocking: the delay (crash...) of one thread cannot delay another thread Generally, lock-based algorithms are not non-blocking Lock-free: some call will complete Michael & Scott queue operations are lock-free (But paper calls it non-blocking; obsolete terminology) Wait-free: every call will complete Michael & Scott queue operations are not wait-free; an enqueue could forever be delayed by other enqueues NB: An algorithm is not lock-free just because it uses no locks! (Why?) 30

30 Correctness notions (H&S ch 3): Quiescent consistency Principle Method calls should appear to happen in a onceat-a-time, sequential order Method calls separated by a period of quiescence should appear to take effect in their real-time order Method calls overlap in time Herlihy & Shavit 31

31 Principles Sequential consistency Method calls should appear to happen in a once-ata-time, sequential order Method calls should appear to take effect in program order Reads and writes on non-volatile fields fail this A B A q.enq(x) B q.enq(y) A q.deq(x) B q.deq(y) B q.enq(y) A q.enq(x) A q.deq(y) B q.deq(x) Two possible orders explaining the execution 32

32 Drawbacks of sequential consistency Not compositional: 33

33 Principles Linearizability Method calls should appear to happen in a onceat-a-time, sequential order Each method call should appear to take effect instantaneously at some moment between its invocation and response That "moment" is the linearization point of the method Linearizability implies sequential consistency 34

34 Lock-free is a bit difficult... java.util.concurrent.concurrentlinkeddequeue source comments (OpenJDK 7-b147): "We believe (without full proof) that all single-element deque operations (e.g., addfirst, peeklast, polllast) are linearizable (see Herlihy and Shavit's book). However, some combinations of operations are known not to be linearizable. In particular, when an addfirst(a) is racing with pollfirst() removing B, it is possible for an observer iterating over the elements to observe A B C and subsequently observe A C, even though no interior removes are ever performed. Nevertheless, iterators behave reasonably, providing the "weakly consistent" guarantees." 35

35 Week 1 Reading Read Goetz et al.: Java Concurrency in Practice, chapters 1, 2, 3, 4, 5 Look at Java Language Specification, section 17.4 and 17.5 Week 2 (this week) Goetz et al.: Java Concurrency in Practice, chapter 15 Michael and Scott: Simple, fast, and practical... Herlihy & Shavit: The Art of Multiprocessor Programming, chapters 3 (and 9) 36

36 Miniproject 2 Groves: Verifying Michael and Scott's Lock- Free Queue Algorithm Using Trace Reduction, 2008 Hand in to sestoft@itu.dk on 15 November see rules in miniproject-2.html 37