The Five-Fold Path. Queues & Stacks. Bounded vs Unbounded. Blocking vs Non-Blocking. Concurrent Queues and Stacks. Another Fundamental Problem

Transcription

1 Concurrent Queues and Stacks The Five-Fold Path Coarse-grained locking Fine-grained locking Optimistic synchronization Lazy synchronization Lock-free synchronization 2 Another Fundamental Problem Queues & Stacks We told you about Sets implemented using linked lists Next: queues Next: stacks Both: pool of items Queue enq() & deq() First-in-first-out (FIFO) order Stack push() & pop() Last-in-first-out (LIFO) order 3 4 Bounded vs Unbounded Blocking vs Non-Blocking Bounded Fixed capacity Good when resources an issue Unbounded Holds any number of objects Problem cases: Removing from empty pool Adding to full (bounded) pool Blocking Caller waits until state changes Non-Blocking Method throws exception 5 6 1

2 This Lecture Queue: Concurrency Bounded, Blocking, Lock-based Queue Unbounded, Non-Blocking, Lock-free Queue ABA problem Unbounded Non-Blocking Lock-free Stack Elimination-Backoff Stack enq(x) enq() and deq() work at different ends of the object y=deq() 7 8 Concurrency Bounded Queue enq(x) y=deq() Sentinel Challenge: what if the queue is empty or full? 9 10 Bounded Queue Bounded Queue First actual item Lock out other deq() calls

3 Bounded Queue Not Done Yet Lock out other enq() calls Need to tell whether queue is full or empty Not Done Yet Not Done Yet 8 Permission to enqueue 8 items 8 Incremented by deq() Decremented by enq() Enqueuer Enqueuer Lock Read 8 8 OK

4 Enqueuer Enqueuer No need to lock Enqueue Node Enqueuer Enqueuer Release lock getanddecrement() Enqueuer Unsuccesful Enqueuer 7 If queue was empty, notify/signal waiting dequeuers 0 Read Uh-oh

5 Dequeuer Dequeuer 8 Lock 7 Read sentinel s next field OK Dequeuer Dequeuer Make first Node new sentinel 7 Read value Dequeuer Dequeuer 8 Increment 7 Release

6 Unsuccesful Dequeuer Bounded Queue 8 Read sentinel s next field uh-oh public class BoundedQueue<T> { ReentrantLock, ; Condition notemptycondition, notfullcondition; AtomicInteger ; Node ; Node ; int capacity; = new ReentrantLock(); notfullcondition =.newcondition(); = new ReentrantLock(); notemptycondition =.newcondition(); Bounded Queue Digression: Monitor Locks public class BoundedQueue<T> { ReentrantLock, ; Condition notemptycondition, notfullcondition; AtomicInteger ; Node ; Node ; int capacity; Enq & deq locks = new ReentrantLock(); notfullcondition =.newcondition(); = new ReentrantLock(); notemptycondition =.newcondition(); Java Synchronized objects and Java ReentrantLocks are monitors Allow blocking on a condition rather than spinning Threads: acquire and release lock wait on a condition The Java Lock Interface The Java Lock Interface public interface Lock { void lock(); void lockinterruptibly() throws InterruptedException; boolean trylock(); boolean trylock(long time, TimeUnit unit); Condition newcondition(); void unlock; Acquire lock public interface Lock { void lock(); void lockinterruptibly() throws InterruptedException; boolean trylock(); boolean trylock(long time, TimeUnit unit); Condition newcondition(); void unlock; Release lock

7 The Java Lock Interface The Java Lock Interface public interface Lock { void lock(); void lockinterruptibly() throws InterruptedException; boolean trylock(); boolean trylock(long time, TimeUnit unit); Condition newcondition(); void unlock; Try for lock, but not too hard public interface Lock { void lock(); void lockinterruptibly() throws InterruptedException; boolean trylock(); boolean trylock(long time, TimeUnit unit); Condition newcondition(); void unlock; Create condition to wait on The Java Lock Interface Lock Conditions public interface Lock { void lock(); void lockinterruptibly() throws InterruptedException; boolean trylock(); boolean trylock(long time, TimeUnit unit); Condition newcondition(); void unlock; Guess what this method does? public interface Condition { void await(); boolean await(long time, TimeUnit unit); void signal(); void signalall(); Lock Conditions Lock Conditions public interface Condition { void await(); boolean await(long time, TimeUnit unit); void signal(); void signalall(); Release lock and wait on condition public interface Condition { void await(); boolean await(long time, TimeUnit unit); void signal(); void signalall(); Wake up one waiting thread

8 Lock Conditions Await public interface Condition { void await(); boolean await(long time, TimeUnit unit); void signal(); void signalall(); Wake up all waiting threads q.await() Releases lock associated with q Sleeps (gives up processor) Awakens (resumes running) Reacquires lock & returns Signal Signal All q.signal(); q.signalall(); Awakens one waiting thread Which will reacquire lock Awakens all waiting threads Which will each reacquire lock A Monitor Lock Unsuccessful Deq Lock() waiting room Lock() waiting room Critical Section unlock() Deq() Critical Section await() Oh no, Empty!

9 Lock() Another One waiting room Enqueur to the Rescue Lock() waiting room Yawn! Yawn! Deq() Critical Section await() Oh no, Empty! Enq( ) Critical Section signalall() unlock() Monitor Signalling waiting room Yawn! Yawn! Dequeurs Signalled waiting room Yawn! Critical Section Awakend thread might still lose lock to outside contender Critical Section Found it Dequeurs Signalled Dollar Short + Day Late waiting room Yawn! waiting room Critical Section Still empty! Critical Section

10 Lock() Lost Wake-Up Yawn! waiting room Lock() Lost Wake-Up Yawn! waiting room Enq( ) Critical Section signal () unlock() Enq( ) Critical Section unlock() Lost Wake-Up Yawn! Lost Wake-Up waiting room waiting room Critical Section Critical Section Found it What s Wrong Here? waiting room zzzz.! Solution to Lost Wakeup Always use signalall and notifyall Not signal and notify Critical Section

11 Java Synchronized Methods public class Queue<T> { int = 0,, = 0; T[QSIZE] items; public synchronized T deq() { while ( == 0) this.wait(); T result = items[ % QSIZE]; ++; this.notifyall(); return result; Java Synchronized Methods public class Queue<T> { int = 0,, = 0; T[QSIZE] items; public synchronized T deq() { while ( == 0) this.wait(); T result = items[ % QSIZE]; ++; this.notifyall(); return result; Each object has an implicit lock with an implicit condition Java Synchronized Methods public class Queue<T> { int = 0,, = 0; T[QSIZE] items; Lock on entry, unlock on return public synchronized T deq() { while ( == 0) this.wait(); T result = items[ % QSIZE]; ++; this.notifyall(); return result; Java Synchronized Methods public class Queue<T> { int = 0,, = 0; T[QSIZE] items; Wait on implicit condition public synchronized T deq() { while ( == 0) this.wait(); T result = items[ % QSIZE]; ++; this.notifyall(); return result; Java Synchronized Methods public class Queue<T> { int = 0,, = 0; T[QSIZE] items; Signal all threads waiting on condition public synchronized T deq() { while ( == 0) this.wait(); T result = items[ % QSIZE]; ++; this.notifyall(); return result; (Pop!) The Bounded Queue public class BoundedQueue<T> { ReentrantLock, ; Condition notemptycondition, notfullcondition; AtomicInteger ; Node ; Node ; int capacity; = new ReentrantLock(); notfullcondition =.newcondition(); = new ReentrantLock(); notemptycondition =.newcondition();

12 Bounded Queue Fields Bounded Queue Fields public class BoundedQueue<T> { ReentrantLock, ; Condition notemptycondition, notfullcondition; AtomicInteger ; Node ; Node ; int capacity; Enq & deq locks = new ReentrantLock(); notfullcondition =.newcondition(); = new ReentrantLock(); notemptycondition =.newcondition(); public class BoundedQueue<T> { ReentrantLock, ; Condition notemptycondition, notfullcondition; AtomicInteger ; Node ; Enq lock s associated Node ; condition int capacity; = new ReentrantLock(); notfullcondition =.newcondition(); = new ReentrantLock(); notemptycondition =.newcondition(); Bounded Queue Fields Bounded Queue Fields public class BoundedQueue<T> { ReentrantLock, ; Condition notemptycondition, notfullcondition; AtomicInteger ; Node ; Node ; int capacity; = new ReentrantLock(); notfullcondition =.newcondition(); = new ReentrantLock(); notemptycondition =.newcondition(); Num : 0 to capacity public class BoundedQueue<T> { ReentrantLock, ; Condition notemptycondition, notfullcondition; AtomicInteger ; Head and Tail Node ; Node ; int capacity; = new ReentrantLock(); notfullcondition =.newcondition(); = new ReentrantLock(); notemptycondition =.newcondition(); Enq Method Part One Enq Method Part One public void enq(t x) { boolean mustwakedequeuers = false;.lock(); try { while (.get() == 0) notfullcondition.await(); Node e = new Node(x);.next = e; = e; if (.getanddecrement() == capacity) mustwakedequeuers = true; finally {.unlock(); 71 public void enq(t x) { boolean mustwakedequeuers = false;.lock(); try { while (.get() == 0) notfullcondition.await(); Node e = new Node(x);.next = e; = e; if (.getanddecrement() == capacity) mustwakedequeuers = true; finally {.unlock(); Lock and unlock enq lock 72 12

13 Enq Method Part One public void enq(t x) { boolean mustwakedequeuers = false;.lock(); try { while (.get() == 0) notfullcondition.await(); Node e = new Node(x);.next = e; = e; if (.getanddecrement() == capacity) mustwakedequeuers = true; finally {.unlock(); If queue is full, patiently await further instructions 73 Be Afraid public void enq(t x) { boolean mustwakedequeuers = false;.lock(); try { while (.get() == 0) notfullcondition.await(); Node e = new Node(x);.next = e; = e; if (.getanddecrement() == capacity) mustwakedequeuers = true; finally {.unlock(); How do we know the field won t change? 74 Enq Method Part One Enq Method Part One public void enq(t x) { boolean mustwakedequeuers = false;.lock(); try { while (.get() == 0) notfullcondition.await(); Node e = new Node(x);.next = e; = e; if (.getanddecrement() == capacity) mustwakedequeuers = true; finally {.unlock(); Add new node 75 public void enq(t x) { boolean mustwakedequeuers = false;.lock(); try { while (.get() == 0) notfullcondition.await(); Node e = new Node(x);.next = e; = e; if (.getanddecrement() == capacity) mustwakedequeuers = true; finally {.unlock(); If queue was empty, wake frustrated dequeuers 76 Enq Method Part Deux Enq Method Part Deux public void enq(t x) { if (mustwakedequeuers) {.lock(); try { notemptycondition.signalall(); finally {.unlock(); public void enq(t x) { if (mustwakedequeuers) {.lock(); try { notemptycondition.signalall(); finally {.unlock(); Are there dequeuers to be signaled?

14 Enq Method Part Deux Enq Method Part Deux public void enq(t x) { if (mustwakedequeuers) {.lock(); try { notemptycondition.signalall(); finally {.unlock(); Lock and unlock deq lock public void enq(t x) { Signal dequeuers that queue if (mustwakedequeuers) no longer empty {.lock(); try { notemptycondition.signalall(); finally {.unlock(); The Enq() & Deq() Methods Split the Counter Share no locks That s good But do share an atomic counter Accessed on every method call That s not so good Can we alleviate this bottleneck? The enq() method Decrements only Cares only if value is zero The deq() method Increments only Cares only if value is capacity Split Counter A Lock-Free Queue Enqueuer decrements enqsidepermits Dequeuer increments deqsidepermits When enqueuer runs out Locks Transfers Intermittent synchronization Not with each method call Need both locks! (careful ) Sentinel

15 Compare and Set Enqueue CAS Enq( ) Enqueue Logical Enqueue CAS Physical Enqueue Enqueue CAS Enqueue Node These two steps are not atomic The field refers to either Actual last Node (good) Penultimate Node (not so good) Be prepared!

16 Enqueue When CASs Fail What do you do if you find A trailing? Stop and help fix it If node has non-null next field CAS the queue s field to.next As in the universal construction During logical enqueue Abandon hope, restart Still lock-free (why?) During physical enqueue Ignore it (why?) Dequeuer Dequeuer Make first Node new sentinel CAS Read value Memory Reuse? What do we do with nodes after we dequeue them? Java: let garbage collector deal? Suppose there is no GC, or we prefer not to use it? CAS Dequeuer Can recycle

17 Simple Solution Each thread has a free list of unused queue nodes Allocate node: pop from list Free node: push onto list Deal with underflow somehow Why Recycling is Hard Want to redirect from zzz grey to red Free pool Both Nodes Reclaimed One Node Recycled zzz Yawn! Free pool Free pool Why Recycling is Hard CAS Final State Bad news OK, here I go! zomg what went wrong? Free pool Free pool

18 The Dreaded ABA Problem Dreaded ABA continued Head pointer has value A Thread reads value A zzz Head pointer has value B Node A freed Dreaded ABA continued Dreaded ABA continued CAS Yawn! Head pointer has value A again Node A recycled & reinitialized CAS succeeds because pointer matches even though pointer s meaning has changed The Dreaded ABA Problem Dreaded ABA A Solution Is a result of CAS() semantics I blame Sun, Intel, AMD, Not with Load-Locked/Store- Conditional Good for IBM? Tag each pointer with a counter Unique over lifetime of node Pointer size vs word size issues Overflow? Don t worry be happy? Bounded tags? AtomicStampedReference class

19 Atomic Stamped Reference AtomicStampedReference class Java.util.concurrent.atomic package Reference Can get reference and stamp atomically address S Concurrent Stack Methods push(x) pop() Last-in, First-out (LIFO) order Lock-Free! Stamp Empty Stack Push Top Top Push Push Top CAS Top

20 Push Push Top Top Push Push Top CAS Top Pop Pop Top Top CAS

21 Pop Pop Top CAS Top CAS mine! Pop Lock-free Stack Top public class LockFreeStack { private AtomicReference top = new AtomicReference(null null); public boolean trypush(node node){ Node oldtop = top.get(); node.next = oldtop; return(top.compareandset(oldtop, node)) public void push(t value) { Node node = new Node(value); while (true true) ) { if (trypush(node)) { else backoff.backoff(); Lock-free Stack Lock-free Stack public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public Boolean trypush(node node){ Node oldtop = top.get(); node.next = oldtop; return(top.compareandset(oldtop, node)) public void push(t value) { Node node = new Node(value); while (true) { if (trypush(node)) { trypush else backoff.backoff() attempts to push a node public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean trypush(node node){ Node oldtop = top.get(); node.next = oldtop; return(top.compareandset(oldtop, node)) public void push(t value) { Node node = new Node(value); while (true) { if (trypush(node)) { Read top value else backoff.backoff()

22 Lock-free Stack Lock-free Stack public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean trypush(node node){ Node oldtop = top.get(); node.next = oldtop; return(top.compareandset(oldtop, node)) public void push(t value) { Node node = new Node(value); while (true) { if (trypush(node)) { else backoff.backoff() current top will be new node s successor 127 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean trypush(node node){ Node oldtop = top.get(); node.next = oldtop; return(top.compareandset(oldtop, node)) public void push(t value) { Node node = new Node(value); while (true) { if (trypush(node)) { else backoff.backoff() Try to swing top, return success or failure 128 Lock-free Stack Lock-free Stack public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean trypush(node node){ Node oldtop = top.get(); node.next = oldtop; return(top.compareandset(oldtop, node)) public void push(t value) { Node node = new Node(value); while (true) { if (trypush(node)) { else backoff.backoff() Push calls trypush public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean trypush(node node){ Node oldtop = top.get(); node.next = oldtop; return(top.compareandset(oldtop, node)) public void push(t value) { Node node = new Node(value); while (true) { if (trypush(node)) { else backoff.backoff() Create new node Lock-free Stack Makes scheduling benevolent so public class LockFreeStack { Method is effectively wait- private AtomicReference top = new AtomicReference(null); free public boolean trypush(node node){ Node oldtop If = trypush() top.get(); fails, node.next = oldtop; return(top.compareandset(oldtop, back off before retrying node)) public void push(t value) { Node node = new Node(value); while (true) { if (trypush(node)) { else backoff.backoff() 131 Lock-free Stack Good No locking Bad Without GC, fear ABA Without backoff, huge contention at top In any case, no parallelism

23 Big Question Elimination-Backoff Stack Are stacks inherently sequential? Reasons why Every pop() call fights for top item Reasons why not Stay tuned How to turn contention into parallelism Replace familiar exponential backoff With alternative elimination-backoff Observation Idea: Elimination Array Push( ) linearizable stack Pick at random Push( ) stack Pop() Yes! After an equal number of pushes and pops, stack stays the same Pop() Pick at random Elimination Array Push Collides With Pop continue No Collision Push( ) stack Push( ) stack Pop() continue Yes! No need to access stack Pop() If pushes collide or If pops no collision, collide access access stack stack

24 Elimination-Backoff Stack Elimination-Backoff Stack Lock-free stack + elimination array Access Lock-free stack, If uncontended, apply operation if contended, back off to elimination array and attempt elimination Push( ) Pop() CAS Top If CAS fails, back off Dynamic Range and Delay Linearizability Push( ) Pick random range and max time to wait for collision based on level of contention encountered Un-eliminated calls linearized as before Eliminated calls: linearize pop() immediately after matching push() Combination is a linearizable stack Un-Eliminated Linearizability Eliminated Linearizability Collision Point pop(v 1 ) pop(v 1 ) push(v 1 ) push(v 1 ) pop(v 2 ) push(v 2 ) Red calls are eliminated time time

25 Backoff Has Dual Effect Elimination introduces parallelism Backoff onto array cuts contention on lock-free stack Elimination in array cuts down total number of threads ever accessing lock-free stack Elimination Array public class EliminationArray { private static final int duration =...; private static final int timeunit =...; Exchanger<T>[] exchanger; public EliminationArray(int int capacity) { exchanger = new Exchanger[capacity]; for (int i = 0; ; i < capacity; i++) exchanger[i] = new Exchanger<T>(); Elimination Array public class EliminationArray { private static final int duration =...; private static final int timeunit =...; Exchanger<T>[] exchanger; public EliminationArray(int capacity) { exchanger = new Exchanger[capacity]; for (int i = 0; ; i < capacity; i++) exchanger[i] = new Exchanger<T>(); An array of exchangers A Lock-Free Exchanger public class Exchanger<T> { AtomicStampedReference<T> slot = new AtomicStampedReference<T>(null null, 0); A Lock-Free Exchanger public class Exchanger<T> { AtomicStampedReference<T> slot = new AtomicStampedReference<T>(null, 0); Atomically modifiable reference + time stamp Atomic Stamped Reference AtomicStampedReference class Java.util.concurrent.atomic package In C or C++: Reference address S Stamp

26 Extracting Reference & Stamp Extracting Reference & Stamp Public T get(int[] stampholder); Public T get(int[] stampholder); Returns reference to object of type T Returns stamp at array index 0! Exchanger Status enum Status {EMPTY, WAITING, BUSY; Exchanger Status enum Status {EMPTY { EMPTY, WAITING, BUSY; Nothing yet Exchange Status Exchange Status enum Status {EMPTY, { WAITING, BUSY; enum Status {EMPTY, { WAITING, BUSY; Nothing yet Nothing yet One thread is waiting for rendez-vous One thread is waiting for rendez-vous Other threads busy with rendez-vous

27 The Exchange The Exchange public T Exchange(T myitem, long nanos) throws TimeoutException { long timebound = System.nanoTime() + nanos; int[] stampholder = {EMPTY; while (true true) ) { if (System.nanoTime() > timebound) throw new TimeoutException(); T int stamp = stampholder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging public T Exchange(T myitem, long nanos) throws TimeoutException { long timebound = System.nanoTime() + nanos; int[] stampholder = {EMPTY; while (true) { if (System.nanoTime() > timebound) throw new TimeoutException(); T int stamp = stampholder[0]; switch(stamp) { Item & timeout case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging The Exchange The Exchange public T Exchange(T myitem, long nanos) throws TimeoutException { long timebound = System.nanoTime() + nanos; int[] stampholder = {EMPTY; while (true) { if (System.nanoTime() > timebound) throw new TimeoutException(); T int stamp = stampholder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging Array to hold timestamp public T Exchange(T myitem, long nanos) throws TimeoutException { long timebound = System.nanoTime() + nanos; int[] stampholder = {0; { while (true true) ) { if (System.nanoTime() > timebound) throw new TimeoutException(); T int stamp = stampholder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging Loop until timeout The Exchange The Exchange public T Exchange(T myitem, long nanos) throws TimeoutException { long timebound = System.nanoTime() + nanos; int[] stampholder = {0; { while (true) { if (System.nanoTime() > timebound) throw new TimeoutException(); T int stamp = stampholder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging Get other s item and timestamp public T Exchange(T myitem, long nanos) throws TimeoutException { long timebound = System.nanoTime() + nanos; Exchanger int[] stampholder slot has = three {0; { states while (true) { if (System.nanoTime() > timebound) throw new TimeoutException(); T int stamp = stampholder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging

28 Lock-free Exchanger Lock-free Exchanger EMPTY CAS EMPTY Lock-free Exchanger Lock-free Exchanger In search of partner WAITING WAITING Try to Lock-free Exchanger Still waiting exchange item and set state to BUSY Lock-free Exchanger Partner showed up, take item and reset to EMPTY Slot Slot CAS WAITING BUSY item stamp/state

29 Lock-free Exchanger Partner showed up, take item and reset to EMPTY item Slot EMPTY BUSY stamp/state 169 Exchanger State EMPTY case EMPTY: // slot is free if (slot.compareandset(heritem, myitem, EMPTY, WAITING)) { while (System.nanoTime() < timebound){ if (stampholder[0] ] == BUSY) { slot.set(null, EMPTY); if (slot.compareandset(myitem, null, WAITING, EMPTY)){throw new TimeoutException(); else { slot.set(null, EMPTY); break; 170 Exchanger State EMPTY Exchanger State EMPTY case EMPTY: // slot is free if (slot.compareandset(heritem, myitem, EMPTY, WAITING)) { while (System.nanoTime() < timebound){ if (stampholder[0] ] == BUSY) { slot.set(null, EMPTY); if (slot.compareandset(myitem, null, WAITING, EMPTY)){throw new TimeoutException(); else { Slot is free, try to insert slot.set(null, myitem and EMPTY); change state return to WAITING heritem; break; 171 case EMPTY: // slot is free if (slot.compareandset(heritem, myitem, EMPTY, WAITING)) { while (System.nanoTime() < timebound){ if (stampholder[0] ] == BUSY) { slot.set(null, EMPTY); if (slot.compareandset(myitem, null, WAITING, EMPTY)){throw new TimeoutException(); else { Loop while still time left to slot.set(null, attempt exchange EMPTY); break; 172 Exchanger State EMPTY case EMPTY: // slot is free if (slot.compareandset(heritem, myitem, WAITING, BUSY)) { while (System.nanoTime() < timebound){ if (stampholder[0] ] == BUSY) { slot.set(null, EMPTY); if (slot.compareandset(myitem, null, WAITING, EMPTY)){throw new TimeoutException(); else { Get item and stamp in slot slot.set(null, and check EMPTY); if state changed return to BUSY heritem; break; 173 Exchanger State EMPTY case EMPTY: // slot is free if (slot.compareandset(heritem, myitem, EMPTY, WAITING)) { while (System.nanoTime() < timebound){ if (stampholder[0] ] == BUSY) { slot.set(null, EMPTY); if (slot.compareandset(myitem, null, WAITING, EMPTY)){throw new TimeoutException(); else { If successful reset slot slot.set(null, state to EMPTY); break;

30 Exchanger State EMPTY case EMPTY: // slot is free if (slot.compareandset(heritem, myitem, WAITING, BUSY)) { while (System.nanoTime() < timebound){ if (stampholder[0] ] == BUSY) { slot.set(null, EMPTY); if (slot.compareandset(myitem, null, WAITING, EMPTY)){throw new TimeoutException(); else { slot.set(null, EMPTY); and return item found in slot break; 175 Exchanger State EMPTY case EMPTY: // slot is free Otherwise if (slot.compareandset(heritem, we ran out of myitem, EMPTY, WAITING)) time, try { to reset state to while (System.nanoTime() < timebound){ EMPTY, if successful time out if (stampholder[0] ] == BUSY) { slot.set(null, EMPTY); if (slot.compareandset(myitem, null, WAITING, EMPTY)){throw new TimeoutException(); else { slot.set(null, EMPTY); break; 176 Exchanger State EMPTY case EMPTY: // slot is free If reset if (slot.compareandset(heritem, failed, someone showed myitem, up WAITING, BUSY)) after { all, take that item while (System.nanoTime() < timebound){ if (stampholder[0] ] == BUSY) { slot.set(null, EMPTY); if (slot.compareandset(myitem, null, WAITING, EMPTY)){throw new TimeoutException(); else { slot.set(null, EMPTY); break; 177 Exchanger State EMPTY case EMPTY: // slot is free Set if slot (slot.compareandset(heritem, to EMPTY with new myitem, EMPTY, WAITING)) timestamp { and return item found while (System.nanoTime() < timebound){ if (stampholder[0] ] == BUSY) { slot.set(null, EMPTY); if (slot.compareandset(myitem, null, WAITING, EMPTY)){throw new TimeoutException(); else { slot.set(null, EMPTY); break; 178 Exchanger State EMPTY States WAITING and BUSY case EMPTY: // slot is free if (slot.compareandset(heritem, myitem, EMPTY, WAITING)) { while (System.nanoTime() < timebound){ if (stampholder[0] ] == BUSY) { slot.set(null, If EMPTY); initial CAS failed then if (slot.compareandset(myitem, null, WAITING, EMPTY)){throw new TimeoutException(); so retry from start else { slot.set(null, EMPTY); break; 179 someone else changed state from EMPTY to WAITING case WAITING: // someone waiting for me if (slot.compareandset(heritem, myitem, WAITING, BUSY)) break; case BUSY: // others in middle of exchanging break; default: : // impossible break;

31 States WAITING and BUSY case WAITING: // someone waiting for me if (slot.compareandset(heritem, myitem, WAITING, BUSY)) break; case BUSY: // others in middle of exchanging break; default: state // impossible WAITING means break; someone is waiting for an exchange, so attempt to CAS my item in and change state to BUSY States WAITING and BUSY case WAITING: // someone waiting for me if (slot.compareandset(heritem, myitem, WAITING, BUSY)) break; case BUSY: // others in middle of exchanging break; default: // impossible break; If successful return her item, state is now BUSY, otherwise someone else took her item so try again from start States WAITING and BUSY case WAITING: // someone waiting for me if (slot.compareandset(heritem, myitem, WAITING, BUSY)) break; case BUSY: // others in middle of exchanging break; default: // impossible If state is BUSY then break; some other threads are using slot to exchange so start again The Exchanger Slot Exchanger is lock-free Because the only way an exchange can fail is if others repeatedly succeeded or no-one showed up The slot we need does not require symmetric exchange Elimination Array Elimination Array public class EliminationArray { public T visit(t value, int Range) throws TimeoutException { int slot = random.nextint(range); int nanodur = converttonanos(duration, timeunit)); return (exchanger[slot].exchange(value, nanodur) public class EliminationArray { public T visit(t value, int Range) throws TimeoutException { int slot = random.nextint(range); int nanodur = converttonanos(duration, timeunit)); return (exchanger[slot].exchange(value, nanodur) visit the elimination array with a value and a range (duration to wait is not dynamic)

32 Elimination Array Elimination Array public class EliminationArray { Pick a random array entry public T visit(t value, int Range) throws TimeoutException { int slot = random.nextint(range); int nanodur = converttonanos(duration, timeunit)); return (exchanger[slot].exchange(value, nanodur) public class EliminationArray { public T visit(t Exchange value, value int or Range) time throws out TimeoutException { int slot = random.nextint(range); int nanodur = converttonanos(duration, timeunit)); return (exchanger[slot].exchange(value, nanodur) Elimination Stack Push Elimination Stack Push public void push(t value) {... while (true true) ) { if (trypush(node)) { else try { T othervalue = eliminationarray.visit(value,policy.range); if (othervalue == null) ) { public void push(t value) {... while (true) { if (trypush(node)) { else try { T othervalue = eliminationarray.visit(value,policy.range); if (othervalue == null) { First try to push Elimination Stack Push Elimination Stack Push public void push(t value) {... while If (true) failed { back-off to try to eliminate if (trypush(node)) { else try { T othervalue = eliminationarray.visit(value,policy.range); if (othervalue == null) { public void push(t value) {... while Value (true) being { pushed and range to try if (trypush(node)) { else try { T othervalue = eliminationarray.visit(value,policy.range); if (othervalue == null) {

33 Elimination Stack Push public void push(t value) {... Only a pop has null value while (true) { so elimination was successful if (trypush(node)) { else try { T othervalue = eliminationarray.visit(value,policy.range); if (othervalue == null) { Elimination Stack Push public void push(t value) {... Else retry push on lock-free stack while (true) { if (trypush(node)) { else try { T othervalue = eliminationarray.visit(value,policy.range); if (othervalue == null) { Elimination Stack Pop Elimination Stack Pop public T pop() {... while (true true) ) { if (trypop()) { return returnnode.value; else try { T othervalue = eliminationarray.visit(null,policy.range; if othervalue!= null) ) { return othervalue; public T pop() {... Same as push, if non-null other while (true) { if (trypop()) thread must { have pushed return so elimnation returnnode.value; succeeds else try { T othervalue = eliminationarray.visit(null,policy.range; if ( othervalue!= null) ) { return othervalue; Summary We saw both lock-based and lockfree implementations of queues and stacks Don t be quick to declare a data structure inherently sequential Linearizable stack is not inherently sequential ABA is a real problem, pay attention 197 This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License. You are free: to Share to copy, distribute and transmit the work to Remix to adapt the work Under the following conditions: Attribution. You must attribute the work to The Art of Multiprocessor Programming (but not in any way that suggests that the authors endorse you or your use of the work). Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights. 33