Concurrency in Object Oriented Programs 3. Object-Oriented Software Development COMP4001 CSE UNSW Sydney Lecturer: John Potter

Transcription

1 Concurrency in Object Oriented Programs 3 Object-Oriented Software Development COMP4001 CSE UNSW Sydney Lecturer: John Potter

2 Outline Explicit Locks in Java Techniques for Concurrent Reading State Dependent Access Deadlocks and Lock Ordering 2

3 Implicit Locks in Java recall: every object has an intrinsic monitor only one thread can enter a monitor at a time provides single threaded access to critical sections of code provides mutual exclusion synchronized statements enforce structured use of the monitor implicit, complementary monitor enter and monitor exit operations locks are automatically released at end of code block/ method, even if exceptions are thrown object monitors are reentrant one thread may enter the implicit monitor more than once object monitors are released while an object is waiting see State Dependent Access in these slides 3

4 Explicit Locks in Java explicit mutex locks c.f. instrinsic monitor extra lock functionality nonblocking acquire client must handle failure to acquire interruptible acquire a blocked attempt may be interrupted acquire with timeout more flexible use without lexical scope e.g. controlling concurrent top-down access to branches of a tree per-node locks must acquire child lock before release of parent lock so-called hand-over-hand or chain locking ensures no overtaking down any branch of a tree but note that tree root is a potential point of contention for concurrent access may support Condition objects see later 4

5 In java.util.concurrent.locks public interface Lock { void lock (); void unlock (); boolean trylock (); boolean trylock (long time, TimeUnit unit) throws InterruptedException; void lockinterruptibly () throws InterruptedException; Condition newcondition (); public class ReentrantLock implements Lock {... 5

6 Use of Explicit Locks in Java programmer is responsible for correct management of locks standard code idiom lock acquire try protected access finally lock release guarantees release of lock, even with exceptions Java code outline: Lock l =...; // e.g. new ReentrantLock () l.lock(); try { // access the resource protected by this lock finally { l.unlock(); 6

7 Mutual Exclusion vs Immutability mutual exclusion in Java objects with methods protected by mutual exclusion are updatable but are NOT concurrently accessible immutable objects no locking required objects, after being "safely" constructed are concurrently accessible but are NOT updatable both approaches can prevent race conditions what about concurrent access to read-mostly objects? updates are infrequent write/read ratio is low but > 0 cannot use immutable objects mutual exclusion is too strong we want concurrent reads 7

8 Outline Explicit Locks in Java Techniques for Concurrent Reading State Dependent Access Deadlocks and Lock Ordering 8

9 Concurrent Reading two approaches may yield better performance when reads outnumber writes read-write locks holding write permission blocks other reads and writes holding read permission blocks writes, not reads copy-on-write only mutable data is a (volatile) reference to an immutable object reads require no synchronization all read methods must only access the reference once to avoid data races with any write methods writes create a new immutable object, possibly reading the reference and reassigning it writes do require synchronization active reads may still be accessing the old data they continue to see the old state of the object see examples in j.u.c package CopyOnWriteArrayList CopyOnWriteArraySet 9

10 Read-Write Locks allow a resource to be accessed by multiple readers or a single writer at a time, but not both allow a greater level of concurrency in accessing shared data than a mutex especially when locks may be held for a long time and most operations are read only in java.util.concurrent.locks: public interface ReadWriteLock { Lock readlock(); Lock writelock(); public class ReentrantReadWriteLock implements ReadWriteLock {... 10

11 Read-Write Lock Example public class Table { private final Map<String, Data> map; private final ReadWriteLock lock = new ReentrantReadWriteLock(); private final Lock r = lock.readlock(); private final Lock w = lock.writelock(); public Data get(string key) { r.lock(); try { return map.get(key finally { r.unlock(); public void put(string key, Data value) { w.lock(); try { map.put(key, value); finally { w.unlock(); Mul3ple threads may run get() Only one thread may run put() 11

12 Read-Write vs Mutex RW offers increased concurrency by allowing concurrent reads but if all reads are short-lived, and there are frequent writes overhead of RW lock may not be justifiable code techniques to allow simple RW/mutex swaps with minimal code change: choose a RWLock factory which makes either a RW lock, e.g. a standard ReentrantReadWriteLock or a mutex, via a RWLock decorator for a standard (mutex) ReentrantLock where the read and write locks are the same use decorators to construct synchronized code synchronized vs explicit mutex vs read-write choice of decorator is when synchronized object is constructed 12

13 Read-Write Lock Policies when both readers and writers are waiting, should a read or a write be granted? should access be granted in-order, or fairly? when a reader is active and a writer is waiting, should a new read be granted (overtaking the waiting writer) reader preference risks writer starvation writer preference may reduce concurrency reentrancy for RW locks: what does it mean? can a thread with a write lock reacquire it? Can it acquire the read lock? can a thread with a read lock reacquire it? can locks be downgraded (from write to read) or upgraded (from read to write), with no intervention by other threads? 13

14 Read-Write Locks and Starvation reading threads may acquire a lock if the lock is held for reading by another thread when will a writer be able to grab the lock? maybe never it has to wait for ALL readers to finish, but before then new readers can always grab the lock and run a solution: writer s preference implemented in Java's j.u.c.l.reentrantreadwritelock do not grant the read lock to a new thread if there is a writer waiting for the lock an optional fairness flag gives the longest-waiting thread priority for locks j.u.c.l. is abbrevia3on for package name: java.u3l.concurrent.locks 14

15 Outline Explicit Locks in Java Techniques for Concurrent Reading State Dependent Access Deadlocks and Lock Ordering 15

16 State Dependence in sequential programming a method precondition specifies the state an object must be in for a call to be correct the client code is responsible for ensuring the target object is in a suitable state or otherwise not calling the method in concurrent programming client code cannot control the state of the target object except by acquiring a mutex, which sacrifices concurrency otherwise, independent threads may modify the object instead clients need to wait for the required condition to occur 16

17 Examples of State Dependent Operations operations on collections, streams, databases remove an element from an empty queue operations on objects maintaining constrained values withdraw money from an empty bank account operations requiring resources print a file operations requiring particular message orderings read an unopened file operations on external controllers shift to reverse gear in a moving car 17

18 Bounded Buffer public class BoundedBuffer { final Object[] buf; final int capacity; int head, tail, count; public BoundedBuffer(int cap){ capacity = cap; buf = new Object[capacity]; head = tail = count = 0; public int count() { return count; public int capacity(){ return capacity; public void put(object item) throws Failure { if (count == capacity) throw new Failure(); else { // store item at tail count++; buf[tail] = item; tail = (tail+1) % capacity; public Object remove() throws Failure { if (count == 0) throw new Failure(); else { // remove item at head Object item = buf[head]; count_--; head = (head+1) % capacity; return item; 18

19 Safety in Bounded Buffer typical usage of the bounded buffer in a sequential environment: caller checks precondition before making call via check-and-act: if (buffer.count()<buffer.capacity()) { buffer.put(item); if (buffer.count()>0) { item = buffer.get(); check-and-act can often fail in concurrent environments condition may be invalidated by a concurrent thread so an action should not go ahead even though its check succeeded condition may be made valid by a concurrent thread so an action could have gone ahead even when its check failed concurrent puts or removes may interfere with each other because they update buffer data simple locks (mutex or RW) can provide safety but clients will still need to synchronize the check-and-act code themselves 19

20 Interfaces and Policies public interface BoundedBuffer { int capacity(); // Inv: capacity() > 0 int count(); // Inv: 0 <= count() <= capacity() // Init: count() == 0 void put(object x); // Pre: count() < capacity() Object remove(); // Pre: count() > 0 interfaces alone cannot convey policy but can suggest policy for example, remove() throws an exception public Object remove() throws Failure { different methods can support different policies can also use annotations indicate whether action is state dependent with design by contract support: make preconditions explicit 20

21 State Dependence operations may succeed only if object is in particular logical state choose the policy (precondition) for the operation interfaces or protocols reflect policy operations check the state they depend on to implement policy 21

22 Policies for State Dependent Actions Blind action: Proceed regardless; no guarantee of outcome Inaction: Ignore request Balking: Report failure Guarding: Wait until in right state Trying: Proceed anyway, check if action succeeded; if not fail Retrying: Keep trying until success Timing out: Wait or retry for a while; then fail Planning: Try to achieve the right state 22

23 Balking check state upon method entry fail immediately if not in the right state responsibility for handling failure thrown to client usable in both concurrent and sequential contexts 23

24 Bounded Buffer with Balking public class BoundedBuffer { final Object[] buf; final int capacity; int head, tail, count; public BoundedBuffer(int cap){ capacity = cap; buf = new Object[capacity]; head = tail = count = 0; public int count() { return count; public int capacity(){ return capacity; synchronized public void put(object item) throws Failure { if (count == capacity) throw new Failure(); else { // store item at tail count++; buf[tail] = item; tail = (tail+1) % capacity; synchronized public Object remove() throws Failure { if (count == 0) throw new Failure(); else { // remove item at head Object item = buf[head]; count_--; head = (head+1) % capacity; return item; 24

25 Propagating Precondition Failure BoundBuffer buffer; while (true) { try { to Callers item = buffer.get(); // throw Failure if buffer is empty break; // if ok, break the loop catch (Failure e) { Thread.sleep(SLEEP_GRANULARITY); // pause before retry // use item Callers have to deal with precondition failures themselves 25

26 Guarding callers do not have to deal with precondition failure the retry mechanism is encapsulated within the methods the calling thread waits until the precondition is true requires another thread to make an update that enables the precondition the waiting thread needs to be notified of the update to allow the precondition to be checked again client code does not implement the retry logic on each call but clients need to be aware that calls may wait 26

27 Guarding locks cause blocking locks cannot be acquired if held by other threads guards cause waiting appropriate for state dependent actions wait until some action precondition is true need other threads to make updates in a sequential context, guarding is useless because no other thread can change the state once waiting, will wait for ever so a client must ensure precondition is true before making a call 27

28 Guarding Mechanisms busy waiting thread continually spins waiting for condition while (!condition) ; // spin consumes CPU time if the condition does not change requires multiple-cpu s or timeslicing it is inappropriate for Java application programming suspension thread stops execution until notified that the condition may be true supported in Java via wait-sets and locks wait(), notify(), notifyall() 28

29 Guarding via wait/notify Java supports guarding with suspension waiting for a condition: synchronized(foo) { while (!condition) { try { foo.wait(); catch ( InterruptedException ex) { condition is true // proceed changing a condition: synchronized(foo) { condition = true; foo.notifyall(); // or foo.notify(); 29

30 Wait-sets and Notification every Java object has a wait-set can only be manipulated while the object lock is held (via synchronized statements) otherwise, IllegalMonitorStateException is thrown threads enter the wait-set by invoking wait() wait() releases the current object lock and the thread is placed in the objects wait-set (and waits) includes releasing a lock held multiple times (via re-entries) no other locks held by the thread are released (caution!) lock is reacquired before wait() returns lock count is restored optional timed-wait: wait(long millisecond) 30

31 Wait-sets and Notification wait(long) causes current thread to wait till another thread invokes the notify()/notifyall() method for this object or a specified amount of time has elapsed threads are released from the wait-set when other threads invoke: notify() releases a single thread from the wait-set which thread is not defined, so avoid using this! only OK to use when you know at most thread can be waiting notifyall() releases all threads 31

32 Bounded Buffer with Guarding synchronized public void put(object item) { while (count == capacity) { try { wait(); catch (InterruptedException e){ count++; buf[tail] = item; // store item at tail tail = (tail+1) % capacity; notifyall(); // no longer empty synchronized public Object get() { while (count == 0) { try { wait(); catch (InterruptedException e){ // remove item at head Object item = buf[head]; count--; head = (head+1) % capacity; notifyall(); // no longer full return item; 32

33 Condition Variables in Java there is only one wait-set per object with notifyall() there is no way to know which threads should be released from the wait-set so they are all released condition variables provide separate wait-sets for different conditions in j.c.u.l a Condition belongs to an explicit Lock a Condition factors out the wait-set methods: called await, signal, signalall like wait, notify, notifyall multiple Conditions provide ability to associate multiple wait-sets with an object explicit conditions allow for specific notifications using signal instead of signalall only one thread is released per signal ensure that the thread must be waiting on that condition and nothing else if a thread may wait on some other condition then a woken thread may await again the notification has been lost in general: avoid specific notification if there are multiple conditions that may cause a thread to wait use signalall to avoid lost notifications in such cases 33

34 Bounded Buffer with Conditions public class BoundedBuffer { final Lock lock = new ReentrantLock(); final Condition notfull = lock.newcondition(); final Condition notempty = lock.newcondition(); final Object[] buf; final int capacity; int head, tail, count; public BoundedBuffer(int cap){ capacity = cap; buf = new Object[capacity]; head = tail = count = 0; public int count() { return count; public int capacity(){ return capacity; public void put(object item) throws InterruptedException { lock.lock(); try { while (count == capacity) notfull.await(); count++; buf[tail] = item; tail = (tail+1) % capacity; notempty.signal(); finally { lock.unlock(); public Object get() throws InterruptedException { lock.lock(); try { while (count == 0) notempty.await(); Object item = buf[head]; count--; head = (head+1) % capacity; notfull.signal(); return item; finally { lock.unlock(); 34

35 Outline Explicit Locks in Java Techniques for Concurrent Reading State Dependent Access Deadlocks and Lock Ordering 35

36 Deadlocks Liveness is a concurrent application's ability to execute in a timely manner tension between safety and liveness safety is achieved via controls that block execution this inhibits liveness Deadlock occurs in a program when some threads are permanently blocked occurs when 2 or more threads are blocked on a shared resource held by another Java applications do not recover from deadlock so we must design to avoid the possibility of deadlock 36

37 Four Conditions for Deadlock Serially reusable resources a resource is either assigned to one thread or it is available No pre-emption only a thread holding a resource may release it Incremental acquisition threads already holding resources may request new resources Wait-for cycle two or more threads form a circular chain 37

38 Preventing Deadlock Removing mutual exclusion no thread may have exclusive access to a resource but then it is difficult to maintain safety See non-blocking synchronization algorithms Allow pre-emption In Java, applications need to repeatedly check for interrupted status Can be difficult to maintain consistent state Remove incremental acquisition a thread acquires all the resources it will need in one hit Avoid circular wait use a partial order on resources e.g. memory addresses / object hashcodes these last 2 points will be considered in this lecture 38 manage resources hierarchically

39 A Trivial Deadlock Example public class LeftRightDeadlock { private final Object left = new Object(); private final Object right = new Object(); public void leftright() { synchronized (left) { synchronized (right) { dosomething(); public void rightleft() { synchronized (right) { synchronized (left) { dosomethingelse(); 39

40 A Trivial Deadlock Example Unlucky Timing in LeftRightDeadlock A program will be free of deadlocks if all threads acquire the locks they need in a fixed global order 40

41 Deadlock Prone? public void transfermoney(account fromaccount, Account toaccount, DollarAmount amount) throws InsufficientFundsException { synchronized (fromaccount) { synchronized (toaccount) { if (fromaccount.getbalance().compareto(amount) < 0) throw new InsufficientFundsException(); else { fromaccount.debit(amount); toaccount.credit(amount); Safety from race condi3ons requires some form of locking on both accounts, as shown, so that transfer will not result in funds being lost 41

42 Dynamic Lock-Ordering Deadlock public void transfermoney(account fromaccount, Account toaccount, DollarAmount amount) throws InsufficientFundsException { synchronized (fromaccount) { synchronized (toaccount) { if (fromaccount.getbalance().compareto(amount) < 0) throw new InsufficientFundsException(); else { fromaccount.debit(amount); toaccount.credit(amount); Deadlock can arise from concurrent calls: transfermoney(myaccount, youraccount, 10); transfermoney(youraccount, myaccount, 20); 42

43 Deadlocks in Java The JVM is not very helpful in resolving deadlocks When a set of Java threads deadlock, that's the end of the game Deadlocks rarely manifest themselves immediately The fact that a class has a potential deadlock doesn't mean that it ever will deadlock Avoiding deadlocks is hard in Java! 43

44 Avoiding Deadlocks The hardest problem in concurrent programming A program will be free of deadlocks if all threads acquire the locks in a fixed global order asymmetry (Dining Philosophers Problem) can use hash code of objects value to order locks System.identityHashCode Deadlock may be caused by coarse-grained locking shrink synchronized methods to synchronized blocks to guard ONLY operations that involve shared state Polled and timed lock attempts see Lock interface in Java 5 (as discussed earlier) acquisition of locks may be interrupted 44

45 Dining Philosophers Problem Dijkstra, 1968 Five philosophers sit around a circular table. Each philosopher spends his life alternately thinking and ea3ng. In the centre of the table is a large bowl of spaghel. A philosopher needs two forks to eat a helping of spaghel One fork is placed between each pair of philosophers and they agree to only use the fork to his immediate right and lem 0 45

46 Dining Philosophers Problem Each fork is a shared resource with ac3ons get and put. FORK lef t phil[1]: PHIL right phil[0]: PHIL lef t FORK right phil[4]: PHIL When hungry, each PHIL must first get his right and lem forks before he can start ea3ng right lef t FORK phil[2]: PHIL right FORK lef t lef t FORK right phil[3]: PHIL 46

47 Dining Philosophers Problem class Fork { private boolean taken=false; final int iden3ty; public Fork (int i) { identity = i; synchronized void put() { taken=false; no3fy(); synchronized void get() throws java.lang.interruptedexcep3on { while (taken) wait(); taken=true; 47

48 Dining Philosophers Problem public class Philosopher extends Thread { final int identity; final Fork left; final Fork right; public Philosopher (int iden3ty, Fork lem, Fork right) { this.iden3ty = iden3ty; this.lem = lem; this.right = public void run() { // see next slide 48

49 Dining Philosophers Problem public void run() { try { while (true) { sleep(thinktime); // thinking // hungry right.get(); // got right fork lem.get(); // got left fork sleep(eattime); // ea?ng right.put(); lem.put(); catch (java.lang.interruptedexcep3on e){ 49

50 Dining Philosophers Problem Code to create the philosopher threads and fork monitors: for (int i =0; i<n; ++i) { fork[i] = new Fork(i); for (int i =0; i<n; ++i) { phil[i] = new Philosopher (i, fork [i],fork [(i+1)%n]); phil[i].start(); 50

51 Solving The Philosophers Problem Deadlock can be avoided by ensuring that a wait-for cycle cannot exist. but how? Introduce an asymmetry into our definition of philosophers Use the identity of a philosopher to make even numbered philosophers get their left forks first Odd numbered philosophers get their right forks first 51

52 Solving The Philosophers Problem public void run() { try { while (true) { sleep(thinktime); // thinking // hungry if (identity%2 == 0) { left.get(); right.get(); else { right.get(); left.get(); // got forks sleep(eattime); // eating right.put(); left.put(); catch (java.lang.interruptedexception e){ No wait- for cycle is possible! Cycle is broken because of reversal of order of acquisi3on by odd Philosophers Philosophers respect a protocol for lock acquisi3on 52

53 Solving The Philosophers Problem There are other solutions to the problem A simpler variation is just to break the cycle that causes the potential deadlock in just one place All Philosophers acquire forks in left right order, except for one e.g. the last one In effect, this places on order on the forks i < j implies Fork[i] < Fork [j] With an acquisition protocol that lower ordered forks must be acquired first So the last Philosopher[N-1] must acquire the right Fork [0] before the left Fork [N-1] Any other Philosopher[i] acquires left Fork[i] before right Fork [i+1] 53

54 Recall Lock-Ordering Deadlock public void transfermoney(account fromaccount, Account toaccount, DollarAmount amount) throws InsufficientFundsException { synchronized (fromaccount) { synchronized (toaccount) { if (fromaccount.getbalance().compareto(amount) < 0) throw new InsufficientFundsException(); else { fromaccount.debit(amount); toaccount.credit(amount); transfermoney(myaccount, youraccount, 10); transfermoney(youraccount, myaccount, 20); 54

55 Ensuring Lock- Ordering The order of arguments can not be controlled within the definition of the transfer method To avoid deadlock, we can impose a global ordering on locks In Java use object identity Or rather, it s hashcode: System.identityHashCode (Object o) BUT occasionally different objects will have the same hashcode So handle that case specially 55

56 Ensuring Lock-Ordering public void transfermoney(account fromacct, Account toacct, DollarAmount amount) throws InsufficientFundsException { int fromhash = System.identityHashCode(fromAcct); int tohash = System.identityHashCode(toAcct); if (fromhash < tohash) { synchronized (fromacct) { synchronized (toacct) { /* do transfer */ else { synchronized (toacct) { synchronized (fromacct) { /* do transfer */ This orders lock acquisi3on according to the hash of the locks object iden3ty. This imposes a global ordering on all locks (except when they have the same hash) 56

57 Ensuring Lock-Ordering private static final Object tielock = new Object(); public void transfermoney(account fromacct, Account toacct, DollarAmount amount) throws InsufficientFundsException { int fromhash = System.identityHashCode(fromAcct); int tohash = System.identityHashCode(toAcct); if (fromhash < tohash) { synchronized (fromacct) { synchronized (toacct) { /* do transfer */ else if (fromhash > tohash) { synchronized (toacct) { synchronized (fromacct) { /* do transfer */ else { synchronized (tielock) { synchronized (fromacct) { synchronized (toacct) { /* do transfer */ Use a single extra lock to deal with 3ed hashcodes. This ensures that both account locks can only be acquired by a single thread. It will rarely be used, so should not be a concurrency bocle- neck. 57

58 Loss of Abstraction with Locks client code may wish to coordinate the synchronization of multiple objects must know which locks to use if we want to provide operations on multiple accounts, we need to know the synchronisation policy for those accounts in particular we need to know what locks are used on what methods?» what if the accounts use explicit locks instead of intrinsic monitor? how to acquire those locks? in what order should multiple locks be acquired? clients need to see some implementation details of the synchronisation policy this is a loss of abstraction it limits the reusability of code with synchronisation 58

59 Software Transactional Memory analogous to database transactions a thread optimistically completes updates of memory ignoring other threads logs reads and writes checks for conflicts with other threads at end of transaction, and commits if none if conflict occurs, transaction fails rollback is necessary with retry, or explicit alternatives provides atomicity for blocks of code nice programming model but has clear performance overheads see: 59

60 Home Work compile and run the examples on locks, deadlock and guarded methods from the Java Concurrency Tutorial newlocks.html deadlock.html guardmeth.html understand why the bounded buffer must call notifyall() and not notify() Read the javadoc for interfaces and classes defined in java.util.concurrent.locks package including the Condition class Start work on assignment 1 60