Internet and Intranet Protocols and Applications Writing Good Multi-Threaded Java Programs April, 2004 Prof. Arthur Goldberg Computer Science Department New York University 1
Why write concurrent programs? Code may need to block for (external) events I/O completion Keyboard input Message arrival Etc. Run in parallel on multiple processors Decomposition Mimic a concurrent system, as in a discrete event simulation 2
Approaches to concurrency We distinguish four forms of practical concurrency (in order of increasing difficulty): Sequential programming + its variants Declarative concurrency + lazy execution (add threads to a functional language and use dataflow to decouple independent calculations) All the programming and reasoning techniques of sequential declarative programming apply (concurrent programs give the same results as sequential ones) Deep characterization: lack of observable nondeterminism Message passing between active objects (Erlang style, each thread runs a functional program, threads communicate through asynchronous channels) Atomic actions on shared state (Java style, using monitors and transactions) The Java style is the most popular, yet it is the most difficult to program Next 3 slides from http://www.info.ucl.ac.be/people/pvr/wflptalk.ppt 2001 P. Van Roy and S. Haridi 3
Declarative concurrency (1) Dataflow = block on data availability Add dataflow variables (i.e., logic variables) to a functional language 1 X0 threads declare X0 X1 X2 X3 Y0 Y1 Y2 in Y0=1+X0 Y1=Y0+X1 Y2=Y1+X2 Y0 Y1 + + X1 X2 X0=1 X2=1 X1=1 thread X0=1 end thread X1=1 end thread X2=1 end + Y2 4
Declarative concurrency (2) A system can be concurrent and still be purely functional Declarative concurrency has this property Here is a simple example: fun {Fibo N if N=<2 then 1 else F1 F2 in thread F1={Fibo N-1 end F2={Fibo N-2 F1+F2 end The results end are the same, whether or not threads are used 5
Common Concurrent Patterns Mutual exclusion relationship Trying to avoid conflicts while sharing a common resource Use synchronized Producer-consumer relationship Actively cooperating on a common goal Use wait and notify or notifyall 6
First, Review Thread Support in Java To create a thread of control, create a Thread object and invoke its start method: new Thread(target).start(); You can extend Thread, and provide a new run method, or implement Runnable, which declares a single method: public void run(); 7
Common Concurrent Patterns Mutual exclusion relationship Trying to avoid conflicts while sharing a common resource Use synchronized Producer-consumer relationship Actively cooperating on a common goal Use wait and notify or notifyall 8
Synchronization support in Java Every object has one lock A method declared synchronized acquires the lock before the method executes, and releases the lock as soon as the method terminates. If the lock is already held when a thread tries to acquire then the thread blocks until the lock is released. Since overriding replaces a method, the programmer decides whether to synchronize the new method. A synchronized statement can request the lock of any object: synchronized( expr ) {statements The expressionexpr must evaluate to an object; this object s lock is acquired before the block is executed, and released as soon as the block is exited. 9
Synchronization support in Java (2) Example public class Account { private double balance; public Account(double initialdeposit) { balance = initialdeposit; public synchronized double getbalance() { return balance; public synchronized void deposit(double amount) { balance += amount; 10
Race conditions (hazards) When the result depends on which thread wins the race to points in their respective code Bad if the code should be deterministic Example Two threads, each waits a random time then sets a value 11
Synchronization support in Java (3) synchronized static methods acquire the lock of the Class object for their class. public class Body { static int nextid; static synchronized int getid() { return nextid++; Alternatively, you can synchronize explicitly on the class literal. public class Body2 { static int nextid; static int getid() { synchronized (Body2.class) { return nextid++; 12
Java s Shared Memory Model Threads may be supported by many hardware processors, by timeslicing a single hardware processor, or by time-slicing many hardware processors. Memory hierarchy model thread working memory: threads keep copies of the values of variables main memory: shared between all threads access a shared variable thread (usually first) obtains a lock and flushes its working memory, which guarantees that shared values will thereafter be loaded from the shared main memory to the thread s working memory thread unlocks a lock it guarantees the values it holds in its working memory will be written back to the main memory 13
Atomicity Java guarantees that reading or writing any variable (except long or double) is atomic the variable will always hold a value written by some thread If a field is declared volatile then the compiler must read or write it from main memory on each access Also, access to a volatilelong or a volatile double is atomic 14
Common Concurrent Patterns Mutual exclusion relationship Trying to avoid conflicts while sharing a common resource Use synchronized Producer-consumer relationship Actively cooperating on a common goal Use wait and notify or notifyall 15
Communicating Between Threads The Object.wait method waits until something happens The methods Object.notify and Object.notifyAll tell waiting threads that something has happened Waiting: synchronized void dowhencondition() { while(!condition ) try { wait(); // do what must be done when condition is true catch (InterruptedException e) { Notifying: synchronized void changecondition() { // make one or more condition s true notifyall(); // or notify(); 16
Notify Using notify is an optimization that applies only when: All threads are waiting for the same condition; At most one thread can benefit from the condition being met; and This is contractually true for all possible subclasses. 17
Reusing Code (1) Suppose you want to use a class C that is not thread-safe in a multithreaded environment Consider these implementation choices Synchronize all access to C in the code that uses C extend C like this public class C { public void m1() { /* body here */ public void m2(int i) { /*BH*/ public class D extends C { synchronized public void m1() { super.m1(); synchronized public void m2(int i) { super.m2(i); 18
Reusing Code (2) If you want to use a class E that implements an interface I and is not thread-safe in a multithreaded environment just provide an alternate implementation that wraps the methods of the interface in synchronized methods that forward the calls to the unsynchronized object that implements the interface. For example, import java.util.*; public class X { Map unsyncmap = new HashMap(); Map syncmap = Collections.synchronizedMap(unsyncMap); // Iterators are not synchronized void foo() { synchronized (syncmap) {// the wrapper synchs on itself System.out.println("the map"); Iterator it = syncmap.keyset().iterator(); while (it.hasnext()) { Object key = it.next(); System.out.println(key + ": " + syncmap.get(key)); 19
Advice Use as few threads as possible Carefully control interaction between threads Use as much synchronization as necessary, but no more (use the finest granularity of locking that is correct) increases concurrency It is safer and more reliable for shared objects to protect themselves (sometimes called server-side synchronization) rather than depend on their clients to do so. Document concurrency well 20
Quality of concurrent programs All issues of sequential programs, plus Performance Satisfactory execution speed Safety Nothing bad every happens Liveness Something eventually happens Fairness All threads obtain some share of the resource (CPU) 21
Multithreaded Designs Strategically add threads for scalability Mainly applicable to multiprocessors Worker Threads Reactors should quickly trigger handlers Handler processing slows down Reactor Offload non-io processing to other threads 22
Common problems Deadlock Busy wait spinning (repeatedly locking and unlocking an object to see whether some internal state has changed), which consumes computational effort Unpredictable results (non-deterministic results) 23
Deadlock Program fails to proceed because some thread(s) is waiting for an event that cannot ever happen Example from page 252 A cycle in the wait-for graph Generic solutions Avoid Break Avoid Assign a priority to everything that can be waited for Only wait for things that have higher priority than things already locked Cycles in the wait-for graph cannot form This technique is known as resource ordering Example xxx Break Can work in DBMSes, but not in arbitrary programs Detect a cycle in the wait-for graph Break the cycle, by killing a thread 24
Does this Deadlock? //deadlock? public class Foo { synchronized void bar() { public static void main(string[] args) { Foo foo = new Foo(); synchronized (foo) { foo.bar(); 25
References Ken Arnold, James Gosling, David Holmes, The Java Programming Language, 3rd edition, June 2000, Paperback, $40, ISBN: 0201704331, June 2000, Pearson Education. Chapter 10. Douglas Lea, Concurrent Programming in Java: Design Principles and Pattern, 2nd edition, October 1999, $50, ISBN: 0201310090, Paperback, 432pp, Addison-Wesley Java Series. Online supplement: http://gee.cs.oswego.edu/dl/cpj/index.html. See also Nik Boyd, A Study Guide for Concurrent Programming in Java, at http://home.labridge.com/~nikboyd/papers/concurrent/concurrent.guide.htm Steve Kleiman, Devang Shah, Bart Smaalders, Programming with Threads, January 1996, ISBN: 0131723898, Hardcover, 576pp, Prentice Hall Professional Technical Reference Per Brinch Hansen, Architecture of Concurrent Programs, Hardcover, July 1977, ISBN: 0130446289, 317pp, July 1977, Prentice Hall Professional Technical Reference C.A.R Hoare, Monitors: An Operating System Structuring Concept, Communications of the ACM, Vol. 17, No. 10. October 1974, pp. 549-557. http://www.acm.org/classics/feb96/. The seminal paper on using monitors to synchronize concurrent programs. Peter Van Roy and Seif Haridi, Concepts, Techniques, and Models of Computer Programming, MIT Press, hardcover, 900pp+xxix, ISBN 0-262-22069-5, March 2004, $65. http://www.info.ucl.ac.be/people/pvr/book.html. A comprehensive scientificallybased programming model, covering all major approaches to concurrency. Java Language Specification, Second Edition, Chapter 17, Threads and Locks. 26
The End 27
Debugging concurrent programs Dependency graph model Heisenbug observe it and it changes 28
Other interaction devices Trylock Spinlock 29
Reader-writer problem and example Shared buffer 30
What thread scheduling strategies does Java support Priority Ready Java makes no promises about scheduling or fairness, or even progress 31