Shared-Memory and Multithread Programming Pruet Boonma pruet@eng.cmu.ac.th Department of Computer Engineering Faculty of Engineering, Chiang Mai University Based on a material by: Bryan Carpenter Pervasive Technology Labs Indiana University
In this week. 2 Shared Memory Parallel Computing with Java Threads Mutual Exclusion. Synchronization between Java Threads using wait() and notify(). Other features of Java Threads.
Threaded programming and Threaded Language 3 Thread is a way to do parallel computing. In C, C++, etc it is possible to do multithreaded programming, given a suitable library. e.g. the pthreads library Thread libraries provide one approach to doing parallel programming on computers with shared memory. Unlike these (traditional HPC) languages, Java integrates threads into the basic language specification in a much tighter way. Every Java Virtual Machine must support threads. Although this close integration doesn t exactly make multithreaded programming in Java easy, it does help avoid common programming errors, and keeps the semantics clean.
Features of Java Threads 4 Java provides a set of synchronization primitives based on monitor and condition variable paradigm of C.A.R. Hoare. Underlying functionality is similar to POSIX threads, for example. Syntactic extension for threads is (deceptively?) small: synchronized attribute on methods. synchronized statement. volatile keyword. Other thread management and synchronization captured in the Thread class and related classes. But the presence of threads has a more wide-ranging effect on the language specification and JVM implementation. e.g., the Java memory model.
Creating New Threads 5 Any Java thread of execution is associated with an instance of the Thread class. Before starting a new thread, you must create a new instance of this class. The Java Thread class implements the interface Runnable. So every Thread instance has a method: public void run() {... When the thread is started, the code executed in the new thread is the body of the run() method. Generally speaking the new thread ends when this method returns.
Making Thread Instances 6 There are two ways to create a thread instance (and define the thread run() method). Choose at your convenience: 1. Extend the Thread class and override the run() method, e.g.: class MyThread extends Thread { public void run() { System.out.println( Hello from another thread ) ;... Thread thread = new MyThread() ; 2. Create a separate Runnable object and pass it to the Thread constructor, e.g.: class MyRunnable implements Runnable { public void run() { System.out.println( Hello from another thread ) ;... Thread thread = new MyThread(new MyRunnable()) ;
Starting a Thread 7 Creating the Thread instance does not in itself start the thread running. To do that you must call the start() method on the new instance: thread.start() ; This operation causes the run() method to start executing concurrently with the original thread. In our example the new thread will print the message Hello from another thread to standard output, then immediately terminate. You can only call the start() method once on any Thread instance. Trying to restart a thread causes an exception to be thrown.
8 Example: Multiple Threads class MyThread extends Thread { MyThread(int id) { this.id = id ; public void run() { System.out.println( Hello from thread + id) ; private int id ;... Thread [] threads = new Thread [p] ; for(int i = 0 ; i < p ; i++) threads [i] = new MyThread(i) ; for(int i = 0 ; i < p ; i++) threads [i].start() ;
Remarks 9 This is one way of creating and starting p new threads to run concurrently. The output might be something like (for p = 4): Hello from thread 3 Hello from thread 4 Hello from thread 2 Hello from thread 1 Of course there is no guarantee of order (or atomicity) of outputs, because the threads are concurrent.
Mutual Exclusion
Avoiding Interference 11 In any non-trivial multithreaded (or shared-memory-parallel) program, interference between threads is an issue. Generally interference (or a race condition) occurs if two threads are trying to do operations on the same variables at the same time. This often results in corrupt data. But not always. It depends on the exact interleaving of instructions. This non-determinism is the worst feature of race conditions. A popular solution is to provide some kind of lock primitive. Only one thread can acquire a particular lock at any particular time. The concurrent program can then be written so that operations on a given group of variables are only ever performed by threads that hold the lock associated with that group. In POSIX threads, for example, the lock objects are called mutexes.
pthread_mutex_lock(&my_mutex) ; 12 Example use of a Mutex (in C) Thread A Thread B /* critical region */ tmp1 = count ; count = tmp1 + 1 ; pthread_mutex_unlock(&my_mutex) ; pthread_mutex_lock(&my_mutex) ; Blocked /* critical region */ tmp2 = count ; count = tmp2-1 ; pthread_mutex_unlock(&my_mutex) ;
Pthreads-style mutexes 13 In POSIX threads, a mutex is allocated then initialized with pthread_mutex_init(). Once a mutex is initialized, a thread can acquire a lock on it by calling e.g. pthread_mutex_lock(). While it holds the lock it performs some update on the variables guarded by the lock (critical region). Then the thread calls pthread_mutex_unlock() to release the lock. Other threads that try to call pthread_mutex_lock() while the critical region is being executed are blocked until the first thread releases the lock. This is fine, but opportunities for error include: There is no built-in association between the lock object (mutex) and the set of variables it guards it is up to the program to maintain a consistent association. If you forget to call pthread_mutex_unlock() after pthread_mutex_lock(), deadlocks will occur.
Monitors 14 Java addresses these problems by adopting a version of the monitors proposed by C.A.R. Hoare. Every Java object is created with its own lock (and every lock is associated with an object there is no way to create an isolated mutex). In Java this lock is often called the monitor lock. Methods of a class can be declared to be synchronized.
Monitors 15 The object s lock is acquired on entry to a synchronized method, and released on exit from the method. Synchronized static methods need slightly different treatment. Assuming methods generally modify the fields (instance variables) of the objects they are called on, this leads to a natural and systematic association between locks and the variables they guard: viz. a lock guards the instance variables of the object it is attached to. The critical region becomes the body of the synchronized method.
16 Example use of Synchronized Methods Thread A call to counter.increment() Thread B // body of synchronized method tmp1 = count ; count = tmp1 + 1 ; counter.increment() returns call to counter.decrement() Blocked // body of synchronized method tmp2 = count ; count = tmp2-1 ; counter.decrement() returns
Caveats 17 This approach helps to encourage good practices, and make multithreaded Java programs less error-prone than, say, multithreaded C programs. But it isn t magic: It still depends on correct identification of the critical regions, to avoid race conditions. The natural association between the lock of the object and its fields relies on the programmer following conventional patterns of object oriented programming (which the language encourages but doesn t enforce). By using the synchronized construct (see later), programs can break this association altogether. There are plenty more insidious ways to introduce deadlocks, besides accidentally forgetting to release a lock! Concurrent programming is hard, and if you start with the assumption Java somehow makes concurrent programming easy, you are probably going to write some broken programs!
18 Example: A Simple Queue public class SimpleQueue { private Node front, back ; public synchronized void add(object data) { if (front!= null) { back.next = new Node(data) ; back = back.next ; else { front = new Node(data) ; back = front ; public synchronized Object rem() { Object result = null ; if (front!= null) { result = front.data ; front = front.next ; return result ;
Remarks 19 This queue is implemented as a linked list with a front pointer and a back pointer. The method add() adds a node to the back of the list; the method rem() removes a node from the front of the list. The Node class just has a data field (type Object) and a next field (type Node). The rem() method immediately returns null when the queue is empty. The following slide gives an example of what could go wrong without mutual exclusion. It assumes two threads concurrently add nodes to the queue. In the initial state, Z is the last item in the queue. In the final state, the X node is orphaned, and the back pointer is null.
Thread A: add(x) The Need for Synchronized Methods Z null 20 back.next = new Node(X) ; back Thread B: add(y) Z X null back.next = new Node(Y) ; back = back.next ; back Z back X Y null null X null Z back Y null X null back = back.next ; Z Y null Corrupt data structure! back null
21 The synchronized construct The keyword synchronized also appears in the synchronized statement, which has syntax like: synchronized (object) { critical region Here object is a reference to any object. The synchronized statement first acquires the lock on this object, then executes the critical region, then releases the lock. Typically you might use this for the lock object, somewhere inside a nonesynchronized method, when the critical region is smaller than the whole method body. In general, though, the synchronized statement allows you to use the lock in any object to guard any code.
General Synchronization
23 Beyond Mutual Exclusion The mutual exclusion provided by synchronized methods and statements is an important special sort of synchronization. But there are other interesting forms of synchronization between threads. Mutual exclusion by itself is not enough to implement these more general sorts of thread interaction (not efficiently, at least). POSIX threads, for example, provides a second kind of synchronization object called a condition variable to implement more general inter-thread synchronization. In Java, condition variables (like locks) are implicit in the definition of objects: every object effectively has a single condition variable associated with it.
A Motivating Example 24 Consider the simple queue from the previous example. If we try to remove an item from the front of the queue when the queue is empty, SimpleQueue was specified to just return null. This is reasonable if our queue is just meant as a data structure buried somewhere in an algorithm. But what if the queue is a message buffer in a communication system? In that case, if the queue is empty, it may be more natural for the remove operation to block until some other thread added a message to the queue.
Busy Waiting 25 One approach would be to add a method that polls the queue until data is ready: public synchronized Object get() { while(true) { Object result = rem() ; if (result!= null) return result ; This works, but it may be inefficient to keep doing the basic rem() operation in a tight loop, if these machine cycles could be used by other threads. This isn t clear cut: sometimes busy waiting is the most efficient approach. Another possibility is to put a sleep() operation in the loop, to deschedule the thread for some fixed interval between polling operations. But then we lose responsiveness.
wait() and notify() 26 In general a more elegant approach is to use the wait() and notify() family of methods. These are defined in the Java Object class. Typically a call to a wait() method puts the calling thread to sleep until another thread wakes it up again by calling a notify() method. In our example, if the queue is currently empty, the get() method would invoke wait(). This causes the get() operation to block. Later when another thread calls add(), putting data on the queue, the add() method invokes notify() to wake up the sleeping thread. The get() method can then return.
A Simplified Example 27 public class Semaphore { int s ; public Semaphore(int s) { this.s = s ; public synchronized void add() { s++ ; notify() ; public synchronized void get() throws InterruptedException { while(s == 0) wait() ; s-- ;
Remarks I 28 Rather than a linked list we have a simple counter, which is required always to be non-negative. add() increments the counter. get() decrements the counter, but if the counter was zero it blocks until another thread increments the counter. The data structures are simplified, but the synchronization features used here are essentially identical to what would be needed in a blocking queue (left as an exercise). You may recognize this as an implementation of a classical semaphore an important synchronization primitive in its own right.
Remarks II 29 wait() and notify() must be used inside synchronized methods of the object they are applied to. The wait() operation pauses the thread that calls it. It also releases the lock which the thread holds on the object (for the duration of the wait() call: the lock will be claimed again before continuing, after the pause). Several threads can wait() simultaneously on the same object. If any threads are waiting on an object, the notify() method wakes up exactly one of those threads. If no threads are waiting on the object, notify() does nothing. A wait() method may throw an InterruptedException (rethrown by by get() in the example). This will be discussed later. Although the logic in the example doesn t strictly require it, universal lore has it that one should always put a wait() call in a loop, in case the condition that caused the thread to sleep has not been resolved when the wait() returns (a programmer flaunting this rule might use if in place of while).
Another Example 30 public class Barrier { private int n, generation = 0, count = 0 ; public Barrier(int n) { this.n = n ; public synchronized void synch() throws InterruptedException { int gennum = generation ; count++ ; if(count == n) { count = 0 ; generation++ ; notifyall() ; else while(generation == gennum) wait() ;
Remarks 31 This class implements barrier synchronization an important operation in shared memory parallel programming. It synchronizes n processes: when n threads make calls to synch() the first n-1 block until the last one has entered the barrier. The method notifyall() generalizes notify(). It wakes up all threads currently waiting on this object. Many authorities consider use of notifyall() to be safer than notify(), and recommend always to use notifyall(). In the example, the generation number labels the current, collective barrier operation: it is only really needed to control the while loop round wait(). And this loop is only really needed to conform to the standard pattern of wait()- usage, mentioned earlier.
Concluding Remarks on Synchronization 32 We illustrated with a couple of simple examples that wait() and notify() allow various interesting patterns of thread synchronization (or thread communication) to be implemented. In some sense these primitives are sufficient to implement general concurrent programming any pattern of thread synchronization can be implemented in terms of these primitives. For example you can easily implement message passing between threads (left as an exercise ) This doesn t mean these are necessarily the last word in synchronization: e.g. for scalable parallel processing one would like a primitive barrier operation more efficient than the O(n) implementation given above.
Other Features of Java Threads
Join Operations 34 The Thread API has a family of join() operations. These implement another simple but useful form of synchronization, by which the current thread can simply wait for another thread to terminate, e.g.: Thread child = new MyThread() ; child.start() ; Do something in current thread child.join() ; // wait for child thread to finish
Priority and Name 35 Thread have properties priority and name, which can be defined by suitable setter methods, before starting the thread, and accessed by getter methods.
Sleeping 36 You can cause a thread to sleep for a fixed interval using the sleep() methods. This operation is distinct from and less powerful than wait(). It is not possible for another thread to prematurely wake up a thread paused using sleep(). If you want to sleep for a fixed interval, but allow another thread to wake you beforehand if necessary, use the variants of wait() with timeouts instead.
Interrupting Threads 37 Calling the method interrupt() on a thread instance requests cancellation of the thread execution. It does this in an advisory way: the code for the interrupted thread must be written to explicitly test whether it has been interrupted, e.g.: public void run() { while(!interrupted()) do something Here interrupted() is a static method of the Thread class which determines whether the current thread has been interrupted If the interrupted thread is executing a blocking operation like wait() or sleep(), the operation may end with an InterruptedException. An interruptible thread should catch this exception and terminate itself. Clearly this mechanism depends wholly on suitable implementation of the thread body. The programmer must decide at the outset whether it is important that a particular thread be responsive to interrupts often it isn t.
Thread-based Parallel Applications and Benchmarks
39 Threads on Symmetric Multiprocessors Most modern implementations of the Java Virtual Machine will map Java threads into native threads of the underlying operating system. For example these may be POSIX threads. On multiprocessor architectures with shared memory, these threads can exploit multiple available processors. Hence it is possible to do true parallel programming using Java threads within a single JVM.
40 Select Application Benchmark Results We present some results, borrowed from the literature in this area. Two codes are described in High-Performance Java Codes for Computational Fluid Dynamics C. Riley, S. Chatterjee, and R. Biswas, ACM 2001 Java Grande/ISCOPE LAURA is a finite-volume flow solver for multiblock, structured grids.
41 Parallel Speedup of LAURA Code
Remarks 42 LAURA speedups are generally fair, and are outstanding with the Jalapeno JVM on PowerPC (unfortunately this is the only JVM that isn t freely available.) LAURA is regular, and the parallelization strategy needs little or no locking.
43 HW: Java Thread Synchronization Following the pattern of the Semaphore class, complete the implementation of a queue with a get() operation that blocks when the queue is empty, using wait() and notify(). Extend this to implement send() and recv() primitives for communicating between threads by message-passing. Try implementing other features of the MPI standard for communication between threads, e.g. synchronous mode sends (where both sender and receive block until both are ready to communicate). Avoid busy-waiting or polling solutions.