Multi-threading in Java Jeff HUANG Software Engineering Group @HKUST
Do you use them? 2
Do u know their internals? 3
Let s see File DB How can they service so many clients simultaneously? l 4
Multi-threading A thread is a single sequential flow of control In a single process, there could be multiple threads Threads share memory The server program spawns multiple threads Each thread handles a client request 5
Example A single core CPU and a task compute readfile compute readfile compute writefile compute 0 1 2 3 4 5 6 7 sec 2 such tasks 6
A simple accounting Single thread T=14 Two threads T=8 7
What s more Multi-core & multi-processor Dual-core AMD 32 processor 4096 processor Cray Athlon X2 Pentium Xeon X1 8
More accounting Number of cores = 1,000,000 = 1M 2M Multi-threading can do such tasks in 8 seconds! What about the single-threaded d version? 9
More benefits Performance User responsiveness Resource utilization Simplicity it of modeling Simplified handling of asynchronous events 10
Multi-threads are everywhere Server programs (web/database server) Client programs (Downloading tools/browser) Editors (Word/Photoshop/Adobe Reader) Search engine IDE (Matlab/Eclipse/Visual studio) JVM Swing and AWT 11
Where are we now? How to program? 12
Various ways of multithreading Thread Runnable Exectutor Framework Timer FutureTask 13
Thread Runnable Two major ways Exectutor Framework Timer FutureTask 14
Creating threads in Java Thread class public class Thread { public void run(){ ; Runnable interface public interface Runnable { public void run(); // work thread 15
Thread Class public class Thread extends Object implements Runnable { public Thread(); public Thread(String name); // Thread name public Thread(Runnable R); // Thread R.run() public Thread(Runnable R, String name); public void run(); // if no R, work for thread public void start(); // begin thread execution... 16
More Thread Class Methods public class Thread extends Object implements Runnable { public static Thread currentthread() public String getname() public void interrupt() public boolean isalive() public void join() public void setdaemon() public void setname() public void setpriority() public static void sleep() public static void yield() 17
1. Thread class Creating Threads in Java Extend Thread class and override the run method Example public class MyThread extends Thread { public void run() { // work for thread MyThread T = new MyThread () ; // create thread T.start(); // begin running thread // thread executing in parallel 18
Creating Threads in Java 2. Runnable interface Create object implementing Runnable interface Pass it to Thread object via Thread constructor Example public class MyRunnable implements Runnable { public void run() { // work for thread MyRunnable myr = new MyRunnable(); // create runnable object Thread T = new Thread(myR); // create thread T.start(); // begin running thread // thread executing in parallel 19
Producer - Consumer class Producer extends Thread class Consumer extends Thread { { private final public BlockingQueue static queue; void main private (String[] final BlockingQueue args) queue; Producer(BlockingQueue q){queue = q; Consumer(BlockingQueue q) { queue = q; public void { run() public void run() { { BlockingQueue q = new SomeQueue(); while(true) while(true) { Producer p = new Producer(q); { queue.put(produce()); consume(queue.take()); Consumer c = new Consumer(q); private Message p.start(); produce() private void consume(message mes) { { c.start(); return new Message(); Example 1 20
Simple WebServer public class MyRunnable implements Runnable { public void run() { handlerequest(connection); Process request public class SimpleWebServer { public static void main (String[] args) throws IOException { ServerSocket socket = new ServerSocket(80); while (true) { Accept client request final Socket connection = socket.accept(); kt t() Runnable task = new MyRunnable(connection); Create Thread t = new Thread(task); Create thread Runnable tstart(); t.start(); start t thread Example 2 21
Let s look a bit deeper Thread states Thread scheduler Difficulties li in Multithreaded lih d programming g 22
Thread States Java thread can be in one of these states New Runnable Running Blocked Dead thread allocated & waiting for start() thread can begin execution thread currently executing thread waiting for event (I/O, etc.) thread finished Transitions between states caused by Invoking methods in class Thread new(), start(), yield(), sleep(), wait(), notify() Other (external) events Scheduler, I/O, returning from run() 23
Thread States new new start runnable scheduler yield, time slice notify, notifyall, IO complete, sleep expired terminate running IO, sleep, wait, join blocked dead 24
Scheduler Thread Scheduling Determines which runnable threads to run Can be based on thread priority Part of OS or Java Virtual Machine (JVM) Scheduling policy Preempted by scheduler Non-preemptive (cooperative) scheduling Preemptive scheduling 25
Let s do a bit analysis Two threads A and B They both do the following work print 1, print 2, print 3 The main thread first starts A, then starts B main (): thread A, thread B thread A: println 1, println 2, println 3 thread B: println 1, println 2, println 3 26
Let s do a bit analysis Non-preemptive scheduler Preemptive scheduler Single-core 1, 2, 3, 1, 2, 3??? Multi-core?????? main (): thread A, thread B thread A: println 1, println 2, println 3 thread B: println 1, println 2, println 3 27
Java Thread Example public class ThreadExample extends Thread { public void run() { for (int i = 1; i <= 3; i++) System.out.println(i); try { sleep((int)(math.random() * 1000)); catch (InterruptedException e) { public static void main(string[] args) { Thread A = new ThreadExample(); Thread B = new ThreadExample(); A.start(); t() B.start(); System.out.println("Done"); 28
Java Thread Example Possible outputs 1,1,Done,2,2,3,3 1Done12233 1,Done,1,2,2,3,3 1,1,Done,2,3,2,3 Done11 Done,1,1, 2233 2,2,3,3 Multi-threads can interleave their executions!!! 29
Another example public class Worker extends Thread { static int x = 0; public void run() { for (int i = 0; i < 1000; i++) { x=x+1; x Firstincreasexby1 increase x = x 1; Then decrease x by 1 public static void main (String[] args) { x = 0; for (int i = 0; i < 1000; i++) new Worker().start(); Start 1000 threads // wait for all threads exit System.out.println(x); 30
Data Race public class Worker extends Thread { static int x = 0; public void run() { for (int i = 0; i < 1000; i++) { x=x+1; x Firstincreasexby1 increase x = x 1; Then decrease x by 1 public static void main (String[] args) { x = 0; for (int i = 0; i < 1000; i++) new Worker().start(); Start 1000 threads // wait for all threads exit Xisnotalways0!! System.out.println(x); 31
Quiz time 32
Answer: Yes! 33
How Can This Happen? Compiler can reorder statements Or keep values in registers The Java memory model is designed to allow aggressive optimization On multi-processor, values not synchronized to global memory Good for performance 34
Observations Multithread execution is non-deterministic Depends on scheduler & single or multi-core Thread scheduling may cause data races Modifying same data from multiple threads Result depends on thread execution order Complier can reorder statements The memory model is designed to allow aggressive optimization 35
How to deal with it? We need protocol to make sure the shared data is manipulated safely We need to force the compiler to ensure that the modified shared data visible to other threads Synchronization Lock Semaphore Monitor Java synchronization 36
Java synchronization Java uses the concept of monitors Java uses the concept that every object is associated with a lock use synchronized keyword Two ways Synchronized methods Synchronized blocks 37
Synchronized method Calling a synchronized method attempts to possess the lock If no one owns the lock, then this thread has/owns the lock and proceeds. Otherwise, it has to wait until the lock is released by some other thread Lock For static method, it is the lock associated with the Class object to which the static method belongs to For instance method, it is the lock associated with the object on which h the method call is being made 38
Synchronized blocks Very similar to synchronized method, but it is Blocks of code, rather than entire methods public void somemethod() { // non-critical section synchronized(someobject) synchronized(this) { // critical section // non-critical section 39
Data Race public class Worker extends Thread { static int x = 0; public void run() { for (int i = 0; i < 1000; i++) { x=x+1; x Firstincreasexby1 increase x = x 1; Then decrease x by 1 public static void main (String[] args) { x = 0; for (int i = 0; i < 1000; i++) new Worker().start(); Start 1000 threads // wait for all threads exit Xisnotalways0!! System.out.println(x); 40
Using synchronization public class Worker extends Thread { static int x = 0; static Object lock = new Object(); public void run() { synchronized(lock) { for (int i = 0; i < 1000; i++) { x = x + 1; x = x 1; 41
Questions What would happen if the lock field were not static? Why don t we just make the run method synchronized? Why don t we just synchronize on x? 42
But, it is hard in practice Holding locks is a global property affects entire program, cannot be hidden behind an abstract interface Results in lack of modularity callers cannot ignore what locks their callees acquire or what locations they access necessary for race avoidance, but also for global l ordering to avoid deadlock part of a method s protocol which lock needs to be held when called, which locks it acquires 43
Bank Account Example class Account { Atomicity violation private int balance = 0; public void deposit(int n) { public read() { int r; synchronized(this) { r = balance; return r; int r = read(); other threads can update balance synchronized(this) { balance = r + n; Race-freedom is not sufficient 44
Optimized Bank Account public read() { return balance; class Account { private int balance = 0; public void deposit(int n) { synchronized(this) { int r = balance; balance = r + n; 45
Another Account Example class Account { private int balance = 0; public synchronized int withdraw(int amt){ public synchronized void deposit(int i) { class Client1 { public synchronized void move(account a1, Account a2) { a2.deposit(a1.withdraw(1000000)); class Client2 // same as Client1 Client1: move(a1,a2); Client2: move(a2,a1); 46
Deadlock client1 holds lock for account a1, waits for account a2 client2 holds lock for account a2, waits for account a1 Both of them can not proceed Client1 Client2 a1 a2 47
Impair performance Frequent lock acquire and lock release operations We are sacrificing performance for data safety But sometimes there is no conflict On some extreme situations, performance is greatly impaired 48
Is it hard? 49
Let s make a conclusion Multithreading is quite useful But you need to be careful Bugs are everywhere 50
What we have learned Benefits of multithreads Performance User responsiveness How to make multithreaded Java programs Thread class/runnable interface Difficulties in Multithreading Statements reorder/data races/atomicity violation/deadlock Java synchronization Syncrhonized methods/blocks 51
Questions What would happen if the lock field were not static? Why don t we just make the run method synchronized? Why don t we just synchronize on x? 52
Using synchronization public class Worker extends Thread { static int x = 0; static Object lock = new Object(); public void run() { synchronized(lock) { for (int i = 0; i < 1000; i++) { x = x + 1; x = x 1; 53
See you next time
Backups
Daemon Threads Java threads types User Daemon Provide general services Typically never terminate Call setdaemon() before start() Program termination 1. All user threads finish 2. Daemon threads are terminated by JVM 3. Main program finishes 56
Atomicity violation 57
Thread flow graph 58
Non-preemptive Scheduling Threads continue execution until Thread terminates Executes instruction causing wait (e.g., IO) Thread volunteering to stop (invoking yield or sleep) 59
Preemptive Scheduling Threads continue execution until Same reasons as non-preemptive scheduling Preempted by scheduler 60
Transactional Memory Instead of using synchronizations, the runtime system ensures that the critical section executes transactionallyti You can just go ahead when entering a critical section; when you are exiting the critical section, the runtime system will check the data safety for you if there is no conflict, just proceed Otherwise, rollback The transaction memory system guarantees other threads cannot see intermediate values of the transaction (all-or-nothing) the transaction cannot see values of other transactions in the middle of its execution 61
Hardware & software TM Hardware transactional memory(htm) Software transactional memory (STM) 62
Advantages No deadlock! we never refer to locks explicitly Composability & modularity no need to know what callees do w.r.t. synchronization Performance could better in some situations 63