Java s Implementation of Concurrency, and how to use it in our applications.

Java s Implementation of Concurrency, and how to use it in our applications. 1

An application running on a single CPU often appears to perform many tasks at the same time. For example, a streaming audio/video application must perform the following tasks: (R) Read compressed digital audio off the network, and place into Buffer A; (D) Read Buffer A, Decompress it and store in Buffer B; (P) Read Buffer B, and Playback the digital audio; and (U) Update the application s display (i.e., the GUI). When this software runs, it appears that all tasks are being run at the same time, because we cannot perceive when one task starts and another ends. The tasks seem to be running at the same time. Software that can do such things is known as concurrent software. 2

On typical systems, the concurrency is perceived. The perceived concurrency is achieved by sequentially running each task on the CPU for a small enough time so that the observer cannot identify which task is currently running. Each task is called a thread of execution, or simply a thread. Thread 1 Application Thread 2 Thread 3 Thread 1... A thread is a unique sequence of statements of an application. Time 3

OS and JVM OS Level (higher level) Applications, such as JVM running some program PowerPoint editing a presentation Clock displaying the current time File manager displaying files Run under and are controlled by the OS. OS Level of Concurrency JVM Level of Concurrency JVM Level (lower level) The threads of your application run under and are controlled by the JVM. We will discuss this level of concurrency. We will create threads for our Java application, and get the JVM to Time Division Multiplex (TDM) them. 4

Time Division Multiplexing (TDM) Consider a streaming audio application running under the JVM. Assume you have split the application into 4 tasks (RDPU). Assume also you have arranged that the JVM will run these tasks as threads in a round robin fashion. 1. At the end of each time-slice, e.g., 1 millisecond, the JVM 2. Stops the current thread (task) from running, and stores the thread s state to memory. 3. Retrieves the next thread s current state, and starts the thread. Goto step (1). Threads share the CPU in a time-sliced fashion. If a thread completes its task, the JVM removes the thread from the list of threads to run. R D P U R D... 0 1 2 3 4 5 6 CPU Time (ms) 5

Java s Implementation of Concurrency 2 Main Components: Java s Thread Class and Runnable Interface, and JVM s Thread Queue JVM Thread Objects run Queue T0:YourApp «interface» Runnable main() T1:Thread run() run() T2:Thread Thread run() start() run(). Tn:Thread Java API Java Runtime Environment run() 6

Java s Implementation of Concurrency Basic Structure Java provides a Runnable interface and a Thread class, so that you can access and use its threading capability in your application. To access the threading capability, you must implement the run method of Java s Runnable interface. Java s Thread class implements the run method, but it does not do anything, i.e., the method has an empty body. If an application (such as yours) wants to create a concurrent thread, you need to override the run method of the Thread class, and put your code that is to be run as a thread in your overriding run method. 7

Java s Implementation of Concurrency Basic Structure Once you have put the code you want to run as a thread in a run method, and you have arranged that your run method overrides the run method of the Thread class, then you can use the start method to start your thread. The start method will add a reference to the run() method of your Thread object to the Thread Queue. The JVM executes the run methods in the queue using TDM in a round robin fashion (other priority schemes are allowed). The main method of an application is automatically entered in the queue, and it is the first thread to run in the round-robin queue. 8

Two Ways to Create Threads There are two ways (approaches) you can use to create threads: 1. Extend the Java Thread class, and override the run method. 2. Implement Java s Runnable Interface, and provide a concrete implementation of the run method In either case, you put the code you want to run as a concurrent thread in the run() method. 9

Approach 1: Subclass of Thread Create a subclass that extends the Java Thread class In your subclass, define a method named run(). Put the code that you want to run as a thread in the run method of your subclass. Note: your subclass may have other methods, such as set and get methods, as usual. Your run method overrides Java s Thread s run method. Instantiate your subclass, thus creating a Thread object. Invoke the start method of the Thread object. The start method enters a reference to your run method of your Thread object into the Thread Objects run Queue for execution. 10

Approach 1: Subclass of Thread Described in UML Class Diagram «interface» Runnable run() JVM Thread Objects run Queue main() Thread Provided by Java MC:MyClass run() run() start() run() MyClass run() get() set() Your Class run().. 11

Example of Subclass Approach public class HelloThread extends Thread { The programmer s HelloThread class is a subclass of Java s Thread class. public void run() { System.out.println( Hello from a thread! ); public static void main(string args[]) { (new HelloThread()).start(); 13

Example of Subclass Approach public class HelloThread extends Thread { The programmer creates a run() method, that is intended to be run as a thread. public void run() { System.out.println( Hello from a thread! ); public static void main(string args[]) { (new HelloThread()).start(); 14

Example of Subclass Approach public class HelloThread extends Thread { public void run() { System.out.println( Hello from a thread! ); public static void main(string args[]) { (new HelloThread()).start(); The programmer creates a Thread object. This object normally will inherit the super class s run() method. However, since this class defines another run() method, it will override the superclass s version. 15

Example of Subclass Approach public class HelloThread extends Thread { This run() method overrides the superclass s version. public void run() { System.out.println( Hello from a thread! ); public static void main(string args[]) { (new HelloThread()).start(); 16

Example of Subclass Approach public class HelloThread extends Thread { public void run() { System.out.println( Hello from a thread! ); public static void main(string args[]) { (new HelloThread()).start(); The code then starts the Thread by invoking the start() method, which is inherited from the Thread class. 17

Example of Subclass Approach public class HelloThread extends Thread { public void run() { System.out.println( Hello from a thread! ); public static void main(string args[]) { (new HelloThread()).start(); The start() method enters a reference to the HelloThread object and the associated run() method into the Thread Queue. 18

Approach 2: Implement Runnable Create a class that implements Java s Runnable Interface Define a concrete implementation of the run method in your class. Put the code that you want to run as a thread in the run method of your class Instantiate your class, thus creating a Runnable object. Instantiate a Thread object, passing a reference to your Runnable object to the constructor of the Thread. The constructor will replace the Thread s run method with the run method of your Runnable object that you passed to the constructor. Invoke the start method of the Thread object. References to your Thread object and run method are entered into the Thread Objects run Queue for execution. 19

Approach 2: Implement Runnable «interface» Runnable run() JVM Thread Objects run Queue main() MC:MyClass run() Your Class MyClass run() get() set() Thread run() start() run().. run() 20

Example 1 of Runnable Approach import java.lang.thread; public class ThreadDemo { public static void main( String[] args ) { ARunnable myrunnable = new ARunnable(); //Create a Runnable Thread t = new Thread(myRunnable); //Create a Thread t.start(); //Place thread object in queue, thus starting it // Let the main thread do other stuff: while(true) System.out.println( Main is running ); class ARunnable implements Runnable { ARunnable() { //Constructor would initialize some fields public void run() { while(true) System.out.println( Thread 2 is running"); 21

«interface» Runnable run() JVM Thread Objects run Queue ThreadDemo main() myrunnable: run() ARunnable run() ThreadDemo main() Thread run() start() 22

Example 2 of Runnable Approach public class SomeClass { public static void main(string args[]) { new Thread( new Runnable() { public void run() {//put thread s code here. ).start(); 23

Example 2 of Runnable Approach public class SomeClass { public static void main(string args[]) { new Thread( Creates a Thread object. new Runnable() { public void run() {//put thread s code here. ).start(); 24

Example 2 of Runnable Approach public class SomeClass { public static void main(string args[]) { new Thread( new Runnable() { public void run() {//put thread s code here. ).start(); Normally, you can t instantiate an interface, because an Interface contains abstract methods. For instance, Runnable has an abstract run() method. But, in this case, you are attaching a concrete implementation of the run() method to the object. Thus, when the Runnable object is created, it appears as though the object was created from a class that implements the Runnable interface. This is an example of an anonymous class. 26

Class Diagram of Example 2 Homework 28

Example 3 of Runnable Approach public class HelloRunnable implements Runnable { Runnable is a Java interface used for implementing threads. It declares an abstract method called run(). public void run() { System.out.println("Hello from a thread!"); public static void main(string[] args) { (new Thread( new HelloRunnable()) ).start(); 30

Example 3 of Runnable Approach public class HelloRunnable implements Runnable { The programmer places the code he/she wants to run as a thread into the run() method of a class that implements Runnable. public void run() { System.out.println("Hello from a thread!"); public static void main(string[] args) { (new Thread( new HelloRunnable()) ).start(); 31

Example 3 of Runnable Approach public class HelloRunnable implements Runnable { public void run() { System.out.println("Hello from a thread!"); public static void main(string[] args) { (new Thread( new HelloRunnable()) ).start(); The programmer creates a Thread object (for example, in the main method) 32

Example 3 of Runnable Approach public class HelloRunnable implements Runnable { public void run() { System.out.println("Hello from a thread!"); public static void main(string[] args) { (new Thread( new HelloRunnable()) ).start(); A reference to a Runnable object is passed to the Thread s constructor, which replaces the Thread s run() method with the run() method of the runnable object. 33

Example 3 of Runnable Approach public class HelloRunnable implements Runnable { This run() method overrides the run()method within the Thread object. public void run() { System.out.println("Hello from a thread!"); public static void main(string[] args) { (new Thread( new HelloRunnable()) ).start(); 34

Example 3 of Runnable Approach public class HelloRunnable implements Runnable { public void run() { System.out.println("Hello from a thread!"); public static void main(string[] args) { (new Thread( new HelloRunnable()) ).start(); The code then starts the Thread by invoking the start() method of the Thread object. 35

Runnable Object vs. Subclass Thread Which of these idioms should you use? The Runnable object approach is more general, because the Runnable object can subclass a class other than Thread. The Thread subclass approach is easier to use in simple applications, but is limited because your class must be a descendant of Thread, and, therefore, cannot be a descendant of another class. 1 st : (new Thread( new HelloRunnable()) ).start(); 2 nd : (new HelloThread()).start(); 36

How many threads can a JVM have and the system can maintain perceived concurrency? Plan: create increasing numbers of threads until dragging a window becomes difficult. What is the maximum number of threads that can run concurrently? 37

class ARunnable implements Runnable { String threadname = "Initial Dummy Name"; int count = 0; long starttime = 0, stoptime = 0, elapsedtime=0; ARunnable(String threadname) { this.threadname = threadname; starttime = 0; stoptime = 0; elapsedtime = 0; public void run() { starttime = System.nanoTime(); while (true) { if (++count == 100 000 000) { elapsedtime = System.nanoTime() - starttime; System.out.println(threadName + " " + elapsedtime); break; 38

import java.lang.thread; public class ThreadTest { public static void main(string[] args) { int count = 0; ARunnable[] myrunnables = new ARunnable[50000]; Thread[] thread = new Thread[50000]; for (int i = 0; i < 50000; i++) { myrunnables[i] = new ARunnable(Integer.toString(i)); thread[i] = new Thread(myRunnables[i]); thread[i].start(); //Start the Thread while (true) //Main thread if (++count == 100 000 000) break; 39

Note: Declaring Arrays of Objects ARunnable[] myrunnables = new ARunnable[50000]; The above statement creates the array myrunnables which can hold references to 50,000 ARunnable objects. In other words, the MyRunnables variable contains 50,000 memory spaces, each of which can store the address of an ARunnable object. The above statement does not create the ARunnable objects themselves. They have to be created separately using the constructor of the ARunnable class. 40

Final Elapsed Time of Each Thread 50000 Threads 41

SOME METHODS OF THREAD Sleep Interrupt Join Thread static sleep() run() start() interrupt() join() 42

THREAD STATIC METHOD: sleep() Pausing Execution with Sleep Thread.sleep(time) causes the current run method to suspend execution for a specified time. Makes more efficient use of the processor For instance: if a thread needs to wait for a certain amount of time before doing something, it is better to invoke sleep than to run a timing loop. void run() { //some tasks to do while (notdone) { for (inti=0; i<waittime; i++); //other tasks to do //No other threads can run during //the for loop. Inefficient void run() { //some tasks to do while (notdone) { Thread.sleep(time); //other tasks to do //This allows other threads to run //during the sleep time. More Efficient 43

THREAD INSTANCE METHOD: interrupt() Interrupts Thread objects can be interrupted by other threads (or by themselves). For example: say a mom object needs to wake up a son thread in order to get him to go to work. Code in the mom object would do: son.interrupt(). while (true) { //Portion of the run method of object son try { Thread.sleep(longTime); catch (InterruptedException e) { //mom has interrupted my sleep: Do work now dowork(); // Then go back to sleep :>) 44

THREAD INSTANCE METHOD: join() join The join method allows one thread to wait for the completion of another. If t is a Thread object whose thread is currently executing, and if a current thread invokes: t.join(), then this causes the current thread to pause execution (sleep) until t's thread terminates. 45

SYNCHRONIZATION Threads communicate primarily by sharing access to fields and objects. This form of communication is useful, but makes two kinds of errors possible: thread interference and memory consistency errors. Java uses synchronization to prevent these types of problems. 46

THREAD INTERFERENCE Interference happens when two operations, running in different threads, but acting on the same data, interleave. Example: consider two threads accessing the shared field, c. Thread A performs c++ : 1. Retrieve the current value of c. 2. Increment the retrieved value by 1. 3. Store the incremented value back in c. Thread B performs c-- : 1. Retrieve the current value of c. 2. Increment the retrieved value by 1. 3. Store the incremented value back in c. Due to the time-slicing nature of thread execution, it is possible that the operations of the two threads will interleave. 47

THREAD INTERFERENCE: INTERLEAVING SCENARIO Suppose thread A invokes increment at about the same time thread B invokes decrement. Assume the initial value of c is 0. Due to the time-slicing, a possible scenario is: 1. Thread A: Retrieve c. 2. Thread B: Retrieve c. 3. Thread A: Increment retrieved value; result is 1. 4. Thread B: Decrement retrieved value; result is -1. 5. Thread A: Store result in c; c is now 1. 6. Thread B: Store result in c; c is now -1. Java s synchronization facility can prevent interleaving. 48

MEMORY CONSISTENCY ERRORS Memory consistency errors occur when different threads have inconsistent views of what should be the same data. E.g., Suppose Thread A s change action to variable c completes in a non-interleaved manner. At a later time, Thread B reads variable c, but it reads the old value instead of the new value. The causes of memory consistency errors are complex, e.g., Compiler reordering of instructions and data caching. Establishing a happens-before relationship prevents memory consistency errors. Java s synchronization can prevent Memory consistency errors. 49

SYNCHRONIZATION: 2 BENEFITS 1. Synchronization establishes a happens-before relationship, and, therefore, prevents memory consistency errors. 2. Synchronization guarantees non-interleaving of operations, and, therefore, prevents thread interference. The Java programming language provides two basic ways to do synchronization: synchronized methods. synchronized statements. 50

SYNCHRONIZED METHODS PREVENTING INTERLEAVING public class SynchronizedCounter { private int c = 0; public synchronized void increment() {c++; public synchronized void decrement() {c--; public synchronized int getvalue() {return c; When one thread is executing a synchronized method defined in some object, all other threads that invoke any synchronized method for the same object will block (suspend execution) until the first thread is done with the object. Thus, interleaving is prevented. 51

SYNCHRONIZED METHODS PREVENTING MEMORY CONSISTENCY ERRORS public class SynchronizedCounter { private int c = 0; public synchronized void increment() {c++; public synchronized void decrement() {c--; public synchronized int getvalue() {return c; When a synchronized method exits, it automatically establishes a happens-before relationship with any subsequent invocation of a synchronized method for the same object. This guarantees that changes to the state of the object are visible to all threads. 52

INTRINSIC LOCKS AND SYNCHRONIZATION Every object has an intrinsic lock associated with it. A thread requiring synchronization has to acquire the object s lock before accessing it, and then release the lock when it s done with accessing the object. A thread is said to own the intrinsic lock between the time it has acquired the lock and released the lock. 53

INTRINSIC LOCKS AND SYNCHRONIZATION As long as a thread owns an intrinsic lock, no other thread can acquire the same lock. The other thread will block when it attempts to acquire the lock. When a thread releases an intrinsic lock, a happens-before relationship is established between When the thread releases the lock and Any subsequent acquisition of the same lock. 54

SYNCHRONIZATION EXAMPLE Thread 1 Locked A Running Idle Suspended increment() Thread 2 Object A Locked A Running Idle Suspended Locked Unlocked decrement() 55

SYNCHRONIZATION EXAMPLE Thread 1 calls the increment() method of Object A, and thus, locks Object A. Thread 1 Locked A Running Idle Suspended increment() Thread 2 Object A Locked A Running Idle Suspended Locked Unlocked decrement() 56

SYNCHRONIZATION EXAMPLE Thread 2 attempts to call decrement() on Object A, but it gets blocked, since Object A is locked. Thread 1 Locked A Running Idle Suspended increment() Thread 2 Object A Locked A Running Idle Suspended Locked Unlocked decrement() 57

SYNCHRONIZATION EXAMPLE When increment() completes, Thread 1 releases the lock on Object A. Thread 1 Locked A Running Idle Suspended increment() Thread 2 Object A Locked A Running Idle Suspended Locked Unlocked decrement() 58

SYNCHRONIZATION EXAMPLE Thread 2 resumes and calls the decrement() method of Object A, and thus, locks Object A. Thread 1 Locked A Running Idle Suspended increment() Thread 2 Object A Locked A Running Idle Suspended Locked Unlocked decrement() 59

SYNCHRONIZATION EXAMPLE When Thread 1 is executing the increment() method, the interleaving of the method s operations on field c is impossible because the Object A that contains c is locked. Thread 1 Locked A Running Idle Suspended increment() Thread 2 Object A Locked A Running Idle Suspended Locked Unlocked decrement() 60

SYNCHRONIZATION EXAMPLE A Happens-before relationship is established from the time Thread 1 releases the lock on Object A, and the Thread 2 s subsequent lock on Object A, i.e., i happens before j. Thread 1 Locked A Running Idle Suspended increment() i j Thread 2 Object A Locked A Running Idle Suspended Locked Unlocked decrement() 61

STATIC SYNCHRONIZED METHODS When a thread invokes a synchronized method, it automatically acquires the intrinsic lock for that method s object and releases it when the method returns. When a static synchronized method is invoked, the thread acquires the intrinsic lock for the Class object associated with the class. Thus access to the class s static fields is controlled by a lock that s distinct from the lock for any instance of the class. 62

SYNCHRONIZED STATEMENTS Another way to create synchronized code. Intended to prevent a problem called deadlock. public void addname(string name) { synchronized(this) { lastname = name; namecount++; namelist.add(name); Not all code within the method is synchronized, only the code within the braces {. Thus, when the synchronized statement is completed, the lock on the object is released. Compare this to waiting for the code in the entire method to complete, before the lock can be released. 63

A concurrent application s execution time not only depends on the time taken to execute T tasks, but it also depends if anyone of the tasks becomes suspended due to locks on objects. A concurrent application s ability to execute in a timely manner is known as its liveness. The most common kind of liveness problems: Deadlock. Starvation and livelock. 64

MODELING A NORMAL SCENARIO Thread1 a:classa Thread2 b:classb HOW IT IS SUPPOSED TO WORK c:classf d:classf invoke method m1 on c; thus locking c Print message 5 ms Thread switching time 5 ms return from m1; thus unlocking c invoke m2 on d; thus locking d return from m2; thus unlocking d invoke method m1 on d; thus locking d invoke m2 on c; thus locking c Print message return from m2; thus unlocking c Print message Print message return from m1; thus unlocking d 65

MODELING A DEADLOCK SCENARIO Thread1 a:classa Thread2 b:classb c:classf HOW IT CAN GO WRONG d:classf 1 ms invoke method m1 on c; thus locking c Print message 1 ms invoke method m1 on d; thus locking d Print message 1 ms invoke m2 on d T1 is blocked due to T2 s lock on d. invoke m2 on c T2 is blocked due to T1 s lock on c. 1 ms 66

STARVATION AND LIVELOCK Starvation Shared resources are made unavailable for long periods by greedy threads. Other threads that also need frequent synchronized access to the same object will often be blocked. Livelock Threads are too busy responding to each other instead of doing useful work. 67