Intro to multi-threaded programming in Java

Transcription

1 Intro to multi-threaded programming in Java

2 Intro 1: Multitasking Most operating systems allow multitasking Multiple tasks are run in parallel (given enough processor cores) Word processor, file manager, web browser Background MP3 player, virus checker 2 Process 1 Program No access Tasks are separate programs with their own code, data, stack Process 2 Program Memory heap Memory heap

3 Intro 2: Threads Multi-threading (lightweight processes) A single program can have multiple threads of execution Executed in parallel (given enough CPU cores) Same heap, where objects are stored Different stack, keeping track local variables + method calls 3 Word Processor GUI Thread Print thread Background spell checker Shared heap No access Web Server Request handler Request handler Request handler Shared heap

4 Intro 3: Threads and Java Java is fundamentally multi-threaded Create threads using the Thread class Manage access to resources using the synchronized keyword Handle communication between threads using wait(), notify() methods 4 Threads for a standard non-gui program (Java 7, Windows): Group: system Max Priority: 10 Reference Handler [pri 10, Daemon] -- Handles weak references Finalizer [pri 8, Daemon] -- Runs finalizers Signal Dispatcher [pri 9, Daemon] Attach Listener [pri 5, Daemon] Group: main Max Priority: 10 main[pri 5] -- The thread that calls main() Additional Threads for a Swing GUI program (Java 7, Windows): Java2D Disposer [pri 10, Daemon] AWT-Windows [pri 6, Daemon] Daemon: Background services, won t prevent the program from terminating

5 Intro 4: Creating Threads To create a thread, first define what it should do The object-oriented way: An object with a specific method to run public class DocSaver implements Runnable { public void run() { save the document to disk Second step: Create an instance of your Runnable class final Runnable saver = new DocSaver(); Runnable interface: A single method 5 Third step: Create a Thread that will execute the Runnable final Thread t1 = new Thread(saver, "Document Saver"); Fourth step: Start the thread! t1.start(); Gives a name to the thread Creates an operating system thread with stack, etc.

6 Intro 5: Thread States Threads can be in one of four states: New created with new Thread(), not yet started Runnable has or wants the CPU The scheduler determines which runnable thread(s) will run Time allocation depends on thread priority Blocked waiting for something to happen Blocking on I/O Called sleep() Called obj.wait() waits for I/O to complete waits n milliseconds waits until notified 6 Thread. start() Done waiting Returns to runnable Dead stopped running The Runnable's run() method has returned But the Thread object still exists (not yet garbage collected)

7 Topic 1: Sharing Resources Threads often share resources 7 Document saving thread Spell checker thread Document editing thread Document object Swing thread (events, painting) What if a document changes while it is being saved? Requires support for coordination, mutual exclusion between threads,

8 Topic 2: Cooperation 8 Threads sometimes cooperate to achieve a common purpose Coordinating thread in image processing software Image Processing on core 1 Image Processing on core 2 Image Processing on core 3 Image Processing on core 4 How to communicate results? Requires support for communication between threads

9 When threads cooperate

10 Communication 1: Producer/Consumer Develop a multi-threaded queue (producer/consumer) 10 Potential delays Producer thread: Read data from disk, append to queue Consumer thread: Read data from queue, play a sound

11 Communication 2: Producer/Consumer 11 class Producer implements Runnable { private final MsgQueue queue; public Producer(MsgQueue q) { this.queue = q; Read data from disk, append to queue public void run() { while (true) { Msg data = [[read]]; queue.put(data); Read from queue, decode MP3, play sound class Consumer implements Runnable { private final MsgQueue queue; public Consumer(MsgQueue q) { this.queue = q; public void run() { while (true) { Msg data = queue.take(); [[play sound from data]];

12 Communication 3: First Attempt 12 First attempt: Let's use an ArrayList! public class MsgQueue { List<Msg> list = new ArrayList<Msg>(); public synchronized void put(msg msg) { list.add(msg); // At the end public synchronized Msg take() { return list.remove(0); Problem! If the queue is empty, there is no element to pop ArrayIndexOutOfBoundsException Solution! The consumer must wait for an element to arrive from the producer! Java provides obj.wait() and obj.notify() for thread communication tricky!

13 Communication 4: Concurrent Queues Fortunately, Java already provides concurrent queues! Usually don't need to use the low-level methods Easy to make mistakes interface BlockingQueue<E> extends Collection<E>: Supports blocking and timeouts 13 What happens if the operation can t be performed (now)? Insert (queue may be full) Remove (may be empty) Examine (may be empty) Throw exception Return null or false add(e) offer(e) put(e) Block (wait until the operation can be performed) Time out (after a specific interval) offer(e, time, unit) remove() poll() take() poll(time, unit) element() peek() not applicable not applicable

14 Concurrent (Blocking) Queues 14 Implemented by ArrayBlockingQueue SynchronousQueue LinkedBlockingQueue LinkedBlockingDeque DelayQueue PriorityBlockingQueue LinkedTransferQueue bounded size zero capacity, each insert matched with a remove each element is available when its delay expires where producers may wait for consumers

15 Shared Resources When is a Program Thread Safe? Mutual Exclusion: Conceptual View and Java Syntax

16 Thread Safety 1: Intro Threads accessing the same data must not interfere! Simple example: ArrayList class used from multiple threads public class ArrayList<E> { private int size = 0; private E[] elements = (E[]) new Object[100]; public void add(e obj) { elements[size] = obj; size++; public void clear() { elements = (E[]) new Object[100]; size = 0; 16

17 Thread Safety 2: Thread Switching 17 With a single CPU core The computer quickly switches between active threads (time slicing) This happens even inside methods! With multiple CPU cores Multiple threads can be running truly simultaneously Timer Swing Timer Swing begin update() continue begin repaint() begin update() begin repaint() continue continue continue

18 Thread Safety 3: Within Methods What happens within a method? 18 Java source (text): Hello.java Compiler: javac Hello.java Bytecode (binary intermediate format): Hello.class Standardized, can be distributed class HelloWorld { public static void main(string[] args) { System.out.println("Hello World!"); Method void main(java.lang.string[]) b2 00 0b 12 0f b c a b2 00 0b bb getstatic #12 <Field java.io.prints ldc #15 <String "Hello, world"> invokevirtual #20 <Method void iconst_0 istore_1 goto 39 getstatic #12 <Field java.io.prints new #23 <Class java.lang.stringbu

19 Thread Safety 4: Fine-Grained Switching 19 Even with one core: Can switch threads after any atomic (indivisible) bytecode operation Very fine-grained! Thread 1 executing size++ push size value onto stack increment top of stack Thread 2 executing a[i] = j push a onto stack push i onto stack push j onto stack arraystore Bytecode instructions store value at top of stack in size

20 Thread Safety 5: Race condition! Suppose two threads call add(a) / add(b) concurrently public void add(e obj) { elements[size] = obj; size++; Might expect: [A,B] or [B,A]. This is not guaranteed! Called a race condition: Depends on "who gets there first" (exactly how far does one thread get before it is interrupted?) 20 read size: 0 store A at [0] read size: 0 inc 1 store size: 1 read size: 1 store B at [1] read size: 1 inc 2 store size: 2 [A,B] read size: 0 store A at [0] read size: 0 read size: 0 store B at [0] read size: 0 inc 1 store size: 1 inc 1 store size: 1 [B] read size: 0 store A at [0] read size: 0 store B at [0] read size: 0 inc 1 store size: 1 read size: 1 inc 2 store size: 2 [B,null]

21 Thread Safety 6: One Requirement One common requirement for thread safety: Parallel execution should be equivalent to some serialization The result should be as if the concurrent calls from multiple threads had been made in some sequence from a single thread Only the result counts if you can still do it concurrently, it's better! 21 If you do this (multiple CPUs) or this (one CPU) The result should be as if you did this or as if you did this. add(a) add(a) add(b) add(b) add(a) add(b) add(b) add(a)

22 Thread Safety 7: Cleverness 22 How is this type of thread safety achieved? Sometimes through very clever algorithms and data structures ConcurrentLinkedQueue threadsafe linked queue ConcurrentLinkedDeque threadsafe linked double-ended queue ConcurrentHashMap ConcurrentSkipListMap Methods do execute like this add(a) add(b) or like this add(a) add(b) efficient multithreaded access without locking another variation...but regardless of scheduling, correct results are guaranteed! Beyond the scope of this course!

23 Thread Safety 8: Mutexes How is this type of thread safety achieved? Usually through mechanisms for mutual exclusion 23 Can t add() to the same list from two threads concurrently One thread will have to wait for the first add() to complete Make sure you can t do this or this You can only do this or this. add(a) add(a) add(b) add(b) add(a) add(b) add(b) add(a)

24 Mutex 1: Semaphore/Monitor 24 One mechanism for mutual exclusion: Semaphore / monitor semaphore.acquire() Waits until no other thread has the semaphore, then "locks" it for this thread semaphore.release() Releases the semaphore, so that some other thread can use it These functions use specific low-level machine instructions to avoid race conditions list.add(a): list.semaphore.acquire(); list.semaphore.release(); list.add(b): list.semaphore.acquire(); list.semaphore.release(); This allows you to use the semaphore at a higher level of abstraction

25 Mutex 2: Monitors in Java Mutual exclusion and semaphores / monitors in Java: Every object is associated with a monitor Acquired and released through synchronized blocks: synchronized(myobject) { // Acquire() myobject s monitor // Implicitly release() it again Synchronized blocks provide structure to monitors Automatically release()d when you exit the block: You can't forget to release them, even if an exception terminates the method 25

26 Mutex 3: Monitors in ArrayList Must use a monitor in the list example Which one? Create a new object public class ArrayList<E> { private int size = 0; private E[] elements = (E[]) new Object[100]; private Object monitor = new Object(); 26 public void add(e obj) { synchronized(monitor) { // acquire monitor elements[size] = obj; size++; // release monitor public void clear() { synchronized(monitor) { elements = (E[]) new Object[100]; size = 0;

27 Mutex 4: Acquiring a Monitor Suppose thread A calls add(a) First: synchronized(monitor) acquires the list s monitor Atomic test-and-set: acquired is false, so make it true Make thread A the owner of the monitor A: acquire A: read size 0 A: store A at 0 27 Conceptual view the monitor is not a Java object with fields! before synchronized(mon) { acquired owner lockqueue false Thread object: Thread A, runnable within synchronized(mon) { acquired owner lockqueue true Thread A runnable

28 Mutex 5: Wait Queue If the scheduler lets B run, before A is done: Method add() calls synchronized(mon) Tries to acquire the monitor Atomic test-and-set: acquired is true, so we fail Thread B is blocked, placed in a wait queue! 28 A: read size 0 B: try acquire B: block [other threads might run here] When thread B is blocked: The scheduler selects another runnable thread (not necessarily A!) acquired owner lockqueue true Thread A runnable Thread B blocked Thread C runnable Thread D runnable

29 Mutex 6: Wait Queue 29 Eventually, thread A is scheduled to run again Finishes its work (no interference!) and releases the monitor Thread B is unblocked (runnable), but thread A continues running acquired owner lockqueue Eventually, thread B is scheduled to run again Thread B is allowed to acquire the monitor acquired owner lockqueue false true Thread A runnable Thread B runnable Thread B runnable A: read size 0 B: try acquire B: block [other threads run] A: inc 1 A: store size 1 A: release A: keep working

30 Mutex 7: Cooperative Be careful: Mutual exclusion is cooperative 30 Not like placing the object in a safe More like this: ArraySet obj. ArraySet obj. header header elements: [] size: 0 elements: [] size: 0 public void getsize() { return size; No synchronized() block can be called in parallel with add() or any other method! Fields can be accessed Methods can be called Everyone must cooperate by checking the note (trying to acquire the monitor)!

31 Mutex 8: All Methods Synchronized? Must all methods use synchronization? Depends on your class // No synchronized() can be called in parallel with other methods public void getsize() { return size; // Calling unsynchronized getsize() in parallel with clear1() // returns either the old size, or the new size 0 OK! public void clear1() { synchronized(monitor) { elements = (E[]) new Object[100]; size = 0; // Calling unsynchronized getsize() in parallel with clear2() // may return an "intermediate" size, that the list never really had! // (One solution: synchronization in getsize() too) public void clear2() { synchronized(monitor) { while (size > 0) elements[--size] = null; 31

32 Mutex 9: Cooperative Protection How to ensure thread safety, if mutexes are cooperative? Anyone can just skip synchronization, deliberately or by mistake 32 Use standard public/private protection! Make sure all fields are private Public fields can be accessed regardless of synchronization Other threads can read intermediate values of public fields Other threads can change values while mutators are being executed But private fields are only accessed within the class which you control! Make sure all methods use synchronization where necessary

33 Conclusions: Complete Thread Safety How do you ensure complete thread safety? The same way you write a bug-free program Think! Consider all possible combinations of circumstances! Do everything right! Requires policy / design decisions, which must be followed Complicated because high parallelism is desired That is the reason why we use threads in the first place Don't want too much mutual exclusion 33 Sorry, but there are no simple answers!

34

35 Synchronization Overhead 35 Synchronization has some overhead! public class SynchTester { public static void main(string[] args) { for (int i = 0; i < 5; i++) doit(); private static void doit() { SynchTester tester = new SynchTester(); long start = System.currentTimeMillis(); ms int value = 1; 1368 ms for (int i = 0; i < 1_000_000_000L; i++) { 539 ms value = tester.increment(value); 545 ms 545 ms System.out.println((System.currentTimeMillis()-start) + "ms: " + value); public synchronized int increment(int var) { return var + 1; Without synchronized: 651 ms 571 ms 566 ms 553 ms 548 ms With synchronized: Huge proportional diff: The method is trivial in itself In this case, the method is dynamically optimized to remove the overhead. Not always possible!

36

37 Synchronization and Collections Collection classes: Often used in single-threaded context Three alternatives: 37 Don t use synchronization public void add(object obj) { // Lots of code adding an element Do use synchronization public void add(object obj) { synchronized(monitor) { // Code adding an element If you use from multiple threads: Risk incorrect results (lost elements, ) Unnecessary overhead in most cases Provide both versions! A lot of redundant code right?

38 Synchronization and Collections (2) 38 Solution: Do provide two versions, don t write collections twice Don t use synchronization public void add(object obj) { // Lots of code adding an element Do use synchronization public synchronizedvoid add(object obj) { // Quick call to the real class: // Delegation

39 Collections: Synchronization Wrappers 39 To achieve this: Use a synchronization wrapper synchlist public class SynchList implements List { private List storage; private Object monitor = ; public SynchList(List storage) { this.storage = storage; public void add(object obj) { synchronized(monitor) { storage.add(obj); "SynchList" Object header: storage (data) ArrayList Object header: (data) size 0 elements Uses the decorator design pattern Implements List can be used everywhere Header length [0] [1] [2] [3] (data) 4

40 Built-in Synchronization Wrappers Synchronization wrappers exist in the Collections framework! List<String> synchlist = Collections.synchronizedList(new ArrayList<String>()); 40

41

42 // Schedule a task to run repeatedly, starting now, // 500 ms from execution n begins to execution n+1 begins t.scheduleatfixedrate(task, new Date(), 500); Timers 42 To schedule arbitrary tasks: java.util.timer Starts a separate thread for each Timer not dependent on Swing timers import java.util.timer; final TimerTask task = new TimerTask() { public void run() { System.out.println( Running!"); ; Timer t = new Timer(); // Schedule a task to run a single time, after 500 ms t.schedule(task, 500); Don t manipulate an existing GUI here: Only permitted in the Swing thread! // Schedule a task to run repeatedly, starting now, // 500 ms from execution n ends to execution n+1 begins t.schedule(task, new Date(), 500);