Computer Networks Process Cooperation Multithreaded Programming in Java ß ß Laszlo Böszörmenyi Computer Networks Processes - 1
Processes and threads A process is a sequential flow of control that is able to cooperate with other processes Heavy-weight light-weight process Thread (of control) lightweight process We use them as synonyms, if difference irrelevant Address space Context Context switch (Heavy) Process Own Large (file descriptors, I/O buffers) Slow Thread Common (but: own stack) Small (a few registers) Fast Laszlo Böszörmenyi Computer Networks Processes - 2
Parallel and concurrent execution True parallel execution Processes run on own processor, with own register set Concurrent (quasi parallel) execution One processor and register set is shared Registers have to be saved and restored Usually slow, but still meaningful for parallel problems Processor Execution Parallel Own Truly parallel Quasi parallel Common Virtually parallel (concurrent) Laszlo Böszörmenyi Computer Networks Processes - 3
Creating processes (fork & join) Calling a procedure suspends the caller Forking a thread creates two parallel threads Each with an own stack Let be P1 and P2 procedures P1 call P1 fork waits. P2 runs runs P2 runs.. return join Laszlo Böszörmenyi Computer Networks Processes - 4
Creating threads in Java A thread is created by creating an instance of either A subclass of the class Thread or A class that implements the Runnable interface Some basic methods of threading The run method must be overridden by required code Call of start causes the run method to be forked join suspends the caller until the target thread completes and merges the 2 threads into 1 sleep causes thread to sleep for a given time (msecs) yield lets run other threads sleep(0) stop, suspend : DO NOT USE (deprecated anyway) Laszlo Böszörmenyi Computer Networks Processes - 5
Example, thread as a subclass class MyThread extends Thread { // MyThread is subclass of Thread public void run() { // overrides Threads run System.out.println("Hello, this is " + getname()); } // run } // MyThread public class ExtendedThread { static public void main(string args[]) { MyThread a = new MyThread(); MyThread b = new MyThread(); a.start(); // forks run of a } } // main Arbitrary order Hello, this is Thread-0 This is main Hello, this is Thread-1 main T0 T1 b.start(); // forks run of b /*3 Threads: a, b, and main run now quasi-parallel*/ System.out.println("This is " + Thread.currentThread().getName()); Laszlo Böszörmenyi Computer Networks Processes - 6
Example, thread implementing runnable class MyThread implements Runnable { public void run() { System.out.println("Hello, this is " + Thread.currentThread().getName()); /* Unqualified getname is now unknown! */ } // run } // MyThread public class RunnableThread { static public void main(string s[]) { MyThread work2do; Thread a, b; work2do = new MyThread(); a = new Thread(work2do); b = new Thread(work2do); a.start(); b.start(); main T0 T1 System.out.println("This is " + Thread.currentThread().getName()); } } Arbitrary order Hello, this is Thread-0 This is main Hello, this is Thread-1 Laszlo Böszörmenyi Computer Networks Processes - 7
Process cooperation Synchronization Processes meet in a point of time ( rendez-vous ) Mutual exclusion and conditional synchronization Communication Synchronous Processes exchange data at synchronization point Asynchronous Exchange data via persistent medium (files, mailboxes, buffers) Kinds of data exchange Common memory, if available in threads Messages, via communications channels If common memory not available normal for distributed systems Laszlo Böszörmenyi Computer Networks Processes - 8
Critical sections A critical section (CS) is a piece of code, in which More than one process access shared data At least on of them wants to modify these data Criteria for correct handling Mutual exclusion (mutex) at most 1 process inside Progress deadlock is avoided Limited wait starvation is avoided Rules of a correct protocol Inside the CS at most 1 process CS must be left in finite time otherwise no progress. In case of crash enforce exit If a process terminates in the rest, this must not have influence on the other processes Steps of mutex Entry protocol Critical section Exit protocol Rest Laszlo Böszörmenyi Computer Networks Processes - 9
Mutual exclusion in Java synchronized keyword marks as critical section an entire method synchronized void P () { // enter // All statements of P are executed under mutual exclusion } // exit a piece of code inside a method void P () { //... Non-synchronized code synchronized (this) { // enter // These statements are executed under mutual exclusion } // exit //... Non-synchronized code } // P Laszlo Böszörmenyi Computer Networks Processes - 10
Conditional synchronization Entering a CS via mutex is unconditional An action may need a precondition fulfilled later Conditional waiting is necessary CS must be left! When condition fulfilled: notify waiting threads Semaphores can be used for both (see later) Mutual exclusion with binary semaphores Conditional synchronization with general semaphores Examples (see later) Bounded buffer Dining philosophers Readers / writers problem Laszlo Böszörmenyi Computer Networks Processes - 11
Monitors (C. A. R. Hoare) Combines the idea of hiding and concurrency A monitor is a special container (class or module) Mutual exclusion is per definition guaranteed for all procedures of a monitor Condition variables with special operations Wait The waiting thread relinquishes the CS (releases lock) Signal (notify) Awakes a thread, which re-enters the CS (gets lock) Awakening my be Immediate (preemptive) or delayed (non-preemptive) Explicit or implicit (automatically by changing the condition) Laszlo Böszörmenyi Computer Networks Processes - 12
Conditional synchronization in Java (1) Each object provides the methods wait, notify, notifyall Wait suspends the thread releases the lock on the object it waits for all other locks are retained Awaking waiting threads by notify and notifyall notification reacquires the lock (reenters the CS) notify wakes one waiting thread notifyall wakes all threads waiting on the object notification remains without effect, if no waiting thread This may cause errors: If signal comes earlier then wait Laszlo Böszörmenyi Computer Networks Processes - 13
Conditional synchronization in Java (2) Signaling is explicit and delayed Additional state-check is necessary, because another thread might have changed the condition in between Proper pattern while (!condition) try {wait();} catch ( ) {}; Thread leaves CS until awaken Re-check condidtion Never use if (!condition) try {wait();} catch ( ) {}; A java class, where all methods are synchronized: a monitor Not perfectly the original Hoare-monitor The synchronized modifier must be set manually Awaking is delayed enabling notifyall (broadcast) CS regained Laszlo Böszörmenyi Computer Networks Processes - 14
Bounded buffer Definition Buffer has a finite capacity (length) Pointer in resp. out point to the next empty resp. filled slot in the buffer Counter used shows the actually filled slots The buffer builds a ring in and out are incremented modulo length Producer threads can deposit objects via put Producers has to wait if buffer full (used == length) Consumer threads can take objects via get Consumers has to wait if buffer empty (used == 0) Access to the buffer elements must be in a CS The CS must be temporarily relinquished if condition false Laszlo Böszörmenyi Computer Networks Processes - 15
cons i cons i cons i prod i prod i prod j Bounded buffer Implementation via monitor public class BoundedBufferMon { private int in, out, used = 0; private Object [] data; public BoundedBufferMon (int bufsize) { in = 0; out = 0; used = 0; data = new Object[bufSize]; // bufsize > 0! } // BoundedBufferSem public synchronized Object get () { while (used == 0) try {wait();} catch(interruptedexception e) {}; // hidden (private) variables Object d = data[out]; used > 0! CS regained out = (out + 1) % data.length; if (used-- == data.length) notifyall(); Advance return d; ring pointer Awake waiting } // get producers public synchronized void put (Object d) { while (used == data.length) try {wait();} catch(interruptedexception e) {}; data[in] = d; in = (in + 1) % data.length; if (used++ == 0) notifyall(); } // put } // BoundedBufferMon Thread leaves CS until awaken used < length! CS regained Laszlo Böszörmenyi Computer Networks Processes - 16
Bounded buffer Producer thread class Producer extends Thread { // Subclass of Thread private int myid; // Thread identifier private BoundedBufferMon buf; // Buffer shared with other threads private boolean isrunning = true; Producer(int identifier, BoundedBufferMon buffer) { // Constructor myid = identifier; buf = buffer; } public void stopp() { isrunning = false; } public void run() { // overrides run method of the Thread class while (isrunning) { String message = new String("P-" + myid + ": " + Calendar.getInstance().getTimeInMillis() % 10000); buf.put(message); try {Thread.sleep(myId*10 + 10);} catch(interruptedexception e) {} } // while buf.put("stop"); } // run } // Producer Laszlo Böszörmenyi Computer Networks Processes - 17
Bounded buffer Consumer thread class Consumer extends Thread { // Subclass of Thread private int myid; // Thread identifier private int RunningP = BufUserMon.NProc; // Number of running producers private BoundedBufferMon buf; // Buffer shared with other threads private boolean isrunning = true; Consumer(int identifier, BoundedBufferMon buffer) { myid = identifier; buf = buffer; } // Constructor public void run() { // overrides run method of the Thread class do { String s = (String) buf.get(); System.out.println("C-" + myid + " read: " + s); if (s.equals("stop") RunningP--; // 1 producer stopped } while (RunningP > 0); // assuming 1 consumer; NProc producers } } // Consumer Laszlo Böszörmenyi Computer Networks Processes - 18
Bounded buffer User class public class BufUserMon { // Uses bounded buffer, NProc producers, 1 consumer public final static int NProc = 2, NBuf = 8; static public void main(string args[]) { Producer [] p = new Producer [NProc]; BoundedBufferMon buffer = new BoundedBufferMon(NBuf); for (int i = 0; i < NProc; i++) { // Create and fork producers p[i] = new Producer(i, buffer); p[i].start(); } // All share the same buffer Consumer c = new Consumer(0, buffer); c.start(); // Fork cons. for (int i = 0; i < NProc; i++) { // Let run producers try {Thread.sleep(100);} catch(interruptedexception e) {} p[i].stopp(); // Stop and join producers try {p[i].join();} catch(interruptedexception e) {}; } // All producers merged with main thread try {c.join();} catch(interruptedexception e) {}; // Join cons. System.out.println(Thread.currentThread().getName()); } // main } // BufUserMon Starts P-0 Starts P-1 Starts C-0 C-0 read: P-0: 13822 C-0 read: P-1: 13822 C-0 read: P-0: 13838 C-0 read: P-0: 13854 C-0 read: P-1: 13854 C-0 read: P-0: 13869 C-0 read: P-0: 13885 C-0 read: P-1: 13885 C-0 read: P-0: 13901 Stops P-0 C-0 read: P-1: 13916 C-0 read: stop C-0 read: P-1: 13947 C-0 read: P-1: 13979 C-0 read: P-1: 14010 Stops P-1 C-0 read: stop Stops C-0 main Laszlo Böszörmenyi Computer Networks Processes - 19
Semaphores (E. W. Dijkstra) Operation P to enter and V to exit np: number of completed P operations nv: number of completed V operations IN: initial value Semaphore invariant: np nv + IN If integer s = IN + nv np: s 0 < > should embrace an atomic action P(s) <await s > 0 s = s 1> V(s) <s = s + 1> To a railway station with IN rails, IN trains may drive in. After this, semaphore switches to red. Laszlo Böszörmenyi Computer Networks Processes - 20
Implementing Semaphores in Java public class Semaphore { private int s = 1; // Semaphore variable (only P and V allowed) public Semaphore (int InitialValue) { s = InitialValue; } public synchronized void P() { // entry (synchronized atomic method) while (s <= 0) { // Waits as long as s becomes positive try {wait();} catch(interruptedexception e) {}; } // s > 0 is guaranteed s--; s > 0! } CS regained public synchronized void V() { s++; notify(); } // exit (synchronized atomic method) // increments semaphore variable // notifies the first waiting thread } // Semaphore Laszlo Böszörmenyi Computer Networks Processes - 21
Bounded buffer with semaphores public class BoundedBufferSem { private int in, out = 0; private Semaphore empty, full, mutex; private Object [] data; public BoundedBufferSem (int size) { // Constructor; size > 0! in = 0; out = 0; data = new Object[size]; empty = new Semaphore(size); full = new Semaphore(0); mutex = new Semaphore(1); } // BoundedBufferSem public Object put () { // not synchr.! empty.p(); mutex.p(); data[in] = d; in = (in + 1) % data.length; mutex.v(); full.v(); } // put public Object get () { // not synchr.! full.p(); mutex.p(); Object d = data[out]; out = (out + 1) % data.length; mutex.v(); empty.v(); return d; } // get Laszlo Böszörmenyi Computer Networks Processes - 22
The dining philosophers 5 philosophers are sitting at a table They are thinking and eating in a cycle forever On the table are 5 plates and 5 forks Unfortunately, for eating one needs 2 forks A protocol is required that excludes Deadlock E.g. all philosophers take the left fork and wait for the right one Starvation Two non-neighboring philosophers eat alternating excluding the one between them Either give priority those who wait longer, or let them leave a virtual room after eating A special case for general resource management Laszlo Böszörmenyi Computer Networks Processes - 23
Dining philosophers fork management public class Forks { // Free of deadlock, not free from starvation private int N; private int [] forks; // forks[i]: number of available forks for p[i] public Forks (int num) { // At start all forks available N = num; forks = new int [N]; for (int i = 0; i < N; i++) forks[i] = 2; } private int left(int id) { if (id == 0) return N-1; else return (id-1) % N; } private int right(int id) { if (id == N-1) return 0; else return (id+1) % N; } public synchronized void pickup(int id) { // Wait as long as both forks available while (forks[id]!= 2) try {wait();} catch(interruptedexception e) {}; forks[left(id)]--; forks[right(id)]--; } // pickup public synchronized void putdown(int id) { forks[left(id)]++; forks[right(id)]++; notifyall(); // NotifyAll is not fair, therefore starvation possible } // putdown } // Forks Laszlo Böszörmenyi Computer Networks Processes - 24
Dining philosophers philosopher threads class Philosopher extends Thread { private int id; private Forks f; private Random rd = new Random(); private boolean isrunning = true; public Philosopher(Forks forks, int processid) { f = forks; id = processid; } public void stopp() { isrunning = false; } public void run() { while (isrunning) { f.pickup(id); System.out.println("Philosopher-" + id + " eating"); try {sleep(rd.nextint(10));} catch(interruptedexception e) {} f.putdown(id); System.out.println("Philosopher-" + id + " thinking"); try {sleep(rd.nextint(100));} catch(interruptedexception e) {} } } // run } // Philosopher Laszlo Böszörmenyi Computer Networks Processes - 25
public class ForkUser { final static int N = 5; Dining philosophers user class static public void main(string args[]) { Forks f = new Forks(N); Philosopher [] ph = new Philosopher [N]; for (int i = 0; i < N; i++) { ph[i] = new Philosopher(f, i); ph[i].start(); } try {Thread.sleep(500);} catch(interruptedexception e) {} for (int i = 0; i < N; i++) { ph[i].stopp(); try {ph[i].join();} catch(interruptedexception e) {} } System.out.println("main stops"); } } // ForkUser Philosopher-2 eating Philosopher-2 thinking Philosopher-0 eating Philosopher-0 thinking Philosopher-4 eating Philosopher-4 thinking Philosopher-2 eating Philosopher-2 thinking Philosopher-0 stops Philosopher-4 eating Philosopher-4 thinking Philosopher-3 eating Philosopher-1 stops Philosopher-3 thinking Philosopher-2 stops Philosopher-4 eating Philosopher-4 thinking Philosopher-4 eating Philosopher-4 thinking Philosopher-3 stops Philosopher-4 stops main stops Laszlo Böszörmenyi Computer Networks Processes - 26
Dining philosophers with semaphores (1) public class ForksSemD { private Semaphore [] forks; private int N; public ForksSem (int num) { N = num; forks = new Semaphore [N]; for (int i = 0; i < N; i++) forks[i] = new Semaphore(1); // Deadlock and starvation danger // mutex for access to forks[i] // At the beginnig al forks available } int left(int id) {if (id == 0) return N-1; else return (id-1) % N;} int right(int id) {if (id == N-1) return 0; else return (id+1) % N; } public void pickup(int id) { forks[left(id)].p(); forks[right(id)].p(); } // pickup public void putdown(int id) { forks[left(id)].v(); forks[right(id)].v(); } // putdown } // ForksSemD Does it help to make these synchronized? Laszlo Böszörmenyi Computer Networks Processes - 27
Dining philosophers with semaphores (2) public class ForksSem { // Neither deadlock nor starvation private Semaphore [] forks; // mutex for access to forks[i] private Semaphore room; private int N; public ForksSem (int num) { // At the beginnig al forks available N = num; forks = new Semaphore [N]; room = new Semaphore(N-1); for (int i = 0; i < N; i++) forks[i] = new Semaphore(1); } int left(int id) {if (id == 0) return N-1; else return (id-1) % N;} int right(int id) {if (id == N-1) return 0; else return (id+1) % N; } public void pickup(int id) { room.p(); forks[left(id)].p(); forks[right(id)].p(); } // pickup public void putdown(int id) { forks[left(id)].v(); forks[right(id)].v(); room.v(); } // putdown } // ForksSem Laszlo Böszörmenyi Computer Networks Processes - 28
The readers/writers problem In many applications many reads, few writes Let allow all readers in parallel, but only 1 writer If all processes read, no conflict may arise Cannot be solved by simple mutual exclusion If a process wants to write Wait as long as all readers finished Do not allow new readers to enter Number of active writers is 0 or 1 If it is 1 then the number of active readers is 0 If number of waiting writers > 0 then allow no more new readers to enter Laszlo Böszörmenyi Computer Networks Processes - 29
Readers/Writers (1) public interface Reader { public void doread(); } // implement in subclasses public interface Writer { public void dowrite(); } // implement in subclasses public abstract class RW implements Reader, Writer { private int activereaders = 0; // threads reading private int activewriters = 0; // always zero or one private int waitingreaders = 0; // threads waiting to read private int waitingwriters = 0; // same for write public void read() { beforeread(); doread(); afterread(); } // read public void write() { beforewrite(); dowrite(); afterwrite(); } // write // Allow arbitrary many readers // Entry protocol for read // Execute actual read implemented elsewhere // Exit protocol for read // Allow exactly 1 writer // Entry protocol for write // Execute actual write implemented elsewhere // Exit protocol for write Laszlo Böszörmenyi Computer Networks Processes - 30
Readers/Writers (2) protected boolean allowreader() { return waitingwriters == 0 && activewriters == 0; protected boolean allowwriter() { return activereaders == 0 && activewriters == 0; } // allowreader } // allowwriter protected synchronized void beforeread() { ++waitingreaders; while (!allowreader()) try { wait(); } catch (InterruptedException ex) {} --waitingreaders; ++activereaders; }// beforeread protected synchronized void afterread() { --activereaders; notifyall(); } // afterread protected synchronized void beforewrite() { ++waitingwriters; while (!allowwriter()) try { wait(); } catch (InterruptedException ex) {} --waitingwriters; ++activewriters; } // beforewrite protected synchronized void afterwrite() { --activewriters; notifyall(); } // afterwrite } // RW Laszlo Böszörmenyi Computer Networks Processes - 31
Readers/Writers reader threads class ReaderThread extends Thread { private int count = 0; private Random rd = new Random(); private RW rw_c; private boolean isrunning = true; public ReaderThread (RW rw_control) {rw_c = rw_control; } public void stopp() { isrunning = false; } public void run() { while (isrunning) { rw_c.read(); count++; System.out.println(getName() + "-read: " + count); try {sleep(rd.nextint(10));} catch(interruptedexception e) {} } System.out.println("Reader-"+ getname() + " stops"); } // run } // ReaderThread Laszlo Böszörmenyi Computer Networks Processes - 32
Readers/Writers writer threads class WriterThread extends Thread { private int count = 0; private Random rd = new Random(); private RW rw_c; private boolean isrunning = true; public WriterThread (RW rw_control) {rw_c = rw_control; } public void stopp() { isrunning = false; } public void run() { while (isrunning) { rw_c.write(); count++; System.out.println(getName() + "-write: " + count); try {sleep(rd.nextint(100));} catch(interruptedexception e) {} } System.out.println("Writer-"+ getname() + " stops"); } // run } // WriterThread Laszlo Böszörmenyi Computer Networks Processes - 33
Readers/Writers user class public class RWUser extends RW { static final int NReaders = 5, NWriters = 2; static int totalreads, totalwrites = 0; public void doread() {totalreads++;} // Overrides abstract read: counts reads public void dowrite() {totalwrites++;} // Overrides abstract write: counts writes static public void main(string args[]) { } } RWUser rw_control = new RWUser(); // Controls access for readers/writers ReaderThread [] r = new ReaderThread [NReaders]; WriterThread [] w = new WriterThread [NWriters]; for (int i = 0; i < NWriters; i++) {w[i] = new WriterThread(rw_control); w[i].start();} for (int i = 0; i < NReaders; i++) {r[i] = new ReaderThread(rw_control); r[i].start(); try {Thread.sleep(500);} catch(interruptedexception e) {} for (int i = 0; i < NWriters; i++) { w[i].stopp(); try { w[i].join(); } catch (InterruptedException ex) {} } for (int i = 0; i < NReaders; i++) { r[i].stopp(); try { r[i].join(); } catch (InterruptedException ex) {} } System.out.println("totalWrites = " + totalwrites); System.out.println("totalReads = " + totalreads); Thread-4-read: 44 Thread-5-read: 46 Thread-6-read: 40 Thread-6-read: 41 totalwrites = 17 totalreads = 212 Laszlo Böszörmenyi Computer Networks Processes - 34
Time-outs in Java sleep(t) thread sleeps for t milliseconds and keeps the lock wait(t) thread sleeps for t milliseconds, but lock is released public synchronized void ProcessLoop() { processone(); try {wait(1000);} catch (Exception e) {} processtwo(); } public void ProcessLoop() { synchronized (this) { processone(); } } // lock released // lock regained // lock released manually try {Thread.sleep(1000);} catch (Exception e) {} synchronized (this) { processtwo(); } // lock regained manually Laszlo Böszörmenyi Computer Networks Processes - 35
Scheduling (1) Controls state-transitions of a system Assigns resources, such as CPU, disk, network etc. Long-term, middle-term, short-term scheduling From scheduling of complex jobs up to individual threads Process (CPU) scheduling controls ready running which ready process gets the CPU(s) Scheduling points 1. running waiting (e.g. I/O is started) 2. running ready (e.g. Interrupt happened) 3. waiting ready (e.g. end of I/O) 4. termination Non-preemptive Scheduling Scheduling only in cases 1 and 4 Laszlo Böszörmenyi Computer Networks Processes - 36
Scheduling (2) Preemptive Scheduling Scheduling in all four scheduling points Threads can be forced to release the CPU (preemption) Time-slicing Every thread gets a certain time quantum If time quantum over: timer interrupt is generated Thread is forced to running ready Fairness Each thread gets its CPU share No thread misses its signal eternally No thread waits in the ready queue eternally Laszlo Böszörmenyi Computer Networks Processes - 37
Scheduling in Java Priority scheduling A priority level is assigned to every thread Threads at the highest priority level run first On a certain priority level round-robin (one after the other) Priority can be set dynamically and explicitly Optional preemption and time-slicing Time-slicing is allowed but not mandatory depends on the current JVM (Java Virtual Machine) implementation The following scheduling points are defined anyway wait, sleep, wait on a lock, wait on I/O, yield Laszlo Böszörmenyi Computer Networks Processes - 38
Problems of priority scheduling Starvation danger Threads with lower priority never get the CPU Priority inversion If a low priority thread keeps a lock High-priority threads must wait for the same lock A low-priority thread slows down high-priority threads Solution for priority inversion: Thread T 1 with priority p 1 keeps lock L Thread T 2 with priority p 2 > p 1 needs Lock L T 1 inherits priority p 2, until the lock is released After that T 1 regains the (lower) priority p 1 again Priority inheritance is implemented by many JVMs Laszlo Böszörmenyi Computer Networks Processes - 39
Implementation of threads Green-thread (user-level) model Scheduling by the JVM Fast (no system calls are needed), easier to port I/O Operations must be non-blocking Native-thread (system-level) model Java-threads are mapped on operating system threads Scheduling is done by the operating system Threads become heavier : Thread-pooling Unused threads are stored in a pool: can be reused Usual implementation on Windows Unix systems (Solaris, Linux) provide generally both models Laszlo Böszörmenyi Computer Networks Processes - 40