Concurrent Programming. Copyright 2017 by Robert M. Dondero, Ph.D. Princeton University

Objectives You will learn/review: What a process is How to fork and wait for processes What a thread is How to spawn and join threads How to handle threads that access shared objects Race conditions and prevention of them Deadlocks and prevention of them How processes can communicate How threads can communicate 2

Agenda 1. Process-level concurrency (C) 2. Thread-level concurrency (Java/Python) 3. Race conditions (Java/Python) 4. Deadlock (Java/Python) 5. Inter-process comm (C) 6. Inter-thread comm (Java/Python) 3

Processes Program Executable code Process An instance of a program in execution A process has its own distinct context 4

Process Context Context consists of: Process id Address space: TEXT, RODATA, DATA, BSS, HEAP, STACK Processor state: RIP, EFLAGS, RAX, RBX, etc. registers 5

Concurrency Options To implement concurrency... Option 1: Multiple processes Process P1 forks process P2 P1 and P2 execute concurrently P1 and P2 do not share objects P1 and P2 have (initially identical but) distinct memory spaces (Relatively) expensive context switching 6

Concurrent Processes Concurrent Processes Running Same Program PROCESS 1 PROCESS 2 HEAP STACK RODATA STACK HEAP DATA TEXT DATA BSS BSS IP REG IP REG 7

Process-Level Concurrency At system boot-up... Several processes are created automatically They run concurrently When you login... ssh process forks a child process Child process execs (overwrites itself with) your login shell (e.g. bash) ssh process and your login shell process run concurrently 8

Process-Level Concurrency When you execute your program as a command... Bash forks a child process Child process execs (overwrites itself with) your program Bash process and your process run concurrently Your process can fork/exec additional processes as desired Your process and child processes run concurrently 9

Process-Level Concurrency COS 333 example CGI programming When HTTP server receives HTTP request: HTTP server forks a child process Child process execs (overwrites itself with) the specified CGI program Parent HTTP server process and CGI process run concurrently 10

Forking Processes How to fork child processes? See forking.c Parent process forks blue child process and red child process Three processes execute concurrently Parent process may exit before child processes Malformed!!! A parent should wait for its children to exit A parent should reap its children 11

Waiting for Processes See waiting.c Parent waits for (reaps) its children Parent process is blocked until both children exit Proper pattern 12

Agenda 1. Process-level concurrency (C) 2. Thread-level concurrency (Java/ Python) 3. Race conditions (Java/Python) 4. Deadlock (Java/Python) 5. Inter-process comm (C) 6. Inter-thread comm (Java/Python) 13

Threads Thread A flow of control within a process A process contains one or more threads Within a process, all threads execute concurrently 14

Concurrency Options To implement concurrency Option 2: Multiple threads Within P1, thread T1 spawns thread T2 T1 and T2 execute concurrently T1 and T2 may share some of P1 s objects (Relatively) inexpensive context switching 15

Concurrent Threads Concurrent Threads within Same Process THREAD 1 THREAD 2 STACK RODATA DATA STACK IP REG HEAP BSS IP REG TEXT 16

Thread-Level Concurrency COS 333 examples Java virtual machine process JVM process must execute given program JVM process also must collect garbage Main thread and garbage collector thread run concurrently 17

Thread-Level Concurrency COS 333 examples Assignment 2 server process Server process must communicate with client process for (relatively) long time periods Server process also must respond to other client processes Server process spawns distinct thread for each client request Main thread and client handler thread run concurrently 18

Thread-Level Concurrency COS 333 examples Assignment 4 browser process Browser process must fetch data Browser process also must respond to user input Via AJAX, Browser main (UI) thread spawns a child thread to fetch data Browser main (UI) thread and fetch data thread run concurrently 19

Spawning Threads The main thread runs at process startup Other threads may run at process startup too The main thread can spawn other threads Note terminology: One process forks another One thread spawns another 20

Spawning Threads in Java See Spawning.java To spawn a thread: Define a subclass of Thread Override run() method Create an object of that class To begin execution of a thread: Call object s start() method start() calls run() Don t call run() directly 21

Spawning Threads in Java See RunnableInterface.java Alternative way to spawn a thread: Define class that implements Runnable Must define a run() method Create a Thread object specifying Runnable object as argument 22

Spawning Threads in Java To begin execution: Call Runnable object s start() method start() calls run() Don t call run() directly Useful when class extends some class other than Thread 23

Spawning Threads in Python See spawning.py (Much the same as Java) No need for Runnable interface approach Python supports multiple inheritance 24

Joining Threads Main thread can join a child thread Main thread can block until child thread terminates Terminology A parent process can wait for a child process A parent thread can join a child thread 25

Joining Threads in Java See Joining.java thread.join() Blocks current thread until thread terminates May throw InterruptedException (Explained in Appendix) 26

Joining Threads in Python See joining.py (Much the same) 27

Race Conditions Problem: Threads can share objects Danger if multiple threads update same object Race condition Outcome depends upon thread scheduling See RaceCondition.java See racecondition.py 29

Locking Solution: Locking Current thread gets lock on shared object Thereby current thread has exclusive use of shared object Other threads cannot obtain lock on shared object until current thread releases lock Adds lots of overhead 30

User-Level Locking Approach 1: Locking in user of shared object 31

User-Level Locking in Java See LockInUser.java synchronized(sharedobj) { stmt; stmt; } Current thread: Give me the lock on sharedobj Other threads cannot obtain the lock on sharedobj during execution of block 32

User-Level Locking in Python See lockinuser.py self._lock = Lock() In BankAcct constructor Creates a lock for the newly instantiated object self._bankacct.getlock().acquire() In each thread Give me the lock on the bankacct object self._bankacct.getlock().release() In each thread Release the lock on the bankacct object 33

Resource-Level Locking Approach 2: Locking in shared resource/ object itself 34

Java Resource-Level Locking See LockInResource.java public synchronized void method() { } Is the same as: public void method() { synchronized(this) { } } Each BankAcct object thus is thread-safe 35

Python Resource-Level Locking See lockinresource.py Same, except... Locking is performed by BankAcct object rather than by thread objects Each BankAcct object thus is thread-safe 36

Locking Strategies Which locking approach is better? Resource-level locking Generally safer; shared object is thread-safe User-level locking Generally faster; avoids locking when unnecessary Maybe easier to understand? 37

Aside: Thread-Safe Collections Thread-safe collections in Java: Vector myvector = new Vector(); Hashmap mymap = new Hashmap(); List<E> mylist = new Collections.synchronizedList(new ArrayList<E>()); Map<K,V> mymap = new Collections.synchronizedMap(new HashMap<K,V>()); ConcurrentLinkedQueue<E> myqueue = new ConcurrentLinkedQueue<E>(); ConcurrentSkipListSet<E> myset = new ConcurrentSkipListSet<E>(); ConcurrentHashMap<K,V> mymap = new ConcurrentHashMap<K,V>(); ConcurrentSkipListMap<K,V> mymap = new ConcurrentSkipListMap<K,V>(); 38

Aside: Thread-Safe Collections Thread-safe collections in Python: from Queue import Queue myqueue = Queue() 39

Conditions Problem: Thread may need to wait for some condition on a locked object to become true Example: Withdraw thread must wait for bank account balance to be sufficiently large Solution: Conditions 40

Conditions Called from within locked code object.wait() Blocks current thread until it is notified Releases the lock object.notifyall() Notifies all waiting threads That some significant event has occurred Thread then should re-check condition 41

Conditions Pattern Thread conditions pattern: consumer() while (! condition) wait(); // Do what should be done when // condition is true. producer() // Change condition. notifyall(); 42

Java Conditions See Conditions.java Consumer method calls object.wait() repeatedly until condition is true Producer method calls object.notifyall() Could handle conditions in users rather than in shared resource 43

Python Conditions See conditions.py Condition wraps around lock Could handle conditions in users rather than in shared resource 44

Deadlock Problem: deadlock Two threads: t1 and t2 Two shared objects: o1 and o2 t1 has lock on o1; needs lock on o2 t2 has lock on o2; needs lock on o1 Both threads block forever 46

Java Deadlock Example See Deadlock.java Deadlock (1) run() (2) run() transferthread1 transferthread2 (3) aliceacct.transferoneto(bobacct) (4) bobacct.transferoneto(aliceacct) aliceacct (5) bobacct.depositone() (6) aliceacct.depositone() bobacct 47

Python Deadlock Example See deadlock.py (Same pattern) 48

Preventing Deadlock Conditions for deadlock Mutual exclusion Hold and wait No preemption Circular chain of events Solution OS level: Negate one of the four conditions App level: Negate circular chain of events 49

Preventing Deadlock NoDeadlock Mutex (1) run() (2) run() transferthread1 (3) get lock (6) release lock (7) get lock (10) release lock transferthread2 (4) alictacct.transferoneto(bobacct) (8) bobacct.transferoneto(aliceacct) aliceacct (5) bobacct.withdrawone() (9) aliceacct.withdrawone() bobacct 50

Java Preventing Deadlock See NoDeadlock1.java Mutex object See NoDeadlock2.java Static fields and static methods Mutex object as a static field in BankAcct class See NoDeadlock3.java Really no need for Mutex object! Any object will suffice E.g., the object which represents the BankAcct class! 51

Python Preventing Deadlock See nodeadlock1.py Mutex object See nodeadlock2.py Static fields and static methods Mutex object as a static field in BankAcct class 52

Inter-Process Comm Facts: Processes on different computers can communicate via sockets See Network lecture Processes on the same computer can communicate via pipes A Unix pipe is a queue for inter-process comm 54

Inter-Process Comm See childtochild.c Parent process creates producer and consumer child processes Producer child process writes data to pipe Consumer child process reads data from pipe 55

Inter-Process Comm See sortmore.c Parent process forks two child processes First child executes sort somefile With stdout redirected to pipe Second child executes more With stdin redirected to pipe 56

Inter-Process Comm Generalization Unix users routinely command the shell to fork communicating concurrent processes E.g. sort somefile more sortmore.c is a hardcoded shell 57

Inter-Thread Comm Facts: Threads can communicate via shared objects Communication via shared objects is dangerous (race conditions, deadlock) Often threads are in a producer/consumer relationship Is there a safer way for producer/ consumer threads to communicate? 59

Java Inter-Thread Comm See ProdConStream.java PipedOutputStream and PipedInputStream Allow producer thread to send data to consumer thread See also java.io.pipedwriter and PipedReader for sending Unicode characters 60

Python Inter-Thread Comm See prodconstream.py No stream mechanism Can t implement Unix pipes model Can t implement Java stream model Instead... Queue class Methods are synchronized Producer thread puts to queue object Consumer thread gets from queue object 61

Concurrency Commentary Process-level concurrency is: Essential (Unix relies on it) Safe (distinct processes share no data) Thread-level concurrency is: Essential (garbage collection, network pgmming, web pgmming, ) Dangerous!!! (distinct threads can share objects) 62

Concurrency Commentary Should methods be synchronized by default? When using thread-level concurrency, should we avoid shared objects? Should we use process-level concurrency instead of thread-level concurrency whenever possible? In the long run, is thread-level concurrency a passing phase? 63

Summary We have covered: What a process is How to fork and wait for processes What a thread is How to spawn and join threads How to handle threads that access shared objects Race conditions and prevention of them Deadlocks and prevention of them How processes can communicate How threads can communicate 64

Appendix 1: Interrupting Threads 65

Interrupting Threads Bad idea to end a thread abnormally May leave shared objects in inconsistent states How to force normal thread termination? That is, how to force normal return from run()? 66

Interrupting Threads Forcing normal thread termination Define interrupted flag in thread object Initially set to false Define public thread.interrupt() method Called (by main thread?) to set interrupted flag to true run() checks interrupted flag periodically true => return from run() 67

Interrupting Threads Problem If thread is joining/waiting/sleeping, then Thread might not check flag for a long time Solution Interrupting threads 68

Java Interrupting Threads b = thread.interrupted() Returns the value (True or False) of thread s interrupted flag thread.interrupt() Sets thread s interrupted flag If thread is joining, waiting, or sleeping... join(), wait(), or sleep() throws InterruptedException Thread.interrupted() Checks interrupted flag of current thread 69

Java Interrupting Threads Suppose Main thread spawns thread t1 Main thread later calls t1.interrupt() What happens? if t1 is running, then t1.interrupt() sets t1 s interrupted flag to True t1.run() calls b = t1.interrupted() at leading edge of its loop b is True => t1.run() returns, and thread terminates else // t1 is blocked (for joining, waiting, or sleeping) join(), wait(), or sleep() throws an InterruptedException t1 catches the InterruptedException Catch clause re-calls t1.interrupt() 70

Java Interrupting Threads Pattern in thread object: public void run() { while ( ) { if (interrupted()) return; } } Pattern in thread object: try { Thread.sleep(1000); } catch (InterruptedException e) { interrupt(); } Pattern in non-thread object: try { Thread.sleep(1000); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } 71

Java Interrupting Threads See Interrupting.java 72

Python Interrupting Threads Not supported!!! Threads cannot be controlled from the outside (Java is particularly good for multithreaded programming) 73

Appendix 2: Daemon Threads 74

Daemon Threads User thread (as seen so far) Example: thread that calls main() Process exits when all user threads have terminated Daemon thread Example: garbage collector thread Exists only to service other threads Process can terminate with running daemon threads 75

Java Daemon Threads thread.setdaemon(b) Sets daemon status of thread to b (true or false) b = thread.isdaemon() Returns daemon status of thread See Daemons.java Daemon threads end when main thread ends 76

Python Daemon Threads thread.setdaemon(b) Sets daemon status of thread to b (True or False) b = thread.isdaemon() Returns daemon status of thread See daemons.py Daemon threads end when main thread ends 77

Appendix 3: Threads in C 78

Threads in C C language does not support threads C standard library does not support threads Use non-standard pthreads library Not object-oriented; uses function pointers Specify -pthread option to gcc command 79

Threads in C An error-handling module See mypthread.h See mypthread.c Spawning threads See spawning.c 80

Threads in C Joining threads See joining.c Locking See lockinuser.c See lockinresource.c Conditions See conditions.c 81

Threads in C Deadlock See deadlock.c See nodeadlock.c Interrupting (Not available in C) Inter-thread comm (Omitted) 82