Overview of Threads and Concurrency

Transcription

1 CS533 Cncepts f Operating Systems Class 2 Overview f Threads and Cncurrency

2 Questins Why study threads and cncurrent prgramming in an OS class? What is a thread? Is multi-threaded prgramming easy? If nt, why nt? CS533 - Cncepts f Operating Systems 2

3 Threads Prcesses have the fllwing cmpnents: a CPU cntext r thread f cntrl an addressing cntext (address space) a cllectin f perating system state On multiprcessr systems, with several CPUs, it wuld make sense fr a prcess t have several CPU cntexts (threads f cntrl) Multiple threads f cntrl culd run in the same address space n a single CPU system t! thread f cntrl and address space are rthgnal cncepts CS533 - Cncepts f Operating Systems 3

4 Threads Threads share an address space with zer r mre ther threads culd be the kernel s address space r that f a user level prcess Threads have their wn PC, SP, register state etc (CPU state) Stack (memry) Why d these need t be private t each thread? what ther OS state shuld be private t threads? A traditinal prcess can be viewed as an address space with a single thread CS533 - Cncepts f Operating Systems 4

5 Single thread state within a prcess CS533 - Cncepts f Operating Systems 5

6 Multiple threads in an address space CS533 - Cncepts f Operating Systems 6

7 Shared state amng related threads Open files User ID, grup ID, prcess/task ID Address space Text Data (static glbal variables) Heap (dynamic data) Changes made t shared state by ne thread will be visible t the thers! Reading & writing shared memry requires synchrnizatin! CS533 - Cncepts f Operating Systems 7

8 Why prgram using threads? Utilize multiple CPU s cncurrently Lw cst cmmunicatin via shared memry Overlap cmputatin and blcking n a single CPU Blcking due t I/O Cmputatin and cmmunicatin Handle asynchrnus events CS533 - Cncepts f Operating Systems 8

9 Why use threads? - example A WWW prcess GET / HTTP/1.0 HTTPD disk CS533 - Cncepts f Operating Systems 9

10 Why use threads? - example A WWW prcess GET / HTTP/1.0 HTTPD disk Why is this nt a gd web server design? CS533 - Cncepts f Operating Systems 10

11 Why use threads? - example A WWW prcess GET / HTTP/1.0 HTTPD HTTPD disk CS533 - Cncepts f Operating Systems 11

12 Why use threads? - example A WWW prcess GET / HTTP/1.0 HTTPD GET / HTTP/1.0 disk CS533 - Cncepts f Operating Systems 12

13 Why use threads? - example A WWW prcess GET / HTTP/1.0 HTTPD GET / HTTP/1.0 GET / HTTP/1.0 GET / HTTP/1.0 disk CS533 - Cncepts f Operating Systems 13

14 What des a typical thread API lk like? POSIX standard threads (Pthreads) First thread exists in main(), typically creates the thers pthread_create (thread,attr,start_rutine,arg) Returns new thread ID in thread Executes rutine specified by start_rutine with argument specified by arg Exits n return frm rutine r when tld explicitly CS533 - Cncepts f Operating Systems 14

15 Thread API (cntinued) pthread_exit (status) Terminates the thread and returns status t any jining thread pthread_jin (threadid,status) Blcks the calling thread until thread specified by threadid terminates Return status frm pthread_exit is passed in status One way f synchrnizing between threads pthread_yield () Thread gives up the CPU and enters the run queue CS533 - Cncepts f Operating Systems 15

16 Using create, jin and exit primitives CS533 - Cncepts f Operating Systems 16

17 An example Pthreads prgram #include <pthread.h> #include <stdi.h> #define NUM_THREADS 5 vid *PrintHell(vid *threadid) { printf("\n%d: Hell Wrld!\n", threadid); pthread_exit(null); } int main (int argc, char *argv[]) { pthread_t threads[num_threads]; int rc, t; fr(t=0; t<num_threads; t++ { printf("creating thread %d\n", t); rc = pthread_create(&threads[t], NULL, PrintHell, (vid *)t); if (rc) { printf("error; return cde frm pthread_create() is %d\n", rc); exit(-1); } } pthread_exit(null); } Prgram Output Creating thread 0 Creating thread 1 0: Hell Wrld! 1: Hell Wrld! Creating thread 2 Creating thread 3 2: Hell Wrld! 3: Hell Wrld! Creating thread 4 4: Hell Wrld! Fr mre examples see: CS533 - Cncepts f Operating Systems 17

18 Prs & cns f threads Prs Cns Overlap I/O with cmputatin! Cheaper cntext switches Better mapping t shared memry multiprcessrs Ptential thread interactins due t cncurrent access t memry Cmplexity f debugging Cmplexity f multi-threaded prgramming Backwards cmpatibility with existing cde CS533 - Cncepts f Operating Systems 18

19 Cncurrent prgramming Assumptins: Example: Prblem: Tw r mre threads Each executes in (pseud) parallel and can t predict exact running speeds The threads can interact via access t shared variables One thread writes a variable The ther thread reads frm the same variable The utcme depends n the rder f these READs and WRITES! CS533 - Cncepts f Operating Systems 19

20 Sequential Prgramming Example int i = 0 i = i + 1 print i What utput d yu expect? Why? CS533 - Cncepts f Operating Systems 20

21 Cncurrent prgramming example int i = 0 Thread 1: i = i + 1 print i What utput d yu expect with 1 thread? Why? CS533 - Cncepts f Operating Systems 21

22 Cncurrent prgramming example int i = 0 Thread 1: i = i + 1 Thread 2: i = i + 1 print i print i What utput d yu expect with 2 threads? Why? CS533 - Cncepts f Operating Systems 22

23 Race cnditins What is a race cnditin? CS533 - Cncepts f Operating Systems 23

24 Race cnditins Hw is i = i + 1 implemented? lad i t register increment register stre register value t i Registers are part f each threads private CPU cntext CS533 - Cncepts f Operating Systems 24

25 Race cnditins Thread 1 Thread 2 lad i regn inc regn stre regn i lad i regn inc regn stre regn i CS533 - Cncepts f Operating Systems 25

26 Race cnditins What is a race cnditin? tw r mre threads have an incnsistent view f a shared memry regin (I.e., a variable) Why d race cnditins ccur? CS533 - Cncepts f Operating Systems 26

27 Race cnditins What is a race cnditin? tw r mre threads have an incnsistent view f a shared memry regin (I.e., a variable) Why d race cnditins ccur? values f memry lcatins are replicated in registers during executin cntext switches ccur at arbitrary times during executin (r prgram runs n a multiprcessr) threads can see stale memry values in registers CS533 - Cncepts f Operating Systems 27

28 Race Cnditins Race cnditin: whenever the utput depends n the precise executin rder f the threads! What slutins can we apply? CS533 - Cncepts f Operating Systems 28

29 Race Cnditins Race cnditin: whenever the utput depends n the precise executin rder f the threads! What slutins can we apply? What is the danger in the previus example? Hw can we make it safe? What price d we pay? CS533 - Cncepts f Operating Systems 29

30 Synchrnizatin by mutual exclusin Divide thread cde int critical sectins Sectins where shared data is accessed (read/written) Only allw ne thread at a time in a critical sectin CS533 - Cncepts f Operating Systems 30

31 Critical sectins with mutual exclusin CS533 - Cncepts f Operating Systems 31

32 Mutual exclusin lcks (mutex) Each shared data has a unique lck assciated with it Threads acquire the lck befre accessing the data Threads release the lck after they are finished with the data The lck can nly be held by ne thread at a time CS533 - Cncepts f Operating Systems 32

33 Lcks - implementatin Hw can we implement a lck? Hw d we test t see if its held? Hw d we acquire it? Hw d we release it? Hw d we blck/wait if it is already held when we test? CS533 - Cncepts f Operating Systems 33

34 Des this apprach wrk? bl lck = false while lck = true; /* wait */ lck = true; /* lck */ critical sectin lck = false; /* unlck */ CS533 - Cncepts f Operating Systems 34

35 Implementing lcks A binary lck variable in memry des nt wrk! Lck and unlck peratins must be atmic! Many cmputers have sme limited hardware supprt fr atmically testing and setting lcks Atmic Test and Set Lck instructin (TSL) Atmic cmpare and swap instructin (CAS) These atmic instructins can be used t implement mutual exclusin (mutex) lcks CS533 - Cncepts f Operating Systems 35

36 Test-and-set-lck instructin (TSL, tset) A lck is a single wrd variable with tw values 0 = FALSE = nt lcked 1 = TRUE = lcked The test-and-set instructin des the fllwing atmically: Get the (ld) value f lck Set the new value f lck t TRUE Return the ld value If the returned value was FALSE... Then yu gt the lck!!! If the returned value was TRUE... Then smene else has the lck (s try again later) CS533 - Cncepts f Operating Systems 36

37 Spin Lcks while TSL (lck); /* while return value is true, spin */ lck = false critical sectin CS533 - Cncepts f Operating Systems 37

38 Spin lcks What price d we pay fr mutual exclusin? Hw well will this wrk n a uniprcessr? CS533 - Cncepts f Operating Systems 38

39 Blcking lcks Hw can we avid wasting CPU cycles? Hw can we implement sleep and wakeup? cntext switch when acquire finds the lck held check and ptential wakeup n lck release system calls t acquire and release lck But hw can we make these system calls atmic? CS533 - Cncepts f Operating Systems 39

40 Blcking lcks Is this better than a spinlck n a uniprcessr? Is this better than a spinlck n a multiprcessr? When wuld yu use a spinlck vs a blcking lck n a multiprcessr? CS533 - Cncepts f Operating Systems 40

41 Enfrcing mutual exclusin Assumptins: Every thread sets the lck befre accessing shared data! Every thread releases the lck after it is dne! Only wrks if yu fllw these prgramming cnventins all the time! Thread 1 Thread 2 Thread 3 Lck Lck A = 2 A = A+1 A = A*B Unlck Unlck CS533 - Cncepts f Operating Systems 41

42 Using Pthread mutex variables Pthread_mutex_lck (mutex) Acquire the lck r blck until it is acquired Pthread_mutex_trylck (mutex) Acquire the lck r return with busy errr cde Pthread_mutex_unlck (mutex) Free the lck CS533 - Cncepts f Operating Systems 42

43 Invariant f a mutex The mutex invariant is the cnditin that must be restred befre: The mutex is released Example Invariant A=B always hlds utside the critical sectin Critical sectin updates A and B CS533 - Cncepts f Operating Systems 43

44 What des thread-safe mean? CS533 - Cncepts f Operating Systems 44

45 What des thread-safe mean? A piece f cde (library) is thread-safe if it defines critical sectins and uses synchrnizatin t cntrl access t them All entry pints must be re-entrant Results nt returned in shared glbal variables nr glbal statically allcated strage All calls shuld be synchrnus CS533 - Cncepts f Operating Systems 45

46 Reentrant cde A functin/methd is said t be reentrant if... A functin that has been invked may be invked again befre the first invcatin has returned, and will still wrk crrectly In the cntext f cncurrent prgramming... A reentrant functin can be executed simultaneusly by mre than ne thread, with n ill effects CS533 - Cncepts f Operating Systems 46

47 Reentrant Cde Cnsider this functin... var cunt: int = 0 functin GetUnique () returns int cunt = cunt + 1 return cunt endfunctin What happens if it is executed by different threads cncurrently? CS533 - Cncepts f Operating Systems 47

48 Reentrant Cde Cnsider this functin... var cunt: int = 0 functin GetUnique () returns int cunt = cunt + 1 return cunt endfunctin What happens if it is executed by different threads cncurrently? The results may be incrrect! This rutine is nt reentrant! CS533 - Cncepts f Operating Systems 48

49 When is cde reentrant? Sme variables are lcal -- t the functin/methd/rutine glbal -- smetimes called static Access t lcal variables? A new stack frame is created fr each invcatin Each thread has its wn stack What abut access t glbal variables? Must use synchrnizatin! CS533 - Cncepts f Operating Systems 49

50 Making this functin reentrant var cunt: int = 0 mylck: Mutex functin GetUnique () returns int var i: int mylck.lck() cunt = cunt + 1 i = cunt mylck.unlck() return i endfunctin CS533 - Cncepts f Operating Systems 50

51 Questin What is the difference between mutual exclusin and cnditin synchrnizatin? CS533 - Cncepts f Operating Systems 51

52 Questin What is the difference between mutual exclusin and cnditin synchrnizatin? Mutual exclusin nly ne at a time in a critical sectin Cnditin synchrnizatin wait until sme cnditin hlds befre prceeding signal when cnditin hlds s thers may prceed CS533 - Cncepts f Operating Systems 52

53 Cnditin variables Mutex lcks allw threads t synchrnize befre accessing the data Cnditin variables allw synchrnizatin based n the value f the data Used in cnjunctin with a mutex lck Allws a thread in a critical sectin t wait fr a cnditin t becme true r signal that a cnditin is true Acquire mutex lck (enter critical sectin) Blck until cnditin becmes true (frees mutex lck) Free mutex lck (leave critical sectin) CS533 - Cncepts f Operating Systems 53

54 Pthread cnditin variables pthread_cnd_wait (cnditin,mutex) Releases mutex and blcks until cnditin is signaled pthread_cnd_signal (cnditin) Signals cnditin which wakes up a thread blcked n cnditin pthread_cnd_bradcast (cnditin) Signals cnditin and wakes up all threads blcked n cnditin CS533 - Cncepts f Operating Systems 54

55 Semantics f cnditin variables Hw many blcked threads shuld be wken n a signal? Which blcked thread shuld be wken n a signal? In what rder shuld newly awken threads acquire the mutex? Shuld the signaler immediately free the mutex? If s, what if it has mre wrk t d? If nt, hw can the signaled prcess cntinue? What if signal is called befre the first wait? CS533 - Cncepts f Operating Systems 55

56 Subtle race cnditins Why des wait n a cnditin variable need t atmically unlck the mutex and blck the thread? Why des the thread need t re-lck the mutex when it wakes up frm wait? Can it assume that the cnditin it waited n nw hlds? CS533 - Cncepts f Operating Systems 56

57 Deadlck Thread A lcks mutex 1 Thread B lcks mutex 2 Thread A blcks trying t lck mutex 2 Thread B blcks trying t lck mutex 1 Can als ccur with cnditin variables Nested mnitr prblem (p. 20) CS533 - Cncepts f Operating Systems 57

58 Deadlck (nested mnitr prblem) Prcedure Get(); BEGIN LOCK a DO LOCK b DO WHILE NOT ready DO wait(b,c) END; END; END; END Get; Prcedure Give(); BEGIN LOCK a DO LOCK b DO ready := TRUE; signal(c); END; END; END Give; CS533 - Cncepts f Operating Systems 58

59 Deadlck in layered systems High layer: Lck M; Call lwer layer; Release M; Lw layer: Lck M; D wrk; Release M; return; Result thread deadlcks with itself! Layer bundaries are suppsed t be paque CS533 - Cncepts f Operating Systems 59

60 Deadlck Why is it better t have a deadlck than a race? CS533 - Cncepts f Operating Systems 60

61 Deadlck Why is it better t have a deadlck than a race? Deadlck can be prevented by impsing a glbal rder n resurces managed by mutexes and cnditin variables i.e., all threads acquire mutexes in the same rder Mutex rdering can be based n layering Allwing upcalls breaks this defense CS533 - Cncepts f Operating Systems 61

62 Pririty inversin Occurs in pririty scheduling Starvatin f high pririty threads Lw pririty thread C lcks M Medium pririty thread B pre-empts C High pririty thread A preempts B then blcks n M B resumes and enters lng cmputatin Result: C never runs s can t unlck M, therefre A never runs Slutin? pririty inheritance CS533 - Cncepts f Operating Systems 62

63 Dangers f blcking in a critical sectin Blcking while hlding M prevents prgress f ther threads that need M Blcking n anther mutex may lead t deadlck Why nt release the mutex befre blcking? Must restre the mutex invariant Must reacquire the mutex n return! Things may have changed while yu were gne CS533 - Cncepts f Operating Systems 63

64 Reader/writer lcking Writers exclude readers and writers Readers exclude writers but nt readers Example, page 15 Gd use f bradcast in ReleaseExclusive() Results in spurius wake-ups and spurius lck cnflicts Hw culd yu use signal instead? Mve signal/bradcast call after release f mutex? Advantages? Disadvantages? Can we avid writer starvatin? CS533 - Cncepts f Operating Systems 64

65 Useful prgramming cnventins All access t shared data must be ptected by a mutex All shared variables have a lck The lck is held by the thread that accesses the variable Hw can this be checked? Statically? Dynamically? CS533 - Cncepts f Operating Systems 65

66 Autmated checking f cnventins Eraser A dynamic checker that uses binary re-writing techniques Gathers an executin histry f reads, writes and lck acquisitins Evaluates cnsistency with rules Is it enugh t simply check that sme lck is held whenever a glbal variable is accessed? CS533 - Cncepts f Operating Systems 66

67 Autmated checking f cnventins Eraser desn t knw ahead f time which lcks prtect which variables It infers which lcks prtect which variables using a lck-set algrithm Assume all lcks are candidates fr a variable ( C(v) is full) Fr each access take intersectin f C(v) and lcks held by thread and make these the candidate set C(v) If C(v) becmes empty, issue warning CS533 - Cncepts f Operating Systems 67

68 Imprving the lcking discipline The standard apprach prduces many false psitives that arise due t special cases: Initializatin N need t lck if n thread has a reference yet Read sharing N need t lck if all threads are readers Reader/writer lcking Distinguish cncurrent readers frm cncurrent readers and writers CS533 - Cncepts f Operating Systems 68

69 Imprved algrithm virgin rd, wr First thread exclusive wr, new thread rd shared Mdified (race?) wr rd, new thread shared wr CS533 - Cncepts f Operating Systems 69

70 Questins Why are threads lightweight? Why assciate thread lifetime with a prcedure? Why blck instead f spin waiting fr a mutex? If a mutex is a resurce scheduling mechanism What is the resurce being scheduled? What is the scheduling plicy and where is it defined? Why d alerts result in cmplicated prgrams? What is carse-grain lcking? What effect des it have n prgram cmplexity? What effect des it have n perfrmance? CS533 - Cncepts f Operating Systems 70

71 Questins What is lck cntentin? Why is it wrse n multiprcessrs than uniprcessrs? What is the slutin? and its cst? What else might cause perfrmance t degrade when yu use multiple threads? CS533 - Cncepts f Operating Systems 71

72 Why is multi-threaded prgramming hard? Many pssible interleavings at the instructin level that make it hard t reasn abut crrectness CS533 - Cncepts f Operating Systems 72