COMP 322: Principles of Parallel Programming Lecture 11: Parallel Programming Issues (Chapter 6) Fall 2009

Transcription

1 COMP 322: Principles of Parallel Programming Lecture 11: Parallel Programming Issues (Chapter 6) Fall 2009 Vivek Sarkar Department of Computer Science Rice University COMP 322 Lecture September 2009

2 Summary of Previous Lecture!! Example of Pipeline Parallelism with HJ Phasers! Overview of POSIX Threads (Chapter 6)! Question for you to think about:!which POSIX Threads examples in the lecture or book cannot be expressed using HJ constructs? 2

3 Acknowledgments for Today"s Lecture!! Course text: Principles of Parallel Programming, Calvin Lin & Lawrence Snyder!Includes resources available at 0,3110, ,00.html! Phasers: a Unified Deadlock-Free Construct for Collective and Point-topoint Synchronization, Jun Shirako, David M. Peixotto, Vivek Sarkar, William N. Scherer III! Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit! 3

4 Pthreads Issues: Fairness!! Consider three POSIX threads (0, 1, 2) that repeatedly execute the following loop: while (true) { pthread_mutex_lock(&lock); printf( Hello from thread %d\n, pthread_self()); pthread_mutex_unlock(&lock); }! What is the relative number of print statements that you might expect from threads 0, 1, 2?! What is the minimum number of print statements that might be guaranteed from threads 0, 1, 2? 4! No guarantee of fairness in the POSIX threads standard! Order in which locks are acquired is not guaranteed to match the order in which the threads attempt to acquire the locks

5 Pthreads Issues: Serializability!! A concurrent execution is serializable if the execution is guaranteed to correspond to some serial execution of those threads. Consider /* threads compete to update global variable best_cost */ if (my_cost < best_cost)!!two threads best_cost = my_cost;!!initial value of best_cost is 100!values of my_cost are 50 and 75 for threads t1 and t2! After execution, best_cost could be 50 or 75! 75 does not correspond to any serialization of the threads! Use mutex to make the parallel program serializable 5

6 Pthreads Issues: Deadlock! Figure 6.7 Deadlock example. Threads T1 and T2 hold locks L1 and L2, respectively, and each thread attempts to acquire the other lock, which cannot be granted." 6

7 Lock Hierarchies!! A simple way to prevent deadlocks is to prevent cycles in the resource allocation graph.! Impose an order on the locks, and require that all threads acquire their locks in the same order!!lock hierarchy!! What if a thread doesn t know a priori which locks it needs to acquire? 1.# On learning of a new lock, it can release all its existing locks and then reacquire all locks in the proper order 2.# Use pthread_mutex_trylock() on the new lock. If that fails then revert to option 1. above. 7

8 Counter Implementation! public class Counter { private long value; } public long getandincrement() { return value++; } 8

9 Counter Implementation! public class Counter { private long value; } public long getandincrement() { return value++; } OK for single thread, not for concurrent threads 9

10 What It Means! public class Counter { private long value; } public long getandincrement() { return value++; } 10

11 What It Means! public class Counter { private long value; } public long getandincrement() { return value++; temp = value; } value = temp + 1; return temp; 11

12 Not so good! Value read 1 write 2 read 2 write 3 read 1 write 2 time 12

13 Is this problem inherent?!!!!! read write write read If we could only glue reads and writes together 13

14 Challenge! public class Counter { private long value; } public long getandincrement() { temp = value; value = temp + 1; return temp; } 14

15 Challenge! public class Counter { private long value; } public long getandincrement() { temp = value; value = temp + 1; return temp; } Make these steps atomic (isolated) 15

16 Hardware Solution! public class Counter { private long value; 16 public long getandincrement() { temp = value; value = temp + 1; return temp; } } ReadModifyWrite() instruction

17 HJ Solution! public class Counter { private long value; } public long getandincrement() { isolated { temp = value; value = temp + 1; } return temp; } 17

18 Mutual Exclusion or Alice & Bob share a pond! A B 18

19 Alice has a pet! A B 19

20 Bob has a pet! A B 20

21 The Problem! A B The pets don t get along 21

22 Formalizing the Problem!! Two types of formal properties in asynchronous computation:! Safety Properties!Nothing bad happens ever! Liveness Properties!Something good happens eventually 22

23 Formalizing our Problem!! Mutual Exclusion!Both pets never in pond simultaneously!this is a safety property! No Deadlock!if only one wants in, it gets in!if both want in, one gets in.!this is a liveness property 23

24 ! Idea!Just look at the pond! Gotcha!Not atomic!trees obscure the view Simple Protocol! 24

25 Interpretation!! Threads can t see what other threads are doing! Explicit communication required for coordination 25

26 ! Idea!Bob calls Alice (or vice-versa)! Gotcha!Bob takes shower!alice recharges battery Cell Phone Protocol!!Bob out shopping for pet food 26

27 ! Message-passing doesn t work! Recipient might not be!listening!there at all! Communication must be!persistent (like writing)!not transient (like speaking) Interpretation! 27

28 Can Protocol! cola cola 28

29 Bob conveys a bit! A B cola 29

30 Bob conveys a bit! A B cola 30

31 ! Idea!Cans on Alice s windowsill Can Protocol!!Strings lead to Bob s house!bob pulls strings, knocks over cans! Gotcha!Cans cannot be reused!bob runs out of cans 31

32 Interpretation!! Cannot solve mutual exclusion with interrupts!sender sets fixed bit in receiver s space!receiver resets bit when ready!requires unbounded number of interrupt bits 32

33 Flag Protocol! A B 33

34 Alice"s Protocol (sort of)! A B 34

35 Bob"s Protocol (sort of)! A B 35

36 Alice"s Protocol!! Raise flag! Wait until Bob s flag is down! Unleash pet! Lower flag when pet returns 36 Art of Multiprocessor Programming 36

37 Bob"s Protocol!! Raise flag! Wait until Alice s flag is down! Unleash pet! Lower flag when pet returns 37

38 Bob"s Protocol (2 nd try)!! Raise flag! While Alice s flag is up!lower flag!wait for Alice s flag to go down!raise flag! Unleash pet! Lower flag when pet returns 38

39 Remarks!! Protocol is unfair!bob s pet might never get in! Protocol uses waiting!if Bob is eaten by his pet, Alice s pet might never get in 39

40 Moral of Story!! Mutual Exclusion cannot be solved by!transient communication (cell phones)!interrupts (cans)! It can be solved by! one-bit shared variables! that can be read or written 40

41 The Fable Continues!! Alice and Bob fall in love & marry 41

42 The Fable Continues!! Alice and Bob fall in love & marry! Then they fall out of love & divorce!she gets the pets!he has to feed them 42

43 The Fable Continues!! Alice and Bob fall in love & marry! Then they fall out of love & divorce!she gets the pets!he has to feed them! Leading to a new coordination problem: Producer-Consumer 43

44 Bob Puts Food in the Pond! A 44

45 Alice releases her pets to Feed! mmm mmm B 45

46 Producer/Consumer!! Alice and Bob can t meet!each has restraining order on other!so he puts food in the pond!and later, she releases the pets! Avoid!Releasing pets when there s no food!putting out food if uneaten food remains 46

47 Producer/Consumer!! Need a mechanism so that!bob lets Alice know when food has been put out!alice lets Bob know when to put out more food 47

48 Surprise Solution! A B cola 48

49 Bob puts food in Pond! A B cola 49

50 Bob knocks over Can! A B cola 50

51 Alice Releases Pets! A yum yum B cola 51

52 Alice Resets Can when Pets are Fed! A B cola 52

53 Pseudocode! while (true) { while (can.isup()){}; pet.release(); pet.recapture(); can.reset(); } Alice s code 53 Art of Multiprocessor Programming 53

54 Pseudocode! while (true) { while (can.isup()){}; Bob s code pet.release(); pet.recapture(); can.reset(); while (true) { } while (can.isdown()){}; pond.stockwithfood(); can.knockover(); } Alice s code 54

55 Correctness!! Mutual Exclusion!Pets and Bob never together in pond 55

56 Correctness!! Mutual Exclusion!Pets and Bob never together in pond! No Starvation if Bob always willing to feed, and pets always famished, then pets eat infinitely often. 56

57 Correctness!! Mutual Exclusion!Pets and Bob never together in pond safety liveness! No Starvation if Bob always willing to feed, and pets always famished, then pets eat infinitely often. safety! Producer/Consumer The pets never enter pond unless there is food, and Bob never provides food if there is unconsumed food. 57

58 Could Also Solve Using Flags! A B 58

59 Figure 6.1! A bounded buffer with producers and consumers. The Put and Get cursors indicate where the producers will insert the next item and where the consumers will remove its next item. 59

60 Figure 6.3 Bounded buffer example using condition variables nonempty and nonfull.! 60

61 Bound option in phasers!! Constructor!phaser(mode m, int bound_size);! next operation!a task registered in SIG mode will block if it is >= bound_size phases past the current phase 61

62 finish {! }! Single-Producer Single-Consumer Bounded Buffer! final phaser ph = new phaser(<sig_wait>, bound_size);! async phased (ph<sig>)! while ( ) { insert(); next; } // producer! async phased (ph<wait>)! while ( ) { next; remove(); } // consumer! 62

63 Summary of Today"s Lecture!! POSIX Threads as an illustration of parallel programming issues:!fairness!serializability!deadlock!safety!liveness!bounded buffer example! Many of these issues arise in other parallel programming languages, but constructs in higher level languages such as HJ can help (async, finish, isolated, phasers) 63