Programming Paradigms for Concurrency Introduction

Transcription

1 Programming Paradigms for Concurrency Introduction Based on companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Modified by Thomas Wies New York University

2 Moore s Law Transistor count still rising Clock speed flattening sharply 2

3 Moore s Law (in practice) 3

4 Nearly Extinct: the Uniprocesor cpu memory 4

5 Endangered: The Shared Memory Multiprocessor (SMP) cache cache Bus cache Bus shared memory 5

6 All on the same chip The New Boss: The Multicore Processor cache (CMP) cache Bus cache shared memory Bus NVidia Tegra 4 with ARM Cortex- A15 6

7 Why is Mooly Eden Smiling? 7

8 Traditional Scaling Process Speedup 1.8x 3.6x 7x User code Traditional Uniprocessor Time: Moore s law 9

9 Ideal Scaling Process Speedup 1.8x 3.6x 7x User code Multicore Unfortunately, not so simple 10

10 Actual Scaling Process Speedup 1.8x 2x 2.9x User code Multicore Parallelization and Synchronization require great care 11

11 Course Overview Fundamentals models, algorithms, impossibility Real-World programming architectures synchronization primitives spin-locks, monitors, barriers paradigms and techniques shared memory concurrency transactional memories message passing 12

12 Languages Used Java shared memory concurrency Scala software transactional memory message passing (actors) 13

13 Administrative Issues Office Hours: Tue 3-4pm, or by appointment Office: CIWW 407 Course web site: Mailing list: 14

14 Grading Weekly Assignments: 30% Term Project: 30% Final Exam: 40% 15

15 Sequential Computation thread memory object object 16

16 Concurrent Computation memory object object 17

17 Asynchrony Sudden unpredictable delays Cache misses (short) Page faults (long) Scheduling quantum used up (really long) 18

18 Model Summary Multiple threads Sometimes called processes Single shared memory Objects live in memory Unpredictable asynchronous delays 19

19 Road Map We are going to focus on principles first, then practice Start with idealized models Look at simplistic problems Emphasize correctness over pragmatism Correctness may be theoretical, but incorrectness has practical impact 20

20 Toyota Unintended Acceleration Incidents ( ) at least one fatal accident more than 9 million vehicles recalled In a 2013 court case, unprotected critical variables in Toyota s real-time OS were linked to the incidents 21

21 Concurrency Jargon Hardware Processors Software Threads, processes Sometimes OK to confuse them, sometimes not. 22

22 Parallel Primality Testing Challenge Print primes from 1 to Given Ten-processor multiprocessor One thread per processor Goal Get ten-fold speedup (or close) 23

23 Load Balancing P 0 P 1 P 9 Split the work evenly Each thread tests range of

24 Procedure for Thread i void primeprint { int i = ThreadID.get(); // IDs in {0..9} for (j = i* , j<(i+1)*10 9 ; j++) { if (isprime(j)) print(j); } } 25

25 Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads Uneven Hard to predict 26

26 Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads Uneven Hard to predict Need dynamic load balancing 27

27 Shared Counter each thread takes a number 17 28

28 Procedure for Thread i int counter = new Counter(1); void primeprint { long j = 0; while (j < ) { j = counter.getandincrement(); if (isprime(j)) print(j); } } 29

29 Procedure for Thread i Counter counter = new Counter(1); void primeprint { long j = 0; while (j < ) { j = counter.getandincrement(); } } if (isprime(j)) print(j); Shared counter object 30

30 Where Things Reside void primeprint { int i = ThreadID.get(); // IDs in {0..9} for (j = i* , j<(i+1)*10 9 ; j++) { if (isprime(j)) print(j); } } code Local variables cache cache Bus cache Bus 1 shared memory shared counter 31

31 Procedure for Thread i Counter counter = new Counter(1); void primeprint { long j = 0; while (j < ) { j = counter.getandincrement(); if (isprime(j)) print(j); } } Stop when every value taken 32

32 Procedure for Thread i Counter counter = new Counter(1); void primeprint { long j = 0; while (j < ) { j = counter.getandincrement(); if (isprime(j)) print(j); } } Increment & return each new value 33

33 Counter Implementation public class Counter { private long value; } public long getandincrement() { return value++; } 34

34 Counter Implementation public class Counter { private long value; } public long getandincrement() { return value++; } 35

35 What It Means public class Counter { private long value; } public long getandincrement() { return value++; } 36

36 What It Means public class Counter { private long value; } public long getandincrement() { return value++; temp = value; } value = temp + 1; return temp; 37

37 Not so good Value read 1 write 2 read 2 write 3 read 1 write 2 time 38

38 Is this problem inherent?!!!! read write write read If we could only glue reads and writes together 39

39 Challenge public class Counter { private long value; } public long getandincrement() { temp = value; value = temp + 1; return temp; } 40

40 Challenge public class Counter { private long value; } public long getandincrement() { temp = value; value = temp + 1; return temp; } Make these steps atomic (indivisible) 41

41 Hardware Solution public class Counter { private long value; public long getandincrement() { temp = value; value = temp + 1; return temp; } } ReadModifyWrite() instruction 42

42 An Aside: Java public class Counter { private long value; } public long getandincrement() { synchronized { temp = value; value = temp + 1; } return temp; } 43

43 An Aside: Java public class Counter { private long value; } public long getandincrement() { synchronized { temp = value; value = temp + 1; } return temp; } Synchronized block 44

44 An Aside: Java public class Counter { private long value; } Mutual Exclusion public long getandincrement() { synchronized { temp = value; value = temp + 1; } return temp; } 45

45 Mutual Exclusion, or Alice & Bob share a pond A B 46

46 Alice has a pet A B 47

47 Bob has a pet A B 48

48 The Problem A B The pets don t get along 49

49 Formalizing the Problem Two types of formal properties in asynchronous computation: Safety Properties Nothing bad happens ever Liveness Properties Something good happens eventually 50

50 Formalizing our Problem Mutual Exclusion Both pets never in pond simultaneously This is a safety property No Deadlock if only one wants in, it gets in if both want in, one gets in. This is a liveness property 51

51 Simple Protocol Idea Just look at the pond Gotcha Not atomic Trees obscure the view 52

52 Interpretation Threads can t see what other threads are doing Explicit communication required for coordination 53

53 Cell Phone Protocol Idea Bob calls Alice (or vice-versa) Gotcha Bob takes shower Alice recharges battery Bob out shopping for pet food 54

54 Interpretation Message-passing doesn t work Recipient might not be Listening There at all Communication must be Persistent (like writing) Not transient (like speaking) 55

55 cola cola Can Protocol 56

56 cola Bob conveys a bit A B 57

57 Bob conveys a bit A B 58

58 Can Protocol Idea Cans on Alice s windowsill Strings lead to Bob s house Bob pulls strings, knocks over cans Gotcha Cans cannot be reused Bob runs out of cans 59

59 Interpretation Cannot solve mutual exclusion with interrupts Sender sets fixed bit in receiver s space Receiver resets bit when ready Requires unbounded number of interrupt bits 60

60 Flag Protocol A B 61

61 Alice s Protocol (sort of) A B 62

62 Bob s Protocol (sort of) A B 63

63 Alice s Protocol Raise flag Wait until Bob s flag is down Unleash pet Lower flag when pet returns 64

64 Bob s Protocol Raise flag Wait until Alice s flag is down Unleash pet Lower flag when pet returns 65

65 Bob s Protocol (2 nd try) Raise flag While Alice s flag is up Lower flag Wait for Alice s flag to go down Raise flag Unleash pet Lower flag when pet returns 66

66 Bob s Protocol Bob defers to Raise flag Alice While Alice s flag is up Lower flag Wait for Alice s flag to go down Raise flag Unleash pet Lower flag when pet returns 67

67 The Flag Principle Raise the flag Look at other s flag Flag Principle: If each raises and looks, then Last to look must see both flags up 68

68 Proof of Mutual Exclusion Assume both pets in pond Derive a contradiction By reasoning backwards Consider the last time Alice and Bob each looked before letting the pets in Without loss of generality assume Alice was the last to look 69

69 Proof Bob last raised flag Alice last raised her flag Alice s last look Bob s last look time Alice must have seen Bob s Flag. A Contradiction 70

70 Proof of No Deadlock If only one pet wants in, it gets in. 71

71 Proof of No Deadlock If only one pet wants in, it gets in. Deadlock requires both continually trying to get in. 72

72 Proof of No Deadlock If only one pet wants in, it gets in. Deadlock requires both continually trying to get in. If Bob sees Alice s flag, he gives her priority (a gentleman ) 73

73 Remarks Protocol is unfair Bob s pet might never get in Protocol uses waiting If Bob is eaten by his pet, Alice s pet might never get in 74

74 Moral of Story Mutual Exclusion cannot be solved by transient communication (cell phones) interrupts (cans) It can be solved by one-bit shared variables that can be read or written 75

75 The Arbiter Problem (an aside) Pick a point Pick a point 76

76 The Fable Continues Alice and Bob fall in love & marry 77

77 The Fable Continues Alice and Bob fall in love & marry Then they fall out of love & divorce She gets the pets He has to feed them 78

78 The Fable Continues Alice and Bob fall in love & marry Then they fall out of love & divorce She gets the pets He has to feed them Leading to a new coordination problem: Producer-Consumer 79

79 Bob Puts Food in the Pond A 80

80 Alice releases her pets to Feed mmm mmm B 81

81 Producer/Consumer Alice and Bob can t meet Each has restraining order on other So he puts food in the pond And later, she releases the pets Avoid Releasing pets when there s no food Putting out food if uneaten food remains 82

82 Producer/Consumer Need a mechanism so that Bob lets Alice know when food has been put out Alice lets Bob know when to put out more food 83

83 cola Surprise Solution A B 84

84 cola Bob puts food in Pond A B 85

85 Bob knocks over Can A B 86

86 Alice Releases Pets A yum yum B 87

87 cola Alice Resets Can when Pets are Fed A B 88

88 Pseudocode while (true) { while (can.isup()){}; pet.release(); pet.recapture(); can.reset(); } Alice s code 89

89 Pseudocode while (true) { while (can.isup()){}; pet.release(); pet.recapture(); can.reset(); while (true) { } while (can.isdown()){}; pond.stockwithfood(); can.knockover(); } Alice s code Bob s code 90

90 Mutual Exclusion Correctness Pets and Bob never together in pond 91

91 Mutual Exclusion Correctness Pets and Bob never together in pond No Starvation if Bob always willing to feed, and pets always famished, then pets eat infinitely often. 92

92 Correctness Mutual Exclusion Pets and Bob never together in pond No Starvation if Bob always willing to feed, and pets always famished, then pets eat infinitely often. safety liveness safety Producer/Consumer The pets never enter pond unless there is food, and Bob never provides food if there is unconsumed food. 93

93 Could Also Solve Using Flags A B 94

94 Waiting Both solutions use waiting while(mumble){} In some cases waiting is problematic If one participant is delayed So is everyone else But delays are common & unpredictable 95

95 The Fable drags on Bob and Alice still have issues 96

96 The Fable drags on Bob and Alice still have issues So they need to communicate 97

97 The Fable drags on Bob and Alice still have issues So they need to communicate They agree to use billboards 98

98 Billboards are Large B A 3 C 1 3 D 2 E 1 Letter Tiles From Scrabble box 99

99 Write One Letter at a Time W 4 A 1 S 1 H 4 B A 3 C 1 3 D 2 E 1 100

100 To post a message W A S H T H E C A R whew 101

101 Let s send another message L S E L L L A V A A M P 3 3 S 1 102

102 Uh-Oh S 1 E 1 L 1 L 1 T 1 H 4 E 1 C A R L 1 OK 103

103 Readers/Writers Devise a protocol so that Writer writes one letter at a time Reader reads one letter at a time Reader sees snapshot Old message or new message No mixed messages 104

104 Readers/Writers (continued) Easy with mutual exclusion But mutual exclusion requires waiting One waits for the other Everyone executes sequentially Remarkably We can solve R/W without mutual exclusion 105

105 Esoteric? Java container size() method Single shared counter? incremented with each add() and decremented with each remove() Threads wait to exclusively access counter 106

106 Readers/Writers Solution Each thread i has size[i] counter only it increments or decrements. To get object s size, a thread reads a snapshot of all counters This eliminates the bottleneck 107

107 Why do we care? We want as much of the code as possible to execute concurrently (in parallel) A larger sequential part implies reduced performance Amdahl s law: this relation is not linear 108

108 Amdahl s Law Speedup= 1-thread execution time n-thread execution time 109

109 Amdahl s Law Speedup= 1 1 p + p n 110

110 Amdahl s Law 1 Parallel fraction Speedup= 1 p + p n 111

111 Amdahl s Law Sequential fraction Speedup= 1 Parallel fraction 1 p + p n 112

112 Amdahl s Law Sequential fraction Speedup= 1 Parallel fraction 1 p + p n Number of threads 113

113 Amdahl s Law (in practice) 114

114 Example Ten processors 60% concurrent, 40% sequential How close to 10-fold speedup? 115

115 Example Ten processors 60% concurrent, 40% sequential How close to 10-fold speedup? Speedup = 2.17=

116 Example Ten processors 80% concurrent, 20% sequential How close to 10-fold speedup? 117

117 Example Ten processors 80% concurrent, 20% sequential How close to 10-fold speedup? Speedup = 3.57=

118 Example Ten processors 90% concurrent, 10% sequential How close to 10-fold speedup? 119

119 Example Ten processors 90% concurrent, 10% sequential How close to 10-fold speedup? Speedup = 5.26=

120 Example Ten processors 99% concurrent, 01% sequential How close to 10-fold speedup? 121

121 Example Ten processors 99% concurrent, 01% sequential How close to 10-fold speedup? Speedup = 9.17=

122 Back to Real-World Multicore Scaling Speedup 1.8x 2x 2.9x User code Multicore Not reducing sequential % of code 123

123 Shared Data Structures Coarse Grained 25% Shared Fine Grained 25% Shared 75% Unshared 75% Unshared 124

124 Shared Data Structures Honk! Honk! Honk! Why only 2.9 speedup Coarse Grained 25% Shared Fine Grained 25% Shared 75% Unshared 75% Unshared 125

125 Honk! Shared Data Structures Honk! Honk! Why fine-grained parallelism maters Coarse Grained 25% Shared Fine Grained 25% Shared 75% Unshared 75% Unshared 126

126 This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License. You are free: to Share to copy, distribute and transmit the work to Remix to adapt the work Under the following conditions: Attribution. You must attribute the work to The Art of Multiprocessor Programming (but not in any way that suggests that the authors endorse you or your use of the work). Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights. 127