Distributed Computing through Combinatorial Topology MITRO207, P4, 2017

Transcription

1 Distributed Computing through MITRO207, P4, 2017

2 Administrivia Language: (fr)anglais? Lectures: Fridays (28.04, , 30.06), Thursday (29.06), 8:30-11:45, B Web page: MITRO / Homeworks Corrected, not graded Office hours: C213-2, appointments by to Credit = written exams: Midterm: June 2, 2017 (1TH) Final: June 30, 2017 (1TH) Bonus for homeworks, participation, discussion of exercises, bugs found

3 Literature Distributed Computing Through Maurice Herlihy, Dmitry Kozlov, Sergio Rajsbaum Morgan Kaufman, 2013, available online (TPT library Lecture notes on Concurrent Computing. R. Guerraoui, P. Kuznetsov, 2017 (constantly under construction) Distributed Computing: Fundamentals, Simulations and Advanced Topics H. Attiya, J. Welch. (2nd edition). Addison Wesley Distributed Algorithms. N. Lynch. Morgan Kaufmann Publishers

4 (Preliminary) road map The matter and the method of distributed computing Basics of combinatorial topology Colorless tasks Simulations and reductions Generic tasks and manifold computations Renaming and oriented manifolds

5 In the Beginning a computer was just a Turing machine 5

6 Today??? Computing is co-ordination and communication 6

7 Moore s Law and CPU speed 2016 P. Kuznetsov

8 Distributed computations unfold in time! No, distributed computations are static mathematical objects! Operational versus combinatorial approaches Background picture: School of Athens, Rafael 8

9 Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 9

10 Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 10

11 There are Many Models 11

12 There are Many Models Communication? 12

13 There are Many Models Communication? Failures? 13

14 There are Many Models Communication? Failures? Timing? 14

15 Communication Message-Passing 15

16 Communication Message-Passing Read-Write Memory 16

17 Communication Message-Passing Read-Write Memory Black-Box Memory 17

18 Message-Passing Prof. James Moriarty Brown University Providence RI Mr. S. Holmes 221B Baker Street London NW1 6XE 18

19 Read-Write Memory

20 Read-Write Models write & read individual locations write & take memory snapshot Group writes together then takes snapshots together 20

21 Layered Read-Write Memory 21

22 Black Box Memory Compare-and-swap Fetch-and-add 22

23 Failures Crash failures: processes halt How many? Which ones? 23

24 Wait-Free Failure Model All but one may crash 24

25 t-resilient Failure Model At most t may crash Here, t = 1. 25

26 Correlated Failures Processes on same server may crash 26

27 Adversaries Walt Disney 27

28 Adversaries Determine which sets of processes can halt. Walt Disney 28

29 Adversaries Determine which sets of processes can halt. worst-case scenario Walt Disney 29

30 Timing Models 30

31 Timing Models Processes share a clock 31

32 Timing Models Processes share a clock Synchronous 32

33 Timing Models Processes share a clock Synchronous Processes do not share a clock 33

34 Timing Models Processes share a clock Synchronous Processes do not share a clock Asynchronous 34

35 Timing Models Processes share a clock Synchronous Processes do not share a clock Asynchronous Processes have approximatelysynchronized clocks 35

36 Timing Models Processes share a clock Synchronous Processes do not share a clock Asynchronous Processes have approximatelysynchronized clocks Semi-synchronous 36

37 Synchronous 37

38 Synchronous Failures! detection easy 38

39 Asynchronous 39

40 Asynchronous Failures?? detection impossible 40

41 Semi-Synchronous 41

42 Semi-Synchronous Failures! detection slow 42

43 Computation Model Space asynchronous semi-synchronous synchronous wait-free adversaries t-resilient messages read-write black-box Which combinations make sense? 43

44 Multicores Asynchronous Wait-free Shared Memory Sun Microsystems 44

45 Distributed Computing Internet Asynchronous Message-passing Sensor network 45

46 Parallel Computing Synchronous Message-passing (or shared memory) GPU 46

47 Local Views 110 Each process has a 3-bit local view 47

48 Multiple Local Views local views differ by 1 bit 48

49 Multiple Local Views local views differ by 1 bit but no process knows which one 49

50 Multiple Local Views each view is represented by a labeled vertex 50

51 Global States compatible views represented by an edge 51

52 All possible global states

53 Communication Each process sends local view to the other 53

54 Communication Each process sends local view to the other but at most one message may be lost! 54

55 One Communication Round ? ? 55

56 One Communication Round ? ? 1 Lost 56

57 One Communication Round ? ? 1 Lost none Lost 57

58 One Communication Round ? ? 1 Lost none Lost 1 Lost 58

59 One Communication Round 110? ? 59

60 All possible global states after one round unreliable communication 110? ?

61

62 Informally. Unreliable communication does not change topology of global states

63 Reliable Communication? 110? ? 63

64 Reliable Communication?

65

66 Tasks 66

67 32 19 Tasks 21 Possible set of input values 67

68 32 19 Tasks 21 Possible set of input values Finite computation 68

69 32 19 Tasks 21 Possible set of input values Finite computation Possible set of output values

70 Task Inputs Specification Outputs computation Protocol decision

71 Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 71

72 Muddy Children 11:00 72

73 Muddy Children 11:01 73

74 Muddy Children 12:00 At least one of you is dirty! 74

75 Muddy Children 12:00 You may not communicate! 75

76 Muddy Children 12:00 When you realize you are dirty, confess on the hour! 76

77 Muddy Children 1:00 (silence ) 77

78 Muddy Children 2:00 Me! Me! 78

79 Operational Explanation 1:00 79

80 Operational Explanation Others are clean, so I must be dirty. 1:00 80

81 Operational Explanation Others are clean, so I must be dirty. 1:00 Me! 81

82 Operational Explanation 1:01 82

83 Operational Explanation He was quiet, so I must be dirty. 1:01 He was quiet, so I must be dirty. 83

84 Combinatorial Explanation 12:00 84

85 Combinatorial Explanation 12:00 85

86 Combinatorial Explanation 12:00 Each process has its own input 86

87 Combinatorial Explanation 01??11 12:00 Each process has its own input 0?1 87

88 Combinatorial Explanation 01??11 12:00 Global State 0?1 88

89 all clean 11:59 00? 0? 0?00 0?1 01??01?1 0 10? 1? 0?11 1?1 11? all dirty

90 12:01 00? 0? 0?00 0?1 01??01?1 0 10? 1? 0?11 1?1 11? all dirty

91 1:01 0?1 01??01?1 0 10? 1? 0?11 1?1 11? all dirty

92 2:01?11 1?1 11? all dirty

93 Road Map Distributed Computing Two Classic Distributed Problems The Muddy Children Coordinated Attack 93

94 Coordinated Attack Alice Bob Red army wins If both sides attack together 94

95 The Two Generals Alice Bob Red generals send messengers across the valley 95

96 The Two Generals Alice Bob Messengers don t always make it 96

97 Your Mission Design a protocol to ensure that Alice and Bob attack simultaneously 97

98 Theorem There is no protocol that ensures that the Red armies attack simultaneously 98

99 Operational Proof Suppose Bob receives a message at 1:00 saying attack at Dawn. 99

100 Operational Proof Suppose Bob receives a message at 1:00 saying attack at Dawn. Are we done? 100

101 Operational Proof Suppose Bob receives a message at 1:00 saying attack at Dawn. Are we done? No, because Alice doesn t know if Bob got that message 101

102 Operational Proof So Bob sends an acknowledgment to Alice 102

103 Operational Proof So Bob sends an acknowledgment to Alice Are we done? 103

104 Operational Proof So Bob sends an acknowledgment to Alice Are we done? No, because Bob doesn t know if Alice got that message 104

105 Operational Proof So Bob sends an acknowledgment to Alice Are we done? No, because Bob doesn t know if Alice got that message 105

106 Attack at dawn! Attack at noon! Noon Bob is Alice is

107 delivered lost delivered 1:00 PM

108 delivered lost lost delivered 2:00 PM

109 delivered lost delivered protocol graph Attack at dawn! decision map Attack at noon! Don t attack! output graph

110 delivered lost delivered protocol graph These edges go here dawn! output graph noon!

111 This graph is connected

112 This graph is not connected

113 delivered lost delivered Map not allowed to tear protocol complex

114 protocol graph decision map output graph

115 Test and Set TAS(X) tests if X = 1, sets X to 1 if not, and returns the old value of X Instruction available on almost all processors TAS(X): atomic if X == 1 return 1; X = 1; return 0; atomic X = 1? yes no 1 X := 1 2 processes, P and Q, perform TAS(X) What is the protocol complex? 0

116 2-Process Test and Set initial state output 0 output 1 output 1 output 0 protocol graph What if there are 3 processes: P, Q, and R?

117 HW: (Simplified) Peterson s lock: 2 processes bool flag[0] = false; bool flag[1] = false; int turn; P0: P1: flag[0] = true; turn = 1; if(flag[1] and turn==1){ return false // failure } return true // critical section flag[1] = true; turn = 0; if(flag[0] and turn==0) { return false // failure } return true // critical section What is the protocol complex? Can we prove that the two processes cannot be both in the critical section

118 Operational Reasoning 118

119 Combinatorial Reasoning 119

120 Combinatorial Reasoning Model-independent properties 120

121 Combinatorial Reasoning Model-independent properties restricted model-dependent reasoning 121

122 This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License. You are free: to Share to copy, distribute and transmit the work to Remix to adapt the work Under the following conditions: Attribution. You must attribute the work to Distributed Computing Through (but not in any way that suggests that the authors endorse you or your use of the work). Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.