Election Algorithms. has elected i. will eventually set elected i

Transcription

1 Election Algorithms Election 8 algorithm designed to designate one unique rocess out of a set of rocesses with similar caabilities to take over certain functions in a distributes system central server for mutual exclusion ring master in token ring networks bus master 8 necessary when system is booted server fails server retires 8 roerties, to be valid during any articular run of the system E1: a rocess i has elected i = (undefined) or elected i = P for some non-crashed rocess P that will be chosen at the end of the run with the largest identifier (safety) E2: all rocesses i will eventually set elected i (liveness) 8 erformance network bandwidth utilization (roortional to total number of messages sent) turnaround time: number of serialized message transmission times between initiation and termination of a single run Distributed Systems - Fall 2001 IV - 27 Stefan Leue 2001

2 Election Algorithms Ring-based Algorithm 8 assumtions all nodes communicate on uni-directional ring structure all rocesses have unique integer id asynchronous, reliable system 8 initially, all rocesss marked non-articiant 8 to begin election, rocess lace election message with own identifier on ring and marks itself articiant 8 uon receit of election message, comare received identifier with own if received id greater than own id, forward message to neighbour if received id smaller than own id, iif own status is non-articiant, then substitute own id in election message and foward on ring iotherwise, do not forward message (already articiant ) if received id is identical to own id ithis rocess s id must be greatest and it becomes elected imarks own status as non-articiant isends out elected message 8 uon any forwarding, mark own state as articiant 8 when receiving elected message mark own status as non-articiant set elected i aroriately and forward elected message Distributed Systems - Fall 2001 IV - 28 Stefan Leue 2001

3 Election Algorithms Ring-based Algorithm 8 roerties E1 satisfied, since all identifiers are comared E2 follows from reliable communication roerty 8 erformance at worst 2N-1 messages for electing the left-hand neighbour another N elected messages 8 failures tolerates no failures Distributed Systems - Fall 2001 IV - 29 Stefan Leue 2001

4 Election Algorithms The Bully-Algorithm 8 works for synchronous networks nodes can crash, and crashes will be detected reliably 8 assumtions each node knows identifiers of all other nodes every node can communicate with every other node 8 message tyes election: announce an election answer: rely to an election message coordinator: announce identity of elected rocess Distributed Systems - Fall 2001 IV - 30 Stefan Leue 2001

5 Election Algorithms The Bully-Algorithm 8 initiation of algorithm: reliable failure detection a eer rocess failed if no answer to request within it = 2T trans + T rocess 8 rocess can decide whether to become coordinator by comaring own id with all other ids (highest wins) announce by sending coordinator message to all other nodes with lower id 8 rocess with lower id can bid to become coordinator by sending election message to all rocesses with higher ID if no resonse within T, considers itself elected coordinator, sends coordinator message to all rocesses with lower id otherwise, wait for another T time units for a coordinator message to arrive from new coordinator iif no resonse, then begin another election rocess 8 rocess receiving election message sets variable election i to the id of the coordinator received in the election message 8 if rocess receives election message, sends back an answer message and begins another election - unless one was already initiated 8 new rocess relacing crashed rocess if highest id, will immediately send coordinator message and bully current coordinator to resign Distributed Systems - Fall 2001 IV - 31 Stefan Leue 2001

6 The Bully Algorithm 8 examle Election Algorithms The election of coordinator 2, after the failure of 4 and then 3 Stage Stage Stage Stage Addison-Wesley Publishers 2000 Distributed Systems - Fall 2001 IV - 32 Stefan Leue 2001

7 The Bully Algorithm 8 examle Election Algorithms election The election of coordinator 2, after the failure of 4 and then 3 Stage 1 1 election answer 2 answer 3 election C 4 Stage 2 election answer election C timeout Stage Eventually... Stage 4 coordinator C Addison-Wesley Publishers 2000 Distributed Systems - Fall 2001 IV - 33 Stefan Leue 2001

8 Election Algorithms The Bully Algorithm 8 roerties E1 satisfied (if no rocess relaced and timeout T estimate accurate) E2 satisfied (synchronous network, reliable transmission) E1 not satisfied if crashed rocess relaced at the same time while another rocess has announced that it is the new coordinator 8 erformance best case: rocess with the second highest identifier detects coordinators failure ielects itself coordinator and sends N-2 coordinator messages requires O(N 2 ) messages in worst case when lowest id detects failure in-1 rocesses with higher IDs start election Distributed Systems - Fall 2001 IV - 34 Stefan Leue 2001

9 Multicast 8 grou communication sending and delivery of messages to more than one receiient ireceiving of message: queueing of arriving message in network interface buffer idelivery of message: assing message from network interface buffer to to target alication membershi in receiient grou transarent to sender ione send oeration to one address without having to send individual messages to all receiients 8 issues addressing coordination iguarantees that messages are received by a grou of receiients idelivery ordering amongst grou members 8 uses of multicast Comuter Suorted Collaborative Work (CSCW) communication with relicated servers (to achieve fault-tolerance) event notification in networks discovery services in sontaneous networking Distributed Systems - Fall 2001 IV - 35 Stefan Leue 2001

10 IP-based Multicast 8 only imlemented by some IP routers 8 available for UDP transort service 8 addressing: multicast address and ort number 8 IP multicast grou class D IP address for which first 4 bits are 1110 in IPv4 membershi is dynamic comuter belongs to multicast grou if one or more rocesses have sockets that belong to a multicast grou 8 imlementation of multicast IP routers on local area networks, use LAN's multicast caabilities (e.g., Ethernet) iuse locally valid multicast address, set Time To Live (TTL) counter in IP header to 1 so that acket will never get routed outside LAN in the Internet, router forwards messages to all other routers that have members in the multicast grou, which in turn forward the datagrams to grou members isession directory (sd) * grahical user interface tool allowing users to advertise multicast sessions as well as their valid multicast addresses 8 no guarantees whatsoever message loss, reordering, dulication, etc. Distributed Systems - Fall 2001 IV - 36 Stefan Leue 2001

11 Proerties of multicast 8 achieves not only transarency, but also enables stronger guarantees than "delivery by hand" efficient use of network hardware irouter sends individual messages iuses tree-like distribution structure if avialabe * two UDP-datagrams to same subnet: two IP ackets, the first delays the second * with IP multicast caabilities, only one IP datagram transmitted iuse of LAN-based multicast caabilities, if available delivery guarantees System model 8 messge m: contains ID of sender and of destination grou multicast(g, m): multicast message m to grou g deliver(m): delivery of a message at receiient 8 multicast grou is closed, if multicast only within oen, if rocesses not member of the grou may send to it Distributed Systems - Fall 2001 IV - 37 Stefan Leue 2001

12 Basic multicast 8 guaranteed delivery, unless multicaster crashes 8 rimitives and imlementation B-multicast(g, m): for each rocess g, send(, m) B-deliver(m) at : when receive(m) at, for all 8 roblem in using concurrent send(, m) oerations ack-imlosion: iall receiients acknowledge receit at about same time ibuffer overflow leads to droing of ack messages iretransmits, even more ack messages Distributed Systems - Fall 2001 IV - 38 Stefan Leue 2001

13 Reliable multicast 8 rimitives R-multicast(m, g) R-deliver(m) 8 desired roerties integrity: a correct rocess delivers a message at most once, and the delivered message is identical to the message sent in the multicast send oeration (safety) validity: if a correct rocess multicasts message m, then it will eventually deliver m (liveness) agreement: if a correct rocess delivers a message m, then all other correct rocesses in the target grou of message m will also deliver message m (additionally) uniform agreement: if a rocess, no matter whether it is correct or fails, delivers a message m, then all correct rocesse in the grou will deliver m as well 8 notes: validity is exressed in terms of self-delivery, for simlicity reasons ivalidity and agreement amount to overall liveness requirement: if one rocess (the sender) delivers a message m, then m will eventually be delivered to all the grou s correct members agreement is similar to atomicity : all-or-nothing semantics Distributed Systems - Fall 2001 IV - 39 Stefan Leue 2001

14 Reliable multicast 8 Imlementation B-multicast to rocesses in grou R-deliver 8 roerties validity: a correct rocess will eventually B-deliver to itself integrity: based on underlying communication medium agreement: B-multicast to all other rocesses after B-deliver 8 inefficient, since each message is sent g times to each rocess Addison-Wesley Publishers 2000 Distributed Systems - Fall 2001 IV - 40 Stefan Leue 2001

15 Reliable Multicast over IP Multicast 8 R-IP-multicast based on observation, that multicast successful in most cases use negative acknowledgement to indicate non-delivery 8 Basic idea closed multicast grous S g : sequence number for grou g that rocess belongs to R g : sequence number of latest message that a rocess has delivered from rocess and that was sent to grou g R-multicasts message to grou g iiggy back onto message * S g * acknowledgements <q, R gq > for all q iip-multicast message and iggy back information iincrement S g by one Distributed Systems - Fall 2001 IV - 41 Stefan Leue 2001

16 Reliable Multicast over IP Multicast 8 Basic idea R-deliver message from ionly if received sequence number S = R g +1 ithen increment R g by 1 iretain any message that cannot yet be delivered in hold-back-queue Message rocessing deliver Hold-back queue Delivery queue Incoming messages When delivery guarantees are met Addison-Wesley Publishers 2000 Distributed Systems - Fall 2001 IV - 42 Stefan Leue 2001

17 Reliable Multicast over IP Multicast 8 Basic idea R-deliver message from iif S R g, then message is already delivered, discard iif S > R g or R > R gq for any enclosed acknowledgement <q, R>, then receiver has missed one or more messages, requests retransmit through negative acknowledgement 8 roerties integrity ifollows from detection of dulicates and roerties of IP multicast (e.g., checksum to detect message corrution) validity imessage loss can only be detected when a successor message is eventually transmitted irequires rocesses to multicast messages indefinitely agreement irequires unbounded history for broadcast messages so that retransmit is always ossible 8 there exist ractical variants that ensure validity and agreement Distributed Systems - Fall 2001 IV - 43 Stefan Leue 2001

18 Ordered Multicast 8 assume: every rocess belongs to at most one grou 8 roerties FIFO ordering: if a correct rocess issues a multicast(g, m) and then multicast(g, m ), then every correct rocess that delivers m will deliver m before m causal ordering: if multicast(g, m) multicast(g, m ), where is induced by message assing only, then every correct rocess that delivers m will deliver m before m total ordering: if a correct rocess delivers m before it delivers m, then any other correct rocess that delivers m will deliver m before m 8 notes causal ordering imlies FIFO ordering FIFO ordering and causal ordering are artial (re-)orders total order allows arbitrary ordering of deliver events relative to multicast events, as long as this order is identical in all correct rocesses atomic multicast: reliable, totally ordered multicast Distributed Systems - Fall 2001 IV - 44 Stefan Leue 2001

19 Ordered Multicast 8 imlementing FIFO ordering S g : sequence number for grou g that rocess belongs to R g : sequence number of latest message that a rocess has delivered from rocess and that was sent to grou g assumtion: non-overlaing grous FO-multicast(m, g) ib-multicast(m, g, < S g >) iincrement S g by 1 uon receit of a message from q with sequence number S iif S = R g +1, then this is the next message, * therefore FO-deliver(m) * R g := S iif S > R g +1, then * lace message on hold-back queue until intervening messages have been delivered and S = R g +1 Distributed Systems - Fall 2001 IV - 45 Stefan Leue 2001

20 Ordered Multicast 8 imlementing total ordering idea: assign totally ordered identifiers to multicast messages so that every rocess makes the same delivery decision based on these identifiers delivery similar to FIFO delivery, only that grou-secific sequence numbers rather than rocess-secific sequence numbers are used assumtion: non-overlaing grous two main methods for the assignment of identifiers isequencer icollective agreement on the assignment of message identifiers Distributed Systems - Fall 2001 IV - 46 Stefan Leue 2001

21 Ordered Multicast 8 imlementing total ordering sequencer irocess wishing to TO-broadcast attaches a unique identifier id(m) to the message imessage is sent to sequencer as well as all members of g isequencer maintains grou-secific sequence number s g which it uses to assign increasing and consecutive sequence numbers to the messages it B-delivers iannounces the order in which members of g have to deliver these messages using a B-multicasted order message Distributed Systems - Fall 2001 IV - 47 Stefan Leue 2001

22 Ordered Multicast 8 imlementing total ordering sequencer Multicast S +1; Addison-Wesley Publishers 2000 Distributed Systems - Fall 2001 IV - 48 Stefan Leue 2001

23 Ordered Multicast 8 imlementing total ordering sequencer is bottleneck (erformance and/or reliability) collective agreement on the assignment of message identifiers iimlemented in the ISIS toolkit igrous may be oen or closed ireceiving rocesses bounce roosed sequence numbers to sender isender returns agreed sequence numbers ieach rocess q in grou g maintains * A gq : the largest agreed sequence number it has observed so far for grou g * P gq : own largest roosed sequence number Distributed Systems - Fall 2001 IV - 49 Stefan Leue 2001

24 Ordered Multicast 8 imlementing total ordering algorithm for collective agreement on the assignment of message identifiers i B-multicasts <m, i> to g, where i is unique identifier for m ieach receiient q relies to g with roosal for agreed sequence number * P gq := max(a gq, P gq ) + 1 * each rocess q rovisionally assigns own roosed sequence number to message and queues message in hold back queue, ordered according to roosed sequence number i chooses largest roosed number as sequence number a i B-multicasts <i, a> to g ieach rocess q in grou * sets A gq := max(a gq, a) * reorders received message in hold-back queue if received sequence number differs from roosed number * only when message at head of hold-back queue is assigned an agreed sequence number, it will be queued in delivery queue Distributed Systems - Fall 2001 IV - 50 Stefan Leue 2001

25 Ordered Multicast 8 imlementing total ordering algorithm for collective agreement on the assignment of message identifiers iobvious, that correct rocesses eventually agree on sequence number ito be shown, that * sequence numbers are monotonically increasing * no rocess delivers message before there is agreement iassume m1 to have agreed sequence number, and to be at head of hold-back queue * a message received after this state should be delivered after m1, it will have larger sequence number than m1 * let m2 another message, not yet with an agreed sequence number, but member of the same queue as m1 * ageedseq(m2) agreedseq(m1) (by construction of algorithm) * roosedseq(m2) > roosedseq(m1) (since m1 at head of queue) * agreedseq(m2) > agreedseq(m1) Distributed Systems - Fall 2001 IV - 51 Stefan Leue 2001