Distributed Systems (5DV020) Group communication. Fall Group communication. Fall Indirect communication

Transcription

1 Distributed Systems (5DV020) Group communication Fall Group communication 2 Indirect communication Communication through an intermediary with no direct coupling between the sender and the receiver(s) 3 1

2 Group communication a message is sent to a group of processes and then this message is delivered to all members of the group 4 p6 join(group_id) group_id p2 p3 p4 p5 p1 multicast(m, group_id) One-to-many communication Management functionality Maintain membership Detect failure of member(s) Provide reliability and ordering guarantees 5 Closed group Open group A group is closed if only members of the group can multicast to it A group is open if processes outside the group may send to it Figure adapted from Instructor s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 Pearson Education 2012 based on Figure

3 A B overlapping group A B non-overlapping group In overlapping groups, processes may be members of multiple groups In non-overlapping groups, processes may belong to at most one group 7 Reliability in group communication Integrity A message sent will be delivered correctly at most once Validity A message sent will eventually be delivered Agreement If a message is delivered to one process, it is delivered to all processes in the group 8 Group membership management Interface for group membership changes Create and destroy groups, add or remove members to a group Failure detection Mark processes as suspected or unsuspected and remove those processes that have (suspected) failed 9 3

4 Notify members of group membership changes Processes that join or leave Perform group address expansion From group id to individual group members (current) Group membership management 10 Summary 11 Recap on group communication One-to-many, indirect communication Different types of groups Open, closed, overlapping, and non-overlapping Reliability in group communication Integrity, validity, and agreement Group membership management changes, failure detection, notification of membership changes, group address expansion 12 4

5 Coordination and agreement in group communication 13 Multicast 14 Multicast Receive vs. Deliver Receive: message has arrived and will be processed Deliver: message is allowed to reach upper layer Static vs. dynamic groups Multicast to groups whose membership is known Unreliable (basic) multicast (using reliable unicast) Send (unicast) to each other process in the group! What if sender fails halfway through? 15 5

6 Receive vs. deliver Multicast Hold-back queue Message processing deliver message delivery Delivery queue Incoming messages message receipt When delivery guarantees are met Figure adapted from Instructor s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 Pearson Education 2012 based on Figure Reliable multicast 17 Integrity Reliable multicast messages delivered at most once Validity if a correct process multicasts message m, it will eventually deliver m Agreement if a correct process delivers m, then all correct processes in the group will eventually deliver m 18 6

7 Use basic multicast to send to all (including self) When basic multicast delivers, check if message has been received before 1. If it has, do nothing further 2. If not, and sender is not own process Basic multicast message to others 3. Deliver message to upper layer Reliable multicast 19 Integrity? Validity? Agreement? Yes! Reliable multicast Insane amounts of traffic? Yes! Every message is sent sizeof(group) to each process! A single message will be sent 100 times if we just have 10 processes 20 Other approaches Reliable multicast Scalable reliable multicasting (SRM) using feedback suppression, hierarchical feedback control IP multicast requires hardware support IP, however, is unreliable Study algorithm on your own Perfect exam material

8 Message ordering Unordered 2. FIFO 3. Total 4. Causal Hybrids such as total-causal & total-fifo 23 FIFO ordering 24 8

9 FIFO Intuition messages from a process should be delivered in the order in which they were sent Solution sender numbers the messages, receivers hold back those that have been received out of order 25 Process P1 S(p 1,g) # of messages that p has sent to the group R(p 2,g) sequence # of latest message that p 1 has delivered from p 2 that was sent to g R(p 3,g) R(p n,g) n members of g FO-multicast Send S(p i,g) with message FIFO B-multicast message Increment by S(p i,g) 1 FO-deliver If S=R(p j, g) + 1 FO-deliver and set R(p j, g) = S If S>R(p j, g) + 1 Place in hold-back queue until S = R(p j, g) Total ordering 27 9

10 Total Intuition: messages from all processes should get a group wide ordering number, so all processes can deliver them in a single order! Mental pitfall: the order itself does not have to make any sense, as long as all processes abide by it! 28 Implementing total ordering 29 Total Sequencer Simple Central server (= single point of failure) P ISIS-algorithm 2 1 Message 3 2 Not as simple 1 Distributed 1 2 P 1 3 Study on your own! 3 Agreed Seq P 4 P 3 Figure adapted from Instructor s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 Pearson Education 2012 based on Figure

11 Total ordering via sequencer 31 Sequencer is logically external to the group Messages are sent to all members and the Sequencer Initially, have no ordering number Sequencer maps message identifiers to ordering numbers Multicasts mapping to group Once a message has an ordering number, it can be delivered according to that number Total 32 Total 1 2 Send the to g and sequencer Wait until right time to deliver (given by sequence # from sequencer) 4 3 multicastsequential # to g sequence # is totally ordered Figure adapted from Instructor s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 Pearson Education 2012 based on Figure

12 Note, again, that the ordering is completely up to the sequencer It could collect all messages for half an hour and then assign numbers according to how many a s there are in the message While annoying to use, this is still a total order, and all processes will have to follow it! Total 34 Causal ordering 35 Intuition: captures causal (cause and effect) relationships via happened-before ordering Causal Using vector clocks, ensures that replies are delivered after the message that they are replying to 36 12

13 Causal delivers previously messages from p j zeroes the vector clock Increases own clock Multicast message and vector clock It has delivered any message that p j delivered at thattime thatit multicasted this message Increases vector clock for p j Figure adapted from Instructor s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 Pearson Education 2012 based on Figure Hybrid orderings 38 Hybrid Causal order is not unique Concurrent messages...neither is FIFO FIFO only guarantees per process, not inter-process Total order only guarantees a unique order Combine with others to get stronger delivery semantics! 39 13

14 You will need to get familiar with these orderings during the assignment Read the book carefully Avoid mistakes Adhere to definitions Get a few very nice hints Summary 41 Multicast, reliable and unreliable Message ordering The ordering in delivering messages is necessary in some cases Ordering is expensive in terms of delivery latency and bandwidth consumption 42 14

15 FIFO order messages from each sender Causal order messages across senders Total same message ordering on all recipients Very important for the assignment 43 Next Lecture Replication 44 15