Chapter 6 Synchronization (2)

Transcription

1 DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN Chapter 6 Synchronization (2)

2 Plan Clock synchronization in distributed systems Physical clocks Logical clocks Ordered multicasting JGroups Mutual exclusion Election

3 JGroups Java toolkit for reliable group communication Join group Send to all or single group members Receive messages from group Unicast Multicast Unreliable UDP IP Multicast Reliable TCP JGroups

4 JGroups Similar to (BSD) sockets pull-based Building blocks for higherlevel functionality E.g., PullPushAdapter, RpcDispatcher Protocol stack Bidirectional list of protocol layers Used, e.g., for replication and load balancing in a number of application servers E.g., JBoss, Tomcat

5 JGroups Hello World

6 JGroups JGroup s flexible protocol stack has a number of ordering possibilities None org.jgroups.protocols.causal org.jgroups.protocols.total

7 JGroups: Alphabet Example Stable process group Send parts of alphabet in sequence Initiator Select address of random member in group Multicasts { A, <address>} Receivers print A, stores A Receiver with address <address> Select address of random member in group Multicasts { B, <address >} And so on... Which ordering do we want?

8 JGroups: Alphabet Example

9

10 JGroups: Stack setup Use UDP communication Ping via mcast Discover subgroups Unicast on top of mcast To make the example more interesting Heartbeat-based failure detection Check that FD is correct Sequence numbers + NAKs Compute msgs seen by all members And finally causal multicast (De-)fragmentation Group Membership Service

11 Plan Clock synchronization in distributed systems Physical clocks Logical clocks Ordered multicasting JGroups Mutual exclusion Election

12 Mutual Exclusion We often want to ensure that only one process accesses a critical region Requirements: ensure Mutually exclusive access Deadlock freedom Starvation freedom Fairness Types of algorithms Permission-based Token-based

13 Mutual Exclusion: A Centralized Algorithm Simulate how mutual exclusion is done in a one-processor system Let one process act as a coordinator Send request to coordinator to ask for permission

14 Mutual Exclusion A Centralized Algorithm Mutually exclusive access Coordinator only lets one process in Deadlock freedom If no process is in critical region, any other that wants to can be granted Starvation freedom Fairness Access granted in order of receipt at coordinator Only 3 messages per ressource use But coordinator is single point of failure and potential bottleneck

15 Mutual Exclusion: A Decentralized Algorithm Sigma: algorithm based on DHTs Logically replicate ressource to n coordinators Name ith replica rname-i and let node with id lookup(rnamei) be responsible for it To enter into criticial region Get grant from m > n/2 coordinators If cannot get enough grants, give up and retry sometimes later Problem: failure/recovery of coordinator May handle up to f failures where n m + f < m f < 2m n If probability that coordinator fails within time t is p then the probability of violating mutual exclusion is n k= 2m n m p k k (1 p) m k

16 Mutual Exclusion: A Decentralized Algorithm Mutual exclusion Average lifetime of node is seconds Performance problems 1. Difference in latency between client and replica 2. Clients are greedy n=32 m=24

17 Mutual Exclusion: A Distributed Algorithm Totally ordered multicast of requests for resources from requester to receivers 1. Receiver did not send request Send OK back 2. Receiver sent request, is accessing resource Wait and queue request 3. Receiver sent request, is not accessing resource Receiver has higher timestamp on its request: Send OK back Receiver has lower timestamp on its: Wait and queue request When requester has OK from all processes it accesses resource When done, send OK to all processes in queue

18 Mutual Exclusion: A Distributed Algorithm Guarantees Mutual exclusion Need OK from all processes Deadlock freedom Starvation freedom Fairness Timestamp used to decide But n-points-of-failure (instead of 1) Could try to detect whether resource has failed n bottlenecks (instead of 1) Could try to just get OK from majority

19 Mutual Exclusion: A Token Ring Algorithm Assume processes can be connected in a (logical) ring Token circulates on the ring Take the token if need to access critical region If token not needed, send it to next neighbor

20 Mutual Exclusion: A Token Ring Algorithm Guarantees Only one token at one process in a given time Mutual exclusion, deadlock freedom, starvation freedom, fairness But What if token is lost? Process crashes?

21 A Comparison of the Four Algorithms Figure A comparison of three mutual exclusion algorithms.

22 Global Positioning Of Nodes (1) Figure Computing a node s position in a two-dimensional space.

23 Global Positioning Of Nodes (2) Figure Inconsistent distance measurements in a one-dimensional space.

24 Election There is often a need for selecting a process with some special role E.g., choose coordinator in a centralized protocol If processes have unique id Try to find process with highest id, make this leader Algorithms differ in how they locate process with highest id (If you had views like in JGroups, how would you do election) If processes do not have unique id (or similar) li Then there is no way to select any of them to be special Well

25 Population Protocol All agents start in same state Eventually one agent is leader

26 The Bully Algorithm Assume that every process knows the ids of every other process Static group, but members may have failed P notices that there is no coordinator P wants to hold an election P sends an ELECTION message to all processes with higher numbers If no one responds, P wins the election and becomes coordinator If one of the higher-ups answers, it takes over P s job is done

27 The Bully Algorithm (1) Figure The bully election algorithm. (a) Process 4 holds an election. (b) Processes 5 and 6 respond, telling 4 to stop. (c) Now 5 and 6 each hold an election.

28 The Bully Algorithm (2) Figure The bully election algorithm. (d) Process 6 tells 5 to stop. (e) Process 6 wins and tells everyone.

29 A Ring Algorithm Build view of processes by circulating message Two steps: ELECTION and COORDINATOR Choose process with highest id from view as coordinator

30 Elections in Wireless Environments (1) Figure Election algorithm in a wireless network, with node a as the source. (a) Initial network. (b) (e) The build-tree phase

31 Elections in Wireless Environments (2) Figure Election algorithm in a wireless network, with node a as the source. (a) Initial network. (b) (e) The build-tree phase

32 Elections in Wireless Environments (3) Figure (e) The build-tree phase. (f) Reporting of best node to source.

33 Elections in Wireless Environments (4) Handling split and merge Use probe/reply messages to see if children in spanning tree are gone Handling node crashes and recovery Crash is a special case of split Recovery is treated as a special case of merge

34 Elections in Large-Scale Systems (1) Requirements for superpeer selection: 1. Normal nodes should have low-latency access to superpeers. 2. Superpeers should be evenly distributed across the overlay network. 3. There should be a predefined portion of superpeers relative to the total number of nodes in the overlay network. 4. Each superpeer should not need to serve more than a fixed number of normal nodes.

35 Elections in Large-Scale Systems (2) Assume we want L leaders in m-bit Chord DHT Reserve k = log 2 ( L) don t care bits lookup(p) = lookup(p && ) k Each superpeer is then responsible for k 2 N with high probability 2 m

36 Summary Mutual exclusion Centralized Decentralized Distributed Election Bully, ring, decentral algorithms