Part 5: Total Order Broadcast

Transcription

1 Dependable Distributed Systems 2 Part 5: Total Order Broadcast Prof Dr. Felix Freiling (based on slides by Rachid Guerraoui, EPFL)

2 Looking Back Asynchronous system model with reliable channels best-effort/reliable/uniform broadcast without failure detectors with or without FIFO/causal order regular/uniform consensus FLP impossibility algorithms with failure detectors weakest failure detectors for consensus Today: back to broadcast 2

3 Consensus Agreement Validity Termination Consensus is a fundamental agreement abstraction (well-studied in the literature) "smallest common agreement problem" stronger agreement abstractions derived from solutions to consensus 3

4 Total Order Broadcast Reliable broadcast with total order all processes see the same delivery order sometimes also called atomic broadcast broadcast() deliver() deliver () broadcast() 4

5 Overview Intuitions: what total order broadcast can be used for? Specifications of total order broadcast Consensus-based total order algorithm 5

6 Uniform Reliable Broadcast Properties: "Safety" and "Liveness" plus Agreement or Uniform Agreement Non-Uniform Reliable Broadcast can be constructed in the obvious way broadcast(m) deliver(m) 6

7 Broadcast Properties URB1. Validity: If pi and pj are correct, then every message broadcast by pi is eventually delivered by pj URB2. No duplication: No message is delivered more than once URB3. No creation: No message is delivered unless it was broadcast URB4. Uniform Agreement: For any message m, if a process delivers m, then every correct process delivers m 7

8 Ordered Reliable Broadcast None, FIFO, causal best-effort FIFO best-effort causal besteffort reliable FIFO reliable causal reliable uniform reliable FIFO uniform reliable causal uniform reliable 8

9 Partial and Total Orders In (uniform) reliable broadcast, the processes are free to deliver messages in any order they wish In causal broadcast, the processes need to deliver messages according to causal order The order imposed by causal broadcast is however partial: some messages might be delivered in different order by the processes 9

10 p1 Reliable Broadcast m3 p2 m3 p3 m3 m3 10

11 p1 Causal Broadcast m3 p2 m3 p3 m3 m3 11

12 Total vs. FIFO/Causal Order In total order broadcast, the processes must deliver messages according to the same order (i.e., the order is now total) Note that this order does not need to respect causality (or even FIFO ordering) Total order is orthogonal to FIFO/causal order Total order broadcast can be made to respect causal (or FIFO) ordering 12

13 Total Order Broadcast? (1/4) FIFO, causal, total? p1 m3 m3 p2 p3 m3 m3 13

14 Total Order Broadcast? (2/4) FIFO, causal, total? p1 m3 p2 m3 p3 m3 m3 14

15 Total Order Broadcast? (3/4) FIFO, causal, total, uniform? p1 m3 p2 m3 p3 m3 15

16 Total Order Broadcast? (4/4) FIFO, causal, total, uniform? p1 m3 p2 p3 m3 m3 16

17 Applications (1/2) A replicated service where the replicas need to treat the requests in the same order to preserve consistency replica 1 replica 2 replica 3 17

18 Applications (2/2) A notification service where the subscribers need to get notifications in the same order 18

19 Overview Intuitions: what total order broadcast can bring? Now: Specifications of total order broadcast two variants: regular and uniform Consensus-based algorithm 19

20 Total order broadcast (tob) Events Request: <tobroadcast, m> Indication: <todeliver, src, m> Properties: RB1, RB2, RB3, RB4 Total order property 20

21 Total order broadcast (utob) Events Request: <utobroadcast, m> Indication: <utodeliver, src, m> Properties: URB1, URB2, URB3, URB4 Uniform Total order property 21

22 (Uniform) Total order broadcast Validity: If pi and pj are correct, then every message broadcast by pi is eventually delivered by pj No duplication: No message is delivered more than once No creation: No message is delivered unless it was broadcast (Uniform) Agreement: For any message m. If a correct (any) process delivers m, then every correct process delivers m 22

23 (Uniform) Total order broadcast (cont.) Total order: Let pi and pj be any two correct processes that deliver a message m. If pi delivers a message m before m, then pj delivers m before m. Uniform Total order: Let pi and pj be any two processes that deliver a message m. If pi delivers a message m before m, then pj delivers m before m. order with respect to any delivered message m 23

24 Exercise Compare the following two properties: Uniform Total order: Let pi and pj be any two processes that deliver a message m. If pi delivers a message m before m, then pj delivers m before m. Naive total order: Let pi and pj be any two processes that deliver two messages m and m. If pi delivers m before m, then pj delivers m before m. Safety/Liveness? UTO NTO? 24

25 Overview Intuitions: what total order broadcast can bring? Specifications of total order broadcast Now: Consensus-based algorithm for Uniform total order broadcast 25

26 Uniform Consensus In the uniform consensus problem, the processes propose values and need to agree on one among these values UC1. Validity: Any value decided is a value proposed UC2. Uniform Agreement: No two processes decide differently UC3. Termination: Every correct process eventually decides UC4. Integrity: Every process decides at most once 26

27 Uniform Consensus Events Request: <ucpropose, v> Indication: <ucdecide, v > Properties: UC1, UC2, UC3, UC4 27

28 Modules of a process indication request request indication (R-U)Consensus 28

29 Algorithm Idea We use uniform reliable broadcast (URB) as a transport mechanism for uniform total order broadcast We use uniform consensus (UC) to agree on total order messages are disseminated using URB delivered (but unordered) messages are stored in a buffer periodically we use UC to agree on a set of to-bedelivered messages (sequence of rounds) deliver these messages in a predefined order 29

30 Algorithm Implements: UniformTotalOrder (uto). Uses: Uniform ReliableBroadcast (urb). Uniform Consensus (ucons); upon event < Init > do unordered = delivered = { }; wait := false; sn := 1; 30

31 Algorithm upon event < utobroadcast, m> do trigger < urbbroadcast, m>; upon event <urbdeliver,sm,m> and (m not in delivered) do unordered := unordered U {(sm,m)}; upon (unordered not empty) and not(wait) do wait := true: trigger < ucpropose, unordered> sn ; 31

32 Algorithm upon event <ucdecide,decided> sn do unordered := unordered \ decided; ordered := deterministicsort(decided); for all (sm,m) in ordered: trigger < utodeliver,sm,m>; delivered := delivered U {m}; sn : = sn + 1; wait := false; 32

33 Example p1 utob() p2 utob(m4) p3 utob(m3) p4 utob() consensus p1 p2 p3 p4,,m3,m3 m3,m4 m3,m4 m3,m4 m3,m4 utod() utod() utod(m3,m4) 33

34 Correctness (1/3) Validity: If pi and pj are correct, then every message broadcast by pi is eventually delivered by pj No duplication: No message is delivered more than once 34

35 Correctness (2/3) No creation: No message is delivered unless it was broadcast Uniform Agreement: For any message m. If any process delivers m, then every correct process delivers m 35

36 Correctness (3/3) Uniform Total order: Let pi and pj be any two processes that deliver a message m. If pi delivers a message m before m, then pj delivers m before m. 36

37 Adding FIFO/causal Order How can we add FIFO order? exchange URB with FIFO uniform reliable broadcast? How add causal order? Exchange URB with causal URB? 37

38 Adding FIFO Order Replace URB with a FIFO URB primitive Local deliveries will respect FIFO order let message be sent by process p before cannot be proposed to consensus unless has been todelivered or is proposed at the same time Take care that deterministicsort respects FIFO order too 38

39 Adding Causal Order Replace URB with a causal URB primitive Same type of argument as for FIFO let message -> is not delivered unless has been delivered cannot be proposed to consensus unless has been todelivered or is proposed at the same time Look out for deterministicsort 39

40 Total Order total order can be added to any type of reliable broadcast reliable FIFO reliable causal reliable total order reliable total order FIFO reliable total order causal reliable 40

41 Total Broadcast in Context So we can build total order broadcast using consensus! Can we build total order broadcast using just reliable broadcast (without consensus)? How can we prove that this is impossible? 41

42 Building Consensus out of Atomic Broadcast We can construct consensus using atomic broadcast? need to map invocations of Propose and Decide to invocations of tobroadcast and todeliver Idea: whenever a process Proposes a value, this value is tobroadcast to everybody other processes receive proposed values using todeliver they decide on the first value received 42

43 Construction Idea propose(x) y,z,x decide(y) propose(y) y,z,x decide(y) decide(y) propose(z) y,z,x Agreement? Validity? Termination? 43

44 Proof Consensus Agreement: follows from total order and URB Agreement Consensus Validity: algorithm does not introduce new values URB doesn't either (no creation property) Termination: follows mainly from URB Termination 44

45 Equivalences 1. One can build consensus with total order broadcast 2. One can build total order broadcast with consensus and reliable broadcast Therefore, consensus and total order broadcast are equivalent problems in a system with reliable channels 45

46 Questions What is the weakest failure detector for total order broadcast? Majority of correct processes? Minority? Given an eventuallp perfect failure detector: can you implement total order broadcast? Can you do it with? 46

47 Summary Total order (atomic) broadcast reliable broadcast with total delivery order consensus-based algorithm Equivalence to consensus Coming next: Other (strong) coordination problems Non-blocking atomic commit Terminating reliable broadcast and their relation to consensus... 47