Distributed Algorithms Reliable Broadcast

Similar documents
Distributed Algorithms Models

Part 5: Total Order Broadcast

Distributed Algorithms Benoît Garbinato

Coordination 2. Today. How can processes agree on an action or a value? l Group communication l Basic, reliable and l ordered multicast

Specifying and Proving Broadcast Properties with TLA

Message-Efficient Uniform Timed Reliable Broadcast Yu Ma and Scott D. Stoller 21 September Introduction In distributed database systems,

CS505: Distributed Systems

Distributed Systems. Multicast and Agreement

Coordination and Agreement

Coordination and Agreement

Ruminations on Domain-Based Reliable Broadcast

Distributed systems. Total Order Broadcast

Understanding Non-Blocking Atomic Commitment

Basic vs. Reliable Multicast

Distributed Systems 2 Introduction

A Dual Digraph Approach for Leaderless Atomic Broadcast

A Case Study of Agreement Problems in Distributed Systems : Non-Blocking Atomic Commitment

The UNIVERSITY of EDINBURGH. SCHOOL of INFORMATICS. CS4/MSc. Distributed Systems. Björn Franke. Room 2414

Chapter 8 Fault Tolerance

Multicast Communication. Brian Nielsen

Two-Phase Atomic Commitment Protocol in Asynchronous Distributed Systems with Crash Failure

Distributed Systems. replication Johan Montelius ID2201. Distributed Systems ID2201

Distributed Systems. 09. State Machine Replication & Virtual Synchrony. Paul Krzyzanowski. Rutgers University. Fall Paul Krzyzanowski

CSE 5306 Distributed Systems

CSE 5306 Distributed Systems. Fault Tolerance

Distributed Algorithms Failure detection and Consensus. Ludovic Henrio CNRS - projet SCALE

Formal Development of Fault Tolerant Transactions for a Replicated Database using Ordered Broadcasts

A General Characterization of Indulgence

Clock Synchronization. Synchronization. Clock Synchronization Algorithms. Physical Clock Synchronization. Tanenbaum Chapter 6 plus additional papers

Replication in Distributed Systems

CSE 486/586 Distributed Systems Reliable Multicast --- 1

CSE 486/586 Distributed Systems

Fault Tolerance. Basic Concepts

Building TMR-Based Reliable Servers Despite Bounded Input Lifetimes

Parsimonious Asynchronous Byzantine-Fault-Tolerant Atomic Broadcast

Process groups and message ordering

Distributed Algorithms. Partha Sarathi Mandal Department of Mathematics IIT Guwahati

Distributed Algorithms (PhD course) Consensus SARDAR MUHAMMAD SULAMAN

Distributed systems. Consensus

Distributed algorithms

Clock and ordering. Yang Wang

EECS 591 DISTRIBUTED SYSTEMS

Fault-Tolerant Distributed Consensus

Distributed Systems (ICE 601) Fault Tolerance

Fault Tolerance. Distributed Software Systems. Definitions

Implementing Shared Registers in Asynchronous Message-Passing Systems, 1995; Attiya, Bar-Noy, Dolev

Initial Assumptions. Modern Distributed Computing. Network Topology. Initial Input

Failures, Elections, and Raft

System Models for Distributed Systems

System models for distributed systems

Dependability and real-time. TDDD07 Real-time Systems Lecture 7: Dependability & Fault tolerance. Early computer systems. Dependable systems

Byzantine Consensus in Directed Graphs

Fault Tolerance via the State Machine Replication Approach. Favian Contreras

Chapter 5: Distributed Systems: Fault Tolerance. Fall 2013 Jussi Kangasharju

Consensus Problem. Pradipta De

Applications of Paxos Algorithm

Coordination and Agreement

Parallel Data Types of Parallelism Replication (Multiple copies of the same data) Better throughput for read-only computations Data safety Partitionin

Distributed Systems. coordination Johan Montelius ID2201. Distributed Systems ID2201

Verteilte Systeme/Distributed Systems Ch. 5: Various distributed algorithms

Concurrency and OS recap. Based on Operating System Concepts with Java, Sixth Edition, 2003, Avi Silberschatz, Peter Galvin e Greg Gagne

Introduction to Distributed Systems Seif Haridi

Asynchronous Reconfiguration for Paxos State Machines

Distributed Systems Principles and Paradigms. Chapter 08: Fault Tolerance

Chapter 8 Fault Tolerance

System Models 2. Lecture - System Models 2 1. Areas for Discussion. Introduction. Introduction. System Models. The Modelling Process - General

Semi-Passive Replication in the Presence of Byzantine Faults

Linearizability CMPT 401. Sequential Consistency. Passive Replication

Cache Coherence in Distributed and Replicated Transactional Memory Systems. Technical Report RT/4/2009

Consensus and related problems

CS555: Distributed Systems [Fall 2017] Dept. Of Computer Science, Colorado State University

Secure Distributed Programming

DISTRIBUTED SYSTEMS [COMP9243] Lecture 5: Synchronisation and Coordination (Part 2) TRANSACTION EXAMPLES TRANSACTIONS.

Distributed Systems Fault Tolerance

DISTRIBUTED SYSTEMS [COMP9243] Lecture 5: Synchronisation and Coordination (Part 2) TRANSACTION EXAMPLES TRANSACTIONS.

Distributed systems. Lecture 6: distributed transactions, elections, consensus and replication. Malte Schwarzkopf

A MODULAR FRAMEWORK TO IMPLEMENT FAULT TOLERANT DISTRIBUTED SERVICES. P. Nicolas Kokkalis

Time in Distributed Systems

6.852: Distributed Algorithms Fall, Class 12

BYZANTINE GENERALS BYZANTINE GENERALS (1) A fable: Michał Szychowiak, 2002 Dependability of Distributed Systems (Byzantine agreement)

Practical Byzantine Fault Tolerance. Miguel Castro and Barbara Liskov

The Long March of BFT. Weird Things Happen in Distributed Systems. A specter is haunting the system s community... A hierarchy of failure models

The Timed Asynchronous Distributed System Model By Flaviu Cristian and Christof Fetzer

21. Distributed Algorithms

CS 138: Practical Byzantine Consensus. CS 138 XX 1 Copyright 2017 Thomas W. Doeppner. All rights reserved.

Self-stabilizing Byzantine Digital Clock Synchronization

System aspects of dependable computers

Distributed Algorithms (PhD course) Consensus SARDAR MUHAMMAD SULAMAN

Correct-by-Construction Attack- Tolerant Systems. Robert Constable Mark Bickford Robbert van Renesse Cornell University

Transactions Between Distributed Ledgers

Sigma: A Fault-Tolerant Mutual Exclusion Algorithm in Dynamic Distributed Systems Subject to Process Crashes and Memory Losses

Replications and Consensus

Eventual Consistency Today: Limitations, Extensions and Beyond

A Distributed and Robust SDN Control Plane for Transactional Network Updates

Distributed Algorithms Introduction

Capacity of Byzantine Agreement: Complete Characterization of Four-Node Networks

Distributed Algorithms 6.046J, Spring, 2015 Part 2. Nancy Lynch

Failure Models. Fault Tolerance. Failure Masking by Redundancy. Agreement in Faulty Systems

Broker B2 for SPA and SPB

Lecture 1: Introduction to distributed Algorithms

Transcription:

Distributed Algorithms Reliable Broadcast Alberto Montresor University of Trento, Italy 2016/04/26 This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Contents 1 Introduction 2 Broadcast specifications and protocols Best-Effort Broadcast Reliable Broadcast Uniform Reliable Broadcast 3 Message ordering Introduction Specification A modular approach Algorithms and proofs Atomic Broadcast

Introduction Introduction Efficient techniques are required, capable of supporting consistent behavior between system components in spite of failures. Examples Reliable Broadcast/Multicast protocols: Ensure reliable message delivery to all participants Agreement protocols: Ensure all participants to have a consistent system view Commit protocols: Implement atomic behavior in transactional types of systems Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 1 / 55

Introduction Broadcast A deliver B broadcast deliver Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 2 / 55

Introduction Broadcast Protocol Layering Application Protocol Application Protocol broadcast deliver broadcast deliver Broadcast Protocol Broadcast Protocol send receive send receive Communication Network Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 3 / 55

Introduction Basic assumptions (1) System is asynchronous No bounds on messages and process execution delays Processes fail by crashing stop executing actions after the crash We do not consider Byzantine failures Correct/Faulty A process that does not fail in a run is correct in that run Otherwise, the process is faulty Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 4 / 55

Introduction Basic assumptions (2) We will consider two failure models for communication: No Failures Validity: If p sends a message to q, and q is correct, then q will eventually receive m Integrity: No message is delivered to a process more than once, and only if it has been sent previously Perfect Channels Validity: If p sends a message to q, and p,q are correct, then q will eventually receive m Integrity: No message is delivered to a process more than once, and only if it has been sent previously Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 5 / 55

Introduction What kind of underlying network? Complete graph Every process can communicate with every other process A routing substrate realizes this abstraction Point-to-point Every process can communicate with a subset of processes (its neighbors) Routing is not implemented at the send/receive level (we may implement it at the level of our protocols) Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 6 / 55

Introduction Different flavors of broadcast Reliability Best-effort Reliable Uniform Reliable Ordering FIFO Casual Atomic FIFO Atomic Causal Atomic Time bounds Timed Reliable Primitives R-Broadcast F-Broadcast C-Broadcast... Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 7 / 55

Broadcast specifications and protocols Best-Effort Broadcast Best-effort broadcast Specification Definition (BEB1 Validity) If p and q are correct, then every message B-broadcast by p is eventually delivered by q Definition (BEB2 Uniform Integrity) m is delivered by a process at most once, and only if it was previously broadcast Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 8 / 55

Broadcast specifications and protocols Best-Effort Broadcast Best-effort broadcast Algorithm Best-effort broadcast protocol executed by p upon B-broadcast(m) do foreach q Π do send m to q upon receive(m) do B-deliver(m) Notation Send to all foreach q Π do send m to q is equivalent to send m to Π Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 9 / 55

Broadcast specifications and protocols Best-Effort Broadcast Best-effort broadcast Proof We can show that the protocol works with Perfect Channels: BEB1 - Validity: By the Validity property of Perfect Channels and the very facts that 1 the sender sends the message to all 2 every correct process that receives a message B-delivers it BEB2 Uniform Integrity: By the Integrity property of Perfect Channels Clearly, it will work also with No Failures Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 10 / 55

Broadcast specifications and protocols Best-Effort Broadcast Best-effort broadcast Example p 1 m p 2 m p 3 m Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 11 / 55

Broadcast specifications and protocols Best-Effort Broadcast Best-effort broadcast Example p 1 crash p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 12 / 55

Broadcast specifications and protocols Best-Effort Broadcast Best-effort broadcast Problem What happens if the sender fails? Even in the absence of communication failures: if the sender crashes before being able to send the message to all some processes will not deliver the message What we do? First we revise the specification of broadcast Then we implement the new specification Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 13 / 55

Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Specification Definition (RB1 Validity) If a correct process broadcasts m, then it eventually delivers m Definition (RB2 Uniform Integrity) m is delivered by a process at most once, and only if it was previously broadcast Definition (RB3 Agreement) If a correct process delivers m, then all correct processes eventually deliver m Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 14 / 55

Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Scenario 1 Does this execution satisfy the RB specification? p 1 crash p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 15 / 55

2016-10-19 DS - Reliable Broadcast Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Scenario 1 Reliable Broadcast Scenario 1 Does this execution satisfy the RB specification? crash p 1 p 2 p 3 m 2 Obviously yes the fact that process p 1 does not deliver m is not a problem, because Validity only requires correct processes to deliver their own messages.

Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Scenario 2 Does this execution satisfy the RB specification? p 1 crash p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 16 / 55

2016-10-19 DS - Reliable Broadcast Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Scenario 2 Reliable Broadcast Scenario 2 Does this execution satisfy the RB specification? crash p 1 p 2 p 3 Obviously yes the fact that no process delivers m is not a problem, because process p 1 is faulty and Validity does not apply; and nobody delivers m, so Agreement does not apply.

Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Scenario 3 Does this execution satisfy the RB specification? p 1 crash p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 17 / 55

2016-10-19 DS - Reliable Broadcast Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Scenario 3 Reliable Broadcast Scenario 3 Does this execution satisfy the RB specification? crash p 1 p 2 p 3 Obviously no Agreement is not satisfied.

Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Algorithm v.1 Reliable broadcast protocol executed by p upon initialization do Set delivered upon R-broadcast(m) do send m to Π {p} R-deliver(m) delivered delivered {m} upon receive(m) from q do if not m delivered then send m to Π {p, q} R-deliver(m) delivered delivered {m} % Messages already delivered Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 18 / 55

Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Scenario 4 Does this execution satisfy the RB specification? p 1 crash p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 19 / 55

2016-10-19 DS - Reliable Broadcast Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Scenario 4 Reliable Broadcast Scenario 4 Does this execution satisfy the RB specification? crash p 1 p 2 p 3 Yes, because before R-delivering the message, p 2 processes. forwards it to all other

Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Proof Algorithm v.1 implements Reliable Broadcast. RB1 Validity: If a correct process broadcasts m, then it eventually delivers m By the code implementing R-broadcast. RB2 Agreement: If a correct process delivers m, then all correct processes eventually deliver m Before R-delivering m, a correct process p forwards m to all processes. By Validity of Perfect Channels and the fact that p is correct, all correct processes will eventually receive m and R-deliver it. RB3 Integrity: m is delivered by a process at most once, and only if it was previously broadcast By the Integrity of Perfect Channels and the use of variable delivered Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 20 / 55

Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Scenario 5 Does this execution satisfy the RB specification? p 1 crash p 2 crash p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 21 / 55

2016-10-19 DS - Reliable Broadcast Broadcast specifications and protocols Reliable Broadcast Reliable Broadcast Scenario 5 Reliable Broadcast Scenario 5 Does this execution satisfy the RB specification? m p 2 crash 1 crash p 2 p 3 Yes, because the fact that has been delivered by p 1 and p 2, which are not correct, does not imply that has to be delivered by p 3 as well. Yet, this situation is not desirable, because two processes deliver something and another one does not.

Broadcast specifications and protocols Uniform Reliable Broadcast Uniform Reliable Broadcast Specification Definition (URB1 Validity) If a correct process broadcasts m, then it eventually delivers m Definition (URB2 Uniform Agreement) If a correct process delivers m, then all correct processes eventually deliver m Definition (URB3 Uniform Integrity) m is delivered by a process at most once, and only if it was previously broadcast Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 22 / 55

Broadcast specifications and protocols Uniform Reliable Broadcast Uniform Reliable Broadcast Proof Algorithm v.1 implements Uniform Reliable Broadcast...... but under different assumptions! URB1, URB2: As RB1, RB2 RB3 Uniform Agreement: If a process delivers m, then all correct processes eventually deliver m Before R-delivering m, a process forwards m to all processes. By Validity of Perfect Channels, all correct processes will eventually receive m and R-deliver it In the absence of communication failures, all correct processes will eventually receive m and R-deliver it Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 23 / 55

Message ordering Introduction Message ordering Problem Given the asynchronous nature of distributed systems, messages may be delivered in any order Some services, such as replication, need messages to be delivered in a consistent manner, otherwise replicas may diverge. Solution We describe a collection of ordering policies and we show how to implement them in a modular way. Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 24 / 55

Message ordering Specification Message ordering Definition (FIFO Order) If a process p broadcasts a message m before it broadcast a message m, the no correct process delivers m unless it has previously delivered m broadcast p (m) broadcast p (m ) deliver q (m) deliver q (m ) Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 25 / 55

Message ordering Specification Definition (Causal Order) If the broadcast of a message m happens-before the broadcast of a message m, then no correct process delivers m unless it has previously delivered m broadcast p (m) broadcast q (m ) deliver r (m) deliver r (m ) Is this causal? No! p 1 p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 26 / 55

Message ordering Specification Definition (Causal Order) If the broadcast of a message m happens-before the broadcast of a message m, then no correct process delivers m unless it has previously delivered m broadcast p (m) broadcast q (m ) deliver r (m) deliver r (m ) Is this causal? Yes! p 1 p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 27 / 55

Message ordering Specification Definition (Causal Order) If the broadcast of a message m happens-before the broadcast of a message m, then no correct process delivers m unless it has previously delivered m broadcast p (m) broadcast q (m ) deliver r (m) deliver r (m ) Is this causal? Yes! p 1 p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 28 / 55

Message ordering Specification Message ordering Problem Causal Broadcast does not impose any order on messages not causally related Example Consider a replicated database with two copies of a bank account Initially, account = 1000$ A user deposits 150$ triggering a broadcast of = {add 150$ to account} At the same time the bank initiates a broadcast of = {add 2% interest to account} Causal Broadcast allows two processes to deliver these updates in different order, creating inconsistency Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 29 / 55

Message ordering Specification Definition (Total Order) If correct processes p and q both deliver messages m,m, then p delivers m before m if and only if q delivers m before m deliver p (m) deliver p (m ) deliver q (m) deliver q (m ) Is this totally ordered? No! p 1 p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 30 / 55

Message ordering Specification Definition (Total Order) If correct processes p and q both deliver messages m,m, then p delivers m before m if and only if q delivers m before m deliver p (m) deliver p (m ) deliver q (m) deliver q (m ) Is this totally ordered? Yes! p 1 p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 31 / 55

Message ordering Specification Uniform Versions Definition (Uniform FIFO Order) If a process p broadcasts a message m before it broadcast a message m, then no correct process delivers m unless it has previously delivered m broadcast p(m) broadcast p(m ) deliver q(m) deliver q(m ) Definition (Uniform Causal Order) If the broadcast of a message m happens-before the broadcast of a message m, then no correct process delivers m unless it has previously delivered m broadcast p(m) broadcast q(m ) deliver r(m) deliver r(m ) Definition (Uniform Total Order) If correct processes p and q both deliver messages m,m, then p delivers m before m if and only if q delivers m before m deliver p(m) deliver p(m ) deliver q(m) deliver q(m ) Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 32 / 55

Message ordering A modular approach A modular approach to Broadcast Reliable Broadcast Total Order Atomic Broadcast FIFO Order FIFO Order FIFO Broadcast Total Order FIFO Atomic Broadcast Causal Order Causal Order Causal Broadcast Total Order Causal Atomic Broadcast Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 33 / 55

Message ordering A modular approach A modular approach to Broadcast Uniform Reliable Broadcast Uniform FIFO Order Uniform Total Order Uniform Atomic Broadcast Uniform FIFO Order Uniform FIFO Broadcast Uniform Total Order Uniform FIFO Atomic Broadcast Uniform Causal Order Uniform Causal Order Uniform Causal Broadcast Uniform Total Order Uniform Causal Atomic Broadcast\ Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 34 / 55

Message ordering A modular approach Transformation Informal definition A broadcast transformation is an algorithm that takes a weaker broadcast algorithm and transform it into a stronger version Definition (Transformation) A transformation from problem A to problem B is an algorithm T A B that converts any algorithm A that solves problem A into an algorithm B that solves problem B Definition (Preservation) A transformation T A B preserves property P if it converts any algorithm for A into an algorithm that solves problem B and also satisfies P Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 35 / 55

Message ordering A modular approach Transformation Properties of weakest RB must be preserved Uniform Integrity: preserved in all transformations No message is created Messages are tagged to avoid re-delivery Validity, (Uniform) Agreement: To be proved case by case To add Total Order: We cannot start from a simple reliable broadcast We need stronger assumptions Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 36 / 55

Message ordering A modular approach Transformation Definition (Blocking transformation) A transformation of one broadcast algorithm into another is blocking if the resulting broadcast algorithm has a run in which a process delays the delivery of a message for a later time. Example FIFO Order Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 37 / 55

Message ordering Algorithms and proofs FIFO Order Algorithm FIFO Order Transformation executed by process p upon initialization do Set buffer integer[ ] next new integer[1... Π ] foreach q Π do next[q] 1 upon F-broadcast(m) do R-broadcast(m) upon R-deliver(m) do buffer buffer {m} while m buffer : sender(m ) = sender(m) and seqn(m ) = next[q] do F-deliver(m ) next[q] next[q] + 1 buffer buffer {m } Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 38 / 55

Message ordering Algorithms and proofs FIFO Order Proof Theorem For any process p, if next p [q] = k then p has F-delivered the first k 1 messages F-broadcast by q Theorem Suppose a correct process p R-delivers a message m from q and F-delivers all the messages that q F-broadcast before m. Then p also F-delivers m Validity, (Uniform) Agreement, (Uniform) Total Order are preserved Uniform FIFO Order is satisfied The transformation is blocking Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 39 / 55

2016-10-19 DS - Reliable Broadcast Message ordering Algorithms and proofs FIFO Order Proof FIFO Order Proof Theorem For any process p, if nextp[q] = k then p has F-delivered the first k 1 messages F-broadcast by q Theorem Suppose a correct process p R-delivers a message m from q and F-delivers all the messages that q F-broadcast before m. Then p also F-delivers m Validity, (Uniform) Agreement, (Uniform) Total Order are preserved Uniform FIFO Order is satisfied The transformation is blocking Clai Proof If m is the last message delivered from sender(m), let k be equal to seqn(m ). By clai, next p[q] = k + 1 and by definition, seqn(m) = k + 1. Thus m will be delivered as the next message. Validity Proof To F-broadcast m, a correct process p R-broadcasts it. It will eventually R-deliver m (Validity of RB) Suppose m is the first message p R-delivers, but not F-delivers For clai, it will eventually F-deliver m. Absurd. Uniform FIFO Order Proof Suppose a process p F-delivers a message m F-broadcast by q. Let seqn(m) = k. By the algorithm, just before F-delivering m, next p[q] = k. By Clai, p has already F-delivered all the k 1 messages that q F-broadcast before m, as wanted.

Message ordering Algorithms and proofs Causal Order - Algorithm Two transformations: Both based on FIFO Reliable Broadcast One is non-blocking Each message is tagged with recent history When a message is F-delivered, all the causal messages that have been F-delivered are locally delivered Does this recall anything? One blocking Based on vector clocks Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 40 / 55

Message ordering Algorithms and proofs Causal Order - Algorith Causal Order Transformation executed by process p upon initialization do Set delivered % Messages already C-delivered Sequence recent % Messages C-delivered since last C-broadcast upon C-broadcast(m) do F-broadcast(recent m) recent upon F-deliver(,..., m k ) do for i 1 to k do if not m i / delivered then delivered delivered {m i } recent recent m i C-deliver(m i ) Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 41 / 55

Message ordering Algorithms and proofs Causal Order - Algorith p 1 p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 42 / 55

Message ordering Algorithms and proofs Causal Order Proof Validity, (Uniform) Agreement, (Uniform) Total Order are preserved Uniform Causal Order is satisfied The transformation is non-blocking Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 43 / 55

Message ordering Algorithms and proofs Causal Order - Algorith Causal Order Transformation executed by process p upon initialization do Set buffer % Messages to be delivered integer[ ] VC {0,..., 0} % Vector clock upon C-broadcast(m) do F-broadcast( m, VC ) upon F-deliver( m, TS ) do buffer buffer { m, TS while m, TS buffer : VC [sender(m )] = TS [sender(m )] 1 s sender(m ) : VC [s] T S [s] do C-deliver(m ) update VC buffer buffer {m} Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 44 / 55

Message ordering Algorithms and proofs Causal Order - Algorith p 1 [1,0,0] [1,2,0] p 2 p 3 Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 45 / 55

Message ordering Algorithms and proofs Causal Order Proof Validity, (Uniform) Agreement, (Uniform) Total Order are preserved Uniform Causal Order is satisfied The transformation is blocking Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 46 / 55

Message ordering Algorithms and proofs A modular approach to Broadcast (Uniform) Reliable Broadcast (Uniform) FIFO Order (Uniform) Total Order (Uniform) Atomic Broadcast (Uniform) FIFO Order (Uniform) FIFO Broadcast (Uniform) Causal Order (Uniform) Total Order (Uniform) FIFO Atomic Broadcast (Uniform) Causal Order (Uniform) Causal Broadcast (Uniform) Total Order (Uniform) Causal Atomic Broadcast Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 47 / 55

Message ordering Atomic Broadcast Atomic Broadcast There are three approaches: 1 We add synchronous assumptions to our system 2 We show that the Atomic Broadcast problem is equivalent to the Consensus problem There is an algorithm T Consensus AtomicBroadcast There is an algorithm T AtomicBroadcast Consensus 3 Through a coordinator (actual implementation, see later in group communication) Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 48 / 55

Message ordering Atomic Broadcast Timed Reliable Broadcast Definition ((Uniform) Real-Time -Timeliness) There is a known constant such that if a message m is broadcast at real-time t, then no correct (any) process delivers m after real-time t + Definition ((Uniform) Local-Time -Timeliness) There is a known constant such that no correct (any) process delivers m after local time TS(m) + on p s clock, where TS(m) is the timestamp obtained by the local clock of the sender Note (Uniform) Real-Time -Timeliness (Uniform) Local-Time -Timeliness Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 49 / 55

Message ordering Atomic Broadcast Atomic Broadcast, Algorith Total Order Transformation executed by process p upon A-broadcast(m) do T-broadcast(m) upon T-deliver(m) do schedule A-deliver(m) at time TS(m) + Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 50 / 55

Message ordering Atomic Broadcast Consensus In the (Uniform) Consensus problem, the processes propose values and need to decide (agree) on one of these values t s 5 Consensus 8 5 2 5 p 5 r q 8 5 7 Crash! Definition (Uniform Validity) Any value decided is a value proposed Definition ((Uniform) Agreement) No two correct (any) processes decide differently Definition (Termination) Every correct process eventually decides Definition (Uniform Integrity) Every process decides at most once Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 51 / 55

Message ordering Atomic Broadcast From Atomic Broadcast to Consensus Transformation executed by process p upon initialization do boolean decided false upon propose(v) do A-broadcast(v) upon A-deliver(v) do if not decided then decided true decide(u) Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 52 / 55

Message ordering Atomic Broadcast From Consensus to Atomic Broadcast Transformation executed by process p upon initialization do Set unordered Set delivered boolean wait false integer s 1 upon A-broadcast(m) do R-broadcast(m) upon R-deliver(m) do if not m delivered then unordered unordered {m} % Messages to be ordered % Messages already delivered % true when Consensus is running % Consensus protocol identifier Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 53 / 55

Message ordering Atomic Broadcast From Consensus to Atomic Broadcast Transformation executed by process p upon decide s (S) do unordered unordered S foreach m S do A-deliver(m) % In some deterministic order delivered delivered S s s + 1 wait false upon unordered and not wait do wait true propose s (unordered) Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 54 / 55

Message ordering Atomic Broadcast Conclusions Summary Consensus and total order broadcast are equivalent problems in an asynchronous system with crashes and Perfect Channels Consensus can be obtained from total order broadcast Total order broadcast can be obtained from Consensus Problem But in this way, we have moved the problem from Atomic Broadcast to Consensus. Next step: can we solve Consensus? Alberto Montresor (UniTN) DS - Reliable Broadcast 2016/04/26 55 / 55

Reading Material V. Hadzilacos and S. Toueg. A modular approach to fault-tolerant broadcasts and related problems. In S. Mullender, editor, Distributed Systems (2 nd ed.). Addison-Wesley, 1993. http://www.disi.unitn.it/~montreso/ds/papers/ftbroadcast.pdf