Byzantine Fault Tolerance and Consensus. Adi Seredinschi Distributed Programming Laboratory

Transcription

1 Byzantine Fault Tolerance and Consensus Adi Seredinschi Distributed Programming Laboratory 1

2 (Original) Problem Correct process General goal: Run a distributed algorithm 2

3 (Original) Problem Correct process Arbitrary failure General goal: Run a distributed algorithm 2

4 (Recasting the) Problem REQUEST REPLY Client Application Distributed algorithm 3

5 Recasting the problem Application requirements: High-Availability (give a reply to a request) Reliability (give correct replies) REQUEST REPLY Distributed algorithm Client Application Boils down to fault-tolerance 4

6 Solution Fault-tolerance basic techniques: Agreement = Consensus REPLY Client Replication = State Machine Replication REQUEST Application In the following we will see PBFT Seminal algorithm for Byzantine Fault Tolerance Distributed algorithm 5

7 PBFT Practical Byzantine Fault Tolerance OSDI 99 Miguel Castro Barbara Liskov 6

8 Modules Application State Machine Replication (SMR) Consensus Perfect Links Channels 7

9 Modules Application State Machine Replication (SMR) PBFT Consensus Perfect Links Channels 7

10 Overview PBFT: System model slightly different from what we ve seen so far SMR Consensus 8

11 System model Processes Three types of processes in this algorithm: Clients n replicas one of them is primary others are backup n 9

12 System model Failure model Arbitrary (Byzantine) faults Clients:. Any client can be faulty Replicas: n = 3f + 1 f faulty (upper bound) n=4 f=1 n=7 f=2 10

13 System model Network & crypto Assume perfect links Direct links between any two processes For messages: Public-key signatures, message authentication codes Avoid spoofing, replays, corruption Clients are authenticated Can revoke access to faulty clients 11

14 Overview PBFT: System model slightly different from what we ve seen so far SMR Consensus 12

15 State Machine Replication (SMR) A fault-tolerance technique. Basic ideas: Application = state machine Process 1. Process n Run the application on multiple processes Each processes is a faithful replica of the application Note: We can ignore the primary/backup distinction in this example 13

16 SMR in a nutshell All SMR replicas start from the same state () 14

17 SMR in a nutshell a S2 a a S2 S2 All SMR replicas start from the same state () Transition to state (S2) due to operation a 14

18 SMR in a nutshell a S2 null a S2 a a S2 c All SMR replicas start from the same state () Transition to state (S2) due to operation a 14

19 SMR in a nutshell a S2 null S2 a S2 a S3 a S2 c S4 All SMR replicas start from the same state () Transition to state (S2) due to operation a A faulty client can wreak havoc 14

20 SMR in a nutshell a S2 null S2 rand() a S2 a S3 rand() a S2 c S4 rand() All SMR replicas start from the same state () Transition to state (S2) due to operation a A faulty client can wreak havoc 14

21 SMR in a nutshell a S2 null S2 rand() S5 a S2 a S3 rand() S6 a S2 c S4 rand() S7 All SMR replicas start from the same state () Transition to state (S2) due to operation a A faulty client can wreak havoc Non-determinism is also harmful 14

22 SMR in a nutshell a S2 null S2 rand() S5 a S2 a S3 rand() S6 a S2 c S4 rand() S7 All SMR replicas start from the same state () Transition to state (S2) due to operation a A faulty client can wreak havoc Non-determinism is also harmful 14 Diverging states!

23 SMR Requirements Avoid diverging states All replicas must: 1. Start in the same state 2. Execute the same sequence of operations 3. Use only provided operation (+parameters), thus avoid non-determinism 15

24 SMR Requirements Avoid diverging states All replicas must: 1. Start in the same state 2. Execute the same sequence of operations 3. Use only provided operation (+parameters), thus avoid non-determinism Simple Consensus Depends on application 15

25 SMR in PBFT Application = a distributed file system Network File System (NFS) Operations = write to a file, delete, etc. Primary/backup distinction is relevant NFS State Machine Replication (SMR) Consensus Perfect Links Channels 16

26 SMR in PBFT Application = a distributed file system Network File System (NFS) Operations = write to a file, delete, etc. Primary/backup distinction is relevant NFS State Machine Replication (SMR) Consensus Perfect Links Channels The replicated state is a file system 16

27 SMR in PBFT REQUEST Client contacts the primary with a request 17

28 SMR in PBFT REQUEST Consensus Client contacts the primary with a request All replicas agree on the request & execute it 17

29 SMR in PBFT REQUEST REPLY Consensus REPLY REPLY REPLY Client contacts the primary with a request All replicas agree on the request & execute it 17 Client gets the same reply from all correct replicas

30 SMR in PBFT REQUEST REPLY Lost Consensus REPLY REPLY REPLY Client contacts the primary with a request All replicas agree on the request & execute it 18 Redundancy in Replies = Cope with failures

31 SMR in PBFT REQUEST REPLY Lost Consensus REPLY REPLY REPLY Client contacts the primary with a request All replicas agree on the request & execute it 18 Redundancy in Replies = Cope with failures

32 Overview PBFT: System model slightly different from what we ve seen so far SMR Consensus 19

33 Consensus The core for many algorithms, including: TRB, Group membership, View synchronous b-cast, State machine replication Traditionally In PBFT: Instance Processes propose values Instance Clients propose requests Primary multicasts the requests to backup replicas Agree on a proposed value Primary & replicas agree on the sequence of request 20

34 Consensus in PBFT We ll assume one client Proposals = requests for application operations Assume: n = 4, f = 1 REQUEST REQUEST REQUEST The faulty replica does not cooperate Concurrent requests: Consensus to agree on a sequential execution of requests 21

35 Consensus in PBFT Algorithm ideas: Client sends requests to the primary replica Execute a sequence of consensus instances: Each instance is dedicated to a request Instances (and therefore requests) are sequentially ordered by the primary Backup replicas adopt requests from the primary in the imposed order Properties: Validity, Agreement, Termination, Integrity 22

36 Consensus in PBFT REQUEST Consensus 23

37 Consensus in PBFT REQUEST Consensus A three-phase protocol 23

38 Consensus instance = three-phase protocol REQUEST

39 Consensus instance = three-phase protocol REQUEST

40 Consensus instance = three-phase protocol REQUEST

41 Consensus instance = three-phase protocol REQUEST

42 Consensus instance = three-phase protocol REQUEST REPLY

43 Consensus instance = three-phase protocol REQUEST REPLY

44 Consensus Corner case What if the primary is faulty, e.g. does not multicast the request to the backups? View change protocol: primary replaced by one of the backups Idea: Replicas are numbered 1 n In view v, the replica p is the primary, where p = v mod n 29

45 Consensus Why 3 phases? 1. PRE-PREPARE 2. PREPARE 3. COMMIT 30

46 Consensus Why 3 phases? 1. PRE-PREPARE Agree on (the order of) requests within the same view 2. PREPARE 3. COMMIT 30

47 Consensus Why 3 phases? 1. PRE-PREPARE Agree on (the order of) requests within the same view 2. PREPARE 3. COMMIT Ensure that requests which execute are totally ordered across different views + Garbage collection 30

48 Practical BFT Reasonable overhead 3% Does not assume synchrony Some clever optimizations: PBFT NFS MD5 replaces digital signatures Message digests Read-only requests, tentative execution 31

49 Further reading Castro, M., & Liskov, B. (1999). Practical Byzantine fault tolerance. OSDI, (February), Available at: Castro, M. (2011). Practical Consensus. Microsoft Research Cambridge. Available at: 32