Lecture XII: Replication
|
|
- Christal Clarke
- 5 years ago
- Views:
Transcription
1 Lecture XII: Replication CMPT 401 Summer 2007 Dr. Alexandra Fedorova
2 Replication 2
3 Why Replicate? (I) Fault-tolerance / High availability As long as one replica is up, the service is available Assume each of nreplicas has same independent probability pto fail. Availability = 1 -p n Fault-Tolerance: Take-Over 3
4 Why Replicate? (II) Fast local access (WAN replication) client can always send requests to closest replica Goal: no communication to remote replicas necessary during request execution Goal: client experiences location transparency since all access is fast local access Fast local access Rome Toronto Montreal 4
5 Why Replicate? Scalability and load distribution (LAN replication) Requests can be distributed among replicas Handle increasing load by adding new replicas to the system cluster instead of bigger server 5
6 Challenges: Data Consistency We will study systems that use data replication It is hard, because data must be kept consistent Users submit operations against the logical copies of data These operations must be translated into operations against one, some, or all physical copies of data Nearly all existing approaches follow a ROWA(A) approach: Read-one-write-all-(available) Update has to be (eventually) executed at all replicas to keep them consistent Read can be performed at any replica 6
7 Challenges: Fault Tolerance The goal is to have data available despite failures If one site fails others should continue providing service How many replicas should we have? It depends on: How many faults we want to tolerate The typesof faults we expect How much we are willing to pay 7
8 Roadmap Replication architectures Active replication Primary-backup (passive, master-slave) replication Design considerations for replicated services Surviving failures 8
9 Active Replication Replicated Servers A Client BA CA 9
10 Active Replication 10
11 Active Replication 1. The client send request to the servers using totally ordered reliable multicast (logical clocks or vector clocks) 2. Server coordination is given by the total order property (assumption: synchronous system) 3. All replicas execute the request in the order they are delivered 4. No additional coordination necessary (Assumption: determinism) All replicas produce the same result 5. All replicas send result to the client; client waits for the first answer 11
12 Fault Tolerance: FailstopFailures As long as at least one replica survives the client will continue receiving service Assuming there are no partitions! Suppose B and C are partitioned, so the cannot communicate They cannot agree on how Client to order client s requests Replicated Servers A BA CA 12
13 Fault Tolerance: Byzantine Failures Can survive Byzantine failures (assuming no partitions) The system must have n 2f + 1 replicas (f is the number of failures) The client will compare results of all replicas, will choose the result returned by the majority f + 1non-faulty replicas This is the idea used in LOCKSS (Lots of Copies Keep Stuff Safe) 13
14 Primary-Backup Replication (PB) Replicated Servers Client A primary If the primary fails, a backup takes over, becomes the primary BA backup CA backup Also known as passive replication 14
15 System Requirements How do we want the system to behave? Just like a single-server system? Must ensure that there is only one primary at a time Data is kept consistent: If a client received a response from an update operation and then the system crashed, the client should find the data reflecting that update Results of operations should be the same as they would be if executed on a single-server system Can we tolerate loose data consistency? The client eventually gets the consistent data, but not right away 15
16 Example of Data Inconsistency Client operations: write(x = 5) read (x) On a replicated system: write (x = 5) read (x) // should return 5 on a single-server system Primary responds to client Primary crashed before propagating update to other replicas A new primary is selected // may return x 5, the new primary does not know about the update to x 16
17 Design Considerations for Replicated Services Where to submit updates? A designated server or any server? When to propagate updates? Eager or lazy? How many replicas to install? 17
18 Where to Submit Updates? Primary Copy: - Each object has a primary copy - Often there is a designated primary -it holds primary copies for all objects - Updates on object x have to be submitted to the primary copy of x - Primary propagates changes on x to secondary copies - Secondary copies are read-only - Also called master/slave approach 18
19 Where to Submit Updates Update Everywhere: Both read and write operations can be submitted to any server This server takes care of the execution of the operation and the propagation of updates to the other copies T1:r(x)w(y) T2:r(y)w(y) 19
20 When to Propagate Updates? Eager: Within the boundaries of the transaction for replicated databases Before response is sent to client for non-transactional services Lazy: After the commit of the transaction for replicated databases After the response is sent to client for non-transactional services 20
21 PB Replication with Eager Updates 1. The client sends the request to the primary 2. There is no initial coordination 3. The primary executes the request 4. The primary coordinates with the other replicas by sending the update information to the backups 5. The primary (or another replica) sends the answer to the client Updates are propagated eagerly, before we respond to client 21
22 Eager Update Propagation 22
23 Eager Update Propagation For Transactional Services On every update At the end of transaction 23
24 When Can a Failure Occur? F1: Primary fails before replica coordination Client receives no response. It will retry. Eventually will get data from new primary. F2: Primary fails during replica coordination Replicas may or may not have reached agreement w.r.t. client s transaction. Client may receive a response after system recovers. The system may fail to recover (if the agreement protocol blocks). F3:Primary fails after replica coordination A new primary responds F1 F2 F3 Phase 1: Client Request Phase 3: Execution Phase 4: Replica Coordination Phase 5: Client response 24
25 Lazy Update Propagation (Transactional Services) Primary Copy: Upon read: read locally and return to user Upon write: write locally and return to user Upon commit/abort: terminate locally Sometime after commit: multicast changed objects in a single message to other sites (in FIFO) Secondary copy: Upon read: read locally Upon message from primary copy: install all changes (FIFO) Upon write from client: refuse (writing clients must submit to primary copy) Upon commit/abort request (only for read-only txn): local commit Note: existing systems allow different objects to have different primary copies A transaction that wants to write X (primary copy is site S1) and Y (primary copy on site S2) is usually disallowed 25
26 Lazy Update Propagation A client may end up with an inconsistent view of the system 26
27 Lazy Propagation: Discussion Lazy replication has no server/agreement coordination within response time Faster Transactions might be lost in case of primary crash Weak data consistency Simple to achieve Secondary copies only need to apply updates in FIFO order Data at secondary copies might be stale Multiple Primaries possible (multi-master replication) More locality 27
28 How Many Replicas? Properties of correct PB protocol Property 1: There is at most one primary at any time Property 2: Each client maintains the identity of the primary, and sends its requests only to the primary Property 3:If a client update arrives at a backup, it is not processed When a primary fails, we must elect a new one Network partitions may cause election of more than one primary We can avoid network by choosing the right number of replicas (under certain failure assumptions) How many replicas do we need to tolerate failures? 28
29 System Model Synchronous system (useful for deriving theoretical results) Fully connected network (exactly one FIFO link between any two processes) Failure model: Crash failures: also known as failstop failures Crash+Linkfailures: A server may crash or a link may lose messages (but links do not delay, duplicate or corrupt messages) Receive-Omission failures: A server may crash and also omit to receive some of the messages send over a non-faulty link Send-Omission failures: A server may fail not only by crashing but also by omitting to send some messages over a non-faulty link General-Omission failures: A server may exhibit send-omission and receive-omission failures 29
30 Lower Bounds on Replication How many replicas n do you need to tolerate ffailures? Failure Model crash Degree of Replication n > f crash+link n> f+1 receive-omission n > send-omission general-omission n > f 3f 2 n > 2f 30
31 Crash Failures, Send-Omission Failures: n > freplicas Becomes primary FAILED (crashedor fail to send) 31
32 Other Failure Models The rest of the failure models may create partitions Partitions: Servers are divided into mutually noncommunicating partitions A primary may emerge in each partition, so we ll have more than one primary against the rules To avoid partitions, we use more replication 32
33 Crash+LinkFailures: n > f+1replicas Scenario 1: f servers fail Scenario 2: f links fail FAILED UNREACHABLE BUT ALIVE Becomes primary Becomes primary Problem! 2 primaries!!! Becomes primary 33
34 Crash+LinkFailures: n > f+1replicas We need another correct node that would serve as a link between the two partitions We can assume that its links will be correct, because we allow no more than f failures UNREACHABLE BUT ALIVE Becomes primary Becomes primary 34
35 Omission Failures Precise definitions of omission failures[perry-toueg86] Notation: sent(pj, Pi) a message sent from Pjto Pi received(pi, Pj) a message received by Pi frompj Receive-omission failure of Pi with respect to Pj: sent(pj, Pi) received(pi, Pj) Send-omission failure of Pi with respect to Pj: Pifails to send a message prescribed by a protocol to Pj General-omission failure of Pi w.r.t. Pj Pi commits both receive-omission and send-omission w.r.t. Pj 35
36 Receive-Omission Failures: n > 3f/2 Replicas f/2 f/2 A FAIL B Server in A becomes primary FAIL C f/2 f servers in B and C fail 36
37 Receive-Omission Failures: n > 3f/2 Replicas f/2 f/2 FAIL A B f servers in A and C fail FAIL C f/2 Server in B becomes primary 37
38 Receive-Omission Failures: n > 3f/2 Replicas Server in A becomes primary f/2 Servers in B: receive-omission failures Servers in A: receive-omission failures w.r.t. processes outside their partition From A servers perspective, everyone else has crashed: partition! A f/2 C B Problem! 2 primaries!!! f/2 Server in B becomes primary Need another nonfailed server that links the partitions 38
39 General-Omission Failures: n>2f Replicas f f FAIL A B Becomes primary f f A Becomes primary FAIL B 39
40 General-Omission Failures: n>2f Replicas A commits general-omission failures w.r.t. servers in B A s servers think all servers in B failed one of them becomes primary B s servers think all servers in A failed one of them becomes primary A server in A becomes a primary, a server in B becomes a primary: We have two primaries To fix this, we need another non-faulty server that will link the two partitions f f Becomes primary A B Becomes primary 40
41 How Many Replicas? Summary We showed how many replicas are needed to prevent partitions in the face of f failures However partitions do happen due to router failures, for example So having extra replicas won t help, because they will also be on one of the sides of the faulty router Next we ll talk about surviving failures despite network partitions 41
42 Surviving Network Partitions Most systems operate under assumption that a partition will eventually be repaired Optimistic approach: Allow updates in all partitions When the partition is repaired, eventually synchronize the data OK for a distributed file system (think about your laptop in disconnected mode) Pessimistic approach: Allow updates only in a single partition used where strong consistency is required (flight reservation system) Which partition? This is usually decided by quorum consensus After partition is repaired update copies of data in the other partition 42
43 Quorum Consensus Quorum is a sub-group of servers whose size gives it the right to carry out the operation Usually the majority gets the quorum Design/implementation challenges: Replicas must agreethat they are behind a partition must rely on timeouts, failure detectors (special devices?) If the quorum set does not contain the primary, the replicas must elect the new primary Quorum Cost consideration: to tolerate one partition, must have at least three servers. Implement one as a simple witness? 43
44 Bringing Replicas Up-to-Date Version numbers: Each copy has a version number (or a timestamp) Only copies that are up-to-date have the current version number Operations should be applied only to copies with the current version number How does a failed server finds out that its not up-to-date? Periodically compare all version numbers? Log sequence numbers: Each operation is written to a log (like a transactional log) Each log record has a log sequence number (LSN) Replica managers compare LSN s to find out if they are not up-todate Used by Berkeley DB replication system 44
45 Discussed replication Summary Used for performance, high availability Active replication Client sends updates to all replicas Replicas co-ordinate amongst themselves, apply updates in order Passive replication (primary copy, primary-backup) Eager/lazy update propagation Number of replicas to prevent partitions Handling partitions Optimistic Pessimistic (quorum consensus) Next time we will look at real systems that use replication 45
Replication in Distributed Systems
Replication in Distributed Systems Replication Basics Multiple copies of data kept in different nodes A set of replicas holding copies of a data Nodes can be physically very close or distributed all over
More informationCSE 544 Principles of Database Management Systems. Alvin Cheung Fall 2015 Lecture 14 Distributed Transactions
CSE 544 Principles of Database Management Systems Alvin Cheung Fall 2015 Lecture 14 Distributed Transactions Transactions Main issues: Concurrency control Recovery from failures 2 Distributed Transactions
More informationBasic vs. Reliable Multicast
Basic vs. Reliable Multicast Basic multicast does not consider process crashes. Reliable multicast does. So far, we considered the basic versions of ordered multicasts. What about the reliable versions?
More informationCSE 444: Database Internals. Section 9: 2-Phase Commit and Replication
CSE 444: Database Internals Section 9: 2-Phase Commit and Replication 1 Today 2-Phase Commit Replication 2 Two-Phase Commit Protocol (2PC) One coordinator and many subordinates Phase 1: Prepare Phase 2:
More informationDistributed systems. Lecture 6: distributed transactions, elections, consensus and replication. Malte Schwarzkopf
Distributed systems Lecture 6: distributed transactions, elections, consensus and replication Malte Schwarzkopf Last time Saw how we can build ordered multicast Messages between processes in a group Need
More informationParallel Data Types of Parallelism Replication (Multiple copies of the same data) Better throughput for read-only computations Data safety Partitionin
Parallel Data Types of Parallelism Replication (Multiple copies of the same data) Better throughput for read-only computations Data safety Partitioning (Different data at different sites More space Better
More informationCSE 444: Database Internals. Lecture 25 Replication
CSE 444: Database Internals Lecture 25 Replication CSE 444 - Winter 2018 1 Announcements Magda s office hour tomorrow: 1:30pm Lab 6: Milestone today and due next week HW6: Due on Friday Master s students:
More informationConsensus and related problems
Consensus and related problems Today l Consensus l Google s Chubby l Paxos for Chubby Consensus and failures How to make process agree on a value after one or more have proposed what the value should be?
More informationLast time. Distributed systems Lecture 6: Elections, distributed transactions, and replication. DrRobert N. M. Watson
Distributed systems Lecture 6: Elections, distributed transactions, and replication DrRobert N. M. Watson 1 Last time Saw how we can build ordered multicast Messages between processes in a group Need to
More informationDistributed Systems COMP 212. Lecture 19 Othon Michail
Distributed Systems COMP 212 Lecture 19 Othon Michail Fault Tolerance 2/31 What is a Distributed System? 3/31 Distributed vs Single-machine Systems A key difference: partial failures One component fails
More informationCSE 486/586 Distributed Systems
CSE 486/586 Distributed Systems Failure Detectors Slides by: Steve Ko Computer Sciences and Engineering University at Buffalo Administrivia Programming Assignment 2 is out Please continue to monitor Piazza
More informationCS 425 / ECE 428 Distributed Systems Fall 2017
CS 425 / ECE 428 Distributed Systems Fall 2017 Indranil Gupta (Indy) Nov 7, 2017 Lecture 21: Replication Control All slides IG Server-side Focus Concurrency Control = how to coordinate multiple concurrent
More informationDistributed Systems. replication Johan Montelius ID2201. Distributed Systems ID2201
Distributed Systems ID2201 replication Johan Montelius 1 The problem The problem we have: servers might be unavailable The solution: keep duplicates at different servers 2 Building a fault-tolerant service
More informationIntuitive distributed algorithms. with F#
Intuitive distributed algorithms with F# Natallia Dzenisenka Alena Hall @nata_dzen @lenadroid A tour of a variety of intuitivedistributed algorithms used in practical distributed systems. and how to prototype
More informationDistributed Systems (ICE 601) Fault Tolerance
Distributed Systems (ICE 601) Fault Tolerance Dongman Lee ICU Introduction Failure Model Fault Tolerance Models state machine primary-backup Class Overview Introduction Dependability availability reliability
More informationTo do. Consensus and related problems. q Failure. q Raft
Consensus and related problems To do q Failure q Consensus and related problems q Raft Consensus We have seen protocols tailored for individual types of consensus/agreements Which process can enter the
More informationLinearizability CMPT 401. Sequential Consistency. Passive Replication
Linearizability CMPT 401 Thursday, March 31, 2005 The execution of a replicated service (potentially with multiple requests interleaved over multiple servers) is said to be linearizable if: The interleaved
More informationReplication and Consistency
Replication and Consistency Today l Replication l Consistency models l Consistency protocols The value of replication For reliability and availability Avoid problems with disconnection, data corruption,
More informationReplication. Feb 10, 2016 CPSC 416
Replication Feb 10, 2016 CPSC 416 How d we get here? Failures & single systems; fault tolerance techniques added redundancy (ECC memory, RAID, etc.) Conceptually, ECC & RAID both put a master in front
More informationDistributed Systems 11. Consensus. Paul Krzyzanowski
Distributed Systems 11. Consensus Paul Krzyzanowski pxk@cs.rutgers.edu 1 Consensus Goal Allow a group of processes to agree on a result All processes must agree on the same value The value must be one
More informationMutual consistency, what for? Replication. Data replication, consistency models & protocols. Difficulties. Execution Model.
Replication Data replication, consistency models & protocols C. L. Roncancio - S. Drapeau Grenoble INP Ensimag / LIG - Obeo 1 Data and process Focus on data: several physical of one logical object What
More informationDistributed Systems. 09. State Machine Replication & Virtual Synchrony. Paul Krzyzanowski. Rutgers University. Fall Paul Krzyzanowski
Distributed Systems 09. State Machine Replication & Virtual Synchrony Paul Krzyzanowski Rutgers University Fall 2016 1 State machine replication 2 State machine replication We want high scalability and
More informationLecture XIII: Replication-II
Lecture XIII: Replication-II CMPT 401 Summer 2007 Dr. Alexandra Fedorova Outline Google File System A real replicated file system Paxos Harp A consensus algorithm used in real systems A replicated research
More informationIntroduction to Distributed Systems Seif Haridi
Introduction to Distributed Systems Seif Haridi haridi@kth.se What is a distributed system? A set of nodes, connected by a network, which appear to its users as a single coherent system p1 p2. pn send
More informationC 1. Recap. CSE 486/586 Distributed Systems Failure Detectors. Today s Question. Two Different System Models. Why, What, and How.
Recap Best Practices Distributed Systems Failure Detectors Steve Ko Computer Sciences and Engineering University at Buffalo 2 Today s Question Two Different System Models How do we handle failures? Cannot
More informationRecall: Primary-Backup. State machine replication. Extend PB for high availability. Consensus 2. Mechanism: Replicate and separate servers
Replicated s, RAFT COS 8: Distributed Systems Lecture 8 Recall: Primary-Backup Mechanism: Replicate and separate servers Goal #: Provide a highly reliable service Goal #: Servers should behave just like
More informationDistributed Systems COMP 212. Revision 2 Othon Michail
Distributed Systems COMP 212 Revision 2 Othon Michail Synchronisation 2/55 How would Lamport s algorithm synchronise the clocks in the following scenario? 3/55 How would Lamport s algorithm synchronise
More information10. Replication. CSEP 545 Transaction Processing Philip A. Bernstein Sameh Elnikety. Copyright 2012 Philip A. Bernstein
10. Replication CSEP 545 Transaction Processing Philip A. Bernstein Sameh Elnikety Copyright 2012 Philip A. Bernstein 1 Outline 1. Introduction 2. Primary-Copy Replication 3. Multi-Master Replication 4.
More informationCS October 2017
Atomic Transactions Transaction An operation composed of a number of discrete steps. Distributed Systems 11. Distributed Commit Protocols All the steps must be completed for the transaction to be committed.
More informationConsistency in Distributed Systems
Consistency in Distributed Systems Recall the fundamental DS properties DS may be large in scale and widely distributed 1. concurrent execution of components 2. independent failure modes 3. transmission
More informationFailures, Elections, and Raft
Failures, Elections, and Raft CS 8 XI Copyright 06 Thomas W. Doeppner, Rodrigo Fonseca. All rights reserved. Distributed Banking SFO add interest based on current balance PVD deposit $000 CS 8 XI Copyright
More informationSystem Models for Distributed Systems
System Models for Distributed Systems INF5040/9040 Autumn 2015 Lecturer: Amir Taherkordi (ifi/uio) August 31, 2015 Outline 1. Introduction 2. Physical Models 4. Fundamental Models 2 INF5040 1 System Models
More informationAssignment 12: Commit Protocols and Replication Solution
Data Modelling and Databases Exercise dates: May 24 / May 25, 2018 Ce Zhang, Gustavo Alonso Last update: June 04, 2018 Spring Semester 2018 Head TA: Ingo Müller Assignment 12: Commit Protocols and Replication
More informationRecall our 2PC commit problem. Recall our 2PC commit problem. Doing failover correctly isn t easy. Consensus I. FLP Impossibility, Paxos
Consensus I Recall our 2PC commit problem FLP Impossibility, Paxos Client C 1 C à TC: go! COS 418: Distributed Systems Lecture 7 Michael Freedman Bank A B 2 TC à A, B: prepare! 3 A, B à P: yes or no 4
More informationDistributed Systems 8L for Part IB
Distributed Systems 8L for Part IB Handout 3 Dr. Steven Hand 1 Distributed Mutual Exclusion In first part of course, saw need to coordinate concurrent processes / threads In particular considered how to
More informationCoordination 1. To do. Mutual exclusion Election algorithms Next time: Global state. q q q
Coordination 1 To do q q q Mutual exclusion Election algorithms Next time: Global state Coordination and agreement in US Congress 1798-2015 Process coordination How can processes coordinate their action?
More informationFault Tolerance. Distributed Systems. September 2002
Fault Tolerance Distributed Systems September 2002 Basics A component provides services to clients. To provide services, the component may require the services from other components a component may depend
More informationModern Database Concepts
Modern Database Concepts Basic Principles Doc. RNDr. Irena Holubova, Ph.D. holubova@ksi.mff.cuni.cz NoSQL Overview Main objective: to implement a distributed state Different objects stored on different
More informationExam 2 Review. Fall 2011
Exam 2 Review Fall 2011 Question 1 What is a drawback of the token ring election algorithm? Bad question! Token ring mutex vs. Ring election! Ring election: multiple concurrent elections message size grows
More informationData Modeling and Databases Ch 14: Data Replication. Gustavo Alonso, Ce Zhang Systems Group Department of Computer Science ETH Zürich
Data Modeling and Databases Ch 14: Data Replication Gustavo Alonso, Ce Zhang Systems Group Department of Computer Science ETH Zürich Database Replication What is database replication The advantages of
More informationDistributed Systems. Characteristics of Distributed Systems. Lecture Notes 1 Basic Concepts. Operating Systems. Anand Tripathi
1 Lecture Notes 1 Basic Concepts Anand Tripathi CSci 8980 Operating Systems Anand Tripathi CSci 8980 1 Distributed Systems A set of computers (hosts or nodes) connected through a communication network.
More informationDistributed Systems. Characteristics of Distributed Systems. Characteristics of Distributed Systems. Goals in Distributed System Designs
1 Anand Tripathi CSci 8980 Operating Systems Lecture Notes 1 Basic Concepts Distributed Systems A set of computers (hosts or nodes) connected through a communication network. Nodes may have different speeds
More informationConsistency and Replication. Some slides are from Prof. Jalal Y. Kawash at Univ. of Calgary
Consistency and Replication Some slides are from Prof. Jalal Y. Kawash at Univ. of Calgary Reasons for Replication Reliability/Availability : Mask failures Mask corrupted data Performance: Scalability
More information11. Replication. Motivation
11. Replication Seite 1 11. Replication Motivation Reliable and high-performance computation on a single instance of a data object is prone to failure. Replicate data to overcome single points of failure
More informationFault Tolerance Part I. CS403/534 Distributed Systems Erkay Savas Sabanci University
Fault Tolerance Part I CS403/534 Distributed Systems Erkay Savas Sabanci University 1 Overview Basic concepts Process resilience Reliable client-server communication Reliable group communication Distributed
More informationDatabase Replication: A Tutorial
Chapter 12 Database Replication: A Tutorial Bettina Kemme, Ricardo Jiménez-Peris, Marta Patiño-Martínez, and Gustavo Alonso Abstract This chapter provides an in-depth introduction to database replication,
More informationProseminar Distributed Systems Summer Semester Paxos algorithm. Stefan Resmerita
Proseminar Distributed Systems Summer Semester 2016 Paxos algorithm stefan.resmerita@cs.uni-salzburg.at The Paxos algorithm Family of protocols for reaching consensus among distributed agents Agents may
More informationModule 8 - Fault Tolerance
Module 8 - Fault Tolerance Dependability Reliability A measure of success with which a system conforms to some authoritative specification of its behavior. Probability that the system has not experienced
More informationReplication and Consistency. Fall 2010 Jussi Kangasharju
Replication and Consistency Fall 2010 Jussi Kangasharju Chapter Outline Replication Consistency models Distribution protocols Consistency protocols 2 Data Replication user B user C user A object object
More informationExtend PB for high availability. PB high availability via 2PC. Recall: Primary-Backup. Putting it all together for SMR:
Putting it all together for SMR: Two-Phase Commit, Leader Election RAFT COS 8: Distributed Systems Lecture Recall: Primary-Backup Mechanism: Replicate and separate servers Goal #: Provide a highly reliable
More informationImportant Lessons. Today's Lecture. Two Views of Distributed Systems
Important Lessons Replication good for performance/ reliability Key challenge keeping replicas up-to-date Wide range of consistency models Will see more next lecture Range of correctness properties L-10
More informationPrimary-Backup Replication
Primary-Backup Replication CS 240: Computing Systems and Concurrency Lecture 7 Marco Canini Credits: Michael Freedman and Kyle Jamieson developed much of the original material. Simplified Fault Tolerance
More informationModule 8 Fault Tolerance CS655! 8-1!
Module 8 Fault Tolerance CS655! 8-1! Module 8 - Fault Tolerance CS655! 8-2! Dependability Reliability! A measure of success with which a system conforms to some authoritative specification of its behavior.!
More informationC 1. Today s Question. CSE 486/586 Distributed Systems Failure Detectors. Two Different System Models. Failure Model. Why, What, and How
CSE 486/586 Distributed Systems Failure Detectors Today s Question I have a feeling that something went wrong Steve Ko Computer Sciences and Engineering University at Buffalo zzz You ll learn new terminologies,
More informationPractical Byzantine Fault Tolerance. Castro and Liskov SOSP 99
Practical Byzantine Fault Tolerance Castro and Liskov SOSP 99 Why this paper? Kind of incredible that it s even possible Let alone a practical NFS implementation with it So far we ve only considered fail-stop
More informationDistributed Systems. 10. Consensus: Paxos. Paul Krzyzanowski. Rutgers University. Fall 2017
Distributed Systems 10. Consensus: Paxos Paul Krzyzanowski Rutgers University Fall 2017 1 Consensus Goal Allow a group of processes to agree on a result All processes must agree on the same value The value
More informationChapter 4: Distributed Systems: Replication and Consistency. Fall 2013 Jussi Kangasharju
Chapter 4: Distributed Systems: Replication and Consistency Fall 2013 Jussi Kangasharju Chapter Outline n Replication n Consistency models n Distribution protocols n Consistency protocols 2 Data Replication
More informationAgreement and Consensus. SWE 622, Spring 2017 Distributed Software Engineering
Agreement and Consensus SWE 622, Spring 2017 Distributed Software Engineering Today General agreement problems Fault tolerance limitations of 2PC 3PC Paxos + ZooKeeper 2 Midterm Recap 200 GMU SWE 622 Midterm
More informationRecap. CSE 486/586 Distributed Systems Paxos. Paxos. Brief History. Brief History. Brief History C 1
Recap Distributed Systems Steve Ko Computer Sciences and Engineering University at Buffalo Facebook photo storage CDN (hot), Haystack (warm), & f4 (very warm) Haystack RAID-6, per stripe: 10 data disks,
More informationConsensus, impossibility results and Paxos. Ken Birman
Consensus, impossibility results and Paxos Ken Birman Consensus a classic problem Consensus abstraction underlies many distributed systems and protocols N processes They start execution with inputs {0,1}
More informationConsensus a classic problem. Consensus, impossibility results and Paxos. Distributed Consensus. Asynchronous networks.
Consensus, impossibility results and Paxos Ken Birman Consensus a classic problem Consensus abstraction underlies many distributed systems and protocols N processes They start execution with inputs {0,1}
More informationAtomicity. Bailu Ding. Oct 18, Bailu Ding Atomicity Oct 18, / 38
Atomicity Bailu Ding Oct 18, 2012 Bailu Ding Atomicity Oct 18, 2012 1 / 38 Outline 1 Introduction 2 State Machine 3 Sinfonia 4 Dangers of Replication Bailu Ding Atomicity Oct 18, 2012 2 / 38 Introduction
More informationImportant Lessons. A Distributed Algorithm (2) Today's Lecture - Replication
Important Lessons Lamport & vector clocks both give a logical timestamps Total ordering vs. causal ordering Other issues in coordinating node activities Exclusive access to resources/data Choosing a single
More informationZooKeeper & Curator. CS 475, Spring 2018 Concurrent & Distributed Systems
ZooKeeper & Curator CS 475, Spring 2018 Concurrent & Distributed Systems Review: Agreement In distributed systems, we have multiple nodes that need to all agree that some object has some state Examples:
More information416 practice questions (PQs)
416 practice questions (PQs) 1. Goal: give you some material to study for the final exam and to help you to more actively engage with the material we cover in class. 2. Format: questions that are in scope
More informationBuilding Consistent Transactions with Inconsistent Replication
Building Consistent Transactions with Inconsistent Replication Irene Zhang, Naveen Kr. Sharma, Adriana Szekeres, Arvind Krishnamurthy, Dan R. K. Ports University of Washington Distributed storage systems
More informationFAULT TOLERANCE. Fault Tolerant Systems. Faults Faults (cont d)
Distributed Systems Fö 9/10-1 Distributed Systems Fö 9/10-2 FAULT TOLERANCE 1. Fault Tolerant Systems 2. Faults and Fault Models. Redundancy 4. Time Redundancy and Backward Recovery. Hardware Redundancy
More information10. Replication. Motivation
10. Replication Page 1 10. Replication Motivation Reliable and high-performance computation on a single instance of a data object is prone to failure. Replicate data to overcome single points of failure
More informationByzantine Failures. Nikola Knezevic. knl
Byzantine Failures Nikola Knezevic knl Different Types of Failures Crash / Fail-stop Send Omissions Receive Omissions General Omission Arbitrary failures, authenticated messages Arbitrary failures Arbitrary
More informationByzantine Fault Tolerance
Byzantine Fault Tolerance CS6450: Distributed Systems Lecture 10 Ryan Stutsman Material taken/derived from Princeton COS-418 materials created by Michael Freedman and Kyle Jamieson at Princeton University.
More informationDistributed Operating Systems
2 Distributed Operating Systems System Models, Processor Allocation, Distributed Scheduling, and Fault Tolerance Steve Goddard goddard@cse.unl.edu http://www.cse.unl.edu/~goddard/courses/csce855 System
More informationSemi-Passive Replication in the Presence of Byzantine Faults
Semi-Passive Replication in the Presence of Byzantine Faults HariGovind V. Ramasamy Adnan Agbaria William H. Sanders University of Illinois at Urbana-Champaign 1308 W. Main Street, Urbana IL 61801, USA
More informationCS 138: Practical Byzantine Consensus. CS 138 XX 1 Copyright 2017 Thomas W. Doeppner. All rights reserved.
CS 138: Practical Byzantine Consensus CS 138 XX 1 Copyright 2017 Thomas W. Doeppner. All rights reserved. Scenario Asynchronous system Signed messages s are state machines It has to be practical CS 138
More informationDistributed Systems Principles and Paradigms. Chapter 08: Fault Tolerance
Distributed Systems Principles and Paradigms Maarten van Steen VU Amsterdam, Dept. Computer Science Room R4.20, steen@cs.vu.nl Chapter 08: Fault Tolerance Version: December 2, 2010 2 / 65 Contents Chapter
More informationCMU SCS CMU SCS Who: What: When: Where: Why: CMU SCS
Carnegie Mellon Univ. Dept. of Computer Science 15-415/615 - DB s C. Faloutsos A. Pavlo Lecture#23: Distributed Database Systems (R&G ch. 22) Administrivia Final Exam Who: You What: R&G Chapters 15-22
More informationDistributed Systems (5DV147)
Distributed Systems (5DV147) Replication and consistency Fall 2013 1 Replication 2 What is replication? Introduction Make different copies of data ensuring that all copies are identical Immutable data
More informationFault Tolerance via the State Machine Replication Approach. Favian Contreras
Fault Tolerance via the State Machine Replication Approach Favian Contreras Implementing Fault-Tolerant Services Using the State Machine Approach: A Tutorial Written by Fred Schneider Why a Tutorial? The
More informationAgreement in Distributed Systems CS 188 Distributed Systems February 19, 2015
Agreement in Distributed Systems CS 188 Distributed Systems February 19, 2015 Page 1 Introduction We frequently want to get a set of nodes in a distributed system to agree Commitment protocols and mutual
More informationCSE 5306 Distributed Systems
CSE 5306 Distributed Systems Fault Tolerance Jia Rao http://ranger.uta.edu/~jrao/ 1 Failure in Distributed Systems Partial failure Happens when one component of a distributed system fails Often leaves
More informationLast Class:Consistency Semantics. Today: More on Consistency
Last Class:Consistency Semantics Consistency models Data-centric consistency models Client-centric consistency models Eventual Consistency and epidemic protocols Lecture 16, page 1 Today: More on Consistency
More informationToday: Fault Tolerance
Today: Fault Tolerance Agreement in presence of faults Two army problem Byzantine generals problem Reliable communication Distributed commit Two phase commit Three phase commit Paxos Failure recovery Checkpointing
More informationCSE 5306 Distributed Systems. Fault Tolerance
CSE 5306 Distributed Systems Fault Tolerance 1 Failure in Distributed Systems Partial failure happens when one component of a distributed system fails often leaves other components unaffected A failure
More informationNoSQL systems: sharding, replication and consistency. Riccardo Torlone Università Roma Tre
NoSQL systems: sharding, replication and consistency Riccardo Torlone Università Roma Tre Data distribution NoSQL systems: data distributed over large clusters Aggregate is a natural unit to use for data
More informationDep. Systems Requirements
Dependable Systems Dep. Systems Requirements Availability the system is ready to be used immediately. A(t) = probability system is available for use at time t MTTF/(MTTF+MTTR) If MTTR can be kept small
More informationClock Synchronization. Synchronization. Clock Synchronization Algorithms. Physical Clock Synchronization. Tanenbaum Chapter 6 plus additional papers
Clock Synchronization Synchronization Tanenbaum Chapter 6 plus additional papers Fig 6-1. In a distributed system, each machine has its own clock. When this is the case, an event that occurred after another
More informationFailure Tolerance. Distributed Systems Santa Clara University
Failure Tolerance Distributed Systems Santa Clara University Distributed Checkpointing Distributed Checkpointing Capture the global state of a distributed system Chandy and Lamport: Distributed snapshot
More informationDistributed Systems. Fault Tolerance. Paul Krzyzanowski
Distributed Systems Fault Tolerance Paul Krzyzanowski Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution 2.5 License. Faults Deviation from expected
More informationDistributed Systems. Multicast and Agreement
Distributed Systems Multicast and Agreement Björn Franke University of Edinburgh 2015/2016 Multicast Send message to multiple nodes A node can join a multicast group, and receives all messages sent to
More informationCprE Fault Tolerance. Dr. Yong Guan. Department of Electrical and Computer Engineering & Information Assurance Center Iowa State University
Fault Tolerance Dr. Yong Guan Department of Electrical and Computer Engineering & Information Assurance Center Iowa State University Outline for Today s Talk Basic Concepts Process Resilience Reliable
More informationAvailability versus consistency. Eventual Consistency: Bayou. Eventual consistency. Bayou: A Weakly Connected Replicated Storage System
Eventual Consistency: Bayou Availability versus consistency Totally-Ordered Multicast kept replicas consistent but had single points of failure Not available under failures COS 418: Distributed Systems
More informationFault Tolerance. Basic Concepts
COP 6611 Advanced Operating System Fault Tolerance Chi Zhang czhang@cs.fiu.edu Dependability Includes Availability Run time / total time Basic Concepts Reliability The length of uninterrupted run time
More informationPractical Byzantine Fault
Practical Byzantine Fault Tolerance Practical Byzantine Fault Tolerance Castro and Liskov, OSDI 1999 Nathan Baker, presenting on 23 September 2005 What is a Byzantine fault? Rationale for Byzantine Fault
More informationCS5412: CONSENSUS AND THE FLP IMPOSSIBILITY RESULT
1 CS5412: CONSENSUS AND THE FLP IMPOSSIBILITY RESULT Lecture XII Ken Birman Generalizing Ron and Hermione s challenge 2 Recall from last time: Ron and Hermione had difficulty agreeing where to meet for
More informationMODELS OF DISTRIBUTED SYSTEMS
Distributed Systems Fö 2/3-1 Distributed Systems Fö 2/3-2 MODELS OF DISTRIBUTED SYSTEMS Basic Elements 1. Architectural Models 2. Interaction Models Resources in a distributed system are shared between
More informationAgenda. What is Replication?
!"#$%% "#&'( Agenda What is Replication? Why Replicate? Approaches to Replication Master/Slave Disconnected Repositories (Git / Bitkeeper / Mercurial / Bazaar) Active/Active Master/Slave vs Active/Active
More informationTradeoffs in Byzantine-Fault-Tolerant State-Machine-Replication Protocol Design
Tradeoffs in Byzantine-Fault-Tolerant State-Machine-Replication Protocol Design Michael G. Merideth March 2008 CMU-ISR-08-110 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213
More informationProcess Synchroniztion Mutual Exclusion & Election Algorithms
Process Synchroniztion Mutual Exclusion & Election Algorithms Paul Krzyzanowski Rutgers University November 2, 2017 1 Introduction Process synchronization is the set of techniques that are used to coordinate
More informationSynchronization. Clock Synchronization
Synchronization Clock Synchronization Logical clocks Global state Election algorithms Mutual exclusion Distributed transactions 1 Clock Synchronization Time is counted based on tick Time judged by query
More informationTransactions. CS 475, Spring 2018 Concurrent & Distributed Systems
Transactions CS 475, Spring 2018 Concurrent & Distributed Systems Review: Transactions boolean transfermoney(person from, Person to, float amount){ if(from.balance >= amount) { from.balance = from.balance
More information