Ovid A Software-Defined Distributed Systems Framework. Deniz Altinbuken, Robbert van Renesse Cornell University
|
|
- Loreen Poole
- 5 years ago
- Views:
Transcription
1 Ovid A Software-Defined Distributed Systems Framework Deniz Altinbuken, Robbert van Renesse Cornell University
2 Ovid Build distributed systems that are easy to evolve easy to reason about easy to compose
3 Approach Create a theoretical model using abstractions Deploy the distributed system using this model
4 Approach Create a theoretical model using abstractions Deploy the distributed system using this model
5
6 agent agent
7 not fault tolerant
8 transform agent to be fault tolerant replicate(,3) =
9 transform agent to be fault tolerant replicate(,3) = *3
10 minority of replicas can fail crash fault tolerant asynchronous environment linearizable }Paxos consensus protocol replicate(,3) = R*3 + *3
11 minority of replicas can fail crash fault tolerant asynchronous environment linearizable replicate(,3) = + R*3 + *3
12 How do we know that a transformation did not break an agent?
13 How do we know that a transformation did not break an agent? virtual agent physical agent physical agent physical agent physical agent physical agent physical agent physical agent
14 Use refinement mappings to prove equivalence virtual agent physical agent physical agent physical agent physical agent physical agent physical agent physical agent
15
16 Use path names to represent transformations /egress1 / /ingress /egress2 / /egress3 / LATT Logical Agent Transformation Tree
17 Supported transformations replication byzantine resistance batching sharding encryption/decryption compression load-balancing deduplication
18 Approach Create a theoretical model using abstractions Deploy the distributed system using this model
19 Deployment Create a configuration from the model Deploy processes on boxes depending on the configuration
20 Deployment controller Create a configuration from the model Deploy processes on boxes depending on the configuration
21 Deployment controller Machine Configuration cpp cpp replica.cpp replica.cpp replica.cpp cpp cpp cpp
22 Deployment controller Machine Configuration cpp cpp replica.cpp replica.cpp replica.cpp cpp cpp cpp
23 Dynamic Routing controller Create routing tables Route messages to destination depending on the model
24 Dynamic Routing controller Routing Configuration,,
25 Dynamic Routing msg: dest payload controller
26 Dynamic Routing msg: dest payload put(1, 6) controller
27 Dynamic Routing msg: dest payload put(1, 6) controller send msg(dest,payload)
28 Dynamic Routing msg: dest payload put(1, 6) controller send msg(dest,payload) look for dest in routing table
29 Dynamic Routing msg: dest payload put(1, 6) controller send msg(dest,payload) look for dest in routing table dest not present
30 Dynamic Routing msg: dest payload put(1, 6) controller lookup(ds) controller send msg(dest,payload) look for dest in routing table dest not present send lookup message to controller
31 Dynamic Routing msg: dest payload put(1, 6) controller lookup(ds) controller send msg(dest,payload) look for dest in routing table dest not present send lookup message to controller
32 Dynamic Routing msg: dest payload put(1, 6) controller lookup(ds) controller Routing Configuration,, send msg(dest,payload) look for dest in routing table dest not present send lookup message to controller
33 Dynamic Routing msg: dest payload put(1, 6) ds, controller send msg(dest,payload) look for dest in routing table dest not present send lookup message to controller get route mapping
34 Dynamic Routing msg: dest payload put(1, 6) ds, controller send msg(dest,payload) look for dest in routing table dest not present send lookup message to controller get route mapping
35 Dynamic Routing msg: dest payload put(1, 6) ds: controller send msg(dest,payload) look for dest in routing table dest not present send lookup message to controller get route mapping update routing table
36 Dynamic Routing msg: dest payload put(1, 6) controller send msg(dest,payload) look for dest in routing table dest not present send lookup message to controller get route mapping update routing table send msg(dest,payload)
37 Dynamic Routing msg: dest payload put(1, 6) controller send msg(dest,payload) look for dest in routing table dest not present send lookup message to controller get route mapping update routing table send msg(dest,payload)
38 Dynamic Routing msg: dest payload put(1, 6) controller send msg(dest,payload) look for dest in routing table dest not present send lookup message to controller get route mapping update routing table send msg(dest,payload)
39 Conclusion Ovid introduces new abstractions and a new way of modeling distributed systems. Ovid can create distributed systems that can be reconfigured and deployed on the fly. Ovid makes building, running, maintaining and evolving distributed systems an easy task.
There Is More Consensus in Egalitarian Parliaments
There Is More Consensus in Egalitarian Parliaments Iulian Moraru, David Andersen, Michael Kaminsky Carnegie Mellon University Intel Labs Fault tolerance Redundancy State Machine Replication 3 State Machine
More informationViewstamped Replication to Practical Byzantine Fault Tolerance. Pradipta De
Viewstamped Replication to Practical Byzantine Fault Tolerance Pradipta De pradipta.de@sunykorea.ac.kr ViewStamped Replication: Basics What does VR solve? VR supports replicated service Abstraction is
More informationFault Tolerance via the State Machine Replication Approach. Favian Contreras
Fault Tolerance via the State Machine Replication Approach Favian Contreras Implementing Fault-Tolerant Services Using the State Machine Approach: A Tutorial Written by Fred Schneider Why a Tutorial? The
More informationDistributed Consensus Paxos
Distributed Consensus Paxos Ethan Cecchetti October 18, 2016 CS6410 Some structure taken from Robert Burgess s 2009 slides on this topic State Machine Replication (SMR) View a server as a state machine.
More informationDistributed Systems. Day 9: Replication [Part 1]
Distributed Systems Day 9: Replication [Part 1] Hash table k 0 v 0 k 1 v 1 k 2 v 2 k 3 v 3 ll Facebook Data Does your client know about all of F s servers? Security issues? Performance issues? How do clients
More informationEnhancing Throughput of
Enhancing Throughput of NCA 2017 Zhongmiao Li, Peter Van Roy and Paolo Romano Enhancing Throughput of Partially Replicated State Machines via NCA 2017 Zhongmiao Li, Peter Van Roy and Paolo Romano Enhancing
More informationBeyond FLP. Acknowledgement for presentation material. Chapter 8: Distributed Systems Principles and Paradigms: Tanenbaum and Van Steen
Beyond FLP Acknowledgement for presentation material Chapter 8: Distributed Systems Principles and Paradigms: Tanenbaum and Van Steen Paper trail blog: http://the-paper-trail.org/blog/consensus-protocols-paxos/
More informationCorrect-by-Construction Attack- Tolerant Systems. Robert Constable Mark Bickford Robbert van Renesse Cornell University
Correct-by-Construction Attack- Tolerant Systems Robert Constable Mark Bickford Robbert van Renesse Cornell University Definition Attack-tolerant distributed systems change their protocols on-the-fly in
More informationDistributed Consensus: Making Impossible Possible
Distributed Consensus: Making Impossible Possible QCon London Tuesday 29/3/2016 Heidi Howard PhD Student @ University of Cambridge heidi.howard@cl.cam.ac.uk @heidiann360 What is Consensus? The process
More informationIntroduction to Distributed Systems Seif Haridi
Introduction to Distributed Systems Seif Haridi haridi@kth.se What is a distributed system? A set of nodes, connected by a network, which appear to its users as a single coherent system p1 p2. pn send
More informationDistributed Consensus: Making Impossible Possible
Distributed Consensus: Making Impossible Possible Heidi Howard PhD Student @ University of Cambridge heidi.howard@cl.cam.ac.uk @heidiann360 hh360.user.srcf.net Sometimes inconsistency is not an option
More informationReplication in Distributed Systems
Replication in Distributed Systems Replication Basics Multiple copies of data kept in different nodes A set of replicas holding copies of a data Nodes can be physically very close or distributed all over
More informationDistributed Systems Final Exam
15-440 Distributed Systems Final Exam Name: Andrew: ID December 12, 2011 Please write your name and Andrew ID above before starting this exam. This exam has 14 pages, including this title page. Please
More informationCS 138: Practical Byzantine Consensus. CS 138 XX 1 Copyright 2017 Thomas W. Doeppner. All rights reserved.
CS 138: Practical Byzantine Consensus CS 138 XX 1 Copyright 2017 Thomas W. Doeppner. All rights reserved. Scenario Asynchronous system Signed messages s are state machines It has to be practical CS 138
More informationBuilding Scalable Stateful Services. Craft Conf 2016
Building Scalable Stateful s Craft Conf 2016 Caitie McCaffrey Distributed Systems Engineer @caitie caitiem.com Stateless s Stateless s Stateless s Stateless s Stateless s Stateless s Stateless s Stateless
More informationAGREEMENT PROTOCOLS. Paxos -a family of protocols for solving consensus
AGREEMENT PROTOCOLS Paxos -a family of protocols for solving consensus OUTLINE History of the Paxos algorithm Paxos Algorithm Family Implementation in existing systems References HISTORY OF THE PAXOS ALGORITHM
More informationReplications and Consensus
CPSC 426/526 Replications and Consensus Ennan Zhai Computer Science Department Yale University Recall: Lec-8 and 9 In the lec-8 and 9, we learned: - Cloud storage and data processing - File system: Google
More informationErasure Coding in Object Stores: Challenges and Opportunities
Erasure Coding in Object Stores: Challenges and Opportunities Lewis Tseng Boston College July 2018, PODC Acknowledgements Nancy Lynch Muriel Medard Kishori Konwar Prakash Narayana Moorthy Viveck R. Cadambe
More informationBasic vs. Reliable Multicast
Basic vs. Reliable Multicast Basic multicast does not consider process crashes. Reliable multicast does. So far, we considered the basic versions of ordered multicasts. What about the reliable versions?
More informationTransactions Between Distributed Ledgers
Transactions Between Distributed Ledgers Ivan Klianev Transactum Pty Ltd High Performance Transaction Systems Asilomar, California, 8-11 October 2017 The Time for Distributed Transactions Has Come Thanks
More informationLinearizability CMPT 401. Sequential Consistency. Passive Replication
Linearizability CMPT 401 Thursday, March 31, 2005 The execution of a replicated service (potentially with multiple requests interleaved over multiple servers) is said to be linearizable if: The interleaved
More informationDistributed Systems. replication Johan Montelius ID2201. Distributed Systems ID2201
Distributed Systems ID2201 replication Johan Montelius 1 The problem The problem we have: servers might be unavailable The solution: keep duplicates at different servers 2 Building a fault-tolerant service
More informationExploiting Commutativity For Practical Fast Replication. Seo Jin Park and John Ousterhout
Exploiting Commutativity For Practical Fast Replication Seo Jin Park and John Ousterhout Overview Problem: replication adds latency and throughput overheads CURP: Consistent Unordered Replication Protocol
More informationPARALLEL CONSENSUS PROTOCOL
CANOPUS: A SCALABLE AND MASSIVELY PARALLEL CONSENSUS PROTOCOL Bernard Wong CoNEXT 2017 Joint work with Sajjad Rizvi and Srinivasan Keshav CONSENSUS PROBLEM Agreement between a set of nodes in the presence
More informationBlockFin A Fork-Tolerant, Leaderless Consensus Protocol April
BlockFin A Fork-Tolerant, Leaderless Consensus Protocol April 2018 @storecoin What are the most desirable features in a blockchain? Scalability (throughput) and decentralization (censorship resistance),
More informationPractical Byzantine Fault Tolerance Consensus and A Simple Distributed Ledger Application Hao Xu Muyun Chen Xin Li
Practical Byzantine Fault Tolerance Consensus and A Simple Distributed Ledger Application Hao Xu Muyun Chen Xin Li Abstract Along with cryptocurrencies become a great success known to the world, how to
More informationVive La Différence: Paxos vs. Viewstamped Replication vs. Zab
IEEE TRANSACTIONS ON DEPENDABLE AND SECURE COMPUTING, MANUSCRIPT ID 1 Vive La Différence: Paxos vs. Viewstamped Replication vs. Zab Robbert van Renesse, Nicolas Schiper, Fred B. Schneider, Fellow, IEEE
More informationTolerant Systems. Mark Bickford Robbert van Renesse
Correct by Construction Attack Tolerant Systems Robert Constable Mark Bickford Robbert van Renesse Cornell lluniversity it and ATC NY DARPA Kickoff Meeting November 3, 2010 Talk Goals Review our experience
More informationLow-Latency Multi-Datacenter Databases using Replicated Commit
Low-Latency Multi-Datacenter Databases using Replicated Commit Hatem Mahmoud, Faisal Nawab, Alexander Pucher, Divyakant Agrawal, Amr El Abbadi UCSB Presented by Ashutosh Dhekne Main Contributions Reduce
More informationGroup Replication: A Journey to the Group Communication Core. Alfranio Correia Principal Software Engineer
Group Replication: A Journey to the Group Communication Core Alfranio Correia (alfranio.correia@oracle.com) Principal Software Engineer 4th of February Copyright 7, Oracle and/or its affiliates. All rights
More informationProseminar Distributed Systems Summer Semester Paxos algorithm. Stefan Resmerita
Proseminar Distributed Systems Summer Semester 2016 Paxos algorithm stefan.resmerita@cs.uni-salzburg.at The Paxos algorithm Family of protocols for reaching consensus among distributed agents Agents may
More informationSystem models for distributed systems
System models for distributed systems INF5040/9040 autumn 2010 lecturer: Frank Eliassen INF5040 H2010, Frank Eliassen 1 System models Purpose illustrate/describe common properties and design choices for
More informationFundamentals Large-Scale Distributed System Design. (a.k.a. Distributed Systems 1)
Fundamentals Large-Scale Distributed System Design (a.k.a. Distributed Systems 1) https://columbia.github.io/ds1-class/ 1 Interested in... 1. scalable web services? 2. big data? 3. and the large-scale
More informationLarge-Scale Key-Value Stores Eventual Consistency Marco Serafini
Large-Scale Key-Value Stores Eventual Consistency Marco Serafini COMPSCI 590S Lecture 13 Goals of Key-Value Stores Export simple API put(key, value) get(key) Simpler and faster than a DBMS Less complexity,
More informationTyphoon: An SDN Enhanced Real-Time Big Data Streaming Framework
Typhoon: An SDN Enhanced Real-Time Big Data Streaming Framework Junguk Cho, Hyunseok Chang, Sarit Mukherjee, T.V. Lakshman, and Jacobus Van der Merwe 1 Big Data Era Big data analysis is increasingly common
More informationIronFleet. Dmitry Bondarenko, Yixuan Chen
IronFleet Dmitry Bondarenko, Yixuan Chen A short survey How many people have the confidence that your Paxos implementation has no bug? How many people have proved that the implementation is correct? Why
More informationDistributed Systems 24. Fault Tolerance
Distributed Systems 24. Fault Tolerance Paul Krzyzanowski pxk@cs.rutgers.edu 1 Faults Deviation from expected behavior Due to a variety of factors: Hardware failure Software bugs Operator errors Network
More informationProphecy: Using History for High Throughput Fault Tolerance
Prophecy: Using History for High Throughput Fault Tolerance Siddhartha Sen Joint work with Wyatt Lloyd and Mike Freedman Princeton University Non crash failures happen Non crash failures happen Model as
More informationZHT: Const Eventual Consistency Support For ZHT. Group Member: Shukun Xie Ran Xin
ZHT: Const Eventual Consistency Support For ZHT Group Member: Shukun Xie Ran Xin Outline Problem Description Project Overview Solution Maintains Replica List for Each Server Operation without Primary Server
More informationReplicated State Machine in Wide-area Networks
Replicated State Machine in Wide-area Networks Yanhua Mao CSE223A WI09 1 Building replicated state machine with consensus General approach to replicate stateful deterministic services Provide strong consistency
More informationVive La Difference: Paxos vs. Viewstamped Replication vs. Zab
472 IEEE TRANSACTIONS ON DEPENDABLE AND SECURE COMPUTING, VOL. 12, NO. 4, JULY/AUGUST 2015 Vive La Difference: Paxos vs. Viewstamped Replication vs. Zab Robbert van Renesse, Nicolas Schiper, and Fred B.
More informationDistributed Systems. Aleardo Manacero Jr.
Distributed Systems Aleardo Manacero Jr. Replication - part 1 Introduction Using multiple servers to attend client requests allow for a better performance in the system Unfortunately, as shown in the study
More informationByzantine Failures. Nikola Knezevic. knl
Byzantine Failures Nikola Knezevic knl Different Types of Failures Crash / Fail-stop Send Omissions Receive Omissions General Omission Arbitrary failures, authenticated messages Arbitrary failures Arbitrary
More informationBYZANTINE FAULT TOLERANT SOFTWARE- DEFINED NETWORKING (SDN) CONTROLLERS
BYZANTINE FAULT TOLERANT SOFTWARE- DEFINED NETWORKING (SDN) CONTROLLERS KARIM ELDEFRAWY* AND TYLER KACZMAREK** * INFORMATION AND SYSTEMS SCIENCES LAB (ISSL), HRL LABORATORIES ** UNIVERSITY OF CALIFORNIA
More informationDistributed systems. Lecture 6: distributed transactions, elections, consensus and replication. Malte Schwarzkopf
Distributed systems Lecture 6: distributed transactions, elections, consensus and replication Malte Schwarzkopf Last time Saw how we can build ordered multicast Messages between processes in a group Need
More informationCS 425 / ECE 428 Distributed Systems Fall 2017
CS 425 / ECE 428 Distributed Systems Fall 2017 Indranil Gupta (Indy) Nov 7, 2017 Lecture 21: Replication Control All slides IG Server-side Focus Concurrency Control = how to coordinate multiple concurrent
More informationPBFT: A Byzantine Renaissance. The Setup. What could possibly go wrong? The General Idea. Practical Byzantine Fault-Tolerance (CL99, CL00)
PBFT: A Byzantine Renaissance Practical Byzantine Fault-Tolerance (CL99, CL00) first to be safe in asynchronous systems live under weak synchrony assumptions -Byzantine Paxos! The Setup Crypto System Model
More informationConsensus Problem. Pradipta De
Consensus Problem Slides are based on the book chapter from Distributed Computing: Principles, Paradigms and Algorithms (Chapter 14) by Kshemkalyani and Singhal Pradipta De pradipta.de@sunykorea.ac.kr
More informationiscsi Target Usage Guide December 15, 2017
December 15, 2017 1 Table of Contents 1. Native VMware Availability Options for vsan 1.1.Native VMware Availability Options for vsan 1.2.Application Clustering Solutions 1.3.Third party solutions 2. Security
More informationScalable State-Machine Replication
Scalable State-Machine Replication Carlos Eduardo Bezerra, Fernando Pedone, Robbert van Renesse University of Lugano, Switzerland Cornell University, USA Universidade Federal do Rio Grande do Sul, Brazil
More informationMENCIUS: BUILDING EFFICIENT
MENCIUS: BUILDING EFFICIENT STATE MACHINE FOR WANS By: Yanhua Mao Flavio P. Junqueira Keith Marzullo Fabian Fuxa, Chun-Yu Hsiung November 14, 2018 AGENDA 1. Motivation 2. Breakthrough 3. Rules of Mencius
More informationPublish & Subscribe Larry Rudolph May 3, 2006 SMA 5508 & MIT Pervasive Computing MIT SMA 5508 Spring 2006 Larry Rudolph
Publish & Subscribe Larry May 3, 2006 SMA 5508 & MIT 6.883 1 Agents An agent is an autonomous program. It executes code and can communicate with other agents. All the components in a pervasive computing
More informationA Byzantine Fault-Tolerant Ordering Service for the Hyperledger Fabric Blockchain Platform
A Byzantine Fault-Tolerant Ordering Service for the Hyperledger Fabric Blockchain Platform João Sousa, Alysson Bessani, Marko Vukolić* Faculdade de Ciências, Universidade de Lisboa *IBM Research Zurich
More informationEECS 498 Introduction to Distributed Systems
EECS 498 Introduction to Distributed Systems Fall 2017 Harsha V. Madhyastha Implementing RSMs Logical clock based ordering of requests Cannot serve requests if any one replica is down Primary-backup replication
More informationLarge-Scale Data Stores and Probabilistic Protocols
Distributed Systems 600.437 Large-Scale Data Stores & Probabilistic Protocols Department of Computer Science The Johns Hopkins University 1 Large-Scale Data Stores and Probabilistic Protocols Lecture 11
More informationVerdi: A Framework for Implementing and Formally Verifying Distributed Systems
Verdi: A Framework for Implementing and Formally Verifying Distributed Systems Key-value store VST James R. Wilcox, Doug Woos, Pavel Panchekha, Zach Tatlock, Xi Wang, Michael D. Ernst, Thomas Anderson
More informationSurviving congestion in geo-distributed storage systems
Surviving congestion in geo-distributed storage systems Brian Cho Marcos K. Aguilera University of Illinois at Urbana-Champaign Microsoft Research Silicon Valley Geo-distributed data centers Web applications
More informationLeveraging Sharding in the Design of Scalable Replication Protocols
Leveraging Sharding in the Design of Scalable Replication Protocols Hussam Abu-Libdeh Dept. of Computer Science Cornell University hussam@cs.cornell.edu Robbert van Renesse Dept. of Computer Science Cornell
More informationExam Distributed Systems
Exam Distributed Systems 5 February 2010, 9:00am 12:00pm Part 2 Prof. R. Wattenhofer Family Name, First Name:..................................................... ETH Student ID Number:.....................................................
More informationChapter 10 DISTRIBUTED OBJECT-BASED SYSTEMS
DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN Chapter 10 DISTRIBUTED OBJECT-BASED SYSTEMS Distributed Objects Figure 10-1. Common organization of a remote
More informationDistributed Algorithms Reliable Broadcast
Distributed Algorithms Reliable Broadcast Alberto Montresor University of Trento, Italy 2016/04/26 This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. Contents
More informationBuilding Consistent Transactions with Inconsistent Replication
DB Reading Group Fall 2015 slides by Dana Van Aken Building Consistent Transactions with Inconsistent Replication Irene Zhang, Naveen Kr. Sharma, Adriana Szekeres, Arvind Krishnamurthy, Dan R. K. Ports
More informationChapter 11 DISTRIBUTED FILE SYSTEMS
DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN Chapter 11 DISTRIBUTED FILE SYSTEMS Client-Server Architectures (1) Figure 11-1. (a) The remote access
More informationThe Building Blocks of Consensus
The Building Blocks of Consensus Yee Jiun Song 1, Robbert van Renesse 1, Fred B. Schneider 1, and Danny Dolev 2 1 Cornell University 2 The Hebrew University of Jerusalem Abstract. Consensus is an important
More informationDistributed Systems COMP 212. Revision 2 Othon Michail
Distributed Systems COMP 212 Revision 2 Othon Michail Synchronisation 2/55 How would Lamport s algorithm synchronise the clocks in the following scenario? 3/55 How would Lamport s algorithm synchronise
More informationPractical Byzantine Fault Tolerance. Castro and Liskov SOSP 99
Practical Byzantine Fault Tolerance Castro and Liskov SOSP 99 Why this paper? Kind of incredible that it s even possible Let alone a practical NFS implementation with it So far we ve only considered fail-stop
More informationEverything You Need to Know About MySQL Group Replication
Everything You Need to Know About MySQL Group Replication Luís Soares (luis.soares@oracle.com) Principal Software Engineer, MySQL Replication Lead Copyright 2017, Oracle and/or its affiliates. All rights
More informationDesigning for Understandability: the Raft Consensus Algorithm. Diego Ongaro John Ousterhout Stanford University
Designing for Understandability: the Raft Consensus Algorithm Diego Ongaro John Ousterhout Stanford University Algorithms Should Be Designed For... Correctness? Efficiency? Conciseness? Understandability!
More informationA Distributed and Robust SDN Control Plane for Transactional Network Updates
A Distributed and Robust SDN Control Plane for Transactional Network Updates Marco Canini (UCL) with Petr Kuznetsov (Télécom ParisTech), Dan Levin (TU Berlin), Stefan Schmid (TU Berlin & T-Labs) 1 Network
More informationThe Building Blocks of Consensus
The Building Blocks of Consensus Yee Jiun Song 1, Robbert van Renesse 1, Fred B. Schneider 1, and Danny Dolev 2 1 Cornell University 2 The Hebrew University of Jerusalem Abstract. Consensus is an important
More informationDistributed Systems 23. Fault Tolerance
Distributed Systems 23. Fault Tolerance Paul Krzyzanowski pxk@cs.rutgers.edu 4/20/2011 1 Faults Deviation from expected behavior Due to a variety of factors: Hardware failure Software bugs Operator errors
More informationParallel Data Types of Parallelism Replication (Multiple copies of the same data) Better throughput for read-only computations Data safety Partitionin
Parallel Data Types of Parallelism Replication (Multiple copies of the same data) Better throughput for read-only computations Data safety Partitioning (Different data at different sites More space Better
More informationPaxos Made Live. An Engineering Perspective. Authors: Tushar Chandra, Robert Griesemer, Joshua Redstone. Presented By: Dipendra Kumar Jha
Paxos Made Live An Engineering Perspective Authors: Tushar Chandra, Robert Griesemer, Joshua Redstone Presented By: Dipendra Kumar Jha Consensus Algorithms Consensus: process of agreeing on one result
More informationFailure models. Byzantine Fault Tolerance. What can go wrong? Paxos is fail-stop tolerant. BFT model. BFT replication 5/25/18
Failure models Byzantine Fault Tolerance Fail-stop: nodes either execute the protocol correctly or just stop Byzantine failures: nodes can behave in any arbitrary way Send illegal messages, try to trick
More informationMINIATURE WORLD: MEASURING AND REPLICATING REAL-WORLD BLOCKCHAIN DEPLOYMENTS
MINIATURE WORLD: MEASURING AND REPLICATING REAL-WORLD BLOCKCHAIN DEPLOYMENTS Adem Efe Gencer with Ittay Eyal, Emin Gün Sirer, and Robbert van Renesse Jan 26 th, 2016 Initiative for CryptoCurrencies and
More informationSystem Models for Distributed Systems
System Models for Distributed Systems INF5040/9040 Autumn 2015 Lecturer: Amir Taherkordi (ifi/uio) August 31, 2015 Outline 1. Introduction 2. Physical Models 4. Fundamental Models 2 INF5040 1 System Models
More informationPractice: Large Systems Part 2, Chapter 2
Practice: Large Systems Part 2, Chapter 2 Overvie Introduction Strong Consistency Crash Failures: Primary Copy, Commit Protocols Crash-Recovery Failures: Paxos, Chubby Byzantine Failures: PBFT, Zyzzyva
More informationMegastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database.
Megastore: Providing Scalable, Highly Available Storage for Interactive Services & Spanner: Google s Globally- Distributed Database. Presented by Kewei Li The Problem db nosql complex legacy tuning expensive
More informationSTATE Machine Replication (SMR) is a well-known approach
TO APPEAR IN IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS (MARCH 17) 1 Elastic State Machine Replication Andre Nogueira, Antonio Casimiro, Alysson Bessani State machine replication (SMR) is a
More informationDistributed Systems. Fall 2017 Exam 3 Review. Paul Krzyzanowski. Rutgers University. Fall 2017
Distributed Systems Fall 2017 Exam 3 Review Paul Krzyzanowski Rutgers University Fall 2017 December 11, 2017 CS 417 2017 Paul Krzyzanowski 1 Question 1 The core task of the user s map function within a
More informationConfiguring Advanced Windows Server 2012 Services
Course 20412 : Configuring Advanced Windows Server 2012 Services Page 1 of 6 Configuring Advanced Windows Server 2012 Services Course 20412: 4 days; Instructor-Led Introduction Course 20412 is part three
More informationCORFU: A Shared Log Design for Flash Clusters
CORFU: A Shared Log Design for Flash Clusters Authors: Mahesh Balakrishnan, Dahlia Malkhi, Vijayan Prabhakaran, Ted Wobber, Michael Wei, John D. Davis EECS 591 11/7/18 Presented by Evan Agattas and Fanzhong
More informationConsensus and related problems
Consensus and related problems Today l Consensus l Google s Chubby l Paxos for Chubby Consensus and failures How to make process agree on a value after one or more have proposed what the value should be?
More informationHow to upgrade MongoDB without downtime
How to upgrade MongoDB without downtime me - @adamotonete Adamo Tonete, Senior Technical Engineer Brazil Agenda Versioning Upgrades Operations that always require downtime Upgrading a replica-set Upgrading
More informationAnnouncements. me your survey: See the Announcements page. Today. Reading. Take a break around 10:15am. Ack: Some figures are from Coulouris
Announcements Email me your survey: See the Announcements page Today Conceptual overview of distributed systems System models Reading Today: Chapter 2 of Coulouris Next topic: client-side processing (HTML,
More informationReliable Distributed Storage A Research Perspective. Idit Keidar Technion
Reliable Distributed Storage A Research Perspective Idit Keidar Technion UnDistributed Enterprise Storage (What I Won t Talk About Today) Expensive Needs to be replaced to scale up Direct fiber access
More informationPractical Byzantine Fault Tolerance
Practical Byzantine Fault Tolerance Robert Grimm New York University (Partially based on notes by Eric Brewer and David Mazières) The Three Questions What is the problem? What is new or different? What
More informationIDS: Signature Detection
IDS: Signature Detection Idea: What is bad, is known What is not bad, is good Determines whether a sequence of instructions being executed is known to violate the site security policy Signatures: Descriptions
More informationDistributed Systems. 27. Engineering Distributed Systems. Paul Krzyzanowski. Rutgers University. Fall 2018
Distributed Systems 27. Engineering Distributed Systems Paul Krzyzanowski Rutgers University Fall 2018 1 We need distributed systems We often have a lot of data to ingest, process, and/or store The data
More informationTech Note: vsphere Replication with vsan First Published On: Last Updated On:
Tech Note: vsphere Replication with vsan First Published On: 10-27-2017 Last Updated On: 11-01-2017 1 Table of Contents 1. Introduction 1.1.Introduction 2. vsphere Replication with vsan Datastores 2.1.vSphere
More informationTAPIR. By Irene Zhang, Naveen Sharma, Adriana Szekeres, Arvind Krishnamurthy, and Dan Ports Presented by Todd Charlton
TAPIR By Irene Zhang, Naveen Sharma, Adriana Szekeres, Arvind Krishnamurthy, and Dan Ports Presented by Todd Charlton Outline Problem Space Inconsistent Replication TAPIR Evaluation Conclusion Problem
More informationPaxos and Replication. Dan Ports, CSEP 552
Paxos and Replication Dan Ports, CSEP 552 Today: achieving consensus with Paxos and how to use this to build a replicated system Last week Scaling a web service using front-end caching but what about the
More informationDistributed Systems. 09. State Machine Replication & Virtual Synchrony. Paul Krzyzanowski. Rutgers University. Fall Paul Krzyzanowski
Distributed Systems 09. State Machine Replication & Virtual Synchrony Paul Krzyzanowski Rutgers University Fall 2016 1 State machine replication 2 State machine replication We want high scalability and
More informationJanus. Consolidating Concurrency Control and Consensus for Commits under Conflicts. Shuai Mu, Lamont Nelson, Wyatt Lloyd, Jinyang Li
Janus Consolidating Concurrency Control and Consensus for Commits under Conflicts Shuai Mu, Lamont Nelson, Wyatt Lloyd, Jinyang Li New York University, University of Southern California State of the Art
More informationDesigning Distributed Systems using Approximate Synchrony in Data Center Networks
Designing Distributed Systems using Approximate Synchrony in Data Center Networks Dan R. K. Ports Jialin Li Naveen Kr. Sharma Vincent Liu Arvind Krishnamurthy University of Washington CSE Today s most
More informationNo Trade Secrets. Microsoft does not claim any trade secret rights in this documentation.
FSO]: Intellectual Property Rights Notice for Open Specifications Documentation Technical Documentation. Microsoft publishes Open Specifications documentation for protocols, file formats, languages, standards
More informationStrong Consistency & CAP Theorem
Strong Consistency & CAP Theorem CS 240: Computing Systems and Concurrency Lecture 15 Marco Canini Credits: Michael Freedman and Kyle Jamieson developed much of the original material. Consistency models
More informationlecture 18: network virtualization platform (NVP) 5590: software defined networking anduo wang, Temple University TTLMAN 401B, R 17:30-20:00
lecture 18: network virtualization platform (NVP) 5590: software defined networking anduo wang, Temple University TTLMAN 401B, R 17:30-20:00 Network Virtualization in multi-tenant Datacenters Teemu Koponen.,
More informationCourse Outline 20743B
Course Outline 20743B Module 1: Installing and configuring Windows Server 2016 This module explains how to install and perform post-installation configuration of Windows Server 2016 servers. Introducing
More information