Advantages of P2P systems. P2P Caching and Archiving. Squirrel. Papers to Discuss. Why bother? Implementation

Size: px
Start display at page:

Download "Advantages of P2P systems. P2P Caching and Archiving. Squirrel. Papers to Discuss. Why bother? Implementation"

Transcription

1 Advantages of P2P systems P2P Caching and Archiving Tyrone Nicholas May 10, 2004 Redundancy built in - by definition there are a large number of servers Lends itself automatically to multiple copies of data Leverage this redundancy for: load distribution (Squirrel) failover (OceanStore) consistency (Lockss) Papers to Discuss Squirrel Web caching OceanStore P2P-backed file system Lockss Archiving academic journals Squirrel Web servers cache served pages Corporate LANs often maintain additional caches in proxy servers/firewalls Web browsers cache recently accessed pages Squirrel enables browsers in a local network to share their caches Why bother? Authors say: centralized caches are unscalable single point of failure Presenter says: Visited web pages vary too widely for this to be useful Implementation Based on Pastry Browsers configured to use Squirrel as proxy cache For each HTTP request, hash the URL using SHA-1 Node whose Pastry ID is closest to the URL s hash become its home node 1

2 Two models for content storage Home-store model Home-store Home node keeps copy Straightforward chain: local cache, home node cache, server cache Directory Home node does not keep actual copy Instead, stores directory of pointers to requesting nodes Delegate subsequent requests to these Partition the web; every possible URL maps to one home node, which is responsible for caching it All refreshes from the server go through home node first Directory model Requesters store content, they contact server directly Home node stores only a directory of pointers to delegates Delegates are simply those nodes which requested the page earlier Which model is better? Home store has significantly lower overhead and vulnerability In directory, first node to access a highimage page gets stuck serving every image; while home-store would partition them If a node goes down, every directory pointing to it suffers Transience New node enters network: Piggyback on Pastry informing peers of leaf set changes Copy content or directory to new node Node leaves network: Future requests for content homed at the departed node treated as a cache miss by the new home Criticisms of Squirrel Requires 100MB cache per host just to match performance of centralized cache Larger site reported more network hops with Squirrel than with centralized cache have to traverse a linked list to reach home directory adds extra hop Only useful if multiple clients are accessing the same web page - rarely the case Centralized caches also perform other functions (security etc.) that would be harder in P2P 2

3 Papers to Discuss Squirrel Web caching OceanStore P2P-backed file system Lockss Archiving academic journals OceanStore Leverage the Internet to provide universally available, highly durable filesystem Must be able to withstand constant arrival/departure of nodes Individual nodes are not trusted Versioning Every version of every data object (file) is kept permanently Each version stored in a B-tree-like structure Each file has a GUID, so does each version and each data block Only deltas between versions need be stored - unchanged content points to GUIDs of blocks in older versions Architecture Built on Tapestry routing system Every Tapestry host provides a GUID for itself and each resource it offers OceanStore hosts publish GUIDs for data blocks they store The Big Picture Primary Replica Each file has one primary replica which manages access control, concurrency Stored in the inner ring, a group of servers chosen by a responsible party server Has a timestamped heartbeat mapping file s GUID to that of the latest version Updates to the file managed by Byzantine agreement 3

4 Byzantine agreement Any decision process in which all non-faulty participants will reach the same decision if at least 2/3 of participants obey the protocol Modified Castro-Liskov: a single public key paired with l private key shares. Each server uses these to generate a signature share. Any k of these produces a full signature If l ~= 3k, correct signatures are proof of Byzantine agreement Storage - erasure codes Divide a block into m fragments, then encode them into n smaller pieces using a code with rate m/n Any m of the n fragments can then be used to reconstruct the entire block Primary replicas distribute these to rest of the system using Tapestry Secondary replicas Cache of erasure-encoded objects Hosts publish decoded copies of files they have used Each file has a dissemination tree of hosts serving as secondary replicas Primary replicas multicast updates to the file down the tree The Pond prototype 50,000 lines of Java code, written by 10 students Does not completely implement the OceanStore specification Performance Large storage overhead; equivalent to multiple copies of every file In a LAN, 15x slower than NFS; in a WAN, 3x slower Criticisms of OceanStore Everything is encrypted, digitally signed etc; this leads to a huge overhead Several of the performance evaluations carried out with benchmarks developed by the same research team Relies on the inner ring members not changing frequently Possible security hole from malicious inner ring members? 4

5 Papers to Discuss Squirrel Web caching OceanStore P2P-backed file system Lockss Archiving academic journals Lots of Copies Keep Stuff Safe Want to mimic paper journals, by having digital copies distributed worldwide Must be as difficult to modify everywhere as twould be to modify paper journals Vote on the content! (system not in use in Florida) Design principles Assume cheap, unreliable hardware Keep no secrets: private keys etc. are discarded after use Make it slow No trusted authorities Intrusion detection part of the system core, not a layer on the surface Lockss Architecture Each member library maintains a cache Each node has a crawler that obtains content from the publisher directly Peers take opinion polls to compare each others versions of the content; majority vote wins. The voting process (1) Each peer has a reference list, other peers who have been in polls called by this peer recently (initialized by humans) At random intervals, peer selects a piece of content to conduct a poll on Selects an inner circle from its reference list and sends Poll messages to each The voting process (2) Peers invited to join an inner circle respond yes or no If quorum is met, poll initiator then calculates poll effort proof and sends these to poll participants Inner circle members then nominate peers from their reference lists to the outer circle ; poll initiator adds some of these to the poll 5

6 Poll proofs Expensive to generate a proof, but cheap to verify An attacker wishing to disseminate fake content is via polls gets slowed down by having to generate these proofs Use memory-bound functions, whose cost between computation and verification is parameterizable Accompanies votes as well as poll requests The voting process (3) Votes received are verified by the proof sent with it If vote is valid, its hash of the content is compared to the initiator s Vote can be one of: Landslide victory: initiator leaves content alone Landslide loss: initiator must repair content Narrow victory/loss: raise alarm After the vote If a repair needed, get content from a poll winner; then hash it and compare to the other winners Update reference list Removes all disagreeing and some agreeing inner circle members from RL add valid outer circle members. Thus, new rogue nodes take a while to worm their way into network add some from friends list remove entries that have not voted recently Defense against the dark arts (1) Rogue peer could suborn a poll by dominating a real peer s reference list Difficult because peers regularly add their friends into ref list, and expel inner circle members Attackers could plot to target specific polls for suborning Peers can demand early commitments, making colluding difficult Defense against the dark arts (2) Attackers could eavesdrop on poll communications All communication is encrypted with pairspecific keys Attackers could pretend to be a real peer Real peers can retransmit their poll-acceptance messages - if the initiator gets more than one from the same source, attack is detected Response to simulated attack Running an accelerated version of the system, takes years for an adversary to suborn significant parts of the system 6

7 Criticisms of Lockss Entire system is very application-dependent, little said about generalizability Attacks developed by the same authors of the original system - circularity Problem of disseminating legitimate corrections or changes to original content not addressed 7

Staggeringly Large Filesystems

Staggeringly Large Filesystems Staggeringly Large Filesystems Evan Danaher CS 6410 - October 27, 2009 Outline 1 Large Filesystems 2 GFS 3 Pond Outline 1 Large Filesystems 2 GFS 3 Pond Internet Scale Web 2.0 GFS Thousands of machines

More information

Staggeringly Large File Systems. Presented by Haoyan Geng

Staggeringly Large File Systems. Presented by Haoyan Geng Staggeringly Large File Systems Presented by Haoyan Geng Large-scale File Systems How Large? Google s file system in 2009 (Jeff Dean, LADIS 09) - 200+ clusters - Thousands of machines per cluster - Pools

More information

The OceanStore Write Path

The OceanStore Write Path The OceanStore Write Path Sean C. Rhea John Kubiatowicz University of California, Berkeley June 11, 2002 Introduction: the OceanStore Write Path Introduction: the OceanStore Write Path The Inner Ring Acts

More information

Practical Byzantine Fault

Practical Byzantine Fault Practical Byzantine Fault Tolerance Practical Byzantine Fault Tolerance Castro and Liskov, OSDI 1999 Nathan Baker, presenting on 23 September 2005 What is a Byzantine fault? Rationale for Byzantine Fault

More information

Byzantine Fault Tolerance

Byzantine Fault Tolerance Byzantine Fault Tolerance CS 240: Computing Systems and Concurrency Lecture 11 Marco Canini Credits: Michael Freedman and Kyle Jamieson developed much of the original material. So far: Fail-stop failures

More information

Distributed Systems. Fault Tolerance. Paul Krzyzanowski

Distributed Systems. Fault Tolerance. Paul Krzyzanowski Distributed Systems Fault Tolerance Paul Krzyzanowski Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution 2.5 License. Faults Deviation from expected

More information

Practical Byzantine Fault Tolerance. Miguel Castro and Barbara Liskov

Practical Byzantine Fault Tolerance. Miguel Castro and Barbara Liskov Practical Byzantine Fault Tolerance Miguel Castro and Barbara Liskov Outline 1. Introduction to Byzantine Fault Tolerance Problem 2. PBFT Algorithm a. Models and overview b. Three-phase protocol c. View-change

More information

Failure models. Byzantine Fault Tolerance. What can go wrong? Paxos is fail-stop tolerant. BFT model. BFT replication 5/25/18

Failure models. Byzantine Fault Tolerance. What can go wrong? Paxos is fail-stop tolerant. BFT model. BFT replication 5/25/18 Failure models Byzantine Fault Tolerance Fail-stop: nodes either execute the protocol correctly or just stop Byzantine failures: nodes can behave in any arbitrary way Send illegal messages, try to trick

More information

Efficient Data Structures for Tamper-Evident Logging

Efficient Data Structures for Tamper-Evident Logging Efficient Data Structures for Tamper-Evident Logging Scott A. Crosby Dan S. Wallach Rice University Everyone has logs Tamper evident solutions Current commercial solutions Write only hardware appliances

More information

Dep. Systems Requirements

Dep. Systems Requirements Dependable Systems Dep. Systems Requirements Availability the system is ready to be used immediately. A(t) = probability system is available for use at time t MTTF/(MTTF+MTTR) If MTTR can be kept small

More information

Today. Why might P2P be a win? What is a Peer-to-Peer (P2P) system? Peer-to-Peer Systems and Distributed Hash Tables

Today. Why might P2P be a win? What is a Peer-to-Peer (P2P) system? Peer-to-Peer Systems and Distributed Hash Tables Peer-to-Peer Systems and Distributed Hash Tables COS 418: Distributed Systems Lecture 7 Today 1. Peer-to-Peer Systems Napster, Gnutella, BitTorrent, challenges 2. Distributed Hash Tables 3. The Chord Lookup

More information

LOCKSS: A Peer-to-Peer Digital Preservation System

LOCKSS: A Peer-to-Peer Digital Preservation System LOCKSS: A Peer-to-Peer Digital Preservation System Petros Maniatis maniatis@cs.stanford.edu David S. H. Rosenthal dshr@stanford.edu Mema Roussopoulos mema@cs.stanford.edu Mary Baker mgbaker@cs.stanford.edu

More information

BlockFin A Fork-Tolerant, Leaderless Consensus Protocol April

BlockFin A Fork-Tolerant, Leaderless Consensus Protocol April BlockFin A Fork-Tolerant, Leaderless Consensus Protocol April 2018 @storecoin What are the most desirable features in a blockchain? Scalability (throughput) and decentralization (censorship resistance),

More information

CS555: Distributed Systems [Fall 2017] Dept. Of Computer Science, Colorado State University

CS555: Distributed Systems [Fall 2017] Dept. Of Computer Science, Colorado State University CS 555: DISTRIBUTED SYSTEMS [DYNAMO & GOOGLE FILE SYSTEM] Frequently asked questions from the previous class survey What s the typical size of an inconsistency window in most production settings? Dynamo?

More information

The Design and Implementation of a Next Generation Name Service for the Internet (CoDoNS) Presented By: Kamalakar Kambhatla

The Design and Implementation of a Next Generation Name Service for the Internet (CoDoNS) Presented By: Kamalakar Kambhatla The Design and Implementation of a Next Generation Name Service for the Internet (CoDoNS) Venugopalan Ramasubramanian Emin Gün Sirer Presented By: Kamalakar Kambhatla * Slides adapted from the paper -

More information

Byzantine Fault Tolerant Raft

Byzantine Fault Tolerant Raft Abstract Byzantine Fault Tolerant Raft Dennis Wang, Nina Tai, Yicheng An {dwang22, ninatai, yicheng} @stanford.edu https://github.com/g60726/zatt For this project, we modified the original Raft design

More information

Slides for Chapter 10: Peer-to-Peer Systems

Slides for Chapter 10: Peer-to-Peer Systems Slides for Chapter 10: Peer-to-Peer Systems From Coulouris, Dollimore, Kindberg and Blair Distributed Systems: Concepts and Design Edition 5, Addison-Wesley 2012 Overview of Chapter Introduction Napster

More information

Distributed Storage Systems: Data Replication using Quorums

Distributed Storage Systems: Data Replication using Quorums Distributed Storage Systems: Data Replication using Quorums Background Software replication focuses on dependability of computations What if we are primarily concerned with integrity and availability (and

More information

CS555: Distributed Systems [Fall 2017] Dept. Of Computer Science, Colorado State University

CS555: Distributed Systems [Fall 2017] Dept. Of Computer Science, Colorado State University CS 555: DISTRIBUTED SYSTEMS [P2P SYSTEMS] Shrideep Pallickara Computer Science Colorado State University Frequently asked questions from the previous class survey Byzantine failures vs malicious nodes

More information

Peer-to-peer systems and overlay networks

Peer-to-peer systems and overlay networks Complex Adaptive Systems C.d.L. Informatica Università di Bologna Peer-to-peer systems and overlay networks Fabio Picconi Dipartimento di Scienze dell Informazione 1 Outline Introduction to P2P systems

More information

Replication in Distributed Systems

Replication in Distributed Systems Replication in Distributed Systems Replication Basics Multiple copies of data kept in different nodes A set of replicas holding copies of a data Nodes can be physically very close or distributed all over

More information

Byzantine Fault Tolerance

Byzantine Fault Tolerance Byzantine Fault Tolerance CS6450: Distributed Systems Lecture 10 Ryan Stutsman Material taken/derived from Princeton COS-418 materials created by Michael Freedman and Kyle Jamieson at Princeton University.

More information

Peer-to-Peer Systems and Distributed Hash Tables

Peer-to-Peer Systems and Distributed Hash Tables Peer-to-Peer Systems and Distributed Hash Tables CS 240: Computing Systems and Concurrency Lecture 8 Marco Canini Credits: Michael Freedman and Kyle Jamieson developed much of the original material. Selected

More information

Security (and finale) Dan Ports, CSEP 552

Security (and finale) Dan Ports, CSEP 552 Security (and finale) Dan Ports, CSEP 552 Today Security: what if parts of your distributed system are malicious? BFT: state machine replication Bitcoin: peer-to-peer currency Course wrap-up Security Too

More information

A Lightweight Blockchain Consensus Protocol

A Lightweight Blockchain Consensus Protocol A Lightweight Blockchain Consensus Protocol Keir Finlow-Bates keir@chainfrog.com Abstract A lightweight yet deterministic and objective consensus protocol would allow blockchain systems to be maintained

More information

Lecture XII: Replication

Lecture XII: Replication Lecture XII: Replication CMPT 401 Summer 2007 Dr. Alexandra Fedorova Replication 2 Why Replicate? (I) Fault-tolerance / High availability As long as one replica is up, the service is available Assume each

More information

Distributed Systems Peer-to-Peer Systems

Distributed Systems Peer-to-Peer Systems Distributed Systems Peer-to-Peer Systems [2] P2P Systems - Goals and Definition Goal: to enable sharing of data and resources on a very large scale by eliminating any requirement for separately-managed

More information

A Survey on Peer-to-Peer File Systems

A Survey on Peer-to-Peer File Systems Christopher Chang May 10, 2007 CSE 598D: Storage Systems Department of Computer Science and Engineering The Pennsylvania State University A Survey on Peer-to-Peer File Systems Introduction Demand for information

More information

Proactive Recovery in a Byzantine-Fault-Tolerant System

Proactive Recovery in a Byzantine-Fault-Tolerant System Proactive Recovery in a Byzantine-Fault-Tolerant System Miguel Castro and Barbara Liskov Laboratory for Computer Science, Massachusetts Institute of Technology, 545 Technology Square, Cambridge, MA 02139

More information

Squirrel case-study. Decentralized peer-to-peer web cache. Traditional centralized web cache. Based on the Pastry peer-to-peer middleware system

Squirrel case-study. Decentralized peer-to-peer web cache. Traditional centralized web cache. Based on the Pastry peer-to-peer middleware system Decentralized peer-to-peer web cache Based on the Pastry peer-to-peer middleware system Traditional centralized web cache 1 2 Decentralized caching of web pages use the resources of peers (web browsers/clients)

More information

Scaling Byzantine Fault-tolerant Replication to Wide Area Networks

Scaling Byzantine Fault-tolerant Replication to Wide Area Networks Scaling Byzantine Fault-tolerant Replication to Wide Area Networks Cristina Nita-Rotaru Dependable and Secure Distributed Systems Lab Department of Computer Science and CERIAS Purdue University Department

More information

CS /29/17. Paul Krzyzanowski 1. Fall 2016: Question 2. Distributed Systems. Fall 2016: Question 2 (cont.) Fall 2016: Question 3

CS /29/17. Paul Krzyzanowski 1. Fall 2016: Question 2. Distributed Systems. Fall 2016: Question 2 (cont.) Fall 2016: Question 3 Fall 2016: Question 2 You have access to a file of class enrollment lists. Each line contains {course_number, student_id}. Distributed Systems 2017 Pre-exam 3 review Selected questions from past exams

More information

Secure Distributed Storage in Peer-to-peer networks

Secure Distributed Storage in Peer-to-peer networks Secure Distributed Storage in Peer-to-peer networks Øyvind Hanssen 07.02.2007 Motivation Mobile and ubiquitous computing Persistent information in untrusted networks Sharing of storage and information

More information

Security for Structured Peer-to-peer Overlay Networks. Acknowledgement. Outline. By Miguel Castro et al. OSDI 02 Presented by Shiping Chen in IT818

Security for Structured Peer-to-peer Overlay Networks. Acknowledgement. Outline. By Miguel Castro et al. OSDI 02 Presented by Shiping Chen in IT818 Security for Structured Peer-to-peer Overlay Networks By Miguel Castro et al. OSDI 02 Presented by Shiping Chen in IT818 1 Acknowledgement Some of the following slides are borrowed from talks by Yun Mao

More information

Large-Scale Data Stores and Probabilistic Protocols

Large-Scale Data Stores and Probabilistic Protocols Distributed Systems 600.437 Large-Scale Data Stores & Probabilistic Protocols Department of Computer Science The Johns Hopkins University 1 Large-Scale Data Stores and Probabilistic Protocols Lecture 11

More information

Practical Byzantine Fault Tolerance

Practical Byzantine Fault Tolerance Appears in the Proceedings of the Third Symposium on Operating Systems Design and Implementation, New Orleans, USA, February 1999 Practical Byzantine Fault Tolerance Miguel Castro and Barbara Liskov Laboratory

More information

Distributed Systems. Lec 10: Distributed File Systems GFS. Slide acks: Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung

Distributed Systems. Lec 10: Distributed File Systems GFS. Slide acks: Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Distributed Systems Lec 10: Distributed File Systems GFS Slide acks: Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung 1 Distributed File Systems NFS AFS GFS Some themes in these classes: Workload-oriented

More information

Presented By: Devarsh Patel

Presented By: Devarsh Patel : Amazon s Highly Available Key-value Store Presented By: Devarsh Patel CS5204 Operating Systems 1 Introduction Amazon s e-commerce platform Requires performance, reliability and efficiency To support

More information

Motivation for peer-to-peer

Motivation for peer-to-peer Peer-to-peer systems INF 5040 autumn 2015 lecturer: Roman Vitenberg INF5040, Frank Eliassen & Roman Vitenberg 1 Motivation for peer-to-peer Ø Inherent restrictions of the standard client/ server model

More information

Practical Byzantine Fault Tolerance. Castro and Liskov SOSP 99

Practical Byzantine Fault Tolerance. Castro and Liskov SOSP 99 Practical Byzantine Fault Tolerance Castro and Liskov SOSP 99 Why this paper? Kind of incredible that it s even possible Let alone a practical NFS implementation with it So far we ve only considered fail-stop

More information

Consistency and Replication 1/65

Consistency and Replication 1/65 Consistency and Replication 1/65 Replicas and Consistency??? Tatiana Maslany in the show Orphan Black: The story of a group of clones that discover each other and the secret organization Dyad, which was

More information

Store, Forget & Check: Using Algebraic Signatures to Check Remotely Administered Storage

Store, Forget & Check: Using Algebraic Signatures to Check Remotely Administered Storage Store, Forget & Check: Using Algebraic Signatures to Check Remotely Administered Storage Ethan L. Miller & Thomas J. E. Schwarz Storage Systems Research Center University of California, Santa Cruz What

More information

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or

More information

Byzantine fault tolerance. Jinyang Li With PBFT slides from Liskov

Byzantine fault tolerance. Jinyang Li With PBFT slides from Liskov Byzantine fault tolerance Jinyang Li With PBFT slides from Liskov What we ve learnt so far: tolerate fail-stop failures Traditional RSM tolerates benign failures Node crashes Network partitions A RSM w/

More information

Trustworthy Coordination of Web Services Atomic Transactions

Trustworthy Coordination of Web Services Atomic Transactions 1 Trustworthy Coordination of Web s Atomic Transactions Honglei Zhang, Hua Chai, Wenbing Zhao, Member, IEEE, P. M. Melliar-Smith, Member, IEEE, L. E. Moser, Member, IEEE Abstract The Web Atomic Transactions

More information

Consistency and Replication 1/62

Consistency and Replication 1/62 Consistency and Replication 1/62 Replicas and Consistency??? Tatiana Maslany in the show Orphan Black: The story of a group of clones that discover each other and the secret organization Dyad, which was

More information

Chapter 10: Peer-to-Peer Systems

Chapter 10: Peer-to-Peer Systems Chapter 10: Peer-to-Peer Systems From Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edition 4, Addison-Wesley 2005 Introduction To enable the sharing of data and resources

More information

Today: Fault Tolerance. Replica Management

Today: Fault Tolerance. Replica Management Today: Fault Tolerance Failure models Agreement in presence of faults Two army problem Byzantine generals problem Reliable communication Distributed commit Two phase commit Three phase commit Failure recovery

More information

Failures, Elections, and Raft

Failures, Elections, and Raft Failures, Elections, and Raft CS 8 XI Copyright 06 Thomas W. Doeppner, Rodrigo Fonseca. All rights reserved. Distributed Banking SFO add interest based on current balance PVD deposit $000 CS 8 XI Copyright

More information

Distributed Systems COMP 212. Revision 2 Othon Michail

Distributed Systems COMP 212. Revision 2 Othon Michail Distributed Systems COMP 212 Revision 2 Othon Michail Synchronisation 2/55 How would Lamport s algorithm synchronise the clocks in the following scenario? 3/55 How would Lamport s algorithm synchronise

More information

Remote E-Voting System

Remote E-Voting System Remote E-Voting System Crypto2-Spring 2013 Benjamin Kaiser Jacob Shedd Jeremy White Phases Initialization Registration Voting Verifying Activities Trusted Authority (TA) distributes 4 keys to Registrar,

More information

CSE 153 Design of Operating Systems

CSE 153 Design of Operating Systems CSE 153 Design of Operating Systems Winter 2018 Lecture 22: File system optimizations and advanced topics There s more to filesystems J Standard Performance improvement techniques Alternative important

More information

Introduction to Peer-to-Peer Systems

Introduction to Peer-to-Peer Systems Introduction Introduction to Peer-to-Peer Systems Peer-to-peer (PP) systems have become extremely popular and contribute to vast amounts of Internet traffic PP basic definition: A PP system is a distributed

More information

Detecting Insider Attacks on Databases using Blockchains

Detecting Insider Attacks on Databases using Blockchains Detecting Insider Attacks on Databases using Blockchains Shubham Sharma, Rahul Gupta, Shubham Sahai Srivastava and Sandeep K. Shukla Department of Computer Science and Engineering Indian Institute of Technology,

More information

E-companion. Quiz for IT-knowledge

E-companion. Quiz for IT-knowledge E-companion Quiz for IT-knowledge 1. The is used in combination with other keys to perform shortcuts and special tasks. A) Control (Ctrl) key B) Function key C) Toggle key D) Windows key 2. is the place

More information

Distributed Systems. Fall 2017 Exam 3 Review. Paul Krzyzanowski. Rutgers University. Fall 2017

Distributed Systems. Fall 2017 Exam 3 Review. Paul Krzyzanowski. Rutgers University. Fall 2017 Distributed Systems Fall 2017 Exam 3 Review Paul Krzyzanowski Rutgers University Fall 2017 December 11, 2017 CS 417 2017 Paul Krzyzanowski 1 Question 1 The core task of the user s map function within a

More information

Remote Data Checking for Network Codingbased. Distributed Storage Systems

Remote Data Checking for Network Codingbased. Distributed Storage Systems CCSW 0 Remote Data Checking for Network Coding-based Bo Chen, Reza Curtmola, Giuseppe Ateniese, Randal Burns New Jersey Institute of Technology Johns Hopkins University Motivation Cloud storage can release

More information

Trojan-tolerant Hardware & Supply Chain Security in Practice

Trojan-tolerant Hardware & Supply Chain Security in Practice Trojan-tolerant Hardware & Supply Chain Security in Practice Who we are Vasilios Mavroudis Doctoral Researcher, UCL Dan Cvrcek CEO, Enigma Bridge George Danezis Professor, UCL Petr Svenda CTO, Enigma Bridge

More information

Distributed Systems

Distributed Systems 15-440 Distributed Systems 11 - Fault Tolerance, Logging and Recovery Tuesday, Oct 2 nd, 2018 Logistics Updates P1 Part A checkpoint Part A due: Saturday 10/6 (6-week drop deadline 10/8) *Please WORK hard

More information

Authenticated Byzantine Fault Tolerance Without Public-Key Cryptography

Authenticated Byzantine Fault Tolerance Without Public-Key Cryptography Appears as Technical Memo MIT/LCS/TM-589, MIT Laboratory for Computer Science, June 999 Authenticated Byzantine Fault Tolerance Without Public-Key Cryptography Miguel Castro and Barbara Liskov Laboratory

More information

Today: Fault Tolerance. Fault Tolerance

Today: Fault Tolerance. Fault Tolerance Today: Fault Tolerance Agreement in presence of faults Two army problem Byzantine generals problem Reliable communication Distributed commit Two phase commit Three phase commit Paxos Failure recovery Checkpointing

More information

Distributed Systems. Characteristics of Distributed Systems. Lecture Notes 1 Basic Concepts. Operating Systems. Anand Tripathi

Distributed Systems. Characteristics of Distributed Systems. Lecture Notes 1 Basic Concepts. Operating Systems. Anand Tripathi 1 Lecture Notes 1 Basic Concepts Anand Tripathi CSci 8980 Operating Systems Anand Tripathi CSci 8980 1 Distributed Systems A set of computers (hosts or nodes) connected through a communication network.

More information

Distributed Systems. Characteristics of Distributed Systems. Characteristics of Distributed Systems. Goals in Distributed System Designs

Distributed Systems. Characteristics of Distributed Systems. Characteristics of Distributed Systems. Goals in Distributed System Designs 1 Anand Tripathi CSci 8980 Operating Systems Lecture Notes 1 Basic Concepts Distributed Systems A set of computers (hosts or nodes) connected through a communication network. Nodes may have different speeds

More information

Key-value store with eventual consistency without trusting individual nodes

Key-value store with eventual consistency without trusting individual nodes basementdb Key-value store with eventual consistency without trusting individual nodes https://github.com/spferical/basementdb 1. Abstract basementdb is an eventually-consistent key-value store, composed

More information

CS5412: CONSENSUS AND THE FLP IMPOSSIBILITY RESULT

CS5412: CONSENSUS AND THE FLP IMPOSSIBILITY RESULT 1 CS5412: CONSENSUS AND THE FLP IMPOSSIBILITY RESULT Lecture XII Ken Birman Generalizing Ron and Hermione s challenge 2 Recall from last time: Ron and Hermione had difficulty agreeing where to meet for

More information

To do. Consensus and related problems. q Failure. q Raft

To do. Consensus and related problems. q Failure. q Raft Consensus and related problems To do q Failure q Consensus and related problems q Raft Consensus We have seen protocols tailored for individual types of consensus/agreements Which process can enter the

More information

Distributed Hash Tables

Distributed Hash Tables Distributed Hash Tables CS6450: Distributed Systems Lecture 11 Ryan Stutsman Material taken/derived from Princeton COS-418 materials created by Michael Freedman and Kyle Jamieson at Princeton University.

More information

Slides for Chapter 10: Peer-to-Peer Systems. From Coulouris, Dollimore, Kindberg and Blair Distributed Systems: Concepts and Design

Slides for Chapter 10: Peer-to-Peer Systems. From Coulouris, Dollimore, Kindberg and Blair Distributed Systems: Concepts and Design Slides for Chapter 10: Peer-to-Peer Systems From Coulouris, Dollimore, Kindberg and Blair Distributed Systems: Concepts and Design Edition 5, Addison-Wesley 2012 Edited and supplemented by Jonne Itkonen,!

More information

Agreement in Distributed Systems CS 188 Distributed Systems February 19, 2015

Agreement in Distributed Systems CS 188 Distributed Systems February 19, 2015 Agreement in Distributed Systems CS 188 Distributed Systems February 19, 2015 Page 1 Introduction We frequently want to get a set of nodes in a distributed system to agree Commitment protocols and mutual

More information

Developer Resources: PIN2

Developer Resources: PIN2 Administrative Technology Services Technology and Data Services Developer Resources: PIN2 Contents Introduction... 2 Registering an Application... 2 Information Required for Registration... 3 Information

More information

CS /15/16. Paul Krzyzanowski 1. Question 1. Distributed Systems 2016 Exam 2 Review. Question 3. Question 2. Question 5.

CS /15/16. Paul Krzyzanowski 1. Question 1. Distributed Systems 2016 Exam 2 Review. Question 3. Question 2. Question 5. Question 1 What makes a message unstable? How does an unstable message become stable? Distributed Systems 2016 Exam 2 Review Paul Krzyzanowski Rutgers University Fall 2016 In virtual sychrony, a message

More information

GFS Overview. Design goals/priorities Design for big-data workloads Huge files, mostly appends, concurrency, huge bandwidth Design for failures

GFS Overview. Design goals/priorities Design for big-data workloads Huge files, mostly appends, concurrency, huge bandwidth Design for failures GFS Overview Design goals/priorities Design for big-data workloads Huge files, mostly appends, concurrency, huge bandwidth Design for failures Interface: non-posix New op: record appends (atomicity matters,

More information

CS514: Intermediate Course in Computer Systems

CS514: Intermediate Course in Computer Systems Distributed Hash Tables (DHT) Overview and Issues Paul Francis CS514: Intermediate Course in Computer Systems Lecture 26: Nov 19, 2003 Distributed Hash Tables (DHT): Overview and Issues What is a Distributed

More information

Address Translation. Tore Larsen Material developed by: Kai Li, Princeton University

Address Translation. Tore Larsen Material developed by: Kai Li, Princeton University Address Translation Tore Larsen Material developed by: Kai Li, Princeton University Topics Virtual memory Virtualization Protection Address translation Base and bound Segmentation Paging Translation look-ahead

More information

Int ernet w orking. Internet Security. Literature: Forouzan: TCP/IP Protocol Suite : Ch 28

Int ernet w orking. Internet Security. Literature: Forouzan: TCP/IP Protocol Suite : Ch 28 Int ernet w orking Internet Security Literature: Forouzan: TCP/IP Protocol Suite : Ch 28 Internet Security Internet security is difficult Internet protocols were not originally designed for security The

More information

Peer-to-Peer Systems. Network Science: Introduction. P2P History: P2P History: 1999 today

Peer-to-Peer Systems. Network Science: Introduction. P2P History: P2P History: 1999 today Network Science: Peer-to-Peer Systems Ozalp Babaoglu Dipartimento di Informatica Scienza e Ingegneria Università di Bologna www.cs.unibo.it/babaoglu/ Introduction Peer-to-peer (PP) systems have become

More information

Proceedings of FAST 03: 2nd USENIX Conference on File and Storage Technologies

Proceedings of FAST 03: 2nd USENIX Conference on File and Storage Technologies USENIX Association Proceedings of FAST 03: 2nd USENIX Conference on File and Storage Technologies San Francisco, CA, USA March 31 April 2, 2003 2003 by The USENIX Association All Rights Reserved For more

More information

SQL, NoSQL, MongoDB. CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden

SQL, NoSQL, MongoDB. CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden SQL, NoSQL, MongoDB CSE-291 (Cloud Computing) Fall 2016 Gregory Kesden SQL Databases Really better called Relational Databases Key construct is the Relation, a.k.a. the table Rows represent records Columns

More information

Peer-to-peer computing research a fad?

Peer-to-peer computing research a fad? Peer-to-peer computing research a fad? Frans Kaashoek kaashoek@lcs.mit.edu NSF Project IRIS http://www.project-iris.net Berkeley, ICSI, MIT, NYU, Rice What is a P2P system? Node Node Node Internet Node

More information

Byzantine Techniques

Byzantine Techniques November 29, 2005 Reliability and Failure There can be no unity without agreement, and there can be no agreement without conciliation René Maowad Reliability and Failure There can be no unity without agreement,

More information

Failure Models. Fault Tolerance. Failure Masking by Redundancy. Agreement in Faulty Systems

Failure Models. Fault Tolerance. Failure Masking by Redundancy. Agreement in Faulty Systems Fault Tolerance Fault cause of an error that might lead to failure; could be transient, intermittent, or permanent Fault tolerance a system can provide its services even in the presence of faults Requirements

More information

MEMBERSHIP & PARTICIPATION

MEMBERSHIP & PARTICIPATION MEMBERSHIP & PARTICIPATION What types of activities can I expect to participate in? There are a variety of activities for you to participate in such as discussion boards, idea exchanges, contests, surveys,

More information

Peer-to-Peer Networks 12 Fast Download

Peer-to-Peer Networks 12 Fast Download Peer-to-Peer Networks 12 Fast Download Arne Vater Technical Faculty Computer Networks and Telematics University of Freiburg IP Multicast Motivation - Transmission of a data stream to many receivers Unicast

More information

Chapter 11: File System Implementation. Objectives

Chapter 11: File System Implementation. Objectives Chapter 11: File System Implementation Objectives To describe the details of implementing local file systems and directory structures To describe the implementation of remote file systems To discuss block

More information

An On-demand Secure Routing Protocol Resilient to Byzantine Failures. Routing: objective. Communication Vulnerabilities

An On-demand Secure Routing Protocol Resilient to Byzantine Failures. Routing: objective. Communication Vulnerabilities An On-demand Secure Routing Protocol Resilient to Byzantine Failures Baruch Awerbuch Johns Hopkins University On-Demand vs. Proactive Routing Security Concerns On-Demand Source Authentication Caching presents

More information

Data Management in Application Servers. Dean Jacobs BEA Systems

Data Management in Application Servers. Dean Jacobs BEA Systems Data Management in Application Servers Dean Jacobs BEA Systems Outline Clustered Application Servers Adding Web Services Java 2 Enterprise Edition (J2EE) The Application Server platform for Java Java Servlets

More information

Last time. Distributed systems Lecture 6: Elections, distributed transactions, and replication. DrRobert N. M. Watson

Last time. Distributed systems Lecture 6: Elections, distributed transactions, and replication. DrRobert N. M. Watson Distributed systems Lecture 6: Elections, distributed transactions, and replication DrRobert N. M. Watson 1 Last time Saw how we can build ordered multicast Messages between processes in a group Need to

More information

An On-demand Secure Routing Protocol Resilient to Byzantine Failures

An On-demand Secure Routing Protocol Resilient to Byzantine Failures An On-demand Secure Routing Protocol Resilient to Byzantine Failures Baruch Awerbuch Johns Hopkins University Joint work with David Holmer, Cristina Nita-Rotaru, and Herbert Rubens Based on paper at WiSe2002

More information

Availability versus consistency. Eventual Consistency: Bayou. Eventual consistency. Bayou: A Weakly Connected Replicated Storage System

Availability versus consistency. Eventual Consistency: Bayou. Eventual consistency. Bayou: A Weakly Connected Replicated Storage System Eventual Consistency: Bayou Availability versus consistency Totally-Ordered Multicast kept replicas consistent but had single points of failure Not available under failures COS 418: Distributed Systems

More information

Security Setup CHAPTER

Security Setup CHAPTER CHAPTER 8 This chapter describes how to set up your bridge s security features. This chapter contains the following sections: Security Overview, page 8-2 Setting Up WEP, page 8-7 Enabling Additional WEP

More information

The Chubby Lock Service for Loosely-coupled Distributed systems

The Chubby Lock Service for Loosely-coupled Distributed systems The Chubby Lock Service for Loosely-coupled Distributed systems Author: Mike Burrows Presenter: Ke Nian University of Waterloo, Waterloo, Canada October 22, 2014 Agenda Distributed Consensus Problem 3

More information

The nominative data coming from the population registry and used to generate the voting cards are never stored in the evoting system database.

The nominative data coming from the population registry and used to generate the voting cards are never stored in the evoting system database. System overview Legal compliance CHVote complies to the swiss federal chancellery ordinance on Electronic Voting (VEleS) for an evoting system offered up to 30% of the resident voters. The election process

More information

Last Class: Consistency Models. Today: Implementation Issues

Last Class: Consistency Models. Today: Implementation Issues Last Class: Consistency Models Need for replication Data-centric consistency Strict, linearizable, sequential, causal, FIFO Lecture 15, page 1 Today: Implementation Issues Replica placement Use web caching

More information

Database Architectures

Database Architectures Database Architectures CPS352: Database Systems Simon Miner Gordon College Last Revised: 4/15/15 Agenda Check-in Parallelism and Distributed Databases Technology Research Project Introduction to NoSQL

More information

Exam : Title : Security Solutions for Systems Engineers(SSSE) Version : Demo

Exam : Title : Security Solutions for Systems Engineers(SSSE) Version : Demo Exam : 642-565 Title : Security Solutions for Systems Engineers(SSSE) Version : Demo 1. SomeCompany, Ltd. wants to implement the the PCI Data Security Standard to protect sensitive cardholder information.

More information

Security and Anonymity

Security and Anonymity Security and Anonymity Distributed Systems need a network to send messages. Any message you send in a network can be looked at by any router or machine it goes through. Further if your machine is on the

More information

Introduction to Distributed Data Systems

Introduction to Distributed Data Systems Introduction to Distributed Data Systems Serge Abiteboul Ioana Manolescu Philippe Rigaux Marie-Christine Rousset Pierre Senellart Web Data Management and Distribution http://webdam.inria.fr/textbook January

More information

Securing our Package Distribution System

Securing our Package Distribution System Securing our Package Distribution System Duncan Coutts and Edsko de Vries August 2015, Haskell Implementors Workshop Copyright 2015 Well-Typed LLP Well-Typed The Haskell Consultants Whither security? Why

More information

Fault Tolerance. Distributed Software Systems. Definitions

Fault Tolerance. Distributed Software Systems. Definitions Fault Tolerance Distributed Software Systems Definitions Availability: probability the system operates correctly at any given moment Reliability: ability to run correctly for a long interval of time Safety:

More information

! Design constraints. " Component failures are the norm. " Files are huge by traditional standards. ! POSIX-like

! Design constraints.  Component failures are the norm.  Files are huge by traditional standards. ! POSIX-like Cloud background Google File System! Warehouse scale systems " 10K-100K nodes " 50MW (1 MW = 1,000 houses) " Power efficient! Located near cheap power! Passive cooling! Power Usage Effectiveness = Total

More information