08 Distributed Hash Tables
|
|
- Beverly Manning
- 6 years ago
- Views:
Transcription
1 08 Distributed Hash Tables
2 2/59 Chord Lookup Algorithm Properties Interface: lookup(key) IP address Efficient: O(log N) messages per lookup N is the total number of servers Scalable: O(log N) state per node Robust: survives massive failures Simple to analyze
3 3/59 Chord IDs Key identifier = SHA-1(key) Node identifier = SHA-1(IP address) SHA-1 distributes both uniformly How to map key IDs to node IDs?
4 4/59 Consistent Hashing [Karger 97] Key 5 Node 105 K5 N105 K20 Circular 7-bit ID space N32 N90 K80 A key is stored at its successor: node with next higher ID
5 5/59 Basic Lookup N120 N10 Where is key 80? N105 N90 has K80 K80 N90 N60 N32
6 6/59 Simple lookup algorithm Lookup(my-id, key-id) n = my_successor if my-id < n < key-id call Lookup(id) on node n // next hop else return my successor // done Correctness depends only on successors
7 7/59 Finger Table Allows log(n)-time Lookups ¼ 1/8 1/16 1/32 1/64 1/128 N80 ½
8 8/59 Finger i Points to Successor of n+2i N ¼ 1/8 1/16 1/32 1/64 1/128 N80 ½
9 9/59 Lookup with Fingers Lookup(my-id, key-id) look in local finger table for highest node n s.t. my-id < n < key-id if n exists call Lookup(id) on node n // next hop else return my_successor // done
10 10/59 Lookups Take O(log(N)) Hops N5 N10 N110 K19 N20 N99 N32 K90 Lookup(K19) Lookup(K90) N80 N60
11 11/59 Joining: Linked List Insert N25 N36 1. Lookup(36) N40 K30 K38
12 12/59 Join (2) N25 2. N36 sets its own successor pointer N36 N40 K26 K30 K38
13 13/59 Join (3) N25 3. Copy keys from N40 to N36 N36 N40 K38 K26 K30
14 14/59 Join (4) N25 4. Set N25 s successor pointer N36 N40 K38 Predecessor pointer allows link to new host Update finger pointers in the background Correct successors produce correct lookups K26 K30
15 15/59 Failures Might Cause Incorrect Lookup N120 N113 N10 N102 N85 Lookup(90) N80 N80 doesn t know correct successor, so incorrect lookup
16 16/59 Solution: Successor Lists Each node knows r immediate successors After failure, will know first live successor Correct successors guarantee correct lookups Guarantee is with some probability
17 17/59 Lookup with Fault Tolerance Lookup(my-id, key-id) look in finger_table + succ_list for highest node n s.t. my-id < n < key-id if n exists call Lookup(id) on node n // next hop if call failed remove n from finger table return Lookup(my-id, key-id) else return my successor // done
18 18/59 Experimental Overview Quick lookup in large systems Low variation in lookup costs Robust despite massive failure Experiments confirm theoretical results
19 19/59 DHash Properties Builds key/value storage on Chord Replicates blocks for availability What happens when DHT partitions, then heals? Which (k, v) pairs do I need? Caches blocks for load balance Authenticates block contents
20 20/59 Replication Data at r Successors N5 N10 N110 N20 N99 N40 Block 17 N50 N80 N68 N60 Replicas are easy to find if successor fails Hashed node IDs ensure independent failure
21 21/59 Pastry
22 Pastry Node IDs are assigned randomly With high probability nodes with adjacent IDs are diverse Considers network locality Seeks to minimize distance messages travel Scalar proximity metric #IP routing hops RTT
23 Pastry: Object distribution O Consistent hashing [Karger et al. 97] objid nodeids 128 bit circular id space * nodeids (uniform random) objids (uniform random) Invariant: node with numerically closest nodeid maintains object. (Recall Chord)
24 Pastry: Node Soft State Storage requirement in each node = O(log N) Immediate Neighbors in ID space (Used for routing) Used for routing Nodes closest according to locality (Used to update routing table) Similar to successor and predecessor Similar to finger table entries
25 Pastry: Node Soft State Leaf Set Neighborhood Set Contains L nodes, closest in the ID space Contains M nodes, closest according to proximity metric Routing Table Entries of row n, shares exactly the first n digits with the local node Nodes are chosen according to proximity metric
26 Pastry: Routing Case I Key within leaf set Case II (Prefix Routing) Key not within leaf set Route to the node in the leaf set with ID closest to key Route a node in the routing table, such that the new node shares one more digit with the key than the local node Case III Key not within leaf set Case II not possible Route to a node which shares at least same number of digits with the key, but is closer to the key than the local node
27 Routing from To To By To Owner By To By
28 Routing Example d471f1 d467c4 d46a1c 65a1fc Cuts the ID space into 1/(2^b) Number of hops needed is log2^bn d462ba d4213f lookup(d46a1c) d13da3
29 Self-organization How are the routing tables and leaf sets initialized and maintained? Node addition Node departure (failure)
30 Self Organization: Node Join d471f1 Z=d467c4 X=d46a1c d462ba d4213f New node: X=d46a1c A is X s neighbor Route(d46a1c) The new node X asks node 65a1fc to route a message to it. Nodes in the route share their routing tables with X A = 65a1fc Node X splits key with d4771f and d467c4 B = d13da3
31 Self Organization: Node Join Leaf set (X) = leaf set (Z) Neighborhood set (A) = neighborhood set (X) d471f1 Z=d467c4 X=d46a1c d462ba d4213f New node: X=d46a1c Route(d46a1c) Routing Table Row zero of X = row zero of A Row one of X = row one of B A = 65a1fc d13da3
32 Pastry: Node departure (failure) Leaf set repair (eager all the time): Routing table repair (lazy upon failure): Leaf set members exchange keep-alive messages Request set from furthest live node in set Get table from peers in the same row, if not found from higher rows Neighborhood set repair (eager)
33 Pastry: Node State Update X informs any nodes that need to be aware of its arrival X also improves its table locality by requesting neighborhood sets from all nodes X knows Optimistic approach
34 CAN Content-Addressable Network
35 System Goals CAN: A distributed infrastructure that provides hash table-like functionality on Internet-like scales. Scalable Fault-tolerant Self-organizing
36 Basic Design A virtual d-dimensional Coordinate space Each node owns a Zone in the virtual space Data is stored as (key, value) pair Hash(key) --> a point P in the virtual space (key, value) pair is stored on the node within whose Zone the point P locates
37 Routing For routing purpose, each node only need to maintain the information of those nodes that hold coordinate zone adjoining its own zone (neighbors) Routing: greedy algorithm if P is within the Zone of current node else return(key, value) or failure (if no such key) forward the query to the neighbor with coordinates closest to P
38 Example: Routing (0, 4) (4, 4) 7 (0,0) (4, 0)
39 Node Insertion A new node N1 is going to join the network: Find a node N2 already in the CAN Randomly choose one point P in the space Send a JOIN request destined for P (P resides in the Zone of N3) N3 splits its Zone and assigns half zone to N1, and N3 sends (key, values) pairs from the half zone to N1 N3 sends neighbors information N3 notices all the neighbors the reallocation of space. Neighbors change their corresponding data
40 Example: Insertion
41 Node departure Explicit departure Hand over its zone to another node to produce a valid single zone Or merge with a smallest zone Node failure Periodic update messages between Neighbors A takeover mechanism merges the zone Prolonged absence of an update message from a neighbor<=>failure with the smallest adjacent zone There is also a background zone-reassignment algorithm to smooth the zone allocation
42 Example: Node Departure
43 Kademlia
44 Kademlia Nodes, files and key words, deploy SHA-1 hash into a 160 bits space. Every node maintains information about files, key words close to itself. The closeness between two objects measured as their bitwise XOR interpreted as an integer. distance(a, b) = a XOR b
45 Kademlia Binary Tree Treat nodes as leaves of a binary tree. Start from root, for any given node, dividing the binary tree into a series of successively lower subtrees that don t contain the node.
46 Kademlia Binary Tree Subtrees of interest for a node 0011
47 Kademlia Binary Tree Every node keeps touch with at least one node from each of its subtrees. (if there is a node in that subtree.) Corresponding to each subtree, there is a k-bucket. Every node keeps a list of (IP-address, Port, Node id) triples, and (key, value) tuples for further exchanging information with others.
48 Kademlia Search An example of lookup: node 0011 is searching for 1110 in the network
49 The XOR Metric d (x,x) = 0 d (x,y) > 0 if x y d (x,y) = d (y,x) d (x,y) + d (y,z) d (x, z) For each x and t, there is exactly one node y for which d (x,y) = t
50 Node state For each i (0 i <160) every node keeps a list of nodes of distance between 2i and 2(i+1) from itself.. Call each list a k-bucket. Least recenly seen The list is sorted by time last seen. The value of k is chosen so that any give set of k nodes is unlikely to fail within an hour. head Most recenly seen The list is updated whenever a node tail receives a message. Gnutella showed that the longer a node is up, the more likely it is to remain up for one more hour k = system-wide replication parameter
51
52
53 Node state The nodes in the k-buckets are the stepping stones of routing. By relying on the oldest nodes, k-buckets promise the probability that they will remain online. Least recenly seen Most recenly seen DoS attack is prevented since the new nodes find it difficult to How is the bucket updated? get into the k-bucket
54 Kademlia RPC PING: to test whether a node is online STORE: instruct a node to store a key FIND_NODE: takes an ID as an argument, a recipient returns (IP address, UDP port, node id) of the k nodes that it knows from the set of nodes closest to ID (node lookup) FIND_VALUE: behaves like FIND_NODE, unless the recipient received a STORE for that key, it just returns the stored value.
55 Kademlia Lookup Kademlia employs a recursive algorithm for node lookups. The lookup initiator starts by picking α nodes from its closest non-empty k-bucket. The initiator then sends parallel, asynchronous FIND_NODE to the α nodes it has chosen. α is a system-wide concurrency parameter, such as 3.
56 Kademlia Lookup When α = 1, the lookup resembles that in Chord in terms of message cost, and the latency of detecting the failed nodes. However, unlike Chord, Kademlia has the flexibility of choosing any one of the k nodes in a bucket, so it can forward with lower latency.
57 Kademlia Lookup The initiator resends the FIND_NODE to nodes it has learned about from previous RPCs. If a round of FIND_NODES fails to return a node any closer than the closest already seen, the initiator resends the FIND_NODE to all of the k closest nodes it has not already queried. The lookup terminates when the initiator has queried and gotten responses from the k closest nodes it has seen.
58 Kademlia Keys Store To store a (key,value) pair, a participant locates the k closest nodes to the key and sends them STORE RPCs. Additionally, each node re-publishes (key,value) pairs as necessary to keep them alive. For Kademlia s file sharing application, the original publisher of a (key,value) pair is required to republish it every 24 hours. Otherwise, (key,value) pairs expire 24 hours after publication.
59 New node join Each node bootstraps by looking for its own ID Search recursively until no closer nodes can be found The nodes passed on the way are stored in the routing table
Kademlia: A P2P Informa2on System Based on the XOR Metric
Kademlia: A P2P Informa2on System Based on the XOR Metric Today! By Petar Mayamounkov and David Mazières, presented at IPTPS 22 Next! Paper presentation and discussion Image from http://www.vs.inf.ethz.ch/about/zeit.jpg
More informationPeer- Peer to -peer Systems
Peer-to-peer Systems Brian Nielsen bnielsen@cs.aau.dk Client-Server Centralized Bottleneck Functional specialization Single point of failure Central administration NFS Client NFS Client NFS Server NFS
More informationOverlay and P2P Networks. Structured Networks and DHTs. Prof. Sasu Tarkoma
Overlay and P2P Networks Structured Networks and DHTs Prof. Sasu Tarkoma 6.2.2014 Contents Today Semantic free indexing Consistent Hashing Distributed Hash Tables (DHTs) Thursday (Dr. Samu Varjonen) DHTs
More informationDistributed Hash Tables: Chord
Distributed Hash Tables: Chord Brad Karp (with many slides contributed by Robert Morris) UCL Computer Science CS M038 / GZ06 12 th February 2016 Today: DHTs, P2P Distributed Hash Tables: a building block
More informationCS514: Intermediate Course in Computer Systems
Distributed Hash Tables (DHT) Overview and Issues Paul Francis CS514: Intermediate Course in Computer Systems Lecture 26: Nov 19, 2003 Distributed Hash Tables (DHT): Overview and Issues What is a Distributed
More informationKademlia: A peer-to peer information system based on XOR. based on XOR Metric,by P. Maymounkov and D. Mazieres
: A peer-to peer information system based on XOR Metric,by P. Maymounkov and D. Mazieres March 10, 2009 : A peer-to peer information system based on XOR Features From past p2p experiences, it has been
More informationPeer-to-Peer Systems and Distributed Hash Tables
Peer-to-Peer Systems and Distributed Hash Tables CS 240: Computing Systems and Concurrency Lecture 8 Marco Canini Credits: Michael Freedman and Kyle Jamieson developed much of the original material. Selected
More informationToday. Why might P2P be a win? What is a Peer-to-Peer (P2P) system? Peer-to-Peer Systems and Distributed Hash Tables
Peer-to-Peer Systems and Distributed Hash Tables COS 418: Distributed Systems Lecture 7 Today 1. Peer-to-Peer Systems Napster, Gnutella, BitTorrent, challenges 2. Distributed Hash Tables 3. The Chord Lookup
More informationMarch 10, Distributed Hash-based Lookup. for Peer-to-Peer Systems. Sandeep Shelke Shrirang Shirodkar MTech I CSE
for for March 10, 2006 Agenda for Peer-to-Peer Sytems Initial approaches to Their Limitations CAN - Applications of CAN Design Details Benefits for Distributed and a decentralized architecture No centralized
More informationP2P Network Structured Networks: Distributed Hash Tables. Pedro García López Universitat Rovira I Virgili
P2P Network Structured Networks: Distributed Hash Tables Pedro García López Universitat Rovira I Virgili Pedro.garcia@urv.net Index Introduction to DHT s Origins of structured overlays Case studies Chord
More informationOverlay networks. To do. Overlay networks. P2P evolution DHTs in general, Chord and Kademlia. Turtles all the way down. q q q
Overlay networks To do q q q Overlay networks P2P evolution DHTs in general, Chord and Kademlia Turtles all the way down Overlay networks virtual networks Different applications with a wide range of needs
More informationCompSci 356: Computer Network Architectures Lecture 21: Overlay Networks Chap 9.4. Xiaowei Yang
CompSci 356: Computer Network Architectures Lecture 21: Overlay Networks Chap 9.4 Xiaowei Yang xwy@cs.duke.edu Overview Problem Evolving solutions IP multicast Proxy caching Content distribution networks
More informationContent Overlays. Nick Feamster CS 7260 March 12, 2007
Content Overlays Nick Feamster CS 7260 March 12, 2007 Content Overlays Distributed content storage and retrieval Two primary approaches: Structured overlay Unstructured overlay Today s paper: Chord Not
More informationDepartment of Computer Science Institute for System Architecture, Chair for Computer Networks. File Sharing
Department of Computer Science Institute for System Architecture, Chair for Computer Networks File Sharing What is file sharing? File sharing is the practice of making files available for other users to
More informationBadri Nath Rutgers University
lookup services Badri Nath Rutgers University badri@cs.rutgers.edu 1. CAN: A scalable content addressable network, Sylvia Ratnasamy et.al. SIGCOMM 2001 2. Chord: A scalable peer-to-peer lookup protocol
More informationDistributed Systems. 17. Distributed Lookup. Paul Krzyzanowski. Rutgers University. Fall 2016
Distributed Systems 17. Distributed Lookup Paul Krzyzanowski Rutgers University Fall 2016 1 Distributed Lookup Look up (key, value) Cooperating set of nodes Ideally: No central coordinator Some nodes can
More informationDistributed Hash Tables
Distributed Hash Tables CS6450: Distributed Systems Lecture 11 Ryan Stutsman Material taken/derived from Princeton COS-418 materials created by Michael Freedman and Kyle Jamieson at Princeton University.
More informationCS555: Distributed Systems [Fall 2017] Dept. Of Computer Science, Colorado State University
CS 555: DISTRIBUTED SYSTEMS [P2P SYSTEMS] Shrideep Pallickara Computer Science Colorado State University Frequently asked questions from the previous class survey Byzantine failures vs malicious nodes
More informationOverlay and P2P Networks. Structured Networks and DHTs. Prof. Sasu Tarkoma
Overlay and P2P Networks Structured Networks and DHTs Prof. Sasu Tarkoma 2.2.2015 Contents Today Geometries revisited Distributed Hash Tables (DHTs) Thursday Power-law networks Geometries revisited The
More informationReminder: Distributed Hash Table (DHT) CS514: Intermediate Course in Operating Systems. Several flavors, each with variants.
CS514: Intermediate Course in Operating Systems Professor Ken Birman Vivek Vishnumurthy: TA Reminder: Distributed Hash Table (DHT) A service, distributed over multiple machines, with hash table semantics
More informationScalability In Peer-to-Peer Systems. Presented by Stavros Nikolaou
Scalability In Peer-to-Peer Systems Presented by Stavros Nikolaou Background on Peer-to-Peer Systems Definition: Distributed systems/applications featuring: No centralized control, no hierarchical organization
More information: Scalable Lookup
6.824 2006: Scalable Lookup Prior focus has been on traditional distributed systems e.g. NFS, DSM/Hypervisor, Harp Machine room: well maintained, centrally located. Relatively stable population: can be
More informationChapter 10: Peer-to-Peer Systems
Chapter 10: Peer-to-Peer Systems From Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edition 4, Addison-Wesley 2005 Introduction To enable the sharing of data and resources
More informationOverlay networks. Today. l Overlays networks l P2P evolution l Pastry as a routing overlay example
Overlay networks Today l Overlays networks l P2P evolution l Pastry as a routing overlay eample Network virtualization and overlays " Different applications with a range of demands/needs network virtualization
More informationKen Birman. Cornell University. CS5410 Fall 2008.
Ken Birman Cornell University. CS5410 Fall 2008. What is a Distributed Hash Table (DHT)? Exactly that A service, distributed over multiple machines, with hash h table semantics Put(key, value), Value =
More informationMotivation for peer-to-peer
Peer-to-peer systems INF 5040 autumn 2015 lecturer: Roman Vitenberg INF5040, Frank Eliassen & Roman Vitenberg 1 Motivation for peer-to-peer Ø Inherent restrictions of the standard client/ server model
More informationDistributed lookup services
Distributed lookup services lookup services Badri Nath Rutgers University badri@cs.rutgers.edu A set of nodes cooperating Peers Run special purpose algorithms/software Doesn t have to be deployed at every
More informationOverlay networks. T o do. Overlay networks. P2P evolution DHTs in general, Chord and Kademlia. q q q. Turtles all the way down
Overlay networks T o do q q q Overlay networks P2P evolution DHTs in general, Chord and Kademlia Turtles all the way down Overlay networks virtual networks Different applications with a wide range of needs
More informationDistributed Hash Table
Distributed Hash Table P2P Routing and Searching Algorithms Ruixuan Li College of Computer Science, HUST rxli@public.wh.hb.cn http://idc.hust.edu.cn/~rxli/ In Courtesy of Xiaodong Zhang, Ohio State Univ
More informationCS4700/CS5700 Fundamentals of Computer Networks
CS4700/CS5700 Fundamentals of Computer Networks Lecture 22: Overlay networks Slides used with permissions from Edward W. Knightly, T. S. Eugene Ng, Ion Stoica, Hui Zhang Alan Mislove amislove at ccs.neu.edu
More informationLECT-05, S-1 FP2P, Javed I.
A Course on Foundations of Peer-to-Peer Systems & Applications LECT-, S- FPP, javed@kent.edu Javed I. Khan@8 CS /99 Foundation of Peer-to-Peer Applications & Systems Kent State University Dept. of Computer
More informationLecture 6: Overlay Networks. CS 598: Advanced Internetworking Matthew Caesar February 15, 2011
Lecture 6: Overlay Networks CS 598: Advanced Internetworking Matthew Caesar February 15, 2011 1 Overlay networks: Motivations Protocol changes in the network happen very slowly Why? Internet is shared
More informationPeer to peer systems: An overview
Peer to peer systems: An overview Gaurav Veda (Y1148) gveda@cse.iitk.ac.in Computer Science & Engineering Indian Institute of Technology Kanpur, UP, INDIA - 208016 Abstract Peer-to-peer (p2p) systems is
More informationTelematics Chapter 9: Peer-to-Peer Networks
Telematics Chapter 9: Peer-to-Peer Networks Beispielbild User watching video clip Server with video clips Application Layer Presentation Layer Application Layer Presentation Layer Session Layer Session
More informationDistributed Systems. 16. Distributed Lookup. Paul Krzyzanowski. Rutgers University. Fall 2017
Distributed Systems 16. Distributed Lookup Paul Krzyzanowski Rutgers University Fall 2017 1 Distributed Lookup Look up (key, value) Cooperating set of nodes Ideally: No central coordinator Some nodes can
More informationModern Technology of Internet
Modern Technology of Internet Jiří Navrátil, Josef Vojtěch, Jan Furman, Tomáš Košnar, Sven Ubik, Milan Šárek, Jan Růžička, Martin Pustka, Laban Mwansa, Rudolf Blažek Katedra počítačových systémů FIT České
More informationWinter CS454/ Assignment 2 Instructor: Bernard Wong Due date: March 12 15, 2012 Group size: 2
Winter CS454/654 2012 Assignment 2 Instructor: Bernard Wong Due date: March 12 15, 2012 Group size: 2 Distributed systems is a rapidly evolving field. Most of the systems we have studied (or will study)
More informationDistributed Hash Tables
Distributed Hash Tables Chord Smruti R. Sarangi Department of Computer Science Indian Institute of Technology New Delhi, India Smruti R. Sarangi Chord 1/29 Outline Overview 1 Overview 2 3 Smruti R. Sarangi
More informationModule SDS: Scalable Distributed Systems. Gabriel Antoniu, KERDATA & Davide Frey, ASAP INRIA
Module SDS: Scalable Distributed Systems Gabriel Antoniu, KERDATA & Davide Frey, ASAP INRIA Staff Gabriel Antoniu, DR INRIA, KERDATA Team gabriel.antoniu@inria.fr Davide Frey, CR INRIA, ASAP Team davide.frey@inria.fr
More informationPeer-to-Peer (P2P) Systems
Peer-to-Peer (P2P) Systems What Does Peer-to-Peer Mean? A generic name for systems in which peers communicate directly and not through a server Characteristics: decentralized self-organizing distributed
More informationP2P Network Structured Networks: Distributed Hash Tables. Pedro García López Universitat Rovira I Virgili
P2P Network Structured Networks: Distributed Hash Tables Pedro García López Universitat Rovira I Virgili Pedro.garcia@urv.net Index Description of CHORD s Location and routing mechanisms Symphony: Distributed
More informationinternet technologies and standards
Institute of Telecommunications Warsaw University of Technology 25 internet technologies and standards Piotr Gajowniczek Andrzej Bąk Michał Jarociński Internet application layer peer-to-peer systems overview
More informationDistributed Hash Tables
Smruti R. Sarangi Department of Computer Science Indian Institute of Technology New Delhi, India Smruti R. Sarangi 1/34 Outline 1 2 Smruti R. Sarangi 2/34 Normal Hashtables Hashtable : Contains a set of
More informationPeer to Peer Networks
Sungkyunkwan University Peer to Peer Networks Prepared by T. Le-Duc and H. Choo Copyright 2000-2018 Networking Laboratory P2P Applications Traditional P2P applications: for file sharing BitTorrent, Emule
More informationPeer-to-Peer Systems and Security IN2194. Chapter 1 Peer-to-Peer Systems 1.3 Structured Networks
Chair for Network Architectures and Services Department of Informatics TU München Prof. Carle Peer-to-Peer Systems and Security IN2194 Chapter 1 Peer-to-Peer Systems 1.3 Structured Networks Prof. Dr.-Ing.
More informationDistributed Hash Tables
KTH ROYAL INSTITUTE OF TECHNOLOGY Distributed Hash Tables Distributed Hash Tables Vladimir Vlassov Large scale data bases hundreds of servers High churn rate servers will come and go Benefits fault tolerant
More informationSearching for Shared Resources: DHT in General
1 ELT-53207 P2P & IoT Systems Searching for Shared Resources: DHT in General Mathieu Devos Tampere University of Technology Department of Electronics and Communications Engineering Based on the original
More informationChord : A Scalable Peer-to-Peer Lookup Protocol for Internet Applications
: A Scalable Peer-to-Peer Lookup Protocol for Internet Applications Ion Stoica, Robert Morris, David Liben-Nowell, David R. Karger, M. Frans Kaashock, Frank Dabek, Hari Balakrishnan March 4, 2013 One slide
More informationP2P: Distributed Hash Tables
P2P: Distributed Hash Tables Chord + Routing Geometries Nirvan Tyagi CS 6410 Fall16 Peer-to-peer (P2P) Peer-to-peer (P2P) Decentralized! Hard to coordinate with peers joining and leaving Peer-to-peer (P2P)
More informationStructured Peer-to-Peer Networks
Structured Peer-to-Peer Networks The P2P Scaling Problem Unstructured P2P Revisited Distributed Indexing Fundamentals of Distributed Hash Tables DHT Algorithms Chord Pastry Can Programming a DHT Graphics
More informationCS5412: TIER 2 OVERLAYS
1 CS5412: TIER 2 OVERLAYS Lecture VI Ken Birman Recap 2 A week ago we discussed RON and Chord: typical examples of P2P network tools popular in the cloud They were invented purely as content indexing systems,
More informationCIS 700/005 Networking Meets Databases
Announcements CIS / Networking Meets Databases Boon Thau Loo Spring Lecture Paper summaries due at noon today. Office hours: Wed - pm ( Levine) Project proposal: due Feb. Student presenter: rd Jan: A Scalable
More informationPeer-to-peer computing research a fad?
Peer-to-peer computing research a fad? Frans Kaashoek kaashoek@lcs.mit.edu NSF Project IRIS http://www.project-iris.net Berkeley, ICSI, MIT, NYU, Rice What is a P2P system? Node Node Node Internet Node
More informationSearching for Shared Resources: DHT in General
1 ELT-53206 Peer-to-Peer Networks Searching for Shared Resources: DHT in General Mathieu Devos Tampere University of Technology Department of Electronics and Communications Engineering Based on the original
More informationLecture-2 Content Sharing in P2P Networks Different P2P Protocols
Lecture-2 Content Sharing in P2P Networks Different P2P Protocols Zhou Shuigeng March 10, 2005 Outline Classification of content sharing P2P systems Content sharing P2P systems Napster; Gnutella; Freenet
More informationScaling Out Systems. COS 518: Advanced Computer Systems Lecture 6. Kyle Jamieson
Scaling Out Systems COS 518: Advanced Computer Systems Lecture 6 Kyle Jamieson [Credits: Selected content adapted from M. Freedman, B. Karp, R. Morris] Today 1. Scaling up: Reliable multicast 2. Scaling
More informationSlides for Chapter 10: Peer-to-Peer Systems. From Coulouris, Dollimore, Kindberg and Blair Distributed Systems: Concepts and Design
Slides for Chapter 10: Peer-to-Peer Systems From Coulouris, Dollimore, Kindberg and Blair Distributed Systems: Concepts and Design Edition 5, Addison-Wesley 2012 Edited and supplemented by Jonne Itkonen,!
More informationSlides for Chapter 10: Peer-to-Peer Systems
Slides for Chapter 10: Peer-to-Peer Systems From Coulouris, Dollimore, Kindberg and Blair Distributed Systems: Concepts and Design Edition 5, Addison-Wesley 2012 Overview of Chapter Introduction Napster
More informationGNUnet Distributed Data Storage
GNUnet Distributed Data Storage DHT and Distance Vector Transport Nathan S. Evans 1 1 Technische Universität München Department of Computer Science Network Architectures and Services July, 24 2010 Overview
More informationPart 1: Introducing DHTs
Uni Innsbruck Informatik - Peer-to to-peer Systems Structured PP file sharing systems Michael Welzl michael.welzl@uibk.ac.at DPS NSG Team http://dps.uibk.ac.at dps.uibk.ac.at/nsg Institute of Computer
More informationPEER-TO-PEER NETWORKS, DHTS, AND CHORD
PEER-TO-PEER NETWORKS, DHTS, AND CHORD George Porter May 25, 2018 ATTRIBUTION These slides are released under an Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) Creative Commons license
More informationL3S Research Center, University of Hannover
, University of Hannover Dynamics of Wolf-Tilo Balke and Wolf Siberski 21.11.2007 *Original slides provided by S. Rieche, H. Niedermayer, S. Götz, K. Wehrle (University of Tübingen) and A. Datta, K. Aberer
More informationINF5070 media storage and distribution systems. to-peer Systems 10/
INF5070 Media Storage and Distribution Systems: Peer-to to-peer Systems 10/11 2003 Client-Server! Traditional distributed computing! Successful architecture, and will continue to be so (adding proxy servers)!
More informationProject Guide for Kademlia
Project Guide for Kademlia Building Go's build tools depend on the value of the GOPATH environment variable. $GOPATH should be the project root: the absolute path of the directory containing {bin,pkg,src}.
More informationCycloid: A constant-degree and lookup-efficient P2P overlay network
Performance Evaluation xxx (2005) xxx xxx Cycloid: A constant-degree and lookup-efficient P2P overlay network Haiying Shen a, Cheng-Zhong Xu a,, Guihai Chen b a Department of Electrical and Computer Engineering,
More informationCPSC 426/526. P2P Lookup Service. Ennan Zhai. Computer Science Department Yale University
CPSC 4/5 PP Lookup Service Ennan Zhai Computer Science Department Yale University Recall: Lec- Network basics: - OSI model and how Internet works - Socket APIs red PP network (Gnutella, KaZaA, etc.) UseNet
More informationToday CSCI Coda. Naming: Volumes. Coda GFS PAST. Instructor: Abhishek Chandra. Main Goals: Volume is a subtree in the naming space
Today CSCI 5105 Coda GFS PAST Instructor: Abhishek Chandra 2 Coda Main Goals: Availability: Work in the presence of disconnection Scalability: Support large number of users Successor of Andrew File System
More informationA Scalable Content- Addressable Network
A Scalable Content- Addressable Network In Proceedings of ACM SIGCOMM 2001 S. Ratnasamy, P. Francis, M. Handley, R. Karp, S. Shenker Presented by L.G. Alex Sung 9th March 2005 for CS856 1 Outline CAN basics
More informationPeer-to-Peer Networks Pastry & Tapestry 4th Week
Peer-to-Peer Networks Pastry & Tapestry 4th Week Department of Computer Science 1 Peer-to-Peer Networks Pastry 2 2 Pastry Peter Druschel Rice University, Houston, Texas now head of Max-Planck-Institute
More informationR/Kademlia: Recursive and Topology-aware Overlay Routing
R/Kademlia: Recursive and Topology-aware Overlay Routing Bernhard Heep ATNAC 2010, Auckland, New Zealand, 11/02/2010, KIT University of the State of Baden-Wuerttemberg and National Research Center of the
More informationCSE 486/586 Distributed Systems
CSE 486/586 Distributed Systems Distributed Hash Tables Slides by Steve Ko Computer Sciences and Engineering University at Buffalo CSE 486/586 Last Time Evolution of peer-to-peer Central directory (Napster)
More informationL3S Research Center, University of Hannover
, University of Hannover Structured Peer-to to-peer Networks Wolf-Tilo Balke and Wolf Siberski 3..6 *Original slides provided by K. Wehrle, S. Götz, S. Rieche (University of Tübingen) Peer-to-Peer Systems
More informationCh06. NoSQL Part III.
Ch06. NoSQL Part III. Joonho Kwon Data Science Laboratory, PNU 2017. Spring Adapted from Dr.-Ing. Sebastian Michel s slides Recap: Configurations R/W Configuration Kind ofconsistency W=N and R=1 Read optimized
More informationDistributed File Systems: An Overview of Peer-to-Peer Architectures. Distributed File Systems
Distributed File Systems: An Overview of Peer-to-Peer Architectures Distributed File Systems Data is distributed among many sources Ex. Distributed database systems Frequently utilize a centralized lookup
More informationLecture 8: Application Layer P2P Applications and DHTs
Lecture 8: Application Layer P2P Applications and DHTs COMP 332, Spring 2018 Victoria Manfredi Acknowledgements: materials adapted from Computer Networking: A Top Down Approach 7 th edition: 1996-2016,
More informationSecuring Chord for ShadowWalker. Nandit Tiku Department of Computer Science University of Illinois at Urbana-Champaign
Securing Chord for ShadowWalker Nandit Tiku Department of Computer Science University of Illinois at Urbana-Champaign tiku1@illinois.edu ABSTRACT Peer to Peer anonymous communication promises to eliminate
More informationTracking Objects in Distributed Systems. Distributed Systems
Tracking Objects in Distributed Systems Arvind Krishnamurthy Fall 2004 Distributed Systems Consider the Shared virtual memory system (SVM) by Kai Li: It tracks pages in a cluster system Simplest version
More informationLast Time. CSE 486/586 Distributed Systems Distributed Hash Tables. What We Want. Today s Question. What We Want. What We Don t Want C 1
Last Time Distributed Systems Distributed Hash Tables Evolution of peer-to-peer Central directory (Napster) Query flooding (Gnutella) Hierarchical overlay (Kazaa, modern Gnutella) BitTorrent Focuses on
More informationC 1. Last Time. CSE 486/586 Distributed Systems Distributed Hash Tables. Today s Question. What We Want. What We Want. What We Don t Want
Last Time Distributed Systems Distributed Hash Tables Evolution of peer-to-peer Central directory (Napster) Query flooding (Gnutella) Hierarchical overlay (Kazaa, modern Gnutella) BitTorrent Focuses on
More informationAthens University of Economics and Business. Dept. of Informatics
Athens University of Economics and Business Athens University of Economics and Business Dept. of Informatics B.Sc. Thesis Project report: Implementation of the PASTRY Distributed Hash Table lookup service
More informationDATA. The main challenge in P2P computing is to design and implement LOOKING UP. in P2P Systems
LOOKING UP DATA in P2P Systems By Hari Balakrishnan, M. Frans Kaashoek, David Karger, Robert Morris, and Ion Stoica The main challenge in P2P computing is to design and implement a robust and scalable
More informationPeer-to-Peer Systems and Security IN2194. Chapter 1 Peer-to-Peer Systems 1.3 Structured Networks
Chair for Network Architectures and Services Department of Informatics TU München Prof. Carle Peer-to-Peer Systems and Security IN2194 Chapter 1 Peer-to-Peer Systems 1.3 Structured Networks Prof. Dr.-Ing.
More informationDistriubted Hash Tables and Scalable Content Adressable Network (CAN)
Distriubted Hash Tables and Scalable Content Adressable Network (CAN) Ines Abdelghani 22.09.2008 Contents 1 Introduction 2 2 Distributed Hash Tables: DHT 2 2.1 Generalities about DHTs............................
More informationPeer to Peer Networks
Sungkyunkwan University Peer to Peer Networks Prepared by T. Le-Duc and H. Choo Copyright 2000-2017 Networking Laboratory Presentation Outline 2.1 Introduction 2.2 Client-Server Paradigm 2.3 Peer-To-Peer
More informationToday s Objec2ves. Kerberos. Kerberos Peer To Peer Overlay Networks Final Projects
Today s Objec2ves Kerberos Peer To Peer Overlay Networks Final Projects Nov 27, 2017 Sprenkle - CSCI325 1 Kerberos Trusted third party, runs by default on port 88 Security objects: Ø Ticket: token, verifying
More informationEffects of Churn on Structured P2P Overlay Networks
International Conference on Automation, Control, Engineering and Computer Science (ACECS'14) Proceedings - Copyright IPCO-214, pp.164-17 ISSN 2356-568 Effects of Churn on Structured P2P Overlay Networks
More informationImproving the Dependability of Prefix-Based Routing in DHTs
Improving the Dependability of Prefix-Based Routing in DHTs Sabina Serbu, Peter Kropf and Pascal Felber University of Neuchâtel, CH-9, Neuchâtel, Switzerland {sabina.serbu, peter.kropf, pascal.felber}@unine.ch
More informationTinyTorrent: Implementing a Kademlia Based DHT for File Sharing
1 TinyTorrent: Implementing a Kademlia Based DHT for File Sharing A CS244B Project Report By Sierra Kaplan-Nelson, Jestin Ma, Jake Rachleff {sierrakn, jestinm, jakerach}@cs.stanford.edu Abstract We implemented
More informationPastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems
Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems Antony Rowstron 1 and Peter Druschel 2 1 Microsoft Research Ltd, St. George House, Guildhall Street, Cambridge,
More informationDebunking some myths about structured and unstructured overlays
Debunking some myths about structured and unstructured overlays Miguel Castro Manuel Costa Antony Rowstron Microsoft Research, 7 J J Thomson Avenue, Cambridge, UK Abstract We present a comparison of structured
More informationDistributed hash table - Wikipedia, the free encyclopedia
Page 1 sur 6 Distributed hash table From Wikipedia, the free encyclopedia Distributed hash tables (DHTs) are a class of decentralized distributed systems that provide a lookup service similar to a hash
More informationCS 43: Computer Networks BitTorrent & Content Distribution. Kevin Webb Swarthmore College September 28, 2017
CS 43: Computer Networks BitTorrent & Content Distribution Kevin Webb Swarthmore College September 28, 2017 Agenda BitTorrent Cooperative file transfers Briefly: Distributed Hash Tables Finding things
More informationCS 640 Introduction to Computer Networks. Today s lecture. What is P2P? Lecture30. Peer to peer applications
Introduction to Computer Networks Lecture30 Today s lecture Peer to peer applications Napster Gnutella KaZaA Chord What is P2P? Significant autonomy from central servers Exploits resources at the edges
More informationDistributed Systems Peer-to-Peer Systems
Distributed Systems Peer-to-Peer Systems [2] P2P Systems - Goals and Definition Goal: to enable sharing of data and resources on a very large scale by eliminating any requirement for separately-managed
More informationFinding Data in the Cloud using Distributed Hash Tables (Chord) IBM Haifa Research Storage Systems
Finding Data in the Cloud using Distributed Hash Tables (Chord) IBM Haifa Research Storage Systems 1 Motivation from the File Systems World The App needs to know the path /home/user/my pictures/ The Filesystem
More informationChapter 6 PEER-TO-PEER COMPUTING
Chapter 6 PEER-TO-PEER COMPUTING Distributed Computing Group Computer Networks Winter 23 / 24 Overview What is Peer-to-Peer? Dictionary Distributed Hashing Search Join & Leave Other systems Case study:
More informationOverview. Peer-to-Peer Systems and Security IN2194. Chapter 1 Peer-to-Peer Systems 1.3 Structured Networks. Motivation. Routing on the Internet
Chair for Network Architectures and Services Department of Informatics TU München Prof. Carle Peer-to-Peer Systems and Security IN2194 Chapter 1 Peer-to-Peer Systems 1.3 Structured Networks Prof. Dr.-Ing.
More informationLocality in Structured Peer-to-Peer Networks
Locality in Structured Peer-to-Peer Networks Ronaldo A. Ferreira Suresh Jagannathan Ananth Grama Department of Computer Sciences Purdue University 25 N. University Street West Lafayette, IN, USA, 4797-266
More informationBuilding a low-latency, proximity-aware DHT-based P2P network
Building a low-latency, proximity-aware DHT-based P2P network Ngoc Ben DANG, Son Tung VU, Hoai Son NGUYEN Department of Computer network College of Technology, Vietnam National University, Hanoi 144 Xuan
More informationPEER-TO-PEER (P2P) systems are now one of the most
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 25, NO. 1, JANUARY 2007 15 Enhancing Peer-to-Peer Systems Through Redundancy Paola Flocchini, Amiya Nayak, Senior Member, IEEE, and Ming Xie Abstract
More information