CS 347 Parallel and Distributed Data Processing
|
|
- Eustacia Wilcox
- 5 years ago
- Views:
Transcription
1 CS 347 Parallel and Distributed Data Processing Spring 2016 Notes 9: Peer-to-Peer Systems
2 Previous Topics Data Database design Queries Query processing Localization Operators Optimization Transactions Concurrency control Reliability Replication CS 347 Notes 9 2
3 Previous Topics Data Database design Queries Query processing Localization Operators Optimization Transactions Concurrency control Reliability Replication Client-server architecture Relational data Good understanding of What the data is Where the data is CS 347 Notes 9 3
4 Next Topic Client-server architecture Relational data Good understanding of What the data is Where the data is? CS 347 Notes 9 4
5 Peer-to-Peer Systems CS 347 Notes 9 5
6 Peer-to-Peer Systems Distributed applications where nodes are Autonomous Very loosely coupled Equal in role or functionality Sharing & exchanging resources with each other CS 347 Notes 9 6
7 Peer-to-Peer Systems Related concepts File sharing P2P is one option Grid computing Focus is on computing Autonomic computing Focus is on self-management CS 347 Notes 9 7
8 Peer-to-Peer Systems Search Essential problem to solve Query Who has X? Node 3 Resources R 3,1, R 3,2,... Node 2 Resources R 2,1, R 2,2,... Node 1 Resources R 1,1, R 1,2,... CS 347 Notes 9 8
9 Peer-to-Peer Systems Search Node 3 Resources R 3,1, R 3,2,... Query Who has X? Node 2 Resources R 2,1, R 2,2,... Answers Node 1 Resources R 1,1, R 1,2,... CS 347 Notes 9 9
10 Peer-to-Peer Systems Search Node 3 Resources R 3,1, R 3,2,... Query Who has X? Node 2 Resources R 2,1, R 2,2,... Answers Request resource Node 1 Resources R 1,1, R 1,2,... Provide resource CS 347 Notes 9 10
11 Distributed Lookup Have < k, v > pairs, each of n nodes holds some pairs Given key k, find matching values { v 1, v 2, } k v 1 a 1 b 4 a 7 c 3 a 1 a 4 d lookup(4) = { a, d } CS 347 Notes 9 11
12 Distributed Lookup Communication overlay network Structured (+) Efficient distributed lookup Unstructured (+) Easy, robust Can handle high churn rates ( ) Flood queries Hybrid E.g., centralized search, decentralized exchange CS 347 Notes 9 12
13 Distributed Lookup Distributed hashing Most common way to create a structured overlay network H(k) is an m-bit number (k is a key) H(X) is an m-bit number (X is a node identifier) Hash function H is good H(k) 0 2 m - 1 k v stored at Y H(X) H(Y) CS 347 Notes 9 13
14 Distributed Lookup Distributed hashing Two approaches Chord Replicated hash table (RHT) CS 347 Notes 9 14
15 Chord The Chord circle N1 m = 6 N51 N48 N56 K54 K10 N8 N14 N42 N21 K38 Using hashed values E.g., N56 is node with id hashing to 56 N38 N32 K24, K30 CS 347 Notes 9 15
16 Chord Ownership rule Consider nodes X, Y such that Y follows X clockwise Node Y owns keys k such that H(k) in (H(X), H(Y)] N54 N3 Stores K55, K56,, K3 CS 347 Notes 9 16
17 Chord Notation X.function( ) (remote) calls function at X X.A returns value A at X If X omitted, refers to current node CS 347 Notes 9 17
18 Chord Successor/predecessor links N1.pred N1 m = 6 N51 N48 N56 N1.succ N8 N14 N42 N21 N38 N32 CS 347 Notes 9 18
19 Chord Search for owner using successor links X.find_succ(k): if k in (pred, X] return X else if k in (X, succ] return succ else return succ.find_succ(k) CS 347 Notes 9 19
20 Chord Value lookup X.lookup(k): Y := X.find_succ(k) return Y.get_value(k) X.get_value(k): // Return local value v for k, if it exists CS 347 Notes 9 20
21 Chord Example Searching for K52 N56 N1 N8 m = 6 N51.find_succ(K52) = N56 N51 N48 N14 N14.find_succ(K52) N42 N21 N38 N32 CS 347 Notes 9 21
22 Chord Finger table N1 m = 6 N56 N51 N48 N42 N8 N14 N21 Finger table for N8 N8 + 1 N14 N8 + 2 N14 N8 + 4 N14 N8 + 8 N21 N N32 N N42 N38 N32 CS 347 Notes 9 22
23 Chord Finger table N1 m = 6 N56 N51 N48 N42 N8 N14 N21 Finger table for N8 N8 + 1 N14 N8 + 2 N14 N8 + 4 N14 N8 + 8 N21 N N32 N N42 N38 N32 Node that owns key = 40 CS 347 Notes 9 23
24 Chord Search using the finger table X.find_succ(k): if k in (pred, X] return X if k in (X, succ] return succ else Y := closest_preceeding(k) return Y.find_succ(k) X.closest_preceeding(k): for i := m downto 1 if finger[i] in (X, k] return finger[i] return nil CS 347 Notes 9 24
25 Chord Example Looking up K54 N56 N1 N8 N8.find_succ(K54) m = 6 N51 N48 N14 N42.find_succ(K54) N42 N21 N38 N32 CS 347 Notes 9 25
26 Chord Example Looking up K54 N56 N1 N8 m = 6 N51.find_succ(K54) N51 N48 N14 N42.find_succ(K54) N42 N21 N38 N32 CS 347 Notes 9 26
27 Chord Example Looking up K54 N56 N1 N8 N8.find_succ(K54) m = 6 N51.find_succ(K54) N51 N48 N14 N42.find_succ(K54) N42 N21 N38 N32 CS 347 Notes 9 27
28 Chord Adding nodes N1 m = 6 Need to 1. Update links 2. Move data N51 N48 N56 N8 N14 N42 Joining node N26 N21 N38 For now, assume nodes never die N32 CS 347 Notes 9 28
29 Chord Adding nodes X.join(Y): // Node Y is known to belong to the circle pred := nil; succ := Y.find_succ(X); CS 347 Notes 9 29
30 Chord Periodic stabilization X.stabilize(): Y := succ.pred if Y in (X, succ) succ := Y succ.notify(x) X.notify(Y): if pred = nil or Y in (pred, X) pred := Y CS 347 Notes 9 30
31 Chord Join example Before updating links N x N p N s CS 347 Notes 9 31
32 Chord Join example After N x.join() nil N x N p N s CS 347 Notes 9 32
33 Chord Join example After N x.stabilize() nil N x N p N s CS 347 Notes 9 33
34 Chord Join example After N p.stabilize() N x N p N s Exercise: fix finger table CS 347 Notes 9 34
35 Chord Moving data When? Keys in (N p, N x ] N x Keys in (N p, N s ] N p N s CS 347 Notes 9 35
36 Chord Moving data After N s.notify(n x ) nil N x N p N s Send all keys in (N p, N x ] when N s.pred gets updated CS 347 Notes 9 36
37 Chord Moving data Revised notify() X.notify(Y): if pred = nil or Y in (pred, X) Y.add(data in (pred, Y]) temp := pred pred := Y X.remove(data in (temp, Y]) Glossing over concurrency issues E.g., what happens to lookups while moving data? CS 347 Notes 9 37
38 Chord Moving data Revised notify() Exercise: when pred = nil, what data gets moved? CS 347 Notes 9 38
39 Chord Moving data Lookup at wrong node nil Keys in (N p, N x ] N x N p Keys in (N x, N s ] N s Lookup for all k in (N p, N x ] directed to N s CS 347 Notes 9 39
40 Chord Moving data Revised lookup() X.lookup(k): X.get_value(k): ok := false if k in (pred, X] while not ok return [true, value for k] Y := X.find_succ(k) else [ok, v] := Y.get_value(k) return [false, nil] return v CS 347 Notes 9 40
41 Chord Moving data Revised lookup() works pred, succ links eventually correct Data ends up at correct node Finger pointers speed up searches, but do not cause problems CS 347 Notes 9 41
42 Chord Performance n = number of live nodes With high probability, the number of nodes that must be contacted to find a successor is O(log n) Although finger table contains room for m entries, only O(log n) need to be stored Experimental results show average lookup time is ~(log n)/2 CS 347 Notes 9 42
43 Chord Node failures k7 k8 k9 v7 v8 v9 N x N p N s Assume N x dies Links become invalid Data gets lost CS 347 Notes 9 43
44 Chord Node failures Fixing links X.check_pred(): if pred failed pred := nil Also, keep backup links to s > 1 successors CS 347 Notes 9 44
45 Chord Node failure example State before failure N x N p N s backup succ link (s = 2) CS 347 Notes 9 45
46 Chord Node failure example After failure After N s.check_pred() N p Ns nil CS 347 Notes 9 46
47 Chord Node failure example After N p discovers that N x is down N p Ns nil CS 347 Notes 9 47
48 Chord Node failure example After stabilization N p Ns CS 347 Notes 9 48
49 Chord Node failure Protection from data loss: e.g., robust nodes (see replication notes) Robust node X Takeover protocols Node X 1 Node X 2 Node X 3 k7 v7 k7 v7 k7 v7 k8 v8 k8 v8 k8 v8 k9 v9 k9 v9 k9 v9 Backup protocols CS 347 Notes 9 49
50 Chord Summary Finding owner nodes Successor and predecessor links Finger table Adding nodes Updating links Moving data Coping with node failures Fixing links Protecting from data loss Performance CS 347 Notes 9 50
51 Replicated Hash Table Node N0 Node N1 Node N2 Node N3 Hash Node Hash Node Hash Node Hash Node 0 N0 0 N0 0 N0 0 N0 1 N1 1 N1 1 N1 1 N1 2 N2 2 N2 2 N2 2 N2 3 N3 3 N3 3 N3 3 N3 Data for keys that hash to 0 Data for keys that hash to 1 Data for keys that hash to 2 Data for keys that hash to 3 CS 347 Notes 9 51
52 Replicated Hash Table Adding nodes E.g., N0 overloaded Node N0 Hash Node 0 N0 1 N0 2 N2 3 N2 Data for keys that hash to 0,1 Node N2 Hash Node 0 N0 1 N0 2 N2 3 N2 Data for keys that hash to 2,3 CS 347 Notes 9 52
53 Replicated Hash Table Adding nodes First, set up N1 Node N0 Hash Node 0 N0 1 N0 2 N2 3 N2 Data for keys that hash to 0,1 Node N1 Hash Node 0 N0 1 N0 2 N2 3 N2 Node N2 Hash Node 0 N0 1 N0 2 N2 3 N2 Data for keys that hash to 2,3 CS 347 Notes 9 53
54 Replicated Hash Table Adding nodes Next, copy data to N1 Node N0 Node N1 Node N2 Hash Node Hash Node Hash Node 0 N0 0 N0 0 N0 1 N0 1 N0 1 N0 2 N2 2 N2 2 N2 3 N2 3 N2 3 N2 Data for keys that hash to 0,1 Data copy Data for keys that hash to 1 Data for keys that hash to 2,3 CS 347 Notes 9 54
55 Replicated Hash Table Adding nodes Next, change control Node N0 Node N1 Node N2 Hash Node Hash Node Hash Node 0 N0 0 N0 0 N0 1 N0 N1 1 N0 N1 1 N0 2 N2 2 N2 2 N2 3 N2 3 N2 3 N2 Data for keys that hash to 0,1 Data for keys that hash to 1 Data for keys that hash to 2,3 CS 347 Notes 9 55
56 Replicated Hash Table Adding nodes Next, remove data from N0 Node N0 Hash Node 0 N0 1 N1 2 N2 3 N2 Data for keys that hash to 0 Node N1 Hash Node 0 N0 1 N1 2 N2 3 N2 Data for keys that hash to 1 Node N2 Hash Node 0 N0 1 N0 2 N2 3 N2 Data for keys that hash to 2,3 CS 347 Notes 9 56
57 Replicated Hash Table Adding nodes Finally, update other nodes Eagerly by N0 or N1? Lazily during future lookups? Node N0 Node N1 Node N2 Hash Node Hash Node Hash Node 0 N0 0 N0 0 N0 1 N1 1 N1 1 N0 N1 2 N2 2 N2 2 N2 3 N2 3 N2 3 N2 Data for keys that hash to 0 Data for keys that hash to 1 Data for keys that hash to 2,3 CS 347 Notes 9 57
58 Replicated Hash Table Adding nodes What about inserts while adding a node (e.g., during copy)? Apply at N0 then copy? Apply at both? Redirect to N1? Node N0 Node N1 Node N2 Hash Node Hash Node Hash Node 0 N0 0 N0 0 N0 1 N0 1 N0 1 N0 2 N2 2 N2 2 N2 3 N2 3 N2 3 N2 Data for keys that hash to 0,1 Data copy Data for keys that hash to 1 Data for keys that hash to 2,3 CS 347 Notes 9 58
59 Chord vs. Replicated Hash Table Which is simpler to implement? Cost of operations Looking up values: O(log n) vs. O(1) Adding nodes Recovering from node failures Storage cost Routing table size: log n vs. n CS 347 Notes 9 59
60 Distributed Lookup Communication overlay network Structured Chord Replicated hash table Unstructured Neighborhood search CS 347 Notes 9 60
61 Neighborhood Search Each node stores its own data Searches nearby nodes E.g., Gnutella Key Value Key Value Key Value 41 g 99 c 14 d 12 a 7 b 13 c 25 a 47 f 12 d 51 x 9 y CS 347 Notes 9 61
62 Neighborhood Search Example 41 g 13 x 41 g 13 c 14 d 13 f 12 c 15 f 12 c 14 d 41 g 13 c 14 d 12 a 7 b 13 c 25 a 47 f 12 d 25 x 9 y 13 d 12 c 41 g 13 f 14 d Node N1 N1.lookup(13), TTL = 4 13 t 1 d 44 s 5 a CS 347 Notes 9 62
63 Neighborhood Search Example 41 g 13 x 41 g 13 c 14 d 13 f 12 c 15 f 12 c 14 d 41 g 13 c 14 d TTL = 1 12 a 7 b 13 c 25 a 47 f 12 d 25 x 9 y 13 d 12 c TTL = 2 41 g 13 f 14 d TTL = 3 Node N1 N1.lookup(13), TTL = 4 Answer so far = { c, f, x } 13 t 1 d 44 s 5 a CS 347 Notes 9 63
64 Neighborhood Search Example 41 g 13 x 41 g 13 c 14 d 13 f 12 c 15 f 12 c 14 d 41 g 13 c 14 d 12 a 7 b 13 c 25 a 47 f 12 d 25 x 9 y 13 d 12 c 41 g 13 f 14 d Node N1 N1.lookup(13), TTL = 4 Answer so far = { c, d, f, x } 13 t 1 d 44 s 5 a Incorrect/incomplete CS 347 Notes 9 64
65 Neighborhood Search Optimization Queries have unique identifiers Nodes keep cache of recent queries (query identifier and TTL) CS 347 Notes 9 65
66 Neighborhood Search Example [ 77, 1 ] 41 g 13 x 41 g 13 c 14 d 13 f 12 c 15 f 12 c [ 77, 2 ] 14 d 41 g 13 c 14 d TTL = 1 Do not reprocess a 7 b 13 c 25 a 47 f 12 d 25 x 9 y 13 d 12 c TTL = 2 41 g 13 f 14 d TTL = 3 Node N1 [ 77, 4 ] N1.lookup(13), TTL = 4, id = t 1 d 44 s 5 a [ 77, 3 ] CS 347 Notes 9 66
67 Neighborhood Search Example TTL = 1 [ 77, 1 ] 41 g 13 x 41 g 13 c 14 d 13 f 12 c 15 f 12 c 14 d [ 77, 2 ] TTL = 2 41 g 13 c 14 d TTL = 3 12 a 7 b 13 c 25 a 47 f 12 d 25 x 9 y 13 d 12 c TTL = 2 Do not reprocess g 13 f 14 d Node N1 [ 77, 4 ] N1.lookup(13), TTL = 4, id = t 1 d 44 s 5 a [ 77, 3 ] CS 347 Notes 9 67
68 Neighborhood Search Bootstrapping Bootstrap server Add to S Known nodes S = N1, N2, Get neighbors N5 N1 N2 N7 CS 347 Notes 9 68
69 Neighborhood Search Problems Unnecessary messages High load and traffic E.g., if nodes have p neighbors, each search ~p TTL messages Low capacity nodes are a bottleneck May not find all answers Network Area searched TTL CS 347 Notes 9 69
70 Neighborhood Search Advantages Can handle complex queries Simple, robust algorithm Works well if data is highly replicated Network Area searched TTL Nodes with data CS 347 Notes 9 70
71 Open Problems Performance Availability Efficiency Load balancing Authenticity DoS prevention Incentives Anonymity Correctness Participation CS 347 Notes 9 71
72 Peer-to-Peer Systems Summary Structured networks Chord Replicated hash table Unstructured networks Neighborhood search Open problems CS 347 Notes 9 72
Chord: A Scalable Peer-to-peer Lookup Service For Internet Applications
Chord: A Scalable Peer-to-peer Lookup Service For Internet Applications Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan Presented by Jibin Yuan ION STOICA Professor of CS
More informationChord : A Scalable Peer-to-Peer Lookup Protocol for Internet Applications
: A Scalable Peer-to-Peer Lookup Protocol for Internet Applications Ion Stoica, Robert Morris, David Liben-Nowell, David R. Karger, M. Frans Kaashock, Frank Dabek, Hari Balakrishnan March 4, 2013 One slide
More informationPeer to Peer I II 1 CS 138. Copyright 2015 Thomas W. Doeppner, Rodrigo Fonseca. All rights reserved.
Peer to Peer I II 1 Roadmap This course will feature key concepts in Distributed Systems, often illustrated by their use in example systems Start with Peer-to-Peer systems, which will be useful for your
More informationChord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications
Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications Amir H. Payberah (amir@sics.se) 1/71 Recap 2/71 Distributed Hash Tables (DHT) An ordinary hash table, which is... Key Fatemeh Sarunas
More informationChord. Advanced issues
Chord Advanced issues Analysis Theorem. Search takes O (log N) 5me (Note that in general, when there are m fingers, 2 m may be much larger than N) Proof. A(er log N forwarding steps, distance to key is
More informationCS 640 Introduction to Computer Networks. Today s lecture. What is P2P? Lecture30. Peer to peer applications
Introduction to Computer Networks Lecture30 Today s lecture Peer to peer applications Napster Gnutella KaZaA Chord What is P2P? Significant autonomy from central servers Exploits resources at the edges
More informationCIS 700/005 Networking Meets Databases
Announcements CIS / Networking Meets Databases Boon Thau Loo Spring Lecture Paper summaries due at noon today. Office hours: Wed - pm ( Levine) Project proposal: due Feb. Student presenter: rd Jan: A Scalable
More informationL3S Research Center, University of Hannover
, University of Hannover Dynamics of Wolf-Tilo Balke and Wolf Siberski 21.11.2007 *Original slides provided by S. Rieche, H. Niedermayer, S. Götz, K. Wehrle (University of Tübingen) and A. Datta, K. Aberer
More informationP2P Network Structured Networks: Distributed Hash Tables. Pedro García López Universitat Rovira I Virgili
P2P Network Structured Networks: Distributed Hash Tables Pedro García López Universitat Rovira I Virgili Pedro.garcia@urv.net Index Introduction to DHT s Origins of structured overlays Case studies Chord
More information*Adapted from slides provided by Stefan Götz and Klaus Wehrle (University of Tübingen)
Distributed Hash Tables (DHT) Jukka K. Nurminen *Adapted from slides provided by Stefan Götz and Klaus Wehrle (University of Tübingen) The Architectures of st and nd Gen. PP Client-Server Peer-to-Peer.
More informationA Survey of Peer-to-Peer Content Distribution Technologies
A Survey of Peer-to-Peer Content Distribution Technologies Stephanos Androutsellis-Theotokis and Diomidis Spinellis ACM Computing Surveys, December 2004 Presenter: Seung-hwan Baek Ja-eun Choi Outline Overview
More informationCompSci 356: Computer Network Architectures Lecture 21: Overlay Networks Chap 9.4. Xiaowei Yang
CompSci 356: Computer Network Architectures Lecture 21: Overlay Networks Chap 9.4 Xiaowei Yang xwy@cs.duke.edu Overview Problem Evolving solutions IP multicast Proxy caching Content distribution networks
More informationDistributed File Systems: An Overview of Peer-to-Peer Architectures. Distributed File Systems
Distributed File Systems: An Overview of Peer-to-Peer Architectures Distributed File Systems Data is distributed among many sources Ex. Distributed database systems Frequently utilize a centralized lookup
More informationCPSC 426/526. P2P Lookup Service. Ennan Zhai. Computer Science Department Yale University
CPSC 4/5 PP Lookup Service Ennan Zhai Computer Science Department Yale University Recall: Lec- Network basics: - OSI model and how Internet works - Socket APIs red PP network (Gnutella, KaZaA, etc.) UseNet
More informationPeer-to-Peer Systems and Distributed Hash Tables
Peer-to-Peer Systems and Distributed Hash Tables CS 240: Computing Systems and Concurrency Lecture 8 Marco Canini Credits: Michael Freedman and Kyle Jamieson developed much of the original material. Selected
More information*Adapted from slides provided by Stefan Götz and Klaus Wehrle (University of Tübingen)
Distributed Hash Tables (DHT) Jukka K. Nurminen *Adapted from slides provided by Stefan Götz and Klaus Wehrle (University of Tübingen) The Architectures of st and nd Gen. PP Client-Server Peer-to-Peer.
More informationDistributed Hash Tables (DHT)
Distributed Hash Tables (DHT) Jukka K. Nurminen Aalto University *Adapted from slides provided by Stefan Götz and Klaus Wehrle (University of Tübingen) The Architectures of st and nd Gen. PP Client-Server
More informationPage 1. How Did it Start?" Model" Main Challenge" CS162 Operating Systems and Systems Programming Lecture 24. Peer-to-Peer Networks"
How Did it Start?" CS162 Operating Systems and Systems Programming Lecture 24 Peer-to-Peer Networks" A killer application: Napster (1999) Free music over the Internet Key idea: share the storage and bandwidth
More informationArchitectures for Distributed Systems
Distributed Systems and Middleware 2013 2: Architectures Architectures for Distributed Systems Components A distributed system consists of components Each component has well-defined interface, can be replaced
More informationPeer-to-Peer (P2P) Systems
Peer-to-Peer (P2P) Systems What Does Peer-to-Peer Mean? A generic name for systems in which peers communicate directly and not through a server Characteristics: decentralized self-organizing distributed
More informationDistributed Hash Table
Distributed Hash Table P2P Routing and Searching Algorithms Ruixuan Li College of Computer Science, HUST rxli@public.wh.hb.cn http://idc.hust.edu.cn/~rxli/ In Courtesy of Xiaodong Zhang, Ohio State Univ
More informationLECT-05, S-1 FP2P, Javed I.
A Course on Foundations of Peer-to-Peer Systems & Applications LECT-, S- FPP, javed@kent.edu Javed I. Khan@8 CS /99 Foundation of Peer-to-Peer Applications & Systems Kent State University Dept. of Computer
More informationContent Overlays. Nick Feamster CS 7260 March 12, 2007
Content Overlays Nick Feamster CS 7260 March 12, 2007 Content Overlays Distributed content storage and retrieval Two primary approaches: Structured overlay Unstructured overlay Today s paper: Chord Not
More informationPeer-to-Peer Systems. Chapter General Characteristics
Chapter 2 Peer-to-Peer Systems Abstract In this chapter, a basic overview is given of P2P systems, architectures, and search strategies in P2P systems. More specific concepts that are outlined include
More informationEE 122: Peer-to-Peer (P2P) Networks. Ion Stoica November 27, 2002
EE 122: Peer-to-Peer (P2P) Networks Ion Stoica November 27, 22 How Did it Start? A killer application: Naptser - Free music over the Internet Key idea: share the storage and bandwidth of individual (home)
More informationFinding Data in the Cloud using Distributed Hash Tables (Chord) IBM Haifa Research Storage Systems
Finding Data in the Cloud using Distributed Hash Tables (Chord) IBM Haifa Research Storage Systems 1 Motivation from the File Systems World The App needs to know the path /home/user/my pictures/ The Filesystem
More informationFinding Data in the Cloud using Distributed Hash Tables (Chord) IBM Haifa Research Storage Systems
Finding Data in the Cloud using Distributed Hash Tables (Chord) IBM Haifa Research Storage Systems 1 In File Systems The App needs to know the path /home/user/my pictures/ The Filesystem looks up the directory
More informationP2P: Distributed Hash Tables
P2P: Distributed Hash Tables Chord + Routing Geometries Nirvan Tyagi CS 6410 Fall16 Peer-to-peer (P2P) Peer-to-peer (P2P) Decentralized! Hard to coordinate with peers joining and leaving Peer-to-peer (P2P)
More informationScalability In Peer-to-Peer Systems. Presented by Stavros Nikolaou
Scalability In Peer-to-Peer Systems Presented by Stavros Nikolaou Background on Peer-to-Peer Systems Definition: Distributed systems/applications featuring: No centralized control, no hierarchical organization
More informationPEER-TO-PEER NETWORKS, DHTS, AND CHORD
PEER-TO-PEER NETWORKS, DHTS, AND CHORD George Porter May 25, 2018 ATTRIBUTION These slides are released under an Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) Creative Commons license
More informationPeer-to-Peer Systems. Network Science: Introduction. P2P History: P2P History: 1999 today
Network Science: Peer-to-Peer Systems Ozalp Babaoglu Dipartimento di Informatica Scienza e Ingegneria Università di Bologna www.cs.unibo.it/babaoglu/ Introduction Peer-to-peer (PP) systems have become
More informationPage 1. Key Value Storage"
Key Value Storage CS162 Operating Systems and Systems Programming Lecture 14 Key Value Storage Systems March 12, 2012 Anthony D. Joseph and Ion Stoica http://inst.eecs.berkeley.edu/~cs162 Handle huge volumes
More informationPeer to Peer Networks
Sungkyunkwan University Peer to Peer Networks Prepared by T. Le-Duc and H. Choo Copyright 2000-2018 Networking Laboratory P2P Applications Traditional P2P applications: for file sharing BitTorrent, Emule
More informationLecture 6: Overlay Networks. CS 598: Advanced Internetworking Matthew Caesar February 15, 2011
Lecture 6: Overlay Networks CS 598: Advanced Internetworking Matthew Caesar February 15, 2011 1 Overlay networks: Motivations Protocol changes in the network happen very slowly Why? Internet is shared
More informationIntroduction to Peer-to-Peer Systems
Introduction Introduction to Peer-to-Peer Systems Peer-to-peer (PP) systems have become extremely popular and contribute to vast amounts of Internet traffic PP basic definition: A PP system is a distributed
More informationLast Time. CSE 486/586 Distributed Systems Distributed Hash Tables. What We Want. Today s Question. What We Want. What We Don t Want C 1
Last Time Distributed Systems Distributed Hash Tables Evolution of peer-to-peer Central directory (Napster) Query flooding (Gnutella) Hierarchical overlay (Kazaa, modern Gnutella) BitTorrent Focuses on
More informationEE 122: Peer-to-Peer Networks
EE 122: Peer-to-Peer Networks Ion Stoica (and Brighten Godfrey) TAs: Lucian Popa, David Zats and Ganesh Ananthanarayanan http://inst.eecs.berkeley.edu/~ee122/ (Materials with thanks to Vern Paxson, Jennifer
More informationChord: A Scalable Peer-to-peer Lookup Service for Internet Applications
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan Presented by Veranika Liaukevich Jacobs University
More informationMarch 10, Distributed Hash-based Lookup. for Peer-to-Peer Systems. Sandeep Shelke Shrirang Shirodkar MTech I CSE
for for March 10, 2006 Agenda for Peer-to-Peer Sytems Initial approaches to Their Limitations CAN - Applications of CAN Design Details Benefits for Distributed and a decentralized architecture No centralized
More informationFlooded Queries (Gnutella) Centralized Lookup (Napster) Routed Queries (Freenet, Chord, etc.) Overview N 2 N 1 N 3 N 4 N 8 N 9 N N 7 N 6 N 9
Peer-to-Peer Networks -: Computer Networking L-: PP Typically each member stores/provides access to content Has quickly grown in popularity Bulk of traffic from/to CMU is Kazaa! Basically a replication
More informationC 1. Last Time. CSE 486/586 Distributed Systems Distributed Hash Tables. Today s Question. What We Want. What We Want. What We Don t Want
Last Time Distributed Systems Distributed Hash Tables Evolution of peer-to-peer Central directory (Napster) Query flooding (Gnutella) Hierarchical overlay (Kazaa, modern Gnutella) BitTorrent Focuses on
More informationGoals. EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks. Solution. Overlay Networks: Motivations.
Goals CS : Introduction to Computer Networks Overlay Networks and PP Networks Ion Stoica Computer Science Division Department of lectrical ngineering and Computer Sciences University of California, Berkeley
More informationToday. Why might P2P be a win? What is a Peer-to-Peer (P2P) system? Peer-to-Peer Systems and Distributed Hash Tables
Peer-to-Peer Systems and Distributed Hash Tables COS 418: Distributed Systems Lecture 7 Today 1. Peer-to-Peer Systems Napster, Gnutella, BitTorrent, challenges 2. Distributed Hash Tables 3. The Chord Lookup
More informationDISTRIBUTED COMPUTER SYSTEMS ARCHITECTURES
DISTRIBUTED COMPUTER SYSTEMS ARCHITECTURES Dr. Jack Lange Computer Science Department University of Pittsburgh Fall 2015 Outline System Architectural Design Issues Centralized Architectures Application
More informationSpecial Topics: CSci 8980 Edge History
Special Topics: CSci 8980 Edge History Jon B. Weissman (jon@cs.umn.edu) Department of Computer Science University of Minnesota P2P: What is it? No always-on server Nodes are at the network edge; come and
More information08 Distributed Hash Tables
08 Distributed Hash Tables 2/59 Chord Lookup Algorithm Properties Interface: lookup(key) IP address Efficient: O(log N) messages per lookup N is the total number of servers Scalable: O(log N) state per
More informationCSE 124 Finding objects in distributed systems: Distributed hash tables and consistent hashing. March 8, 2016 Prof. George Porter
CSE 124 Finding objects in distributed systems: Distributed hash tables and consistent hashing March 8, 2016 rof. George orter Outline Today: eer-to-peer networking Distributed hash tables Consistent hashing
More informationTelematics Chapter 9: Peer-to-Peer Networks
Telematics Chapter 9: Peer-to-Peer Networks Beispielbild User watching video clip Server with video clips Application Layer Presentation Layer Application Layer Presentation Layer Session Layer Session
More informationSimulations of Chord and Freenet Peer-to-Peer Networking Protocols Mid-Term Report
Simulations of Chord and Freenet Peer-to-Peer Networking Protocols Mid-Term Report Computer Communications and Networking (SC 546) Professor D. Starobinksi Brian Mitchell U09-62-9095 James Nunan U38-03-0277
More information: Scalable Lookup
6.824 2006: Scalable Lookup Prior focus has been on traditional distributed systems e.g. NFS, DSM/Hypervisor, Harp Machine room: well maintained, centrally located. Relatively stable population: can be
More informationSearching for Shared Resources: DHT in General
1 ELT-53206 Peer-to-Peer Networks Searching for Shared Resources: DHT in General Mathieu Devos Tampere University of Technology Department of Electronics and Communications Engineering Based on the original
More informationSearching for Shared Resources: DHT in General
1 ELT-53207 P2P & IoT Systems Searching for Shared Resources: DHT in General Mathieu Devos Tampere University of Technology Department of Electronics and Communications Engineering Based on the original
More informationEECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks. Overlay Networks: Motivations
EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks Ion Stoica Computer Science Division Department of Electrical Engineering and Computer Sciences University of California, Berkeley
More informationP2P Network Structured Networks: Distributed Hash Tables. Pedro García López Universitat Rovira I Virgili
P2P Network Structured Networks: Distributed Hash Tables Pedro García López Universitat Rovira I Virgili Pedro.garcia@urv.net Index Description of CHORD s Location and routing mechanisms Symphony: Distributed
More informationMaking Gnutella-like P2P Systems Scalable
Making Gnutella-like P2P Systems Scalable Y. Chawathe, S. Ratnasamy, L. Breslau, N. Lanham, S. Shenker Presented by: Herman Li Mar 2, 2005 Outline What are peer-to-peer (P2P) systems? Early P2P systems
More informationDistributed Hash Tables
Distributed Hash Tables Chord Smruti R. Sarangi Department of Computer Science Indian Institute of Technology New Delhi, India Smruti R. Sarangi Chord 1/29 Outline Overview 1 Overview 2 3 Smruti R. Sarangi
More informationBadri Nath Rutgers University
lookup services Badri Nath Rutgers University badri@cs.rutgers.edu 1. CAN: A scalable content addressable network, Sylvia Ratnasamy et.al. SIGCOMM 2001 2. Chord: A scalable peer-to-peer lookup protocol
More informationIntroduction to P2P Computing
Introduction to P2P Computing Nicola Dragoni Embedded Systems Engineering DTU Compute 1. Introduction A. Peer-to-Peer vs. Client/Server B. Overlay Networks 2. Common Topologies 3. Data Location 4. Gnutella
More informationLecture 8: Application Layer P2P Applications and DHTs
Lecture 8: Application Layer P2P Applications and DHTs COMP 332, Spring 2018 Victoria Manfredi Acknowledgements: materials adapted from Computer Networking: A Top Down Approach 7 th edition: 1996-2016,
More informationDISTRIBUTED SYSTEMS CSCI 4963/ /4/2015
1 DISTRIBUTED SYSTEMS CSCI 4963/6963 12/4/2015 2 Info Quiz 7 on Tuesday. Project 2 submission URL is posted on the web site Submit your source code and project report (PDF!!!) in a single zip file. If
More informationDistributed Systems. 17. Distributed Lookup. Paul Krzyzanowski. Rutgers University. Fall 2016
Distributed Systems 17. Distributed Lookup Paul Krzyzanowski Rutgers University Fall 2016 1 Distributed Lookup Look up (key, value) Cooperating set of nodes Ideally: No central coordinator Some nodes can
More informationPeer-to-Peer Internet Applications: A Review
Peer-to-Peer Internet Applications: A Review Davide Quaglia 01/14/10 Introduction Key points Lookup task Outline Centralized (Napster) Query flooding (Gnutella) Distributed Hash Table (Chord) Simulation
More informationDistributed Hash Tables
Distributed Hash Tables CS6450: Distributed Systems Lecture 11 Ryan Stutsman Material taken/derived from Princeton COS-418 materials created by Michael Freedman and Kyle Jamieson at Princeton University.
More informationScalable overlay Networks
overlay Networks Dr. Samu Varjonen 1 Lectures MO 15.01. C122 Introduction. Exercises. Motivation. TH 18.01. DK117 Unstructured networks I MO 22.01. C122 Unstructured networks II TH 25.01. DK117 Bittorrent
More informationOverlay Networks: Motivations. EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks. Motivations (cont d) Goals.
Overlay Networks: Motivations CS : Introduction to Computer Networks Overlay Networks and PP Networks Ion Stoica Computer Science Division Department of lectrical ngineering and Computer Sciences University
More informationCS555: Distributed Systems [Fall 2017] Dept. Of Computer Science, Colorado State University
CS 555: DISTRIBUTED SYSTEMS [P2P SYSTEMS] Shrideep Pallickara Computer Science Colorado State University Frequently asked questions from the previous class survey Byzantine failures vs malicious nodes
More informationDecentralized Object Location In Dynamic Peer-to-Peer Distributed Systems
Decentralized Object Location In Dynamic Peer-to-Peer Distributed Systems George Fletcher Project 3, B649, Dr. Plale July 16, 2003 1 Introduction One of the key requirements for global level scalability
More informationL3S Research Center, University of Hannover
, University of Hannover Structured Peer-to to-peer Networks Wolf-Tilo Balke and Wolf Siberski 3..6 *Original slides provided by K. Wehrle, S. Götz, S. Rieche (University of Tübingen) Peer-to-Peer Systems
More informationA Decentralized Content-based Aggregation Service for Pervasive Environments
A Decentralized Content-based Aggregation Service for Pervasive Environments Nanyan Jiang, Cristina Schmidt, Manish Parashar The Applied Software Systems Laboratory Rutgers, The State University of New
More informationDistributed lookup services
Distributed lookup services lookup services Badri Nath Rutgers University badri@cs.rutgers.edu A set of nodes cooperating Peers Run special purpose algorithms/software Doesn t have to be deployed at every
More informationAdvanced Distributed Systems. Peer to peer systems. Reference. Reference. What is P2P? Unstructured P2P Systems Structured P2P Systems
Advanced Distributed Systems Peer to peer systems Karl M. Göschka Karl.Goeschka@tuwien.ac.at http://www.infosys.tuwien.ac.at/teaching/courses/ AdvancedDistributedSystems/ What is P2P Unstructured P2P Systems
More informationCS535 Big Data Fall 2017 Colorado State University 11/7/2017 Sangmi Lee Pallickara Week 12- A.
CS Big Data Fall Colorado State University http://www.cs.colostate.edu/~cs // Week - A // CS Big Data - Fall Week -A- CS BIG DATA FAQs PA deadline has been extended (/) PART. SCALABLE FRAMEWORKS FOR REAL-TIME
More informationCSCI-1680 P2P Rodrigo Fonseca
CSCI-1680 P2P Rodrigo Fonseca Based partly on lecture notes by Ion Stoica, Sco5 Shenker, Joe Hellerstein Today Overlay networks and Peer-to-Peer Motivation Suppose you want to write a routing protocol
More informationDistributed Hash Tables: Chord
Distributed Hash Tables: Chord Brad Karp (with many slides contributed by Robert Morris) UCL Computer Science CS M038 / GZ06 12 th February 2016 Today: DHTs, P2P Distributed Hash Tables: a building block
More informationSecuring Chord for ShadowWalker. Nandit Tiku Department of Computer Science University of Illinois at Urbana-Champaign
Securing Chord for ShadowWalker Nandit Tiku Department of Computer Science University of Illinois at Urbana-Champaign tiku1@illinois.edu ABSTRACT Peer to Peer anonymous communication promises to eliminate
More informationSmall-World Overlay P2P Networks: Construction and Handling Dynamic Flash Crowd
Small-World Overlay P2P Networks: Construction and Handling Dynamic Flash Crowd Ken Y.K. Hui John C. S. Lui David K.Y. Yau Dept. of Computer Science & Engineering Computer Science Department The Chinese
More informationPart 1: Introducing DHTs
Uni Innsbruck Informatik - Peer-to to-peer Systems Structured PP file sharing systems Michael Welzl michael.welzl@uibk.ac.at DPS NSG Team http://dps.uibk.ac.at dps.uibk.ac.at/nsg Institute of Computer
More informationDistributed Hash Tables
KTH ROYAL INSTITUTE OF TECHNOLOGY Distributed Hash Tables Distributed Hash Tables Vladimir Vlassov Large scale data bases hundreds of servers High churn rate servers will come and go Benefits fault tolerant
More informationFORMAL METHODS IN NETWORKING COMPUTER SCIENCE 598D, SPRING 2010 PRINCETON UNIVERSITY LIGHTWEIGHT MODELING IN PROMELA/SPIN AND ALLOY
FORMAL METHODS IN NETWORKING COMPUTER SCIENCE 598D, SPRING 2010 PRINCETON UNIVERSITY LIGHTWEIGHT MODELING IN PROMELA/SPIN AND ALLOY Pamela Zave AT&T Laboratories Research Florham Park, New Jersey, USA
More informationDepartment of Computer Science Institute for System Architecture, Chair for Computer Networks. File Sharing
Department of Computer Science Institute for System Architecture, Chair for Computer Networks File Sharing What is file sharing? File sharing is the practice of making files available for other users to
More informationPeer-to-peer systems and overlay networks
Complex Adaptive Systems C.d.L. Informatica Università di Bologna Peer-to-peer systems and overlay networks Fabio Picconi Dipartimento di Scienze dell Informazione 1 Outline Introduction to P2P systems
More informationCS514: Intermediate Course in Computer Systems
Distributed Hash Tables (DHT) Overview and Issues Paul Francis CS514: Intermediate Course in Computer Systems Lecture 26: Nov 19, 2003 Distributed Hash Tables (DHT): Overview and Issues What is a Distributed
More informationSaarland University Faculty of Natural Sciences and Technology I Department of Computer Science. Masters Thesis
Saarland University Faculty of Natural Sciences and Technology I Department of Computer Science Masters Thesis Experimental Comparison of Byzantine Fault Tolerant Distributed Hash Tables submitted by Supriti
More informationDistributed Systems. 16. Distributed Lookup. Paul Krzyzanowski. Rutgers University. Fall 2017
Distributed Systems 16. Distributed Lookup Paul Krzyzanowski Rutgers University Fall 2017 1 Distributed Lookup Look up (key, value) Cooperating set of nodes Ideally: No central coordinator Some nodes can
More information416 Distributed Systems. Mar 3, Peer-to-Peer Part 2
416 Distributed Systems Mar 3, Peer-to-Peer Part 2 Scaling Problem Millions of clients server and network meltdown 2 P2P System Leverage the resources of client machines (peers) Traditional: Computation,
More informationIn this assignment, you will implement a basic CHORD distributed hash table (DHT) as described in this paper:
Project 2: Chord Distributed Hash Table Introduction In this assignment, you will implement a basic CHORD distributed hash table (DHT) as described in this paper: https://pdos.csail.mit.edu/papers/ton:chord/paper-ton.pdf
More informationAn Expresway over Chord in Peer-to-Peer Systems
An Expresway over Chord in Peer-to-Peer Systems Hathai Tanta-ngai Technical Report CS-2005-19 October 18, 2005 Faculty of Computer Science 6050 University Ave., Halifax, Nova Scotia, B3H 1W5, Canada An
More informationTracking Objects in Distributed Systems. Distributed Systems
Tracking Objects in Distributed Systems Arvind Krishnamurthy Fall 2004 Distributed Systems Consider the Shared virtual memory system (SVM) by Kai Li: It tracks pages in a cluster system Simplest version
More informationAdvanced Computer Networks
Advanced Computer Networks P2P Systems Jianping Pan Summer 2007 5/30/07 csc485b/586b/seng480b 1 C/S vs P2P Client-server server is well-known server may become a bottleneck Peer-to-peer everyone is a (potential)
More informationDistributed Systems Final Exam
15-440 Distributed Systems Final Exam Name: Andrew: ID December 12, 2011 Please write your name and Andrew ID above before starting this exam. This exam has 14 pages, including this title page. Please
More informationA Hybrid Peer-to-Peer Architecture for Global Geospatial Web Service Discovery
A Hybrid Peer-to-Peer Architecture for Global Geospatial Web Service Discovery Shawn Chen 1, Steve Liang 2 1 Geomatics, University of Calgary, hschen@ucalgary.ca 2 Geomatics, University of Calgary, steve.liang@ucalgary.ca
More informationCSCI-1680 Web Performance, Content Distribution P2P John Jannotti
CSCI-1680 Web Performance, Content Distribution P2P John Jannotti Based partly on lecture notes by Sco2 Shenker and Rodrigo Fonseca Last time HTTP and the WWW Today: HTTP Performance Persistent Connections,
More informationCSCI-1680 Web Performance, Content Distribution P2P Rodrigo Fonseca
CSCI-1680 Web Performance, Content Distribution P2P Rodrigo Fonseca Based partly on lecture notes by Scott Shenker and John Jannotti Last time HTTP and the WWW Today: HTTP Performance Persistent Connections,
More informationDistributed K-Ary System
A seminar presentation Arne Vater and Prof. Schindelhauer Professorship for Computer Networks and Telematik Department of Computer Science University of Freiburg 2007-03-01 Outline 1 2 3 4 5 Outline 1
More informationOpportunistic Application Flows in Sensor-based Pervasive Environments
Opportunistic Application Flows in Sensor-based Pervasive Environments Nanyan Jiang, Cristina Schmidt, Vincent Matossian, and Manish Parashar ICPS 2004 1 Outline Introduction to pervasive sensor-based
More information15-744: Computer Networking P2P/DHT
15-744: Computer Networking P2P/DHT Overview P2P Lookup Overview Centralized/Flooded Lookups Routed Lookups Chord Comparison of DHTs 2 Peer-to-Peer Networks Typically each member stores/provides access
More informationNaming. Naming. Naming versus Locating Entities. Flat Name-to-Address in a LAN
Naming Naming Tanenbaum Ch. 5 Distributed Software Systems CS 707 A name in a distributed system is a string of bits or characters that is used to refer to an entity Types of names: Address: an access
More informationInternet Services & Protocols
Department of Computer Science Institute for System Architecture, Chair for Computer Networks Internet Services & Protocols Content Distribution Dr.-Ing. Stephan Groß Room: INF 3099 E-Mail: stephan.gross@tu-dresden.de
More informationFlexible Routing Tables in a Distributed Key-Value Store
Flexible Routing Tables in a Distributed Key-Value Store Diplomarbeit zur Erlangung des akademischen Grades Diplominformatiker Humboldt-Universität zu Berlin Mathematisch-Naturwissenschaftliche Fakultät
More informationResilient GIA. Keywords-component; GIA; peer to peer; Resilient; Unstructured; Voting Algorithm
Rusheel Jain 1 Computer Science & Information Systems Department BITS Pilani, Hyderabad Campus Hyderabad, A.P. (INDIA) F2008901@bits-hyderabad.ac.in Chittaranjan Hota 2 Computer Science & Information Systems
More information