CPSC 4/5 PP Lookup Service Ennan Zhai Computer Science Department Yale University
Recall: Lec- Network basics: - OSI model and how Internet works - Socket APIs red PP network (Gnutella, KaZaA, etc.) UseNet and gossip protocol: - UseNet format and peer identity - How gossip protocol runs, pros. and cons. Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)
Lecture Roadmap Background Chord Design Evaluating Chord Comparison
Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)
Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)
Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)
Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)
Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.)? - More (Hybrid PP, BitTorrent, etc.) Pros. Cons.?
Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, - Easy BitTorrent, to maintainetc.) Pros. { Cons. { - Finding popular stuff - Fancy local queries - Finding unpopular stuff - No guarantees - Data poisoning and tracking
Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)
Why Chord? Chord s important guarantees: - Decentralized (V.S. Napster) - Scalability: O(log(n)) - Availability (V.S. Gnutella)
Lecture Roadmap Background Chord Design Evaluating Chord Comparison
Chord In short: A PP lookup service Solves problem of locating a data in a collection of decentralized nodes Supports just one operation: given a key, it maps the key onto a node
Initialization: Node 7 0 5 3 4
Initialization: Node 7 0 5 3 4
Initialization: Object 7 0 5 4 3
Initialization: Object 7 0 5 4 3
Initialization: Table 7 0 [,3) 3 3 [3,5) 3 5 [5,) 0 5 3 4
Initialization: Table 7 0 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Initialization: Table [,) [,4) 3 4 [4,0) 0 0 7 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Initialization: Table [,) [,4) 3 4 [4,0) 0 0 7 Search an object? [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Search [,) [,4) 3 4 [4,0) 0 7 0 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Search [,) [,4) 3 4 [4,0) 0 7 0 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Search [,) [,4) 3 4 [4,0) 0 7 0 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Search [,) [,4) 3 4 [4,0) 0 7 0 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Search [,) [,4) 3 4 [4,0) 0 7 0 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Search [,) [,4) 3 4 [4,0) 0 7 0 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Search [,) [,4) 3 4 [4,0) 0 7 0 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Search [,) [,4) 3 4 [4,0) 0 0 7 I have! [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Search [,) [,4) 3 4 [4,0) 0 0 7 I have! [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
If new node joins?
Node Joins [,) [,4) 3 4 [4,0) 0 7 0 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Node Joins [,) [,4) 3 4 [4,0) 0 7 0 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Object Location Changed [,) [,4) 3 4 [4,0) 0 0 7 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Object Location Changed [,) [,4) 3 4 [4,0) 0 0 7 [,3) 3 3 [3,5) 3 Who will be affected? 5 [5,) 0 5 4 3 4 [4,5) 0 5 [5,7) 0 7 [7,3) 0
Table Changed [,) [,4) 3 4 [4,0) 0 0 7 [,3) 3 3 [3,5) 3 5 [5,) 0 5 4 3 4 [4,5) 0-> 5 [5,7) 0-> 7 [7,3) 0
Table Changed [,) [,4) 3 4 [4,0) 0 0 7 [,3) 3 3 [3,5) 3 5 [5,) 0 -> 5 4 3 4 [4,5) 5 [5,7) 7 [7,3) 0
Table Changed [,) [,4) 3 4 [4,0) 0 -> 0 7 [,3) 3 3 [3,5) 3 5 [5,) 5 4 3 4 [4,5) 5 [5,7) 7 [7,3) 0
New Table [,) [,4) 3 4 [4,0) 7 0 [,3) 3 3 [3,5) 3 5 [5,) 7 [7,0) 0 0 [0,) 0 [,) 3 5 4 3 4 [4,5) 5 [5,7) 7 [7,3) 0
If node leaves?
Node Leaves [,) [,4) 3 4 [4,0) 0 7 [,3) 3 3 [3,5) 3 5 [5,) 7 [7,0) 0 0 [0,) 0 [,) 3 5 4 3 4 [4,5) 5 [5,7) 7 [7,3) 0
7 [7,0) 0 0 [0,) 0 [,) 3 [,) [,4) 3 4 [4,0) 0 7 Node Leaves Who will be affected? 5 3 4 [,3) 3 3 [3,5) 3 5 [5,) 4 [4,5) 5 [5,7) 7 [7,3) 0
Node Leaves [,) [,4) 3 4 [4,0) 0 7 [,3) 3 3 [3,5) 3 5 [5,) 7 [7,0) 0 0 [0,) 0 [,) 3 5 4 3 4 [4,5) 5 [5,7) 7 [7,3) 0
Node Leaves [,) 3 [,4) 3 4 [4,0) 0 7 7 [7,0) 0 0 [0,) 0 [,) 3 5 4 3 4 [4,5) 5 [5,7) 7 [7,3) 0
Practical Issues Concurrent operations and failures Failure recovery
Concurrent Operations & Errors Basic stabilization protocol is used to keep nodes successor pointers up to date, which is sufficient to ensure the correctness of lookups. Every node runs stabilize periodically to find newly joined/left nodes.
Concurrent Operations & Errors Basic stabilization protocol is used to keep nodes successor pointers up to date, which is sufficient to ensure the correctness of lookups. Every node runs stabilize periodically to find newly joined/left nodes.
Failure Recovery We should maintain correct successor pointers To do this, each node maintains a successor-list of its r nearest successors on the ring If node n notices that its successor has failed, it replaces it with the first live entry in the list Stabilize will correct finger table and successor-list entries pointing to failed node
Failure Recovery We should maintain correct successor pointers To do this, each node maintains a successor-list of its r nearest successors on the ring If node n notices that its successor has failed, it replaces it with the first live entry in the list Stabilize will correct finger table and successor-list entries pointing to failed node
Failure Recovery We should maintain correct successor pointers To do this, each node maintains a successor-list of its r nearest successors on the ring If node n notices that its successor has failed, it replaces it with the first live entry in the list Stabilize will correct finger table and successor-list entries pointing to failed node
Failure Recovery We should maintain correct successor pointers To do this, each node maintains a successor-list of its r nearest successors on the ring If node n notices that its successor has failed, it replaces it with the first live entry in the list Stabilize will correct finger table and successor-list entries pointing to failed node
Lecture Roadmap Background Chord Design Evaluating Chord Comparison
Distributed Hash Table All the structured PP models implement DHT abstraction.
Provable Guarantees Efficient: O(log N) messages per lookup Scalable: O(log N) state per node Robust: survives massive failures Update only affects O(log N) nodes
Final Notes Chord only offers Lookup(key) -> IP address Chord typically does not store the data Object and Node IDs are from SHA- Applications?
Other Structured PP Models
CAN [SIGCOMM 0]
Pastry [Middleware 0]
SkipNet
PP Search Models PP search model: - Napster and BitTorrent - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - Hybrid (Yyepas, Gnutella+Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)
Lecture Roadmap Background Chord Design Evaluating Chord Comparison
Structured PP vs Unstructured PP Searching Routing performance Peers join/leave overhead Maintenance Scalability Lookups need O(logN) messages The complexity of finger table needs O(logN) entries.
Next Lecture In the lec-5, I will cover: - Pollution problems in PP content sharing systems - Reputation system is the most representative solution, including peer-based and object-based reputation systems - Case study: Credence