CPSC 426/526. P2P Lookup Service. Ennan Zhai. Computer Science Department Yale University

Transcription

1 CPSC 4/5 PP Lookup Service Ennan Zhai Computer Science Department Yale University

2 Recall: Lec- Network basics: - OSI model and how Internet works - Socket APIs red PP network (Gnutella, KaZaA, etc.) UseNet and gossip protocol: - UseNet format and peer identity - How gossip protocol runs, pros. and cons. Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)

3 Lecture Roadmap Background Chord Design Evaluating Chord Comparison

4 Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)

5 Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)

6 Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)

7 Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)

8 Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.)? - More (Hybrid PP, BitTorrent, etc.) Pros. Cons.?

9 Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, - Easy BitTorrent, to maintainetc.) Pros. { Cons. { - Finding popular stuff - Fancy local queries - Finding unpopular stuff - No guarantees - Data poisoning and tracking

10 Background History of Peer-to-Peer (PP) systems: - Napster, the first PP content sharing system - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)

11 Why Chord? Chord s important guarantees: - Decentralized (V.S. Napster) - Scalability: O(log(n)) - Availability (V.S. Gnutella)

12 Lecture Roadmap Background Chord Design Evaluating Chord Comparison

13 Chord In short: A PP lookup service Solves problem of locating a data in a collection of decentralized nodes Supports just one operation: given a key, it maps the key onto a node

14 Initialization: Node

15 Initialization: Node

16 Initialization: Object

17 Initialization: Object

18 Initialization: Table 7 0 [,3) 3 3 [3,5) 3 5 [5,)

19 Initialization: Table 7 0 [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

20 Initialization: Table [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

21 Initialization: Table [,) [,4) 3 4 [4,0) Search an object? [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

22 Search [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

23 Search [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

24 Search [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

25 Search [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

26 Search [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

27 Search [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

28 Search [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

29 Search [,) [,4) 3 4 [4,0) I have! [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

30 Search [,) [,4) 3 4 [4,0) I have! [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

31 If new node joins?

32 Node Joins [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

33 Node Joins [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

34 Object Location Changed [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

35 Object Location Changed [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 Who will be affected? 5 [5,) [4,5) 0 5 [5,7) 0 7 [7,3) 0

36 Table Changed [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) [4,5) 0-> 5 [5,7) 0-> 7 [7,3) 0

37 Table Changed [,) [,4) 3 4 [4,0) [,3) 3 3 [3,5) 3 5 [5,) 0 -> [4,5) 5 [5,7) 7 [7,3) 0

38 Table Changed [,) [,4) 3 4 [4,0) 0 -> 0 7 [,3) 3 3 [3,5) 3 5 [5,) [4,5) 5 [5,7) 7 [7,3) 0

39 New Table [,) [,4) 3 4 [4,0) 7 0 [,3) 3 3 [3,5) 3 5 [5,) 7 [7,0) 0 0 [0,) 0 [,) [4,5) 5 [5,7) 7 [7,3) 0

40 If node leaves?

41 Node Leaves [,) [,4) 3 4 [4,0) 0 7 [,3) 3 3 [3,5) 3 5 [5,) 7 [7,0) 0 0 [0,) 0 [,) [4,5) 5 [5,7) 7 [7,3) 0

42 7 [7,0) 0 0 [0,) 0 [,) 3 [,) [,4) 3 4 [4,0) 0 7 Node Leaves Who will be affected? [,3) 3 3 [3,5) 3 5 [5,) 4 [4,5) 5 [5,7) 7 [7,3) 0

43 Node Leaves [,) [,4) 3 4 [4,0) 0 7 [,3) 3 3 [3,5) 3 5 [5,) 7 [7,0) 0 0 [0,) 0 [,) [4,5) 5 [5,7) 7 [7,3) 0

44 Node Leaves [,) 3 [,4) 3 4 [4,0) [7,0) 0 0 [0,) 0 [,) [4,5) 5 [5,7) 7 [7,3) 0

45 Practical Issues Concurrent operations and failures Failure recovery

46 Concurrent Operations & Errors Basic stabilization protocol is used to keep nodes successor pointers up to date, which is sufficient to ensure the correctness of lookups. Every node runs stabilize periodically to find newly joined/left nodes.

47 Concurrent Operations & Errors Basic stabilization protocol is used to keep nodes successor pointers up to date, which is sufficient to ensure the correctness of lookups. Every node runs stabilize periodically to find newly joined/left nodes.

48 Failure Recovery We should maintain correct successor pointers To do this, each node maintains a successor-list of its r nearest successors on the ring If node n notices that its successor has failed, it replaces it with the first live entry in the list Stabilize will correct finger table and successor-list entries pointing to failed node

49 Failure Recovery We should maintain correct successor pointers To do this, each node maintains a successor-list of its r nearest successors on the ring If node n notices that its successor has failed, it replaces it with the first live entry in the list Stabilize will correct finger table and successor-list entries pointing to failed node

50 Failure Recovery We should maintain correct successor pointers To do this, each node maintains a successor-list of its r nearest successors on the ring If node n notices that its successor has failed, it replaces it with the first live entry in the list Stabilize will correct finger table and successor-list entries pointing to failed node

51 Failure Recovery We should maintain correct successor pointers To do this, each node maintains a successor-list of its r nearest successors on the ring If node n notices that its successor has failed, it replaces it with the first live entry in the list Stabilize will correct finger table and successor-list entries pointing to failed node

52 Lecture Roadmap Background Chord Design Evaluating Chord Comparison

53 Distributed Hash Table All the structured PP models implement DHT abstraction.

54 Provable Guarantees Efficient: O(log N) messages per lookup Scalable: O(log N) state per node Robust: survives massive failures Update only affects O(log N) nodes

55 Final Notes Chord only offers Lookup(key) -> IP address Chord typically does not store the data Object and Node IDs are from SHA- Applications?

56 Other Structured PP Models

57 CAN [SIGCOMM 0]

58 Pastry [Middleware 0]

59 SkipNet

60 PP Search Models PP search model: - Napster and BitTorrent - Unstructured PP network (Gnutella, KaZaA, etc.) - Structured PP network (Chord, CAN, Pastry, etc.) - Hybrid (Yyepas, Gnutella+Pastry, etc.) - More (Hybrid PP, BitTorrent, etc.)

61 Lecture Roadmap Background Chord Design Evaluating Chord Comparison

62 Structured PP vs Unstructured PP Searching Routing performance Peers join/leave overhead Maintenance Scalability Lookups need O(logN) messages The complexity of finger table needs O(logN) entries.

63 Next Lecture In the lec-5, I will cover: - Pollution problems in PP content sharing systems - Reputation system is the most representative solution, including peer-based and object-based reputation systems - Case study: Credence