Main Challenge. Other Challenges. How Did it Start? Napster. Model. EE 122: Peer-to-Peer Networks. Find where a particular file is stored

Transcription

1 Main hallenge ind where a particular file is stored : Peer-to-Peer Networks Ion Stoica (and righten Godfrey) Ts: Lucian Popa, avid Zats and Ganesh nanthanarayanan (Materials with thanks to Vern Paxson, Jennifer Rexford, and colleagues at U erkeley)? How id it Start? killer application: Naptser (999) ree music over the Internet Key idea: share the storage and bandwidth of individual (home) users Other hallenges Scale: up to hundred of thousands or millions of machines ynamicity: machines can come and go any time Internet Model Napster ach user stores a subset of files ach user has access (can download) files from all users in the system ssume a centralized index system that maps files (songs) to machines that are alive How to find a file (song) Query the index system return a machine that stores the required file Ideally this is the closest/least-loaded machine ftp the file dvantages: Simplicity, easy to implement sophisticated search engines on top of the index system isadvantages: Robustness, scalability (?)

2 Napster: xample Gnutella: xample m ssume: m s neighbors are m and m; m s neighbors are m and m; m m?? m m m m m m m m m??? m m m m m? m m The ftermath Recording Industry ssociation of merica (RI) Sues Music Startup Napster for $ illion ecember 999 Napster ordered to remove copyrighted material March Main legal argument: Napster owns the index system, so it is directly responsible for disseminating copyrighted material Two-Level Hierarchy urrent Gnutella implementation, Oct rawl KaZaa on Gnutella Leaf nodes are connected to a small number of ultrapeers (suppernodes) Query leaf sends query to its ultrapeers If ultrapeers don t know the answer, they flood the query to Ultrapeer nodes other ultrapeers Leaf nodes More scalable: looding only among ultrapeers Gnutella () Skype () istribute file location Idea: broadcast the request How to find a file? Send request to all neighbors Neighbors recursively multicast the request ventually a machine that has the file receives the request, and it sends back the answer dvantages: Totally decentralized, highly robust isadvantages: Not scalable; the entire network can be swamped with requests (to alleviate this problem, each request has a TTL) 9 Peer-to-peer Internet Telephony Two-level hierarchy like KaZaa Ultrapeers used to route traffic between NTed end-hosts (see next slide) plus a login server to authenticate users ensure that names are unique across network login server Messages exchanged to login server ata traffic (Note*: probable protocol; Skype protocol is not published)

3 ittorrent () llow fast downloads even when sources have low up-link capacity How does it work? Seed (origin) site storing the file to be downloaded Tracker server maintaining the list of peers in system Split each file into pieces (~ K each), and each piece into sub-pieces (~ K each) The loader loads one piece at a time Within one piece, the loader can load up to five subpieces in parallel Minute reak Questions efore We Proceed? ittorrent: Join Procedure ) Peer contacts tracker responsible for file it wants to download ) Tracker returns a list of peer (-) downloading same file ) Peer connects to peer in the list join peer list Tracker Seed (origin server) istributed Hash Tables Problem: Given an I, map to a host hallenges Scalability: hundreds of thousands or millions of machines Instability hanges in routes, congestion, availability of machines Heterogeneity Latency: ms to ms andwidth: Kb/s to Mb/s Nodes stay in system from s to a year Trust Selfish users Malicious users ittorrent: ownload lgorithm ownload consists of three phases: Start: get a piece as soon as possible Select a random piece Middle: spread all pieces as soon as possible Select rarest piece next nd: avoid getting stuck with a slow source, when downloading the last sub-pieces Request in parallel the same sub-piece ancel slowest downloads once a sub-piece has been received ontent ddressable Network (N, ) ssociate to each node and item a unique id in a d-dimensional space Properties Routing table size O(d) Guarantees that a file is found in at most d*n /d steps, where n is the total number of nodes (or details see:

4 N xample: Two imensional N xample: Two imensional divided between nodes ll nodes cover the entire space ach node covers either a square or a rectangular area of ratios : or : xample: ssume space size ( x ) Node n:(, ) first node that joins cover the entire space n Nodes n:(, ) and n:(,) join n n n n n 9 N xample: Two imensional N xample: Two imensional Node n:(, ) joins space is divided between n and n Nodes: n:(, ); n:(,); n:(, ); n:(,);n:(,) Items: f:(,); f:(,); f:(,); f:(,); n n n f n n n f f n f N xample: Two imensional N xample: Two imensional Node n:(, ) joins space is divided between n and n n ach item is stored by the node who owns its mapping in the space n n n f n n n f f n f

5 N: Query xample ach node knows its neighbors in the d-space orward query to the neighbor that is closest to the query id xample: assume n queries f n f f n n n n f Lookup ach node maintains its successor Route packet (I, data) to the node responsible for I using successor pointers node= lookup() f hord () ssociate to each node and item a unique id in an uni-dimensional space.. m - ach node periodically sends a stabilize() message to its successor Key design decision ecouple correctness from efficiency Properties Routing table size O(log(N)), where N is the total number of nodes Guarantees that a file is found in O(log(N)) steps Upon receiving a stabilize() message, node returns its predecessor =pred() to by sending a notify( ) message Upon receiving notify( ) from, if is between and, updates its successor to otherwise, doesn t do anything 9 Identifier to Node Mapping xample Node maps [,] Node maps [9,] Node maps [, ] Node maps [9, ] Node with id= joins the ring Node needs to know at least one node already in the succ=nil system pred=nil ssume known node is succ= pred= succ= pred= ach node maintains a pointer to its successor

6 Node : send join() to node Node : returns node Node updates its successor to succ=nil succ= pred=nil succ= pred= join() (cont d) Node sends a stabilize message to its new successor, node Node sets its predecessor to node succ= pred= pred=nil succ= pred= Stabilize() succ= pred= succ= pred= (cont d) Node : send stabilize() to node Node : update predecessor to send notify() back succ= pred=nil notify(pred=) succ= pred= succ= pred= pred= stabilize() This completes the joining operation! succ= pred= succ= pred= (cont d) succ= Node sends a stabilize pred= message to its successor, node Node reply with a notify message Node updates its stabilize() successor to succ= pred=nil succ= succ= pred= notify(pred=) chieving fficiency: finger tables inger Table at i ft[i] ( + ) mod = Say m= i m ith entry at peer with id n is first peer with id >= n + (mod )

7 chieving Robustness To improve robustness each node maintains the k (> ) immediate successors instead of only one successor In the notify() message, node can send its k- successors to its predecessor Upon receiving notify() message, can update its successor list by concatenating the successor list received from with itself iscussion Query can be implemented Iteratively Recursively Performance: routing in the overlay network can be more expensive than in the underlying network ecause usually there is no correlation between node ids and their locality; a query can repeatedly jump from urope to North merica, though both the initiator and the node that store the item are in urope! Solutions: Tapestry takes care of this implicitly; N and hord maintain multiple copies for each entry in their routing tables and choose the closest in terms of network distance onclusions The key challenge of building wide area PP systems is a scalable and robust directory service Solutions covered in this lecture Naptser: centralized location service Gnutella: gossip-based decentralized location service N, hord, Tapestry, Pastry: intelligent-routing decentralized solution Guarantee correctness Tapestry, Pastry provide more efficient routing, but more complex 9