Deploying and Optimizing Squid in a Filesharing Application

Transcription

1 Deploying and Optimizing Squid in a Filesharing Application Dan Halperin DIMACS REU 2004 with Dr. Manish Parashar and Cristina Schmidt Deploying and Optimizing Squid in a Filesharing Application p.1/16

2 Outline Peer-to-Peer (P2P) Background Differing P2P Models Napster Gnutella Squid My project Deploying and Optimizing Squid in a Filesharing Application p.2/16

3 P2P Background Main Idea: Peers (i.e. networked machines - computers, cell phones, palms, etc) on a network connect directly to each other instead of through a centralized server. Why? Scalability, speed, fault tolerance, etc. Deploying and Optimizing Squid in a Filesharing Application p.3/16

4 P2P Background - Examples Hardware sharing: the GRID computing project, technologies that provide seamless and scalable access to wide-area distributed resources [1] (e.g. supercomputers, clusters, storage, data, instruments). Information Sharing: share network information, services, and resources without centralized servers (e.g. filesharing such as Gnutella). Deploying and Optimizing Squid in a Filesharing Application p.4/16

5 Napster P2P Model NapServer NapServer Servers index all clients data. Clients search to servers, download P2P. Pro: complex queries Con: not scalable, central point of failure, fraction of the network is searchable Deploying and Optimizing Squid in a Filesharing Application p.5/16

6 Gnutella P2P Model Peers index own data. Search propagated via flooding. Pro: fully scalable, supports complex queries. Con: no search guarantees, scarcity compounded, repeated queries. Deploying and Optimizing Squid in a Filesharing Application p.6/16

7 Squid P2P Model Finger table at node finger = the successor of (this node identifier + 2 i - 1 ) mod 2 m, 1 <= i <= m Structured overlay - efficient peer lookup. Index in Distributed Hash Table (DHT). Search is hashed, then DHT is queried. Pro: guaranteed results, complex queries (keywords, wildcards, ranges). Con: high overlay maintenance costs. Deploying and Optimizing Squid in a Filesharing Application p.7/16

8 Squid How Squid uses the DHT d keywords form a d-d keyword space K. Peers given IDs in [0, R] (1-D Peer ID space P). (R = maximum keyword d ) Hash function H : K P yields location to search for results for any query. Overlay provides efficient routing. Deploying and Optimizing Squid in a Filesharing Application p.8/16

9 Squid Hash Function: Space-Filling Curves (SFCs) The Hilbert SFC fills K. Query covers some subregion of K, and will contain some parts of the curve. The indices along the curve are the images of the hash function in P. Deploying and Optimizing Squid in a Filesharing Application p.9/16

10 Squid Some SFCs and Hilbert Refinement (a) (b) (c) (c1) (c2) (c3) Curve locality. Deploying and Optimizing Squid in a Filesharing Application p.10/16

11 My Project For a given application, What is the best SFC? What is the best overlay? What other optimizations can we make? The best way to approach this: What application-specific assumptions can we make? How can we exploit them? Deploying and Optimizing Squid in a Filesharing Application p.11/16

12 Filesharing Application We re creating a filesharing application. Assumptions we can make: Interchangeable axes. No ranges, except for wildcards. Idea: Use a modified Z-curve for the SFC. Motivation: Do not need to preserve locality in all directions. Use simpler, more customizable Z-curve. Deploying and Optimizing Squid in a Filesharing Application p.12/16

13 Sample Z SFC searches (b*,**) (ab,**) (c*,b*) (cc,c*) (aa,ca) (**,ab) Deploying and Optimizing Squid in a Filesharing Application p.13/16

14 Z-Curve Advantages: Curve follows query shapes. Simple to generate. Easy hash: ( cat, dog, *** ) cd ao tg 26. Customizes to alphabet. Count clusters: Σ ^ (# internal stars). Deploying and Optimizing Squid in a Filesharing Application p.14/16

15 Bibliography References [1] GRID Computing Web Page. [2] A. Oram, ed. Peer-To-Peer: Harnessing the Power of Disruptive Technologies, O Reilly & Associates, Inc., [3] C. Schmidt and M. Parashar, Enabling Flexible Queries with Guarantees in P2P Systems, IEEE Internet Computing (May-June 2004), pp [4] C. Schmidt and M. Parashar, Peer to Peer Information Storage and Discovery Systems, forthcoming. Deploying and Optimizing Squid in a Filesharing Application p.15/16

16 Acknowledgments DIMACS REU program Rutgers University Dr. Manish Parashar Cristina Schmidt Deploying and Optimizing Squid in a Filesharing Application p.16/16