Indexing Through Querying in Unstructured Peer-to-Peer Overlay Networks

Transcription

1 Indexing Through Querying in Unstructured Peer-to-Peer Overlay Networks K Haribabu 1, Chittaranjan Hota 2, and Antti Ylä-Jääski 3 1 Computer Sc. & Information Systems Group, Birla Institute of Technology and Science Pilani, ajasthan, INDIA 2 Computer Sc. & Information Syst. Group, Birla Institute of Technology and Science, Pilani Hyderabad Campus, Hyderabad, Andhra Pradesh, INDIA 3 Data Communications Software Laboratory, Deptartment of Computer Sc. & Engineering Helsinki University of Technology, Espoo, FINLAND khari@bits-pilani.ac.in, hota@bits-hyderabad.ac.in, antii.yla-jaaski@hut.fi Abstract. The efficiency of a Peer-to-Peer file sharing overlay is measured in terms of the scalability and versatility of its object lookup strategy. In these networks peers carry out distributed query relaying to discover the service providers. Existing lookup mechanisms like flooding and random walks in unstructured P2P overlays create huge communication overhead and increased response time. In this work we propose efficient lookup in unstructured peer-topeer overlay networks using indexing through querying, distributing indices through queries. Our simulation studies show that by our approach more than 97% of the queries are answered in one hop and the rest in few hops thus reducing the network load. Our approach is efficient in worst case scenarios where contents are distributed over thousands of peers and the overlay network condition is highly dynamic. Keywords: Peer-to-Peer, Overlay Networks, Indexing, Lookup, Bloom Filters 1 Introduction The emergence of Peer-to-Peer (P2P) overlay file sharing networks has increased the interest amongst the Internet users to use the Internet beyond web browsing and exchanging s. These networks have recently gained a lot of attention in academic and industrial circles. The P2P traffic occupies major chunk of the Internet traffic [1]. The kind of impact they generate on the whole Internet motivates the researchers to study and improve the scalability and performance of these networks. P2P overlay networks are application-level logical networks built on top of the physical networks as shown in Fig 1. These networks maintain separate addressing and routing mechanisms to enable efficient search and data exchange between peers. They don t require any special administrative or financial arrangement. They are selforganizing and adaptive, distributed and decentralized. There are two kinds of requests that are sent over the overlay edges, i.e. content requests to find an object and

2 keep-alive requests to stay connected in the overlay. P2P overlay networks are categorized as unstructured and structured. An unstructured P2P system is composed of peers joining the network with some loose rules, without any prior knowledge of the topology. Gnutella [2], and KaZaA [3] are examples of unstructured P2P overlay networks. In structured P2P overlay networks, network topology is tightly controlled and content is placed not at random peers but at specified locations that will make subsequent queries more efficient. Most of the structured P2P overlays are Distributed Hash Table (DHT) based. Content Addressable Network (CAN) [4], Chord [5], and Pastry [6] are some examples of structured P2P overlay networks. In unstructured P2P networks, the lookup for an object involves forwarding the queries to all P2P neighbors [7]. This generates huge message traffic in the network. As the queries are forwarded, the traffic exponentially multiplies at each hop. This degrades the network performance thus limits the scalability in terms of number of queries. As the number of queries increase in the network, the whole network bandwidth is consumed by these query messages, thus burdening the peers with unnecessary duplicate messages, message processing and giving very huge response times to the querying peers. core Fig. 1. An example Overlay Network This has prompted us to think of ways to utilize the huge traffic generated and reduce further the query explosion. In this paper we present a query-based index propagation approach for disseminating the content indices through the query traffic. Each index record is the collection of bits generated by Bloom Filter [8]. Every query carries a summarized index of contents of the peers that are on its way. At every node, the indices are copied to local cache and the local indices are updated in the query message. The nodes can lookup the local index cache and directly contact the node for the object. If the index record is not found, it uses the usual lookup algorithm. Through the traffic the indexes are disseminated. As the time passes, the query traffic reduces and reaches a steady state. The approach takes advantage of huge traffic for a good purpose. It offers many advantages. First it reduces the aggregate bandwidth consumption in the network. This improves the scalability. Second, it reduces the search latency by maintaining an index of the objects in the neighborhood. This improves the quality of the search giving satisfaction to the user. Third, the nodes which participate in processing the maximum traffic get the maximum benefit. Such nodes get one-hop access to the most of objects in the network. Fourth, the index is disseminated equally for both popular as well as unpopular objects. One of the demerits of unstructured networks is that it doesn t guarantee retrieval of an object although it is present in the network. This is due to few replicas of unpopular objects in the network. Due to this index distribution, many pointers to such objects are distributed thus reducing the risk of not finding objects. However, we do not address the free riding problem in P2P overlays in this work. IP

3 The simulation results show that Indexing Through Querying (ITQ) approach compared with Flooding and k-random walk techniques reduce the average bandwidth consumption and search latency by more than 95%. However, the ITQ approach is less efficient when there is very less query traffic in the system and there are very few replicas of the objects, which is of course an unlike scenario in today s peer-to-peer file sharing systems. 2 elated Work Gnutella [2] used query flooding for locating the objects which obstructs the scalability of any network. In local indices technique [9], each node maintains an index of data of all nodes within r hops of itself. When a node receives the query, it processes it on behalf of all the nodes within r hops of radius. Creating and maintaining such index involves extra overhead on the system. In routing indices technique [1], index is created for different topics in different routes. The index is used for choosing a neighbor to forward the query. In this technique, the aggregate updates are exchanged among the nodes to keep the index up to date. By sending only the aggregated vectors, the overhead is reduced. In attenuated bloom filter technique [11], an index for every neighbor and up to d number of hops is maintained. Index for neighbor n, stores the bits of objects generated by Bloom Filter [8] available at hop d through neighbor n. The index is maintained by exchanging the changed bits. In [12], a global but partial index is built using a Distributed Hash Tables (DHT) built on top of the unstructured overlay. The index consists of peer s top interests which is reflected by the nature of queries and objects peer has. Changes in the interests of peers involve changing the neighbors. In [13], super peers develop multi-attribute based content index by observing the QueryHit messages that pass through it. They use this index to efficiently forward the queries to selected neighboring super peers. There is separate index table for every attribute. The entries are deleted based on age factor. In advertisement based technique [14], a local index is built by requesting neighbors up to certain hop level to send their ads or indexes. In esearch [15], index for every term is created. One node is responsible for maintaining an index of one term. Nodes analyze their documents and find top terms and publish them to the respective nodes responsible for those terms. Our approach differs from above that the index is propagated by the queries. Nodes have passive involvement of loading the indices to and from queries. So the index distribution is fully dependent on query distribution. 3 Indexing Through Querying In this section we discuss design of our approach.

4 3.1 Scope In this paper we focus on unstructured peer-to-peer networks. The approach encourages the idea that instead of creating and maintaining indexes by individual nodes, index information is disseminated by the query traffic. Each node observes the query traffic that passes through it and updates its local index cache. There is always a trade-off between the scope of the index and maintenance cost. In one extreme every node can keep index of all the objects available in the network. That will surely make all the queries one-hop (zero hops for lookup and one hop for getting the object). But the cost associated with maintaining such an index prohibits such an approach. On the other extreme searching loses its grace without indexing. Indexing is the most important tool for searching [16]. Our approach is based on day-to-day observation that information about festivals, events and government policies are propagated by sticking the posters on the public transportation buses, trains etc. The vehicles wherever they go, by their presence they communicate the information to the people. This doesn t cost extra. We augment that idea to suit the peer-to-peer networks. Each query can be seen as vehicle carrying the index information of all the nodes it has passed through. The nodes receiving the query copy all the index information and add their indices to the query. This way whoever the query reaches gets benefited. When the node needs to find the object, it looks up its index cache and if there is match it will immediately contact the peer having that object. Our approach is fundamentally based on the observation that query traffic of public file sharing p2p networks occupies most of the internet traffic. In [1], it is observed that p2p network traffic occupies the 4% of total internet traffic. In [17], it is reported that there are 562 transactions per second happening in Kazaa [3] network. This clearly gives the extent of queries in p2p systems. Not only the successful queries but even the unsuccessful queries can be used as vehicles for carrying the index information. Not only the queries but even the query replies and download requests can be used for this purpose. That way we have huge percentage of traffic at our disposal for conveying. But in this paper we focus on query messages. 3.2 Data Structure The query passes through many nodes and if every node keeps its index in the query, query will become bulky and will consume huge bandwidth. To address this issue we use Bloom Filers to only store the summarized index for specified number of files in one node. Bloom Filters [14] are widely used to represent elements of a set using a bit set hashed through a set of hash functions. The filter may report an element is present even though it is not present in the set. This is known as false positive. But the filter never reports absence of an element when it is present in the set. The false positive probability (fpp) of representing set of elements of size n depends on size m of Bloom Filter and size k of set of hash functions. From (2), it can be noted that fpp reduces exponentially as n is reduced, and k and m are increased. The m and fpp are computed as follows [26]. m =(k * n)/ln2 (1)

5 fpp=(.6185) m/n (2) We have set of constraints when we apply Bloom Filters to our approach. First, query message size is limited. Maximum query size as per Gnutella protocol.6 [2], is 4 KB. The number of files shared by users varies from zero to thousands. So to have one bloom filter for all files in a node is not feasible. Second, Bloom Filters don t have an option of deleting elements. It can only add new elements into the filter. So nodes exchange indices in units of Bloom Filter not a part of it. It requires identifying each Bloom Filter uniquely across the network so that it is possible to replace old Bloom Filter with new one. As observed in [18], in a p2p file sharing network, 75% of nodes share 1 or less number of files. So it will be appropriate to group 1 files of a node and represent their file names in one Bloom Filter. Nodes having more than 1 files will have more than one Bloom Filter to convey their indices. By fixing size of n at 1, we have various alternatives in choosing size of m and k as listed in Table 1. As k increases fpp falls exponentially. The choice of k effects the bandwidth consumption, and the time required to disseminate the indices. For supporting efficient dissemination and Table 1. k and m and fpp m k fpp (bytes) updating of indices across the network without extra cost we propose the following structure called Query Index ecord (QI). QI record is denoted as (N, B, {, S, D c }). N refers to the node identity such as IP address and B refers to the bandwidth (mbps) of the connection available at node N, and rest is the set of Bloom Filter records (BF) of N. Each BF has, the record number unique for N, S refers to m bit array, and D c refers to the date on which this bloom filter is created by N. It is measured as seconds that have elapsed since the starting of the network. This date is used to compare the freshness of the BF. A BF is uniquely identified by the combination of N and. A query carries several QI records as many as it can fit in maximum message size. As observed in [18], a node is sharing maximum of 8 files. Each BF can store summary of 1 files as discussed above. The value of doesn t exceed 1 in worst case thus takes 1 byte space. For k=6, the size of BF record in bytes is (1, 19, 4) = 114 bytes. The size of QI record in bytes is (4, 1, size of set of BF). Also each node has a local cache of Node index records (NI). The structure of NI is (N, B, {, S, D c, D lu, D ll }). For each BF, node stores additional information like the date on which it was last successfully used (D lu ), and the date on which it was loaded on to a query (D ll ). D lu is used for removing the NI after certain time and D ll is used in deciding which indices to be loaded onto the query. Let us apply the above structure to Gnutella.6 [2] with k=6. A Gnutella Query message takes maximum of 54 bytes assuming that keywords usually take less than 3 characters thus maximum of 3 bytes. Maximum message size is 4 KB. Then it is possible to fit in (496 54) / 119 = 35 index records in each Query message. The 35 index records convey the locations of 35 objects in the network. What should be

6 done when there are more than 35 index records to load on to a query is discussed in next section. 3.3 Algorithm Overview When a node joins the network, during the initial handshake with neighbors, it collects the set of NI from each neighbor and updates its local index cache. When a node wants to lookup for set of keywords W, it first looks up its own files and index as shown in the search algorithm in Fig 2. Out of set of hosts H, chosehostwithmaxbw procedure chooses the host h with maximum bandwidth. If there are no matches, it will resort to usual lookup method i.e, flooding or random walk. The query Q is populated with the cached indices. Incase the node has more indices than it could fit in the query, it will choose the indices based on D ll as shown in Fig 3. The indices are chosen in a round-robin fashion ensuring that all the indices are propagated. The sort method sorts the index records on D ll in ascending order. Similarly when a node receives the query, it will scan the query Q for the QI records and either replaces or adds them to local cache. The criterion for either replacing or adding is shown in Fig 4. After updating the local index cache, it will search files and the index cache similar to search algorithm Search(Keywords: W) { esult: Host: H for each file f Є F if match(f, W) = true then f If = then for each index record i Є I if BloomFilterMatch(i, W) then H H ( N(i), B(i)) else return If H = then Q makequery(w) Q loadindex(q, I) for each neighbor n Є P forwardquery(n, Q) else h chosehostwithmaxbw(h) requestdownload(h, W) } Fig. 2. Search algorithm for searching W at a querying node with index cache I, set of Files F and set of neighbors P LoadIndex(Query:Q, Index: I) { number: QSize = I sort(i, D ll ) //sorts I on D ll Q clearindex(q) for each index record i Є I Q load(q,n(i),b(i),- (i),s(i),d c (i)) D ll (i) now() QSize QSize + Z if QSize + Z >M then return } Fig. 3. Algorithm for adding index records in Query Q at a node with index cache I and network wide maximum query size M and query index record size Z UpdateIndexInNode(Query: Q, Index: I) { boolean found for each index record r Є found false for each index record i Є I if N(i) = N(r) and (i) = (r) then found true if Dc(i) < Dc(r) then i r if found = false then I I r } Fig. 4. Algorithm for adding index record set of Query Q to a node s index cache I

7 in Fig 2. If it finds a match(s), it will make reply message and includes the addresses and the bandwidth information of all the nodes it found to have the desired object. But it doesn t confirm whether the object really exists in the network. So the result may include the false positives also. If it doesn t find a match, it will forward the query to its neighbors. 4 Simulation esults We used a 1 node random graph generated by GT-ITM [21] with average degree of 1. We used a pure P2P model where there is no distinction among the peers. All peers have equal functionalities. There are 5 distinct objects in the network with the 1 st object being the most popular and 5 th object being the least popular. The objects and queries are distributed according to the observations made in [22]. The replica distribution of these objects is done according to the Zifian distribution with parameter α =.82. The replicas are placed at randomly selected nodes. Queries for these objects also follow the Zifian distribution (α =.82) i.e. the popular objects receive more queries than that of less popular objects. The queries are generated according to a Poisson process with average generating rate at 8 queries per second. For each query, a node is selected randomly to fire that query. The simulation was run for 75 queries. To model the churn scenario, we remove 1% active nodes and insert equal number of new nodes into the system. The new nodes start functioning without any prior knowledge. The objects are replicated onto these new nodes according to Zifian distribution. To model the object deletions and insertions, randomly selected objects are removed from randomly selected nodes and new replicas and new objects are added at randomly selected nodes. The topology change happens every 3 minutes and object relocation happens every 3 minutes during each run. For simulating the search method flooding, the TTL is set to 7 and for k-random walk 5 walkers are used and TTL for each is set to 124. The parameters for Bloom Filter are number of files n= 1, k=6 and maximum query size is 496 bytes. We simulate Flooding (FL), k-andom walk (W), Index Dissemination with Flooding (ITQFL), Index dissemination with k-andom walk (ITQW). The comparison is done in terms of bandwidth consumption, query response time, and false positives. In Fig 5, we show the average messages produced per query. There is 99% reduction in average messages in ITQ approaches. The difference between flooding and random walk is due to forwarding only to few randomly selected neighbors. But the huge difference between ITQ and others is due to the spreading of the object indices through the queries. By spreading indices, the location information of the objects is available within fewer hops of the requesting nodes. As the query is answered within fewer hops, correspondingly the message replication also reduces exponentially. In Fig 6, we compare index propagation through random walk and flooding. In both the approaches, the curve rises steeply for the first few seconds and rises slow after that. This is because; initially both approaches behave just like their counterparts. But the difference is that the initial query traffic carries the indices and spreads to all the nodes on its way. So there is decline in steepness of the curves. There are rises in the steepness of the curve at 184, 385, and 522 seconds indicating

8 the churn scenarios. But it can be clearly seen that the rise in the steepness reduces over next few seconds and becomes normal. Thus our approach adapts to the churn scenarios and stabilizes the network. 1 8 Me s s a ge 6 C ount Mes s age Count IT Q(FL) k=6 IT Q( W) k=6 Flooding a ndomwa lk IT QFL k=6 IT Q W k= Avg Ms gs T ime (s ec) Fig. 5. Average messages per query Fig. 6. CDF of bandwidth consumption Message 2 s T ime (s e c ) Me ssa ge s T ime (s e c ) Fig. 7. Traffic distribution for ITQFL k=3 Fig. 8. Traffic distribution for ITQFL k=7 3 2 Milli se c s e s pons e T ime 4 IT Q(FL) k=6 IT Q( W) k=6 Flooding a ndomwa lk IT QFL k=6 IT Q W k=6 Avg T ime Que r y No Fig. 9. Comparison of avg response time Fig. 1. CDF of query response time Time Unit s Query No 1 % 5 Flooding a ndomw IT Q W IT Q W IT QFL IT QFL % Fig. 11. CDF of response time (ITQFL k=6) Fig. 12. Percent queries answered in one-hop

9 As k increases, the false positive rate reduces and the length of Bloom Filter or BF record increases thus loading fewer records onto the query. This can be observed in Fig 7 and 8. First in Fig 7 there is less query traffic in the starting of the network compared to Fig 8. This is attributed to more query hits that occur in the first few hops and they don t further forward the query. The more query hits are due to high false positive rate. In Table 2, we observe that there are on an average 33.6 false results for every query. As k, increases the false positives decrease considerably. Second as k increases, the number of indices that can be loaded onto the query reduce linearly. Thus it takes time to stabilize the network. The comparison of average query response time is depicted in Fig 9. We see that random walk has maximum search latency. This is attributed to depth first search. There is Table 2. False positives per query in ITQFL k fp drastic reduction in search latencies of ITQ approaches when compared to others. The comparison between search latencies of ITQW and ITQFL is presented in Fig 1. The difference in steepness is due to, first, in random walk, the indices are spread not breadth wise but depth wise. So only few nodes come in contact with the queries. Second the nodes are selected randomly to forward the query. This means a node may be selected several times leaving other nodes not getting selected at all. This way the network has rare chance of getting stabilized. The churn further complicates this. Although in Fig 1, the curve for ITQFL seems to be constant, but if we closely look at it, it looks as in Fig 11. For most of the queries, the curve is flat, indicating that queries are answered in one-hop. The rises in steepness of the curve are due to the churn. But soon the curve becomes flat indicating that the network has recovered from the churn. The percentage of queries answered with the locally available index (one-hop queries) is shown in Fig Conclusion In this paper we propose a novel approach, indexing through querying that utilizes the huge amount of traffic to propagate the indices of the objects. Our approach utilizes the benefits of query forwarding to improve upon the scalability of P2P lookup. It also improves upon the availability in the face of individual failures through index replication. Experimental evaluations have shown that our approach works very efficiently in conjunction with flooding in improving the search efficiency. The search cost is reduced over by 95% and over 97% queries were answered one-hop. We plan to address the free riding problem in the future work. eferences 1 Saroiu, S., Gummadi, K., Dunn,., Gribble, S., Levy H.: An Analysis of Internet Content Delivery Systems. In: Proceedings of the Fifth Symposium on Operating Systems Design and Implementation, pp , USENIX Association, Berkeley, CA, USA (22) 2 Gnutella Protocol Specification Version.4

10 3 Kazaa. 4 atnasamy, S., Francis, P., Handley, M., Karp,., Shenker, S.: A Scalable Content Addressable Network. In: Proceedings of the 21 ACM Annual Conference of the Special Interest Group on Data Communication (SIGCOMM), pp , ACM Press, (21) 5 Stoica, I., Morris,., Liben-Nowell, D., Karger, D., Kaashoek, M.F., Dabek, F., Balakrishnan, H.: Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications. IEEE/ACM Transactions on Networking 11, (23) 6 owstron, A., Druschel, P.: Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems. In: Proceedings of IFIP/ACM International Conference on Distributed Systems Platforms (Middleware), pp , Springer-Verlag, London (21) 7 Yang B., Patrick, V., Garcia-Molina, H.: Evaluating GUESS and Non-Forwarding Peer-to- Peer Search. In: Proceedings of IEEE 24th International Conference on Distributed Computing Systems (ICDCS), pp , IEEE Computer Society, Washington DC, USA (24) 8 Bloom, B.H.: Space/time trade offs in hash coding with allowable errors. Communications of the ACM, 13(7), (197) 9 Yang, B., Garcia-Molina H.: Improving search in peer-to-peer networks. In: Proceedings of the 22nd International Conference on Distributed Computing Systems (ICDCS), pp. 5, IEEE Computer Society, Washington DC (22) 1 Crespo, A., Garcia-Molina, H.: outing indices for peer-to-peer systems. In: Proceedings of the 22nd International Conference on Distributed Computing Systems (ICDCS), pp , IEEE Computer Society, Washington DC (22) 11 Sean, C.., Kubiatowicz J.: Probabilistic location and routing. In: Proceedings of 21st Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM2), pp , IEEE Computer Society, Washington DC (22) 12 Zhang,., Hu, Y.C.: Assisted Peer-to-Peer Search with Partial Indexing, IEEE Transactions on Parallel and Distributed Systems. 18, (27) 13 Praveen, K., Srividya, G., Sridhar, V.: Multi-attribute Based Similar Content Indexing in Hybrid Peer-to-Peer Networks. In: Proceedings of the International conference on Networking and Services (ICNS), pp. 25, IEEE Computer Society, (26) 14 Peng, G., Jun, W., Hailong, C.: ASAP: An Advertisement-based Search Algorithm for Unstructured Peer-to-peer Systems. In: Proceedings of the 27 International Conference on Parallel Processing (ICPP), pp. 8, IEEE Computer Sociey, Wasinghton DC, USA (27) 15 Tang, C., Dwarkadas, S.: Hybrid global-local indexing for efficient peer-to-peer information retrieval. In: Proceedings of the 1st Symposium on Networked Systems Design and Implementation, pp. 16, USENIX Association Berkeley, CA, USA (24) 16 Baeza-Yates,., ibeiro-neto, B.: Modern Information etrieval. Addison Wesley Longman, Boston (1999) 17 Gummadi, K.P., Dunn,.J., Saroiu, S., Gribble, S.D., Levy, H.M., Zahorjan, J.: Measurement, Modeling, and Analysis of a Peer-to-Peer File-Sharing Workload. ACM SIGOPS Operating Systems eview. 37, , (23) 18 Saroiu, S., Gummadi, P.K., Gribble, S.D.: A measurement study of peer-to-peer file sharing systems. In: Proceedings of Multimedia Computing and Networking (MMCN) (22) 19 Fan, L., cao, P., Almeida, J., broder, A.: Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol. IEEE/ACM Transactions on Networking. 8, (2) 2 Gnutella protocol Zegura, E.W.,Calvert, K.L., Bhattacharjee, S.: How to Model an Internetwork. In: Proceedings of Fifteenth Annual Joint Conference of the IEEE Computer Societies, pp , IEEE Computer Society, New York, (1996) 22 Chu, J., Labonte, K., Levine, B.: Availability and Locality Measurements of Peer-to-Peer File Systems. In: Proceedings of SPIE (22)