Characterizing Guarded Hosts in Peer-to-Peer File Sharing Systems

Transcription

1 Characterizing Guarded Hosts in Peer-to-Peer File Sharing Systems Wenjie Wang Hyunseok Chang Amgad Zeitoun Sugih Jamin Department of Electrical Engineering and Computer Science, The University of Michigan, Ann Arbor, MI, 489, USA {wenjiew, hschang, azeitoun, Abstract We call end-hosts behind Network Address Translator (NAT) gateways or firewalls guarded hosts,and otherwise open hosts. Due to mounting security threats and growing population of high-speed Internet users equipped with connection sharing, guarded hosts become increasingly common in today s Internet. However, the presence of guarded hosts has a detrimental effect on the performance of peer-to-peer file sharing systems as they may disallow incoming connections initiated by remote peers to download files or to build an overlay. In this paper, we empirically measure the prevalence of guarded hosts in two popular peerto-peer file sharing systems, edonkey and Gnutella, and study the characteristics of their shared files. By performing passive and active probes, we found that about 25 36% of edonkey and Gnutella users reside on guarded hosts and that the ratio of files shared by guarded hosts is also non-trivial. When discounting guarded hosts, we found that a popular file s availability, i.e., the number of copies available for download, decreases by 25 3%. Our measurement study testifies to the significant impact guarded hosts may have on the performance of current peer-topeer file sharing systems, and points to a need to consider their presence when designing next generation peer-to-peer systems. I. INTRODUCTION AND MOTIVATION A peer-to-peer file sharing system [ 5] provides a framework that facilitates locating and exchanging files among different peers. Due to mounting security threats and growing popularity of high-speed Internet connections shared by multiple machines, NAT gateways or firewalls become increasingly common among peer-to-peer file sharing users. We call hosts behind Network Address Translator (NAT) gateways or firewalls guarded hosts, and otherwise open hosts. The presence of peers on guarded hosts violates two implicit assumptions inherent in any file sharing system: unique identification of peers and two-way connectivity between peers. That is, while a given peer X can always initiate a connection to another peer on an open host, the reverse may not be true if peer X is on a guarded host. To access files shared by a guarded host X, a remote host A needs a proxy server (or another peer) S that maintains a connection to X. S can route queries or download requests to X on A s behalf. X then pushes the results or requested files back to the actual recipient A. This push mechanism introduces extra overhead on the proxy server and may create long access latency for those peers making requests. There have been several measurement studies about peer-topeer file sharing systems. For example, Saroiu et al. reported that end-hosts in Napster and Gnutella systems are highly heterogeneous in terms of bottleneck bandwidth, latency and For the purpose of our study, we consider guarded hosts that enable port forwarding open hosts. availability [6]. Sen and Wang analyzed flow-level traffic in FastTrack, Gnutella, and DirectConnect systems, and found that the spatial distribution for traffic volumes is extremely skewed but relatively stable [7]. Markatos collected queries and replies issued by Gnutella peers and found that Gnutella queries exhibit significant temporal locality [8]. To our best knowledge, however, there is no existing work that reports on guarded hosts in peer-to-peer file sharing systems. In this paper, we empirically measure the prevalence of guarded hosts participating in two popular peer-to-peer file sharing systems, edonkey and Gnutella, and carefully examine the characteristics of their shared files. By performing passive and active probes, we found that about 25 36% of edonkey and Gnutella users are located on guarded hosts. We present some evidence that this percentage is a reliable ballpark figure. Furthermore, we found that a similar ratio of files are shared by guarded hosts in Gnutella. As far as the characteristics of shared files are concerned, we found that file sharing patterns of guarded hosts are qualitatively similar to those of open hosts; there are several guarded hosts that share a significant number of files, but many of them contributes only a few files. When discounting guarded hosts, we found that a popular file s availability, which is measured by the number of copies available for download in peer-to-peer systems, decreases by 25 3%. Finally, we report that in Gnutella, a significant number of search queries are returned by guarded hosts, which may degrade the performance of Gnutella network. The rest of the paper is structured as follows. In Section II, we briefly describe existing peer-to-peer file sharing systems. We explain in Section III our methodology for estimating the population of guarded hosts in edonkey and Gnutella systems. We report the ratio of guarded hosts participating in those systems in Section IV, and their file sharing characteristics in Section V. In Section VI, we provide evidence that our result on guarded host population is reliable. Finally, we discuss in Section VII the negative impacts that guarded hosts may have on the performance of peer-to-peer systems. We conclude the paper in Section VIII. II. PEER-TO-PEER FILE SHARING SYSTEMS The main difference between various types of peer-to-peer file sharing systems lies in the way peers locate files. In Napster [] and edonkey [2], dedicated centralized servers maintain information associating files indices with a list of peers storing these files. In order to locate a file in such a system, a peer simply contacts one or more servers for a list of other peers storing the file. Gnutella [3] and Myster [5] employ

2 a decentralized approach for file searching. In such systems, peers form an overlay network to propagate search messages. When a peer searches for a file, it sends out a request by controlled flooding through the overlay. If a peer receives a request for a file that is stored locally, it sends a response back to the requesting peer. Some peer-to-peer systems, such as KaZaA [4], adopt a hybrid approach. In KaZaA, some peers with sufficient bandwidth and processing power are elected dynamically to act as super-peers. Super-peers are then contacted by other peers for file searching. In our study, we examine edonkey and Gnutella, which are the popularly used centralized and decentralized versions of peer-to-peer file sharing systems. These two applications are among the top five most downloaded file sharing systems at III. MEASUREMENT METHODOLOGY In this section, we outline our main approach for estimating the prevalence of guarded hosts in the edonkey and Gnutella file sharing systems. A. The edonkey file sharing system In edonkey, a set of dedicated servers allow peers to search for files. Upon startup, a peer connects to a server, which then assigns a unique ID to that peer. There are two types of IDs assigned by each server, HighID and LowID. A peer on an open host is assigned a HighID, which is the decimal representation of the host s IP address. A peer on a guarded host is assigned a LowID, which is an arbitrary 32-bit unique number managed by its connected server. Each server determines whether a host should be assigned a HighID or LowID based on the result of a proprietary probing. Comparing the number of HighID peers and LowID peers gives a good estimate on the population of edonkey peers running on guarded hosts. When a peer X queries its connected server for a file, the server returns a list of peers, along with their IDs, that have the requested file. Upon receiving a set of matched peers from the server, peer X starts to download the file from these peers in parallel. Since we do not have control over any existing edonkey server, we modified emule [9], an open-source client, to collect the list of peers returned by the server. Specifically, our modified emule client connects to a server, and asks for files with well-known suffixes (avi, mp3, mpg, and wmv). From the list of files returned for each suffix, the client requests the 5 highest-ranked files in terms of their availability. 2 Each request prompts the server to return a list of peers storing a requested file. From these lists, it is straightforward to distinguish between HighID and LowID peers. Our emule client stays connected to each server for 2 minutes to repeat this process three times and then randomly switches to another 2 A file s availability denotes the number of peers sharing that file, as known to the connected server. When returning a list of files matched with a query, a server also reveals the availability of each file to the querying client. By selecting files with high availability in our experiment, we can trace many peers with a few downloads. server. Our study indicates that downloading the top-5 files for 2 minutes can already provide us reliable statistic on guarded host population (see Section VI for further details). B. The Gnutella file sharing system In Gnutella, there are no centralized servers that maintain a list of peers and indices for shared files. Instead, we have to empirically discover (guarded or open) peers on our own. For this purpose, we instrumented mutella [], a terminalmode Gnutella client, to perform active probing to discover live peers and collect their file sharing status. When a new peer A comes up, it sends a Ping message to well-known bootstrapping nodes, which in turn broadcast A s Ping message to the network by controlled flooding. When an existing peer B receives the message, it replies to A with a Pong message which contains B s IP address for A to connect to and information on B s shared files. The primary mechanism for a peer to locate a file in Gnutella networks is by exchanging Query and QueryHit messages. Peer A searches for files by broadcasting a Query message that describes files of its interest. Whichever peer, Q, that has some matched files in its local storage generates a QueryHit message and routes it back to A. The QueryHit message contains Q s IP address and information on each matched file (i.e., file name, file size). Our modified mutella client runs a separate probe thread which extracts IP addresses of live peers from Pong and QueryHit messages, and actively probes them on the fly to detect guarded hosts. When a new live peer is found from a QueryHit or Pong message, the probe thread attempts to connect to the remote peer on the well-known Gnutella port (6346) for five seconds. In order to minimize falsenegatives due to temporary network congestion, the probe thread attempts to connect to each remote peer three times at five-second intervals. If the connection succeeds at least once, the remote peer is considered to be running on an open host, otherwise on a guarded host. IV. PREVALENCE OF GUARDED HOSTS In this section, we report on the empirically-inferred population of guarded hosts in edonkey and Gnutella systems. In order to discern whether a given live peer is located on an open or guarded host, we check the peer s ID (edonkey) or actively probe the peer (Gnutella). A. edonkey In order to collect live clients from edonkey network, we first obtain an up-to-date list of edonkey servers to query, which contains around 5 highly available servers []. Among them, we pick the most popular 35 servers with the maximum number of clients allowed ranging from 5, to 5,. With these 35 servers, we perform the experiment described in Section III-A for a period of two days. Fig. shows the total number of hosts we discovered on each server and the percentage of hosts detected as guarded among them. The overall average of guarded host percentage

3 Fig.. Number of Discovered Hosts Server ID Ratio of guarded hosts at different edonkey servers. TABLE I RATIO OF GUARDED HOSTS ON GNUTELLA NETWORKS (FEB. 4, 24). QueryHit-responding hosts Pong-responding hosts Prober # of probed hosts Guarded % # of probed hosts Guarded % A 28, % 8,6 34.8% B 8, % 22, % C 2, % 24, % D,797 3.% 7, % E 3, % 6, % F 5, % 5, % Total 84, % 78, % on those 35 servers is about 36%. However, the ratio of guarded hosts on each server varies significantly from % to 8%. This is because an edonkey server can control the percentage of LowID hosts connected to itself, or even disallow LowID hosts. B. Gnutella We run our modified mutella client from six different machines on local campus network for a period of one day. Table I summarizes the results from our one-day probing experiment. The second and the third columns in the table describe probing results with the set of hosts retrieved from QueryHit messages, and the right two columns with the set of hosts extracted from Pong messages. In both cases, the percentage in the last row Total indicates the fraction of hosts considered guarded by all six probing machines. Overall, about 25 36% of the hosts that have issued QueryHit or Pong messages are identified as guarded hosts by our experiment. V. FILE SHARING CHARACTERISTICS A. edonkey In our study of file sharing characteristics in edonkey network, We limit ourselves to the four most popular file types (avi, mp3, mpg, and wmv). Table II summarizes the collected meta data collected from edonkey network. As described before, our modified client connects to each of the selected 35 servers three times. Each time it is connected to a server, it requests the 5 most popular files of the four different media types. The set of the 5 most popular files may vary among servers. The second and the third column of Table II report the total number of files we thus discovered, and their median size respectively. The fourth column reports the total number of peers that share any of the files reported in the second column. Finally, the fifth column shows the ratio of guarded hosts among them. TABLE II RATIO OF GUARDED HOSTS IN EDONKEY NETWORK (FEB. 9, 24). File type # offiles Median size # of sharing hosts Guarded % avi MB 45, % mp MB 6, % mpg MB 33,6 3.3% wmv MB 9, % total, MB 2, % ) File availability: With the,486 files reported in Table II, we then examine the availability of each file (i.e., the number of peers that share a given file). Given a file, we first check how many peers share the file, and then look at how many of them are actually located on open hosts. In a sense, the number of open host-based peers that share a given file better represents the file s availability as those peers on guarded hosts do not openly contribute to file sharing. Fig. 2 plots the number of open hosts that share a given file. We sort files based on the number of hosts sharing them and assign a rank ID to each file. Given a file f, we report in y-axis (i) the total number of hosts that share f (labeled all hosts ) and (ii) the number of open hosts among them (labeled open hosts ). Discounting those peers on guarded hosts, we find out that the availability of each file (especially for those files whose availability are larger than ) decreases roughly by one-third. When focusing on specific file types (e.g., avi and mp3) and considering them separately, we observe similar trends. 2) Number of files per-host: We also study the number of files discovered on each individual peer. Without running a server on our own, we cannot obtain a complete list of files shared by a remote peer in edonkey. 3 By querying servers with search keywords, we can only get a partial list of shared files for a peer. Although requesting a larger number of files from servers would reveal more complete file sharing status of each peer, the search configuration of a typical edonkey server does not allow one to request more than several thousands of files from each server. In Fig. 3, we examine the number of files shared by each open or guarded host in edonkey. Given two sets of hosts (open and guarded), we sort them separately by the number of their shared files and identify them by their ranks on x-axis. The number on y-axis represents the number of files shared by a corresponding host. According to the figure, most open hosts share less than ten files, but several open hosts share over files. We have discovered fewer files for guarded hosts, with maximum number of files less than ten. B. Gnutella As described before, our mutella client gleans file sharing status of individual peers from Pong and QueryHit messages. A Pong message reports the total number and the total byte amount of files that are shared by a given peer. A QueryHit message, on the other hand, reveals information on a specific 3 There is a feature called friend list of the emule client software that allows a peer s friend to see its shared file list. However, this feature is disabled by default and our experiment exploiting this feature turned out to be unsuccessful.

4 9 8 7 all files: all hosts all files: open hosts open hosts guarded hosts 6 5 Number of Clients Number of Files GB File ID Host AVI RM MPG MP3 RMVB BIN ISO RAR ZIP WMV Fig. 2. Number of hosts per-file. Fig. 3. Number of files per-host. Fig. 4. Top- file types of Gnutella networks. file (i.e., file size and file name) shared by a peer. While collecting per-peer file sharing summary from Pong messages allows us to estimate the size of data storage that each peer makes available for potential upload, snooping QueryHit messages enables us to locate where popular files (which are requested frequently) are hosted. While we were conducting our experiment, we identified a handful of peers that generate a significant number of QueryHit messages which are apparently bogus. Also, we encountered many peers with private addresses (e.g., 92.68/6). In the rest of our study, we discard all the control messages that originate from those misbehaving peers or privately addressed ones. ) QueryHit-based sharing info: We extract from each QueryHit message a (<file size>, <file extension>, <peer s IP address>) tuple. Given a set of three-element QueryHit tuples, we estimate the availability of each shared file. To calculate file availability, we group all the QueryHit tuples associated with an identical file size, and count the distinct IP addresses in them. In our analysis, a set of duplicate QueryHit tuples (if any) are counted just once. We also distinguish between peers on open hosts and guarded hosts. We present in Fig. 4 the top- most popular file types that are shared in Gnutella. We classify all collected QueryHit tuples based on their file types, and then sum up file sizes for each type. If a given file is shared by multiple peers, we sum up its size as many times as the number of sharing peers. In the figure, the height of each bar denotes the total amount of storage for shared files of a given type, and a smaller darker bar represents the ratio of the storage used by guarded hosts. The top- most popular file types include well-known video (e.g., avi, rm, mpg, rmvb, wmv), audio (e.g., mp3), and archive (e.g., bin, iso, rar, zip) formats. The fraction of files that are shared by guarded hosts ranges from 25% to 45%, which is comparable to the guarded hosts percentage reported in Table I. This means that guarded hosts are non-negligible not only in terms of their numbers, but also in terms of their sharing volume. Fig. 5 shows two distribution of file availability. In the distribution labeled, we do not distinguish peers on guarded hosts and those on open hosts while the distribution labeled only reports the number of peers on open host. As can be seen from the figure, file availability exhibits heavy-tailed distribution, where the availability spans over more than two orders of magnitude. When discounting peers on guarded hosts, the availability of the most popular file decreases from 8 to 6, but the overall distribution still remains qualitatively the same. 2) Pong-based sharing info: Unlike edonkey, Gnutella allows one to retrieve a complete file sharing summary from remote peers through Pong messages. We now examine the file sharing information collected from Pong messages. In Fig. 6, we plot the (-CDF) distribution for per-host sharing volume that we infer from Pong messages. In the figure, we plot two distributions, one for open hosts and the other for guarded hosts. The evidence for the abundance of freeriders in Gnutella is prominent in the figure, where about 4 5% of peers do not share any file. Adar and Huberman made a similar observation [2], but without distinguishing between open hosts and guarded hosts. Comparing two distribution for open and guarded hosts, one can see that peers on open hosts tend to store more files than those on guarded hosts. The entire distributions still remain qualitatively similar. VI. REPRESENTATIVENESS OF GUARDED HOST PERCENTAGE Since we do not have control of any edonkey server, we can only collect a partial list of live peers in edonkey network by querying existing servers. We ask whether the client set we collected is representative of edonkey population as far as guarded host percentage is concerned. There are two factors that may affect our estimation of guarded host percentage: the number of files queried and the time period our collector client stays connected to a server. By requesting more files, a client can increase a chance to discover more peers from the connected server. Also, a client that stays connected to a server for a longer period may collect more peers from the server by querying it periodically. To examine the effect of these two factors, we perform the following experiment. We select four highly populated servers, one in North America, two in Europe, and one in Asia. For each server, our client stays connected for two hours, and downloads the top-5 files of well-known types (avi, mp3, and mpg). To examine whether searching for a varied number of files yields different guarded host percentage, we calculate the ratio of guarded hosts that share files of the top-n availability, and check how the ratio changes as we vary n from to 5. We report the ratio in Figs. 8(a) and (b) for file suffix avi

5 - CDF.... e-5 - CDF.... Number of QueryHits Issued e-6 Availability per File Fig. 5. Per-file availability distribution. e-5 Number of Files Shared by a Host Fig. 6. Pong-inferred sharing volume distribution. Host Rank Fig. 7. Frequency of QueryHits. Ratio of Guarded Hosts server server2 server3 server File Rank (a) AVI file Fig. 8. Ratio of Guarded Hosts server server2 server3 server File Rank (b) MP3 file Ratio of guarded hosts on different servers. and mp3. Clearly the ratio of guarded host flattens beyond file rank ten. Therefore, focusing on top-ten files (in terms of their availability) sufficiently captures the ratio of guarded hosts reliably. We also checked how sensitive the guarded ratio is to the duration of our collection period. When our client remains connected to a server more than 2 minutes, the ratio of guarded hosts discovered on the server stabilizes sufficiently. As one of our future work, we plan to measure the guarded host population in Gnutella and edonkey networks at different time periods to check the temporal variation. VII. IMPACTS OF GUARDED HOSTS In this section, we examine one possible impact that guarded hosts may have on the performance of peer-to-peer file sharing systems. Here we focus on Gnutella system and ask the following question. When node A has issued a Query message and later receives a QueryHit message from a remote peer, how likely is the QueryHit message to originate from an open host (or a guarded host)? This question is practically important as QueryHit messages from a remote peer on a guarded host may introduce extra overhead for A to retrieve a corresponding file. Here we compare the frequency of QueryHit messages issued from open hosts and those from guarded host. We use the set of QueryHit messages collected by Prober A in Table I. In Fig. 7, we plot the frequency of QueryHit messages issued by each peer. All known open peers (labeled Open hosts ) are sorted in decreasing order of their QueryHit messages frequency. Based on this order, each peer has a rank (the x-axis). For each ranked host we plot in the y-axis the number of QueryHit messages issued by that host. In the the same figure, we also plot the frequency of QueryHit messages issued only from guarded hosts (labeled ). The top most ranked open host issued more than twice as many QueryHit messages as the top most ranked guarded host. In both cases, about % of top-ranked hosts are responsible for 8% of all the QueryHit messages issued. In particular, the presence of guarded hosts that flood QueryHit messages (e.g., top-% guarded hosts) has a detrimental effect on the performance of Gnutella networks as the files exposed in their QueryHit messages may not be available to all peers. VIII. CONCLUSIONS Facing the on-going threat of malicious Internet traffic and security breaches, an increasing percentage of end-hosts are now protected by firewalls. Nowadays many residential customers of high-speed Internet are equipped with software or hardware mechanisms to protect their computers and to share their connections. These trends accelerate the growth of socalled guarded host population in today s Internet. Our study empirically confirmed the prevalence of guarded hosts and the significance of their file sharing in two of the most popular peer-to-peer file sharing systems. We found that about 25-36% of edonkey and Gnutella users are located on guarded hosts and that they share a nontrivial amount of files on peer-to-peer systems. The prevalence of guarded hosts not only inhibits fair sharing among peers, but also may interfere with efficient overlay construction for overlay-based peer-to-peer systems. In this sense, our measurement study points to a need to consider their presence when improving an existing peer-to-peer system or designing a new one. REFERENCES [] Napster, [2] edonkey Network, [3] Gnutella, [4] KaZaA, [5] Myster, [6] S. Saroiu, K. Gummadi, and S. Gribble, A Measurement Study of Peer-to-Peer File Sharing Systems, in Proc. of MMCN, January 22. [7] Subhabrata Sen and Jia Wang, Analyzing peer-to-peer traffic across large networks, in Proc. of Internet Measurement Workshop, 22. [8] Evangelos P. Markatos, Tracing a large-scale Peer to Peer System: an hour in the life of Gnutella, in Proc. of IEEE/ACM International Symposium on Cluster Computing and the Grid, 22. [9] emule Client, [] Mutella, [] edonkey server list, [2] Eytan Adar and Bernardo A. Huberman, Free Riding on Gnutella, First Monday, vol. 5, no., 2.