Analyzing the Characteristics of Gnutella Overlays

Transcription

1 Analyzing the Characteristics of Gnutella Overlays Yong Wang Institute of Computing Technology the Chinese Academy of Sciences Beijing China Xiaochun Yun School of Computer Science and Technology Harbin Institute of Technology Harbin China Yifei Li School of Computer Science Sichuan University Chengdu China Abstract Mapping and analyzing the topological properties of P2P overlay network will benefit the further design and development of the P2P networks. In this paper, the measured Gnutella network topology is basically taken as an example. The properties of degree-rank distribution and frequency-degree distributions of the measured topology graphs are analyzed in detail. The small world characteristics for Gnutella network are discussed. The results indicate that each tier of Gnutella network shows individual characters, namely, the top level graph fits the power law in degree-rank distribution, but follows the Gaussian function in frequency-degree distribution. The bottom level graph shows power law both in its degreerank distribution and in its frequency-degree distribution. Fitting results indicate that power law could fit better for the degree-rank distribution and frequency-degree distribution of bottom level graphs, while Gaussian could describe the frequency-degree distribution of the top level graphs. Gnutella overlay network has the small world characters, but it is not a scale-free network, which has developed over time following a different set of growth processes from those of the BA (Barabási-Albert) model. The measured results show that Gnutella network has pretty well scalability as well as the abilities to tolerating failures and attacks against peers, but with low routing efficiencies. Keywords: P2P overlay network, topology measurements, scale-free network, power laws, and small world. Introduction With the rapid development of P2P networks, great attentions have been paid on P2P network topology analysis. Knowledge of appropriate metric values of P2P graphs may influence the engineering of future topologies, repair strategies in the face of failure, and P2P overlays managements. Unfortunately, most of the work in this field is focused on measuring and analyzing the degree distributions of the P2P overlay networks [~4]. The detailed scrutiny of the topological properties of complex networks has pointed out that graphs with the same degree distributions may have totally different structures; The degree rank distribution, degree distribution for each tier, and small world phenomenon of the network is one of the most important properties of topologies, which also are crucial for understanding and predicting the performance, robustness, and scalability of P2P applications. The most common method for achieving P2P overlay network topology is to construct a crawler, which works like spider of WWW network. However, discovering P2P overlay topology accurately has tremendous difficulties in that: First of all, P2P network has a large scale of size and grows rapidly, which requires high speed of peer s information collecting. Secondly, P2P overlay network has the characteristic of dynamics in nature, for happenings of peers joining and leaving network, new links adding onto P2P network are constant. Thirdly, P2P overlay network itself is highly heterogeneous. Peers connect into the P2P network by different methods, which make parts of peers behind firewalls hardly detectable [5]. Finally, zero knowledge issue. That is to say, so little is known about P2P overlay topology that it is unlikely to make much presumption due to its inherent dynamic nature, which also makes it hard to evaluate completeness, accuracy and validating of achieved topology data. The longer the crawling duration is, the more peers could be achieved, but with the less accuracy of snapshots. Therefore, the crawling system has to make trade off between topology data accuracy and completeness. Analyzing the topology characteristics of measured P2P network instances is still a big challenge for the lack of proper theories related to such complex networks. In this paper, we analyze the properties of degree and degree rank distributions, as well as small world of each tier of the Gnutella network topology with snapshots achieved The work described in this paper is supported by the National Natural Science Foundation of China under grant No

2 during Aug to Mar by our distributed Gnutella topology capturing system (called D-Crawler). 2. Related work Recently, P2P file-sharing systems have evolved in many ways to accommodate growing numbers of participating peers. New features have changed the properties of their topology. Jovanovic [6] measured the Gnutella system in 200 for the first time, which find that the Gnutella network topology is small-world and its degree distribution follows power law. As the limitations of Gnutella network old protocols, the number of total nodes achieved by Jovanovic is about K, which weakens its validation in representing the whole Gnutella network. In 2002, S. Saroiu et al. [7] has performed a detail study of the two popular P2P file sharing systems, namely Napster and Gnutella, which characterizes the population of enduser hosts participating in these two systems, including bottleneck bandwidth between hosts, IP-level latencies, frequency of hosts connecting and disconnecting from the systems, and the degree of cooperation between the hosts etc. The measurements show that there is significant heterogeneity and lack of cooperation across peers in these two P2P systems. The number of hosts achieved is about K for the lack of new methods to discover nodes in the P2P networks. Features of topology properties are not analyzed. M. Ripeanu et al. [8] implement a distributed crawling system of Gnutella network, which could capture about 30K peers in a few hours. The study concludes that the degree distributions of Gnutella network follows power-law, but the overlay network does not match the underlying Internet topology which leads to ineffective use of physical networking infrastructure. With the evolvement of the Gnutella, the properties changed significantly. A new kind of high-speed distributed crawler (called Cruiser) of Gnutella network is constructed [5], which utilizes hierarchical structure of the new Gnutella protocols; it can achieve nodes information at the speed of 40K per minute with 6 linux box machines. The degree distributions between ultra peers, ultra and leaf peers are analyzed, which have drawn conclusions that Gnutella overlay network is small-world but the degree distributions do not follow power-law. The current analysis of Gnutella overlays is mainly focusing on degree distributions. For there exist many different graphs having the same degree distributions, some of which may be considered opposites from the viewpoint of network engineering, it is incomplete and in need for corrective actions. Therefore, in this paper, we analyze properties of Gnutella s each level topology graphs by plotting and fitting degree-rank distributions, frequency-degree distributions, as well as small world phenomenon. These analyzing methods could help to understand the topological characteristics of Gnutella in detail, which has been an open problem in P2P network modeling and measurements. 3. Gnutella topology collecting We have designed and developed a distributed crawler of Gnutella system (called D-Crawler) based on positive feed back crawling strategies, which positively contacts known ultra peers to obtain several pieces of information including: () Client s version string; (2) Peer s type (ultra peer or leaf peer); (3) A list of peer s neighbors; (4) A list of peer s leaf neighbors. The system can automatically choose stable graphs and adapt its crawling behaviors according to the statistical properties of the snapshot achieved previously. D-Crawler system can capture more accurate and complete snapshots with the average nodes information achieving speed at about 60K per minute using three P4 2.8GHz/G RAM PCs. Figure illustrates the framework of D-Crawler system, the detailed analysis about its performances is discussed in ref. [9]. After required information is collected from all peers, some post-processing should be performed to remove any obvious inconsistence that has been introduced due to the dynamic changes in the Gnutella network during the crawling period, which includes: () converting snapshots into bidirectional graphs; (2) ignoring leaves which declaring themselves as the neighbors of ultra or legacy peers; (3) ignoring leaves that are parents of other leaves. About % of nodes and links are influenced by the post-processing. Figure. The framework of D-Crawler system We have captured and analyzed hundreds of snapshots during Aug to Mar. 2006, which shows that the topology relations validated by D-Crawler system have similar topological properties. To make it brief and to the point, we randomly select three snapshots as samples in this paper, the overall information of these snapshots is shown in table.

3 Table. The overall information of the snapshots Avg. Total nodes,758,76,729,63,727,945,738,773 Top level 463, ,80 465, ,55 Leaves,294,876,262,803,26,986,273,22 Avg. of top-level Avg. of leaves Characterization of Gnutella topologies Due to the topology nature of Gnutella, in this section, we analyze the following characterizations of the modern Gnutella topology: () -rank distributions, and (2) Frequency-degree distributions; (3) small world properties. These three kinds of distributions can describe the structural characteristics of Gnutella topology. It is worth clarifying that the snapshots are treated as bidirectional graphs due to the properties of TCP links. 4. -rank distribution -rank distribution, which is a powerful metric for characterizing families of graphs, can distinguish network topology of different nature. Modern Gnutella consists of two-tier sub-networks as described in section 3. Each tier has its own neighbor-choosing strategy, so it is necessary to analyze the distributions respectively. Figure 2 shows the degree-rank distributions [0] and fitness of the top-level, ultrapeers leaves and leaves parents in log-log scale, from which we find: () The degree-rank distribution of top-level (ultra to ultra peers) follows power law by three segments, the rank exponent R of each is , and respectively; (2) The degree-rank distribution of ultrapeers leaves (ultra to leaf peers) can be approximated well by the two linear regressions, the absolute correlation coefficient (ACC) is higher than 0.95 for the first segment and for the second one with R equals and for each segment; (3) The degree-rank distribution of leaves parents (leaf to ultra peers) also follows power-law by three segments, the R of each is , , and respectively. It is observed in the degree-rank distribution for the leaf to ultra peers of figure 2 that, the degree of the top 00 ranked leaves is larger than 000, the fraction of which is about 0.0% percent of total number of leaves. This is not consistent with the Gnutella protocols which specify leaf nodes should have few neighbors. The further observations show the reasons lie in: () there exist testing points in the Gnutella overlay network, which connect ultrapeers frequently to monitoring query messages in top-level graph; (2) The topology dynamics may increase degrees of the leaves which leave and join Gnutella very constantly; The distortion of topology can be reduced by decreasing crawling duration, but, for the reason of (), the testing points have changed the topology characters of Gnutella, so, the leaves with very large degree (typically larger than 000 in our observations) should be ignored when analyzing and modeling the bottom-level graphs. We also noticed that the percentage of leaves that have less than 2 parents is about 82.7%~84.3%, which is coincident with those listed in table. On the other hand, the percentage of ultrapeers with neighbors between 30 and 00 is about 62%, and the slope of the plot is nearly flat. The results indicate that the frequency-degree distribution (see section 4.2) of top-level would likely be normal Ultra to Ultra Peers exp(8.8667)*x**( ) ACC= exp(.75486)*x**( ) ACC= exp( )*x**( ) ACC= Rank U ltra to L e a f P e e rs exp( )*x**(-7.275) ACC= exp(2.0364)*x**( ) ACC= Rank

4 Leaf to Ultra Peers exp(.57382)*x**( ) A C C = exp( )*x**(-2.637) A C C = exp( )*x**( ) A C C = Rank Figure 2. The -Rank power law fitness of the measured topology graphs (in log-log plot) 4.2 Frequency-degree distribution Furthermore, we analyze frequency-degree distribution on three different objects, as top-level, ultrapeers leaves, and leaves parents. Four Probability Distribution Functions (PDFs) are selected for verifying the fitness of each distribution, listed in table 2. Table 2. Four PDFs for degree distribution fitness Function Name PDFs Power law f ( x) = Cx α ( C > 0, α < ) Gaussian Double Exponential Weibull f ( x) A ( x x0) exp 2πσ 2σ = 2 f ( x) = exp 2β γ x f ( x) = α α γ x x β 0 x exp α Figure 3 shows the frequency-degree distributions and PDFs fitness of the measured topology graphs, which indicates that: () in log-log scale, the distribution of leaves parents (leaf to ultra peers) follows power law with R=0.252 (ACC=0.989 for the fitness), but does not fit Weibull distribution well as the ACC=0.705; (2) Frequency-degree distribution of Ultrapeers leaves (ultra to leaf peers) is bell shape which, in linear scale, is fitted well by Gaussian with σ=0.933 (ACC=0.939), but double exponential function fits a little worse as ACC=0.836; (3) The distribution of top-level (ultra to ultra peers) exhibits double peeks Gaussian distribution, which, in linear scale, both Gaussian function and double exponential function can fit well with. By analyzing frequency-degree distributions of peers with different γ 2 version in Gnutella, we find that the double peeks are formed by Limewire clients with the peek around 32 overlapping upon BearShare clients with the peek 25. P(=x) P(degree=x) P(degree=x) E-3 E-4 E-5 E Leaf to Ultra Peers Power law fitting C=0.840, Alpha= ACC= Weilbull fitting Gama=.6382, Alpha= ACC= Ultra to Leaf Peers Gaussian fitting Sigma=0.9326, x0=29.409, A= ACC= Double Exponential fitting Belta=2.0402, x0= ACC= Ultra to Ultra Peers Gaussian fitting Sigma=.909, x0= , A= Sigma2=.343, x02= A2= ACC= Double Exponential fitting Belta=4.7493, x0= Belta2= , x02= ACC= Figure 3. Frequency-degree distributions and the PDF fitness of the measured topology graphs Leaves in Gnutella can choose their parents freely, Most of which have small number of parents due to the limitations of their bandwidth and computing abilities. There exist several rich ones connecting to large

5 number of parents (ultrapeers) as well as a mass of poor leaves only having one or two parents. This is consisting with the results described in (). On the other hand, Ultrapeers perform as routers in Gnutella network, the (2) and (3) mentioned above can claim that ultrapeers in Gnutella seems equal, namely, they have similarly number of neighbors which is mainly affected by the client s neighbor chosen strategies. 4.3 Small world Supposed that a vertex v in an undirected graph has d v neighbors; then there at most exist K v =d v (d v -)/2 edges between its neighbors. Let E v denote the actually existed edges between these neighbors; clustering coefficient C v of vertex v is defined as C v =Ev/Kv. Define average clustering coefficient C as the average of C v over all v: N C = C i N For friendship networks, C v reflects the extent to which friends of v are also friends of each other; and thus C measures the cliquishness of a typical friendship circle []. The Characteristic Path Length (CPL) is a measurement of the average distance needed to pass from one node to another within the graph. Nodes in Small World networks are highly clustered, at the same time, the CPL of which is bigger but pretty much equal to that of completely random graphs with the same number vertex and edges. A study by Jovanic et al. [6] in 200 concluded that the Gnutella network exhibit small world properties. In this section, we verify to what extent topologies of the Gnutella network still exhibit small world properties despite its rapid growth in network size and changes in network structure. Figure 4 illustrates the probability density distribution of shortest path between nodes on top level of the three snapshots. For the reason of the complexity calculating each node s clustering coefficient and pair-wise distance of Gnutella topology, we select 5000 nodes randomly each time to work around. The figure indicates that more than 63.3% pair-wise distance length is 4 and the whole distribution can be fit by normal distribution with sigma=0.40. Table 3 shows the ranges of clustering coefficient and CPL for the top level of the three snapshots as well as the mean values of random graphs with correspondingly the same number of vertex and edges. Besides, it includes information presented in reference [6] and three classic small world networks [2]. A graph can be loosely identified as small world network when its CPL is close to that of random graph with the same size (i.e., L gnutella L random ), but its clustering coefficient is much larger then the i= corresponding random graphs (i.e., C gnutella >>C random ). By comparing information listed in table 3, we can see that Gnutella network is small world network. The clustering coefficient increases with the size of Gnutella network, which indicates that Gnutella network is becoming tightly connected. The high clustering coefficient influences efficiency of message routing in Gnutella network, constructing a less clustered unstructured overlay network is still an open problem. P(hops=x) Gaussian fitting Sigma= x0= A=.008 ACC= Shotest path length (hops) Figure 4. The distribution and the PDF fitness of the shortest path length Network Original Gnutella Movie Actors Power Grid C. Elegans Table 3. Small world characteristics; C gnutella represents ranges of clustering coefficient for each actual graph, L gnutella denotes CPL of the graphs. C random and L random are respectively values of random graphs with the same number of vertices and edges. Networks C gnutella C random L gnutella L random Gnutella In reference [3], it is pointed out that the clustering coefficient of BA model (Barabási-Albert model) decreases with the network size, following approximately a power law C~N But, by comparing our measurements results with that of [6], it reflects that clustering coefficient of modern Gnutella increases with network size, which shows the developments of the Gnutella network over time does not follow the growth processes of BA model.

6 5 Conclusions and future work We draw conclusions from analysis in this paper as: () degree-rank distribution of the top-level exhibits three segments power laws, whereas, frequency-degree distribution is double peeks Gaussian; (2) degree-rank distribution of the ultrapeers leaves shows two-segment power laws, but frequency-degree distribution of which displayed as bell-shape ; (3) degree-rank distribution of the leaves parents is slightly abnormal, the reason of which lies in that some leaves are deployed for the purpose of query monitoring, while their frequencydegree distribution follows power law; (4) Modern Gnutella network has the properties of small world, but its evolvements follow a different set of growth processes from those of the BA model. The measured results indicate that the Gnutella overlay network has pretty well scalability as well as abilities of tolerating randomly nodes failures, but the message routing efficiency of the overlay network is low for the reason of high clustering coefficient. On the other hand, the current P2P overlay topological generators based on power-law degree distributions (the BA model) may not describe the real P2P networks (e.g., Gnutella) properly, more parameters such as degree-rank distributions and clustering coefficients et al. should be taken into considerations. Finally, the bell-shape degree distributions show that Gnutella is not scale-free network and that no super-hubs exist in the network, which are crucial to the global connectivity of the network. This implies that Gnutella overlay network (top-level graphs), which is not like SF networks, can tolerate attacks based on hub-removal strategies. We are pursuing this work in several directions, primarily based on continuously collecting more accurate snapshots from Gnutella. First, we intend to study more closely several other key topology properties (e.g., topology correlations, spectral density). Second, we plane to examine dynamics of clients participation and variations in topology structure. Finally, some security problems related to topology structure will be analyzed. [4] V Paxson, S Floyd, Wide Area Traffic: The Failure of Possion Modeling, IEEE/ACM Transaction on Networking, 995, 3 (3): [5] D. Stutzbach, R. Rejaie, Characterizing Today s Gnutella Topology, Technical Report, CIS-TR-04-02, Department of Computer Science, University or Oregon, Dec. 7, [6] M. A. Jovanovic, Modeling Large-Scale Peer-to-Peer Networks and a Case Study of Gnutella, MS. Thesis, University of Cincinnati, 200. [7] S. Saroiu, P. K. Gummadi, S. D. Gribble, A Measure me-nt Study of Peer-to-Peer File Sharing Systems, in: M ulti-media Computing and Networking (MMCN2002), Sa n Jose, CA, Jan [8] M. Ripeanu, I. Foster, A. Iamnitchi, Mapping the Gnutella Network: Properties of Large-Scale Peer-to-Peer Systems and Implications for System Design, in: IEEE Internet Computing Journal special issue on peer-to-peer networking, 2002, 6(). [9] Y. Wang, X. Yun, and Y. Li, An P2P Network Accurate Topology Capturing System, to be published in Journal of Computer Engineering (in Chinese with English abstract), 2007,9. [0] M. Faloutsos, P. Faloutsos, and C. Faloutsos, On Power-law Relationship of the Internet Topology, ACM SIGCOMM Computer Communication Review, 999, 29(4): [] D. J. Watts, S. H. Strogatz, Collective Dynamics of Small-World Networks, Nature, 998, 393(6684): [2] D.J. Watts, Six s. In: The Essense of a Connected Egde, 2003, ACM Press. [3] R. Albert, A. Barabási, A Statistical Mechanics of Complex Networks, Reviews of Modern Physics, 2002, 74():47-97 References [] C. Wang, B. Li, Peer-to-peer overlay networks: A survey, Technical Report, Department of Computer Science, HKUST, Feb, [2] O. S. Kaya, A Glance at Peer to Peer Systems, Technical Report, Centre for Telematics and Information Technology, Univ. of Twente, the Netherlands ISSN , [3] S. Sen, J. Wang, Analyzing Peer-to-Peer Traffic across Large Networks, ACM/IEEE Transactions on Networking, 2004, 2(2):