Survey of DHT Evaluation Methods

Transcription

1 Survey of DHT Evaluation Methods Markus Meriläinen Helsinki University of Technology Abstract In this paper, we present an overview of factors affecting the performance of the DHT-algorithms and give examples of how these factors have been measured from the DHT algorithms. We analyse the strengths and weaknesses of evaluation methods that are based on simulation, and make observations on the growing importance of globally distributed testbeds such as PlanetLab. We conclude that the best evaluation method of DHT is depends on the intended application for the DHT. KEYWORDS: DHT evaluation, P2PSIP 1 Introduction Peer-to-peer (P2P) networks have existed already for several years. The early systems were designed for file-sharing, but in recent years also other types of communication needs, such as Voice over Internet Protocol (VoIP) and streaming video content over the Internet, have been deployed with these technologies. Commercial applications using these technologies already exist, such as Skype [4] and P2PSIP Core Development Kit [1]. Often communication protocols are not standardized and thus applications providing essentially the same service are unable to communicate between each other. There is ongoing standardisation work at the Internet Engineering Task Force(IETF) working group called Peer-to-Peer Session Initiation Protocol(P2PSIP) [2]. The working group is set up to develop a protocol, that could be used to store and retrieve information on a network formed by peers. We refer to this network as a P2PSIP overlay. The goal of the working group is to submit P2PSIP Peer Protocol document to the IESG July 2008 [2], but the development work around the protocol is likely to continue many years forward. At present there are several competing proposal for the P2PSIP protocol. All of the proposals are using a Distributed Hash Table(DHT) algorithm as the key component for making routing decisions between the nodes of a P2PSIP overlay. For example, the DHT algorithm determine how the next hop on the route of a message is selected, and how the overlay changes as a result of nodes joining and leaving. The choice of a DHT algorithm can have a significant effect on the overall performance of the overlay. 1.1 Evaluating Distributed Hash Table Algorithms The basic function of any DHT is to map a key provided by the user to an identifier of a node in the overlay. The overlay consists of nodes, hereinafter called as peers. In order to retrieve the key associated value, a DHT provides a primitive called lookup. Lookup is a query send to the overlay and routed towards the node which is responsible for that portion of the key address space the key belongs to. If the associated value is found, it is then routed back to the initiator of the lookup. The first four DHT algorithms: Chord [21], CAN [17], Pastry [20] and Tapestry [22] were introduced roughly at the same time and the research related to the algorithms has continued active since then. Previous work has evaluated DHTs based on how well they handle churn [19], how fast they perform lookups [22], how well they are suited for interpersonal communication [9] and what the effect of overlays topology is [8]. In addition to these metrics, the DHTs can be evaluated based on how much maintenance traffic they generate, how many connections they create, etc. In this paper we focus on two of these metrics, the amount of the maintenance traffic and the length of the lookup latency. The DHTs can be evaluated from performance point of view by analysing them mathematically [16, 13], by using simulations [10, 8, 19, 11], by running experiments on existing implementations of the algorithms in laboratory networks [19], or by running the experiments over the Internet. The DHTs can also be evaluated based on how well they perform in specific task [9]. Simulations can be used to verify the correctness of the algorithm. Besides that, they can be used for determining the theoretical scalability, bandwith usage, and lookup performance. Simulations allow tests to be made in a scale that would be impractical or even impossible with actual systems. However, using simulators requires time and effort, both in terms of setting up the simulations and in terms of running them. In addition to this, most of the currently available simulators provide very limited means to collect statistics of the simulations [15]. Most of the reviewed simulators [15] did not provide means to tune the parameters of the network layer. However, in real network environments, packets may be lost or delayed and are thus not correctly modelled in the reviewed simulators. In order to get more reliable results, the measurements should be made in large scale using real nodes instead of simulations. The measurements could be made for example by deploying the DHT implementations in the nodes of the PlanetLab [3]. The PlanetLab is network of geographi-

2 cally distributed servers that provide users an opportunity to test the functionality of their applications at a global scale. Therefore it has been frequently used [5, 18, 12, 6] to provide performance data of DHT algorithms. PlanetLab reveals the performance bottlenecks of DHTs by introducing nodes with poor performance and slow response time [18]. 1.2 Handling Churn in DHTs There are several factors that affect the overall performance of a DHT. One of the most important ones, in terms of lookup time and correctness of lookups, is how well the DHT handles churn. The three most important factors identified in the study [19] by Rhea et al. are: Figure 1: The effect of PNS [19] reactive versus periodic recovery from failures calculation of message timeouts during lookups choice of nearby over distant neighbors Reactive recovery [19] means that a node in the network tries to find a replacement for a failed node immediately after it has detected that one of its neighbors has failed. In contrast, when using periodic recovery, node updates its neighbor information with periodic fixed interval. The study shows that periodic recovery improves the performance by reducing the amount of overlay maintenance traffic. Request are often sent to nodes that do not respond immediately and therefore appear to have left the overlay. In such cases, re-sending the request after a short timeout to an alternative node in the message path may not be the best solution, since the original query may not yet have reached the recipient, the message may still be in processing, or the reply is on the way back. In such cases creating a new query would just generate unnecessary traffic on the overlay. Broken links can be easily created by trusting blindly on the neighbor table information received from closest neighbours. The connectivity to the new nodes should be always checked before adding them to nodes own neighbour table [7]. Some of the DHTs implement a technique called Proximity Neighbor Selection(PNS), which is a process of choosing a neighbor among the potential neighbors for any given routing table entry according to their network latency to the choosing node. The study by Rhea et al. [19] suggest that using a small amount of technique called global sampling is necessary to achieve near optimal PNS. The use of suitable amount of PNS can improve lookup results, and bring performance gains with little or no increase in bandwith consumption, as can be seen in Figure 1. 2 How to Measure Performance 2.1 Comparing the Amount of Signaling Data Desired feature in any DHT algorithm is to keep the the amount of maintenance traffic at a minimum. When the algorithm is used in bandwith limited environment, such as Figure 2: Bandwith vs. lookup latency [11] mobile network, every byte is important. The calculations should also include the amount of traffic that is needed to keep the nodes data up to date [11]. Simulations have shown that there is no major differences in the lookup times of the algorithms, if the algorithms are allowed to use unlimited bandwith for the maintenance operations [11]. Typically the DHT algorithm generates a lot of traffic in the Join process, when the joining node exchanges routing and neighbour table information with its neighbors. After this initial exchange, the number of routing table entries (base) and the stabilization interval have the most decisive effect on the amount of generated data. The reason for this is that on every stabilization the node updates its routing table pointers and more entries means more traffic [11]. 2.2 Comparing Lookup-time The user of a DHT is naturally interested in the lookup time as the main indicator of the system efficiency. The DHT lookup can be compared to the use of a search engine, such as Google. Since all DHTs basically provide the same service, the user naturally, will want to use the fastest one. Reaching faster lookup times often means higher bandwith consumption, and vice versa as can be seen in the Figure 2. Typically, lookup time has been evaluated by running simulations of several DHTs and tweaking their parameters in order to reach faster lookup times[11, 10]. Some of the nodes in the PlanetLab are quite slow [18], which will make all lookups that go through those nodes

3 appear extremely slow also. Therefore it is important to use use techniques such as global sampling in order to prevent accepting neighbors that are behind slow links. In global sampling, the lookup functionality of a DHT is used to find new neighbors. Rhea et al. [19] describe the technique like this: "for a routing table entry that requires a neighbor with prefix p, we perform a lookup for a random identifier with prefix p. The node returned by this lookup will almost always have the desired prefix." Another scheme, proposed by Ratnasamy et al. [17], that is used in CAN DHT for PNS is called Landmark ordering. In this scheme the joining node measure its latency to few well known nodes in the overlay and based on this data orders the landmarks in order of increasing Round Trip Time(RTT). The RTT is measured by sending a message to a destination node and measuring the time from sending to the moment the sender has received a response. In CAN DHT a new node utilises its landmark ordering information by joining at random point in the portition of the coordinate space associated with its landmark ordering. 3 Factors Affecting Suitability for Interpersonal Communication DHTs can be evaluated from the viewpoint of interpersonal communication. This means that some characteristics are required in order to make the communication possible for example in mobile environment. In the Figure 3 Hautakorpi et al. [9] have gathered eleven factors that should be considered when choosing a DHT to be used in communication application: Lookup methods Parallel lookups Proximity support Graceful departure Replication & caching Complexity Bandwith consumption Node join & departure Configuration parameters Extendability Notification framework We will now go through few of them to see why these factors are so important. By lookup methods the authors refer into three different ways messages are delivered between the peers of an overlay. These methods are presented in the Figure 4. The simplest way which is generally supported by all of the DHTs is to use iterative routing. This means that during lookup when node A wants to lookup node R, the Figure 4: Routing methods query is first send to the closest node the sender knows which we shall call node B. Node B then replies with the address of which will reply with the address of a even closer node to the destination, which we shall call node C. Now node A makes a new query to the node C, which will reply with a closer match. This activity is carried out until the node R is found, or some intermediary node will determine, that there is no closer match, and the lookup fails. Some DHTs also support recursive routing, which differs from iterative routing in a way that the intermediary nodes forward the messages directly to the closest node they know instead of sending the address of the closest node back to the sender. In this method, the replies are routed back the same path back to the sender. The third method called semi-recursive routing differs from the recursive routing in a way that the replies are send directly back from the destination node to the sender node without going through the intermediate nodes that were on the message path. The iterative and semi-recursive method are not directly applicable in environments where Network Address Translation(NAT) is being used. The semi-recursive method should provide the shortest lookup delays out of these three methods, under assumption that the connections between the nodes are already established. Bandwith consumption is another important factor when it comes to running the algorithm in mobile devices, since every transaction consumes energy and mobile devices typically run on battery [9]. 4 Future Work We are planning to implement Chord and CAN algorithms using C programming language on the Linux platform and in later stage on mobile devices. The performance measurements will be run on both, the nodes of the laboratory test bed, as well as the nodes of the PlanetLab. The evaluation of DHTs will be done from the viewpoint of interpersonal communication. In later stage we are planning to expand our prototype with support to Kademlia DHT [14]. We are trying to determine how well the selected DHTs perform in environment, where NATs or bandwith limitations are present. There is an ongoing discussion in the

4 Figure 3: Feature comparison of DHT algorithms [9] P2PSIP working group [2] of whether a P2PSIP protocol standard should define a default DHT algorithm that must be present at all the protocol implementations. The results of our evaluation work could be used for making the decision on the default DHT. 5 Conclusions In this paper, we have summarized the methods that have been used for evaluating DHTs, and provided justification for some of the evaluation criteria. We discovered that most of the evaluation results so far have been obtained by using simulations. Lately globally distributed testbeds such as PlanetLab are gaining momentum as provider of reliable results. We also discussed the factors that are relevant when measuring the performance of DHT implementations to be used in interpersonal communication, and concluded that lookup time and bandwith usage both play an important role from users perspective. The upcoming P2PSIP protocol will surely increase the interest towards DHT development outside academia. The best evaluation method depends on the intended application for the DHT. In file sharing applications, multiple copies and stale data is not such a big problem, but when using DHT to store SIP contact information, these factors have significant importance. References [1] P2psip core development kit. sipeerior.com/products.php. Referenced [2] P2psip working group. html.charters/p2psip-charter.html. Referenced [3] Planelab. Referenced [4] Skype. Referenced [5] F. Dabek, J. Li, E. Sit, J. Robertson, M. F. Kaashoek,, and R. Morris. Designing a dht for low latency and high throughput. In USENIX 04: Proceedings of the First Symposium on Networked Systems Design and Implementation, March [6] J. Falkner, M. Piatek, J. P. John, A. Krishnamurthy, and T. Anderson. Profiling a million user dht. In IMC 07: Proceedings of the 7th ACM SIGCOMM conference on Internet measurement, pages , New York, NY, USA, ACM. [7] M. J. Freedman, K. Lakshminarayanan, S. Rhea, and I. Stoica. Non-transitive connectivity and DHTs. In WORLDS 05: Proceedings of the Second Workshop on Real, Large Distributed Systems, December [8] K. Gummadi, R. Gummadi, S. Gribble, S. Ratnasamy, S. Shenker, and I. Stoica. The impact of DHT routing geometry on resilience and proximity. In SIGCOMM 03: Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications, pages , New York, NY, USA, ACM. [9] J. Hautakorpi and G. Camarillo. Evaluation of DHTs from the viewpoint of interpersonal communications.

5 In MUM 07: Proceedings of the 6th international conference on Mobile and ubiquitous multimedia, pages 74 83, New York, NY, USA, ACM. [10] D. Kato and T. Kamiya. Evaluating DHT implementations in complex environments by network emulator. In IPTPS 07: Proceedings of the 6th International Workshop on Peer-to-Peer Systems, Bellevue, WA, USA, February [11] J. Li, J. Stribling, T. M. Gil, R. Morris, and M. F. Kaashoek. Comparing the performance of distributed hash tables under churn. In IPTPS 04: Proceedings of the 3rd International Workshop on Peer-to-Peer Systems, San Diego, CA, USA, February [12] J. Li, J. Stribling, R. Morris, M. Kaashoek, and T. Gil. A performance vs. cost framework for evaluating DHT design tradeoffs under churn. In INFOCOM 2005: Proceedings of the 24th Annual Joint Conference of the IEEE Computer and Communications Societies, [20] A. I. T. Rowstron and P. Druschel. Pastry: Scalable, decentralized object location, and routing for largescale peer-to-peer systems. In Middleware 01: Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms Heidelberg, pages , London, UK, Springer-Verlag. [21] I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan. Chord: A scalable peer-to-peer lookup service for internet applications. In SIGCOMM 01: Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications, pages , New York, NY, USA, ACM. [22] B. Y. Zhao, J. D. Kubiatowicz, and A. D. Joseph. Tapestry: An infrastructure for fault-tolerant wide-area location and routing. Technical report, Berkeley, CA, USA, April [13] D. Liben-Nowell, H. Balakrishnan, and D. Karger. Analysis of the evolution of peer-to-peer systems. In PODC 02: Proceedings of the twenty-first annual symposium on Principles of distributed computing, pages , New York, NY, USA, ACM. [14] P. Maymounkov and D. Mazières. Kademlia: A peerto-peer information system based on the XOR metric. In IPTPS 01: Revised Papers from the First International Workshop on Peer-to-Peer Systems, pages 53 65, London, UK, Springer-Verlag. [15] S. Naicken, B. Livingston, A. Basu, S. Rodhetbhai, I. Wakeman, and D. Chalmers. The state of peer-topeer simulators and simulations. SIGCOMM Comput. Commun. Rev., 37(2):95 98, [16] D. Qiu and R. Srikant. Modeling and performance analysis of bittorrent-like peer-to-peer networks. In SIGCOMM 04: Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, pages , New York, NY, USA, ACM. [17] S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Schenker. A scalable content-addressable network. In SIGCOMM 01: Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications, pages , New York, NY, USA, ACM. [18] S. Rhea, B. Chun, J. Kubiatowicz, and S. Shenker. Fixing the embarrassing slowness of OpenDHT on planetlab. In WORLDS 05: Proceedings of the Second Workshop on Real, Large Distributed Systems, December [19] S. Rhea, D. Geels, T. Roscoe, and J. Kubiatowicz. Handling churn in a DHT. In Proceedings of the USENIX 2004 Annual Technical Conference, Boston, MA, USA, June 2004.