APPROVAL PAGE FOR GRADUATE THESIS OR PROJECT

Transcription

1 APPROVAL PAGE FOR GRADUATE THESIS OR PROJECT SUBMITTED IN PARTIAL FULFILLMENT OF REQUIREMENTS FOR DEGEREE OF MASTER OF SCIENCE AT CALIFORNIA STATE UNIVERSITY, LOS ANGELES BY SNEHAL NIRAV PATEL Candidate COMPUTER SCIENCE Field of Concentration TITLE: Peer Discovery Protocol for JXTA Peer-to-Peer Network APPROVED: Dr. HUIPING GUO Faculty Member Dr. EUN-YOUNG KANG Faculty Member Dr. RAJ PAMULA Department Chairperson Signature Signature Signature DATE: 12/1/2006

2 PEER DISCOVERY PROTOCOL FOR JXTA PEER-TO-PEER NETWORK A Thesis Presented to The Faculty of the Department of Computer Science California State University, Los Angeles In Partial Fulfillment Of the Requirements for the Degree Master of Science By Snehal Patel December 2006

3 2006 Snehal Nirav Patel ALL RIGHTS RESERVED ii

4 ACKNOWLEDGEMENTS I would like to express my deep and sincere gratitude to my advisor, Professor Huiping Guo, for her invaluable guidance and encouragement, without which this thesis would not have been completed. Her advice and knowledge have been invaluable. I would also like to thank Professor Raj Pamula, Professor Russell Abbott, and Professor Eun-Young Kang for their suggestions and support. I would also like to thank my colleagues Alla, Deepak, Laila, Geetha, and Roudabeh for their encouragement. I would like to thank my husband Nirav, without whose love and cooperation, this thesis would not have been finished. I would like to dedicate this thesis to my parents and my husband. iii

5 ABSTRACT Peer Discovery Protocol for JXTA Peer-to-Peer Network By Snehal Nirav Patel Peer-to-peer (P2P) networking has recently emerged as a new paradigm for resource sharing, for building distributed network, and for grid technology. Peer-to-peer networks offer the significant advantages of decentralized, faulttolerant, and load-balancing systems, with simplicity and ease of use. Despite the multidimensional advantages of Peer-to-peer networks, they face some serious challenges such as flooding, security, and scalability. JXTA is relative newcomer in P2P technologies. JXTA peer discovery mechanism is fully decentralized and it works with query request and query response. This thesis addresses one of the major deficiencies of P2P network: scalability. Additionally, the thesis proposes a schema for peer discovery for JXTA P2P network. The proposed schema can be used for any peer-to-peer networks and is based on a semi-centralized hierarchical tree structure, with edge peers that are grouped together based on their network proximity. Each group will be joined to a service provider called an agent and these agents can send requests and respond to messages on behalf of peers. The rendezvous peers are designed as an agent. This type of schema would provide improved scalability than the fully decentralized peer discovery protocol currently incorporated in JXTA. iv

6 TABLE OF CONTENTS ACKNOWLEDGEMENTS...III ABSTRACT...IV LIST OF TABLES...VII LIST OF FIGURES...VIII CHAPTER 1. INTRODUCTION JXTA The JXTA Protocols Categories of Peers The JXTA Peers and Peer Groups Peers Peer Groups The JXTA Advertisements The JXTA Pipes RELATED WORK A Decentralized and Self-Organizing Mechanism An Incentive Message Relaying Mechanism PEER DISCOVERY PROTOCOL Overview of the Model Agent Based Framework v

7 4.2.1 Determination of the Number of Agents and the Threshold for Each Agent The Grouping Function The Caching Mechanism of the Peer Evaluation of the Framework The Agents as Service Providers The Update Mechanism (Join/Leave) Message Types in the Hierarchical Model Failure Handling and Recovery Performance of the Hierarchical Model FUTURE WORK CONCLUSION BIBLIOGRAPHY APPENDIX: DISCOVERY MECHANISM PARAMETERS vi

8 LIST OF TABLES Table 1 Notation for grouping function Table 2 The Hierarchical Model Agents Table 3 The Seven Basic Message Types vii

9 LIST OF FIGURES Figure 1 The JXTA three-layer Architecture... 8 Figure 2 The JXTA Protocol Figure 3 The JXTA Peer Discovery Protocol viii

10 CHAPTER 1 Introduction The concept of peer-to-peer (P2P) networking is certainly among the top contenders for the highest impact in the shortest amount of time since its introduction. The field is driven by a huge demand in tasks such as file sharing, messaging, and grid resource sharing. The user of the modern Internet, being aware of the resources that are available on other users' systems, would like to share these resources with a minimal overhead and without third-parties being involved in the process. Peer-to-peer networking is the technology that has the most potential to satisfy this demand. The technical aspects of peer-to-peer networking have evolved rapidly, always driven by the applications that the new network should be able to support. Contrary to the development of the traditional Internet which first emphasized a set of generalized precepts and then used them to implement applications, in the realm of peer-to-peer networks, the application has always been the main focus. Only after deconstructing end-applications have the theoretical concepts of peer-to-peer networks been elucidated, though the fundamental idea to directly share resources that are located on users' computers has remained unchanged. However, the implementations thereof differ greatly, beginning with peer-to-peer networks with centralized management of which resources are available and only leaving the sharing of the resources itself up to the users. Since then, the trend has been to pull these 1

11 management and discovery operations into the peer-to-peer network itself. This goal has been achieved with various levels of success in different implementations and still regarded as an area of ongoing work and research. Peer-to-peer computing is defined as the sharing of computer resources and information through direct exchange. P2P systems and applications are distributed systems without any centralized control or hierarchical organization, where the software running at each node is equivalent in functionality. These huge networks do not depend on any dedicated network servers to provide services. Each peer is considered a server and a client simultaneously called a servent. They provide client-side interfaces through which users can issue queries and view search results, while at the same time also accepting queries from other servents, checking for matches against their local data set, and responding with applicable results. The P2P network is self-organized and selfadministrative, as the nodes automatically try to find new peers if the current peers are disconnected from the network. There is no restriction on the number of peers in the network and the peers can join and leave the group dynamically without explicit knowledge of their memberships. Peer-to-peer networks are becoming increasing popular because they offer the significant advantages of simplicity and ease of use, scalability, and robustness. The most popular peer-to-peer networks are Gnutella and Napster. JXTA is newly comer in peer-to-peer networks. The increased need to share resources and information in large scale distributed environments, the 2

12 widespread use of networks and software technologies, and decreasing cost of powerful workstations have increased the usage of P2P networks in real-time communication. The advantages of P2P networks are multidimensional; they improve scalability by enabling direct and real-time sharing of services and information; enable knowledge sharing by aggregating information and resources from nodes that are located on geographically distributed and potentially heterogeneous platforms; and provide high availability by eliminating the need for a single centralized component [10]. Despite the significant potential of P2P networks, they have certain limitations. One of the most important problems of the P2P network is scalability. Large P2P computing spaces raise issues in complexity and scalability of file sharing. For example, scalability becomes a problem in some P2P networks, like Gnutella, due to the flooding of messages. Flooding means that a peer host sends a message to all the other peers in the network. Flooding on every request is clearly not scalable, and because the flooding has to be curtailed at some point, a user may fail to find the content that is actually in the system. Additionally, the network might be bombarded with lots of messages and after exceeding its capacity, may fail. Some may argue that the Gnutella protocol cannot exceed more than a few thousand-peer nodes [36]. Conversely, in Napster, a central server stores the index of all the files available within the Napster user community. To retrieve a file, a user queries this central server using the desired file s well-known name and obtains the IP 3

13 address of a user machine storing the requested file. Hence, when millions of nodes connect to the network, the centralized server might get overloaded and there is a chance of failure. Again, Napster is clearly not scalable. Different P2P networks have emerged with a variety of designs, but most have scalability problems because of bombardment of messages in the network. Current P2P networks lack efficient group communication. Different messaging systems are known that address the increased communication demand of distributed systems, however they cannot be used in P2P network as they depend on centralized message servers and are designed in a way to provide more reliability than performance. But, in a P2P network, performance is considered to be more important than reliability. Hence, special systems are required for P2P networks. The group communication mechanisms in current P2P network are very weak. The network is composed of heterogeneous peers that have different bandwidth and latency. As the topology of the network is very random, peers with high latency will slow down the entire system as messages need to traverse all of the peers that lie on a communication route, even if they are not involved. This results in consuming bandwidth of the uninvolved peers and slows down the entire communication system [11]. The topology of the current P2P network is such that it cannot dynamically adapt to the changes needed when certain peers join or leave a group. Current P2P networks do not provide anonymity to both producers and consumers. Because of a continued desire among individuals to protect the 4

14 privacy of their authorship or the readership of sensitive information, current P2P network should provide security, reliability, and anonymity to their users. Gnutella violates the anonymity feature as the immediate peer host knows the source because value zero in the Hops field indicates the source address. In the queryhit packet, there is the IP address field. This is a peer host holding the desired files. Any hosts, which receive the packet, know the address [12]. Current P2P networks also lack dynamic storage of information in the peers and delete the records as and when necessary. All of the above problems could be resolved if the design of the network topology is based on the information about the hosts. Current P2P networks are designed in such a way that they lack information about the characteristics of hosts that choose to participate in currently popular P2P systems. The system must take into account the suitability of a given peer for a specific task before explicitly or implicitly delegating the task to the peer. As mentioned earlier, the major problem of the P2P network is scalability in terms of the number of users. A common P2P file-sharing network like Gnutella has the scalability problem because of its flooding method. A peer host sends a message to all the other peer hosts which cause bombardment of messages in the network and hence this kind of approach is not scalable. Let s take an example. Suppose, in a P2P network like Gnutella, one million hosts connect to the network and a particular host is trying to search a file. What the Gnutella protocol will do is it will broadcast the query message to all of the 5

15 other hosts that are connected to the network, creating a bombardment of messages. In a similar way, if every host tries to query a file, then there will be million times million messages in the network and the network might collapse at a certain point. Also, in the worst case, every host will receive a million messages, which will just consume the bandwidth of each host. In a similar way, Napster is not scalable. It depends on one a centralized server. If a million hosts connect to the network, the centralized server might get overloaded and will start dropping messages. Hence, the query message never reaches the server and the query is not served to the querying host. Also, if the server fails, then no messages will be served and the system will be a total failure. Hence, as the number of users increases, most of the current P2P files sharing networks are not scalable. The goal of this thesis is to suggest a peer discovery protocol for JXTA P2P network to provide scalability but this protocol can also be used for any P2P network. The model that we are proposing for peer discovery protocol for P2P network is an agent-based hierarchical model with agents at different levels of hierarchy and a grouping of peer hosts based on their network closeness. Each group is connected to an agent and the agent provides service to its hosts. The major problem that we will address in our thesis is the discovery a peer host s agent and subsequent grouping mechanism, to provide scalability. 6

16 CHAPTER 2 JXTA The JXTA (juxtapose or juxta) technology was introduced by Sun Microsystems. JXTA is a set of open, generalized peer-to-peer protocol specifications that allows any connected device, from cell phone and PDAs, to desktops and servers; to network, communicate and collaborate in a fully decentralized manner [1, 2, 3]. JXTA is an open-source project. This chapter introduces the JXTA peer-to-peer network and elucidates the basic concepts of the six main protocols of JXTA. In JXTA all of the crucial work is done by these six protocols. The peers in JXTA are divided into edge peers or super peers. The peer is an individual peer or it can be grouped with other peers. As JXTA is developed in Java and XML, all of the request and replies are carried through a XML-based template called advertisements. The communication is carried through JXTA pipes. The JXTA protocols are language- and platform-independent, defining a set of XML messages to coordinate some aspect of peer-to-peer networking [4]. XML allows implementers of the JXTA protocols to leverage existing toolsets for XML parsing and formatting [2]. The JXTA protocols make it possible to implement peer-to-peer solutions on any device. The typical problem with peer-to-peer network applications is that they have specific protocols for specific peer-to-peer applications only. The users on one peer-to-peer application cannot communicate with the users on a different 7

17 peer-to-peer application. This problem is solved by the JXTA as it uses standard protocols specification. The JXTA platform is divided into three layers: JXTA applications, JXTA services, and JXTA core. Each layer builds on the capabilities of the layer below, adding functionality and behavioral complexity [5]. The core layer is where the code for implementing the protocol is found. The protocols provide the functionality for peers, peer groups, security, monitoring, message passing, and network protocols. Figure 1 The JXTA three-layer Architecture The services layer is built on top of the core layer, which uses the protocols to accomplish a given task. All of the services that could be created will allow peer-to-peer applications to be written more quickly and also allow the sharing of code. 8

18 The application layer is where the developer is able to construct a peerto-peer application that will be used by others in the Internet community. 2.1 The JXTA Protocols JXTA has six main protocols based on XML messages. The protocols can be implemented using any language, thus allowing heterogeneous devices to exist and communicate with one another. Each of the JXTA protocols addresses exactly one fundamental aspect of peer-to-peer networking. Each protocol conversation is divided into two parts. A portion conducted by the local peer and another portion conducted by the remote peer. The local peer s half of the protocol is responsible for generating messages and sending them to the remote peer. The remote peer s half of the protocol is responsible for handling the incoming message and processing the message to perform a task [2, 4]. The six main protocols of JXTA: Peer Resolver Protocol: Used to send a query to any number of other peers and to receive a response. Peer Discovery Protocol: Used to advertise content and discover content. Peer Information Protocol: Used to obtain peer status information. Pipe Binding Protocol: Used to create a communication path between peers. Peer Endpoint Protocol: Used to find a route from one peer to another peer. 9

19 Rendezvous Protocol: Used to propagate messages in the network. Figure 2 The JXTA Protocol Each protocol is semi-independent of the others. The JXTA protocols are implemented above the network layer. A peer can select to implement only a subset of protocols to provide functionality. For example, a peer could rely on a preconfigured set of router peers and so, would not require an implementation 10

20 of the Endpoint Routing Protocol. These six protocols are all that are needed for an individual peer to exist in a decentralized peer-to-peer environment [1, 2, 3, 5]. 2.2 Categories of Peers The JXTA has two main types of peers: edge peers and super peers. In the JXTA each peer has well-defined roles: Edge Peers: Edge peers can be defined as peers who have low bandwidth network connectivity. These types of peers usually reside on the border of the Internet, hidden behind the firewalls or accessing the network through non-dedicated connections. Edge peers implement the minimum JXTA capability and will either use a TCP/IP or an HTTP connection to the JXTA network. The edge peers maintain lists of rendezvous peers [1, 2, 3]. Super Peers: Super peers can be further divided in to rendezvous peers and relay peers. Rendezvous peers are super peers in the JXTA network that help an edge peers discover other available peer resources. Relay peers allow the peers who are behind firewalls or NAT systems to take part in the JXTA network [5, 6, 8]. Each peer in JXTA is either an edge peer or a rendezvous peer (super peer). 11

21 2.3 The JXTA Peers and Peer Groups Peers The most common component of any peer-to-peer system is the peer. A peer is an application, executing on a computer device, which has the ability to communicate with other peers. In JXTA the peer has the following properties: Peer Identity: A peer needs to be known by other peers. Peer Membership: A single peer is not very useful. Peer Transport: Peers must communicate with other peers to survive Peer Groups When several peers get together to share files or work on a large, computationally intensive problem, they have formed a group. The formation of a group is usually attributed to several things: Membership to a shared system Common transport Access to a centralized server Peer groups have a number of services, which have been defined as a core set by the specification. The services are as below: Discovery Service: Allows searching for peer group content. Membership Service: Permits validation of a peer group. Access Service: Permits validation of a peer. Pipe Service: Allows creation and use of pipes. 12

22 Resolver Service: Allows queries and responses for peer services. Monitoring Service: Enables peers to monitor other peers and groups. 2.4 The JXTA Advertisements When the peers and peer groups have services that they want to make known to the peer-to-peer network, they use an advertisement. An advertisement is an XML-based document that describes JXTA resource and content. The JXTA protocols use advertisement to pass information. All advertisements are hierarchical in nature and will contain elements specific to the advertisement type [1, 2]. 2.5 The JXTA Pipes JXTA uses pipes as its communication mechanism from the UNIX operating system and its shell. Information is put in one end of the pipe, and it comes out at the other end. Through pipes, messages can be sent between peers without having to know anything about the underlying infrastructure. Pipes use the concept of an endpoint to indicate the input and output points of communication [1]. The Java implementation of the JXTA specification has three pipe types: Unicast: This is one-way, insecure, and unreliable type of pipe. Unicast Secure: This is one-way, secure, and unreliable type of pipe. Propagating: This is propagating pipe, insecure, and unreliable. 13

23 2.6 Problems with JXTA One important element that JXTA does not try to address is how services are invoked other than the core services [6, 7]. Several other arguments arise against the flexibility that the designers of JXTA infused throughout the JXTA Protocols Specification. Although JXTA s use of XML specifies all aspects of P2P communication for any generic P2P application, JXTA might not be suited to a specific standalone P2P application [1]. The platform s abstraction of the network transport is another potential area of excess. One of these problems is global naming. JXTA attaches a unique ID generated by the users with each peer in the group but does not solve the global problem because it does not guarantee uniqueness across the entire community of millions of users. JXTA's peer discovery protocol is fully decentralized. Because of this arrangement, the network can be overloaded with queries. An overloaded peer will discard requests from other peers. The thesis proposes a semi-centralized, agent-based hierarchical schema for peer discovery for JXTA. In the suggested model, edge peers are considered as a host and super peers are considered as an agent. An agent acts as a service provider. This chapter highlights the basic terminology used for JXTA, including its protocols, peers, advertisements, and pipes. In addition, problems with JXTA were also explored. In the subsequent chapter we will examine the related work that has been completed in peer discovery of P2P networks. 14

24 CHAPTER 3 Related Work In this chapter, we will illustrate recent methodologies used for peer discovery and discuss other suggested techniques for peer discovery. One of them is a decentralized and self-organizing discovery, and the other is an incentive message relaying mechanism. Peer-to-peer systems are typically composed of a large number of distributed, heterogeneous, autonomous, and highly dynamic peers. Dynamic or mobile networks with peer-to-peer functionality require addressing new networking challenges. In such environments peers manage data, and collectively, large amounts of data dynamically enter, leave, and migrate around the network. Discovery of data is a basic functionality that enhances the accessibility of information in the distributed context. In a dynamic environment, network services such as discovery should be decentralized in order to avoid potential computation bottlenecks. Such bottlenecks result from the need for knowledge about a centralized location, and the need for updates to that location whenever local data properties may change. Also, as network systems become large or highly dynamic, self-organization provides a reasonable mechanism for the ease of configuration and administration of such a dynamic network. As a result, peer-to-peer applications need sophisticated discovery mechanisms to enable peers to find, identify, and communicate with other peers. 15

25 Centralized discovery mechanisms may not scale, while decentralized discovery mechanisms may not provide the guarantees required by the application [18]. One of the main goals of peer-to-peer networks is to find the peer who has resources. The scalability of network depends upon efficiency of the peer discovery. There have been many strategies proposed for the peer discovery mechanism. Many existing peer-to-peer discovery mechanisms focus on implementing a robust distributed hash table where data reference hash values are stored and retrieved [15]. These mechanisms focus on locating individual hash values, which may limit the performance of other types of queries. For example, objects that are not exactly similar will be hashed into relatively random locations on the overlay network. This may make it costly to locate other objects that are similar to a query since the number of overlay network hops between similar objects may be quite large. Some other existing mechanisms use history to adapt the discovery mechanism [19]. History represents performance of past discoveries and is used to improve the performance of frequent queries over time. Widely used peer-to-peer network such as Gnutella uses breadth-first search (BFS), while Freenet uses depth-first search (DFS) [34, 33]. In breadthfirst search each search query message is assigned a maximum time-to-live (TTL), which limits the search depth in a number of hops. The requesting peer sends a query message to all its neighbors, and all nodes who receive the 16

26 message relay it to all their neighbors, who forward the message to the next depth, and so on. The process ends when the TTL is reached. In depth-first search each node relays the query to a single neighbor at one time, and waits for the response from the neighbor before forwarding the message to another neighbor or forwarding the result back to the requestor or its upstream node. The search depth is also limited. With BFS the messages are flooded in the system. Therefore the consumption of bandwidth is enormous, although results can be found very quickly. With DFS searches can be terminated once a result is found, and therefore use less bandwidth. But the response time could be very long and is exponential in the depth limit. It is also more difficult to implement than the breadth-first search model. The latest strategies proposed for peer discovery mechanism are a decentralized and self-organizing discovery mechanism, and an incentive message relaying mechanism [16, 17, 22]. The strategies are explained in the following section. 3.1 A Decentralized and Self-Organizing Mechanism In a decentralized and self-organizing discovery mechanism, they propose a discovery mechanism that structures relationships and forwards discovery queries based on keyword similarity, usage history, and small-world clustering. The peers in the networks are considered as agents that contain a limited number of relationships to the other agents. The algorithm described by Moore and Suda is divided into following sections: 17

27 the definition of agents and their relationships how the overall network of relationships are to be organized how relationships are acquired and selected to meet this desired relationship network organization how the relationship organization is used for discovery forwarding [16] Agents represent data, applications, or users available on the network and contain keywords that describe attributes they have or services they might provide. These agents contain links to, and information about, several other agents. Relationships and information contained in relationships provide a highly decentralized mechanism to discover other agents while remaining adaptive to a dynamic network environment. Discovery messages are forwarded along these relationships, attempting to reach agents that would satisfy a particular discovery query. In order to remain decentralized, agents in the proposed discovery mechanism improve discovery performance based only on localized information [16]. An agent may establish a relationship with another agent by actively searching for other agents nearby, through introduction via other agents, and through discovery-related interactions. The proposed discovery mechanism uses the relationship network in two phases: discovery query forwarding, discovery result backtracking. The discovery query forwarding process moves discovery queries through relationships, seeking agents that would satisfy the 18

28 discovery queries. Keyword similarity and relationship history are used at each agent to determine which relationships have priority in forwarding discovery queries [16]. The peer mechanism described in the paper is limited to small world concepts. The usefulness of the discovery mechanisms depends on the distribution of object data and distribution of object query types in the system. 3.2 An Incentive Message Relaying Mechanism This mechanism for peer discovery addresses the incentive problem. The protocols assume that peers will follow the design and contribute their bandwidth and resources to relaying messages as required. However, a P2P network is a highly decentralized system and each peer may present a different self-interested entity. A peer may manipulate the local information to take advantage of other peers resources [11, 17]. For example, a peer may simply drop a message that is sent from other peers for relaying, for the purpose of saving communication bandwidth and energy. Therefore a message relaying P2P system is vulnerable to the free riding problem. Since a sound P2P system relies on the contribution of resources from each individual peers, free riding can cause severe degradation of the system performance or even paralyze the system. It is important to design an incentive mechanism that motivates each peer to behave rationally in a system efficient way. [17, 13] 19

29 An incentive mechanism of message relaying for peer discovery overcomes the flooding problem of BFS search while reserving the quick response property and good reliability. In peer discovery there is no guidance about who the message should be sent to. The unknown destination implies more uncertainty and less control in the message relaying process for peer discovery. [17] The mechanism is as follows: The source peer sends the query to some neighbors and promises some payment to each receiver if the resource provider is found via a transmission route composed of the receiver. Depending on the offer each receiver decides the number of neighbors she relays the message to and also the promised payment to its immediate downstream peers. Each of the new receivers again makes the same decisions, until the maximum number of hops is reached. This mechanism does not price the relaying activities, but instead prices the relaying result, which influences the relaying activities. This mechanism is motivated by the communication efficiency, reliability, and information locality. An incentive message relying mechanism is not feasible for an anonymous message relaying process. The mechanism s efficiency is depended upon peers involvement and sharing of resources into the peer-to-peer networks. It can cause the free-riding problem very easily. Thus, a decentralized mechanism is limited to a small world. In a real P2P network it is hard to find relationships with all the peers in the network 20

30 secondary to the heterogeneity and decentralization, and would be very time consuming. An incentive message relaying assumes that peers in a network are willing to share resources, which is not the case in real peer-to-peer networks. This chapter elaborates current methodologies used for peer discovery. It discusses the disadvantages and advantages associated with decentralized and self-organizing, and instant message relaying mechanisms for peer discovery. To address these existing disadvantages, the thesis proposes a combination of these two methodologies. It uses an agent-based hierarchical model for peer discovery and message relaying for communication between agents. This proposed methodology is not limited to a small world; it can be used for any peer-to-peer network. 21

31 CHAPTER 4 Peer Discovery Protocol 4.1 Overview of the Model JXTA s peer discovery model is fully decentralized and works with query request and query response between peers. The peer who needs resources from other peers will send the query request to the JXTA network and the peers who have resources available will respond with a query response. Because of this arrangement the network can easily be overloaded with queries which can cause the flooding problem. In JXTA, peers are divided either as edge peers or rendezvous peers. The peers in JXTA discover resources by sending a request to another peer. The edge peers send request to rendezvous peers through advertisements and rendezvous peers forward request to other rendezvous peers. The request can overlap with different rendezvous peers which can degrade network efficiency. The typical routine for JXTA peer discovery protocol has only two messages: request messages and response messages. The request message is used for discovery of advertisements and the response message is used for responding to a discovery request. These messages are the Discovery Query Message and the Discovery Response Message. Because of JXTA s fully decentralized architecture, the peer who is receiving many query request messages might not be able to respond. This can happen for multiple reasons such as if a request did not generate the proper results or the request is denied 22

32 by the overloaded peers. The overloaded peers can decrease the efficiency of a network. Figure 3 The JXTA Peer Discovery Protocol To provide better efficiency for peer discovery of JXTA, this thesis proposes a semi-centralized, agent-based hierarchical tree schema for peer discovery of JXTA. The agent-based hierarchical tree schema provides better scalability than a fully decentralized schema for peer discovery and increased efficiency to locate the node, which contains the querying resource. The schema proposed by Moore and Suda [16] for peer discovery protocol is fully decentralized. The usefulness of the discovery mechanisms of Moore and Suda depends on the distribution of object data and the distribution of queries in the network. The schema is designed for only small networks [16]. The schema 23

33 proposed in this thesis is applicable to any size of peer-to-peer network. The fundamental proposal of this thesis is to group the edge peers based on their network proximity, which is based on their geographical location. The rendezvous peers are designated as agents. As in JXTA, rendezvous peers are considered as super peers and are much more sophisticated than the edge peers. The agent in this schema acts as the service providers at different levels of hierarchy. In this agent based hierarchical tree structure schema, the actual hosts are connected at level 0. In this model, edge peers are considered as regular peers and they are grouped together depending upon their network proximity. These edge peers are then connected to the agent at level 1. Each edge peer should be identified through their unique ID. The unique ID is achieved by using an IP address. There is also a maximum limit of edge peers that can join the agents for better efficiency and load balancing. The agents are again grouped together based on their network proximity and they are connected to higher level agents. The hierarchical agent based model provides better scalability and load balancing which is a vital part of JXTA network, as the numbers of peers to each agent levels are limited. This type of schema would provide better performance than a fully decentralized system. 4.2 Agent Based Framework In the previous section, we mentioned the agent-based hierarchical model for JXTA network in which the edge peers are grouped together based on 24

34 their network proximity. In this section, we will describe the construction of the agent based framework and also about the components required to build the model. Then we will discuss the grouping mechanism of the peers in which we will provide a grouping function to determine the appropriate agent for a host. The major problems that we will address in this section regarding construction of the model are: how to determine the number of agents at each level, how to determine the threshold of each agent, and how to determine the closeness between an agent and a peer based on response time. The goal is to provide scalability using this hierarchical model to maintain load balancing of agents and grouping of peers based on network closeness, which at the same time provide efficient query. The solutions of the above mentioned problems are discussed in sections and The components used to construct our framework are: Registration Server Agents at different hierarchical levels Registration Server (RS): The main component that we are using to construct the framework is a globally unique Registration Server. This Registration Server maintains a database of the IP address of all the agents at different levels of hierarchy. Each agent and host in the model knows about the IP address of this globally unique Registration Server. Initially, all the agents of every level send a register request message to the RS. The RS then saves all the information like IP address, level information, etc. about all the agents in its 25

35 database. The main function of this Registration Server is to provide the peer host with the IP address of all the agents in level one. Whenever an agent or host sends a query message to the RS, it sends back a response message to that agent or the host about the information of the agents at its upper level. To provide scalability and load balancing, we can maintain some hierarchical Registration Servers who contain the exact replica of the globally unique RS but remains inactive. That means that the other hierarchical Registration Servers also maintain a database regarding the information about all the agents. The only time replicated servers will become active is when the first RS is not able to provide service to the hosts as more and more hosts join the network at the same time. The globally unique RS forwards the query message to the next RS it is connected to when it is unable to provide service to the hosts as it may be busy serving other hosts. The next RS then can provide the service to the querying host. Agents: We are maintaining a large number of agents at each level, which are distributed around the world. The agents act as a service provider. Like the hosts, the agents also know about the IP address of the globally unique RS. All the agents at every level are registered with the Registration Server. Each agent at level one has a threshold or maximum limit of the number of hosts that can be grouped to it. Similarly, each agent at level two has a maximum limit of the number of agents level one can be grouped to it and so on. The functionality of all the agents is described in detail in section

36 As we mentioned before, we should be very careful to construct the model so as to provide better scalability of the model and load balancing of the agents and at the same time group the peers based on network proximity. For this, we need to determine the appropriate number of the agents at each level and carefully determine the threshold of each agent. The major issues that we are going to address in the subsections are: The determination of the number of agents and the threshold for each agent The discovery of a peer host s agent and the grouping mechanism Determination of the Number of Agents and the Threshold for Each Agent Initially, the question is how to determine the number of agents at each level and what is the threshold of each agent. The answer is that we will go with the number of hosts a particular Domain Name System (DNS) name server has. The DNS is a wonderful example of how a scalable distributed database can be implemented in the Internet. The main function of DNS is the hostname-to-ipaddress translation. Hence, the record of DNS name servers has the most current total number of hosts in today s Internet. As the number of hosts in the JXTA network will be much less than the number of DNS hosts, we will go with certain percentage of DNS hosts and compute accordingly the suitable number of agents with an appropriate threshold. Each agent also maintains a database about all the peer hosts that are grouped to it. 27

37 The number of agents and the threshold for each agent is calculated based on the following formula. Definition 1: Let, N [DNS] is the total number of hosts maintained in the different DNS Name Servers database. N [P] is the maximum number of hosts that can join the JXTA network. N [A] is the number of agents at level 1. N [G] is the number of peer hosts grouped under one agent, which is the threshold of each agent at level 1. Algorithm 1 Determine the number of agents and their threshold Input: N [DNS] Output: N [A], N [G] 1: N [P] = c % * N [DNS] where c is constant; 5 < c <10 2: N [A] = p % * N [P] = p % * c% * N [DNS] where c is constant 3: N [G] = N [P] / N [A]; N [A] < N [G] As the number of hosts in a P2P network will be much less than the number of DNS hosts, we will assume, the maximum number of peers that can join the network will be N[P] = c % * N[DNS], where c is a constant. As the number of DNS hosts increases, N [P] will automatically increase. N [A] = p % * N [P] = p % * c% * N [DNS]. As the number of agents should be much smaller compare to the number of P2P hosts, the value of p should be small. N [G] = N [P] / N [A], the threshold of each agent at level 1. N [A] < N [G] Using the above formula, we can determine the number of agents at level 1 and the threshold for each agent at level 1. 28

38 Approximate ratio between agents at different levels: The next step is to group the agents at level i under an agent at level (i + 1). We will approximate a ratio between the number of agents at level i and the number of agents at level (i + 1). The approximate ratio will be N [A (i + 1)]: N [A (i)] = 1: c, where c can be any number between 5 and 10. The reason for keeping c below 10 is that if N[G] is large, the total number of hosts under A (i + 1) would be very large because number of hosts under agent is A (i + 1) = c * N [G]. For example, let c = 7 and N [G] = 700, then A (i + 1) = 7 * A (i) = 7 * N [G] = 7000 for i = 1 which means the total number of hosts under an agent at level 2 is If the value of c is increased, A (i + 1) will be increased which will make the search space large. The above approach is supposed to provide the scalability and load balancing of the model. It will be scalable because it is based on the DNS number. The workloads are equally distributed among all the agents. In the future, if the number of DNS hosts increases, the hosts in the JXTA network will also increase. In that case, even if an agent reaches its threshold, we can increase the capacity of the agents. Another solution is that the agent can share its load with its subagents, which will be completely transparent to the host. The interface will be with the actual agent but it may share its load with its subagents. Hence, the model always remains scalable as more hosts join the network. 29

39 4.2.2 Discovery of a Peer Host s Agent and the Grouping Mechanism Currently in JXTA, a unique ID generated by the users with each peer in the group is attached. However this does not solve the problem because it does not guarantee uniqueness across the entire community of millions of users. Certain P2P networks provide a grouping or partition mechanism of location independent peers. In such kind of partition, the peers are grouped together based on some kind of common properties they have, for example, the peers that have files on the same topic. The main assumption in this kind of partition is that every peer knows about every other peer. But, in the real world, where the peers might be located in geographically distant places, it is impossible for a peer to know about other peers that are connected to the network. Hence, in the global world, this kind of partition is not feasible. The grouping of the peers based on their network proximity is practical in real P2P networks. This closeness-based grouping mechanism increases the scalability of the system and provides load balancing and decreases the total number of messages in the network. Network information can be used to improve the topology of the network, that is, we can classify the results based on network closeness of two peer hosts. We can provide load balancing of the agents depending on closeness. For example, agents closest to the host could provide fastest access and consequently more hosts can be served. Also, grouping mechanism provides an efficient query mechanism as the peers do not need to have knowledge about other peers a priori. 30

40 In our hierarchical model, the peers that are located close to each other are grouped together; the partition is location dependent. The actual peer hosts in the P2P network are the leaf nodes in the hierarchical tree structure. The major problem would be to discover an agent among several agents who is located close to the peer host in the network. The discovery of a peer host s agent to which the peer should be grouped to, is based on the algorithm and the grouping function discussed below. Before we define the grouping function, the notations used for the definitions will be described in Table 1. 31

41 Table 1 Notation for grouping function Name Description RS Ah[i] H[j] ST[k] T[i] The globally unique Registration Server The total number of agents at level h, where h = 1, 2, 3..and i = 1 n j is the host willing to join the P2P network. The sample times between a host and an agent computed by ping messages at time t0, t1, t2,. and k = 1..n The average of all the sample times for any i T The time recorded between a host and an min agent which is minimum among the times recorded between the host and all other agents. D The distance between a host and an agent min this is the minimum among the distance between the host and all other agents. 32

42 The information at the Registration Server (RS): Ah[i] where {i = 1 n}: The IP addresses of all the agents at level h, h = 1, 2, 3... Querying peer ID: The IP address of the peer host that sent a query message to the RS to get back a response The Grouping Function In this section, we will give an algorithm for construction of the agentbased framework and this algorithm defines a grouping function to perform the grouping mechanism for peers. Definition 2: The register message is denoted by register [Ah[i] for every i; RS] which means that all the agents at level h sends a register message to the RS. The query message is denoted by query [H[j] for any j: RS] where the first parameter is the source and the second parameter is the destination. This query message means that the host H[j], for any j, sends a query message to the RS. The response message is denoted by response [RS; H[j] for any j] where the first parameter is the source and the second parameter is the destination. This query message means that the RS sends a response message to any particular host. 33

43 The Ping message is denoted by Ping[ H[j] for any j ; A1[i] for every i ] where the first parameter is the source and the second parameter is the destination. The join message is denoted by join[ H[j] ; A1[i] with T min ] which means that any host H[j] sends a join message to the Agent A1[i] for any particular i to include him into the group under that agent. Algorithm 2 Construction of Agent based Framework Input: RS, Ah[i], H[j] Output: join [H[j]; A1 [i]] 1: register [A1[i] for every i ; RS ]; 2: query [H[j] for any j; RS]; 3: response [RS; H[j] for any j ]; 4: Ping [H[j] for any j; A1 [i] for every i]; 5: ST[k] Record time [H[j] for any j; A1 [i] for every i ] for k = 1,2.. N 6: Compute T[i] = Σ ST[k] / n for every k = 1 7: Compute T min = min [T[i] for every i]; 8: join [H[j]; A1 [i] with T min ]; Initially, all the agents of every level send a register message to the Registration Server. The RS maintains a database about all of the agents. When a host is willing to join the JXTA network, a query message is sent to the RS asking information about all the agents of level 1. The RS then looks up the 34

44 database and sends a response message to the querying host containing a list of IP addresses of all the agents at level 1. The host then pings all the agents and records the time needed for the ping message to go to every agent. The host can ping the agents several times and each time it records a sample time. The actual time is determined by averaging all the sample times. It then computes the minimum time among all the times recorded and locates the agent that received the ping message in the shortest time. Finally, the host sends a group message to that particular agent requesting to join the group under that agent. The agent who received the ping in the shortest time is located closest to the host in the network. The host then caches the IP address of that agent and the ping time in its database which will be further discussed later. Thus, the first level of the hierarchical model is built with the closely located hosts grouped together. In a similar way, the other levels of the hierarchical model can be built where an agent at level 1 sends a ping message to all the agents at level 2 and subsequentially grouped to an agent at level 2 following the algorithm. This process continues up to the higher levels of the hierarchy. Reducing the overhead: The above mechanism works well if there are a fewer number of agents at level 1. But, if there are a large numbers of agents, then the grouping mechanism produces a huge overhead as every host who wants to join the network has to ping all the agents at level 1. This overhead can be significantly reduced by following the mechanism described below. 35

45 Whenever a host gets a list of IP addresses from the Registration Server, it randomly selects 10 to 15 agents out of the whole list to which it can send ping messages to. The host then sends ping messages to all those 15 agents. The host can set a parameter, say, k seconds to determine which agents should be considered for grouping. The agents whose ping times are more than k seconds are not considered. If no such agents existence whose ping times are less than k seconds, the host then randomly selects another 20 agents and ping them. The host considers only those agents whose ping times is less than k seconds for further pinging. The host then pings all the agents different times with different size of messages. The idea is to discard those agents having ping time more than k seconds at first place because they may be connected by links having high traffic, low bandwidth, or may be located far from the host in the network and consider only the closely located agents based on response time. Based on the above algorithm, we can derive a lemma for the grouping function. Lemma 1: The Grouping Function The minimum time recorded between a host an agent implies the closeness between them based on response time and the agent having the minimum ping time is the most appropriate agent to be grouped to. Symbolically, T min [H[j]; A1 [i]] A1[i] is appropriate for H[j] 36

46 Proof: We know that it is very obvious messages can travel between two hosts faster whenever they are closely located. So, it follows that the shortest time recorded between a host and an agent means closeness between the host and an agent based on response time. The problem arises when there is congestion in the network. It may happen that an agent who is closely located to a host in the network might be connected by the link, which has suddenly become congested. This is the reason for taking the average of multiple ping times to find the actual response time. We can claim that the agent having the shortest ping time is the appropriate agent for a host to be grouped to. Let us consider three parameters to prove the claim. They are Message size, Bandwidth Available, and Network Traffic. For Fixed Message size = k bytes, (k ) bytes, (k ) bytes For i = 1, 2, max Ping_time [H[j] ; A1[i] ] depends on------> [ location of agent in the network, Bandwidth Available, Network traffic] Find Avg_Ping_time i and Message size Min_Avg_Ping_time [closely located in the network, High bandwidth available low Network traffic] The host pings the agents with different message sizes and takes an average of all the ping times. The agent having the minimum average ping time is considered as the appropriate agent for that host as the ping time may 37

47 depend on the location of agent in the network, the bandwidth and traffic of the link by which that agent is connected. The agent having the minimum ping time means it is connected by the link having high bandwidth, low traffic, and also located close so that message can be routed faster to it. Hence, we can safely assume that the minimum time between a host and an agent means that agent is appropriate for the host to be grouped to The Caching Mechanism of the Peer The grouping mechanism discussed in the prior section is time consuming and could cause an extra overhead every time a peer tries to join the network, as it has to go through all of the ping messages again. To overcome this problem, we propose another important mechanism, which is the caching mechanism used by all the hosts in the JXTA network. Initially, whenever a peer host intends to join the network, it pings all the agents of level 1 to discover its closest agent. It then computes the shortest time that a ping message took to reach an agent. Then it sends a group message to that agent and caches the IP address of that agent with the recorded time in its database. At the same time, the peer host will also cache the IP addresses and times of some agents having the next shortest times. The main advantage of this mechanism is that if a host leaves the network and tries to join again, it doesn t have to go through all the query and ping messages. It can directly ping the few agents whose IP addresses are cached to its database and accordingly select the agent based on minimum time ping results. This kind of caching 38

48 mechanism subsequently reduces the search space of the peer host to discover its agent when it intends to join the network very often. It also reduces the time and the number of messages required to discover an agent for a peer. When a host wants to leave the network, it simply sends a quit message to its agent and an agent removes that host from its database Evaluation of the Framework The major contribution of this thesis is to provide a scalable model for the JXTA network and maintaining at the same time load balancing of the nodes and closeness feature of the grouping mechanism. The workload is equally distributed among all the agents. As more and more hosts are joining the network, the capability of the agent to serve them is note limited. This is because the proposed mechanism uses data from the DNS and there should be enough room in each of the agent (as mentioned in section 4.2.1), unlike Napster. The model is very dynamic in nature as once the framework is constructed, any host can join and leave the network dynamically without incurring any overhead or congestion in the network because of the reduction in the number of messages. The grouping based on network closeness also provides load balancing of the agents to some extent. As the hosts are closely located to the agent, the agent could provide faster service and consequently more hosts can be served. Hence, an agent always has the capability to serve as more and more hosts join its group. This grouping mechanism based on ping also allows a peer to have 39

49 the flexibility or choice to join a particular group of interest in a different place. The construction of the framework can be applied to any kind of peer-to-peer construction based on topic-based or context-based grouping. Easy maintenance and less complexity make the model even more unique. Hence, we can claim that the network closeness grouping and load balancing is maintained, as the model becomes more and more scalable. Despite of certain advantages, the construction of the model has some limitations. The advantage of scalability is achieved after the model is built. The peers are grouped together under an agent. But, the construction of the model itself has certain limitations. There is a cost factor involved with the maintenance of agents throughout the world. The self-organization property of P2P network is violated; hence the model is more like a hybrid P2P model rather than a pure P2P model. The complexity depends on the number of agents at the lower level since each peer needs to ping all the agents. The complexity for the construction of the model is O(m) where m is the number of agents whereas the worst case complexity for the communication between two peers after the model is constructed is O(log n), where n is the number of peer nodes, which is discussed in later sections. Hence, to achieve scalability and better network performance, these tradeoffs needs to be considered and a decision should be taken accordingly. 40

50 4.3 The Agents as Service Providers The term agent is used to refer to an active component, which has autonomous, communication, and proactive features. In the model we use the rendezvous peer as an agent, as the rendezvous peer is considered a powerful peer for JXTA network. Multiple agents are used in these hierarchical models that perform different functionality. As mentioned earlier, the agents here are the service providers at different levels of hierarchy. Each agent focuses on its own specialized tasks, manages its own resources, and makes specific decisions according to the situation [16]. Agents participate in many important tasks like sending requests, responding to messages on behalf of their group members, retrieving resources from repositories, doing calculations, and making decisions according to the available information. The agent maintain its resources by maintaining a database which keeps record of the number of agents or hosts connected to it as well as about its parent. Here, the agents perform the task of resource management. Initially, a peer host sends a query message to the agent at level 1, which is one level up in the hierarchy. Whenever the agent receives a query message, it looks up the database and finds the ID s of different hosts connected to it. The agent here performs the task of retrieval of resources. It then broadcasts the query message downwards to all the peers connected to it. If any host has the queried file, it sends a response message to the agent at level 1, which in turn forwards the message to the querying host. If the file is not found, the agent at level 1 41

51 sends the message to agent level 2, which is one level up in the hierarchy and the process continues until the file is found. Hence, one of the main task of different service providers or agents here is to forward the query and response messages. The communication between agents for the query message is of two types. One of the functions is to send a query message to the agents one level up in the hierarchy. Another one is to broadcast the query message to the agents or hosts one level down in the hierarchy that they are connected to it. Similarly, the communication between the agents for the response message is of two types. The responding agent, that is, the agent who found the queried file in one of its hosts, sends the response message back to the agent that queried for it one level up in the hierarchy. In another communication, the agent forwards the response message to the agent or host situated one level down in the hierarchy. Another important task of the agent is decision-making. Whenever multiple hosts have the queried file, the agent at level 1 needs to calculate the fastest link using some kind of mechanism like round trip time (RTT), and make a decision based on that information. After the fastest link is determined, the agents decide to retrieve the file from that agent or host that is connected with the fastest link. Table 2 describes the agents of the hierarchical model in terms of their type and role. 42

52 Table 2 The Hierarchical Model Agents Agent at Level 1 Agent Type Agent at Level i for i = 2,3,4,.. Role Maintains a database for the number of hosts connected to it and its parent, updates the database when a host is added or deleted, broadcasts the query message to all the hosts connected, forwards the query message to agent level 2 if file is not found, sends the response message to the appropriate agent, calculates the fastest link for the hosts if multiple hosts have the queried file, makes decisions about which host to retrieve a queried file from. Maintains a database for the number of lower agents connected to it and its parent, updates the database when an agent is added or deleted, broadcasts the query message to all the lower agents connected, forwards the query message to upward agent if file is not found, sends the response message if file is found. 43

53 The communication between the agents for both the query and response message is of two types. down_level } Algorithm 3 Communication between the Agents Input: send query message Output: query message received by agents or hosts if (query message) } { send_query_message to the agent one Up_level if (message_received) { Up_level agent broadcast_query_message to agents one Input: send response message Output: response message received by agents or hosts if (response message) { send_response_message to the agent one Up_level if (message_received) { Up_level agent forward_response_message to agent one down_level } } 44

54 4.4 The Update Mechanism (Join/Leave) In the hierarchical model, all the agents can have a limited number of agents or hosts that it can be connected to as discussed in section This type of arrangement provides load balancing of the agents and scalability of the system. There is a maximum limit set a priori about the number of agents or hosts for all the agents. The update mechanism for our model is suggested from the token ring topology [37]. Whenever a new peer wants to join the system for the first time, in the hierarchical model, it will first send a request message to the Registration Server asking for the IP addresses of all the agents at level 1. The Registration Server sends a response message to the peer host with a list of all the IP addresses of the agents at level 1. Then the new host sends a ping message to all the agents at level 1 and based on the shortest time of the ping message, it discovers its agent. Actually, the peer computes sample ping times to all the agents and then takes the average to find the shortest ping time of an agent to which it can be grouped to. It also caches the IP addresses of first few agents with the minimum ping times. After a peer finds an agent to whom it can connect to, it joins under that particular agent. The agent then needs to update the database about its newly joined node and keep the machine ID of the new node in its repository. If a peer wants to rejoin the system, it does not have to go through all these steps. Instead it can ping the agents whose IP addresses are in its cache 45

55 and again based on minimum ping time, it can join under that agent. The joining of a peer host, which is the grouping mechanism of the peers, is described in detail in section The leaving of the peer host is quite simple compared to the addition process. Whenever a peer wants to quit the system, it will send a quit message to its agent. The agent then breaks all connection with that host and updates its database by deleting all of the information about that host. The decision for adding a new host to the network is based on the following algorithm. Algorithm 4 Peer join/leave mechanism Input: Host, Agents at level 1 Output: Join the group for every peer host { send_query_message (host [source], RS [destination] ) send_response_message ( RS[source], host [destination] ) send_ping_message ( host [source], Agent_level_1(i) where i = 1,2,3,.[destination] ) compute sample ping times(host [source], Agent_level_1(i) where i = 1,2,3,.[destination]) compute_average_time ( host [source], Agent_level_1(i) where i = 1,2,3,.[destination] ) 46

56 compute min_average_time send_join_message ( host[source], Agent_level_1 with min_average_time [destination] ) cache the IP addresses of few agents with the minimum ping times } Input: Host, Agent, quit message Output: leave the group Host sends a quit message to its agent The agent breaks the connection with the host The agent updates the database by deleting the record of that host from its database. 4.5 Message Types in the Hierarchical Model To handle all the functionality of the model, we have defined seven basic message types. Table 3 contains a summary. 47

57 Table 3 The Seven Basic Message Types Type Code Message Type Description 1 Register 2 Query 3 Broadcast 4 Response The agents sends a register message to the Registration Server Two types: 1. A peer host sends a query message to the Registration Server 2. A peer host or an agent sends a message to search for a peer An agent sends the message to all the downward agents or hosts connected to it. Two types: 1. The Registration Server sends a response message to the host with a list of IP addresses of the agents at level 1 2. A host or an agent sends a message in response to a query message. 5 Ping The peer host sends a ping message to all the agents at level 1. 6 Join/Quit A peer host sends a message whenever it wants to join the group or quit the system. 7 KeepAlive Actively test peer connectivity Each message is divided into header and data areas. Each header has a type field that identifies the content of the message. 48

58 4.6 Failure Handling and Recovery The hierarchical model provides an extensive failure handing and recovery mechanism. The failure handling and recovery mechanism for the model is suggested from token ring topology [37]. There can be a situation when any agent or link might fail. In that case when a peer host sends a query message to its agent and it fails, there is no way that message can be delivered into the network and the query is never served. So, when any agent crashes, the whole network could be affected. However, due to simplicity of the tree structure, back up procedures can be established very easily. In this section, we discuss how agents handle detected failure and recover when the failure is healed. The proposed solution is to create some back up links in the model. Each peer host is connected to their own agent as well as to another nearest agent at level 1. Similarly, each agent at level 1 has also connection to another agent at level 2 other than their own agent at level 2. In a similar way, all the agents in the hierarchical model are connected to an agent higher in the hierarchy as well as to an agent located nearest to that higher agent in the hierarchy. Now the question may arise, how to determine the next nearest upper agent. As we have already mentioned in previous sections that whenever a peer host pings all the agents for the first time, it also caches the IP addresses of first few agents which have the shortest ping times. So back up links can be easily created with the agent which have the second shortest ping time response. 49

59 When a host or an agent sends a query message to another agent, but does not get any response after a certain interval of time; it can detect failure in the link or failure of the agent as discussed below. Then it forwards the same message to its nearest upward agent to which it is connected (by using the second shortest ping time). This type of back up links can be used to continue sending messages in the network even when some of the agents might crash. The failure detection can be done using any kind of failure detection techniques like using KeepAlive messages. The idea is that, two agents or a host and an agent periodically exchange KeepAlive messages to test network connectivity and to verify that both parties continue to function. A KeepAlive message consists of the standard message header with no additional data. As mentioned in section 4.5, whenever any party sends a KeepAlive message, the header of that message contains a value 7 in the TYPE field. Periodic KeepAlive messages are needed because some transport protocols like TCP do not include a mechanism to continually test whether a connection endpoint is reachable. Also, KeepAlive s conserve bandwidth compared to other messages. Hence, whenever an agent or a link fails, it stops sending KeepAlive messages and the other party can detect the failure of the agent. The functioning agent then sends the message using the back up links as mentioned above and the search process continues. So, failure of any links or agents does not affect the network. 50

60 When the failure is healed, that is, when the agents or the links start functioning again, it can recover by start exchanging KeepAlive messages to let the other party know that it is alive. The other party then can continue sending messages over that link. Hence, our hierarchical model handles failure very effectively and provides recovery after the failure has occurred. 4.7 Performance of the Hierarchical Model In this section we describe the characteristics of the proposed technique in comparison with the Gnutella protocol. We first consider the performance of the hierarchical model and compare it with the Gnutella protocol. In the hierarchical model with tree topology, the grouping mechanism based on peers network location searches the nodes of the graph more efficiently that is, sends fewer messages than the Gnutella protocol. Consider a random graph G with n nodes and e edges, which has average degree d which means each node can have at average d nodes connected to it. For a given node v, let N k (v) be the set of nodes at distance at most k from v. When a node v starts a Gnutella search with a TTL = k (time to live as per the Gnutella search protocol), v sends approximately d messages to its neighbors, each being propagated k times. Since, the search mechanism explores all the edges of the graph, the number of messages send by the Gnutella protocol is at least N k (v) * ( N k (v) /n) * d [10]. On the other hand, in the hierarchical model because of the grouping mechanism, the number of messages in the network is much less than the Gnutella search, which is proved in the following lemma. 51

61 Lemma 2: The number of messages M in the hierarchical model is M = d(d k 1)/(d 1) where k is the height of the tree and d is the average degree of each node. Proof: The search approach is bottom up. Whenever a querying host sends a query message to the agent one level up in the hierarchy, the agent broadcast the messages to all the d nodes, since the average degree of each node is d. The search process continues and the agent sends the message again to another agent up in the hierarchy which in turn broadcast the message to all the d nodes and each d node sends the message again to all the d nodes connected to it. Hence, the total number of messages in the network will be M = d + d 2 + d 3 + +d k where k is the distance between the root node and the leaf node (height of the tree). This is a geometric progression (G.P) series and the sum will be M = d(d k 1)/(d 1) For example, consider a network with total 85 nodes with k =2 and d =3 which has 60 leaf nodes. In Gnutella search, the number of messages in the network will be N k (v) * ( N k (v) /n) * d = 60 * 60/85 * 3 = 127 On the other hand, the number of messages in the hierarchical model will be d (d k 2 1)/(d 1) = 3*( 3 1)/(3 1) = 12 52

62 Hence, the JXTA hierarchical model uses significantly fewer messages compared to the standard Gnutella strategy, and scales better with respect to the size of the network because it can search a larger network using the same number of messages. Performance in terms of latency: The model is also scalable in terms of latency. Each request from an agent can pass one or more agents in order to find a target agent that can provide the required service. The performance of the discovery process is mainly based on the number of routing connections, that is, number of messages sent in the network. Fewer connections have a faster discovery process and higher system performance. In the whole system, there may be simultaneous service requests. The average service discovery speed v can be defined as v = r/d Where r is the total number of requests during a certain period and d is the total number of connections made for the discovery [34]. As we can see that the model has fewer numbers of connections and fewer numbers of messages compared to Gnutella to discover or locate a host, and discovery speed is inversely proportional to the number of messages, so the discovery speed of the proposed model is much higher. Hence, the model is also scalable in terms of latency. 53

63 The model is scalable in terms of latency also because of the tree topology that we are using. The maximum distance between two nodes only depends on a logarithmic function of n where n is the total number of nodes. Thus, the topology can scale up to large number of n with a limited latency penalty [11]. Lemma 3: The maximum distance between two leaf nodes in a tree depends on the logarithmic function of n where n is the total number of nodes in the tree. Proof: We assume a balanced tree and that each node can have d child nodes. The maximum number of node n of a balanced tree with height k is: n = d 0 + d 1 + d 2 + d d k d * n = d 1 + d 2 + d 3 + d d k + 1 d * n = d ( d k + 1 1) / (d 1) n ( d 1) = d k d k + 1 = n ( d 1) + 1 k + 1 = log d [n ( d 1 ) + 1] k = log d [n ( d 1 ) + 1] 1 Let, the maximum distance between two leaf nodes be h, which requires traversing the root node: h = 2k = 2 log d [n ( d 1 ) + 1] 2 Approximating n ( d 1) + 1 by nd, we have, h 2 log d (nd) 2 = 2 log d n + 2 log d d 2 = 2 log d n Hence, tree topologies are scaleable in means of latency. The maximum distance between two leaf nodes depends only on a logarithmic function of n. 54

64 Thus, the topology can scale up to large numbers of n with a limited latency penalty [11]. Overall complexity: As we have already discussed, the complexity for construction of the framework is O (m) where m is the number of agents at level 1, since a host needs to ping all the agents at level 1 when it wants to join the network first time. After the model is constructed, the complexity for the search mechanism will be O (log n) where n is the total number of nodes in the network as discussed in this section which is the same as Chord [10]. Hence, the overall complexity for the hierarchical model is O (m) + O (log n). 55

65 CHAPTER 5 Future Work The hierarchical agent-based model for peer discovery provides better scalability than a fully decentralized model of JXTA. However, there are many other areas in which this model can be further improved, for example peer grouping, which is very inefficient. One possible solution involves the introduction of common interest peers into a single cluster. Another area for improvement is to optimize the threshold for each agent and peers based on actual DNS data. A third area of improvement is to provide intelligence to the registration server, which currently operates on a very simple philosophy. By enabling the registration server to make decisions, it can select the appropriate agents for a particular host among all the agents available and send a list of agent IP address accordingly. Lastly, we can also introduce searching and resource retrieval mechanisms for the proposed model. The current model is designed only for peer discovery. Obviously, further development needs to be undertaken in these various areas to optimize JXTA as a P2P network. 56

66 CHAPTER 6 Conclusion Peer-to-peer (P2P) networking has recently emerged as a new paradigm for resource sharing, for building distributed network, and for grid technology. Peer-to-peer networks offer the significant advantages of decentralized, faulttolerant, and load-balancing systems, with simplicity and ease of use. The increase in need to share resources and information in a large-scale, distributed environment; the widespread use of networks and software technologies; and decreasing cost of powerful workstations have increased the usage of peer-topeer networks in real-time communication. Despite the significant potential of the peer-to-peer network, they have certain limitations. One of the major problems of the peer-to-peer network is scalability as more and more hosts join the network. Current widely distributed peer-to-peer networks like Gnutella have scalability and efficient search problems. Peer-to-peer also lacks efficient grouping mechanisms, as it is assumed that each peer knows about every other peer. This is not possible in real world, as with the enormous growth of the Internet, there might be millions of hosts connected to the peer-to-peer network. Hence, scalability is one of the major issues of existing peer-to-peer networks. JXTA is a relative newcomer in peer-to-peer technologies. JXTA s peer discovery protocol used a fully decentralized model and works with query request and query response. 57

67 The thesis investigates current peer-to-peer systems and specifically JXTA, and proposes an agent-based hierarchical model to overcome some of the existing problems of the JXTA network. The thesis proposes a dynamic grouping mechanism of the peers based on their network closeness. The model consists of a semi-centralized hierarchical tree structure with actual peers being the leaves of the tree, and the all the other nodes acting as agents at different levels of the hierarchy that can send request and response messages. Peers are grouped together based on their network closeness. Each peer group is connected to an agent and the communication between the group members is done via the agent. It solves the scalability problem as the numbers of agents are calculated based on the data from the DNS. Also, there is a threshold on the number of group members in a particular group hence providing load balancing of each agent. The search process is efficient as it starts locally in its region and slowly spreads out searching the local nodes, next closest nodes, and continues all the way up to the root. In the best case, the search file could be found locally as the message is broadcast within the group. This decreases the number of messages and latency in the network. This type of model can have significantly less messages in the network than the Gnutella network. Additionally, caching and timeout mechanisms are suggested which can further improve the efficiency of the system. The only disadvantage in our model is the overhead incurred while constructing the framework to discover a peer host s agent. Whenever a new 58

68 host wants to join the network, it sends a query message to the Registration Server to get the IP addresses of all the agents at level 1. The RS has the information about all the agents and it sends back a response to the host with a list of IP addresses of the agents at level 1. The host then pings all the agents to determine the closest agent based on minimum ping time. This is the overhead of the construction of the framework as the complexity is O (number of agents). However, this overhead is highly reduced when a node rejoins the network as all the information about the agent is cached. The semi-centralized agent based hierarchical model would provide better scalability than a fully decentralized model for peer discovery. It also provides efficient grouping mechanism which will also contribute to improve the efficiency. 59

69 BIBLIOGRAPHY [1] JXTA First Edition. Brendon J. Wilson, June [2] Mastering JXTA: Building Java Peer-to-Peer Applications. Joseph D. Gradecki. [3] J. Verbeke, N. Nadgir, G. Ruetsch, et al., "Framework for Peer-to-Peer Distributed Computing in a Heterogeneous, Decentralized Environment," [4] "Project JXTA Community Home Page," [5] JXTA Technology: Creating Connected Communities, Sun Microsystems, Jan [6] Project JXTA: An Open, Innovative Collaboration, Sun Microsystems, April 25, [7] Emir Halepovic, Ralph Deters, The Costs of Using JXTA, Department of Computer Science, University of Saskatchewan [8] Buyya, Rajkumar, Steve Chapin, and David DiNucci, Architectural Models for Resource Management in the Grid. In First IEEE/ACM International Workshop on Grid Computing (GRID 2000), Bangalore, India, December [9] Clarke, Ian, Oskar Sandberg, Brandon Wiley, and Theodor W. Hong., "Freenet: A distributed anonymous information storage and retrieval system". In Proceedings of the Workshop on Design Issues in Anonymity and Unobservability, July 2000, pp

70 [10] Kalogeraki, Vana, Dimitrios Gunopulos, and D. Zeinalipour-Yiatzi," A local search mechanism for peer-to-peer networks". In Proceedings of the Eleventh International Conference on Information and Knowledge Management, 2002, pp [11] Junginger Markus, and Yugyung Lee, "A high performance messaging system for peer-to-peer computing". In Proceedings of IEEE P2P Conference 2002, pp [12] Lee, Yugyung, Changgyu Oh, and E. K. Park, "Intelligent knowledge discovery in peer-to-peer file sharing". In Proceedings of the Eleventh International Conference on Information and Knowledge Management, 2002, pp [13] Napster. Available from [14] Ratnasamy, Sylvia, Paul Francis, Mark Handley, Richard Karp, and Scott Shenker, "A scalable content-addressable network". In proceedings of the 2001 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications August 2001 [15] Stoica, Ion, Robert Morris, David Karger, M. Frans Kaashoek, and Hari Balakrishnan, "CHORD: A scalable peer-to-peer lookup service for Internet applications". Technical report TR-819, MIT, March 2001 [16] Michael Moore and Tatsuya Suda, "A Decentralized and Self-Organizing Discovery Mechanism. Department of Information and Computer Science, University of California, Irvine 61

71 [17] Cuihong Li, Bin Yu, and Katia Sycara, "An incentive mechanism for message relaying in peer-to-peer discovery". In the proceedings of second workshop on the economics of peer-to-peer systems, 2004 [18] M. Kelaskar, V. Matossian, P. Mehra, D. Paul, and M. Parashar, "A Study of Discovery Mechanisms for Peer-to-Peer Applications". Department of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ [19] Carlo Mastroianni, Domenico Talia, and Oreste Verta, ""A Super-Peer Model for Building Resource Discovery Services in Grids: Design and Simulation Analysis". In the proceeding of EGC 2005, LNCS 3470, pp , 2005 [20] Cameron Ross Dunne, "Using Mobile Agents for Network Resource Discovery in Peer-to-Peer Networks. School of Computer Applications, Dublin City University, Dublin 9, Ireland [21] Luciana dos Santos Lima, Antônio Tadeu Azevedo Gomes, Artur Ziviani, Markus Endler, Luiz Fernando Gomes Soares, and Bruno Schulze, "Peerto-Peer Resource Discovery in Mobile Grids". Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio), Laboratório Nacional de Computação Científica (LNCC), August, 2005 [22] Zhanping Yin and Victor C.M. Leung, "Third-Party Handshake Protocol for Efficient Peer Discovery and Route Optimization in IEEE WPANs". In the proceeding of IEEE BroadNets 2005, Boston, MA, Oct

72 [23] Michael J. Freedman and Robert Morris, "Tarzan: A Peer-to-Peer Anonymizing Network Layer". In the proceeding of CCS 02, November 2002 [24] Jan Sacha, Jim Dowling, Raymond Cunningham and René Meier, "Using Aggregation for Adaptive Super-Peer Discovery on the Gradient Topology". In the proceeding of Digital Business Ecosystem Project IST [25] Adeep S. Cheema, Moosa Muhammad, and Indranil Gupta, "Peer-to-peer Discovery of Computational Resources for Grid Applications". In the proceeding of NSF CAREER, June 3, 2005 [26] Andraž Jere, Marko Meža, Boštjan Marušič, Štefan Dobravec, Tomaž Finkšt, and Jurij F. Tasič, "Peer to Peer Search Engine and Collaboration Platform Based on JXTA Protocol". In the proceeding of EUROCON 2003 [27] Massimo Paolucci, Katia Sycara, Takuya Nishimura, and Naveen Srinivasan, "Using DAML-S for P2P Discovery". Carnegie Mellon University, USA [28] Michael Moore and Tatsuya Suda, "Adaptable Peer-to-Peer Discovery of Objects that Match Multiple Keywords". In the proceeding of the 2004 International Symposium on Applications and the Internet Workshops, 2004 [29] Tie-Yan Li, Zhi-Gang Zhao, and Si-Zhen You, "A-peer: An Agent Platform Integrating Peer-to-Peer Network". In the proceeding of the Proceedings of 63

73 the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003 [30] Jan Sacha, Jim Dowling, Raymond Cunningham, and René Meier, "Discovery of Stable Peers in a Self-Organizing Peer-to-Peer Gradient Topology". In the proceeding of International Federation for Information Processing [31] Naveen Srinivasan and Timothy Finin, "Enabling Peer-to-Peer SDP". University of Maryland Baltimore County, Baltimore MD [32] P.A. Felber, E.W. Biersack, L. Garcés-Erice, K.W. Ross, and G. Urvoy-Keller, "Data Indexing and Querying in DHT Peer-to-Peer Networks". Institute EURECOM, Sophia Antipolis, France [33] Mujtaba Khambatti, Kyung Ryu, and Partha Dasgupta, "Peer-to-Peer Communities: Formation and Discovery". Research paper granted by AFSOR, DARPA, Microsoft and NSF [34] Yatin Chawathe, Sylvia Ratnasamy, Lee Breslau, Nick Lanham and Scott Shenker, "Making Gnutella-like P2P Systems Scalable". In the proceeding of SIGCOMM 2003 Conference, August 2003 [35] Piet Van Mieghem, "Dividing a Network into Peer Groups to Build a Hierarchical Structure". Alcatel Corporate Research [36] Gnutella. Available from [37] Token Ring. Available from 64

74 APPENDIX Discovery Mechanism Parameters 65