Structured P2P. Complexity O(log N)

Transcription

1 Structured P2P. Complexity O(log N) Student: Santiago Peña Luque Communication Technologies 1 Year : 2005

2 INDEX 1. Introduction to P2P systems 2. Complexity 3. Structured Systems:DHT 4. Specific DHT algorithms 5. Problems 6. Conclusion 7. Sources

3 1.INTRODUCTION TO P2P SYSTEMS Definition : In general, a computer network p2p (peer to peer) is a Net without fixed clients and servers, and instead has a group of nodes that play both roles : client and server. This net model is different from the client-server model. Any node can initiate or finish a transaction. The nodes can have different local configuration, transfer bit-rate and storage capacity. The majority of computers have not a fixed IP, therefore they can not connect because they do not know the address before. The normal solution is to connect to a server with a fixed address, and this server has to update its IP addresses of all the other clients, server of the net, and normally also an index with the information that offer the clients. After the clients have all the necessary information about the net and they can exchange data, without any server more. Characteristics of P2P Systems - No central coordination. In the P2P nets there is no server. All the nodes take the role of server and clients at the same time. But there are some exceptions. The most primitives P2P nets used a server (Centralized P2P) to control the addresses of the rest of the nodes.the most popular were Napster. The problem of using a server is the existence of one responsible of the net, and it will be punished by law if there is an illegal distribution of data. - No central data base. It is supposed to distribute all de data information between all the nodes. - No peer has a global view of the system. Each node knows only where are some neighbours or other far nodes, but does not have a complete list of the node s addresses. - Global behaviour emerges from local iteractions. Due to the no existence of global view of the system, the net actions are composed by many local interactions. - All existing data and services are accessible from any peer. Normally there is no hierarchiecal structure. Therefore all the node have the same rights and the data are accessible for all of them. - Peers and connections are unreliable. Most of the nodes are home users in a data interchange p2p net, and these connections can fail. A p2p net can not trust completely in any node, and must be prepared for it. - Self-organizing evolution of the system (growth etc.) Each node has to be programmed to execute the protocol correctly. The global behaviour is made of small steps.

4 Categorization of P2P systems Architecture Centralized (Napster, centralized index) Decentralized (Gnutella, Freenet, etc.) Hierarchical (Farsite?) Searching for Data (decentralized architecture) Unstructured (Gnutella, flooding) Structured (Pastry, n-ary search trees) Storing of Data Fix (e.g. hashing) Adaptive (Freenet) 2.COMPLEXITY IN P2P SYSTEMS All the p2p systems must affront the following problems: Location of data in the system. In a P2P net the nodes offer their data and in the oder side they look for other data. These data should be located in any of the nodes of the system, but there are several ways to organize it. Depending in the selected method of location of data, the searching overhead and storage overhead will be different. Where will be stored the data items How does request find storage location To keep overhead small: search and storage of Data. In the actual P2P nets there is a very inefficient searching system due to the unstructured Systems. To reduce these overheads is a task of the actual p2p investigations. Robustness against faults and modifications. The p2p systems are based on unreliable nodes and connections. Therefore the structure of the net should take this in to account. Another important factor is the dynamicity of the nodes. Big O notation In complexity theory, computer science, and mathematics, the Big O notation is a mathematical notation used to describe the asymptotic behavior of functions. More exactly,

5 it is used to describe an asymptotic upper bound for the magnitude of a function in terms of another, usually simpler, function. It was first introduced by German number theorist Paul Bachmann in his 1892 book Analytische Zahlentheorie. The notation was popularized in the work of another German number theorist Edmund Landau, hence it is sometimes called a Landau symbol. The big-o, standing for "order of", was originally a capital omicron; today the capital letter O is used, but never the digit zero. Formal Definition: f(n) = O(g(n)) means there are positive constants c and k, such that 0 f(n) cg(n) for all n k. The values of c and k must be fixed for the function f and must not depend on n. Big O notation is useful when analyzing algorithms for efficiency. For example, the time (or the number of steps) it takes to complete a problem of size n might be found to be T(n) = 4n2-2n + 2. As n grows large, the n2 term will come to dominate, so that all other terms can be neglected. Further, the coefficients will depend on the precise details of the implementation and the hardware it runs on, so they should also be neglected. Big O notation captures what remains: we write O (n²) and say that the algorithm has order of n² time complexity. Search for Content To study the complexity of the searching algorithms in p2p systems, we will see first what happens in the extreme cases: Possibility 1 : Central Server - There is a Client / Server protocol. - The nodes must publish their content/data, and the server will store these locations. - When a node query for a specific data, it ask to the server for the location of data, and after, it will be able to connect to the desired node.

6 Complexity: - Searching: the algorithm has a complexity of O(1). That means that the number of steps of the algorithm of searching is constant, independent of the number of nodes N of the system. - Storage: the central server must save the location of all the information in the net. This problem has a complexity of O(N),therefore, the number of allocations that the server must save is directly proportional to the number of nodes. Very inefficient, and derives in a bad scalabily. More: - The existence of the server implicates the owner as a responsible of what happens in this system. That happened to Napster, because of the use of the system for piracy propousals. - For small systems is recommended. Possibility 2: Unstructured P2P systems - Data/files are stored at source node (provider). - Unstructured systems do not manage any routing informations. That means that there is no information about the best route to destination. - Hash values are used to identify uniquely content not for efficient routing. - Alternative to search in the system: o Flooding: high traffic load on network, does not scale.

7 Complexity: - Searching : the algorithm of flooding is totally inefficient, but almost the unique solution to find data in a unstructured system. The complexity is O(N). - Storage: Each node has to storage only a constant number of neighbours. Therefore has a complexity of O(1). As a result of both possibilities we obtain: If we center the effort in the middle, we will need to have a system with both complexities better than O(N) but worst than O(1). In this kind of system each node will have to storage some more information about the structure than in the case of instructured, but also not all the information. In this way, the searching of data will be much more efficient in exchange for some more storage overhead in each node. This systems are based in DISTRIBUTED HASH TABLES (DHT)

8 3.STRUCTURED SYSTEMS: DHT -Distributed hash tables (DHTs) are a class of decentralized, distributed systems and algorithms being developed to provide a scalable, self-configuring infrastructure with a clean programming interface. This infrastructure can then be used to support more complex services. DHTs can be used to store data, as well as route and disseminate information. DHTs are named after hash tables because they assign responsibility for a piece of data based on a hash function (often SHA-1); each node acts like a bucket in a hash table. A DHT provides an efficient lookup algorithm (or network routing method) that allows one participating node to quickly determine which other machine is responsible for a given piece of data. DHT typically seeks to achieve some or all of the following properties: -Decentralized operation: every node should be able to function independently and collectively from the complete system without any central coordination. -Scalability: the system should function efficiently even with large number of nodes. That is, it should scale. -Load balance: keys (i.e. data) should be distributed fairly among the different participants, particularly important when they have dissimilar capabilities or commitments. -Fault tolerance: the system should be reliable (in some sense) even if nodes fail or leave the system. -Performance: Operations such as routing and data storage or retrieval should complete quickly. -Data integrity: It should be easy to verify the correctness of data stored in or retrieved from the system. -Data replication: the system automatically makes multiple copies of important data, to protect from data loss, increase availability, and reduce routing delays. -Security/Robustness: The system should continue to function "correctly" even if some (possibly large) fraction of the nodes are conspiring to prevent correct operation. -Anonymity: The system should not allow observers to determine who is doing what inside the system. It is difficult to achieve all of these properties simultaneously; research into achieving these goals is on-going.

9 Form of operation: 1. Mapping of nodes and data into a name space (hash domain). 2. Storage of Content 3. Search for data at corresponding nodes 1.Mapping into Range - Mapping of content and nodes into value range by means of a hash function: H(string) mod 2m - In this way all the nodes and server will have an ID in the same hash domain. -Distribution of value range over DHT nodes ( responsibilities ). All the nodes will be responsible of one range of the hash domain and must control the allocation on the data in this range. Example: - In this case the Hash domain is There are 8 nodes. The first will be responsible of the range And the ID of the first node is 0. - The film PM123.avi, through the hash function, gives an ID equal to In the example, the node A will be responsible of this data, because has the range Storage of Content There are 2 possibilities: Direct storage: Content is stored on the node responsible. inflexible for large content o.k. if not too much data Indirect storage: Nodes in DHT store pairs (key,value). Key is the ID of one data Value is often real storage address of content: (IP, Port) =( , 4711)

10 Instead of copying the data in the responsible node, now the node only stores the real address of the content. More flexible, but one step more to reach content Example of Indirect Storage: The responsible node of the content 2313 has a pointer to the real location of the data. 3. Routing to Data Example: In this example, an arbitrary node looks for a film and must ask to the responsible. The first step is to know who is the responsible. Then this node apply the hash function to the film and obtain an ID Key = Hash( movie )-> 2313 Then the node ask to the responsible of The query goes through the net direct to the responsible. This node has the range 2001 to The responsible node has also a list with pairs (content, address). When the query of the film arrives, it answers with the real address.

11 Now the first node knows the address of the film and the connection is prepared to be made. DHT Updating: Joining and Exit of Nodes Joining of a new node Assignment of a particular hash range Copying of K/V pairs of hash range (usually with redundancy) Binding into routing environment Exit of a node Partitioning of hash range to neighbor nodes Copying of K/V pairs to corresponding nodes Unbinding from routing environment Failure of a node Use of redundant K/V pairs (if a node fails) Use of redundant / alternative routing paths Key-value usually still reachable if at least one copy remains Structure Nodes in a DHT are organized in a network overlay with a particular topology (such as a circle or a hypercube) over some space (such as the real interval [0,1)). Each node has a logical identifier that determines its logical position in the overlay. A join protocol allows a new node to bootstrap into the existing system, usually by contacting a node that is known to be in the system already. This protocol introduces the node to a set of neighbors and typically facilitates the construction of the new node's routing table. Routing tables are used by DHT nodes to efficiently determine what other node is responsible for a given piece of data. Data is given a key (in the same identifier space) and assigned to the closest node in the overlay. The definition of closest varies depending on the DHT and the topology chosen and usually does not have to do with the physical distance between nodes for example, a DHT that places nodes in a Euclidean space might simply choose the node

12 whose coordinates give the smallest Euclidean distance to the key. The routing table allows any node to find the closest node to any given key efficiently, often in O(logn) network hops (or in some cases O(1) hops). This style of routing is sometimes called key based routing. 4.SPECIFIC DHT ALGORITHMS Actually there is diverse researching efforts in different DHT algorithms. Most of them are developed in North American Universities and institutions. Chord - UC Berkeley, MIT Pastry - Microsoft Research, Rice University Tapestry - UC Berkeley CAN - UC Berkeley, ICSI P-Grid - EPFL Lausanne Nice, ZigZag 4.1 CHORD The Chord project aims to build a scalable, symmetric and robust distributed systems using peer-to-peer ideas. The basis for much of our work is the Chord distributed hash table(dht)lookup primitive. Chord is completely decentralized and can find data using only log(n) messages, where N is the number of nodes in the system. Chord's lookup mechanism is probably robust in the face of frequent node failures and re-joins. Consistent Hashing Chord has been built on principle of consistent hashing. Hash function assigns m-bit identifier to each node and to each key: - Hash-Funktion: SHA-1 - ID(node) = Hash(IP, Port) - ID(key) = Hash(key) Properties of consistent hashing: - Load balancing: An average all nodes manage the same number of pairs - In case the n-th node leaves or joins Knoten beitritt only O(1/n) pairs have to be relocated. The structure is a ring, and is called CHORD RING:

13 Value range organized in ring form: 0 k < 2M - Node k manages all keys between k and predecessor - Key s is managed in successor(s) node. Little routing information is needed to realize consistent hashing in distributed systems - Each node has just to know its successor in the ring - Requests for keys are forwarded to successor nodes along the ring 1. Search for key K is started at arbitrary node N 2. If node N maintains requested key K, pair (K, V) is returned 3. If not then request is forwarded in clock-wise order until node N >= K has been found - Reliable answer: if existing then K will be found - Concept works - Scalable but inefficient: overhead of O(N) steps!! Improvement of Consistent hashing -Each node knows (several), R, immediately succeeding nodes (neighbour set), and not only the successor.

14 Consequences: -Overhead O(N/R). Reduces the overhead. But what happens if R := N, is feasible?: - Search cost: O(1). Really Good! - But: Storage cost O(N). Really Bad. Each node would have to know the location of all the nodes of the systems. It is like all the nodes are servers in the Centralized System. Efficient Routing Instead of know the R inmediate successors, is better to have finger pointing further nodes, in exponentially way. Exponential pointers in hash range: - Node refers with M = log N pointers to value range - Routing: Search for a node in finger table that is closest to or immediately before K - Overhead: - O(log N) hops->searching - O(log N) pointers->storage Example. Search for K18 (M=7)

15 But the system needs a Periodic stabilization in order to prevent inconsistency: - Periodic checking of predecessor of successor - Periodic updating of finger tables That makes the system very inefficient, due to the high traffic needed to update the routing tables of each node. Conclusions about Chord In the steady state,in an N-node network, each node maintains routing information for only about O(logN) other nodes, and resolves all lookups via O(logN) messages to other nodes. Updates to the routing information for nodes leaving and joining require onlyo(log²n)messages. Attractive features of Chord include its simplicity, provable correctness, and provable performance even in the face of concurrent node arrivals and departures. It continues to function correctly, albeit at degraded performance, when a node s information is only partially correct. Our theoretical analysis, simulations, and experimental results confirm that Chord scales well with the number of nodes, recovers from large numbers of simultaneous node failures and joins, and answers most lookups correctly even during recovery. Some Experts believes that Chord will be a valuable component for peer-to-peer, large-scale distributed applications such as cooperative file sharing, time-shared available storage systems, distributed indices for document and service discovery, and large-scale distributed computing platforms. 4.2 PASTRY Previous: The Plaxton Approximation. Prefixed Based Routing. -Prefix based Routing-Algorithm

16 - Hash range (Nodes and Data): -Values to base b - b=16: hexadecimal system - b=4: base four -system - Example: The blue points are the nodes. The structure of the system is based on the prefixes of the ID. The first digit separate the nodes in b groups, and b is the base, and so on. node-id = 0221 Base = 3 -Data (key-value pairs) are stored at the next node in numeric order. Example: Key ->Node: >0002, 01** ->0110 Interconnection inside the prefix areas: - Nodes inside one prefix group know each other,(ip addresses are stored) - Each node in a group knows one/multiple nodes from a different group ( bridges i. e. in Small Worlds). Each prefix group must know b-1 nodes outside of the own group. Example: Prefix based routing algorithm - In each routing step a hop occurs to the node with the closest ID

17 - Normally the algorithm routes to a node with a prefix that is at least one digit closer to the target - O (log b N) routing steps - If not possible: Just route to the node with the closest id Needed data structures in Pastry nodes - Pastry routing table - Leaf set - Neighborhood set

18 Complexity of Pastry Routing The choice of b implies: Size of the routing table inversely proportional to path lengths Example: b = nodes: -75 entries in the routing table -Hop count = nodes: -105 entries in the routing table -Hop count = 7 Routing Performance - Count of Pastry hops (b=16, L = 16, M = 32, requests - Result: O(log N) for the hop count in the overlay Complexity: - O(log N) hops to destination - Often improved by additional use - O(log N) storage overhead per - Disadvantage: Logarithmic overhead - Often times even better

19 4.3 CAN Content Addressable Network - Value range as D-dimensional space - Hash value corresponds to point in D-dimensional space - Example: - D = 2 - H( Matrix.avi ) 4711 (0.7, 0.2) - DHT manages (key, value) - Node 4 manages all values of range x є [0.5, 1], y є [0, 0.25] - Bordering regions are referred to as neighbors - Coordinate spans overlap along D-1 dimensions - Nodes 6, 2, 4 and 3 are neighbors of node 5 (node 1, however, is not!) - wrap around at DHT outer limits - Expected number of neighbors for a D-dimensional space partitioned into n equal zones: 2D - Independent of size of CAN network Data are managed by the node of the respective range

20 Routing in CAN Route along the shortes path in D-dimensional space - More precisely: Neighbor with shortest distance to destination is selected as next hop! - Distance to destination is continuously decreased - Cost: O(DN 1/D ) Complexity of CAN: - State information per node : O(D) (independently of N!) - Routing: O(DN 1/D ) hops (in overlay!) (approximates linearity!) - Cost = O(log N), if D = log N is chosen - Problem: N not know a priorily Conclusions about CAN Reduced search and storage overhead: - O(log N) with Pastry and Chord - CAN approaches linear cost for search - But only O(D) storage cost - Pastry and Chord are quite similar - Pastry performs better in case of locality - Problems: Security - DoS attacks to single node can be tolerated - Problem: insertion of a node with intentionally misleading information - Partitioning: problems may occur with some DHTs

21 5 UPDATING in DHT systems: Problem: In DHT based P2P systems, a routing procedure can be satisfied fast and efficiently. To achieve this purpose, each node must maintain certain amount of routing information. As nodes join/leave system, the routing data structure must be created on the nodes or the affected routing information on several other nodes must be modified accordingly. To make system work properly, each node periodically sends hello messages to its neighbors to check their availability. Clearly, DHT algorithms achieve better routing performance than non-structured P2P algorithms at the cost of increasing system routing information maintenance overhead. In a large-scale P2P system, this overhead can be high, especially in a dynamic environment, system generates considerable routing maintenance traffic. When system detects a node joins, it will send update messages to all affected nodes. However, for a frequently join/leave node, after the affected nodes modified their routing information, it may leaves the system and the system need to send node leave messages to all these nodes to update their routing information again which causes a routing information thrashing problem. Since substantial users use unstable dial-up connections, this is a big problem which decreases DHT system routing performance. Possible Solution: Current DHT system design did not pay enough attention on the diversity of system nodes. Although the principle of P2P system is the equal treatment of all system members, peers have different behavior and have different effect on system performance. Some peers are powerful and stable which can make more contributions than other peers, some other peers only increase maintenance overhead without contributing much. As it was mentioned, in DHT algorithms, when a node joins the system, the corresponding routing information must be created on it and affected routing tables on other nodes must be modified. As a node leaves the system or a node failure occurs, the affected routing information on some nodes must be updated. or heavily dynamic nodes, system maintenance work is useless. On the other hand, for relative stable nodes, system has no much maintenance work to do. The purpose is to use stable nodes to take most system workload and reduce the DHT algorithm maintenance overhead. It is evaluated the availability of each node during the time it joins the system. It is prefered to add stable nodes into routing data structures instead of frequently join/leave nodes. Using supernodes in P2P systems is a hot research topic recently. In these systems, the powerful nodes are selected as supernodes and take most system tasks. By creating another supernode overlay network on existing P2P network, the routing tasks can finished much faster. The general approach can be viewed as the similar strategy, the supernodes are generally stay longer time than other nodes. However, these systems do not address the DHT maintenance problem, the routing information still need to be updated each time a node join/leave the system. The approach can reduce the overhead by preventing frequently join/leave nodes to pollute system routing information.

22 6.CONCLUSION - Structured P2P systems seems to be the next generation of P2P nets. The low overhead for searching makes it a powerful system. - The problem of updating makes the system inefficient. Therefore this system must wait for new solutions about updating the routing information. Until this moment it will be not commercially implanted. 7. SOURCES Zhiyong Xu, Rui Min and Yiming Hu, Reducing Maintenance Overhead in DHT Based Peer-to-Peer Algorithms, Department of Electrical & Computer Engineering and Computer Science Brighten Godfrey Karthik Lakshminarayanan Sonesh Surana Richard Karp Ion Stoica, Load Balancing in Dynamic Structured P2P Systems /04 (C) 2004 IEEE, University of Berkeley Seminar Peer-to-Peer-Systeme Praktische Informatik,Fakultät für Mathematik und Informatik Universität Mannheim Lecture SS 2004 Peer-to-Peer and Ad-Hoc Networks, Prof. Dr. H. De Meer,Universität Passau I. Stoica, R. Morris, D. Karger, M.F. Kaashoek, H. Balakrishnan: "Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications", Proc. of ACM SIGCOMM'01, San Diego, September S. Ratnasami, P. Francis, M. Handley, R. Karp, S. Shenker: A Scalable Content- Addressable Network Proc. ACM IGCOMM 2002, San Diego