Simple Determination of Stabilization Bounds for Overlay Networks A. Introduction The landscape of computing has changed dramatically in the past half-century. Computers are now smaller, faster, and near-omnipresent. Computer ownership has gone from one per major corporation to several per person. Technology has become so advanced that now smart phones allow continuous connections amongst individuals. The computing of today is nearly unrecognizable from the computing of the past. These changes have had a huge impact on distributed computation. Past work in distributed computing is mostly based upon a model of computer networks with fixed topologies and static process membership. Today, many networks have a very dynamic process membership and changing topologies, such as with peer-to-peer (P2P), sensor, or mobile social networks. While there are still a large number of static networks (for instance, the many large data centers sprouting up recently), an increasing number of these new dynamic networks have created the need for new distributed algorithms. One recent area of research that has been created for these new networks has been self-stabilizing overlay networks. Instead of fixed physical links, an overlay network uses logical links between processes for communication, where a single logical link is composed of one or more physical links. The use of logical links means processes can change the topology easily, allowing creation of any topology. This gives overlay networks the ability to have desirable properties like low diameter or low degree, even with very dynamic network membership. Self-stabilizing overlay networks promise that a group of processes can form a desired topology when starting from any arbitrary weakly-connected topology. As many overlay networks are deployed in hostile environments, the design of such systems is desirable for researchers and practitioners alike. Self-stabilizing overlay network research is still quite new and unexplored. To guide the exploration of these networks, we present in this paper several optimality bounds for self-stabilizing overlay network construction. Specifically, we define two measures (the fault- 1
detector distance and the detector diameter) that can be used to bound the optimal convergence time of any self-stabilizing overlay network algorithm. Furthermore, we show that all families of overlay networks have a fault-detector distance bounded below by the diameter of the network. We finish with the presentation of a special form of graphs for which the detector diameter is easily bounded. These results are the first of their kind in this field, and we believe they will be quite useful for guiding future research. B. Related Work Self-stabilization is an established area of distributed computing research, created in 1974 by Dijkstra [3]. Self-stabilization promises a program will always reach some desired state when starting from any arbitrary initial state. Many self-stabilizing algorithms for distributed systems have been created. An excellent exposition of self-stabilization can be found in Dolev s book, Self-stabilization[4]. Overlay networks have become popular in the past few decades, largely due to the rise in peer-to-peer networking. Overlay networks can be divided into two types - structured and unstructured. In an unstructured network, there are no restrictions on the neighborhood of each node. While maintenance of these networks is easy, performance (such as routing times or search success rates) usually suffers. One very popular example of an unstructured network is the P2P network Gnutella, which has thousands of connected computers[8]. Conversely, structured networks place restrictions on the links in the overlay network. These restrictions make maintenance more expensive, but allow desirable properties such as low diameter for efficient routing and low node degree for scalable state space requirements. Current structured P2P networks, such as Chord[9], are not self-stabilizing, however the legal topology can only be guaranteed when starting from a subset of possible states. Recently, a few researchers have begun exploring the combination of self-stabilization and overlay network construction. Self-stabilizing overlay networks promise that, regardless of the initial weakly-connected topology, a correct configuration will always be created. The few examples of self-stabilizing overlay networks are silent algorithms. A silent self-stabilizing 2
algorithm is one where the execution of the algorithm ceases for all processes after a legal configuration is reached. Silent self-stabilizing overlay networks are desirable in that they do not consume additional resources once a legal state is reached. A key requirement for the building of silent self-stabilizing overlay networks is local checkability[1]. Informally, a locally-checkable network is one where any configuration that does not match the correct configuration contains at least one process that can detect, using only local information, that the network is in an illegal configuration. If a network is not locally-checkable, then there exists at least one configuration such that the network is incorrect or faulty, but every process believes the topology is correct. In such a situation, the only means to guarantee stabilization is to continuously search the network, compiling global information which can be used to detect a fault. Clearly, this method is not silent, and requires maintenance overhead throughout the network s life. One of the first examples of a self-stabilizing overlay network was the simple problem of linearization arranging nodes in a line sorted by unique immutable identifiers[7]. Later, a more advanced topology was created in a self-stabilizing fashion the Skip+ graph[6]. The Skip+ graph is a locally-checkable version of the Skip graph. The Skip+ graph has low diameter (O(log n)) and low node degree (O(log n) w.h.p.), two major scalability concerns for overlay networks. Currently, the Skip+ graph is the only published self-stabilizing overlay network that has a scalable diameter and node degree. These two topologies currently are the only examples of self-stabilizing overlay networks, demonstrating how nascent the field is. C. Model As is done for other self-stabilizing overlay network research, we use a model that is a modified version of the traditional distributed system models. As with the traditional model, we represent the distributed system as a graph G = (V, E), with processes modeled as the nodes of the graph V and communication links modeled as the edges E. Process v is considered a neighbor of process u if and only if the edge (u, v) E. The set of all neighbors 3
of u is called u s neighborhood, and is represented N(u). The state of each process is the value of its local variables. A process may read and write to it s own state, and read the state of its neighbors. The collection of the states of all processes is called a configuration. Processes may also contain some immutable information, such as an identifier or random sequence. Each process executes a program, which consists of a set of guarded actions of the form <guard> <action>. The guard is a Boolean expression over a process s local state and the state of its neighbors. When the guard evaluates to true, the action can be executed. A process with a guard that evaluates to true is considered enabled. The system proceeds in synchronous rounds, with each enabled process executing an action in every round. A computation is a (potentially infinite) series of configurations, S 0, S 1,, S i, S i+1, such that S i+1 is reachable from S i by executing all enabled actions in S i. Overlay networks require a different model than traditional distributed systems, however, as overlay networks allow each process to change its communication links that is, E is actually a dynamic set that may change at every step of a computation. Process i s neighborhood N(i) is now part of process i s state. Therefore, not only can a process read the local state of all neighbors, it can also identify the neighbors of its neighbors. In other words, a process i is aware of it s 2-neighborhood, represented as N 2 (i). Each time a process changes its neighborhood, it also changes the states it is capable of reading. This unique aspect is what differentiates overlay network models from prior work in distributed computation. The goal of the computation in self-stabilizing overlay network construction is to create a specific network topology. We consider a family of overlay networks as a mapping ON : V G, where V is the set of all possible sets of processes and G is the set of all directed graphs. ON(V ), then, takes a specific set of processes V (and any immutable information associated with these processes) and returns a directed graph from the family of the overlay network. We consider this returned graph to be the ideal configuration. As a final part of defining our model, we present the following terms. Node i is considered 4
a detector if the calculated ideal 2-neighborhood ON(N 2 (i)) does not match the current subgraph induced by i s 2-neighborhood. Detectors can initiate corrective actions to move the network to an ideal configuration. Therefore, their location in the network is important for determining optimal convergence time. Besides detectors, convergence also depends upon the location of faulty nodes. A faulty node u V is a node whose current neighborhood N(u) does not match the ideal neighborhood of u from ON(V ). Note a faulty node need not be a detector, nor does a detector need to be faulty. D. Bounds To define the bounds of overlay network construction, we need several measures of detector distribution the detector diameter (first introduced in a prior result[2]), and the fault-detector distance (defined here in our work). Imagine some weakly-connected directed graph G consisting of all nodes in a set V, with V = n. The detector diameter of G is the maximum shortest path between any faulty node and its closest detector. The fault-detector distance of G is defined as the maximum shortest path between any faulty node and its closest detector. The detector diameter D(n) (fault-detector distance F DD(n)) for the family of overlay networks ON is the largest detector diameter (fault-detector distance) taken over all weakly-connected graphs of V. In other words, regardless of the network configuration, all nodes are within distance D(n) of a detector, while every faulty node is within F DD(n) of a detector. We may now present the following theorem. Theorem 1. A silent self-stabilizing algorithm for constructing an overlay network ON(λ) cannot guarantee convergence in less than Ω(F DD(n)) rounds. Proof. Imagine a network with fault-detector distance F DD(n) = d f, and let node u be a faulty node of distance d f from its closest detector. In order for the faulty node u to repair itself, it must detect a fault and execute corrective actions. Consider the subgraph G df induced by all nodes of distance at most (d f 1) from node u. Note that no node in this subgraph is a detector. As the self-stabilizing algorithm is silent, there exists some setting of the local variables 5
of all nodes in G df such that no node in G df changes its neighborhood. At this point, G df contains no detectors, and has reached a silent state such that the neighborhoods of all nodes in G df will not change unless a fault is detected. For a fault to be detected in the silent G df, then, either the 2-neighborhood of a node in G df must change, or a node in G df must read a neighboring node s state and discover a detector exists in the system. In the first round, no node inside G df will execute an action. Therefore, a node outside of G df must execute in a faulty configuration, which may be detected by nodes that are distance (d f 1) from node u. Again, in the second round, only nodes outside of G df, as well as nodes of distance (d f 1) from node u will execute. Now nodes at distance (d f 2) from u may detect a fault. In general, nodes that are distance (d f i) from node u cannot become detectors until after i rounds. This implies at least d f rounds are required for node u to become a detector, and thus begin corrective action. Therefore, no silent self-stabilizing algorithm for an overlay network may converge in less than Ω(F DD(n)) rounds. Notice we can bound the fault-detector distance for overlay networks. Theorem 2. The fault-detector distance F DD(n) for a family of overlay networks ON is at least d, where d is the diameter of the ideal network ON(V ), for V = n. 2 Proof. We prove Theorem 2 by constructing an overlay network with fault-detector distance at least d. Imagine n nodes arranged in their ideal configuration G = ON(V ) and having 2 diameter d. There exists a shortest path consisting of distinct nodes p 0, p 1,, p d in the network. Let G = ON(V ) (where V = (V \ p 0 )) that is, G is the network that results from removing node p 0 and reconfiguring to a legal configuration. There are two cases concerning the distance between nodes p 1 and p d in G. Case 1: dist G (p 1, p d ) > d 2 - If the distance between nodes p 1 and p d remains at least d in 2 G, then we can insert node p 0 as a neighbor to node p d. The network is now faulty, as p 0 must be neighbors with p 1 in the ideal configuration. Furthermore, only p d and its 6
immediate neighbors have knowledge of node p 0, and these nodes are at least d away from 2 node p 1. Therefore, the only detectors (node p d and neighbors) are at least distance d from 2 p 1, and it holds that F DD(n) > d. 2 Case 2: dist G (p 1, p d ) < d 2 - Let nodes p 1 and p d be closer than d 2 in G. A node within d 2 of p d must have then changed its neighborhood from G to G, or else the set of nodes within d of p 2 d will not have changed, and p 1 remains at least d from p 2 d. Notice, too, that all nodes within d 2 of p d in G are also at least d 2 from p 1 in G. Therefore, if node p 0 is removed from G and the network is not reconfigured, only node p 1 and its immediate neighbors are detectors, and a faulty node exists at least d 2 away from p 1. Thus, removing p 0 from G results in a network with F DD(n) > d 2. Notice that this bound, the first of its kind, is significant in that it allows an easy method for evaluating overlay networks. For instance, it is now simple to show optimal convergence time simply by referencing the diameter of the network. Furthermore, it can help direct future research by focusing efforts on scalable convergence time to those networks with low diameter e.g. designing scalable algorithms for linearization is impossible, as it cannot converge faster than O(n) time. E. A Special Class of Networks Above, we provided a lower bound on the stabilization time of all overlay networks. It turns out that for a particular class of graphs we can provide a tight optimal convergence bound. Prior research has proven that any overlay network can be built in D(n) + log(n) rounds[2]. Therefore, if we can provide an upper bound on the detector diameter for some networks, we can tightly bound the optimal convergence time for these networks. To do this, we offer the following axioms. Axiom 1. Let G be an arbitrary network configuration, with desired configuration equal to G = ON(λ). Consider a subgraph G k of G induced by all nodes of distance k from some node u. If all nodes in this subgraph were not detectors in G, then the subgraph is a legal 7
overlay network (G k = ON(λ ), where λ is all nodes and associated identifiers and random sequences from G k ). Axiom 2. Let G λ = ON(λ) and G λ = ON(λ ) be two legal overlay networks such that the nodes in G λ are a subset of the nodes in G λ. The diameter of G λ is no larger than the diameter of G λ. We now bound the detector diameter from above. Theorem 3. Let d be the diameter of G λ = ON(λ). For all locally-checkable overlay networks satisfying Axioms 1 and 2, the detector diameter is no more than (d + 1). Proof. We prove Theorem 3 by contradiction. Imagine that D(n) > d + 1. This means in some configuration G there exists a node u whose closest detector is greater than distance d + 1 away. Let G u be the ball induced on G by all nodes of distance at most D(n) 1 from u. Notice that this ball contains no detectors. By Axiom 1, if all nodes except those in G u are removed, then G u forms a correct overlay network - that is, no new edges are introduced to G u that were not in G. Note also that G u has a diameter of at least d + 1, and contains at least one less node than G. However, by Axiom 2, this is impossible. Therefore, the detector diameter can be no more than d + 1. For networks satisfying Axioms 1 and 2, D(n) is at most (d + 1). Therefore, we have defined a set of overlay networks for which the tight bound on optimal convergence time is equal to the diameter of the ideal network (assuming the diameter is at least O(log n)). The following example shows the application of our results. We consider the seminal problem of silent self-stabilizing overlay networks linearization[7]. An example linearized graph is shown in Figure 1. In a linearized graph, every node u has a unique integer identifier, u.id. Furthermore, every node u has at most two neighbors, u.l and u.r. We use to represent the case where no neighbor exists. A linearized graph satisfies the following conditions. 8
1. u.l.id < u.id u.l = 2. u.r.id > u.id u.r = Several papers have been published investigating linearization and striving towards polylogarithmic convergence time[7, 5]. In the following paragraphs, we prove such a result cannot exist. Lemma 1. Linearization is locally-checkable. Proof. This result follows from the definition of a linearized graph. The conditions defining a linearized graph depend only upon a node u and it s immediate neighbors. If we assume the conditions are true for all nodes and represent these as a conjunction (p 0.L = p 0.R.id > p 0.id p n 1.R = ), it clearly represents the single correct configuration for linearization. Lemma 2. Linearization satisfies Axiom 1. Proof. Consider what happens when we have a subgraph of only non-detectors. At most two nodes have their neighborhoods changed. If previously these nodes satisfied the conditions above by having a left or right neighbor of, this condition still holds in our subgraph. If the nodes previously had left or right neighbors, they now have as the left or right neighbor, which still satisfies the above conditions. Therefore, a subgraph of non-detectors is a legal overlay network. Lemma 3. Linearization satisfies Axiom 2. Proof. Notice that a correct linearized graph has a diameter equivalent to the number of nodes in the graph. Therefore, as G λ has fewer nodes than G λ, it also has a smaller diameter, and Axiom 2 holds. Theorem 4. The optimal convergence time for linearization is Θ(n). 9
Proof. By Theorem 1, convergence time is bounded by F DD(n). Furthermore, by Theorem 2, the fault-detector distance of linearization must be at least d. The detector diameter for 2 linearization is no more than d + 1, which follows from Theorem 3 and Lemmas 2 and 3. By definition, F DD(n) D(n). Therefore, F DD(n) O(d). As known by the prior result[2], a linearized topology can be built in Θ(n) rounds, so our optimal convergence bound is tight. F. Conclusions In this paper, we have presented several bounds for self-stabilizing overlay network construction. Furthermore, we have introduced a special class of graphs for which optimal convergence time is easily bounded by the network diameter. These bounds are the first of their kind in this area of research. Using these bounds, researchers and practitioners can easily select topologies to begin investigating for self-stabilization. Likewise, they can rule out certain topologies as poor choices for scalable convergence (for instance, linearization). There are several open questions implied by our results. First, one can examine whether our presented axioms are the weakest possible, or if in fact networks that do not satisfy these axioms still have optimal convergence time equivalent to the diameter. Also, as local checkability is a requirement for silent self-stabilizing overlay network construction, examining how to prove a network is locally checkable is an open and meaningful problem (for instance, the Skip+ graph[6] does not have a proof of local checkability). Finally, an obvious area of future work is designing other self-stabilizing algorithms for different network topologies. Our framework will greatly assist this wide-open field by allowing easy calculation of convergence time, and by helping researchers select the desired topology. As the nature of distributed computation will continue to change in the future, our work in specific, and self-stabilizing overlay networks in general, will continue to grow and develop into an important area of computing. 10
References [1] B. Awerbuch, B. Patt-Shamir, and G. Varghese. Self-stabilization by local checking and correction (extended abstract). In Proceedings of the 32nd annual SFCS 91, pages 268 277, Washington, DC, USA, 1991. IEEE Computer Society. [2] A. Berns, S. Ghosh, and S. V. Pemmaraju. Brief announcement: a framework for building self-stabilizing overlay networks. In Proceeding of PODC 10, pages 398 399, New York, NY, USA, 2010. ACM. [3] E. W. Dijkstra. Self-stabilizing systems in spite of distributed control. Commun. ACM, 17(11):643 644, 1974. [4] S. Dolev. Self-stabilization. MIT Press, Cambridge, MA, USA, 2000. [5] D. Gall, R. Jacob, A. Richa, C. Scheideler, S. Schmid, and H. Täubig. Modeling scalability in distributed self-stabilization: The case of graph linearization. Technical Report TUM-I0835, TU Munich, November 2008. [6] R. Jacob, A. Richa, C. Scheideler, S. Schmid, and H. Täubig. A distributed polylogarithmic time algorithm for self-stabilizing skip graphs. In Proceedings of PODC 09, pages 131 140, New York, NY, USA, 2009. ACM. [7] M. Onus, A. W. Richa, and C. Scheideler. Linearization: Locally self-stabilizing sorting in graphs. In ALENEX, 2007. [8] S. Saroiu, K. P. Gummadi, and S. D. Gribble. Measuring and analyzing the characteristics of napster and gnutella hosts. Multimedia Syst., 9(2):170 184, 2003. [9] I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan. Chord: A scalable peer-to-peer lookup service for internet applications. SIGCOMM Comput. Commun. Rev., 31(4):149 160, 2001. 11
Figures Figure 1: An Example of Linearization with Eight (8) Nodes 12