Efficient and Accurate Delaunay Triangulation Protocols under Churn

Transcription

1 Efficient and Accrate Delanay Trianglation Protocols nder Chrn Dong-Yong Lee and Simon S. Lam Department of Compter Sciences The University of Texas at Astin {dylee, November 9, 2007 Technical Report # TR Abstract We design a new site of protocols for a set of nodes in d-dimension (d > 1) to constrct and maintain a distribted Delanay trianglation (DT) in a dynamic environment. The site incldes join, leave, failre, and maintenance protocols. The join, leave, and failre protocols are proved to be correct for a single join, leave, and failre, respectively. In practice, protocol processing of events may be concrrent. For a system nder chrn, it is impossible to maintain a correct distribted DT continally. We define an accracy metric sch that accracy is 100% if and only if the distribted DT is correct. The maintenance protocol is designed to recover from incorrect system states and to improve accracy. In designing the protocols in this paper, we made se of two new observations to sbstantially improve their efficiency. First, in the neighbor discovery process of a node, many replies to the node s qeries contain redndant information. Second, the se of a new failre protocol that employs a proactive approach to recovery is better than the reactive approach sed in [1, 2]. Experimental reslts show that or new site of protocols maintains high accracy for systems nder chrn and each system converges to 100% accracy after chrning stopped. They are mch more efficient than protocols in prior work [1, 2]. 1. Introdction Delanay trianglation [3] and Voronoi diagram [4] have a long history and many applications in different fields of science and engineering inclding networking applications, sch as greedy roting [5], finding the closest node to a given point, broadcast, mlticast, etc.[1]. A trianglation for a given set S of nodes in a 2D space is a sbdivision of the convex hll of nodes in S into non-overlapping triangles sch that the vertexes of each triangle are nodes in S. A Delanay triangla- Research sponsored by National Science Fondation grant CNS tion in 2D is sally defined as a trianglation sch that the circmcircle of each triangle does not inclde any node other than the vertexes of the triangle. Delanay trianglation can be similarly generalized to a d-dimensional space 1 (d > 1) sing simplexes instead of triangles [6]. We will se DT as abbreviation for Delanay trianglation. Or objective is a site of protocols for a set of nodes in a d-dimensional space (d > 1) to constrct and maintain a distribted DT. In designing these protocols, we allow the set of nodes to change with time. New nodes join the set (system) and existing nodes leave (graceflly) or fail 2 over time. The system is said to be nder chrn and the rate at which changes occr said to be the chrn rate. We present in this paper a site of for protocols for join, leave, failre, and maintenance. In a distribted DT, each node maintains a set of its neighbors. By definition, a distribted DT of a set of nodes S is correct if and only if, for every node S on the distribted DT, s neighbor set is the same as the set of s neighbors on the (centralized) DT of S [1]. For convenience, we will sometimes say the system state is correct to mean the distribted DT is correct. In a previos paper [1], we discovered and proved a necessary and sfficient condition for a distribted DT to be correct. In designing the site of protocols in this paper, we aim to achieve three properties: correctness, accracy, and efficiency: Correctness We prove the join, leave, and failre protocols to be correct for a single, join, leave, and failre, respectively. For the join protocol, we prove that if the system state is correct before a new node joins, and no other node joins, leaves, or fails dring the join protocol exection, then the system state is correct after join pro- 1 Delanay trianglation is defined in a Eclidean space. When we say a d-dimensional space in this paper, we mean a d-dimensional Eclidean space. 2 When a node fails, it becomes silent. We do not consider Byzantine failres.

2 tocol exection. A similar correctness property is proved for the leave and failre protocols. Note that these three protocols are adeqate for a system whose chrn rate is so low that joins, leaves, and failres occr serially, i.e., protocol exection finishes for each event (join, leave, or failre) before another event occrs. In general, for systems with a higher chrn rate, we also provide a maintenance protocol, which is rn periodically by each node. Accracy It is impossible to maintain a correct distribted DT continally for a system nder chrn. Note that correctness of a distribted DT is broken as soon as a join/leave/failre occrs and is recovered only after the join/leave/failre protocol finishes exection. Fortnately, some applications, sch as greedy roting, can work well on a reasonably accrate distribted DT. We previosly presented an accracy metric for a distribted DT [1]; we will show that the accracy of a distribted DT is 1 if and only if the distribted DT is correct. The maintenance protocol is designed to recover from incorrect system states de to concrrent protocol processing and to improve accracy. We fond that in all of or experiments condcted to date with the maintenance protocol, each system that had been nder chrn wold converge to 100% accracy some time after chrning stopped. Efficiency We se the total nmber of messages sent dring protocol exection as the measre of efficiency. Protocols are said to be more efficient when their exection reqires the se of fewer messages. In a previos paper[1], we presented examples of networking applications to rn on top of a distribted DT, namely: greedy roting, finding the closest existing node to a given point, clstering of network nodes, as well as broadcast and mlticast withing a given radis withot session states. Three DT protocols were presented: join and leave protocols that were proved correct and a maintenance protocol that was shown to converge to 100% accracy after system chrn. However, the join and maintenance protocols in this site (the old protocols) were designed with correctness as the main goal and their exection reqires the se of a large nmber of messages. To make the new join and maintenance protocols in this paper mch more efficient, we make two novel observations. First, the objective of the join protocols is for a new node n to identify its neighbors (on the global DT), and for n s neighbors to detect n s join. n sends a reqest to an existing node for n s neighbors in s local information. When n receives a reply, it learns new neighbors and sends reqests to those newly-learned neighbors. This process is recrsively repeated ntil does not find any more new neighbor. Whereas it is necessary to send messages to all neighbors, since the neighbors need to be notified that n has joined, we discovered that n only needs to hear back from jst one neighbor in each simplex that incldes n rather than from all neighbors. Frthermore, qeries as well as replies for some simplexes can be combined so that jst one qery-reply between n and one neighbor is needed for mltiple simplexes. Based on this observation, we designed a new join protocol. We fond that the new join protocol is mch more efficient than or old join protocol. We have proved the new join protocol to be correct for a single join. We also apply the above observation to sbstantially redce the nmber of messages sed by the new maintenance protocol. Frthermore, we make the following second observation to greatly redce the total nmber of all protocol messages per nit time by redcing the freqency at which the new maintenance protocol rns. We keep the leave protocol in [1] nchanged in this paper becase it can efficiently address gracefl node leaves. However, with the old site of protocols, it is the old maintenance protocol s job to detect node failres and repair the reslting distribted DT. To detect a node failre, the node was probed by all of its neighbors. Frthermore, the distribted DT was repaired in a reactive fashion. The process of reactively repairing a distribted DT after a failre is inevitably costly, becase the information needed for the repair was at the failed node and lost after failre. To improve overall efficiency, we designed a new failre protocol to handle node failres. The failre protocol employs a proactive approach. Each node designates one of its neighbors as its monitor node. In the failre protocol, a node is probed only by its monitor node, eliminating dplicate probes. In addition, each node prepares a contingency plan and gives the contingency plan to its monitor node. The contingency plan incldes all information to correctly pdate the distribted DT after its failre. Once the failre of a node is detected by its monitor node, the monitor node initiates failre recovery. That is, each neighbor of the failed node is notified of the failre as well as any new neighbor(s) that it shold have after the failre. In this way, node failres are handled almost as efficiently as gracefl node leaves in the leave protocol. We have proved the new failre protocol to be correct for a single failre. Each node rns the maintenance protocol (new or old) periodically. The commnication cost of the maintenance protocol increases as the period decreases (or freqency increases). Generally, as the chrn rate increases, the maintenance protocol needs to be rn more freqently. In the old protocol site, moreover, the old maintenance protocol needs to be rn at the probing freqency becase one of its fnctions is to recover from node failres. With the inclsion of an efficient failre protocol in the new protocol site to handle failres separately, the new maintenance protocol can be rn less often. We fond that the overall efficiency of the protocols as a whole is greatly improved as a reslt. 2

3 To the best of or knowledge, the only previos work for a dynamic distribted DT in a d-dimensional space is by Simon et al. [2]. They proposed two sets of distribted algorithms: basic generalized algorithms and improved generalized algorithms. Each set consists of an entity insertion (node join) algorithm and an entity deletion (node failre) algorithm. Their basic entity insertion algorithm is similar to or old join protocol. Their improved entity insertion algorithm is based on a centralized flip algorithm [7] whereas or join protocols are based on a candidate-set approach and or correctness condition for a distribted DT. The two approaches are fndamentally different. Their entity deletion algorithm and or failre protocols are also different. Or failre protocol is sbstantially more efficient than their improved entity deletion algorithm, which ses a reactive approach and allows dplicate probes. The centralized flip algorithm is known to be correct [8]. However, correctness of their distribted algorithms is not explicitly proved. Lastly, they do not have any algorithm, like or maintenance protocols, for recovery from concrrent processing of joins and failres de to system chrn. As a reslt, their algorithms failed to converge to 100% accracy after system chrn in or simlation experiments. A qick comparison of the for sets of protocols/algorithms is shown in Table 1. More detailed experimental reslts are presented in Section 7 of this paper. efficiency convergence to 100% accracy after system chrn Simon et al. s basic medim No algorithms Simon et al. s improved high No algorithms Or old protocols low Yes Or new protocols high Yes Table 1. A comparison of the old and new protocols with Simon et al. s basic and improved algorithms. The organization of this paper is as follows. In Section 2, we introdce the concepts and definitions of a distribted DT, present or system model, and a correctness condition for a distribted DT. We present the new join protocol in Section 3, the new failre protocol in Section 4, and the new maintenance protocol in Section 5. The accracy metric is defined in Section 6 and experimental reslts are presented in Section 7. We conclde in Section 8. v Figre 1. A Voronoi diagram (dashed lines) and the corresponding DT (solid lines) in a 2- dimensional space. 2. Distribted Delanay trianglation 2.1. Concepts and definitions Definition 1. Consider a set of nodes S in a Eclidean space. The Voronoi diagram of S is a partitioning of the space into cells sch that a node S is the closest node to all points within its Voronoi cell V C S (). That is, V C S () = {p D(p, ) D(p, w), for any w S} where D(x, y) denotes the distance between x and y. Note that a Voronoi cell in a d-dimensional space is a convex d- dimensional polytope enclosed by (d 1)-dimensional facets. Definition 2. Consider a set of nodes S in a Eclidean space. V C S () and V C S (v) are neighboring Voronoi cells, or neighbors of each other, if and only if V C S () and V C S (v) share a facet, which is denoted by V F S (, v). Definition 3. Consider a set of nodes S in a Eclidean space. The Delanay trianglation of S is a graph on S where two nodes and v in S have an edge between them if and only if V C S () and V C S (v) are neighbors of each other. Figre 1 shows a Voronoi diagram (dashed lines) for a set of nodes in a 2D space and a DT (solid lines) for the same set of nodes. V C S () and V C S (v) are neighbors of each other. We also say that and v are neighbors of each other when V C S () and V C S (v) are neighbors of each other. Note that facets of a Voronoi cell perpendiclarly bisect edges of a DT. Therefore, a DT is the dal of a Voronoi diagram. 3 Let s denote the DT of S as DT(S). Definition 4. A distribted Delanay trianglation of a set of nodes S is specified by {<, N > S}, where N represents the set of s neighbor nodes, which is locally determined by. 3 In geometry, polyhedra are associated into pairs called dals, where the vertices of one correspond to the faces of the other. 3

4 Definition 5. A distribted Delanay trianglation of a set of nodes S is correct if and only if both of the following conditions hold for every pair of nodes, v S: i) if there exists an edge between and v on the global DT of S, then v N and N v, and ii) if there does not exist an edge between and v on the global DT of S, then v N and N v. That is, a distribted DT is correct when for every node, N is the same as the neighbors of on DT(S). Since does not have global knowledge, it is not straightforward to achieve correctness System model Or approach to constrct a distribted DT is as follows. We assme that each node is associated with its coordinates in a d-dimensional Eclidean space. Each node has prior knowledge of its own coordinates, as is assmed in previos work [9, 10, 2, 11, 12]. The mechanism to obtain coordinates is beyond the scope of this stdy. Coordinates may be given by an application, a GPS device[13], or topology-aware virtal coordinates[14]. 4 Also when we say a node knows another node v, we assme that knows v s coordinates as well. Let S be a set of nodes to constrct a distribted DT from. We will present protocols to enable each node S to get to know a set of its nearby nodes inclding itself, denoted as C, to be referred to as s candidate set. Then determines the set of its neighbor nodes N by calclating a local DT of C, denoted by DT(C ). That is, v N if and only if there exists an edge between and v on DT(C ). To simply protocol descriptions, we assme that message delivery is reliable. In a real implementation, additional mechanisms sch as ARQ may be sed to ensre reliable message delivery Correctness condition for a distribted Delanay trianglation Recall that a distribted DT is correct when for every node, N is the same as the neighbors of on DT(S). Since N is the set of s neighbor nodes on DT(C ) in or model, to achieve a correct distribted DT, the neighbors of on DT(C ) mst be the same as the neighbors of on DT(S). Note that C is local information of while S is global knowledge. Therefore in designing or protocols, we need to ensre that C has enogh information for to correctly identify its global neighbors. If C is too limited, cannot identify its global neighbors. For the extreme case of C = S, can identify its neighbors on the global DT since DT(C ) = DT(S); however, the commnication overhead for each node to acqire global knowledge wold be extremely high. 4 Application performance on a DT may be affected by the accracy of virtal coordinates. Theorem 1 (Correctness Condition). Let S be a set of nodes and for each node S, knows C, sch that C S. The distribted DT of S is correct if and only if, for every S, C incldes all the neighbor nodes of on DT(S). Theorem 1, which was presented in or previos paper [1], identifies a necessary and sfficient condition for a distribted DT to be correct, namely: the candidate set of each node contains all of its global neighbors. In the following sbsections, we se the above correctness condition as a gide to design or protocols. A proof of Theorem 1 is presented in [15]. 3. Join protocols 3.1. Flip algorithm in a d-dimensional space Flipping is a well-known and often-sed techniqe to incrementally constrct DT in 2D and 3D spaces. A centralized flip algorithm was also proposed to be sed for a d- dimensional space [7] and was proved to be correct [8]. Note that two triangles in a 2D space are flipped into two other triangles, and two tetrahedra in a 3D space are flipped into three tetrahedra. In general, two simplexes in a d-dimensional space are flipped into d simplexes. This transformation is called 2-d flipping. Incremental constrction of DT based on flipping is as follows. When a new node is inserted, the simplex that encloses the new node is divided into (d + 1) new simplexes. Recall that the circm-hypersphere of a simplex on a DT shold not inclde any other node except for the vertexes of the simplex. Each new simplex is checked whether its circm-hypersphere incldes any other node. In case a simplex does inclde another node, it is flipped to generate new simplexes. The new simplexes are checked, and flipped if necessary. This process contines recrsively. The flip algorithm reqires a general position assmption, namely: no d + 1 nodes are on the same hyperplane and no d + 2 nodes are on the same hypersphere [6]. Figre 2 shows an example of flipping in a 2D space. A node n is inserted to a distribted DT. First, the simplex vw that encloses n is divided into three new simplexes (top figre). Then each new simplex is checked whether its circm-hypersphere incldes any other node. In this example, the circm-hypersphere of nv incldes another node e. Therefore nv and ev are flipped into ne and vne (bottom figre). Distribted flip algorithms for joining were proposed for 2D[9], 3D[10], and a d-dimensional space [2]. The centralized flip algorithm is known to be correct (for a single join or serial joins). Since a simplex in d-dimension has d + 1 nodes, operations at the d + 1 nodes mst be consistent in a distribted algorithm. Correctness has not been explicitly proved for any of the distribted algorithms. 4

5 w n n e e v v (a) (b) (c) (d) Candidate-set approach A joining node n is led to a closest existing node z. z calclates local DT sing C z and n, and sends n s neighbors on DT(C z ) to n. n contacts each of its new neighbors to see whether there are other potential neighbors. n recrsively contacts new neighbors. Flip algorithm A joining node n is led to a closest existing node. The simplex that encloses n is divided into (d + 1) simplexes. The new simplexes are checked and flipped if necessary. New (flipped) simplexes are recrsively checked. w Figre 2. An example of flipping in 2D Candidate-set approach In a previos paper [1], we proposed a join protocol based on the distribted system model sing candidate sets and the correctness condition for a distribted DT introdced in Section 2. When a new node n joins a distribted DT, it is first led to the closest existing node z. 5 Then n sends a reqest to z for mtal neighbors of n and z on DT(C z ). When n receives the reply, n pts the mtal neighbors in its candidate set (C n ) and re-calclates its neighbor set (N n ). If n finds any new neighbors, n sends reqests to the new neighbors. This process is repeated recrsively. We proved correctness of this protocol [15]. The flip algorithm and candidate-set approach are fndamentally different. However, it is interesting to note that there is a correspondence between the two. Table 2 shows how steps of the two different approaches correspond to each other. Whereas the two approaches have corresponding steps, the steps are not exactly the same. For example, in step (b), n initially learns (d + 1) neighbors in the flip algorithm. In step (b) of the candidate-set approach, n may be informed of any nodes that z knows. In step (c) of the candidate-set approach, mltiple neighbors may send dplicate messages to n to inform n of the same new neighbor. In step (c) of the flip algorithm, only one node may reply that a simplex is flipped. This observation gave s an idea to sbstantially improve the efficiency of or join protocol. [1]. 5 This can be done sing the protocol for finding a closest existing node in Table 2. Correspondence between join protocol in the candidate-set approach and flip algorithm New join protocol Using the observation described above, we designed a new join protocol that is sbstantially more efficient than or old one. In addition to C n and N n, a joining node n keeps Nn qeried, which incldes the neighbors that are already qeried dring its join process. Instead of qerying all new neighbors, n qeries only one neighbor for each simplex that does not inclde any node in Nn qeried. Note that only one neighbor in each simplex needs to be qeried. If a simplex incldes a node v Nn qeried, it means that the simplex has already been checked by v. Frthermore, qeries as well as replies for mltiple simplexes may be combined. The new join protocol reqires the general position assmption, which was not reqired for the old join protocol. Or new join protocol is still based on or candidate-set model and its correctness for a single join is proved sing the correctness condition Theorem 1. The new join protocol still has some dplicate comptation. Even thogh there is only one qery-reply interaction for each simplex, DT is calclated independently at each node. Psedocode of the new join protocol at a node is given in Figre 3. The protocol exection loop at a joining node, say n, and the response actions at an existing node, say v, are presented below. 6 Protocol exection loop at a joining node n At a joining node n, the join protocol rns as follows with a loop over steps 3-6: 6 For clarity of presentation, the new join protocol is presented sch that the joining node processes each NEIGHBOR SET REPLY message one at a time. We note that if the joining node can process mltiple reply messages in step 3 of the loop, the nmber of qery messages may be redced; this change wold not affect the correctness proof for the join protocol in the next section. 5

6 1. A joining node n is first led to a closest existing node z. 2. n sends a NEIGHBOR SET REQUEST message to z. C n is set to {n, z} and Nn qeried is set to {z}. Repeat steps 3-6 below ntil a reply has been received for every NEIGHBOR SET REQUEST message sent: 3. n receives a NEIGHBOR SET REPLY message from a node, say v. The message incldes mtal neighbors of n and v on DT(C v ). 4. n adds the newly learned neighbors (if any) to C n, and calclates DT(C n ). 5. Among simplexes that inclde n on DT(C n ), simplexes that do not inclde any node in Nn qeried are identified as nchecked simplexes. n selects some of its neighbors sch that each nchecked simplex incldes at least one selected neighbor. 6. n sends NEIGHBOR SET REQUEST messages to the selected neighbors. Nn qeried is pdated to inclde the selected neighbors. For the non-selected new neighbors, NEIGHBOR NOTIFICATION messages are sent. Response actions at an existing node v When v receives NEIGHBOR SET REQUEST from n, v pts n into C v and re-calclates DT(C v ). Then v sends to n NEIGHBOR SET REPLY that incldes mtal neighbors of v and n on DT(C v ). When v receives NEIGHBOR NOTIFICATION from n, v incldes n into C v and re-calclates DT(C v ). Bt v does not reply to n Correctness of the new join protocol Lemma 1. Let S be a set of nodes. For any sbset C of S, let C and v C, if v is a neighbor of on DT(S), v is also a neighbor of on DT(C). Lemma 1 is proved in [15] (Lemma 3 in [15]). Lemma 2. Let n denote a new joining node, S be a set of existing nodes, and S = S {n}. Sppose that the existing distribted DT of S is correct and no other node joins, leaves, or fails. Let T be a simplex that exists on DT(C n ) at some time dring protocol exection and does not exists on DT(S ). Let x n be a node in T. Sppose that n sends a NEIGHBOR SET REQUEST to x. When n receives a NEIGHBOR SET REPLY from x, T is removed from DT(C n ). Proof. Since the existing distribted DT of S is correct, C x incldes all the neighbors of x on DT(S). After x receives NEIGHBOR SET REQUEST, C x will inclde n, and ths C x will inclde all the neighbors of x on DT(S ). Consider the space that T occpies on DT(C n ). Since T does not exist on DT(S ), the space is occpied by two or more different simplexes on DT(S ). Let T be one of those Join(z) of node ; Inpt: is the joining node, if is the only node in the system, z = NULL; otherwise z is the closest existing node to. if z NULL then Send(z, NEIGHBOR SET REQUEST) C {, z}, N, N qeried {z} else C {}, N, N qeried On s receiving NEIGHBOR SET REQUEST from w if w C then C C {w} N neighbor nodes of on DT(C ) N w {x x is a neighbor of w on DT(C )} Send(w, NEIGHBOR SET REPLY(N w)) On s receiving NEIGHBOR SET REPLY(N w ) from w C C N w Update Neighbors(C, N ) On s receiving NEIGHBOR NOTIFICATION from w if w C then C C {w} N neighbor nodes of on DT(C ) Update Neighbors(C, N ) of node N old N N neighbor nodes of on DT(C ) N new N N old simplexes that contain on DT(C ) and do not T new contain any node in N qeried N check Get Neighbors To Check(T new for all v N check do Send(v, NEIGHBOR SET REQUEST) N qeried N notify N qeried N new for all v N notify N check N check do Send(v, NEIGHBOR NOTIFICATION) Get Neighbors To Check(T new ) of node N while T new do n a node in T new N N n ) remove each simplex that contains n from T new end while Retrn N Figre 3. New join protocol at a node. simplexes that incldes both n and x. Note that there are d 1 other nodes in T, which are mtal neighbors of n and x on 6

7 DT(S ), and, by Lemma 1, on DT(C x ) as well. These d 1 nodes are inclded in the NEIGHBOR SET REPLY message from x to n. When n receives the NEIGHBOR SET REPLY message, the d 1 nodes are inclded in C n and, by Lemma 1, become neighbors of n on DT(C n ). As a reslt, T is created on DT(C n ). That means T, which overlaps with T, is removed from DT(C n ). Lemma 3. Let n denote a new joining node, S be a set of existing nodes, and S = S {n}. Sppose that the existing distribted DT of S is correct and no other node joins, leaves, or fails. Then when the new join protocol finishes, C n incldes all the neighbor nodes of n on DT(S ). Proof. Consider a neighbor v of n on DT(S ). We show that v will be inclded in C n when the join protocol finishes. At step 4 of the protocol exection loop, n has some nodes in C n and calclates DT(C n ). Sppose that v is not yet inclded in C n. Consider a straight line l from n to v. Let T be the first simplex on DT(C n ) that l crosses. Sch a simplex exists becase v is not yet a neighbor of n on DT(C n ). Note that T incldes n. Let the other nodes of T be x 1, x 2,..., x d. Since l is an edge on DT(S ), T does not exist on DT(S ). We show by contradiction that n has not received a NEIGHBOR SET REPLY message from any node in T. Sppose that n has received a NEIGHBOR SET REPLY from a node in T. Then by Lemma 2, T cannot exist on DT(C n ), which contradicts the earlier assmption that T exists becase v is not inclded in C n yet. Next we show that n will receive a NEIGH- BOR SET REPLY message from a node in T. Either T incldes a node x a in Nn qeried or T does not inclde any node in Nn qeried. In the former case, n has sent a NEIGHBOR SET REQUEST to x a and will receive a NEIGHBOR SET REPLY message from x a. In the latter case, by step 5 of the protocol exection loop, n will send a NEIGHBOR SET REQUEST to a node x b in T and then receive a NEIGHBOR SET REPLY message from x b. In each case, when n receives the NEIGHBOR SET REPLY message, T is removed from DT(C n ) by Lemma 2. Afterwards, if v is not a neighbor of n and l crosses another simplex on DT(C n ), protocol exection contines and the above process repeats. This process finishes in a finite nmber of iterations since the nmber of nodes in S is finite and the nmber of possible simplexes in S is also finite. When there is no simplex that l crosses on DT(C n ), l is an edge on DT(C n ). Therefore v is inclded in C n. The following theorem shows that or new join protocol is correct for a single join. Theorem 2. Let n denote a new joining node, S be a set of existing nodes, and S = S {n}. Sppose that the existing distribted DT of S is correct and no other node joins, leaves, or fails. Then when the new join protocol finishes, the pdated distribted DT is correct. Proof. By Lemma 3, when the join process finishes, C n will inclde all of its neighbor nodes on DT(S ). In addition, whenever n discovers a neighbor node v dring the process, n sends either NEIGHBOR SET REQUEST or NEIGHBOR NOTIFICATION message to v so that v incldes n into C v. Since the candidate sets of all existing nodes as well as the joining node are correctly pdated, the pdated distribted DT is correct by Theorem Leave and failre protocols In [1], we designed a leave protocol for nodes that leave graceflly. In the leave protocol, a leaving node sends to each of its neighbors v a notification informing v of v s new neighbors after leaves, as well as the fact that is leaving. The leave protocol is very efficient. We keep it in or new protocol site. In this section, we propose a proactive approach to address node failres. Or failre protocol is almost as efficient as the leave protocol. It is proved to be correct for a single failre. The main idea is that every node prepares a contingency plan in case it fails. That is, calclates a local DT of only s neighbors, not inclding itself. The contingency plan incldes, for each neighbor v of, new neighbor nodes of v after deleting. selects one of its neighbors, say m, and gives the contingency plan to m. m is called the monitor node of. Then m periodically probes to check whether is alive. When m detects failre of, m sends to each of s former neighbors its portion of the the contingency plan. The protocol psedocode is given in Figre 4. The failre protocol takes over one of the fnctions of the old maintenance protocol. As a reslt, the new maintenance protocol may be rn mch less freqently, redcing overall cost of the system. As will be demonstrated by experiments for a system of nodes nder chrn, the new maintenance protocol is still necessary to recover from incorrect system states reslting from concrrent event occrrences. Unlike the old maintenance protocol, probes are not dplicated in the failre protocol, since is probed only by its monitor node. Frthermore, each former neighbor of receives exactly 1 message pon s failre. On the other hand, the failre protocol has the overhead of pdating a contingency plan whenever a neighbor is added or deleted Correctness of the failre protocol The following lemmas are proved in [15] (they correspond to Lemmas 4 and 10 in [15]). Lemma 4. Let S be a set of nodes. Consider S and a sbset C of S that incldes all the neighbor nodes of on 7

8 On change in N m the neighbor in N with the least ID Calclate DT(N ) ; Note: N for all v N do N v {w w is a neighbor of v on DT(N )} Send(m, CONTINGENCY PLAN({< v, N v > v N }) On s receiving CONTINGENCY PLAN(CP v) from v Set FAILURE TIMER v to T + F ; T is crrent time, F is the period of failre probe. On s expiration of FAILURE TIMER v Send(v, PING) Set PING TIMEOUT TIMER v to T + TO ; T is crrent time, TO is the timeot vale. On s receiving PING from v if v = m then Send(v, PONG(tre)) else Send(v, PONG(false)) On s receiving PONG(flag) from v if flag = tre then Set FAILURE TIMER v to T + F ; T is crrent time, F is the period of failre probe. else Cancel FAILURE TIMER v On s expiration of PING TIMEOUT TIMER v for all w that CP v contains < w, N v w > do Send(w, FAILURE NOTIFICATION(v, N v w) C C {v} N v N neighbor nodes of on DT(C ) Cancel FAILURE TIMER v On s receiving FAILURE NOTIFICATION(v, N v ) from v C C {v} N v N neighbor nodes of on DT(C ) Figre 4. Failre protocol at a node. DT(S). If v C is a neighbor of on DT(C ), then v is also a neighbor of on DT(S). Lemma 5. Let S be a set of nodes and S = S {}. Let v be a neighbor node of on DT(S). If w is a neighbor node of v on DT(S ), then w is a neighbor node of v on DT(S) or w is a neighbor node of on DT(S). The following theorem shows that the failre protocol is correct for a single failre. Theorem 3. Let S be a set of nodes with a correct distribted DT. Sppose that a node S fails and its failre is detected by its monitor node m S, which then exectes the failre protocol. Assme that there is no other join, leave, or failre. After the failre protocol finishes, the pdated distribted DT is correct. Proof. Let S = S {}. Consider a node v S. The following case A shows that if v is not a neighbor of, then v is not affected by the failre of. Case B shows that if v is a neighbor of, v will receive enogh information from m to correctly pdate its candidate set. Case A) Sppose that v is not a neighbor of on DT(S). Consider a node w S, w v. If w is a neighbor of v on DT(S ), w is also a neighbor of v on DT(S) by Lemma 1. If w is a neighbor of v on DT(S), w is also a neighbor of v on DT(S ) by Lemma 4. Therefore the neighbors of v on DT(S) are the same as the neighbors of v on DT(S ) and v is not affected by failre of. Case B) Sppose that v is a neighbor of on DT(S). Consider a node w S, w v. If w is a neighbor of v on DT(S ), by Lemma 5, either w is already in C v or w was a neighbor of on DT(S). In the latter case, s monitor node will notify v that w is its neighbor. Therefore C v will inclde all the neighbor nodes of v on DT(S ). From cases A and B, for each node v S, C v incldes all the neighbor nodes of v on DT(S ). Therefore by Theorem 1, the pdated distribted DT is correct. 5. New maintenance protocol The last member of or DT protocol site is a new maintenance protocol. Even thogh the other protocols in the site the new join protocol, the (old) leave protocol, the new failre protocol are proved to be correct for a single join, leave, and failre, respectively, nodes may join, leave, and fail concrrently for a system nder chrn. As to be shown by experimental reslts in Figre 9, neither or protocols (withot a maintenance protocol) nor Simon et al. s algorithms can recover a correct distribted DT after system chrn. In that sense, or protocol site is incomplete withot a maintenance protocol, and so is Simon et al. s set of insertion and deletion algorithms. By Theorem 1, for a distribted DT to be correct, each node mst inclde in its neighbor set C all of its neighbor nodes on the global DT. This was one goal that or old maintenance protocol [1, 15] was designed to achieve. To that end, each node periodically qeries each of its neighbors to find any new neighbor of that is not aware of. We fond that rnning the maintenance protocol freqently reqires a large commnication cost. Note that the goal of a maintenance protocol is similar to that of a join protocol, namely, finding new neighbors. Therefore, we se the same techniqe as in the case of or new join protocol. That is, we redce commnication cost of or maintenance protocol by eliminating messages with redndant information. 8

9 Instead of qerying all neighbors, a node qeries only one node for each simplex that incldes. Since a neighbor node may be inclded in mltiple simplexes, the nmber of qeried neighbors is mch less than that of all neighbors. Another goal of the old maintenance protocol was failre detection and recovery. In the old maintenance protocol, probing a node was carried ot by all neighbors of. In or new set of protocols, the new failre protocol takes over the task of failre detection and recovery, where a node is probed by only one of its neighbor nodes. Ths the overall cost of or new maintenance and failre protocols is mch less than the cost of the old maintenance protocol. Althogh failre recovery is not a major goal of the new maintenance protocol, if a failre is detected by a message timeot, this information is propagated via DELETE messages. This may be necessary in case of concrrent failres. DELETE messages are propagated sing the GRPB (greedy reverse-path broadcast) protocol in [1]. The maintenance protocol psedocode (inclding GRPB) is given in Figre 5. Actions for receiving a NEIGH- BOR SET REQUEST message and the fnctions of Update Neighbors and Get Neighbors To Check are the same as in Figre Accracy metric for a system nder chrn We define an accracy metric as in [1], which we will se for experiments for a system of nodes nder chrn. We consider a node to be in-system from when it finishes joining to when it starts leaving. Let DDT S be a distribted DT of a set of in-system nodes S. (Note that some nodes may be in the process of joining or leaving and not inclded.) Let N correct (DDT S ) be the nmber of correct neighbor entries of all nodes and N wrong (DDT S ) be the nmber of wrong neighbor entries of all nodes on DDT S. A neighbor entry v of a node is correct when v is a neighbor of on the global DT (namely, DT(S)), and wrong when and v are not neighbors on the global DT. Let N(DT(S)) be the nmber of edges on DT(S). Note that edges on a global DT are ndirectional and ths are conted twice when compared with neighbor entries. The accracy of DDT S is defined as follows: accracy(ddt S ) = N correct(ddt S ) N wrong (DDT S ). 2 N(DT(S)) Observation 1. The accracy of a distribted DT is 1 if and only if the distribted DT is correct. Proof. (if) If the distribted DT is correct, N correct (DDT S ) = 2 N(DT(S)) and N wrong (DDT S ) = 0, reslting in accracy of 1. (only if) When accracy is 1, we have N correct (DDT S ) N wrong (DDT S ) = 2 N(DT(S)). Since N wrong (DDT S ) 0, we get N correct (DDT S ) On s expiration of PERIOD TIMER N qeried T simplexes that contain on DT(N {}) N check Get Neighbors To Check(T ) for all v N check do Send(v, NEIGHBOR SET REQUEST) Set NS TIMEOUT TIMER v to T + TO ; T is crrent time, TO is the timeot vale. Set PERIOD TIMER to T + P ; T is crrent time, P is the period of failre probe. On s receiving NEIGHBOR SET REPLY(N v ) from v C C N v Update Neighbors(C, N ) Cancel NS TIMEOUT TIMER v On s expiration of NS TIMEOUT TIMER v C C {v} Update Neighbors(C, N ) for all w N do Send(w, DELETE(v, )) On s receiving DELETE(v, w) from x if v C then C C {v} Update Neighbors(C, N ) for all y N, Dist(y,w) > Dist(, w) do N y nodes that share a simplex with both and y on DT(C ) if is the closest node to w in N y then Send(y, DELETE(v, w)) Figre 5. New maintenance protocol at a node. 2 N(DT(S)). Also, N correct (DDT S ) 2 N(DT(S)). It then follows that N correct (DDT S ) = 2 N(DT(S)) and N wrong (DDT S ) = 0. That means the distribted DT is correct. To demonstrate effectiveness of the new maintenance protocol, we designed an experiment for a system with an initial ring configration. The system begins with a barely connected graph of 100 nodes, in which each node initially knows only one other node. That is, node p i, 1 i 99, initially has only p i 1 in its candidate set and its neighbor set; node p 0 knows p 99. Figre 6 shows change in accracy of the distribted DT as the new maintenance protocol rns. The new maintenance protocol achieved a correct distribted DT within a few ronds of protocol exection. 9

10 Accracy D D 4D 5D Rond Figre 6. Accracy of the maintenance protocol for a system with an initial ring configration. Nmber of messages old join protocol 2000 Simon et al. insertion algorithm new join protocol Dimensionality Figre 7. Costs of join protocols for 100 serial joins. 7. Experimental reslts 7.1. Join protocols Figre 7 shows commnication costs of join protocols. Each crve shows the nmber of messages for 100 serial joins, increasing the system size from 200 nodes to 300 nodes, for different dimensionalities. Or new join protocol has mch less cost than or old join protocol, and is slightly better than Simon et al. s distribted algorithm Failre protocol Figre 8 compares commnication costs of or failre protocol and Simon et al. s entity deletion algorithm. The nmber of messages sed to recover from 100 serial failres from 300 initial nodes is measred. Both or failre protocol and Simon et al. s deletion algorithm se the same probing period of 10 seconds. Or failre protocol is mch more efficient Nmber of messages or failre protocol (10) Simon et al. deletion algorithm (10) Dimensionality Figre 8. Costs of failre protocols for 100 serial failres. Accracy new join and failre protocols (10) Simon et al. algorithms (10) Time Figre 9. Accracy withot a maintenance protocol nder system chrn (join and fail). than Simon et al. s entity deletion algorithm Maintenance protocols Figre 9 shows accracy of or protocols withot a maintenance protocol and Simon et al. s algorithms nder system chrn. From a correct distribted DT of 400 initial nodes in 3D, 100 concrrent joins and 100 concrrent failres occr from time 10 to 110 second, with an average inter-arrival time of 1 second, respectively. 7 In both or failre protocol and Simon et al. s entity deletion algorithm, nodes are probed every 10 seconds. The accracy of the distribted DT is measred every 10 seconds. Both or new join and failre protocols and Simon et al. s entity insertion and deletion algorithms cannot flly recover after system chrn, reslting in an incorrect distribted DT. 7 By Little s Law, for a system size of 400 nodes, the average lifetime of a node is 400 seconds. For P2P file sharing systems, this is a very high chrn rate [16]. 10

11 Accracy 0.97 Accracy old maintenance protocol (10) new maintenance protocol (10) failre (10) and new maintenance (30) protocols Time Figre 10. Accracy of the old and new maintenance protocols nder system chrn (join and fail) old protocol site new protocol site Time Figre 12. Accracy of the old and new protocol sites nder system chrn (join, leave, and fail). Nmber of messages 1.8e e e e+06 1e old maintenance protocol (10) new maintenance protocol (10) failre (10) and new maintenance (30) protocols Dimensionality Figre 11. Costs of the old and new maintenance protocols nder system chrn (join and fail). Figre 10 compares the accracy of or protocols inclding the maintenance protocols in the same scenario as in Figre 9. The old maintenance protocol is rn every 10 seconds. The new maintenance protocol is rn every 10 seconds when it was rn alone, and every 30 seconds when it was rn along with the failre protocol which ses a probing period of 10 seconds. After system chrn stops at time 110 second, accracy converges to 100% by the maintenance protocols. The new maintenance protocol alone shows slightly lower accracy than the old maintenance protocol. However, together with the failre protocol, the new maintenance protocol shows similar accracy to that of the old maintenance protocol alone. The new maintenance protocol also took a longer time to converge to a correct distribted DT. Figre 11 shows the commnication costs in the above experiment. As expected, the new maintenance protocol has several times less commnication cost compared to the old maintenance protocol. Efficiency is frther improved when the new maintenance protocol is combined with the failre protocol, since the new maintenance protocol is rn less freqently. Figre 12 compares the accracy of or old and new protocol sites nder system chrn, where nodes join, leave, and fail concrrently. The scenario is similar to that of the previos experiment, except that nodes either graceflly leave or fail instead of all failing. From a correct distribted DT of 400 initial nodes, 100 joins, 50 leaves, and 50 failres occr from time 10 to 110. The average inter-arrival time of joins is 1 second and those of leaves and failres are 2 seconds, respectively. The old maintenance protocol is rn every 10 seconds. The new maintenance protocol is rn every 30 seconds along with the failre protocol which ses a probing period of 10 seconds. The new protocol site maintains abot the same accracy as the old protocol site. In the end, the new protocol site took a slightly longer time to converge to a correct distribted DT, becase it ses a longer period (30 seconds instead of 10 seconds). Figre 13 shows the commnication costs of or old and new protocol sites in the same chrn experiment. The new protocol site has many times less commnication cost compared to the old protocol site. 8. Conclsions While DT has been known and sed for a long time in different fields of science and engineering, the design of protocols for constrcting and maintaining a distribted DT for a dynamic system has not received mch attention. In a previos paper [1], we investigated the design of several application-level protocols to spport DT applications on networks, as well as join, leave, and maintenance protocols to constrct and maintain a distribted DT. Or focs therein 11

12 Nmber of messages 1.8e e e e+06 1e old protocol site new protocol site Dimensionality Figre 13. Costs of the old and new protocol sites nder system chrn (join, leave, and fail). was to ensre the correctness of a distribted DT. Towards that goal, we defined a correct distribted DT, discovered a necessary and sfficient condition for a distribted DT to be correct, and defined an accracy metric. Or old join and leave protocols were proved to be correct for a single join and leave, respectively. Or old maintenance protocol was fond to converge to a correct distribted DT with 100% accracy after system chrn had stopped. Efficiency, however, was not or primary protocol design goal in [1]. In this paper, we design a new site of DT protocols that are correct, accrate, and efficient. The new protocols in this paper are vastly more efficient as a reslt of two new ideas in this paper. First, in the neighbor discovery process of a node, say n, it needs to hear back from jst one neighbor in each simplex that incldes n rather than from all neighbors. Frthermore, qeries as well as replies for some simplexes can be combined in a single qery-reply. We made se of this idea to greatly improve the efficiency of the new join and maintenance protocols. Second, we have added an efficient failre protocol that employs a proactive approach to failre recovery, instead of relying on the maintenance protocol which recovers from failres reactively. With addition of the failre protocol, the maintenance protocol can be rn less often and the overall system efficiency improves. Both the new join and failre protocols are proved to be correct for a single join and failre, respectively. The old leave protocol was highly efficient and it is kept in the new protocol site. Experimental reslts show that or new site of protocols maintains high accracy for systems nder chrn and each system converges to 100% accracy after chrning stopped. Acknowledgement We thank Professor Pal A. G. Sivilotti at the Ohio State University for motivating this stdy. Correspondence between the join protocol based on candidate sets and the flip algorithm was fond dring discssions with him. References [1] Dong-Yong Lee and Simon S. Lam, Protocol Design for Dynamic Delanay Trianglation, Proceedings of IEEE ICDCS 2007, Toronto, Canada, Jly [2] Gwendal Simon, Moritz Steiner, Ernst Biersack, Distribted Dynamic Delanay Trianglation in d-dimensional Spaces, Technical report, Institt Erecom, Agst [3] B. Delanay, Sr la sphère vide, Otdelenie Matematicheskikh i Estestvennykh Nak, 7: , [4] G. Voronoï, Novelles applications des paramètres contins à la théorie des formes qandratiqes, Jornal für die Reine and Angewandte Mathematik, 133:97-178, [5] Prosenjt Bose and Pat Morin, Online roting in trianglations, SIAM Jornal on Compting, 33(4): , [6] Edited by P. M. Grber and J. M. Wills, Handbook of Convex Geometry, North-Holland, [7] D. F. Watson, Compting the n-dimensional Delanay tessellation with application to Voronoi Polytopes, The Compter Jornal, 24(2): , [8] H. Edelsbrnner and N. R. Shah, Incremental topological flipping works for reglar trianglation, Algorithmica, 15: , [9] Jörg Liebeherr, Michael Nahas, Application-Layer Mlticasting With Delanay trianglation Overlays, IEEE Jornal on Selected Areas in Commnications, VOL. 20, NO. 8, October [10] Moritz Steiner, Ernst Biersack, A flly distribted peer to peer strctre based on 3D Delanay Trianglation, Proceedings of Algotel [11] Masaaki Ohnishi, Ryo Nishide, Shinichi Ueshima, Incremental Constrction of Delanay Overlaid Network for Virtal Collaborative Space, Proceedings of the third international conference on creating, connecting and collaborating throgh compting, [12] Taewon Yoo, Hyonik Lee, Jinwon Lee, Snghee Choi, Jnehwa Song, Distribted Kinetic Delanay Trianglation, CS/TR , KAIST, Korea, [13] Tommaso Melodia, Dario Pompili, Ian F. Akyildiz, A Commnication Architectre for Mobile Wireless Sensor and Actor Networks, Proceedings of IEEE SECON 2006, September [14] T. S. Egene Ng and Hi Zhang, Predicting Internet Network Distance with Coordinates-Based Approaches, Proceedings of IEEE Infocom 2002, Jne [15] Dong-Yong Lee and Simon S. Lam, Protocol design for dynamic Delanay trianglation, The Univ. of Texas at Astin, Dept. of Compter Sciences, Technical Report TR-06-48, Oct [16] S. Sario, P. K. Gmmadi, and S. D. Gribble, A measrement stdy of peer-to-peer file sharing systems, Proceedings of Mltimedia Compting and Networking, Janary