Õ(Congestion + Dilation) Hot-Potato Routing on Leveled Networks

Transcription

1 Õ(Congestion + Dilation) Hot-Potato Routing on Leveled Networks Costas Busch Rensselaer Polytechnic Institute buschc@cs.rpi.edu ABSTRACT We study packet routing problems, in which we route a set o N packets on preselected paths with congestion C and dilation D. Forstore-and-orward routing, in which nodes have buers or packets in transit, there are routing algorithms with perormance that matches the lower bound Ω(C + D). Motivated rom optical networks, we study the extreme case o hot-potato routing in which the nodes are buerless. In hot-potato routing, packets may be unable to ollow the preselected paths towards the destination nodes; thus it may take more time or packets to be routed. An interesting question is how much is the perormance o routing algorithms aected rom the absence o buers. Here, we answer this question or the general class o leveled networks, in which the nodes are partitioned into L + distinct levels. We present a randomized hot-potato routing algorithm or leveled networks, which routes the packets in Õ(C + L)time with high probability. For routing problems with dilation O(L), this bound is within polylogarithmic actors rom the lower bound Ω(C + L). Our algorithm demonstrates that the beneit rom using buers is no more than polylogarithmic; thus, hot-potato routing is an eicient way to route packets in leveled networks. Our algorithm is online, that is, routing decisions are taken at real time at each node, while packets are routed in the network. A novel characteristic o our algorithm is that during the course o routing, packets may deviate rom their preselected paths. To our knowledge, this is the irst hot-potato algorithm designed and analyzed, in terms o congestion and dilation, or arbitrary leveled networks. Categories and Subject Descriptors F.2.m [Analysis o Algorithms and Problem Complexity]: Miscellaneous Supported rom research unds o Rensselaer Polytechnic Institute. Permission to make digital or hard copies o all or part o this work or personal or classroom use is granted without ee provided that copies are not made or distributed or pro t or commercial advantage and that copies bear this notice and the ull citation on the rst page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speci c permission and/or a ee. SPAA 02, August 0-3, 2002, Winnipeg, Manitoba, Canada. Copyright 2002 ACM /02/ $5.00. General Terms Algorithms, Perormance, Theory Keywords Hot-potato routing, Leveled networks, Congestion, Dilation. INTRODUCTION. Background In a network, nodes communicate by sending packets through the nodes and the links (edges)o the network. In a packet routing problem, we are given a set o packets, where each packet has a source node and a destination node, and we route these packets to their destination nodes. A routing algorithm (routing protocol)speciies the decisions that the nodes take while they route the packets. The goal o a routing algorithm is to deliver the packets to their destinations as ast as possible. We consider hot-potato (or delection)routing algorithms, which were introduced by Baran [3]. In hot-potato routing, the nodes in the network have no buers to store packets in transit: any packet that arrives at a node, it is immediately orwarded to another node; in other words, a packet is treated like a hot potato. Hot-potato routing algorithms have been observed to work well in practice [4], and have been used in parallel machines such as the HEP multiprocessor [25], the Connection machine [3], and the Caltech Mosaic C [24], as well as in high speed communication networks [9]. Hot-potato routing algorithms are well-suited or optical networks since it is diicult to buer optical messages [, 26]. We study hot-potato routing algorithms or the wide class o leveled networks. A leveled network with depth L consists rom L + levels o nodes, numbered 0 to L, such that each node belongs to exactly one level, and edges connect nodes that belong to consecutive levels, as shown in Figure. Many multiprocessor network architectures can be represented as leveled networks. Typical examples are the butterly network and the mesh network, or which speciic cases are shown in Figure. In particular, the mesh network can be viewed in our dierent ways as a leveled network, according to which corner node is level 0. Other examples o networks that can be treated as leveled networks are the shule-exchange network, multidimensional array, hypercube, andat-tree [6]. We consider leveled networks in which the nodes are synchronous; namely, time is discrete, and at each time step, 20

2 0 2 L L Butterly... Leveled Network Figure : Leveled networks Mesh a node receives packets, makes a routing decision, and then orwards the packets to the adjacent nodes. At each time step, a node is allowed to send at most one packet per link. We study a special case o many-to-one packet routing problems, in which each node is the source o at most one packet, and each node is the destination o an arbitrary number o packets. Each packet has a preselected path, rom the source node to the destination node, which the packet ollows during routing. In leveled networks, the paths are oriented rom lower levels to higher levels; that is, rom let to right on the network on the top o Figure. There are two parameters associated with the preselected paths that inluence the perormance o any routing algorithm: the congestion C, the maximum number o packets that traverse any edge in the network, and the dilation D, the maximum length o any path. Since at most one packet can traverse any edge at a time step, there is a trivial lower bound or any routing problem: Ω(C + D). In leveled networks the path length does not exceed L, and there are routing problems or which D = O(L)or which the lower bound is Ω(C + L). It is desirable to design routing algorithms with routing time that matches this lower bound. In hot-potato routing, an interesting situation occurs when two or more packets appear in the same node at the same time and all o them wish to ollow the same link; note that there could be up to C such packets. This situation represents a conlict among the packets, since only one o them can ollow the link. One o these packets will successully ollow the link, while the rest packets are sent to alternative links, since there is no buer to store these packets in the current node; we say that such packets are delected. Due to delections, a packet may not be able to ollow the links Note that at any time step at most two packets can traverse a link, one packet at each direction o the link. The packet paths are selected beore the routing begins. In this work we do not consider how these paths are selected, but how to design ast routing algorithms given the paths. o its preselected path towards the destination node. This makes the analysis o hot-potato algorithms challenging..2 Contribution We present a new hot-potato routing algorithm or leveled networks. Our algorithm is randomized and routes N packets in O((C + L)ln 9 (LN)) time steps with probability at least /LN. Note that this time is optimal within polylogarithmic actors. Our algorithm enjoys the ollowing attributes: It is an online algorithm: all routing decisions are made in the nodes during the course o routing. It is a local (distributed)algorithm: the routing decisions o the nodes depend only on their local states, and the packets they receive. The algorithm works or any leveled network, and its perormance doesn t depend on the edge degrees o the nodes. A high level description o our algorithm is as ollows. We separate the packets randomly and uniormly in O(C) disjoint sets, which we call rontier-sets, such that the congestion among packets o the same rontier-set is at most ln(ln). We route the packets o each rontier-set separately inside areas o the network that we call rontier-rames (see igure 2); there is a rontier-rame or each rontier-set. A rontier-rame contains a polylogarithmic number, in terms o L and N, o consecutive levels. The algorithm consists o a sequence o phases, the duration o each phase is also polylogarithmic. The position o a rontier-rame changes at each phase. Initially, a rontier-rame appears at the lowest level o the network, and then, at each subsequent phase, the rontier-rame shits to one level higher. Dierent rontierrames are pipelined one ater the other, without overlapping, and all rontier-rames shit simultaneously. A packet o a rontier-set is injected into the network at the right moment, when the respective rontier-rame passes through the source node. While the packet is routed to its destination node, the packet remains inside the rontierrame; that is, while the rontier-rame shits to higher levels, the packet shits together with it. The packet reaches its destination node, when its rontier-rame passes through that node. Thus, by the time when the rontier-rame reaches the highest-level in the network, all the packets o the respective rontier-set must have been delivered to their destinations. The algorithm terminates when the last rontierrame reaches the highest level in the network, at which point all the packets in the network have been delivered to their destinations. We show that this event occurs no later than time O(C + L)multiplied by polylogarithmic actors. In the analysis, we establish that due to the logarithmic congestion between packets o the same rontier-set, these packets will remain within a polylogarithmic number o levels away rom each other. This property enables the packets to remain inside the rontier-rame during the course o routing. An interesting characteristic o our algorithm is that due to delections, packets may deviate rom their preselected paths. We show in the analysis o our algorithm that this doesn t aect signiicantly the perormance o the algorithm, since when packets are inside their rontier-rames, they stay very close to their preselected paths (within polylogarithmic distance). 2

3 We would like to note that due to the big polylogarithmic actor in the time expression, our algorithm is not really practical, in the sense o direct applicability. Nevertheless, the analysis shows that it is at least possible to come close to the optimal O(C + L)bound, which might motivate searching or more practical algorithms that come to the same or even better perormance guarantee..3 Related Work Hot-potato routing algorithms have been studied or speciic network multiprocessor architectures such as the 2- dimensional mesh and torus [5, 9, 0, 2, 4], the d- dimensional mesh [5, 7], the hypercube [8, 2], and trees [2]. Meyer au der Heide and Scheideler [20] study the more general class o vertex symmetric networks. For more about multiprocessor architectures you can look at [5]. Bhatt et al. [6] study hot-potato routing on leveled networks, but or dierent routing problems than the ones we consider here. Leveled networks have been studied in the context o store-and-orward routing, by Leighton et al. [6], where they present an O(C+L+log N)randomized algorithm with constant size buers. For store-and-orward routing, there has been a lot o research or obtaining optimal O(C + D) algorithms or arbitrary networks [7, 8, 2, 22, 23]. The technique o decreasing congestion, by assigning packets to dierent sets, was irst used in [7]. In our algorithm (Section 3)packets have states with dierent priorities; this particular technique was introduced in []. To our knowledge, there is no previous work on hot-potato routing on arbitrary networks, not even on leveled networks, or the packet routing problems we consider here (routing problems analyzed in terms o congestion and dilation). Paper Outline We proceed in our paper as ollows. In Section 2, we give some necessary preliminaries. In Section 3, we present our algorithm; the time analysis o our algorithm appears in Section 4. In section 5, we conclude with a discussion. 2. PRELIMINARIES 2. Basic Formulas and De nitions We use the ollowing parameters. a = 2e3 ln(ln) m =ln 2 (LN)+5 q = w =4em 2 ln(ln)ln(p )+3m + p 0 = 2LN p = m 2 ln(ln) (amc + L)2amCLN 2 We deine the unction p(k)or any k 0, as ollows. { p0 (k =0) p(k) = p(k )( amcnp ) (k>0) We use the ollowing inequalities. For all real numbers n and k, such that n and k n, ( ) ( e k k2 + k ) n e k. () n n Some o the parameters should be integers, but we don t use the ceiling operation on them to avoid notational clutter; this doesn t aect our results signiicantly. For all real numbers n and k, such that 0 <n<andk, kn ( n) k. (2) Denote by Pr(X)the probability that event X is true. 2.2 Paths In a leveled network with depth L, wedenoteanedgeas e =(v,v 2)such that nodes v and v 2 are in respective levels l and l+, where 0 l<l; that is, edges are oriented rom lower levels to higher levels. A path is a sequence o nodes connected by edges e,e 2,...,e n (we denote a path with the edge sequence). A valid path is any path e,e 2,...,e n in which or any two consecutive edges e i = (v i,v j)and e i+ =(v k,v l ),itholdsv j = v k ; that is, the nodes in a valid path are in consecutive levels o the network starting rom a lower level and ending at a higher level. A subpath o a path is any subsequence o consecutive edges o the path. Note that any subpath o a valid path is also a valid path. The length o a path is the number o its edges. For any packet π, thepreselected path o the packet is a valid path (v,v 2), (v 2,v 3),...,(v n,v n), where v is the source node o π and v n is its destination node. For convenience, we assume that the path o a packet is stored in the header o a packet in the orm o a list o edges which we reer to as the path list. 2.3 Routing Initially, a packet waits in the source node until it is injected into the network. Ater the packet is injected we say that the packet is active. An active packet ollows the links o its preselected path until the packet reaches its destination node (where the packet is absorbed). When a packet ollows a link then we remove that link rom the packet s path list. Thus, at any time step, the path list o a packet consists rom the subpath (o its preselected path), starting rom the current node, at which the packet resides, and ending at the destination node. We will reer to this subpath as the current path o the packet. Clearly, at time 0 the current path o a packet is its preselected path. I a packet π moves rom a lower level to a higher level then we say that the packet moves orward, otherwise that the packet moves backward. Wesaythattwoormorepackets meet i they appear in the same node at the same time step. We say that a packet is injected in isolation i at the time step it is injected into the network it doesn t meet with other packets at the source node. A packet injected in isolation is guaranteed to ollow the irst link o its preselected path. As we will prove in Section 4, in our algorithm packets are always injected in isolation. Let s assume that at time t packet π is in node v at level l>0andπ wishes to ollow link e towards level l +. A conlict occurs when other packets appear at node v and time t that wish to ollow link e. Only one o these packets will ollow link e. I packet π loses in the conlict then it is sent to an alternative link e e connected to v, andwe say that packet π is delected. I link e connects node v to level l (l+)then we say that the delection is backward (delection is orward), and at time t+ packet π appears in level l (l + ). Note that when π is delected it may not be directed on the link it ollowed the previous time step. We avoid to use the term directed edge, since during routing, the edges are used in both directions. 22

4 Thus, packet π may not be able to stay on its preselected path during the course o routing. On a delection, we update the current path o a packet as ollows. Let g denote the current path o packet π at time t. Whenpacketπ is delected at time t andsentonedgee, we update the current path o packet π so that at time t + the irst link is e and the rest is g. In the case where edge e was traversed in the opposite direction by another packet σ at the previous time step t, and σ didn t ollow edge e due to a delection, then we call the delection o packet π a sae delection. Essentially, in a sae delection the edge e is transerred rom the path list o packet σ to the path list o packet π. In other words, with sae delections, edges are recycled, that is, transerred rom the path list o one packet to the path list o another packet. This property is important in our algorithm or preserving the congestion during routing, as we will show in the analysis. We show that delections can always be backwards and sae; this guarantees that current paths o packets are valid. Lemma 2.. I packets are injected in isolation, then packets are delected backwards, delections are sae, and the current paths o packets are valid. Proo. We prove the claim by induction on time t. For the basis case, we consider time step 0. All the packets that appear in the network at time step 0 must have just been injected in isolation; thus, there are no delections and the current paths o packets are trivially valid. Let s assume now that the claim holds or any time step t,where0<t <t. We will prove that the claim holds or time step t. First, we show that the current paths o packets at time step t are valid. Consider a packet π whichisinnodev at time step t. Let g denote the current path o π at time step t. From the induction hypothesis, g is valid. Furthermore, there are only three possible ways that π arrived at v: (i)packetπ is injected at time t, and thus its current path is trivially valid, or (ii)packet π moved orward at time t tonodev (without being delected at time t ), and thus the current path o π in v is a subpath o g, whichis valid, or (iii)packet π got delected at time t andmoved backwards to node v by ollowing some edge e, and thus the current path o π in v is edge e ollowed by g, whichisa valid path. Thereore, in all cases the current path o π is still valid. Next, we prove that all delections at time t are backwards and sae. Notice that at time t all current paths are valid and thus packets wish to move orward. Consider a conlict situation in a node v o level l and at time t, inwhichx> packets wish to ollow the same edge e towards level l +. At time step t, it must be that rom the x packets at least x moved orward, without being delected, rom level l to node v (since current paths are valid); let E denote the set o edges that these packets ollowed (clearly, E x ). At time step t, exactly one packet moves orward on edge e. Therestx packets have to be delected; they can be delected backwards to level l by ollowing any o the edgesosete. Thus, at time t delections are backwards and sae. 2.4 Congestion For any edge e and any time step t, theedge congestion c t e o e is deined as the number o packets that have on their current paths the edge e; we take into account active and non-active packets. The congestion C t at time t is deined as the maximum c t e taken over all the edges e o the network. Denote C = C 0,namely,C is the congestion o the preselected paths. At time t, the dilation D t is deined as the maximum length o any current path. Denote D = D 0, namely, D is the dilation o the preselected paths. From Lemma 2., we know that i packets are injected in isolation then the current paths o packets are valid. Since the length o a valid path cannot exceed L, wehaveoranytimestep t 0, D t L. The congestion C determines the maximum number o packetsthatcouldmeetatthesamenodeatthesametime, and wish to ollow the same edge. To reduce the congestion we separate the packets into ac sets S 0,S,...,S ac, which we call rontier-sets. Each packet belongs to exactly one rontier-set and this set is chosen uniormly and at random among the ac rontier-sets, beore routing begins. Clearly, N = ac i=0 S k. For any rontier-set S i,edgee, and time t, therontier-set edge congestion c t e,i is deined as the number o packets o S i that include e on their current paths; we take into account active and non-active packets. The rontier-set congestion Ci t at time t is deined as the maximum rontier-set edge congestion c t e,i, taken over all the edges e o the network. Let c e,i = c 0 e,i and C i = Ci 0, namely, c e,i and C i correspond to the rontier-set congestion o the preselected paths o packets in S i. Using a Cherno-type bound, we can show, with high probability, that the congestion o the preselected paths in all the rontier-sets is no more than ln(ln). Lemma 2.2. For all i, where0 i ac, C i ln(ln) with probability at least p Phases, Frontiers, and Target Nodes In our algorithm, time is divided into consecutive time periods called phases (starting rom phase 0). Each phase consists o m consecutive time periods called rounds (starting rom round 0). Each round is w time steps long; each phase is mw time steps long. The beginning (end)o a time period is its irst (last)time step. A rontier is a pointer that points to a level l =, as shown in Figure 2. In our algorithm, there are ac rontiers 0,,..., ac, as many as the rontier-sets. At phase 0, i = im, orall0 i ac. At the beginning o each subsequent phase, the rontier i increases by one ( i moves orward). Thus, at the beginning o phase im, rontier i points to level 0 in the network, and during the next L phases, the rontier i points to some subsequent level in the network. Note that all rontiers move orward simultaneously, and any rontier i points to some level in the network only or L + phases. The rontier-rame F i, o rontier i, consists o levels i, i,... i m +, or all 0 i ac, as shown in Figure 2. Actually, at any time step, the rontier-rame consists only o those levels that exist in the network, which depends on where i points to; or example, in Figure 2, the rontier-rames F i+2, andf i 2 are not complete. We call the levels o a rontier-rame F i the inner-levels o F i, andwenumberthemrom0tom ; thus, inner-level k corresponds to level i k, or all k, where0 k m. A rontier-rame F i shits orward (one level higher)whenever the respective rontier i moves orward. Note that all 23

5 rontier: level: inner-level: rontier-rame: i+2 shit direction i F i+2 i F i+ F i F i i F i 2 Figure 2: The rontier-rames o a leveled network with L =and m =3 rontier-rames shit orward simultaneously, and that dierent rontier-rames do not overlap. Each rontier-rame F i has a target level. The target level changes at each round o phase. At rounds 0 and, the target level o F i is its inner-level 0. At each subsequent round j o the current phase, where 2 j m, the target level shits one level backwards, such that it points to innerlevel j of i. Note that at each round, the target level shits backwards, while at each phase, the rontier-rame shits orward. The packets o rontier-set S i are associated with rontier-rame F i. At any time step, any packet π S i has a target node which is deined as ollows. Let g denote the current path o π. Ig is valid and g contains a node v in the target level o F i (that is, g goes through the target level)then v is the target node o π; otherwise, the target node o π is its destination node. 3. THE ALGORITHM We describe the algorithm in terms o the actions o some arbitrary packet π o rontier-set S i, during some particular phase. The actions o the rest packets o rontier-set S i are similar to the actions o packet π. First, we give a high level description o the algorithm. Let s assume that packet π is in its rontier-rame F i, in a round j>0o the current phase. Let s assume that during round j the current path o π is valid and the target node v o π is in the target level o F i. The goal o packet π is to reach the target node v by ollowing the current path towards v. When packet π reaches the target node v, thenpacket π enters a wait state. In the wait state, packet π waits at the target node by oscillating (moving back and orth) between the target node and an adjacent node in the immediate higher inner-level. I packet π enters the wait state on target node v, and remains in the wait state, uninterrupted rom other packets, until the end o the current round j, then packet π will wait on node v or every subsequent round until the end o the current phase. Since we consider hot-potato routing, there is a possibility that during round j packet π may ail to reach target node v, orpacketπ nay ail to remain in the wait state on node v, due to conlicts with other packets. In this case, packet π attempts again in the next round j + to reach the new target node and enter the wait state on that node; note that in round j +, the target node appears in one inner-level higher, since the target level o F i moves backwards one level. This procedure is repeated in the subsequent rounds. In the analysis we show that at some round o the current phase, packet π will eventually successully reach some target node in F i and remain in the wait state on that node. The eect o the algorithm is that the packets o rontierset S i irst attempt to wait at the lower inner-levels o rontier-rame F i, and i they ail then they attempt to wait at higher inner-levels. Since the target level moves backwards, packets that already wait at lower inner-levels will not be aected by packets that attempt to enter the wait state at higher inner-levels at subsequent rounds. Consequently, at the end o the phase, packets accumulate at the lower inner-levels o F i (near rontier i), where they wait on their target nodes, and the higher (at least the last three)inner-levels o F i are ree rom packets. Thus, when at the next phase rontier-rame F i shits one level orward, all the packets o S i appear inside the new position o F i; essentially, the packets o set S i have shited with F i (even though the packets haven t really moved). Every time that rontier-rame F i shits, the packets o rontier-set S i shit together with F i, and the packets are getting closer to their destinations. In our algorithm, in order to guarantee that packet π reaches its target level, there are two more packet states: normal and excited. Originally, packet π is in the normal state and ollows its current path to the target node. Frequently though, packet π changes its state and becomes an excited packet, which has the highest priority. The excited packet π has very good chances to reach its target node, uninterrupted rom other packets. We now give the detailed description o the algorithm or packet π. Packet Injection. Packet π is injected into the network at the beginning o the phase in which its source node is in inner-level m o rontier-rame F i;thatis,π is injected at the last inner-level o F i. In the extreme case, where packet π is unable to be injected at that particular time, because all links connected to the source node are occupied by other packets, then packet π will be injected at any subsequent time step in which there is an available link rom the source node. Packet State. The active packet π has a state which is one o the ollowing: normal, excited, wait. States have priorities and the order o priorities rom higher to lower is: excited, normal, wait. In conlicts, packets with higher priority win over packets with lower priority. Conlicts among packets o the same priority are resolved arbitrarily. I at any time step a packet reaches its destination node then the packet is absorbed. The state o packet π changesasollows. Normal state. When packet π is injected into the network, its priority is set to normal. Inthenormal state, packet π ollows the current path towards its target node. Excited state. At any time step t, thenormal packet π attempts to change its priority, and becomes an excited packet with probability q. Intheexcited state, packet π ollows the current path towards its target node. I at any time step, the excited packet is delected, then it becomes a normal packet. At the end o each round, the excited packet π becomes a normal packet. In the analysis we show that this extreme case doesn t occur (with high probability). 24

6 Wait state. At any time step t that the excited or normal packet π reaches its target node v, packetπ enters the wait state. Let e =(v,v)be the last link that π traversed when it reached the target node v. In the subsequent time steps, packet π remainsinthe wait state by oscillating on edge e, that is, packet π moves back and orth between nodes v and v. I packet π is not delected until the end o the current round, while it is in the wait state on node v, thenπ remainsinthewait state on node v or all subsequent rounds, until the end o the current phase. I however, at any time step the wait packet π is delected rom other packets, then it becomes normal. Attheend o each phase, the wait packet π becomes a normal packet. 4. ANALYSIS OF THE ALGORITHM In the analysis o our algorithm, we determine how much time it is needed until all packets are absorbed. Since our algorithm works in phases, we will prove that during each phase our algorithm satisies some speciic correctness conditions which guarantee the ast delivery o the packets to their destinations. These correctness conditions are stated in the orm o invariants which are preserved rom one phase to the next, with high probability. In particular, given N packets, we will prove that the ollowing invariants hold at the end o each phase k 0, with probability at least p(k). I k a : Up to the end o phase k, packets are injected in isolation. I k b : Up to the end o phase k, packets are delected backwards, delections are sae, and the current paths o packets are valid. I k c : Uptotheendophasek, active packets with rontierrame F i are always inside F i. I k d : Uptotheendophasek, packets o dierent rontiersets do not meet. Ie k : At any time step t up to the end o phase k, Ci t ln(ln). I k : At the end o phase k, all active packets with rontierrame F i appear in an inner-level l m 4. The most important invariant is Ic k, which states that the packetsorontier-sets i remain inside the rontier-rame F i or the duration o routing. Eventually all the packets o rontier-set S i will be absorbed while rontier-rame F i passes through the respective destination nodes. The rest o the invariants are important in order to prove the truth o invariant Ic k. Invariant I k is the most diicult to prove. It states that at the end o each phase, the last three innerlevels o rontier-rame F i are empty rom packets; thus, at the next phase, when F i shits one level orward, all the packets o F i appear again inside F i; essentially, this means that the packets o rontier-set S i shit together with F i. We prove the truth o all the invariants by induction on the phase k. Note that the oscillation o packet π on edge e can be thought o as a sae delection, in which the edge e remains in the path list o π, while π oscillates on e. Basis case (k =0). According to the algorithm, in phase 0, no packets are injected into the network, so invariants Ia, 0 Ib 0, Ic 0, Id 0 and I 0, are trivially true. From Lemma 2.2, invariant Ie 0 is also true with probability at least p 0. Induction hypothesis (k < n). Assume invariants Ia n,...,i n hold with probability at least p(n ). Induction step (k = n). We will prove that the invariants Ia n,...,i n are true with probability at least p(n). In Sections 4., and 4.2, we prove the truth o invariants Ia n,...,i n, given that the invariants Ia n,...,i n hold deterministically. In section 4.3, we determine the probability o the induction step, by including the probability p(n )o the induction hypothesis; this completes the proo o the induction step. In section 4.4, we determine the total time o our algorithm. 4. Proo o Invariants I n a, I n b, I n c, I n d, and I n e Consider the packets o rontier-set S i, or some particular i, where0 i ac. Lemma 4.. At the beginning o phase n, any active packet π with rontier-rame F i, which was active at the previous phase, appears in an inner-level l m 2. Proo. Packet π was active at the end o phase n. Thereore, rom the truth o invariant I n,attheendo phase n, packet π is in an inner-level l n o the rame F i such that l n m 4. Since at the beginning o phase n rontier-rame F i shits to one higher level, we have that l n corresponds to inner-level l n in phase n, such that l n m 3. Since packet π could have been delected, or move orward at the end o phase n, we have that packet π appears in the beginning o phase n in inner-level l such that l m 2. Proposition 4.2. I n a is true. Proo. From Lemma 4., at the beginning o phase n there are no active packets, which were active at the previous phase, in inner-level m o rontier-rame F i. From the description o the algorithm, we have that packets are injected only at the beginning o phase n, and at inner-level m o rontier-rame F i. Recall that we consider routing problems in which at most one packet is injected at any node. Thereore, packets are injected in isolation at the beginning o phase n. Thus, since Ia n holds, Ia n is true. From Invariant I n b, Proposition 4.2, and Lemma 2., we obtain: Proposition 4.3. I n b is true. From Lemma 4., and the proo o Proposition 4.2, we obtain: Lemma 4.4. At the beginning o phase n, any active packet π with rontier-rame F i appears in an inner-level l m. Lemma 4.5. At any time step during phase n, any active packet π with rontier i appears in some level l o the network with l i. 25

7 Proo. From Lemma 4.4, at the beginning o phase n, the packet π with rontier i appears in a level l with l i. During phase n, the rightmost target node (considering all rounds)o packet π is in level i. From the description o the algorithm, and rom Proposition 4.3, packet π never moves to a level higher than i during phase n. Thereore, l i. Lemma 4.6. During phase n, any active packet with rontier i, appears in a level higher than i+. Proo. Assume or contradiction that there is a packet π with rontier i that appears in level i+ or lower during phase n. From Lemma 4.4, at the beginning o phase n, packet π appears in a level higher than i+. In order or packet π to appear in a subsequent time step in level i+ or lower, it must be that packet π got delected at time t rom some node v o level i+ +toanodev o level i+, and thus, it ollowed the edge e =(v,v). Let s assume, without loss o generality, that packet π is the irst packet delected rom level i+ +tolevel i+; thus,t is the earliest time that such a delection occurs. Note that t cannot be the irst time step o phase n, since rom the proo o Proposition 4.2, at that time, all packets at level i+ are newly injected packets, which move orward. It must be that packet π got delected because o conlicts with other packets that appear in node v at time t. From Proposition 4.3, we have that delections are sae, and thus thereisapacketσ which appears in node v at time t, such that at the previous time step packet σ traversed edge e, and then the edge e was transerred rom the path list o σ to the path list o π. From Lemma 4.5, packet σ must be at the beginning o phase n in a level higher than i+ (packet σ must have rontier k,wherek i). Furthermore, packet σ must have been delected rom level i+ + to level i+ at a time step earlier than t, which violates the assumption that π is the irst packet delected rom level i+ + to level i+. A contradiction. Combining Lemmas 4.4, 4.5 and 4.6 we obtain: Lemma 4.7. During phase n, all active packets with rontier-rame F i remain inside F i. From Lemma 4.7 and invariant Ic n, we obtain: Proposition 4.8. I n c is true. Since any two rontier-rames do not overlap, Proposition 4.8 and invariant I n d, imply together that packets with dierent rontiers do not meet during phase n. Proposition 4.9. I n d is true. We continue with the proo o invariant Ie n. First, we show that during phase n the rontier-set congestion doesn t increase. Lemma 4.0. At any time step t during phase n, Ci t ln(ln), or all 0 i ac. Note that packet π cannot possibly be in the wait state and oscillate on e, sincev cannot be a target node or π. Proo. Consider any edge e =(v,v 2)in the network and a rontier-set S i. Let c e,i denote the rontier-set edge congestion o e at the end o phase n. Write c t e,i = c e,i + k. From Proposition 4.9, during phase n, packetso rontier-set S i meet only with packets o the same rontierset. Furthermore, rom Proposition 4.3, delections are sae. Thus, every time t t, apacketπ o rontier-set S i traverses edge e =(v,v 2)rom node v 2 to v (delection), another packet σ o rontier-set S i must have traversed edge e at the previous time step rom v to v 2,andedgeeis transerred rom the path list o packet σ to the path list o packet π. Thereore, packet π increases k by one, while packet σ decreases k by one. Thus, k does not increase more than 0, which implies that c t e,i doesn t increase more than c e,i. Subsequently, Ci t doesn t increase more than C i,wherec i denotes the rontier-set congestion o S i at the end o phase n. From Invariant Ie n we have that C i ln(ln). Thereore, Ci t ln(ln), as needed. From invariant Ie n, and Lemma 4.0, we obtain: Proposition 4.. I n e is true. 4.2 Proo o Invariant I n We consider time steps only in phase n. We also consider the packets o a particular rontier-set S i. or some 0 i ac. From Proposition 4.8, we know that during phase n, the active packets o rontier-set S i are inside rontier-rame F i. From Proposition 4.3 the current paths o packets are valid, and thus the target nodes o these packets are inside rontier-rame F i. First we consider round 0 o phase n. Let A denote the set o active packets with destinations inside rontier-rame F i. From the description o the algorithm, we have that during round 0 the target nodes or these packets is also their destination nodes. We prove that all packets in A will be absorbed in round 0 with high probability. We consider the ollowing events. X π: Packet π A is absorbed in round 0. X: AllpacketsoA are absorbed in round 0. We will ind a lower bound on probability Pr(X π)and then we will use this bound to determine a lower bound on probability Pr(X). We need to show irst that excited packets have a good chance to reach their target nodes (destination nodes). This will help us to estimate the probability that packets reach their target nodes (no matter their state), since packets become excited with probability q. Lemma 4.2. For any particular time step t, and any particular edge e =(v 0,v ) inside F i, the event that node v 0 does not contain any excited packets that wish to traverse e, occurs with probability at least ( mq) ln(ln). Proo. Let l be the inner-level o node v 0. Let π be an excited packet that appears at node v 0 and at time t and wishes to ollow e. Packetπ must have become excited with probability p, at some node v in a inner-level l l o F i, and at time t = t (l l). Packet π has exactly one Note that this act holds even when packet π is in the wait state and oscillates on e, since in that case e remains to π. 26

8 chance to become excited at node v so that it appears at node v 0 at time t; according to the algorithm, this chance is given to π at time t with probability q. Since there are at most m inner-levels like l, the probability that packet π gets excited in any o these levels and then appears at node v 0 and time t is at most mq. Thereore, the probability that π doesn t appear at node v 0 at time t in the excited state, is at least mq. From Proposition 4., there are at most ln(ln)packets similar to π that wish to traverse edge e. Thereore, none o these ln(ln)packets appear in the excited state in node v 0 at time t with probability at least ( mq) ln(ln). Let t 0 denote the irst time step o round 0. Notice that i apacketπ becomes excited beoretimestept 0 +w m, then π might be able to reach its destination node beore the endoround0. Lemma 4.3. I a packet π becomes excited at any particular time t, wheret 0 t t 0 + w m, thenπ will reach its destination node with probability at least /2e. Proo. Let s assume that packet π is in node v at time t and wishes to ollow the edge e =(v 0,v )o F i.packetπ will not ollow the edge e only i it conlicts with other excited packets on edge e. From Lemma 4.2, with probability at least ( mq) ln(ln), there is no other excited packet on node v and time t, that wishes to ollow e; thuspacketπ will successully ollow edge e with at least that probability. Since there are less than m edges on the path o π towards the destination node, starting rom node v 0,theexcited packet π will successully ollow all these edges and reach the destination node with probability at least ( mq) m ln(ln). From the deinition o q, and Equation we obtain, ( mq) m ln(ln) ( m ln(ln) ) m ln(ln) 2e. Since a packet becomes excited with probability q, we obtain rom Lemma 4.3: Lemma 4.4. At any particular time step t, wheret 0 t t 0 + w m, the event that a normal packet changes its state and becomes an excited packet, which then reaches its destination node, occurs with probability at least q/2e. We are now ready to estimate the probability Pr(X π). Lemma 4.5. For any packet π A, Pr(X π) p. Xπ Proo. Let denote the event in which packet π doesn t reach its destination node at the end o round 0; clearly, Pr( X π) = Pr(X π). During the time period [t 0,t 0 + w m ], packet π traverses w m links. From these link traversals, at most (w m )/2+m are towards the destination node, and at least (w m )/2 m are taking π urther rom the destination node, due to delections rom other packets. Immediately ater each delection, packet π is always in the normal state. Thereore, rom Lemma 4.4, packet π has a chance ater each delection to become excited and then reach its destination at a subsequent time step with probability at least q/2e, and ails to do so with probability at most q/2e. Since there are at least (w m )/2 m delections, packet π, will ail to take its chance ater each delection to become excited and reach its destination with probability at most Pr( X π) ( q/2e) (w m )/2 m. From the deinition o q and w, and Equation, we obtain Pr( X π) Subsequently, Pr(X π) p. We continue to prove: ( ) 2em 2 ln(ln)ln(p ) 2em 2 ln(ln) e = p. ln(p ) Lemma 4.6. Pr(X) A p. Proo. Let π A. FromLemma4.5,wehavethat packet π is not absorbed in round 0 with probability Pr( X π) p.let X denote the event that some packet π A is not absorbed in round 0; clearly, Pr(X) = Pr( X). We have, Pr( X) = π A Pr( X π) A p. Subsequently, Pr(X) A p. Now we consider round j o phase n, wherem j. Denote B j the set o active packets in round j, whicharenot in the wait state at the beginning o round j. We consider the ollowing events. Y j π : Packet π B j, reaches its target node in round j. Y j : All packets o B j reach their target nodes in round j. Assume or the moment that event X is given, and that or each event Y j+,eventy j is given, where m j 2. With proos very similar to the proos o Lemmas 4.5 and 4.6 we obtain the ollowing two lemmas. Lemma 4.7. For any packet π B j, Pr(Y j π ) p, or any j, m j. Lemma 4.8. Pr(Y j ) B j p, or any j, m j. We now prove that the number o packets at each round which are not in the wait state, is decreasing. The reason is that at each round a big raction o the packets become wait packets. Lemma 4.9. At the end o any particular round j, where m j, the event that at least B j / ln(ln) packets o set B j are in the wait state, occurs with probability at least B j p. Proo. In round j, the target nodes o the packets in B j are in inner-level j o rontier-rame F i.lete denote the set o edges between inner-levels j 2andj, rom which the packets o B j approach their target nodes. From Lemma 4.8, during round j every packet o B j reaches its target node with probability at least B j p. According to the algorithm, when a packet reaches its target node then it becomes a wait packet on the edge o E which is connected to its target node. Notice that a wait packet π can be delected only by another packet σ (excited or normal)which 27

9 becomes a wait packet on the same edge with π. Thereore, with probability at least B j p, at least one packet will become a wait packet on each edge o E. From Proposition 4., the maximum capacity o any edge in rontier-rame F i is ln(ln). Thereore, E B j / ln(ln). Thus, at least B j / ln(ln)packets enter the wait state, with probability at least B j p. Now, we consider all rounds j with m j 2. We only assume now that the event X is given. Lemma For all j, where m j 2, B j B j ( / ln(ln)) with probability at least ( S i p ) m 2. Proo. Consider some round j, wherem j 2. From Lemma 4.9, we have that at least B j / ln(ln) packets o B j are wait packets at the end o round j, with probability at least B j p. Thereore, the packets which are not in a wait state, and continue in a non-wait state to the next round, are at most B j B j / ln(ln). Thereore, B j B j ( / ln(ln)) with probability at least B j p. Considering now all the rounds j, where m j 2, we have that B j B j ( / ln(ln)) with probability at least m j=2 ( Bj p).since Bj Si, or all j where m j 2, this probability is at least ( S i p ) m 2. Next, we show that higher inner-levels do not contain any active packets. Lemma 4.2. For all j such that m j ln 2 (LN)+2, B j =0with probability at least ( S i p ) m 2. Proo. From Lemma 4.20, or all j where m j 2, B j B j ( / ln(ln)), with probability at least ( S i p ) m 2. Thereore, with at least that probability, B j B ( / ln(ln)) j. Since B N, B j N( / ln(ln)) j. From Equation, when j ln 2 (LN)+ 2, then ( / ln(ln)) j < /N, and thus B j <, which implies that B j = 0. Thereore, B j = 0 or all j such that m j ln 2 (LN)+ 2, with probability at least ( S i p ) m 2. Now we consider all rounds o phase n, including round 0, without making any assumptions (without assuming that X is given). Lemma At the end o phase n, there are no active packets in any inner-level j o rontier-rame F i,wherem j ln 2 (LN)+2, with probability at least ( S i p ) m. Proo. From Lemma 4.6, Pr(X) A p. Multiplying this probability with the probability o Lemma 4.2, and since A S i, wehavethattherearenoactive packets in in any o these inner-levels, with probability at least ( S i p ) m 2 ( A p ) ( S i p ) m ( S i p ) m. Now, we consider all rontier-rames F i,where0 i ac. Lemma At the end o phase n, there are no active packets in all inner-levels j o all rontier-rames F i,where m j ln 2 (LN)+2, and0 i ac, with probability at least amcnp. Proo. From Lemma 4.22, the claim holds or any particular rontier-rame F i, with probability at least ( S i p ) m. Considering all rontier-rames we have that the claim holds with probability at least ac i=0 ( S i p ) m. Since S i N, or all i where 0 i ac, we have that this probability is at least ( Np ) amc. From Equation 2, this probability is at least amcnp. From Lemma 4.23, and invariant I d, we obtain: Proposition Invariant I n at least amcnp. 4.3 All Invariants is true with probability We have shown in Propositions 4.2, 4.3, 4.3, 4.9, 4., and 4.24, that when the invariants Ia n,...,i n, hold deterministically, then invariants Ia n,...,ie n are deterministically true and, invariant I n is true with probability at least p mn. Since rom the induction hypothesis, invariants Ia n,..., I n hold with probability at least p(n ), we have that the invariants Ia n,...,ie n hold with probability at least p(n )and invariant I n holds with probability at least p(n )( amcnp )=p(n). Thus, all invariants Ia n,...,i n hold with probability at least p(n). This completes the proo o the induction step. 4.4 Total Time Invariant Ic k implies that any packet π remains inside its respective rontier-rame rom the moment that the packet is injected into the network until it is absorbed in its destination node. A rontier-rame traverses the network rom a lower level to a higher level, and at some point o time leaves the network completely. By that time, all the packets o the rontier-rame must have been absorbed. Thereore, by the time when the last rontier-rame F ac leaves the network, all packets in the network have been absorbed. Frontierrame F ac leaves the network at the beginning o phase (ac )m + L + m = amc + L, which occurs at time step (amc + L)mw = am 2 wc + mwl. Since invariant Ic amc+l holds with probability at least p(amc + L), we obtain: Proposition By time am 2 wc + mwl, all packets will be absorbed with probability at least p(amc + L). We are now ready to prove our main theorem. Theorem By time O((C +L)ln 9 (LN)), all packets are absorbed with probability at least /LN. Proo. From Proposition 4.25 and rom the deinitions o a, m, and w, we have that all packets will be absorbed by time O((C +L)ln 9 (LN)) with probability at least p(amc + L). By unolding the unction p(amc + L),and rom Equation 2, we have that this probability is at least p(amc + L) =p 0( amcnp ) amc+l ) amc+l amcn = p 0 ( (amc + L)2amCLN 2 ( p 0 ) ( = ) 2 2LN 2LN LN. 28

10 5. DISCUSSION We presented an Õ(C + L)hot-potato routing algorithm or leveled networks. An immediate application o our algorithm is on routing in multiprocessor networks which are represented as leveled networks. For example, in [6] the authors describe how to obtain optimal paths or the n n mesh with congestion and dilation n, and our algorithm can be used to route these packets with time close to the optimal up to polylogarithmic actors. It is interesting to extend our work or arbitrary network topologies. Acknowledgments We thank the reviewers o this conerence or their comments and corrections. We are indebted to Marios Mavronicolas or his thorough comments and suggestions. We also thank Malik Magdon-Ismail and Roger Wattenhoer or helpul discussions. 6. REFERENCES [] A. S. Acampora and S. I. A. Shah. Multihop lightwave networks: a comparison o store-and-orward and hot-potato routing. In Proc. IEEE INFOCOM, pages 0 9, 99. [2] S. Alstrup, J. Holm, K. de Lichtenberg, and M. Thorup. Direct routing on trees. In Proceedings o the Ninth Annual ACM-SIAM Symposium on Discrete Algorithms, pages , 999. [3] P. Baran. On distributed communications networks. IEEE Transactions on Communications, pages 9, 964. [4] C. Bartzis, I. Caragiannis, C. Kaklamanis, and I. Vergados. Experimental evaluation o hot-potato routing algorithms on 2-dimensional processor arrays. In EUROPAR: Parallel Processing, 6th International EURO-PAR Conerence, pages LNCS, [5] A. Ben-Dor, S. Halevi, and A. Schuster. Potential unction analysis o greedy hot-potato routing. Theory o Computing Systems, 3():4 6, Jan./Feb [6] S. N. Bhatt, G. Bilardi, G. Pucci, A. G. Ranade, A. L. Rosenberg, and E. J. Schwabe. On buerless routing o variable-length message in leveled networks. In Proc. European Symposium on Algorithms, pages 49 60, 993. [7] A. Borodin, Y. Rabani, and B. Schieber. Deterministic many-to-many hot potato routing. IEEE Transactions on Parallel and Distributed Systems, 8(6): , June 997. [8] J. T. Brassil and R. L. Cruz. Bounds on maximum delay in networks with delection routing. IEEE Transactions on Parallel and Distributed Systems, 6(7): , July 995. [9] A. Broder and E. Upal. Dynamic delection routing on arrays. In Proceedings o the Twenty-Eighth Annual ACM Symposium on the Theory o Computing, pages , May 996. [0] C. Busch, M. Herlihy, and R. Wattenhoer. Hard-potato routing. In Proceedings o the 32nd Annual ACM Symposium on Theory o Computing, pages , May [] C. Busch, M. Herlihy, and R. Wattenhoer. Randomized greedy hot-potato routing. In Proceedings o the Eleventh Annual ACM-SIAM Symposium on Discrete Algorithms, pages , Jan [2] U. Feige and P. Raghavan. Exact analysis o hot-potato routing. In IEEE, editor, Proceedings o the 33rd Annual Symposium on Foundations o Computer Science, pages , Pittsburgh, PN, Oct [3] W. D. Hillis. The Connection Machine. MIT press, 985. [4] C. Kaklamanis, D. Krizanc, and S. Rao. Hot-potato routing on processor arrays. In Proceedings o the 5th Annual ACM Symposium on Parallel Algorithms and Architectures, pages , Velen, Germany, June 30 July 2, 993. [5] F. T. Leighton. Introduction to Parallel Algorithms and Architectures: Arrays - Trees - Hypercubes. Morgan Kaumann, San Mateo, 992. [6] F. T. Leighton, B. M. Maggs, A. G. Ranade, and S. B. Rao. Randomized routing and sorting on ixed-connection networks. J. Algorithms, 7():57 205, 994. [7] F. T. Leighton, B. M. Maggs, and S. B. Rao. Packet routing and job-scheduling in O(congestion + dilation) steps. Combinatorica, 4:67 86, 994. [8] T. Leighton, B. Maggs, and A. W. Richa. Fast algorithms or inding O(congestion + dilation)packet routing schedules. Combinatorica, 9:375 40, 999. [9] N. F. Maxemchuk. Comparison o delection and store and orward techniuques in the Manhattan street and shule exchange networks. In Proc. IEEE INFOCOM, pages , 989. [20] F. Meyer au der Heide and C. Scheideler. Routing with bounded buers and hot-potato routing in vertex-symmetric networks. In P. G. Spirakis, editor, Proceedings o the Third Annual European Symposium on Algorithms, volume 979 o LNCS, pages , Coru, Greece, Sept [2] F. Meyer au der Heide and B. Vöcking. Shortest-path routing in arbitrary networks. Journal o Algorithms, 3():05 3, Apr [22] R. Ostrovsky and Y. Rabani. Universal O(congestion+dilation+log +ε N)local control packet switching algorithms. In Proceedings o the 29th Annual ACM Symposium on the Theory o Computing, pages , New York, May 997. [23] Y. Rabani and É. Tardos. Distributed packet switching in arbitrary networks. In Proceedings o the Twenty-Eighth Annual ACM Symposium on the Theory o Computing, pages , Philadelphia, Pennsylvania, May 996. [24] C. L. Seitz. The caltech mosaic C: An experimental, ine-grain multicomputer. In Proceedings o the 4th Symp. on Parallel Algorithms and Architectures, June 992. Keynote Speech. [25] B. Smith. Architecture and applications o the HEP multiprocessor computer system. In Proceedings o the 4th Symp. Real Time Signal Processing IV, pages SPIE, 98. [26] T. Szymanski. An analysis o hot potato routing in a iber optic packet switched hypercube. In Proc. IEEE INFOCOM, pages ,