8.1 Shortest Path Trees

Transcription

1 I tudy my Bible a I gather apple. Firt I hake the whole tree, that the ripet might fall. Then I climb the tree and hake each limb, and then each branch and then each twig, and then I look under each leaf. attributed to Martin Luther (c. 5) Life i an unfoldment, and the further we travel the more truth we can comprehend. To undertand the thing that are at our door i the bet preparation for undertanding thoe that lie beyond. attributed to Hypatia of Alexandria (c. ) by Elbert Hubbard in Little Journey to the Home of Great Teacher () Your mind will anwer mot quetion if you learn to relax and wait for the anwer. Like one of thoe thinking machine, you feed in your quetion, it back, and wait... William S. Burrough. Naked Lunch (5) The method given in thi paper require no foreight or ingenuity, and hence deerve to be called algorithm. Edward R. Moore, The Shortet Path Through a Maze (5) Shortet Path Suppoe we are given a weighted directed graph G = (V, E, w) with two pecial vertice, and we want to find the hortet path from a ource vertex to a target vertex t. That i, we want to find the directed path P tarting at and ending at t that minimize the function w(p) := w(u v). u v P For example, if I want to anwer the quetion What the fatet way to drive from my old apartment in Champaign, Illinoi to my wife old apartment in Columbu, Ohio?, I might ue a graph whoe vertice are citie, edge are road, weight are driving time, i Champaign, and t i Columbu. The graph i directed, becaue driving time along the ame road might be different Wet on Church, north on Propect, eat on I-, outh on I-5, eat on Airport Expreway, north on I-5, eat on I-, north on Grandview, eat on 5th, north on Olentangy River, eat on Dodridge, north on High, wet on Kelo, outh on Neil. Depending on traffic. We live in Urbana now.

2 . SHORTEST PATHS in different direction. (At one time, there wa a peed trap on I- jut eat of the Indiana/Ohio border, but only for eatbound traffic.). Shortet Path Tree Almot every algorithm known for computing hortet path from one vertex to another actually olve (large portion of) the following more general ingle ource hortet path or SSSP problem: Find hortet path from the ource vertex to every other vertex in the graph. Thi problem i uually olved by finding a hortet path tree rooted at that contain all the deired hortet path. It not hard to ee that if hortet path are unique, then they form a tree, becaue any ubpath of a hortet path i itelf a hortet path. If there are multiple hortet path to ome vertice, we can alway chooe one hortet path to each vertex o that the union of the path i a tree. If there are hortet path to two vertice u and v that diverge, then meet, then diverge again, we can modify one of the path without changing it length o that the two path only diverge once. b c u a d x y v Figure.. If a b c d v (olid) and a x y d u (dahed) are hortet path, then a b c d u (along the top) i alo a hortet path. Although they are both optimal panning tree, hortet-path tree and minimum panning tree are very different creature. Shortet-path tree are rooted and directed; minimum panning tree are unrooted and undirected. Shortet-path tree are mot naturally defined for directed graph; minimum panning tree are more naturally defined for undirected graph. If edge weight are ditinct, there i only one minimum panning tree, but every ource vertex induce a different hortet-path tree; moreover, it i poible for every hortet path tree to ue a different et of edge from the minimum panning tree.. Negative Edge For mot hortet-path problem, where the edge weight correpond to ditance or length or time, it i natural to aume that all edge weight are non-negative, or even poitive. However, for many application of hortet-path algorithm, it i natural to conider negative edge. For example, the weight of an edge might

3 .. Negative Edge 5 5 Figure.. A minimum panning tree and a hortet path tree of the ame undirected graph. repreent the cot of moving from one vertex to another, o negative-weight edge repreent tranition with negative cot, or equivalently, tranition that earn a profit. Negative edge are a thorn in the ide of mot hortet-path problem, becaue the preence of a negative cycle might imply that there i no hortet path. To be precie, a hortet path from to t exit if and only if there i at leat one path from to t, but there i no path from to t that touche a negative cycle. For any path from to t that touche a negative cycle, there i a horter path from to t that goe around the cycle one more time. Thu, if at leat one path from to t touche a negative cycle, there i no hortet path from to t. 5 t Figure.. There i no hortet walk from to t. In part becaue we need to conider negative wedge weight, thi chapter explicitly conider only directed graph. All of the algorithm decribed here alo work for undirected graph with eentially trivial modification, if and only if negative edge are prohibited. Correctly handling negative edge in undirected graph i coniderably more ubtle. We cannot imply replace every undirected edge with a pair of directed edge, becaue thi would tranform any negative edge into a hort negative cycle. Subpath of an undirected hortet path that contain a negative edge are not necearily hortet path; conequently, the et of all undirected hortet path from a ingle ource vertex may not define a tree, even if hortet path are unique. Technically, we hould be dicuing hortet walk here, rather than hortet path, but the abue of terminology i tandard. If can reach t, there mut be a hortet imple path from to t; it jut NP-hard to compute (when there are negative cycle). On the other hand, if there i a hortet walk from to t, that walk mut be a imple path, and therefore mut be the hortet imple path from to t. Blerg.

4 . SHORTEST PATHS u v u v u v Figure.. An undirected graph where hortet path from are unique but do not define a tree. A complete treatment of undirected graph with negative edge i beyond the cope of thi book. I will only mention, for people who want to follow up via Google, that a ingle hortet path in an undirected graph with negative edge can be computed in O(V E + V log V ) time, by a reduction to maximum weighted matching.. The Only SSSP Algorithm Jut like graph traveral and minimum panning tree, many different SSSP algorithm can be decribed a pecial cae of a ingle generic algorithm, firt propoed by Leter Ford in 5 and independently decribed by George Dantzig in 5 and again by George Minty in 5. Each vertex v in the graph tore two value, which (inductively) decribe a tentative hortet path from to v. dit(v) i the length of the tentative hortet v path, or if there i no uch path. pred(v) i the predeceor of v in the tentative hortet v path, or Null if there i no uch vertex. The predeceor pointer automatically define a tentative hortet-path tree rooted at ; thee pointer are exactly the ame a the parent pointer in our generic graph traveral algorithm. At the beginning of the algorithm, we initialize the ditance and predeceor a follow: InitSSSP(): dit() pred() Null for all vertice v dit(v) pred(v) Null During the execution of the algorithm, an edge u v i tene if dit(u)+w(u v) < dit(v). If u v i tene, the tentative hortet path v i clearly incorrect, becaue the path u v i horter. We can correct (or at leat improve) thi obviou overetimate by relaxing the edge a follow: Specifically, Dantzig howed that the hortet path problem can be phraed a a linear programming problem, and then decribed an interpretation of hi implex method in term of the original graph. Hi decription i (morally) equivalent to Ford relaxation trategy.

5 .. The Only SSSP Algorithm Relax(u v): dit(v) dit(u) + w(u v) pred(v) u Now that everything i et up, Ford generic algorithm ha a imple one-line decription: Repeatedly relax tene edge, until there are no more tene edge. FordSSSP(): InitSSSP() while there i at leat one tene edge Relax any tene edge If FordSSSP eventually terminate (becaue there are no more tene edge), then the predeceor pointer correctly define a hortet-path tree, and each value dit(v) i the actual hortet-path ditance from to v. In particular, if cannot reach v, then dit(v) =, and if any negative cycle i reachable from, then the algorithm never terminate. The correctne of Ford generic algorithm follow from the following erie of impler claim:. At any moment during the execution of the algorithm, for every vertex v, the ditance dit(v) i either or the length of a walk from to v. Thi claim can be proved by induction on the number of relaxation.. If the graph ha no negative cycle, then dit(v) i either or the length of ome imple path from to v. Specifically, if dit(v) i the length of a walk from to v that contain a directed cycle, that cycle mut have negative length. Thi claim implie that if G ha no negative cycle, the relaxation algorithm eventually halt, becaue there are only a finite number of imple path in G.. If no edge in G i tene, then for every vertex v, the ditance dit(v) i the length of the predeceor path pred(pred(v)) pred(v) v. Specifically, if v violate thi condition but it predeceor pred(v) doe not, the edge pred(v) v i tene.. If no edge in G i tene, then for every vertex v, the path of predeceor edge pred(pred(v)) pred(v) v i in fact a hortet path from to v. Specifically, if v violate thi condition but it predeceor u in ome hortet path doe not, the edge u v i tene. Thi claim alo implie that if G ha a negative cycle, then ome edge i alway tene, o the generic algorithm never halt. So far I haven t aid anything about how to find tene edge, or which tene edge to relax if there i more than one. Jut a with graph traveral, there are 5

6 . SHORTEST PATHS everal different intantiation of Ford generic relaxation algorithm. Unlike graph traveral, however, the efficiency and correctne of each earch trategy depend on the tructure of the input graph. In the ret of thi chapter, we dicu the four mot common intantiation of Ford algorithm, each of which i the bet choice for a different cla of input graph. I ll leave the remaining detail of the generic correctne proof a exercie, and intead give (more informative) correctne proof for each of thee four pecific algorithm.. Unweighted Graph: Breadth-Firt Search In the implet pecial cae of the hortet path problem, all edge have weight, and the length of a path i jut the number of edge. Thi pecial cae can be olved by a pecie of our generic graph-traveral algorithm called breadth-firt earch. Breadth-firt earch i often attributed to Edward Moore, who decribed it in 5 (a Algorithm A ) a the firt publihed method to find the hortet path through a maze. Epecially in the context of VLSI wiring and robot path planning, breadth-firt earch i ometime attributed to Chin Yang Lee, who decribed everal application of Moore Algorithm A (with proper credit to Moore) in. However, in 5, more than a decade before Moore conidered maze, Konrad Zue decribed an implementation of breadth-firt earch, a a method to count and label the component of a diconnected graph. Moore wa motivated by a weakne in Claude Shannon maze-olving robot Theeu, which Shannon deigned and contructed in 5. (Theeu ued a memoized verion of depthfirt earch, implemented uing electromechanical relay; thi wa almot certainly the firt implementation of depth-firt earch in graph.) According to Moore, When thi machine wa ued with a maze which had more than one olution, a viitor aked why it had not been built to alway find the hortet path. Shannon and I each attempted to find economical method of doing thi by machine. He found everal method uitable for analog computation, 5 and I obtained thee algorithm. 5 Analog method for computing hortet path through maze have been propoed uing ball bearing, fluid/plama flow, chemical reaction wave, chemotaxi, reitor network, electric circuit with LED, memritor network, glow dicharge in microfluidic chip, growing plant, lime mold, amoeba, ant, bee, nematode, and tourit. Konrad Zue wa one of the early pioneer of computing; he deigned and built hi firt programmable computer (later dubbed the Z) in the late from metal trip and rod in hi parent living room; the Z and it original blueprint were detroyed by a Britih air raid in. Zue 5 PhD thei decribe the very firt high-level programming language, called Plankalkül. The firt complete example of a Plankalkül program in Zue thei i an implementation of breadth-firt earch to count component, along with a peudocode explanation and an illutrated tep-by-tep trace of the algorithm execution on a diconnected graph with eight vertice. Due to the collape of the Nazi government, Zue wa unable to ubmit hi PhD thei, and Plankalkül remained unpublihed until. The firt Plankalkül compiler wa finally implemented in 5 by Joachim Hohmann.

7 .. Unweighted Graph: Breadth-Firt Search Breadth-firt earch maintain a firt-in-firt-out queue of vertice, which initially contain only the ource vertex. At each iteration, the algorithm Pull a vertex u from the front of the queue and examine each of it outgoing edge u v. Whenever the algorithm dicover an outgoing tene edge u v, it relaxe that edge and Puhe vertex v onto the queue. The algorithm end when the queue become empty. BFS(): InitSSSP() Puh() while the queue i not empty u Pull( ) for all edge u v if dit(v) > dit(u) + dit(v) dit(u) + pred(v) u Puh(v) if u v i tene relax u v Breadth-firt earch i omewhat eaier to analyze if we break it execution into phae, by introducing an imaginary token. Before we Pull any vertice, we Puh the token into the queue. The current phae end when we Pull the token out of the queue; we begin the next phae when we Puh the token into the queue again. Thu, the firt phae conit entirely of canning the ource vertex. The algorithm end when the queue contain only the token. The modified algorithm i hown in Figure.5, and Figure. how an example of thi algorithm in action. Let me emphaize that thee modification are merely a convenience for analyi; with or without the token, the algorithm Puhe and Pull vertice in the ame order, can edge in the ame order, and output exactly the ame ditance and predeceor. BFSWithToken(): InitSSSP() Puh() Puh( ) tart the firt phae while the queue contain at leat one vertex u Pull( ) if u = Puh( ) tart the next phae ele for all edge u v if dit(v) > dit(u) + if u v i tene dit(v) dit(u) + relax u v pred(v) u Puh(v) Figure.5. Breadth-firt earch with an end-of-phae token ( ); bold red line are only for analyi.

8 . SHORTEST PATHS h h h e f g e f g e f g a b c d a b c d b d a b c d c a g h h h e f g e f g e f g a b c d h a b c d f e a b c d Figure.. A complete run of breadth-firt earch in a directed graph. Vertice are pulled from the queue in the order b d c a g f e h, where i the end-of-phae token. Bold vertice are in the queue at the end of each phae. Bold edge decribe the evolving hortet path tree. Let me emphaize that in the following lemma, dit(v) i jut a variable maintained by the algorithm. While dit(v) intuitively repreent a tentative hortet-path ditance, we cannot aume (yet) that dit(v) i ever actually equal to the true hortet-path ditance from to v. Don t worry; we ll get there. Lemma.. For every integer i and every vertex v, at the end of the ith phae, either dit(v) = or dit(v) i, and v i in the queue if and only if dit(v) = i. Proof: The proof proceed by induction on i. The bae cae i = i traightforward: At the tart of the firt phae ( at the end of the zeroth phae ), the queue contain only the tart vertex and the token, and InitSSSP jut et dit() and dit(v) for all v. So fix an integer i >. The inductive hypothei implie that at the tart of the ith phae, the queue contain every vertex u with dit(u) = i, followed by the token. In other word, the queue look like thi: i i i Thu, before we Pull the token from the queue, ending the ith phae, we Pull every vertex u with dit(u) = i. For each uch vertex u, we conider every outgoing edge u v. If u v i tene, we et dit(v) dit(u) +, o that dit(v) = i, and then immediately

9 .. Unweighted Graph: Breadth-Firt Search Puh v into the queue. Thee are the only aignment to ditance label during the ith phae. Thu, by induction, during the entire ith phae, the queue contain ome vertice with ditance label i, followed by the token, followed by ome vertice with ditance label i: i i i i In particular, jut before the ith phae end, the queue contain the token, followed by ome vertice with ditance label i. i i i Moreover, vertex v appear in thi final queue if and only if dit(v) wa changed during the ith phae. Thu, at the end of the ith phae, the queue contain every vertex v with dit(v) = i. Lemma. implie that the main body of BFS aign ditance label in nondecreaing order; on the other hand, the ditance label dit(v) of each vertex v never increae. It follow that for each vertex v, the line dit(v) dit(u) + i executed at mot once, during phae dit(v). Similarly: Each predeceor pointer pred(v) i changed at mot once, during phae dit(v). Each vertex v i Puhed into the queue at mot once, during phae dit(v). Each vertex u i Pulled from the queue at mot once, during phae dit(u)+. For each edge u v, the comparion i dit(v) > dit(u) + i performed at mot once, during phae dit(u) +. Altogether, thee obervation imply that breadth-firt earch run in O(V + E) time. Intuitively, we can think of the vertice in the queue a a wavefront expanding monotonically outward from the ource vertex, paing over each vertex and edge of the graph at mot once. Thi expanding wavefront analogy wa already propoed by Chin Yang Lee in, inpired by viualization produced by hi implementation of Moore Algorithm A. Thee obervation alo imply that we can replace the condition if dit(v) > dit(u) + by the (arguably) impler tet if dit(v) =. Then ditance play the ame role a the mark maintained by other graph-traveral algorithm, which enure that each vertex i viited only once. Specifically, a vertex i marked if and only if it ditance label i finite. But we till need to prove that the final ditance label are correct! Theorem.. When BFS end, dit(v) i the length of the hortet path in G from to v, for every vertex v.

10 . SHORTEST PATHS Proof: Fix an arbitrary vertex v, and conider an arbitrary path v v v l in G, where v = and v l = v. I claim that dit(v j ) j for each index j; in particular dit(v) l. We can prove thi claim by induction on j a follow. Trivially dit(v ) = dit() =. For any index j >, the induction hypothei implie dit(v j ) j. Immediately after we Pull vertex v j from the queue, either dit(v i ) dit(v j ) + already, or we et dit(v i ) dit(v j ) +. In either cae, we have dit(v j ) dit(v j ) + j. We jut proved that dit(v) i at mot the length of an arbitrary path from to v; it follow that dit(v) i at mot the length of the hortet path from to v. A imilar induction proof implie that dit(v) i the length of the predeceor path pred(pred(v)) pred(v) v, o thi mut be the hortet path..5 Directed Acyclic Graph: Depth-Firt Search Shortet path are alo eay to compute in directed acyclic graph, even when the edge are weighted, and in particular, even when ome edge have negative weight. (We don t have to worry about negative cycle, becaue by definition, dag don t have any cycle!) Indeed, thi i a completely tandard dynamic programming algorithm. Let G be a directed graph with weighted edge, and let be the fixed tart vertex. For any vertex v, let dit(v) denote the length of the hortet path in G from to v. Thi function atifie the following imple recurrence: if v = dit(v) = min (dit(u) + w(u v)) otherwie u v In fact, thi identity hold for all directed graph, but it i only a recurrence for directed acyclic graph. If the input graph G contained a cycle, a recurive evaluation of thi function would fall into an infinite loop; however, becaue G i a dag, each recurive call viit an earlier vertex in topological order. The dependency graph for thi recurrence i the reveral of the input graph G: ubproblem dit(v) depend on dit(v) if and only if u v i an edge in G. Thu, we compute the ditance of every in O(V + E) time by performing a depth-firt earch in the reveral of G and conidering vertice in potorder. Equivalently, we can conider the vertice in the original graph G in topological order, a hown in Figure.. The reulting dynamic-programming algorithm i another example of Ford generic relaxation algorithm! To make thi connection clearer, we can move the initialization dit(v) outide the main loop and add computation of predeceor pointer, a hown in Figure.. Figure. how thi algorithm in action.

11 .5. Directed Acyclic Graph: Depth-Firt Search DagSSSP(): for all vertice v in topological order if v = dit(v) ele dit(v) for all edge u v if dit(v) > dit(u) + w(u v) dit(v) dit(u) + w(u v) if u v i tene relax u v Figure.. Computing hortet path in a dag uing dynamic programming DagSSSP(): InitSSSP() for all vertice v in topological order for all edge u v if u v i tene Relax(u v) Figure.. Computing hortet path in a dag uing Ford algorithm. (Thee are the ame algorithm.) Figure.. Computing hortet path in a dag, by relaxing incoming edge in topological order. In each iteration, bold edge indicate predeceor, and the bold vertex i about to be canned. Compare with Figure...

12 . SHORTEST PATHS DagSSSP differ from breadth-firt earch and other intance of Ford relaxation trategy in one minor repect. Whenever thee other hortet-path algorithm conider a vertex, they attempt to relax each of it outgoing edge, intuitively puhing the wavefront outward; wherea, DagSSSP attempt to relax each of the incoming edge of each vertex, intuitively pulling the wavefront forward. However, if we modify DagSSSP to relax outgoing edge intead of incoming edge, we obtain another algorithm that compute hortet path in dag in O(V + E) time and that more cloely reemble other hortet-path algorithm. PuhDagSSSP(): InitSSSP() for all vertice u in topological order for all outgoing edge u v if u v i tene Relax(u v) Figure. how an execution of thi modified algorithm on the ame graph a Figure.. The correctne of PuhDagSSSP follow immediately from the correctne of Ford general relaxation trategy, but it not hard to prove correctne directly, by induction over the vertice in topological order Figure.. Computing hortet path in a dag, by relaxing outgoing edge in topological order. In each iteration, bold edge indicate predeceor, and the bold vertex i about to be canned. Compare with Figure..

13 .. Bet-Firt: Dijktra Algorithm. Bet-Firt: Dijktra Algorithm If we replace the FIFO queue in breadth-firt earch with a priority queue, where the key of a vertex v i it tentative ditance dit(v), we obtain an algorithm firt publihed in 5 by a team of reearcher at the Cae Intitute of Technology, in an annual project report for the Combat Development Department of the US Army Electronic Proving Ground. The ame algorithm wa independently dicovered by Edger Dijktra in 5 (but not publihed until 5), again by George Minty ome time before, and again by P. D. Whiting and J. A. Hillier in. A nearly identical algorithm wa alo decribed by George Dantzig in 5. Although everal early ource called it Minty algorithm, thi approach i now univerally known a Dijktra algorithm, in full accordance with Stigler Law. Peudocode for thi algorithm i hown in Figure.. Dijktra(): InitSSSP() Inert(, ) while the priority queue i not empty u ExtractMin( ) for all edge u v if u v i tene Relax(u v) if v i in the priority queue DecreaeKey(v, dit(v)) ele Inert(v, dit(v)) Figure.. Dijktra algorithm. An eay induction proof implie that, at all time during the execution of thi algorithm, an edge u v i tene if and only if vertex u i either in the priority queue or i the vertex mot recently Extracted from the priority queue. Thu, Dijktra algorithm i an intance of Ford general trategy, which implie that it correctly compute hortet path, provided there are no negative cycle in G. No Negative Edge Dijktra algorithm i particularly well-behaved when the input graph ha no negative-weight edge. In thi etting, the algorithm intuitively expand a wavefront outward from the ource vertex, paing over vertice in increaing I will follow thi common convention, depite the hitorical inaccuracy, partly becaue I don t think anybody want to read about the Leyzorek-Gray-Johnon-Ladew-Meaker-Petry-Seitz- Dantzig-Dijktra-Minty-Whiting-Hillier algorithm, and partly becaue paper that aren t actually publihed don t count.

14 . SHORTEST PATHS order of their ditance from, imilarly to breadth-firt earch. Figure. how the algorithm in action Figure.. The firt four iteration of Dijktra algorithm on a graph with no negative edge. In each iteration, bold edge indicate predeceor; haded vertice are in the priority queue; and the bold vertex i about to be canned. The remaining iteration do not change the ditance or the hortet-path tree. We can derive a elf-contained proof of correctne for Dijktra algorithm in thi etting by formalizing thi wavefront intuition. For each integer i, let u i denote the vertex returned by the ith call to ExtractMin, and let d i be the value of dit(u i ) jut after thi Extraction. In particular, we have u = and d =. We cannot aume at thi point that the vertice u i are ditinct; in principle, the ame vertex might be Extracted more than once. Lemma.. If G ha no negative-weight edge, then for all i < j, we have d i d j. Proof: Aume G ha no negative weight edge. Fix an arbitrary index i; to prove the lemma, it uffice to prove that d i+ d i. There are two cae to conider. Suppoe the edge u i u i+ i relaxed during the ith iteration of the main loop. Then at the end of the ith iteration, we have dit(u i+ ) = dit(u i ) + w(u i u i+ ) dit(u i ), becaue all edge weight are non-negative. On the other hand, uppoe the edge u i u i+ i not relaxed during the ith iteration of the main loop. Then at the tart of the ith iteration, u i+ mut already be in the priority queue, with priority dit(u i+ ) dit(u i ), becaue u i i the vertex returned by ExtractMin. Moreover, dit(u i+ ) doe not change during the ith iteration.

15 .. Bet-Firt: Dijktra Algorithm In both cae, we conclude that d i+ d i. The lemma now follow immediately by induction on i. Lemma.. If G ha no negative-weight edge, each vertex of G i Extracted from the priority queue at mot once. Proof: Suppoe v i Extracted more than once. Specifically, uppoe v i Extracted in the ith iteration of the main loop, reinerted during the jth iteration, and reextracted during the kth iteration, for ome indice i < j < k. Then in the notation of the previou proof, we have v = u i = u k. The ditance label dit(v) never increae. Moreover, dit(v) trictly decreae during the jth iteration, jut before v i reinerted. It follow that d i > d k. Therefore, by the previou lemma, G ha at leat one negative-weight edge. Lemma. immediately implie that each vertex i canned at mot once, and thu that each edge i relaxed at mot once. However, unlike in breadth-firt earch, each ditance label dit(v) can change multiple time. The firt time dit(v) change from, we Inert v into the priority queue; after that, each change to dit(v) i followed by a call to DecreaeKey. After v i Extracted from the priority queue, it ditance label never change. The ret of the correctne proof i almot identical to breadth-firt earch. Theorem.5. If G ha no negative-weight edge, then when Dijktra end, dit(v) i the length of the hortet path in G from to v, for every vertex v. Proof: Fix an arbitrary vertex v, and conider an arbitrary path v v v l in G, where v = and v l = v. For any index j, let L j denote the length of the ubpath v v v j. We prove by induction that dit(v j ) L j for all j. Trivially dit(v ) = dit() = = L. For any index j >, the induction hypothei implie dit(v j ) L j. Immediately after we Pull vertex v j from the queue, either dit(v i ) dit(v j )+w(v j v j ) already, or we et dit(v i ) dit(v j )+w(v j v j ). In either cae, we have dit(v j ) dit(v j ) + w(v j v j ) L j + w(v j v j ) = L j. We jut proved that dit(v) i at mot the length of an arbitrary path from to v; it follow that dit(v) i at mot the length of the hortet path from to v. On the other hand, a imilar induction proof implie that dit(v) i the length of the predeceor path pred(pred(v)) pred(v) v. 5

16 . SHORTEST PATHS It remain only to bound the algorithm running time. Altogether Dijktra perform at mot E DecreaeKey operation, and at mot V Inert and ExtractMin operation. Thu, if we implement the underlying priority queue uing a tandard binary heap, which upport each operation in O(log V ) time, Dijktra run in O(E log V) time. If we know in advance that our input graph will never have negative edge, we can implify Dijktra algorithm lightly, by Inerting every vertex into the priority queue in the initialization phae, and then only calling DecreaeKey in the main loop, a hown in Figure.. Thi i the verion of Dijktra algorithm preented by mot algorithm textbook, Wikipedia, and even Dijktra original paper; it alo the verion of Dijktra algorithm that I decribed a bet-firt earch in Chapter??. NonnegativeDijktra(): InitSSSP() for all vertice v Inert(v, dit(v)) while the priority queue i not empty u ExtractMin( ) for all edge u v if u v i tene Relax(u v) DecreaeKey(v, dit(v)) Figure.. Dijktra algorithm very lightly implified for graph without negative edge. Difference from Dijktra are bold red. Negative Edge However, NonnegativeDijktra doe not correctly compute hortet path in graph with negative edge. Moreover, even when all edge weight are poitive, NonnegativeDijktra i no fater than Dijktra (either in theory or in practice). For both of thee reaon, I think Dijktra i more deerving of the name Dijktra algorithm than NonnegativeDijktra. Even Edger Dijktra would have agreed that a correct algorithm that i ometime (and in practice, rarely) low i better than a fat algorithm that doen t alway work! Shortet-path paper from the 5 never mentioned priority queue. Dijktra propoed a brute-force can of all vertice on the wavefront at every iteration; hi original algorithm run in O(V ) time, which i actually fater than the binary-heap implementation when E = Ω(V )! Minty propoed a brute-force can of all edge u v uch that dit(u) i finite but dit(v) i not; thu, hi original algorithm run in O(V E) time. The ue of a priority queue, implemented a a binary heap, to obtain near-linear running time wa propoed by Donald Johnon in. The running time can be improved to O(E + V log V) uing a more complex priority queue data tructure called a Fibonacci heap. There are even fater algorithm, uing even more ophiticated priority queue, for the pecial cae of integer edge weight.

17 .. Relax ALL the Edge: Bellman-Ford Unfortunately, when the input graph ha negative edge, the familiar expanding wavefront intuition i no longer accurate. The ame vertex could be Extracted multiple time; the ame edge could be relaxed multiple time; and ditance might not be dicovered in increaing order. Figure.5 how an example execution where the top left vertex i Extracted ix time, and the top three edge are each relaxed twice. For graph without negative cycle, but no other retriction on edge weight, the wort-cae running time of Dijktra i actually exponential. Figure. how particularly imple family of graph (due to Dougla Shier and Chritoph Witzgall) that force Dijktra to perform Θ( V / ) relaxation. A more complex family of graph (which I ll leave a an exercie) force Θ( V ) relaxation, which i the wort poible. In practice, however Dijktra algorithm i uually quite fat even for graph with negative edge. k k Figure.. A directed graph with negative edge that force DIJKSTRA to run in exponential time.. Relax ALL the Edge: Bellman-Ford The implet implementation of Ford generic hortet-path algorithm wa firt ketched by Alfono Shimbel in 5, decribed in more detail by Edward Moore in 5, and independently redicovered by Max Woodbury and George Dantzig in 5, by Richard Bellman in 5, and by George Minty in 5. (Neither Woodbury and Dantzig nor Minty publihed their algorithm.) In full compliance with Stigler Law, the algorithm i almot univerally known a Bellman-Ford, becaue Bellman explicitly ued Ford 5 formulation of relaxing edge, although ome author refer to Bellman-Kalaba and a few early ource refer to Bellman-Shimbel. Amuingly, Shier and Witzgall example i a dag with only O(V ) edge, which mean hortet path can be computed in O(V ) time, even if we didn t already notice that the zig-zag path along the top i the hortet path tree. I will follow thi common convention, depite the hitorical inaccuracy, partly becaue I don t think anyone really want read about the Shimbel / Moore / Woodbury-Dantzig / Bellman- Ford / Kalaba / Minty algorithm, and partly becaue I m tired of people looking at me funny when I talk about Shimbel algorithm. Thi name i mot likely a reference to Richard Bellman and Robert Kalaba 5 monograph on dynamic programming and control theory, which decribe Bellman algorithm. Bellman and Kalaba alo publihed an extenion of Bellman algorithm in that compute kth hortet path, for any contant k.

18 . SHORTEST PATHS Figure.5. A complete run of Dijktra algorithm on a graph with negative edge. At each iteration, bold edge indicate predeceor; haded vertice are in the priority queue; and the bold vertex i the next to be canned. Compare with Figure..

19 .. Relax ALL the Edge: Bellman-Ford The Shimbel / Moore / Woodbury-Dantzig / Bellman-Ford / Kalaba / Minty / Broh algorithm can be ummarized in one line: Bellman-Ford: Relax ALL the tene edge, then recure. BellmanFord() InitSSSP() while there i at leat one tene edge for every edge u v if u v i tene Relax(u v) The following lemma i the key to proving both correctne and efficiency of Bellman-Ford. For every vertex v and non-negative integer i, let dit i (v) denote the length of the hortet walk in G from to v coniting of at mot i edge. In particular, dit () = and dit (v) = for all v. Lemma.. For every vertex v and non-negative integer i, after i iteration of the main loop of BellmanFord, we have dit(v) dit i (v). Proof: The proof proceed by induction on i. The bae cae i = i trivial, o aume i >. Fix a vertex v, and let W be the hortet walk from to v coniting of at mot i edge (breaking tie arbitrarily). By definition, W ha length dit i (v). There are two cae to conider. Suppoe W ha no edge. Then W mut be the trivial walk from to, o v = and dit i () =. We et dit() in InitSSSP, and dit() can never increae, o we alway have dit(). Otherwie, let u v be the lat edge of W. The induction hypothei implie that after i iteration, dit(u) dit i (u). During the ith iteration of the outer loop, when we conider the edge u v in the inner loop, either dit(v) < dit(u) + w(u v) already, or we et dit(v) dit(u) + w(u v). In both cae, we have dit(v) dit i (u) + w(u v) = dit i (v). A uual, dit(v) cannot increae (although dit(v) might decreae further before the ith iteration of the outer loop end). In both cae, we conclude that dit(v) dit i (v) at the end of the ith iteration. If the input graph ha no negative cycle, the hortet walk from to any other vertex i a imple path with at mot V edge; it follow that Bellman- Ford halt with the correct hortet-path ditance after at mot V iteration. Go read everything in Hyperbole and a Half again. And then adopt another cat, o you can buy it another copy of the book.

20 . SHORTEST PATHS Said differently, if any edge i till tene after V iteration, then the input graph mut contain a negative cycle! Thu, we can rewrite the algorithm more concretely a follow: BellmanFord() InitSSSP() repeat V time for every edge u v if u v i tene Relax(u v) for every edge u v if u v i tene return Negative cycle! Each iteration of the inner loop trivially require O(E) time, o the overall algorithm run in O(V E) time. Thu, Bellman-Ford i alway efficient, even if the graph ha negative edge, and in fact even if the graph ha negative cycle. If all edge weight are non-negative, however, Dijktra algorithm i fater, at leat in the wort cae. (In practice, Dijktra algorithm i often fater than Bellman-Ford even for graph with negative edge.) Moore Improvement Neither Moore nor Bellman decribed the Bellman-Ford algorithm in the form I ve preented here. Moore preented hi verion of the algorithm ("Algorithm D") in the ame paper that propoed breadth-firt earch ("Algorithm A") for unweighted graph; indeed, the two algorithm are nearly identical. Although Moore algorithm ha the ame O(V E) wort-cae running time a BellmanFord, it i often ignificantly fater in practice, intuitively becaue it avoid checking edge that are obviouly not tene. Moore derived hi weighted hortet-path algorithm by making two modification to breadth-firt earch. Firt, replace each + with +w(u v)" in the innermot loop, to take the edge weight into account. Second, check whether a vertex i already in the FIFO queue before Inerting it, o that the queue alway contain at mot one copy of each vertex. Following our earlier analyi of breadth-firt earch, I ll introduce a token to break the execution of the algorithm into phae. Jut like BFS, each phae begin when the token i Puhed into the queue, and end when the token i Pulled out of the queue again. Jut like BFS, the algorithm end when the queue contain only the token. The reulting algorithm i hown in Figure.. Becaue the queue contain at mot one copy of each vertex at any time, each vertex i Pulled from the queue at mot once in each phae, and therefore Moore algorithm i till correct without thi check, but the O(V E) time bound i not.

21 .. Relax ALL the Edge: Bellman-Ford Moore(): InitSSSP() Puh() Puh( ) tart the firt phae while the queue contain at leat one vertex u Pull( ) if u = Puh( ) tart the next phae ele for all edge u v if u v i tene Relax(u v) if v i not already in the queue Puh(v) Figure.. Moore hortet-path algorithm. Bold red line involving the token are only for analyi. each edge u v i checked for tenene at mot once in each phae. Moreover, every edge that i tene when a phae begin i relaxed during that phae. (Some edge that become tene during the phae might alo be relaxed during that phae, and ome relaxed edge might become tene again in the ame phae.) Thu, Moore can be viewed a a refinement of BellmanFord that ue a queue to maintain tene edge, rather than teting every edge by brute force. In particular, a imilar inductive proof etablihe the following analogue of Lemma.: Lemma.. For every vertex v and non-negative integer i, after i phae of Moore, we have dit(v) dit i (v). Thu, if the input graph ha no negative cycle, Moore halt after at mot V phae. In each phae, we can each vertex at mot once, o we relax each edge at mot once, o the wort-cae running time of a ingle phae i O(E). Thu, the overall running time of Moore i O(VE). In practice, however, Moore often compute hortet path coniderably fater than BellmanFord, becaue it only can an edge u v if dit(u) wa changed in the previou phae. If the input graph contain a negative cycle, Moore never halt. Fortunately, like BellmanFord, it i eay to modify Moore algorithm to report negative cycle if they exit. Perhap the eaiet modification i to actually maintain a token, and count the number of time the token i Pulled from the queue. Then the input graph contain a negative cycle if and only if the queue i non-empty immediately after the token i Pulled for the (V )th time.

22 . SHORTEST PATHS e 5 e 5 e 5 d f d f d f b 5 b 5 a b c b 5 a c a c a c d f e 5 5 e 5 5 e 5 5 d f d f d f b 5 d b 5 d c b 5 a c a c a c Figure.. A complete run of Moore algorithm on a directed graph with negative edge. Node are pulled from the queue in the order a b c d f d c d, where i the end-of-phae token. At the tart of each phae, bold edge indicate predeceor, and haded vertice are in the vertex queue. Compare with Figure. and.5. Dynamic Programming Formulation Like almot everything ele with hi name on it, Richard Bellman derived the Bellman-Ford hortet-path algorithm via dynamic programming. A uual, we need to tart with a recurive definition of hortet path ditance. It tempting to ue the ame identity that we exploited for directed acyclic graph: dit(v) = min (dit(u) + w(u v)) otherwie u v if v = Unfortunately, if the input graph i not a dag, thi recurrence doen t work! Suppoe the input graph contain the directed cycle u v w u. To compute dit(w) we firt need dit(v), and to compute dit(v) we firt need dit(u), but to compute dit(u) we firt need dit(w). If the input graph ha any directed cycle, we get tuck in an infinite loop! To upport a proper recurrence, we need to add an additional tructural parameter to the ditance function, which decreae monotonically at each recurive call, defined o that the function i trivial to evaluate when the

23 .. Relax ALL the Edge: Bellman-Ford parameter reache. Bellman choe the maximum number of edge a thi additional parameter. A in our earlier analyi, let dit i (v) denote the length of the hortet walk from to v coniting of at mot i edge. Bellman oberved that thi function obey the following Bellman equation recurrence: if i = and v = if i = and v dit i (v) = dit i (v) min min (dit otherwie i (u) + w(u v)) u v Let aume that the graph ha no negative cycle, o our goal i to compute dit V (v) for every vertex v. Here i a traightforward dynamic-programming evaluation of thi recurrence, where dit[i, v] tore the value of dit i (v). Correctne of the final hortet-path ditance follow from the correctne of the recurrence, and the O(V E) running time i obviou. Thi i eentially how Bellman preented hi hortet-path algorithm. BellmanFordDP() dit[, ] for every vertex v dit[, v] for i to V for every vertex v dit[i, v] dit[i, v] for every edge u v if dit[i, v] > dit[i, u] + w(u v) dit[i, v] dit[i, u] + w(u v) We can tranform thi dynamic programming algorithm into our original formulation of BellmanFord through a hort erie of minor modification. Firt, each iteration of the outermot loop conider each edge u v exactly once, but the order in which we conider thoe edge doen t actually matter. Thu, we can afely remove one level of indentation from the lat three line! The modified algorithm may conider edge in a different order, but it till correctly compute dit i (v) for all i and v. A we ll ee in the next chapter, thi i not the only reaonable choice.

24 . SHORTEST PATHS BellmanFordDP() dit[, ] for every vertex v dit[, v] for i to V for every vertex v dit[i, v] dit[i, v] for every edge u v if dit[i, v] > dit[i, u] + w (u v) dit[i, v] dit[i, u] + w (u v) Next we change the indice in the lat two line from i to i. Thi change may caue the ditance dit[i, v] to approach the true hortet-path ditance more quickly than before, but the algorithm correctly compute the true hortet path ditance. Intead of dit[i, v] = dit i (v), we now have dit[i, v] dit i (v) for all i and v, mirroring Lemma. and.. BellmanFordDP() dit[, ] for every vertex v dit[, v] for i to V for every vertex v dit[i, v] dit[i, v] for every edge u v if dit[i, v] > dit[i, u] + w(u v) not i! dit[i, v] dit[i, u] + w(u v) not i! But thi algorithm i a little illy. In the ith iteration of the outermot loop, we firt copy the (i )th row of the array dit[, ] to the ith row, and then modify the element of the ith row. So we really don t need a two-dimenional array at all; the iteration index i i completely redundant! In our final modification, we maintain only a one-dimenional array of tentative ditance. BellmanFordFinal() dit[] for every vertex v dit[v] for i to V for every edge u v if dit[v] > dit[u] + w(u v) dit[v] dit[u] + w(u v) Thi final dynamic programming algorithm i almot identical to our original formulation of BellmanFord! The firt three line initialize the hortet path ditance, and the lat two line relax the edge u v if that edge i tene.

25 Exercie BellmanFordFinal i miing only two feature of our earlier formulation: It doe not maintain predeceor pointer or detect negative cycle. Fortunately, adding thoe feature i traightforward. Exercie. Let G be a directed graph with arbitrary edge weight (which may be poitive, negative, or zero), poibly with negative cycle, and let be an arbitrary vertex of G. (a) Suppoe every vertex v tore a number dit(v) (but no predeceor pointer). Decribe and analyze an algorithm to determine whether dit(v) i the hortet-path ditance from to v, for every vertex v. (b) Suppoe intead that every vertex v tore a pointer pred(v) to another vertex in G (but no ditance). Decribe and analyze an algorithm to determine whether thee predeceor pointer define a ingle-ource hortet path tree rooted at.. A looped tree i a weighted, directed graph built from a binary tree by adding an edge from every leaf back to the root. Every edge ha non-negative weight. 5 A looped tree. (a) How much time would Dijktra algorithm require to compute the hortet path between two vertice u and v in a looped tree with n node? (b) Decribe and analyze a fater algorithm.. Suppoe we are given a directed graph G with weighted edge and two vertice and t. (a) Decribe and analyze an algorithm to find the hortet path from to t when exactly one edge in G ha negative weight. [Hint: Modify Dijktra algorithm. Or don t.] 5

26 . SHORTEST PATHS (b) Decribe and analyze an algorithm to find the hortet path from to t when exactly k edge in G have negative weight. How doe the running time of your algorithm depend on k?. Suppoe we are given an undirected graph G in which every vertex ha a poitive weight. (a) Decribe and analyze an algorithm to find a panning tree of G with minimum total weight. (The total weight of a panning tree i the um of the weight of it vertice.) (b) Decribe and analyze an algorithm to find a path in G from one given vertex to another given vertex t with minimum total weight. (The total weight of a path i the um of the weight of it vertice.) [Hint: One of thee problem i trivial.]. For any edge e in any graph G, let G \ e denote the graph obtained by deleting e from G. Suppoe we are given a graph G and two vertice and t. The replacement path problem ak u to compute the hortet-path ditance from to t in G \ e, for every edge e of G. The output i an array of E ditance, one for each edge of G. (a) Suppoe G i a directed graph, and the hortet path from vertex to vertex t pae through every vertex of G. Decribe an algorithm to olve thi pecial cae of the replacement path problem in O(E log V ) time. (b) Decribe an algorithm to olve the replacement path problem for arbitrary undirected graph in O(E log V ) time. In both ubproblem, you may aume that all edge weight are non-negative. [Hint: If we delete an edge of the original hortet path, how do the old and new hortet path overlap?] 5. Let G = (V, E) be a connected directed graph with non-negative edge weight, let and t be vertice of G, and let H be a ubgraph of G obtained by deleting ome edge. Suppoe we want to reinert exactly one edge from G back into H, o that the hortet path from to t in the reulting graph i a hort a poible. Decribe and analyze an algorithm that chooe the bet edge to reinert, in O(E log V ) time.. (a) Decribe and analyze a modification of Bellman-Ford that actually return a negative cycle if any uch cycle i reachable from, or a hortet-path tree if there i no uch cycle. The modified algorithm hould till run in O(V E) time.

27 Exercie (b) Decribe and analyze a modification of Bellman-Ford that compute the correct hortet path ditance from to every other vertex of the input graph, even if the graph contain negative cycle. Specifically, if any walk from to v contain a negative cycle, your algorithm hould end with dit(v) = ; otherwie, dit(v) hould contain the length of the hortet path from to v. The modified algorithm hould till run in O(V E) time. (c) Repeat part (a) and (b), but for Ford generic relaxation algorithm. You may aume that the unmodified algorithm halt in O( V ) tep if there i no negative cycle; your modified algorithm hould alo run in O( V ) time.. Conider the following even looer variant of Ford generic relaxation algorithm: FellmanBored(): InitSSSP() repeat e i any edge in G if e i i tene Relax(e i ) until you get bored or whatever Prove that if FellmanBored examine the edge of any walk W tarting from, in order along W, then the lat ditance label in W i at mot the length of W. More formally: If the edge of any walk v v v l, where v =, define a ubequence of the edge e, e, e,... examined by FellmanBored, then we have dit(l) l i= w(v i v i ). [Hint: Thi property i almot eaier to prove than it i to tate correctly.]. Thi problem conider everal way to detect negative cycle uing Ford generic relaxation algorithm. (a) Prove that if pred() ever change after InitSSSP, then the input graph contain a negative cycle through. (b) Show that pred() might never change after InitSSSP, even when the input graph contain a negative cycle through. (c) Let P denote the current graph of predeceor edge pred(v) v, and let X denote the et of all currently tene edge; both of thee et evolve a the algorithm execute. Prove that the input graph ha no negative cycle if and only if P X i alway a dag. (d) Let R denote the et of all edge that have been relaxed o far; thi et grow a the algorithm execute. Prove that the input graph ha no negative cycle if and only if R i alway a dag.

28 . SHORTEST PATHS. Prove that Dijktra algorithm perform Ω( V ) relaxation in the wort cae when edge are allowed to have negative weight, even if the underlying graph i acyclic. Specifically, for every poitive integer n, contruct a n-vertex dag G n with weighted edge, uch that Dijktra algorithm call Relax Ω( n ) time when G n i the input graph. [Hint: Tower of Hanoi.]. Prove that Ford generic relaxation algorithm (and therefore Dijktra algorithm) halt after at mot O( V ) relaxation, unle the input graph contain a negative cycle. [Hint: See Problem (d).]. Although we typically peak of the hortet path between two node, ingle graph could contain everal minimum-length path with the ame endpoint. Even for weighted graph, it i often convenient to chooe a minimum-weight path with the fewet edge; call thi a bet path from to t. Suppoe we are given a directed graph G with poitive edge weight and a ource vertex in G. Decribe and analyze an algorithm to compute bet path in G from to every other vertex Figure.. Four (of many) equal-length hortet path. The firt path i the bet hortet path.. Decribe and analyze an algorithm to determine the number of hortet path from a ource vertex to a target vertex t in an arbitrary directed graph G with weighted edge. You may aume that all edge weight are poitive and that all neceary arithmetic operation can be performed in O() time. [Hint: Compute hortet path ditance from to every other vertex. Throw away all edge that cannot be part of a hortet path from to another vertex. What left?]. You jut dicovered your bet friend from elementary chool on Twitbook. You both want to meet a oon a poible, but you live in two different cite that are far apart. To minimize travel time, you agree to meet at an intermediate city, and then you imultaneouly hop in your car and tart driving toward each other. But where exactly hould you meet? You are given a weighted graph G = (V, E), where the vertice V repreent citie and the edge E repreent road that directly connect citie. Each edge e ha a weight w(e) equal to the time required to travel between the