Lecture 14: Minimum Spanning Tree I

Transcription

1 COMPSCI 0: Deign and Analyi of Algorithm October 4, 07 Lecture 4: Minimum Spanning Tree I Lecturer: Rong Ge Scribe: Fred Zhang Overview Thi lecture we finih our dicuion of the hortet path problem and introduce the Bellman-Ford algorithm for dealing with negative edge length. We then introduce the minimum panning tree (MST) problem and prove a key property of MST. Shortet Path with Negative Edge Length Today we would like to deal with problem of finding hortet path in graph when the graph ha negative length edge. One of the common cenario with negative edge i the currency exchange arbitrage.. Motivation: Currency Exchange Suppoe u, v are different currencie, and the exchange rate i C(u, v) (i.e., unit of v i worth C(u, v) unit of u). We et the length of the edge a w(u, v) = log C(u, v). We take the log becaue, in currency exchange, the rate are multiplicative factor, but in hortet path, path length i defined a um of edge length. By thi converion, the length of a path i log of the total exchange rate. Becaue of the logarithm, the edge length w(u, v) can be negative or poitive. It i negative if C(u, v) i le than, and a we mentioned, a negative cycle in the graph mean currency arbitrage. In general, one can walk along a negative cycle infinitely many time, and a path that contain the cycle ha length of negative infinity. Therefore, our algorithm will try to find hortet path, auming there i no negative cycle.. Bellman-Ford Algorithm We have looked at the Bellman-Ford algorithm, which ue the dynamic programming paradigm. Firt, we define the tate. Let d[u, i] be the length of the hortet path to get to vertex u with (exactly) i tep. A a property of hortet path, we know that any ub-path of a hortet path i alo the hortet. Therefore, the tranition function i d[v, i + ] = min (u,v) E w[u, v] }{{} length of the lat tep + d[u, i] }{{} hortet path to a predeceor For thi tranition function, it i eay to figure out the correct ordering of evaluating the entrie. We firt calculate d[v, i] for all v firt and then are able to compute d[v, i + ]. We do not have to worry about the negative cycle, a we did before.. Lecture 4: Minimum Spanning Tree I-

2 . Implementation Before analyzing it running time, let u dicu it implementation. It follow from the guideline for implementing any dynamic programming algorithm. If you have one parameter, write one for loop. If you have two parameter, write a double for loop. Then try to fill in the table in the correct order. For Bellman-Ford, let u lightly modify the definition of tate. Redefine d[u, i] a the length of the hortet path to get to u with at mot i tep. The tranition function i given by { } d[v, i + ] = min d[v, i], min (w [u + v] + d[u, i]). (u,v) E The difference i that we now have the option of reaching to v jut uing (at mot) i tep when computing d[v, i + ]. The econd part of the tranition function i till the ame. The implementation i fairly eay, given below by Algorithm. Algorithm : The Bellman-Ford Algorithm Initialize d[, 0] 0, d[u, 0] for all other vertice u. for i = to n do Initialize d[u, i] d[u, i ] for all i. for all edge (u, v) do if w[u, v] + d[u, i ] d[v, i] then d[v, i] w[u, v] + d[u, i ]. if there i a vertex u uch that d[u, n] d[u, n ] then Report there i a negative cycle. The lat part of the algorithm i to try detecting a negative cycle. Every vertex i reachable within n tep, o if ome vertex ditance to can be decreaed uing n tep, compared to uing n tep, then there i a negative cycle. The proof of thi fact i left a an exercie..4 Example of Running Bellman-Ford Let u now ee how the Bellman-Ford algorithm run on a graph. Conider the graph below with negative edge length but no negative cycle Figure : A graph with edge length and vertex label. Now run the Bellman-Ford algorithm tarting at vertex. For initialization, when i = 0, every vertex ha ditance d[u, 0] =. We thu get Figure a. Lecture 4: Minimum Spanning Tree I-

3 - - (a) The t iteration at i = 0. Vertice are labeled with d[u, 0]. (b) The nd iteration at i =. Vertice are labeled with d[u, ]. Figure : Firt two iteration. When i =, the algorithm conider the two edge incident on (hown in Figure a) and update the two relevant vertice. Their ditance become and ; ee Figure b. Three outgoing edge from thee two vertice will are relevant for the next tep. The red edge i particularly important to notice. If there i no negative edge, we hould not conider it at thi moment, a the Dijktra algorithm would do. But maybe thi edge i negative, and we might get a horter path to the upper vertex by re-routing from the lower left vertex (a) The rd iteration at i =. Vertice are labeled with d[u, ] (b) The 4th iteration at i =. Vertice are labeled with d[u, ] Figure : Next two iteration. When i =, the algorithm conider the three edge, hown in Figure b, and update ditance accordingly. The reult i Figure a. Again, the algorithm will next conider all outgoing edge from vertice that were jut updated. When i =, the algorithm update the ditance of two vertice and prepare to conider their outgoing edge Figure 4: The 5th iteration at i = 4. Vertice are labeled with d[u, 4], the final olution. When i = 4, thi i the lat iteration, becaue n = 5 and we know there i no negative cycle. The algorithm make one more update thi iteration. Lecture 4: Minimum Spanning Tree I-

4 .5 Analyi Correctne. We have undertood that if there i no negative cycle in the input graph, then thi algorithm baed on dynamic programming i correct. However, how can the algorithm correctly detect a negative cycle if there i one? We need two claim. When the graph ha no negative cycle, a hortet path can only contain n edge. Intuitively, thi i becaue if there i no negative cycle, then the hortet path hould not viit a vertex twice. Of coure, if no vertex i repeated in a hortet path, every hortet path i contain at mot n edge. The proof i left a an exercie. On the other hand, we alo want to prove that if the graph ha a negative cycle, then there i a vertex whoe hortet path with n edge i horter than all path with at mot n edge. Then in the lat iteration of the algorithm (i = n), one vertex ditance label will decreae, indicating that there i a negative cycle. Running Time. We have een a naïve implementation of the Bellman-Ford algorithm (Algorithm ). What i it running time? Each iteration take O(m) time, ince the algorithm check, for every edge, if it can be ued to update a hortet path. There are n iteration, o the total running time i O(mn). Thi i clearly lower than the Dijktra algorithm, which run in time O(m log n). The gap here i almot a factor of n. Thi i uually a big deal becaue if you have a graph of jut 0, 000 vertice, then the running time would be 0, 000 lower, and it i pretty bad. In practice, people ue many heuritic, though they do not affect the wort-cae analyi of O(mn). A major heuritic i to only conider outgoing edge from vertice that were updated lat iteration. Oberve that thi i exactly what we have done in Section.4. Minimum Spanning Tree Now let u go to minimum panning tree (MST). The algorithm for MST are fairly eay, but proving correctne i much harder, o we are going to pend more time on that. Formally, MST i the following problem. Input: an undirected graph with edge weight. (There i another verion of the problem defined on directed graph, but it require very different algorithm.) The edge weight w[u, v] i the cot of connecting two vertice u, v. We aume they are non-negative. Goal: elect a ubet of edge uch that every pair of vertice can be connected by thee edge. Minimize the total weight of the edge elected.. Example Now let u take the graph below a an example (Figure 5a) and conider it MST. Our goal i jut to elect a et of edge connecting every vertex. Of coure, if one keep every edge, the graph i connected. But we would like to remove ome to minimize the total weight of edge that are left. The final olution (Figure 5b) doen t look like a tree becaue we didn t draw it upide down and name a root. But in general, a tree i jut a connected graph with no cycle. The graph in Figure 5b clearly qualifie for that. Lecture 4: Minimum Spanning Tree I-4

5 5 4 7 (a) Input graph with non-negative edge weight (b) Solution. An MST of cot. Figure 5: An MST intance. We call thi a panning tree becaue all edge are from the original graph. In other word, we are required to return a tree a a ubgraph of the input. Moreover, to connect every vertex, we hould pick a tree for minimum cot. If there i a cycle, we could remove an edge from the cycle, and the graph remain connected. 4 Key Property of MST Previouly in hortet path, we identified the hortet path property. Thi lead to many dynamic programming-baed algorithm. In MST, we follow the ame idea and look for a key property that enable algorithm deign. Here, one may wonder if an analogy of the hortet path property would hold. Namely, a ubtree T of an MST i alo an MST for the vertice in T. In fact, thi i true becaue if there i another cheaper tree T connecting thee vertice, one hould have picked T intead of T in the large MST, a thi decreae the cot. Now can we ue dynamic programming baed on thi property? The problem i that there are n ubproblem, where we want to find an MST for every ubet of vertice. Thi cannot lead to an efficient algorithm. 4. Swap Operation Let u go back to MST and oberve that adding any non-tree edge to a tree will introduce a cycle and, on the other hand, for a graph with a cycle, removing any edge in the cycle reult in a tree. Now let u make it a baic operation of adding an edge and removing another one in the cycle created, and call thi a wap. Thi operation i ueful. Conider again Figure 5a. 4 4 (a) A uboptimal panning tree of cot. (b) Swapping the red edge give MST. Figure : A wap operation. Lecture 4: Minimum Spanning Tree I-5

6 Why are we conidering thi operation? Becaue we will ue it in the proof of the algorithm correctne. It turn out the MST algorithm are greedy algorithm. Recall that for proving greedy algorithm, the general recipe i to aume optimal olution i different from the current olution. Then try to modify the optimal olution to make it look imilar to the algorithm olution. We will ee oon how the wap operation i ued in the proof. 4. Cut in Graph Before delving into detail, let u define the notion of the cut of a graph. A cut i a et of edge that eparate the vertice into two part. Uually, we pecify a cut by a ubet of vertice ( S, S ), where S i a et of vertice and S i the remaining vertice. Figure 7: A graph cut. The yellow edge are cut edge. Now we want to deign a greedy algorithm. Recall that a greedy algorithm break down the problem into a equence of deciion and for each deciion make the obviou choice. Uually it i eay to come up with the obviou choice, but not o clear in MST. So we need the notion of graph cut. Notice that from every cut ( S, S ), an MST mut elect at leat one edge in order to be connected. Otherwie, a vertex in S i not connected to a vertex in S, o it i not a valid panning tree. Now ince we want to be greedy, let u chooe the cheapet edge for each cut. Thi i the obviou choice. 4. Key Lemma Why i thi greedy rule good? We claim the following key lemma. Lemma (Key Lemma). Suppoe F i a et of edge inide ome MST T. If (S, S) i a cut that doe not contain any edge in F and e i the minimum cot edge in the cut, then it i afe to add e to F. Let u undertand thi claim. Suppoe we already have a ubtree F of ome MST. Let (S, S) be a cut not connected by F yet and e be the cheapet cut edge. The lemma claim that adding e to F keep u on the right track. Formally, F {e} i till a ubtree of ome MST T. Hence, if we keep adding edge in thi manner, we would end up with ome MST. Proof (Key Lemma). The proof will be quite graphical. Fix a cut (S, S) that F ha not connected yet. Conider the MST T that contain F. In Figure 8, green edge are thoe in T but not in F. Now conider we add e into T. A we can ee, it will create a cycle in the middle. Since thi cycle croe the cut (S, S), there mut be another edge e T that i in the cycle and i a cut edge. We wap e and e. The cot of e certainly cannot be le than e, ince we aume e i the cheapet Lecture 4: Minimum Spanning Tree I-

7 e e S S Figure 8: Swap e and e one in the cut. Therefore, the wap doe not increae the cot. Moreover, after the witching, we till get a panning tree. Therefore, if T = T {e} \ {e }, w(t ) w(t ), and the MST T contain F. Thi complete the proof. Lecture 4: Minimum Spanning Tree I-7