Minimum Spanning Trees

Minimum Spanning Trees

Problem A town has a set of houses and a set of roads. A road connects 2 and only 2 houses. A road connecting houses u and v has a repair cost w(u, v). Goal: Repair enough (and no more) roads such that 1. everyone stays connected: can reach every house from all other houses, and 2. total repair cost is minimum.

Model as a graph: Undirected graph G = (V, E). Weight w(u, v) on each edge (u, v) E. Find T E such that 1. T connects all vertices (T is a spanning tree), and 2. w(t ) = w(u, v) is minimized. (u,v) T A spanning tree whose weight is minimum over all spanning trees is called a minimum spanning tree, or MST. Example of such a graph [edges in MST are shaded] : a 10 12 9 b c 8 8 d 7 e 3 3 1 f 5 6 g h 9 2 11 i In this example, there is more than one MST. Replace edge (e, f ) by (c, e). Get a different spanning tree with the same weight.

Some properties of an MST: It has V 1 edges. It has no cycles. It might not be unique. 8 8 b d g 10 2 7 5 a 9 e 9 i 3 3 12 11 c f h 1 6 In this example, there is more than one MST.

Growing a minimum spanning tree Building up the solution We will build a set A of edges. Initially, A has no edges. As we add edges to A, maintain a loop invariant: Loop invariant: A is a subset of some MST. Add only edges that maintain the invariant. If A is a subset of some MST, an edge (u, v) is safe for A if and only if A {(u, v)} is also a subset of some MST. So we will add only safe edges.

Generic MST algorithm GENERIC-MST(G, w) A while A is not a spanning tree do find an edge (u, v) that is safe for A A A {(u, v)} return A

Finding a safe edge How do we find safe edges? Let s look at the example. Edge (c, f ) has the lowest weight of any edge in the graph. Is it safe for A =? Intuitively: Let S V be any set of vertices that includes c but not f (so that 8 8 b d g 10 2 7 5 a 9 e 9 i 3 3 12 11 c f h 1 6 In this example, there is more than one MST.

Cuts and light edges A cut (S, V S) is a partition of vertices into disjoint sets V and S V. Edge (u, v) E crosses cut (S, V S) if one endpoint is in S and the other is in V S. A cut respects A if and only if no edge in A crosses the cut. An edge is a light edge crossing a cut if and only if its weight is minimum over all edges crossing the cut. For a given cut, there can be > 1 light edge crossing it.

Kruskal s algorithm

KRUSKAL(V, E, w) A for each vertex v V do MAKE-SET(v) sort E into nondecreasing order by weight w for each (u, v) taken from the sorted list do if FIND-SET(u) = FIND-SET(v) then A A {(u, v)} UNION(u, v) return A

Prim s algorithm Builds one tree, so A is always a tree. Starts from an arbitrary root r. At each step, find a light edge crossing cut (V A, V V A ), where V A = vertices that A is incident on. Add this edge to A. V A light edge

PRIM(V, E, w, r) Q for each u V do key[u] π[u] NIL INSERT(Q, u) DECREASE-KEY(Q, r, 0) key[r] 0 while Q = do u EXTRACT-MIN(Q) for each v Adj[u] do if v Q and w(u, v) < key[v] then π[v] u DECREASE-KEY(Q, v, w(u, v)) Do example from previous graph. [Let a student pick the root.]

Single-Source Shortest Path

Shortest paths How to find the shortest route between two points on a map. Input: Directed graph G = (V, E) Weight function w : E R Weight of path p = v 0, v 1,..., v k k = w(v i 1, v i ) i=1 = sum of edge weights on path p. Shortest-path weight u to v: { { p δ(u, v) = min w(p) : u v } if there exists a path u v, otherwise. Shortest path u to v is any path p such that w( p) = δ(u, v).

Example: shortest paths from s s 0 3 5 2 t x 6 3 9 1 4 2 7 3 5 11 6 y z s 0 3 5 2 t x 6 3 9 1 4 2 7 3 5 11 6 y z This example shows that the shortest path might not be unique. It also shows that when we look at shortest paths from one vertex to all o

Negative-weight edges OK, as long as no negative-weight cycles are reachable from the source. If we have a negative-weight cycle, we can just keep going around it, and get w(s, v) = for all v on the cycle. But OK if the negative-weight cycle is not reachable from the source. Some algorithms work only if there are no negative-weight edges in the graph. We ll be clear when they re allowed and not allowed.

Output of single-source shortest-path algorithm For each vertex v V : d[v] = δ(s, v). Initially, d[v] =. Reduces as algorithms progress. But always maintain d[v] δ(s, v). Call d[v] a shortest-path estimate. π[v] = predecessor of v on a shortest path from s. If no predecessor, π[v] = NIL. π induces a tree shortest-path tree. We won t prove properties of π in lecture see text.

Initialization All the shortest-paths algorithms start with INIT-SINGLE-SOURCE. INIT-SINGLE-SOURCE(V, s) for each v V do d[v] π[v] NIL d[s] 0

Relaxing an edge (u, v) Can we improve the shortest-path estimate for v by going through u and taking (u, v)?

Relaxing an edge (u, v) Can we improve the shortest-path estimate for v by going through u and taking (u, v)? RELAX(u, v, w) if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u u v 3 3 4 10 4 6 RELAX 4 7 RELAX 4 6

The Bellman-Ford algorithm

BELLMAN-FORD(V, E, w, s) INIT-SINGLE-SOURCE(V, s) for i 1 to V 1 do for each edge (u, v) E do RELAX(u, v, w) for each edge (u, v) E do if d[v] > d[u] + w(u, v) then return FALSE return TRUE

The Bellman-Ford algorithm Allows negative-weight edges. Computes d[v] and π[v] for all v V. Returns TRUE if no negative-weight cycles reachable from s, FALSE otherwise. Core: The first for loop relaxes all edges V 1 times. Time: (V E).

The Bellman-Ford algorithm guaranteed to converge after V 1 passes, assuming no negative-weight les. cycles. Proof Use path-relaxation property. Let v be reachable from s, and let p = v 0, v 1,..., v k be a shortest path from s to v, where v 0 = s and v k = v. Since p is acyclic, it has V 1 edges, so k V 1. Each iteration of the for loop relaxes all edges: First iteration relaxes (v 0, v 1 ). Second iteration relaxes (v 1, v 2 ). kth iteration relaxes (v k 1, v k ). By the path-relaxation property, d[v] = d[v k ] = δ(s, v k ) = δ(s, v).

Dijkstra s algorithm No negative-weight edges. Essentially a weighted version of breadth-first search. Instead of a FIFO queue, uses a priority queue. Keys are shortest-path weights (d[v]). Have two sets of vertices: S = vertices whose final shortest-path weights are determined, Q = priority queue = V S. D V E s

= = DIJKSTRA(V, E, w, s) INIT-SINGLE-SOURCE(V, s) S Q V i.e., insert all vertices into Q while Q = do u EXTRACT-MIN(Q) S S {u} for each vertex v Adj[u] do RELAX(u, v, w) Looks a lot like Prim s algorithm, but compu

Single-source shortest paths in a directed acyclic graph Since a dag, we re guaranteed no negative-weight cycles. DAG-SHORTEST-PATHS(V, E, w, s) topologically sort the vertices INIT-SINGLE-SOURCE(V, s) for each vertex u, taken in topologically sorted order do for each vertex v Adj[u] do RELAX(u, v, w) Example: 6 1 s t x y z 2 7 1 2 0 2 6 5 3 4 2 Time: (V + E).

All-Pairs Shortest Paths

Given a directed graph G = (V, E), weight function w : E R, V = n. Goal: create an n n matrix of shortest-path distances δ(u, v). Could run BELLMAN-FORD once from each vertex: O(V 2 E) which is O(V 4 ) if the graph is dense (E = (V 2 )). If no negative-weight edges, could run Dijkstra s algorithm once from each vertex: O(V E lg V ) with binary heap O(V 3 lg V ) if dense, O(V 2 lg V + V E) with Fibonacci heap O(V 3 ) if dense. We ll see how to do in O(V 3 ) in all cases, with no fancy data structure.

Shortest paths a Assume that G is given as adjacency matrix of weights: W = (w ij ), with vertices numbered 1 to n. 0 if i = j, w ij = weight of (i, j) if i = j, (i, j) E, if i = j, (i, j) / E. Output is matrix =, where =. Won t worry about predeces- Optimal substructure: Recall: subpaths of shortest paths are shortest paths. m =

Floyd-Warshall algorithm

Floyd-Warshall algorithm A different dynamic-programming approach. For path p = v 1, v 2,..., v l, an intermediate vertex is any vertex of p other than v 1 or v l. Let d (k) i j = shortest-path weight of any path i j with all intermediate vertices in {1, 2,..., k}. p Consider a shortest path i j with all intermediate vertices in {1, 2,..., k}: If k is not an intermediate vertex, then all intermediate vertices of p are in {1, 2,..., k 1}. If k is an intermediate vertex: p 1 p 2 i k j all intermediate vertices in {1, 2,..., k 1}

p 1 p 2 i k j all intermediate vertices in {1, 2,..., k 1} Recursive formulation { d (k) wij if k = 0, ij = min ( d (k 1) ij, d (k 1) ik + d (k 1) ) kj if k 1. (Have d (0) ij Want D (n) = ( d (n) ij = w ij because can t have intermediate vertices 1 edge.) ), since all vertices numbered n.

= ( ) Compute bottom-up Compute in increasing order of k: FLOYD-WARSHALL(W, n) D (0) W for k 1 to n do for i 1 to n return D (n) do for j 1 to n do d (k) ij min ( d (k 1) ij, d (k 1) ik + d (k 1) ) kj

= ( ) Compute bottom-up Compute in increasing order of k: FLOYD-WARSHALL(W, n) D (0) W for k 1 to n do for i 1 to n return D (n) do for j 1 to n do d (k) ij min ( d (k 1) ij, d (k 1) ik + d (k 1) ) kj Time: (n 3 ).

Transitive closure Given G = (V, = E), directed. Compute G = (V, E ). E = {(i, j) : there is a path i j in G}. Could assign weight of 1 to each edge, then run FLOYD-WARSHALL. If d ij < n, then there is a path i j. Otherwise, d ij = and there is no path.

Simpler way: Substitute other values and operators in FLOYD-WARSHALL. Use unweighted adjacency matrix min (OR) + (AND) { t (k) 1 if there is path i j with all intermediate vertices in {1, 2,..., k}, ij = 0 otherwise. { t (0) 0 if i = j and (i, j) / E, ij = 1 if i = j or (i, j) E. t (k) ij = t (k 1) ij ( t (k 1) ik t (k 1) kj ). T -C E n

{ = ( ) TRANSITIVE-CLOSURE(E, n) for i 1 to n do for j 1 to n do if i = j or (i, j) E[G] then t (0) ij 1 else t (0) ij 0 for k 1 to n do for i 1 to n return T (n) do for j 1 to n do t (k) ij t (k 1) ij ( t (k 1) ik t (k 1) ) kj

{ = ( ) TRANSITIVE-CLOSURE(E, n) for i 1 to n do for j 1 to n do if i = j or (i, j) E[G] then t (0) ij 1 else t (0) ij 0 for k 1 to n do for i 1 to n do for j 1 to n do t (k) ij t (k 1) ij ( t (k 1) ik return T (n) ( ) t (k 1) ) kj Time: (n 3 ), but simpler operations than FLOYD-WARSHALL.