CS420/520 Algorithm Analysis Spring 2009 Lecture 14

CS420/520 Algorithm Analysis Spring 2009 Lecture 14 "A Computational Analysis of Alternative Algorithms for Labeling Techniques for Finding Shortest Path Trees", Dial, Glover, Karney, and Klingman, Networks Volume 9 (1979) pages 215-248 Decomposition Algorithm for Shortest Paths in a Network. Journal ORSA, Vol 16, No 1, 1968, pp. 91-102. notes from Integer, Network, and Combinatorial Algorithms class, Dr. Richard Soland, professor, GWU I. Chapter 24 Single-Source Shortest Paths A. introductory comments and overview i. Professor Patrick s routing problem Indianapolis to Pheonix. ii. in shortest-paths problem we are given a weighted, directed graph G=(V,E), with weight function w:e Real Numbers mapping edges to real-valued weights iii. the weight of path p = <v 0, v 1,..., v k > is the sum of the weights of its constituent edges k w(p)= w( vi-1, vi ) i=1 p min { w(p) : u v } path from u to v iv. shortest-path weight from u to v by (u,v)= otherwise v. a path is simple if all its vertices are distinct vi. this is a generalization of breadth-first-search to handle weighted graphs (i.e., when all edge weights = 1 for BFS) vii. Variants (1) single-destination shortest-paths problem: from every vertex v to specific t destination [n paths]; reversing direction of each edge in the path can reduce to a single-source problem (2) single-pair shortest-path problem: [n 1 paths] find a shortest path from u to v for given vertices u and v; all known alg for this problem have same worst-case asymptotic running time as the best single-source alg (3) all-pairs shortest-paths problem: [n(n 1) paths] (also called multiterminal shortest path); focus of chapter 25 viii. Optimal Substructure - proof by cut and paste method (1) A shortest path between two vertices contains other shortest paths within it (2) Triangle Inequality (a) Theorem: (u,v) (u,x)+ (x,v) (this is a generalization of Lemma 24.1) page 645. ix. Negative-weight edges (Well-definedness of solution) (1) Example: edge weights represented energy absorption (negative weight) and energy release (positive weight) (2) Tarjan Theorem 7.1: Let s and t be vertices such that t is reachable from s. There is a shortest path from s to t if and only if no path from s to t contains a cycle of negative length. If there is any shortest path from s to t, there is one that is simple. Spring 2010 1

(3) if graph G contains no negative-weight cycles reachable from the source s, then for all v V, the shortest path (s,v) remains well-defined even if it has a negative value (otherwise you could always go around the negative cycle again and get a shorter path) (4) Dijkstra's assumes all non-negative weights (5) Bellman-Ford allows negative edges with no negative weight cycles are reachable from the source node. x. foundations (1) assume (a) all d ij > 0 (d ij > if there is no arc e ij in the network (nonnegativity) (b) d ij + d jk < d ik for some i,j,k (nontrivial) (c) d ij d ji for some i,j (nonsymmetric) (2) renumber the n nodes V 0, V 1,...,V n-1. We want the shortest paths from V 0 to all V i for i=1,2,...n-1 and their lengths (3) observations (a) the length of a path is always longer that the length of any of its subpaths (true because assumption (1) all costs > 0) (b) any subpaths of a shortest path must also be a shortest path (true even if some d ij < 0) (c) any shortest path contains at most (n-1) arcs (because there are no cycles and n nodes in the network) (d) the shortest path P k contains at most k arcs (i) now let P i be the shortest path from V 0 to V i (i = 1,...,n-1) and let l(p i ) be its length. Renumber the nodes so that l(p 1 ) l(p 2 )... l(p n-1 ) (ii) P 1 contains only 1 arc or there would be a subpath of P 1 that was shorter (iii) P 2 contains at most 2 arcs;... (4) to find P 1 we look at only 1 arc paths (5) to find P 2 we look at 2 or fewer arc paths (a) l(p 2 ) = d 02 one arc (b) l(p 2 ) = d 01 + d 12 two arcs (6) to find P 3 we look at (a) l(p 3 ) = d 03 (1 arc) (b) l(p 3 ) = l(p 1 ) + d 13 (2 arcs) (c) l(p 3 ) = l(p 2 ) + d 23 (3 arcs) (7) in general to find P k we see that either (a) l(p k ) = d 0k (1 arc) (b) l(p k ) = l(p j ) + d jk ( > 1 arc) with j < k in which case note that V k must be a successor to V j (8) definitions (a) V s is a successor to V r if there is an arc e rs (b) V r is a predecessor of V s if there is an arc e rs (9) thus V k must be a successor to V 0 or V 1, or... V k-1. When we find P k (and hence V k ), we can note the predecessor V j such that l(p k ) = l(p j ) + d jk xi. Cycles (1) Can a shortest path contain a cycle (a) Cannot contain a negative-weight cycle (b) Cannot contain a positive-weight cycle (c) What about 0-weight cycles (i) There must be a path between the two vertices that does not contain the 0- weight cycle Spring 2010 2

(2) For any acyclic path in a graph G=(V,E) contains at most V distinct vertices and it contains at most V -1 edges (3) We can restrict our attention to shortest paths of at most V -1 edges. xii. Representing Shortest-paths tree (1) Representation: given a graph G = (V,E), represented by an adjacency list format as in section 22.1, maintain for each vertex in V a predecessor [v] that is either another vertex or NIL (a) use PRINT-PATH algorithm page 601 to print the shortest path from s to v once the [v] have been set (2) Interested in the predecessor subgraph (a) G = (V, E ) (b) V ={v V: v. NIL} {s} (c) E = { v., v) E: v V - {s}}. (3) Definition: (page 647) let G = (V,E) be a weighted, directed graph with weight function w:e Real Numbers, and assume that G contains no negative-weight cycles reachable from the source vertex s V, so that the shortest paths are well defined. A shortest-paths tree rooted at s is a directed subgraph of G = (V,E ), where V V and E E, such that (a) V is the set of vertices reachable from s in G, (b) G forms a rooted tree with root s, and (c) For all v V, the unique simple path from s to v in G is a shortest path from s to v in G. (4) note not unique, as shown in figure 24.2 (5) Tarjan Theorem 7.2 G contains shortest paths from a source to all other vertices if and only if G contains a shortest-path tree rooted at s. (6) Tarjan Theorem 7.3 T is a shortest-path tree if and only if, for every edge [v, w], distance(v) + length(v,w) distance(w) (a) Can be used to test in O(m) time whether a given spanning tree is a shortestpath tree: We compute distance(v) for every vertex v by processing the edges in preorder, and then we test the distance inequality for each edge (b) Also suggests a way to construct shortest-path trees by iterative improvement. For each vertex v, we maintain a tentative distance dist(v) from s to v and a tentative parent p(v). We initialize dist(s) = 0, dist(v) = for v s, p(v) = null for all v, and repeat the following step until distance(v) + length(v,w) distance(w) for every edge [v, w]: (i) Labeling Step (Ford) Select an edge [v, w] such that distance(v) + length(v,w) < distance(w). Replace dist(w) by distance(v) + length(v,w) and p(w) by v. (ii) Tarjan Lemma 7.5 If at some time during the labeling method p k (v) = v for some vertex v and integer k, then the corresponding cycle in G is negative (iii) Tarjan Theorem 7.5 When the labeling method halts, the parent pointers define a shortest-path tree for the subgraph of G induced by the vertices reachable from s (iv) Algorithm can run in 2 m 1 steps, so need refinements for efficiency 1. Labeling and scanning method: partition the vertices into three states (unlabeled, labeled, and scanned). Initially s is labeled and every other vertex is unlabeled. Repeat the following step until there are no unlabeled vertices: 2. Scanning Step. Select a labeled vertex v and scan it, thereby Spring 2010 3

converting it to the scanned state, by applying the labeling step to each edge [v, w] such that distance(v) + length(v,w) < distance(w), thereby converting w to the labeled state a. Needs an efficient scanning order i. Topological used for acyclic graphs (e.g., PERT); running time O(m) including time for topological ordering. Order the vertices reachable from s so that if [v, w] is an edge, v appears before w in the order, and we apply the scanning step once to each vertex in order ii. Shortest first if there are no negative length edges; among labeled vertices, always scan one whose tentative distance in minimum; uses a d-heap of labeled vertices iii. Breadth-first among labeled vertices, scan the one least recently labeled. Needs a queue of labeled vertices. If there is no negative cycle reachable from s, then breadth-first scanning runs in O(nm) time, stopping at the end of pass n 1. Otherwise the method never halts (algorithm can be augmented to detect negative length cycles). iv. Breadth-first/depth-first hybrid xiii. labeling algorithm approach (Tarjan 87-89) (1) label the nodes with distances (a) temporary labels l k (b) permanent labels l k * (2) a permanent node is one with a permanent label (3) a temporary node is one with a temporary label (4) the permanent label on V k is the true shortest path distance from V 0 to V k (5) the temporary label on V k is the minimum of (a) d 0k (b) minimum over V j with permanent labels {l(p j ) + d jk } (6) to find P k we find the minimum temporary label of all successors of V 0, V 1,..., V k-1 and make that label (and the node) permanent. Record the predecessor node V j such that l(p k ) = l(p j ) + d jk. Update the temporary labels of all successors of the node V k that are temporary nodes (a) if l i * + d ij is less than the current temporary label of V j replace the current temporary label by the smaller value xiv. Relaxation (1) For each vertex v V, maintain an attribute d[v] which is the upper bound on the shortest path from s to v. d[v] is the shortest path estimate (2) initialize-single-source and relax pseudo code pages 648-649 xv. Properties of shortest paths and relaxation page 650 (1) relaxation either updates triangle inequality or changes nothing and in either case triangle inequality is satisfied after the relaxation step (2) relaxation initialize single source sets shortest path estimates at either their lowest value or a value that will be reduced by the relaxation step, and in either case once the shortest path estimate is at its minimum for a vertex it never changes (3) the shortest path estimate for vertex u plus the edge weight from u to v results in the shortest path estimate for vertex v B. 24.1 The Bellman-Ford Algorithm i. most basic single-source shortest-paths algorithm (1) edge weights may be negative Spring 2010 4

(2) indicates with a Boolean value if there is a negative weight cycle reachable from the source (in which case no solution exists); otherwise algorithm produces the shortest paths and their weights (3) finds shortest path weights from source s to all v V (4) actual paths can be reconstructed ii. pseudo code page 651 (1) initialize d, which will converge to the shortest path value (2) relaxation: V - 1 times, do relaxation step for each edge (3) test whether got a solution (gets a solution iff no negative-weight cycles) (4) Time = O(VE) (5) iii. Correctness (1) Pages 651-653 (2) Reliance on optimal substructure property C. 24.2 Single-source shortest paths in directed acyclic graphs i. common problem: finding shortest paths in directed acyclic graph (project management functions) ii. Algorithm page 655 DAG-SHORTEST-PATHS (1) Topological sort algorithm Section 22.4 pages 612-614, (V + E) iii. How fast can it be done? (1) Bellman-Ford takes O(VE) (2) If this is a topological sort problem, and we are lucky and process vertices on each shortest path from left to right it would be done in 1 pass (3) every path in a DAG is a subsequence of the topologically sorted vertex order, so if you do topological sort and process vertices in that order, the algorithm will do each path in forward order (it will never relax edges out of a vertex until it has done all edges into a vertex) So only 1 pass is needed (4) runs in linear O(V+E) time iv. quick comments on the longest distance V 0 to V i denoted by L i in an acyclic network (1) L 0 = 0 (2) L 1 = d 01 (3) L 2 = max[(l 0 + d 02 ), (L 1 + d 12 )] (4) L k = max over j (L j + d jk ) for (j < k) (5) d ij = - if no directed arc V i to V j (6) O(n 2 ) algorithm (7) can use for shortest path problem with negative arcs Spring 2010 5

(a) replace max by min (b) l i = shortest distance (c) d ij = implies no directed arc V i to V j D. 24.3 Dijkstra's algorithm (a greedy algorithm) i. non-negative edge weights (if no edge weights are negative, can beat Bellman-Ford) ii. solves the single-source shortest paths problem iii. like BFS; if all weights = 1 you should use BFS iv. use Q = priority queue keyed by d[v] (BFS used FIFO queue) v. pseudo code page 658 vi. notice that it is similar to Prim's MST algorithm vii. viii. Correctness using loop invariant page 659-660 (a) Note that for all v, d[v] (s,v) (b) Let u be the first vertex chosen such that there exists a shorter path than d[u] which implies that d[u] > (s,u) (c) Let y be the first vertex V S on the actual shortest path from s to u which implies that d[y] > (s,y) (d) d[u] > (s,u) = (s,y) + (y,u) by optimal substructure = d[y] + (y,u) d[y] by no negative weights but d[u] > d[y] implies that the algorithm would have chosen y to process next, not u (CONTRADICTION) ix. Analysis page 598 (1) EXTRACT-MIN executed V times (2) DECREASE-KEY executed E times (3) Time = V * T EXTRACT-MIN + E * T DECREASE-KEY Q T EXTRACT-MIN T DECREASE-KEY Total Array O(V) O(1) O(V 2 ) Binary Heap O(lgV) O(lgV) O(ElgV) Fibonacci Heap O(lgV) O(1) O(VlgV + E) Amortized amortized worst case Spring 2010 6

x. Dijkstra's algorithm (shortest path distance to all other nodes) (1) implemented using d-heaps in Tarjan page 91 Step 0 Set l i = d 0i (i = 1,2,, n-1) V 0 gets a perm label l i = if there does not exist e 0i STOP All labels Perm? Step 1 Pick l k = min over i{l i } successors of perm labeled nodes Change l k to l k *. Suppose this is node V k. Label V k perm. Step 2 successors V i of V k that have temp labels, replace l i by the min of {l i, l k * + d kj } Record the predecessor node, V k, if the temp label is updated (2) (3) do the example in the class worksheet (4) note Dijkstra's algorithm always produces a (spanning) tree (ignoring direction) (a) each iteration labels a new node (b) no cycles (c) all nodes are connected ( shortest path from node 0 to every other node) (5) amount of work done in Dijkstra's algorithm (# of adds and # of comparisons (a) in step 1 there are n-2 comparisons the first time; n-3 the second time; n-4 the third time;...; 1 the last time n-2 (n -1)(n - 2) (i) i= i=1 2 (b) in step 2 there are n-2 additions and n-2 comparisons the first time; n-3 adds and n-3 comparisons the second time;... (n -1)(n - 2) (i) additions and comparisons 2 (c) the total amount of work is on the order of (3n 2 /2) so Dijkstra's algorithm is O(n 2 ) (i) hence Dijkstra's algorithm is time-optimal (ii) Dijkstra's algorithm is Ω(n 2 ) (iii) therefor Dijkstra's algorithm is Θ(n 2 ) (6) no shortest path algorithm in general needs O(nlogn) steps because there are O(n 2 ) arcs in a network of n nodes E. 24.4. Difference constraints and shortest paths i. relationship to linear programming ii. skip this semester Spring 2010 7

F. 24.5 Proof of Shortest-Paths Properties i. The triangle inequality ii. Effects of relaxation on shortest-path estimates iii. Relaxation and shortest-path trees. II. Chapter 25 All-Pairs Shortest Paths A. Multiterminal Shortest Path Intro i. we allow some d ij 0 as long as there are no negative cycles ii. recall that any subpath of a shortest path is itself a shortest path (1) let (e ij, e jk,...,e pq ) be a shortest path from V i to V q. then the shortest path from V i to V j must be the single arc e ij,... (2) An arc e ij is a basic arc if it is the shortest path from V i to V j (3) thus a shortest path can consist only of basic arcs iii. the algorithm focuses on basic arcs and "creates" an arc between every ordered pair of nodes not connected by a basic arc. The length of the arc created (call it d*) is the length of the shortest path between the two nodes iv. the triple operation: for a given V j find minimum(d ik *, d ij * + d jk *). Replace d ik * with this minimum. We'll do this for all distinct ordered paris (i,k) such that i j and k j. We can do this successively for j=1,2,...,n. It can be shown that after we have done it for j=n, the d ij * are the lengths of the time shortest path v. we can keep track of the actual paths by using a matrix P, where at the end, p ik is the index j of V j such that the shortest path from V i to V k first goes from V i to V j. We start with all p ik = k. The rule is then (1) p ik = p ij if d ik * > d ij * + d jk * (2) p ik = p ik if d ik * d ij * + d jk * (3) set all p ii = i vi. we also keep all entries d ij * in an n x n matrix D*, where d ii = 0 and initially we have d ij * = d ij B. Text Introduction Section i. directed graph G=(V,E), weight function w:e Real Number, V = n ii. Goal: create an n x n matrix of shortest path distances (u,v) (1) u form rows and v form columns, intersection in matrix is shortest path from u to v iii. could solve by running Bellman-Ford once for each vertex. What would be the runtime? (1) O(V 2 E) or on a dense graph O(V 4 ) (2) Dijsktra's with priority queue O(V 3 ); binary heap O(V E lgv) for a sparse graph; with Fibonacci heap O(V 2 lgv + V E). iv. Adjacency-matrix representation of a graph: (1) n x n adjacency matrix W=(w ij ) of edge weights (2) assume w ii = 0 i because the shortest path to self has no edges, in absence of negative-weight cycles) v. must also keep a predecessor matrix (1) For each vertex i V define the predecessor subgraph (a) G,i = (V,i, E,i ) (b) V,i ={j V: ij NIL} {i} (c) E,i = { ( ij, j) : j = V,i - {i}}. (2) = ( ij ) where ij is NIL if i = j or there is no path from I to j, otherwise ij is the predecessor of j on some shortest path from i. (3) The subgraph induced by the ith row of should be a shortest-paths tree with root I Spring 2010 8

(4) Algorithm page 685 PRINT-ALL-PAIRS-SHORTEST-PATH C. 25.1 Shortest paths and matrix multiplication i. Steps for developing a dynamic-programming algorithm (1) characterize the structure of an optimal solution (2) recursively define the value of an optimal solution (3) compute the value of an optimal solution in a bottom-up fashion (4) construct an optimal solution from computed information ii. The structure of the shortest path (1) weight of shortest path from i to j that uses at most m edges (2) (i,j) = (i,k) + w kj iii. A recursive solution to the all-pairs shortest-paths problem (1) Recursive definition 25.2 page 687 (2) Uses solution of (m-1) edges and smallest weight connecting an (m-1) edge solution and the new node iv. Computing the shortest-path weights bottom up (1) EXTEND-SHORTEST-PATHS is the relaxation step for this problem (2) relationship to matrix multiplication (a) O(n 3 ) time for straightforward matrix multiplication (3) SLOW-ALL-PAIRS-SHORTEST-PATHS algorithm page 689 runs in (n 4 ) time v. Improving the running time (1) Recognize that the above algorithm does repeated squarings W 2n = W n x W n (2) Sequence computed is W, W 2, W 4, W 8, W lg(n-1) (3) (lgn) squarings (4) FASTER-ALL-PAIRS-SHORTEST-PATHS algorithm page 691 D. 25.2 The Floyd-Warshall algorithm (a dynamic programming algorithm) i. allows negative edge weights if no negative weight cycles exist ii. structure of the shortest path (1) use of intermediate vertices (2) we don't have to check all intermediate vertices, we just need to know if the shortest path does or does not include a particular vertex iii. A recursive solution to the all pairs shortest-paths problem (eq 25.5 page 694) iv. Computing the shortest-path weights bottom up (1) FLOYD-WARSHALL algorithm page 630 (2) runs in O(n 3 ); best time is O(V 2 lgv + VE) (3) for a dense graph ( E V 2 ) can get all pairs shortest paths with Floyd-Warshall from the same cost as getting the single source shortest path with Bellman-Ford! ( (VE) = (n 3 )) v. Constructing a shortest path (1) Requires augmenting the algorithm to update the matrix vi. Transitive Closure: is there a path in G from i to j for all ij V? (1) Substitute logical OR for minimum (2) Substitute logical AND for addition (3) Use weights of 1 E. Alternative algorithm statement Spring 2010 9

STOP Initialization yes i, j (i j) p ij = j, d ij * = d ij p ii = 1, d ii * = 0 j = 1 no j > n i, j and j i, k j, i k p ik p ij d ik * d ij * + d jk * no d ik d ij * + d jk * yes j j + 1 i. ii. do the handout example iii. amount of work done (1) there are n(n-1)(n-2) = (n 3-3n 2 + 2n) possible triple operations. Hence the algorithm is O(n 3 ) and it is a good algorithm. It is O(n 2 ) in terms of space F. 25.3 Johnson s algorithm for sparse graphs i. skip this semester Spring 2010 10