ntroduction Computer Science & Engineering 2/82 Design and Analysis of Algorithms Lecture 06 Single-Source Shortest Paths (Chapter 2) Stephen Scott (Adapted from Vinodchandran N. Variyam) sscott@cse.unl.edu Given a weighted, directed graph G =(V, E) with weight function w : E! R The weight of path p = hv 0, v,...,v k i is the sum of the weights of its edges: w(p) = w(v i, v i ) Then the shortest-path weight from u to v is ( min{w(p) :u p v} if there is a path from u to v (u, v) = otherwise A shortest path from u to v is any path p with weight w(p) = (u, v) Applications: Network routing, driving directions Types of Shortest Path Problems Optimal Substructure of a Shortest Path Given G as described earlier, Single-Source Shortest Paths: Find shortest paths from source node s to every other node Single-Destination Shortest Paths: Find shortest paths from every node to destination t Can solve with SSSP solution. How? Single-Pair Shortest Path: Find shortest path from specific node u to specific node v Can solve via SSSP; no asymptotically faster algorithm known All-Pairs Shortest Paths: Find shortest paths between every pair of nodes Can solve via repeated application of SSSP, but can do better The shortest paths problem has the optimal substructure property: f p = hv 0, v,...,v k i is a SP from v 0 to v k,thenfor0apple i apple j apple k, p ij = hv i, v i+,...,v j i is a SP from v i to v j Proof: Let p = v 0 p 0i vi p ij v j p jk v k with weight w(p) =w(p 0i )+w(p ij )+w(p jk ). f there exists a path p 0 ij from v i to v j with w(pij 0 ) < w(p p 0i pij 0 p jk ij), then p is not a SP since v 0 vi v j v k has less weight than p Negative-Weight Edges () Negative-Weight Edges (2) What happens if the graph G has edges with negative weights? Dijkstra s algorithm cannot handle this, Bellman-Ford can, under the right circumstances (which circumstances?)
Cycles Relaxation What kinds of cycles might appear in a shortest path? Negative-weight cycle Zero-weight cycle Positive-weight cycle Given weighted graph G =(V, E) with source node s 2 V and other node v 2 V (v 6= s), we ll maintain d[v], which is upper bound on (s, v) Relaxation of an edge (u, v) is the process of testing whether we can decrease d[v], yielding a tighter upper bound nitialize-single-source(g, s) Relax(u, v, w) for each vertex v 2 V do 2 d[v] = [v] =nil end 5 d[s] =0 if d[v] > d[u]+w(u, v) then 2 d[v] =d[u]+w(u, v) [v] =u Relaxation Example Bellman-Ford Algorithm Works with negative-weight edges and detects if there is a negative-weight cycle Makes V passes over all edges, relaxing each edge during each pass No cycles implies all shortest paths have apple V edges, so that number of relaxations is su cient Numbers in nodes are values of d
Bellman-Ford(G, w, s) Bellman-Ford Algorithm Example () nitialize-single-source(g, s) 2 for i =to V do for each edge (u, v) 2 E do Relax(u, v, w) 5 end 6 end 7 for each edge (u, v) 2 E do 8 if d[v] > d[u]+w(u, v) then 9 return false // G has a negative-wt cycle 0 end 2 return true // G has no neg-wt cycle reachable frm s Within each pass, edges relaxed in this order: (t, x), (t, y), (t, z), (x, t), (y, x), (y, z), (z, x), (z, s), (s, t), (s, y) Bellman-Ford Algorithm Example (2) Time Complexity of Bellman-Ford Algorithm nitialize-single-source takes how much time? Relax takes how much time? What is time complexity of relaxation steps (nested loops)? What is time complexity of steps to check for negative-weight cycles? What is total time complexity? Within each pass, edges relaxed in this order: (t, x), (t, y), (t, z), (x, t), (y, x), (y, z), (z, x), (z, s), (s, t), (s, y) Correctness of Bellman-Ford: Finds SP Lengths Assume no negative-weight cycles Since no cycles appear in SPs, every SP has at most V Then define sets S 0, S,...S V : edges Correctness of Bellman-Ford: Detects Negative-Weight Cycles Let c = hv 0, v,...,v k = v 0 i be neg-weight cycle reachable from s: w(v i, v i ) < 0 S k = {v 2 V : 9s p v s.t. (s, v) =w(p) and p apple k} Loop invariant: After ith iteration of outer relaxation loop (Line 2), for all v 2 S i,wehaved[v] = (s, v) aka path-relaxation property (Lemma 2.5) Can prove via induction on i: Obvious for i =0 f holds for v 2 S i, thendefinitionofrelaxationandoptimalsubstructure ) holds for v 2 S i mplies that, after V iterations, d[v] = (s, v) for all v 2 V = S V f algorithm incorrectly returns true, then (due to Line 8) for all nodes in the cycle (i =, 2,...,k), By summing, we get d[v i ] apple d[v i ]+w(v i, v i ) d[v i ] apple d[v i ]+ w(v i, v i ) Since v 0 = v k, P k d[v i]= P k d[v i ] This implies that 0 apple P k w(v i, v i ), a contradiction
SSSPs in Directed Acyclic Graphs Why did Bellman-Ford have to run V Dag-Shortest-Paths(G, w, s) iterations of edge relaxations? To confirm that SP information fully propagated to all nodes (path-relaxation property) 6 topologically sort the vertices of G nitialize-single-source(g, s) for each vertex u 2 V, taken in topo sorted order do for each v 2 Adj[u] do Relax(u, v, w ) end 7 end 2 5 What if we knew that, after we relaxed an edge just once, we would be completely done with it? Can do this if G a dag and we relax edges in correct order (what order?) SSSP dag Example () SSSP dag Example (2) Analysis Dijkstra s Algorithm Correctness follows from path-relaxation property similar to Bellman-Ford, except that relaxing edges in topologically sorted order implies we relax the edges of a shortest path in order Topological sort takes how much time? nitialize-single-source takes how much time? How many calls to Relax? What is total time complexity? Greedy algorithm Faster than Bellman-Ford Requires all edge weights to be nonnegative Maintains set S of vertices whose final shortest path weights from s have been determined Repeatedly select u 2 V \ S with minimum SP estimate, add u to S, and relax all edges leaving u Uses min-priority queue to repeatedly make greedy choice
Dijkstra(G, w, s) Dijkstra s Algorithm Example () nitialize-single-source(g, s) 2 S = ; Q = V while Q 6= ; do 5 u = Extract-Min(Q) 6 S = S [ {u} 7 for each v 2 Adj[u] do 8 Relax(u, v, w) 9 end 0 end Dijkstra s Algorithm Example (2) Time Complexity of Dijkstra s Algorithm Using array to implement priority queue, nitialize-single-source takes how much time? What is time complexity to create Q? How many calls to Extract-Min? What is time complexity of Extract-Min? How many calls to Relax? What is time complexity of Relax? What is total time complexity? Using heap to implement priority queue, what are the answers to the above questions? When might you choose one queue implementation over another? Correctness of Dijkstra s Algorithm Linear Programming nvariant: At the start of each iteration of the while loop, d[v] = (s, v) for all v 2 S Proof: Let u be first node added to S where d[u] 6= (s, u) p Let p = s x! y p2 u be SP to u and y first node on p in V S Since y s predecessor x 2 S, d[y] = (s, y) due to relaxation of (x, y) Since y precedes u in p and edge wts non-negative: d[y] = (s, y) apple (s, u) apple d[u] Since u was chosen before y in line 5, d[u] apple d[y], so d[y] = (s, y) = (s, u) = d[u], a contradiction Since all vertices eventually end up in S, get correctness of the algorithm Given an m n matrix A and a size-m vector b and a size-n vector c, find avectorx of n elements that maximizes P n c ix i subject to Ax apple b 2 2 E.g., c = 2 22, A = 2 5, b = 5 implies: 0 8 maximize 2x x 2 subject to Solution: x = 6, x 2 =6 x + x 2 apple 22 x 2x 2 apple x 8
Di erence Constraints and Feasibility Di erence Constraints and Feasibility (2) Decision version of this problem: No objective function to maximize; simply want to know if there exists a feasible solution, i.e.,anx that satisfies Ax apple b Special case is when each row of A has exactly one and one, resulting in a set of di erence constraints of the form x j x i apple b k Applications: Any application in which a certain amount of time must pass between events (x variables represent times of events) 2 A = 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 and b = 7 6 5 0 5 7 5 Di erence Constraints and Feasibility () Constraint Graphs s there a setting for x,...,x 5 satisfying: x x 2 apple 0 x x 5 apple x 2 x 5 apple x x apple 5 x x apple x x apple x 5 x apple x 5 x apple Can represent instances of this problem in a constraint graph G =(V, E) Define a vertex for each variable, plus one more: f variables are x,...,x n, get V = {v 0, v,...,v n } Add a directed edge for each constraint, plus an edge from v 0 to each other vertex: E = {(v i, v j ):x j x i apple b k is a constraint} [{(v 0, v ), (v 0, v 2 ),...,(v 0, v n )} Weight of edge (v i, v j )isb k, weight of (v 0, v`) is0forall` 6= 0 One solution: x =( 5,, 0,, ) Constraint Graph Example x x 2 apple 0 x x 5 apple x 2 x 5 apple x x apple 5 x x apple x x apple x 5 x apple x 5 x apple ( 5,, 0,, ) Solving Feasibility with Bellman-Ford Theorem: Let G be constraint graph for system of di erence constraints. f G has a negative-weight cycle, then there is no feasible solution. f G has no negative-weight cycle, then a feasible solution is x =[ (v 0, v ), (v 0, v 2 ),..., (v 0, v n )] Proof: For any edge (v i, v j ) 2 E, triangle inequality says (v 0, v j ) apple (v 0, v i )+w(v i, v j ), so (v 0, v j ) (v 0, v i ) apple w(v i, v j ) ) x i = (v 0, v i )andx j = (v 0, v j ) satisfies constraint x i x j apple w(v i, v j ) f there is a negative-weight cycle c = hv i, v i+,...,v k = v i i,thenthere is a system of inequalities x i+ x i apple w(v i, v i+ ), x i+2 x i+ apple w(v i+, v i+2 ),..., x k x k apple w(v k, v k ). Summing both sides gives 0 apple w(c) < 0, implying that a negative-weight cycle indicates no solution Can solve with Bellman-Ford in time O(n 2 + nm)