Shortest Paths Nishant Mehta Lectures 0 and
Communication Speeds in a Computer Network Find fastest way to route a data packet between two computers 6 Kbps 4 0 Mbps 6 6 Kbps 6 Kbps Gbps 00 Mbps 8 6 Kbps Gbps 00 Mbps 0 0 Mbps 00 Mbps Gbps 7 00 Mbps 9
Shortest Paths in Weighted Graphs Find fastest way to travel across the country using a graph representing roads, with edge weights represented by: distances travel times between cities (might account for speed limits, traffic, etc.) Find a fastest way using flights Flight distances between airports. Might also allow for warps in space-time continuum. Negative travel time
Single-Source Shortest Paths If graph is unweighted: Breadth-First Search is a solution More on this soon If graph is weighted Every edge is associated with a number: integers, rational numbers, real numbers (maybe even negative!) An edge weight can represent: Distance, connection cost, affinity
Single-Source Shortest Paths Problem Input: A weighted directed graph G = (V, E) and a source vertex s Output: All single-source shortest paths for s in G, i.e. for all other vertices v in G, a shortest path from s to v. A path p =(v 0,v,...,v k ) from s = v 0 to v = v k is shortest if its length w(p) = P k j= w(v j,v j ) is the minimum possible among all paths
Dijkstra s Algorithm
Dijkstra s algorithm: a greedy algorithm 9 4 7
Dijkstra s algorithm: Initializing 0 4 9 7
Dijkstra s algorithm: Initializing Cloud C (consisting of solved subgraph) 0 4 9 7
Dijkstra s algorithm: pull v into C 0 4 9 7
Dijkstra s algorithm: update C s neighborhood 0 4 9 4 7
Dijkstra s algorithm: pick closest vertex u outside C 0 4 9 4 7
Dijkstra s algorithm: pull u into C 0 4 9 4 relax 7
Dijkstra s algorithm: update C s neighborhood 0 4 9 7
Dijkstra s algorithm: pick closest vertex u outside C 0 4 9 7
Dijkstra s algorithm: pull u into C 0 4 9 7
Dijkstra s algorithm: update C s neighborhood 0 4 9 7
Dijkstra s algorithm: pick closest vertex u outside C 0 4 9 7
Dijkstra s algorithm: pull u into C 0 4 9 7
Dijkstra s algorithm: update C s neighborhood 0 4 9 7 0
Dijkstra s algorithm: pick closest vertex u outside C 0 4 9 7 0
Dijkstra s algorithm: pull u into C 0 4 4 9 7 0
Dijkstra s algorithm: update C s neighborhood 0 4 4 9 7 0
Dijkstra s algorithm: pick closest vertex u outside C 0 4 4 9 7 0
Dijkstra s algorithm: pull u into C 0 4 relax 4 9 7 0
Dijkstra s algorithm: update C s neighborhood 0 4 8 4 9 7 0
Dijkstra s algorithm: pick closest vertex u outside C 0 4 8 4 9 7 0
Dijkstra s algorithm: pull u into C 0 4 8 4 9 7 0
Dijkstra s algorithm: update C s neighborhood 0 4 8 4 9 7 0
Dijkstra s algorithm: pick closest vertex u outside C 0 4 8 4 9 7 0
Dijkstra s algorithm: pull u into C 0 4 8 4 9 7 0
Dijkstra s algorithm: update C s neighborhood 0 4 8 4 9 7 0
Dijkstra s algorithm: pick closest vertex u outside C 0 4 8 4 9 7 0
Dijkstra s algorithm: pull u into C 0 4 8 4 9 7 0
Dijkstra s algorithm: update C s neighborhood 0 4 8 4 9 7 0
Dijkstra s algorithm: pick closest vertex u outside C 0 4 8 4 9 7 0
Dijkstra s algorithm: pull u into C 0 4 8 4 9 7 0
Dijkstra s Algorithm Input: A simple directed graph G with nonnegative edge-weights and a source vertex s in G Output: A number d[u] for each vertex u in G such that d[u] is the weight of the shortest path in G from s to u
Dijkstra s Algorithm Dijkstra(V,E,s): For v in V d[v] = ; [v] = null; d[s] = 0 S = ; Q = BuildPriorityQueue(V, d) While Q not empty u = ExtractMin(Q) S = S [ u For v in Adj[u] Relax(u,v)
Dijkstra vs Prim Dijkstra(V,E,s): For v in V d[v] = ; [v] = null; d[s] = 0 S = ; Q = BuildPriorityQueue(V, d) While Q not empty u = ExtractMin(Q) S = S [ u For v in Adj[u] If d[u] + w(u,v) < d[v] d[v] = d[u] + w(u,v) [v] = u UpdatePQ(v, [v]) Prim(V,E,s): For v in V d[v] = ; [v] = null; d[s] = 0 S = ; Q = BuildPriorityQueue(V, d) While Q not empty u = ExtractMin(Q) S = S [ u For v in Adj[u] If w(u,v) < d[v] d[v] = w(u,v) [v] = u UpdatePQ(v, [v])
Dijkstra vs Prim Dijkstra(V,E,s): For v in V d[v] = ; [v] = null; d[s] = 0 S = ; Q = BuildPriorityQueue(V, d) While Q not empty u = ExtractMin(Q) S = S [ u For v in Adj[u] If d[u] + w(u,v) < d[v] d[v] = d[u] + w(u,v) [v] = u UpdatePQ(v, [v]) Prim(V,E,s): For v in V d[v] = ; [v] = null; d[s] = 0 S = ; Q = BuildPriorityQueue(V, d) While Q not empty u = ExtractMin(Q) S = S [ u For v in Adj[u] If w(u,v) < d[v] d[v] = w(u,v) [v] = u UpdatePQ(v, [v])
Dijkstra s Algorithm - Runtime Dijkstra(V,E,s): V calls at most E calls For v in V d[v] = ; [v] = null; d[s] = 0 S = ; Q = BuildPriorityQueue(V, d) While Q not empty u = ExtractMin(Q) S = S [ u For v in Adj[u] If d[u] + w(u,v) < d[v] d[v] = d[u] + w(u,v) [v] = u UpdatePQ(v, [v]) O(V) for binary or Fibonacci heap O(log V)/call for binary heap O()/call for Fibonacci heap O(log V)/call for binary or Fibonacci heaps
RELAX preserves upper bound property of d[v] Upper bound property: Any sequence of calls to RELAX maintains the invariant that d[v] (s, v) for all v V Proof: simple exercise Important consequence: If no path from s to v, then d[v] = always!
Dijkstra s Algorithm - Correctness Claim: for all v in S, the algorithm s path shortest s-v path Proof by induction Base case: S =, with S = {s} P v from s-v is a Clearly, P s = (s) is a shortest s-s path (of length zero!) Inductive step Suppose the claim holds for S = k Prove that it holds for S = k +
Dijkstra s Algorithm - Correctness Let S = k and suppose algorithm is about to add v to S, by way of u in S Let P v be the algorithm s s-v path after the addition, with penultimate vertex u in S Consider an arbitrary alternative path P 0 v Pv 0 has a first edge (x,y) that crosses the cut (S, V \ S) w(pv 0 ) (s, x)+w(x, y) = d[x]+w(x, y) S x y d[u]+w(u, v) = (s, u)+w(u, v) s u v
Dijkstra s Algorithm - Correctness Consider an arbitrary alternative path Pv 0 has a first edge (x,y) that crosses the cut S x y P 0 v (S, V \ S) s u v P 0 v Path cannot be shorter than P v w(pv 0 ) (s, x)+w(x, y) = d[x]+w(x, y) d[u]+w(u, v) = (s, u)+w(u, v) = P v (inductive hypothesis) (v is next vertex added to S) (inductive hypothesis)
Dijkstra s Algorithm - Negative Weights v - u s What would Dijkstra do?
Greed is good. -Gordon Gekko
Greed is good. -Gordon Gekko ``Greed is not good (when a graph has negative edge weights). ' - Bernie Sanders
Bellman-Ford Algorithm
Recall: Path Relaxation Property Let p = (v 0, v, v k ) be a shortest path from v 0 to v k Initialize d and with source s Suppose that a sequence of relaxations occurs which includes the subsequence: RELAX(v 0, v ), RELAX(v, v ),, RELAX(v k-, v k ) Then after the last relaxation in this subsequence and for all times thereafter, we have d[v k ]= (s, v k ) (Proved last time)
Bellman-Ford Algorithm An Observation: Suppose shortest path from vertex s to vertex t consists of edge: p = (v 0, v ) with s = v 0 and t = v Then after calling RELAX(v 0, v ): d(t) =d(v )= (v 0, v )= (s, t) Shortest path from s to t has been found! How to ensure RELAX(v 0, v ) gets called? Initialize d and with source s For each edge (u,v) E RELAX(u,v)
Bellman-Ford Algorithm An Observation: Suppose shortest path from vertex s to vertex t consists of edges: p = (s = v 0, v, v = t) Then after calling RELAX(v 0, v ), RELAX(v, v ): d(t) =d(v )= (v 0, v )= (s, t) Shortest path from s to t has been found! How to ensure RELAX(v 0, v ), RELAX(v, v ) gets called? Initialize d and For j = with source s For each edge (u,v) RELAX(u,v) E
Bellman-Ford Algorithm If no negative cycles, shortest path from vertex s to vertex t consists of (at most) n- edges: p = (v 0, v,, v k ) with k apple n- After calling RELAX(v 0, v ), RELAX(v, v ),, RELAX(v k-, v k ): d(t) =d(v k )= (v 0, v k )= (s, t) Shortest path from s to t has been found! How to ensure subsequence of calls to RELAX(v 0, v ) occurs? Initialize d and with source s For j = n- For each edge (u,v) RELAX(u,v) E
Bellman-Ford Algorithm BELLMAN-FORD(G, w, s) Initialize d and with source s For j = n- For each edge (u,v) RELAX(u,v) For each edge (u,v) E E RELAX(u,v) If d[u] + w(u,v) < d[v] d[v] = d[u] + w(u,v) If d[v] > d[u] + w(u,v) Return False Return True
Bellman-Ford Algorithm - Correctness Claim : (A) If there are no negative cycles, the algorithm correctly finds the shortest paths ( d[v] = (s, v) for all v) and predecessor array is correct. (B) The algorithm returns True. Claim : If there is a negative cycle, the algorithm detects it and returns False
Bellman-Ford Algorithm - Correctness Claim : (A) If there are no negative cycles, the algorithm correctly finds the shortest paths ( d[v] = (s, v) for all v) and predecessor array is correct. Proof: This we already showed in the derivation of the algorithm! The desired subsequence of calls to RELAX occurs, which is all that is required.
Bellman-Ford Algorithm - Correctness Claim : (B) The algorithm returns True Proof: We only need to verify that d[v] apple d[u]+w(u, v) for all edges (u, v) E From Claim (A), this is equivalent to (s, v) apple (s, u)+w(u, v) for all edges (u, v) E This must be the case. Why? An s-v path that first visits u and then follows edge (u,v) cannot have less weight than the shortest s-v path
Bellman-Ford Algorithm - Correctness Claim : Proof: If there is a negative cycle, the algorithm detects it and returns False Suppose (for a contradiction) that the algorithm returns True Then d[v] apple d[u]+w(u, v) for all edges (u, v) E ( ) Let the negative cycle be (v 0, v,..., v k ), where v 0 = v k Summing inequality ( ) over each edge in the cycle yields: kx j= d[v j ] apple kx (d[v j ]+w(v j, v j )) j=
Bellman-Ford Algorithm - Correctness (Proof of Claim ) Suppose (for a contradiction) that the algorithm returns True Then d[v] apple d[u]+w(u, v) for all edges (u, v) E ( ) Let the negative cycle be (v 0, v,..., v k ), where v 0 = v k Summing inequality ( ) over each edge in the cycle yields: kx kx d[v j ] apple (d[v j ]+w(v j, v j )) j= j= But since v 0 = v k, it holds that kx d[v j ]= kx d[v j ] j= j= The above implies that 0 apple P k j= w(v j, v j ), a contradiction of the cycle being negative!
Bellman-Ford Algorithm - Time Complexity O(V E) (For loop from to n-) * (Nested for loop over edges) Compare to Dijkstra s algorithm O(E log V) Dealing with negative-weight edges has a cost!
Single-Source Shortest Paths Algorithms Type of Graph Algorithm Time complexity Unweighted graph BFS O(V + E) Weighted DAG Topological sort/dfs-based O(V + E) Weighted directed graph (nonnegative weights) Weighted directed graph (any weights) Dijkstra s Algorithm O(E log V) Bellman-Ford O(V E)
All-Pairs Shortest Paths Problem Input: A weighted directed graph G = (V, E) Output: All shortest paths in G, i.e. for all pairs of vertices s, t in V, a shortest path from s to t
All-Pairs Shortest Paths First approach - run single-source shortest paths algorithm n times, once per choice of source vertex Type of Graph Algorithm Time complexity Dense Graph Time complexity Nonnegative weights Dijkstra s - Binary heap O(V E log V) O(V log V) Dijkstra s - Fibonacci heap O(V log V + E V) O(V log V) Any weights Bellman-Ford O(V E) O(V 4 )
All-Pairs Shortest Paths Need to store upper bound on shortest paths for every pair of vertices Switch from array d to matrix D of size n n D ij = upper bound on shortest path from i to j Switch from predecessor array to predecessor matrix ij = predecessor of j in some shortest path from source i
Floyd-Warshall Algorithm
Floyd-Warshall Algorithm Let p be a shortest path from i to j. Clearly, all intermediate vertices in path p are in {,, n} Also, we can break down p into at most paths whose intermediate vertices are in {,, n-} i p j all intermediate vertices in {,, n} Case Case i p j p n p all intermediate vertices in {,, n-} i all intermediate vertices in {,, n-} j
Floyd-Warshall Algorithm For k = 0,,, n: Let be the weight of the shortest path from i to j for which all intermediate vertices are in {,, k} D (k) ij p i j all intermediate vertices in {,, n} Case Case i p j p n p all intermediate vertices in {,, n-} i all intermediate vertices in {,, n-} j D (n) ij = D (n ) ij D (n) (n ) ij = D in + D (n ) nj
All-Pairs Shortest Paths For k = 0,,, n: Let be the weight of the shortest path from i to j for which all intermediate vertices are in {,, k} D (k) ij p i j all intermediate vertices in {,, k} Case Case i p j p k p all intermediate vertices in {,, k-} i all intermediate vertices in {,, k-} j D (k) ij = D (k ) ij D (k) (k ) ij = D ik + D (k ) kj
Floyd-Warshall Algorithm Recurrence: D (k) ij =min n (k ) (k ) D ij, D ik + D (k ) kj o Base case: D (0) ij = w(i, j) Why? Because no intermediate vertices can be used FLOYD-WARSHALL( ) D (0) = W W For k = n For i = n Return For j = n n D (k) (k ) (k ) ij =min D ij, D ik + D D (n) (k ) kj o
Floyd-Warshall Algorithm FLOYD-WARSHALL( ) D (0) = W W Time Complexity O(V ) For k = For i = Return n n For j = n n D (k) (k ) (k ) ij =min D ij, D ik + D D (n) (k ) kj o Correctness? ij is weight of shortest path with intermediate vertices in {,, n}. This is shortest path itself! D (n)
Floyd-Warshall Algorithm What about that predecessor matrix? How do we print a shortest path? (k ) D Case : ik + D Path will not change (k ) (k ) kj D ij Reuse predecessor from before: (k) ij = (k ) ij (k ) D (k ) (k ) Case : ik + D kj < D ij Path updates to (path from i to k) (path from k to j) (k) (k ) ij = kj Set predecessor of j in shortest path from source i using intermediate vertices in {,, k} to be predecessor of j in shortest path from source k using intermediate vertices in {,, k-}