Breadth-First Search, Slides for CIS DPV Chapter Jim Royer EECS October, 00 Definition In an undirected graph, the distance between two vertices is the length of the shortest path between them. (If there is no path, the distance is.) Breadth-First Search is based on distance from the starting vertex (The distance-d vertices are all explored before the (d + )-distance ones). procedure bfs(g, s) // Input: G = (V, E), directed or undirected; s V // Output: For all verts u, dist[u] = the distance from s to u. dist[u] dist[s] = 0; Q [s] // = a queue containing just s while Q is not empty do u dequeue(q) for each v adjacent to u do if dist[v] = then enqueue(q, v); dist[v] = dist[u] + return dist Royer CIS Slides / Royer CIS Slides / Breadth-First Search, Breadth-First Search, procedure bfs(g, s) // Input: G = (V, E), s V // Output: u, dist[u] = the // distance from s to u dist[u] Q [s] while Q is not empty do u dequeue(q) for each v adjacent to u do if dist[v] = then enqueue(q, v); dist[v] = dist[u] + return dist Lemma bfs is correct. Proof outline. Proof by induction on distance d. Base case: d = 0. OK since dist[s] = 0. Induction step: d > 0. IH: bfs is correct for (d )-distant verts. When visiting the (d )-distant verts, bfs enqueues all and only the d-distant verts. each d-distant vert, u has dist[u] = d. each d-distant vert, u is visited. procedure bfs(g, s) // Input: G = (V, E), s V // Output: u, dist[u] = the // distance from s to u dist[u] Q [s] while Q is not empty do u dequeue(q) for each v adjacent to u do if dist[v] = then enqueue(q, v); dist[v] = dist[u] + return dist Lemma bfs runs in O( V + E ) time. Proof outline. Every u V enters Q at most once. Each edge in E is used at most twice (in the inner for loop). Royer CIS Slides / Royer CIS Slides /
Breadth-First Search, procedure bfs(g, s) // Input: G = (V, E), s V // Output: u, dist[u] = the F: E: // distance from s to u dist[u] Q [s] D: while Q is not empty do u dequeue(q) for each v adjacent to u do if dist[v] = then enqueue(q, v); dist[v] = dist[u] + return dist B: S:0 C: Single Source Shortest Paths Problem: Single Source Shortest Paths Given: G = (V, E), len: E R +, and s V. (G directed or undirected) Find: For each u, dist[u] = the shortest distance from s. 0 Royer CIS Slides / Royer CIS Slides / Dijkstra s Algorithm, Digression: Priority Queues Problem: Single Source Shortest Paths Given: G = (V, E), len: E R +, and s V. (G directed or undirected) Find: For each u, dist[u] = the shortest distance from s. // v has priority dist[v] dist[v] dist[u] + len(u, v) decreasekey(pq, v, dist[v]) Dijkstra s algorithm BFS with a priority queue dist[u] = the shortest distance through solved vertices Always choose the unsolved u with minimal dist[u] Declare the u solved and update dist[v] for v adjacent to u Each element has two fields: datum and priority (an integer) Priority queue operations: newqueue(... ): makes a new priority queue deletemin(pq): returns item with smallest priority and removes it from the priority queue add(pq,d,p): add a datum d with priority p decreasekey(pq,d,p): change the priority of datum d to p More later. Royer CIS Slides / Royer CIS Slides 8 /
Dijkstra s Algorithm, Dijkstra s Algorithm, 0 0 node dist 0 prev null null null null null null node dist 0 prev null null null Royer CIS Slides / Royer CIS Slides 0 / Dijkstra s Algorithm, Dijkstra s Algorithm, 0 0 node dist 0 prev null node dist 0 prev null Royer CIS Slides / Royer CIS Slides /
Dijkstra s Algorithm, Dijkstra s Algorithm, 0 0 node dist 0 8 prev null node dist 0 8 prev null Royer CIS Slides / Royer CIS Slides / Dijkstra s Algorithm, 8 Dijkstra s Algorithm, 0 node dist 0 8 prev null while (PQ is not empty) do Lemma For each step of the while-loop: (a) If w is not in PQ, then dist[w] = the shortest distance from s to w. (b) If w is in PQ, then dist[w] = the shortest distance from s to w in which all the vertices on the path (except w) are not in PQ. Proof: Base case. Initially, all vertices are in PQ, dist[s] = 0, and for w = s, dist[w] =. (a) holds vacuously (as all verts are in PQ). Part (b) also holds. (Why?) Royer CIS Slides / Royer CIS Slides /
Dijkstra s Algorithm, 0 Dijkstra s Algorithm, while (PQ is not empty) do Lemma For each step of the while-loop: (a) If w is not in PQ, then dist[w] = the shortest distance from s to w. (b) If w is in PQ, then dist[w] = the shortest distance from s to w in which all the vertices on the path (except w) are not in PQ. Proof: Induction step. Induction Hypothesis: The Lemma holds up to the current step of the while-loop. Suppose u is the vertex deletemin removes from PQ in this step. Claim : (a) holds for this u. Why? Draw the picture! Claim : (b) also holds for this step. Why? Draw the picture! while (PQ is not empty) do Runtime analysis. This depends on the implementation of priority queue used. Royer CIS Slides / Royer CIS Slides 8 / Simply implementing Priority Queues via an Array A simple minded array implementation. PQ[..n] : array of items last : index Implementing Priority Queues via a Binary Min-Heap Definition A binary tree has the min-heap property when each node has the min-value in the subtree below it. v add(pq,dat,p): last last+; PQ[item] newitem(dat, p) decreasekey(pq,i,p): PQ[i].priority p Θ() time Θ() time For example: values >= v values >= v deletemin(pq): Find and remove the item with smallest priority O(n) time Royer CIS Slides / Royer CIS Slides 0 /
Binary Min-Heap: Adding an Element & DecreaseKey Binary Min-Heap: DeleteMin procedure bubbleup(ptr) while (ptr = root) & (ptr s key ptr s parent s key) do swap ptr with its parent ptr ptr s parent procedure add(ky) ptr a new leaf node in the tree ptr.key ky bubbleup(ptr) procedure decreasekey(ptr, nk) ptr.key nk bubbleup(ptr) procedure sinkdown(ptr) q ptr; l ptr.left; r ptr.right while (q is not a leaf) do if l = null & q.key > l.key then q l if r = null & q.key > r.key then q r if q = ptr then return swap ptr.key and q.key ptr q procedure deletemin() if the heap is empty then error v root.key Pick a leaf node p root.key p.key delete p from the tree sinkdown(p) Animation: http://people.ksp.sk/~kuko/bak/index.html Royer CIS Slides / Royer CIS Slides / Binary Min-Heap: Complete Binary Trees Dijkstra s Algorithm, Run Time Analysis Definition A complete binary tree is one where all levels are complete except perhaps for the lowest level which is filled in left to right. Fact: A complete binary tree of n elements is of height + log n. If the heap tree is complete, then add: O(log n) time decreasekey: O(log n) time deletemin: O(log n) time In fact: There is a very simple array representation of a min-heap. (See Text.) while (PQ is not empty) do Runtime analysis. If a binary min-heap is used to implement priority queues, the run time is O(( V + E ) log V ). (Why?) Using Fibonacci heaps can get this down to O( V log V + E ). (See text.) Royer CIS Slides / Royer CIS Slides /
Negative weight edges Relaxation Dijkstra s Algorithm breaks if edges are allows to have negative weights.... and shortest paths may not exists if we have negative weight cycles. Consider: B A 00 - E D - - - procedure for each v V do dist[v] ; prev[v] null // Try improving dist[v] via (u, v) E // for Dijkstra s Alg, add: // while (PQ is not empty) do relax(v, u, len) C F Royer CIS Slides / Royer CIS Slides / Properties of Shortest Paths and Relaxation, Notation: u v there is no path from u to v u v there is a path from u to v u w v a path from u to v with final edge (w, v) δ(u, v) = the length of a shortest path from u to v. (= u v) Triangle inequality For each (u, v) E, δ(s, v) δ(s, u) + len(u, v). Why: Any s u v cannot be any smaller than δ(s, v). Upper-bound property For all v V, dist[v] δ(s, v) at every point in the algorithm. Why: If dist[v] <, then it s value comes from finding the length of some path s v. Properties of Shortest Paths and Relaxation, Notation: u v there is no path from u to v u v there is a path from u to v u w v a path from u to v with final edge (w, v) δ(u, v) = the length of a shortest path from u to v. (= if u v) No-path property If s v, then we always have dist[v] = δ(s, v) =. Why: (dist[v] < ) the algorithm found some path s v. Convergence property If s u v is a shortest path in G and if dist[u] = δ(s, u) before relaxing (u, v), then dist[v] = δ(s, v) afterwards. Why: Shortest paths have optimal substructure: Subpaths of shortest paths are themselves shortest paths. Royer CIS Slides / Royer CIS Slides 8 /
Properties of Shortest Paths and Relaxation, Notation: u v there is no path from u to v u v there is a path from u to v u w v a path from u to v with final edge (w, v) δ(u, v) = the length of a shortest path from u to v. (= if u v) Convergence property If s u v is a shortest path in G and if dist[u] = δ(s, u) before relaxing (u, v), then dist[v] = δ(s, v) afterwards. Properties of Shortest Paths and Relaxation, Notation: u v there is no path from u to v u v there is a path from u to v u w v a path from u to v with final edge (w, v) δ(u, v) = the length of a shortest path from u to v. (= if u v) Convergence property If s u v is a shortest path in G and if dist[u] = δ(s, u) before relaxing (u, v), then dist[v] = δ(s, v) afterwards. Path-relaxation property If s, v,..., v k is a shortest path & (s, v ), (v, v ),..., (v k, v k ) are relaxed in that order, then dist[v k ] = δ(s, v k ). Why: This follows from the Convergence property. Predecessor-subgraph property Once dist[v] = δ(s, v) for each v V, then the prev[v] s give a shortest-path tree rooted at s. Why: This also follows from the Convergence property. Royer CIS Slides / Royer CIS Slides 0 / Bellman-Ford: Just Relax (a lot) Bellman-Ford: Example, procedure for each v V do dist[v] ; prev[v] null // Try improving dist[v] via (u, v) E Run time analysis: Θ( V E ) time. for i to V do relax(u, v, len) (Why?) procedure for each v V do dist[v] ; prev[v] null ; for i to V do relax(u, v, len) Just before the relaxation loop Images from http://areshsaharkhiz.blogspot.com/ Royer CIS Slides / Royer CIS Slides /
Bellman-Ford: Example, Bellman-Ford: Example, procedure for each v V do dist[v] ; prev[v] null procedure for each v V do dist[v] ; prev[v] null ; ; for i to V do relax(u, v, len) for i to V do relax(u, v, len) Just after the st pass through the relaxation loop Just after the nd pass through the relaxation loop Royer CIS Slides / Royer CIS Slides / Bellman-Ford: Example, Bellman-Ford: Example, procedure for each v V do dist[v] ; prev[v] null ; procedure for each v V do dist[v] ; prev[v] null ; for i to V do relax(u, v, len) for i to V do relax(u, v, len) Just after the nd pass through the relaxation loop Just after the th pass through the relaxation loop Royer CIS Slides / Royer CIS Slides /
Bellman-Ford: Correctness, Bellman-Ford: Correctness, for i to V do // Relax loop relax(u, v, len) // Neg.wght s? Convergence property If s u v is a shortest path in G and if dist[u] = δ(s, u) before relaxing (u, v), then dist[v] = δ(s, v) afterwards. Path-relaxation property If P = s, v,..., v k is a shortest path & (s, v ), (v, v ),..., (v k, v k ) are relaxed in that order, then dist[v k ] = δ(s, v k ). Case: There are no negative weight cycles in G. Suppose v 0, v, v,..., v k is a shortest path in G where s = v 0. Note: k V. (Why?) After the i-th iteration of the relaxation loop, (v i, v i ) is relaxed. So by the Path-relaxation and convergence properties, after iteration k, dist[v k ] = δ(s, v k ). Moreover, Bellman-Ford returns (TRUE, dist) for i to V do // Relax loop relax(u, v, len) // Neg.wght s? Convergence property If s u v is a shortest path in G and if dist[u] = δ(s, u) before relaxing (u, v), then dist[v] = δ(s, v) afterwards. Path-relaxation property If P = s, v,..., v k is a shortest path & (s, v ), (v, v ),..., (v k, v k ) are relaxed in that order, then dist[v k ] = δ(s, v k ). Claim: G has a neg. wght cycle B.F. returns (FALSE, dist) Proof: Suppose there is a neg. wght cycle. Then every time we relax each edge in G, some dist[v] improves. (Why?) The function returns (FALSE, dist). Suppose B.F. returns (FALSE, dist) Then some length- V path to a vertex gives a cheaper path than the length-( V ) paths. Hence, this length- V path must include a cycle. Since this length- V path gave an improvement, the cycle must be negative weight. Royer CIS Slides / Royer CIS Slides 8 /