Algorithm Design and Analysis LECTURE 7 Greedy Graph Algorithms Topological sort Shortest paths Adam Smith
The (Algorithm) Design Process 1. Work out the answer for some examples. Look for a general principle Does it work on *all* your examples? 3. Write pseudocode 4. Test your algorithm by hand or computer Does it work on *all* your examples? 5. Prove your algorithm is always correct 6. Check running time Be prepared to go back to step 1!
Writing algorithms Clear and unambiguous Test: You should be able to hand it to any student in the class, and have them convert it into working code. Homework pitfalls: remember to specify data structures writing recursive algorithms: don t confuse the recursive subroutine with the first call label global variables clearly
Writing proofs Purpose Determine for yourself that algorithm is correct Convince reader Who is your audience? Yourself Your classmate Not the TA/grader Main goal: Find your own mistakes
Exploring a graph Classic problem: Given vertices s,t V, is there a path from s to t? Idea: explore all vertices reachable from s Two basic techniques: Breadth-first search (BFS) Explore children in order of distance to start node Depth-first search (DFS) How to convert these descriptions to precise algorithms? Recursively explore vertex s children before exploring siblings
Adjacency-matrix representation The adjacency matrix of a graph G = (V, E), where V = {1,,, n}, is the matrix A[1.. n, 1.. n] given by A[i, j] = A 1 3 4 1 if (i, j) E, 0 if (i, j) E. 1 1 0 1 1 0 0 0 1 0 Storage: (V ) Good for dense graphs. 3 4 3 4 0 0 0 0 0 0 1 0 Lookup: O(1) time List all neighbors: O( V )
Adjacency list representation An adjacency list of a vertex v V is the list Adj[v] of vertices adjacent to v. 1 3 4 Adj[1] = {, 3} Adj[] = {3} Adj[3] = {} Adj[4] = {3} Typical notation: n = V = # vertices m = E = # edges For undirected graphs, Adj[v] = degree(v). For digraphs, Adj[v] = out-degree(v). How many entries in lists? E ( handshaking lemma ) Total (V + E) storage good for sparse graphs.
DFS pseudocode Maintain a global counter time Maintain for each vertex v Two timestamps: v.d = time first discovered v.f = time when finished color : v.color white = unexplored gray = in process black = finished Parent v. in DFS tree
DFS pseudocode DFS.G/ for each u G:V u:color D WHITE time D 0 for each u G:V if u:color == WHITE DFS-VISIT.G; u/ DFS-VISIT.G; u/ time D time C 1 u:d D time u:color D GRAY // discover u for each G:AdjŒu // explore.u; / if :color == WHITE DFS-VISIT./ u:color D BLACK time D time C 1 u:f D time // finish u Note: recursive function different from first call Exercise: change code to set parent pointer?
DFS example d f 1 1 T 8 11 C 13 16 T T 7 B F C 9 10 T C T T 3 4 C 5 6 C C 14 15 T = tree edge (drawn in red) F = forward edge (to a descendant in DFS forest) B= back edge (to an ancestor in DFS forest) C = cross edge (goes to a vertex that is neither ancestor nor descendant)
DFS with adjacency lists DFS.G/ for each u G:V u:color D WHITE time D 0 for each u G:V if u:color == WHITE DFS-VISIT.G; u/ DFS-VISIT.G; u/ time D time C 1 u:d D time u:color D GRAY // discover u for each G:AdjŒu // explore.u; / if :color == WHITE DFS-VISIT./ u:color D BLACK time D time C 1 u:f D time // finish u Outer code runs once, takes time O(n) (not counting time to execute recursive calls) Recursive calls: Run once per vertex time = O(degree(v)) Total: O(m+n)
Toplogical Sort
Directed Acyclic Graphs Def. An DAG is a directed graph that contains no directed cycles. Ex. Precedence constraints: edge (v i, v j ) means v i must precede v j. Def. A topological order of a directed graph G = (V, E) is an ordering of its nodes as v 1, v,, v n so that for every edge (v i, v j ) we have i < j. v v 3 v 6 v 5 v 4 v 1 v v 3 v 4 v 5 v 6 v 7 v 7 v 1 a DAG a topological ordering
Precedence Constraints Precedence constraints. Edge (v i, v j ) means task v i must occur before v j. Applications. Course prerequisite graph: course v i must be taken before v j. Compilation: module v i must be compiled before v j. Pipeline of computing jobs: output of job v i needed to determine input of job v j. Getting dressed mittens shirt socks boots jacket underwear pants
Recall from book Every DAG has a topological order If G graph has a topological order, then G is a DAG.
Review Suppose your run DFS on a DAG G=(V,E) True or false? Sorting by discovery time gives a topological order Sorting by finish time gives a topological order
Shortest Paths
Shortest Path Problem Input: Directed graph G = (V, E). Source node s, destination node t. for each edge e, length (e) = length of e. length path = sum of edge lengths Find: shortest directed path from s to t. s 9 14 6 3 18 6 3 15 5 0 30 5 11 16 4 6 19 Length of path (s,,3,5,t) is 9 + 3 + + 16 = 50. 7 44 t
Rough algorithm (Dijkstra) Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known. Initialize S = { s }, d(s) = 0. Repeatedly choose unexplored node v which minimizes add v to S, and set d(v) = (v). shortest path to some u in explored part, followed by a single edge (u, v) S d(u) u (e) v s
Rough algorithm (Dijkstra) Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known. Invariant: d(u) is known Initialize S = { s }, d(s) = 0. for all vertices in S Repeatedly choose unexplored node v which minimizes add v to S, and set d(v) = (v). shortest path to some u in explored part, followed by a single edge (u, v) BFS with weighted edges S d(u) u (e) v s
Demo of Dijkstra s Algorithm Graph with nonnegative edge lengths: A 10 B D 8 1 4 7 9 3 C E
Demo of Dijkstra s Algorithm Initialize: 10 B D 0 A 8 1 4 7 9 Q: A B C D E 0 3 C E S: {}
Demo of Dijkstra s Algorithm A EXTRACT-MIN(Q): 10 B D 0 A 8 1 4 7 9 Q: A B C D E 0 3 C E S: { A }
Demo of Dijkstra s Algorithm Explore edges leaving A: 0 A 10 10 B D 8 1 4 7 9 Q: A B C D E 0 10 3 3 C S: { A } E 3
Demo of Dijkstra s Algorithm C EXTRACT-MIN(Q): 0 A 10 10 B D 8 1 4 7 9 Q: A B C D E 0 10 3 3 C S: { A, C } E 3
Demo of Dijkstra s Algorithm Explore edges leaving C: 0 A 10 7 B 11 D 8 1 4 7 9 Q: A B C D E 0 10 3 7 11 5 3 C S: { A, C } E 3 5
Demo of Dijkstra s Algorithm E EXTRACT-MIN(Q): 0 A 10 7 B 11 D 8 1 4 7 9 Q: A B C D E 0 10 3 7 11 5 3 C S: { A, C, E } E 3 5
Demo of Dijkstra s Algorithm Explore edges leaving E: 0 A 10 7 B 11 D 8 1 4 7 9 Q: A B C D E 0 10 3 7 11 5 7 11 3 C S: { A, C, E } E 3 5
Demo of Dijkstra s Algorithm B EXTRACT-MIN(Q): 0 A 10 7 B 11 D 8 1 4 7 9 Q: A B C D E 0 10 3 7 11 5 7 11 3 C S: { A, C, E, B } E 3 5
Demo of Dijkstra s Algorithm Explore edges leaving B: 0 A 10 7 B 9 D 8 1 4 7 9 Q: A B C D E 0 10 3 7 11 5 7 11 9 3 C S: { A, C, E, B } E 3 5
Demo of Dijkstra s Algorithm D EXTRACT-MIN(Q): 0 A 10 7 B 9 D 8 1 4 7 9 Q: A B C D E 0 10 3 7 11 5 7 11 9 3 C E 3 5 S: { A, C, E, B, D }
Proof of Correctness (Greedy Stays Ahead) Invariant. For each node u S, d(u) is the length of the shortest path from s to u. P P' x Proof: (by induction on S ) s Base case: S = 1 is trivial. S u v Inductive hypothesis: Assume for S = k 1. Let v be next node added to S, and let (u,v) be the chosen edge. The shortest s-u path plus (u,v) is an s-v path of length (v). Consider any s-v path P. We'll see that it's no shorter than (v). Let (x,y) be the first edge in P that leaves S, and let P' be the subpath to x. P + (x,y) has length d(x)+ (x,y) (y) (v) y
Review Question Is Dijsktra s algorithm correct with negative edge weights?
Implementation For unexplored nodes, maintain Next node to explore = node with minimum (v). When exploring v, for each edge e = (v,w), update Priority Queue Efficient implementation: Maintain a priority queue of unexplored nodes, prioritized by (v).
Priority queues Maintain a set of items with priorities (= keys ) Example: jobs to be performed Operations: Insert Increase key Decrease key Extract-min: find and remove item with least key Common data structure: heap Time: O(log n) per operation
Pseudocode for Dijkstra(G, ) d[s] 0 for each v V {s} do d[v] ; [v] S Q V Q is a priority queue maintaining V S, keyed on [v] while Q do u EXTRACT-MIN(Q) S S {u}; d[u] [u] for each v Adjacency-list[u] do if [v] > [u] + (u, v) then [v] d[u] + (u, v) explore edges leaving v Implicit DECREASE-KEY
Analysis of Dijkstra n times degree(u) times while Q do u EXTRACT-MIN(Q) S S {u} for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) Handshaking Lemma m implicit DECREASE-KEY s. PQ Operation ExtractMin DecreaseKey Dijkstra n m Array n 1 Binary heap log n log n d-way Heap HW3 HW3 Fib heap log n 1 Total n m log n m log m/n n m + n log n Individual ops are amortized bounds