Algorithm Design and Analysis

Transcription

1 Algorithm Design and Analysis LECTURE 7 Greedy Graph Algorithms Topological sort Shortest paths Adam Smith

2 The (Algorithm) Design Process 1. Work out the answer for some examples. Look for a general principle Does it work on *all* your examples? 3. Write pseudocode 4. Test your algorithm by hand or computer Does it work on *all* your examples? 5. Prove your algorithm is always correct 6. Check running time Be prepared to go back to step 1!

3 Writing algorithms Clear and unambiguous Test: You should be able to hand it to any student in the class, and have them convert it into working code. Homework pitfalls: remember to specify data structures writing recursive algorithms: don t confuse the recursive subroutine with the first call label global variables clearly

4 Writing proofs Purpose Determine for yourself that algorithm is correct Convince reader Who is your audience? Yourself Your classmate Not the TA/grader Main goal: Find your own mistakes

5 Exploring a graph Classic problem: Given vertices s,t V, is there a path from s to t? Idea: explore all vertices reachable from s Two basic techniques: Breadth-first search (BFS) Explore children in order of distance to start node Depth-first search (DFS) How to convert these descriptions to precise algorithms? Recursively explore vertex s children before exploring siblings

6 Adjacency-matrix representation The adjacency matrix of a graph G = (V, E), where V = {1,,, n}, is the matrix A[1.. n, 1.. n] given by A[i, j] = A if (i, j) E, 0 if (i, j) E Storage: (V ) Good for dense graphs Lookup: O(1) time List all neighbors: O( V )

7 Adjacency list representation An adjacency list of a vertex v V is the list Adj[v] of vertices adjacent to v Adj[1] = {, 3} Adj[] = {3} Adj[3] = {} Adj[4] = {3} Typical notation: n = V = # vertices m = E = # edges For undirected graphs, Adj[v] = degree(v). For digraphs, Adj[v] = out-degree(v). How many entries in lists? E ( handshaking lemma ) Total (V + E) storage good for sparse graphs.

8 DFS pseudocode Maintain a global counter time Maintain for each vertex v Two timestamps: v.d = time first discovered v.f = time when finished color : v.color white = unexplored gray = in process black = finished Parent v. in DFS tree

9 DFS pseudocode DFS.G/ for each u G:V u:color D WHITE time D 0 for each u G:V if u:color == WHITE DFS-VISIT.G; u/ DFS-VISIT.G; u/ time D time C 1 u:d D time u:color D GRAY // discover u for each G:AdjŒu // explore.u; / if :color == WHITE DFS-VISIT./ u:color D BLACK time D time C 1 u:f D time // finish u Note: recursive function different from first call Exercise: change code to set parent pointer?

10 DFS example d f 1 1 T 8 11 C T T 7 B F C 9 10 T C T T 3 4 C 5 6 C C T = tree edge (drawn in red) F = forward edge (to a descendant in DFS forest) B= back edge (to an ancestor in DFS forest) C = cross edge (goes to a vertex that is neither ancestor nor descendant)

11 DFS with adjacency lists DFS.G/ for each u G:V u:color D WHITE time D 0 for each u G:V if u:color == WHITE DFS-VISIT.G; u/ DFS-VISIT.G; u/ time D time C 1 u:d D time u:color D GRAY // discover u for each G:AdjŒu // explore.u; / if :color == WHITE DFS-VISIT./ u:color D BLACK time D time C 1 u:f D time // finish u Outer code runs once, takes time O(n) (not counting time to execute recursive calls) Recursive calls: Run once per vertex time = O(degree(v)) Total: O(m+n)

12 Toplogical Sort

13 Directed Acyclic Graphs Def. An DAG is a directed graph that contains no directed cycles. Ex. Precedence constraints: edge (v i, v j ) means v i must precede v j. Def. A topological order of a directed graph G = (V, E) is an ordering of its nodes as v 1, v,, v n so that for every edge (v i, v j ) we have i < j. v v 3 v 6 v 5 v 4 v 1 v v 3 v 4 v 5 v 6 v 7 v 7 v 1 a DAG a topological ordering

14 Precedence Constraints Precedence constraints. Edge (v i, v j ) means task v i must occur before v j. Applications. Course prerequisite graph: course v i must be taken before v j. Compilation: module v i must be compiled before v j. Pipeline of computing jobs: output of job v i needed to determine input of job v j. Getting dressed mittens shirt socks boots jacket underwear pants

15 Recall from book Every DAG has a topological order If G graph has a topological order, then G is a DAG.

16 Review Suppose your run DFS on a DAG G=(V,E) True or false? Sorting by discovery time gives a topological order Sorting by finish time gives a topological order

17 Shortest Paths

18 Shortest Path Problem Input: Directed graph G = (V, E). Source node s, destination node t. for each edge e, length (e) = length of e. length path = sum of edge lengths Find: shortest directed path from s to t. s Length of path (s,,3,5,t) is = t

19 Rough algorithm (Dijkstra) Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known. Initialize S = { s }, d(s) = 0. Repeatedly choose unexplored node v which minimizes add v to S, and set d(v) = (v). shortest path to some u in explored part, followed by a single edge (u, v) S d(u) u (e) v s

20 Rough algorithm (Dijkstra) Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known. Invariant: d(u) is known Initialize S = { s }, d(s) = 0. for all vertices in S Repeatedly choose unexplored node v which minimizes add v to S, and set d(v) = (v). shortest path to some u in explored part, followed by a single edge (u, v) BFS with weighted edges S d(u) u (e) v s

21 Demo of Dijkstra s Algorithm Graph with nonnegative edge lengths: A 10 B D C E

22 Demo of Dijkstra s Algorithm Initialize: 10 B D 0 A Q: A B C D E 0 3 C E S: {}

23 Demo of Dijkstra s Algorithm A EXTRACT-MIN(Q): 10 B D 0 A Q: A B C D E 0 3 C E S: { A }

24 Demo of Dijkstra s Algorithm Explore edges leaving A: 0 A B D Q: A B C D E C S: { A } E 3

25 Demo of Dijkstra s Algorithm C EXTRACT-MIN(Q): 0 A B D Q: A B C D E C S: { A, C } E 3

26 Demo of Dijkstra s Algorithm Explore edges leaving C: 0 A 10 7 B 11 D Q: A B C D E C S: { A, C } E 3 5

27 Demo of Dijkstra s Algorithm E EXTRACT-MIN(Q): 0 A 10 7 B 11 D Q: A B C D E C S: { A, C, E } E 3 5

28 Demo of Dijkstra s Algorithm Explore edges leaving E: 0 A 10 7 B 11 D Q: A B C D E C S: { A, C, E } E 3 5

29 Demo of Dijkstra s Algorithm B EXTRACT-MIN(Q): 0 A 10 7 B 11 D Q: A B C D E C S: { A, C, E, B } E 3 5

30 Demo of Dijkstra s Algorithm Explore edges leaving B: 0 A 10 7 B 9 D Q: A B C D E C S: { A, C, E, B } E 3 5

31 Demo of Dijkstra s Algorithm D EXTRACT-MIN(Q): 0 A 10 7 B 9 D Q: A B C D E C E 3 5 S: { A, C, E, B, D }

32 Proof of Correctness (Greedy Stays Ahead) Invariant. For each node u S, d(u) is the length of the shortest path from s to u. P P' x Proof: (by induction on S ) s Base case: S = 1 is trivial. S u v Inductive hypothesis: Assume for S = k 1. Let v be next node added to S, and let (u,v) be the chosen edge. The shortest s-u path plus (u,v) is an s-v path of length (v). Consider any s-v path P. We'll see that it's no shorter than (v). Let (x,y) be the first edge in P that leaves S, and let P' be the subpath to x. P + (x,y) has length d(x)+ (x,y) (y) (v) y

33 Review Question Is Dijsktra s algorithm correct with negative edge weights?

34 Implementation For unexplored nodes, maintain Next node to explore = node with minimum (v). When exploring v, for each edge e = (v,w), update Priority Queue Efficient implementation: Maintain a priority queue of unexplored nodes, prioritized by (v).

35 Priority queues Maintain a set of items with priorities (= keys ) Example: jobs to be performed Operations: Insert Increase key Decrease key Extract-min: find and remove item with least key Common data structure: heap Time: O(log n) per operation

36 Pseudocode for Dijkstra(G, ) d[s] 0 for each v V {s} do d[v] ; [v] S Q V Q is a priority queue maintaining V S, keyed on [v] while Q do u EXTRACT-MIN(Q) S S {u}; d[u] [u] for each v Adjacency-list[u] do if [v] > [u] + (u, v) then [v] d[u] + (u, v) explore edges leaving v Implicit DECREASE-KEY

37 Analysis of Dijkstra n times degree(u) times while Q do u EXTRACT-MIN(Q) S S {u} for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) Handshaking Lemma m implicit DECREASE-KEY s. PQ Operation ExtractMin DecreaseKey Dijkstra n m Array n 1 Binary heap log n log n d-way Heap HW3 HW3 Fib heap log n 1 Total n m log n m log m/n n m + n log n Individual ops are amortized bounds