Design and Analysis of Algorithms 演算法設計與分析. Lecture 13 December 18, 2013 洪國寶

Transcription

1 Design and Analysis of Algorithms 演算法設計與分析 Lecture 13 December 18, 2013 洪國寶 1

2 Homework # (p. 591 / p. 654) (p. 592 / p. 655) (p. 600 / p. 663) (p. 601) / (p. 664) (p. 615) / 24-2 (p. 678) Due 2

3 Outline Review Graph algorithms Shortest paths All pairs NP-completeness 3

4 Review: Shortest paths A weighted, directed graph G = (V, E) with the weight function w: E R. Weight of path p = <v0, v1,, vk>: k w(p) = (, ) i w 1 vi 1 vi. Shortest-path weight from u to v, (u, v): -- A shortest path from u to v is any path p with weight w(p) = δ(u,v) 4

5 Optimal Substructure of a Shortest Path Subpaths of shortest paths are shortest paths. Let p = <v 1, v 2,, v k > be a shortest path from vertex v 1 to vertex v k, and let p ij = <v i, v i+1,, v j > be the subpath of p from vertex v i to vertex v j, 1 i j k. Then, p ij is a shortest path from v i to v j. Suppose that a shortest path p from a source s to a vertex v can be decomposed into. Then, (s, v) = (s, u) + w(u, v). For all edges (u, v) E, (s, v) (s, u) + w(u, v). 5

6 Review: Shortest paths and relaxation The main technique used by the algorithms is relaxation, a method that repeatedly decreases an upper bound on the actual shortest-path weight of each vertex until the upper bound equals the shortest-path weight. d[v] = upper bound on the weight of a shortest path from source s to v (shortest path estimate) 6

7 Review: Shortest paths and relaxation All algorithms consist of a procedure to set d[v] = (Initialize-Single-Source) followed by repeatedly relax edges (Relax) The algorithms differ in how many times they relax each and the order in which they relax each edge. Dijkstra s algorithm: each edge is relax exactly once. Bellman-Ford algorithm: each edge relaxed V-1 times. 7

8 Review: Relaxation Initialize-Single-Source(G, s) 1. for each vertex v V[G] 2. d[v] ; /* upper bound on the weight of a shortest path from s to v */ 3. [v] NIL; /* predecessor of v */ 4. d[s] 0; Relax(u, v, w) 1. if d[v] > d[u]+w(u, v) 2. d[v] d[u]+w(u, v); 3. [v] u; d[v] d[u] + w(u, v) after calling Relax(u, v, w). d[v] (s, v) during the relaxation steps; once d[v] achieves its lower bound (s, v), it never changes. Let be a shortest path. If d[u] = (s, u) prior to the call Relax(u, v, w), then d[v] = (s, v) after the call. 8

9 Dijkstra's Shortest-Path Algorithm Dijkstra's algorithm solves the single-source shortest-paths problem on a weighted, directed graph G = (V, E) for the case in which all edge weights are nonnegative. Dijkstra's algorithm maintains a set S of vertices whose final shortest-path weights from the source s have already been determined. The algorithm repeatedly selects the vertex u V - S with the minimum shortest-path estimate, inserts u into S, and relaxes all edges leaving u. In implementation, we maintain a priority queue Q that contains all the vertices in V - S, keyed by their d values. The implementation assumes that graph G is represented by adjacency lists. Dijkstra(G, w, s) 1. Initialize-Single-Source(G, s); 2. S ; 3. Q V[G]; 4. while Q 5. u Extract-Min(Q); 6. S S {u}; 7. for each vertex v Adj[u] 8. Relax(u, v, w); 9

10 Negative-weight edges Re-weight Use dynamic programming instead of greedy method 0 if v = s d[i,v] = if i = 0, v s min{d[i-1,v],d[i-1,u]+w(u,v)} otherwise d[i,v] is the minimum weight of any path from vertex s to vertex v that contains at most i edges. If no negative weight cycle reachable from s then δ(s,v) = d[ V -1,v] (a shortest path has at most V -1 edges) 10

11 The Bellman-Ford Algorithm for SSSP Bellman-Ford(G,w, s) 1. Initialize-Single-Source(G, s); 2. for i 1 to V for each edge (u, v) E 4. Relax(u, v, w); 5. for each edge (u, v) E 6. if d[v] > d[u] + w(u, v) 7. return FALSE; 8. return TRUE Solves the case where edge weights can be negative. Returns FALSE if there exists a negative weight cycle reachable from the source; TRUE otherwise. Time complexity: O(VE). 11

12 SSSPs in Directed Acyclic Graphs (DAGs) DAG-Shortest-Paths(G, w, s) 1. topologically sort the vertices of G; 2. Initialize-Single-Source(G, s); 3. for each vertex u taken in topologically sorted order 4. for each vertex v Adj[u] 5. Relax(u, v, w); Time complexity: O(V+E) (adjacency-list representation). The running time of this algorithm is determined by line 1 and by the for loop of lines 3-5. The topological sort can be performed in (V + E) time. In the for loop of lines 3-5, as in Dijkstra's algorithm, there is one iteration per vertex. For each vertex, the edges that leave the vertex are each examined exactly once. Unlike Dijkstra's algorithm, however, we use only O( 1 ) time per edge. The running time is thus (V + E), which is linear in the size of an adjacency-list representation of the graph. 12

13 Outline Review Graph algorithms Shortest paths All pairs NP-completeness 13

14 All-Pairs Shortest Paths (APSP) The All-Pairs Shortest Path (APSP) Problem Given: A directed graph G=(V, E) with edge weights Goal: Find a minimum weight path (or cost) between every pair of vertices in V. Method 1: Extends the SSSP algorithms. No negative-weight edges: Run Dijkstra's algorithm V times, once with each v V as the source. Adjacency list + Fibonacci heap: O(V 2 log V + VE) time. With negative-weight edges: Run the Bellman-Ford algorithm V times, once with each v V as the source. Adjacency list: O(V 2 E) time. 14

15 All-Pairs Shortest Paths (APSP) Method 2: DP based on matrix multiplication. Each major loop of the dynamic program will invoke an operation that is very similar to matrix multiplication, so that the algorithm will look like repeated matrix multiplication. Adjacency matrix: We shall start by developing a O(V 4 )-time algorithm for the all-pairs shortest-paths problem and then improve its running time to O(V 3 log V). Method 3: Applies the Floyd-Warshall algorithm (negativeweight edges allowed). Adjacency matrix: O(V 3 ) time. Method 4: Applies Johnson's algorithm for sparse graphs (negative-weight edges allowed). Adjacency list: O(V 2 log V + VE) time. 15

16 All-Pairs Shortest Paths (APSP) Unlike the single-source algorithms, which assume an adjacency-list representation of the graph, most of the algorithms in this chapter use an adjacency-matrix representation. (Johnson's algorithm for sparse graphs uses adjacency lists.) The input is an n x n matrix W representing the edge weights of an n-vertex directed graph G = (V, E). That is, W = (wij), where

17 All-Pairs Shortest Paths (APSP) The tabular output of the all-pairs shortest-paths algorithms is an n x n matrix D = (dij), where entry dij contains the weight of a shortest path from vertex i to vertex j. That is, if we let δ(i,j) denote the shortest-path weight from vertex i to vertex j, then dij = δ(i,j) at termination. To solve the all-pairs shortest-paths problem on an input adjacency matrix, we need to compute not only the shortest path weights but also a predecessor matrix Π = ( πij ), where πij is NIL if either i = j or there is no path from i to j, and otherwise πij is some predecessor of j on a shortest path from i. 17

18 Print all pairs shortest paths 18

19 All-Pairs Shortest Paths (APSP) Method 2: DP based on matrix multiplication. Each major loop of the dynamic program will invoke an operation that is very similar to matrix multiplication, so that the algorithm will look like repeated matrix multiplication. Adjacency matrix: We shall start by developing a O(V 4 )-time algorithm for the all-pairs shortest-paths problem and then improve its running time to O(V 3 log V). Method 3: Applies the Floyd-Warshall algorithm (negativeweight edges allowed). Adjacency matrix: O(V 3 ) time. Method 4: Applies Johnson's algorithm for sparse graphs (negative-weight edges allowed). Adjacency list: O(V 2 log V + VE) time. 19

20 Shortest paths and matrix multiplication Let be the minimum weight of any path from vertex i to vertex j that contains at most m edges. When m = 0, there is a shortest path from i to j with no edges if and only if i = j. Thus, For m 1, we compute as the minimum of (the weight of the shortest path from i to j consisting of at most m -1 edges) and the minimum weight of any path from i to j consisting of at most m edges, obtained by looking at all possible predecessors k of j. Thus, we recursively define

21 Shortest paths and matrix multiplication Observe that since for all vertices i, j V, we have D (1) = W. Taking as our input the matrix W = (w ij ), we now compute a series of matrices D (1), D (2),..., D (n-1), where for m = 1, 2,... n-1,, we have. The final matrix D (n-1) contains the actual shortest-path weights. The heart of the algorithm is procedure: EXTEND-SHORTEST-PATHS(D,W) 21

22 Shortest paths and matrix multiplication EXTEND-SHORTEST-PATHS(D,W) 1 n rows[d] 2 let D' = (d'ij) be an n n matrix 3 for i 1 to n 4 do for j 1 to n 5 do d'ij 6 for k 1 to n 7 do d'ij min(d'ij, dik + wkj) 8 return D'

23 Shortest paths and matrix multiplication EXTEND-SHORTEST-PATHS(D,W) 1 n rows[d] 2 let D' = (d'ij) be an n n matrix 3 for i 1 to n 4 do for j 1 to n 5 do d'ij 6 for k 1 to n 7 do d'ij min(d'ij, dik + wkj) 8 return D' MATRIX-MULTIPLY(A, B) 1 n rows[a] 2 let C be an n n matrix 3 for i 1 to n 4 do for j 1 to n 5 do cij 0 6 for k 1 to n 7 do cij cij + aik bkj 8 return C

24 Shortest paths and matrix multiplication EXTEND-SHORTEST-PATHS(D,W) 1 n rows[d] 2 let D' = (d'ij) be an n n matrix 3 for i 1 to n 4 do for j 1 to n 5 do d'ij 6 for k 1 to n 7 do d'ij min(d'ij, dik + wkj) 8 return D' Observe the correlations D A, W B, d c, w b, 0, min +, + MATRIX-MULTIPLY(A, B) 1 n rows[a] 2 let C be an n n matrix 3 for i 1 to n 4 do for j 1 to n 5 do cij 0 6 for k 1 to n 7 do cij cij + aik bkj 8 return C 24

25 Slow all pairs shortest paths D (1) = D (0) W = W, D (2) = D (1) W = W 2, D (3) = D (2) W = W 3, D (n-1) = D (n-2) W = W n-1. The following procedure computes this sequence in O(n 4 ) time where n= V. 25

26 Slow all pairs shortest paths: example 26

27 D (1) = W, D (2) = W 2 = W W, D (4) = W 4 = W 2 W 2 D (8) = W 8 = W 4 W 4, Faster all pairs shortest paths D (2^lg(n-1) ) = W 2^lg(n-1) = W 2^lg(n-1) -1 W 2^lg(n-1) -1. Since 2 log(n-1) n - 1, the final product D (2^1og(n-1) ) is equal to D (n-1) 27

28 Faster all pairs shortest paths In each iteration of the while loop of lines 4-6, we compute D (2m) = (D (m))2, starting with m = 1. At the end of each iteration, we double the value of m. The final iteration computes D (n-1) by actually computing D (2m) for some n - 1 2m < 2n - 2. The running time is O(n 3 log n) 28

29 Faster all pairs shortest paths: example A directed graph and the sequence of matrices D (m) computed by SLOW-ALL -PAIRS-SHORTEST-PATHS. We can verify that D (5) = D (4) and thus D (m) = D (4) for all m 4. 29

30 All-Pairs Shortest Paths (APSP) Method 2: DP based on matrix multiplication. Each major loop of the dynamic program will invoke an operation that is very similar to matrix multiplication, so that the algorithm will look like repeated matrix multiplication. Adjacency matrix: We shall start by developing a O(V 4 )-time algorithm for the all-pairs shortest-paths problem and then improve its running time to O(V 3 log V). Method 3: Applies the Floyd-Warshall algorithm (negativeweight edges allowed). Adjacency matrix: O(V 3 ) time. Method 4: Applies Johnson's algorithm for sparse graphs (negative-weight edges allowed). Adjacency list: O(V 2 log V + VE) time. 30

31 Overview of Floyd-Warshall APSP Algorithm Applies dynamic programming. 1. Characterize the structure of an optimal solution 2. Recursively define the value of an optimal solution 3. Compute the value of an optimal solution in a bottom-up fashion 4. Construct an optimal solution from computed information Uses adjacency matrix A for G = (V, E) : Goal: Compute V V matrix D where D[i, j] = d ij is the weight of a shortest i-to-j path. Allows negative-weight edges. Runs in O(V 3 ) time. 31

32 Overview of Floyd-Warshall APSP Algorithm We shall assume that there are no negativeweight cycles. We use a different characterization of the structure of a shortest path than we used in the matrix-multiplication-based all-pairs algorithms. The algorithm considers the "intermediate" vertices of a shortest path. 32

33 Shortest-Path Structure An intermediate vertex of a simple path <v 1, v 2,, v l > is any vertex in {v 2, v 3,, v l-1 }. Let be the weight of a shortest path from vertex i to vertex j with all intermediate vertices in {1, 2,, k}. The path does not use vertex k:. The path uses vertex k:. 33

34 Shortest-Path Structure 34

35 The Floyd-Warshall Algorithm for APSP D (k) [i, j] = : weight of a shortest i-to-j path with intermediate vertices in {1, 2,, k}. D (0) = A: original adjacency matrix (paths are single edges). D (n) = ( ): the final answer ( = (i, j)). Time complexity: O(V 3 ). Q: How to detect negative-weight cycles? 35

36 Constructing a Shortest Path Predecessor matrix = ( ij ): ij is NIL if either i=j or there is no path from i to j, or Some predecessor of j on a shortest path from i Compute (0), (1),, (n), where is defined to be the predecessor of vertex j on a shortest path from i with all intermediate vertices in {1, 2,, k}. Print-All-Pairs-Shortest-Path(, i, j) 1. if i=j 2. then print i; 3. else if ij = NIL 4. then print no path from i to j exists ; 5. else Print-All-Pairs-Shortest-Path(, i, ij ); 6. print j; 36

37 Example: The Floyd-Warshall Algorithm 37

38 Recap: APSP and DP Time complexity O(V 4 ) (m) (m/2) (m/2) d ij = min{d ik + d kj } 1 k n Time complexity O(V 3 log V) (k) (k-1) (k-1) (k-1) d ij = min{d ij, d ik + d kj } Time complexity O(V 3 ) 38

39 Transitive Closure of a Directed Graph The Transitive Closure Problem: Input: a directed graph G = (V, E) Output: a V V matrix T s.t. T[i, j] = 1 iff there is a path from i to j in G. Method 1: Run the Floyd-Warshall algorithm on unit-weight graphs. If there is a path from i to j (T[i, j] = 1), then d ij < n; d ij = otherwise. Runs in O(V 3 ) time and uses O(V 2 ) space. Needs to compute lengths of shortest paths. ( ) Method 2: Define t k ij = 1 if there exists a path in G from i to j with all intermediate vertices in {1, 2,, k}; 0 otherwise. t t t t ( k) ( k 1) ( k 1) ( k 1) ( ) ij ij ik kj for k 1,. Runs in O(V 3 ) time and uses O(V 2 ) space, but only single-bit Boolean values are involved faster! 39

40 Transitive Closure of a Directed Graph The Transitive Closure Problem: Input: a directed graph G = (V, E) Output: a V V matrix T s.t. T[i, j] = 1 iff there is a path from i to j in G. Method 1: Run the Floyd-Warshall algorithm on unit-weight graphs. If there is a path from i to j (T[i, j] = 1), then d ij < n; d ij = otherwise. Runs in O(V 3 ) time and uses O(V 2 ) space. Needs to compute lengths of shortest paths. ( ) Method 2: Define t k ij = 1 if there exists a path in G from i to j with all intermediate vertices in {1, 2,, k}; 0 otherwise. t t t t ( k) ( k 1) ( k 1) ( k 1) ( ) ij ij ik kj for k 1,. Runs in O(V 3 ) time and uses O(V 2 ) space, but only single-bit Boolean values are involved faster! 40

41 Transitive Closure of a Directed Graph The Transitive Closure Problem: Input: a directed graph G = (V, E) Output: a V V matrix T s.t. T[i, j] = 1 iff there is a path from i to j in G. Method 1: Run the Floyd-Warshall algorithm on unit-weight graphs. If there is a path from i to j (T[i, j] = 1), then d ij < n; d ij = otherwise. Runs in O(V 3 ) time and uses O(V 2 ) space. Needs to compute lengths of shortest paths. ( ) Method 2: Define t k ij = 1 if there exists a path in G from i to j with all intermediate vertices in {1, 2,, k}; 0 otherwise. t t t t ( k) ( k 1) ( k 1) ( k 1) ( ) ij ij ik kj for k 1,. Runs in O(V 3 ) time and uses O(V 2 ) space, but only single-bit Boolean values are involved faster! 41

42 Transitive-Closure Algorithm 42

43 Transitive-Closure Algorithm 43

44 All-Pairs Shortest Paths (APSP) Method 2: DP based on matrix multiplication. Each major loop of the dynamic program will invoke an operation that is very similar to matrix multiplication, so that the algorithm will look like repeated matrix multiplication. Adjacency matrix: We shall start by developing a O(V 4 )-time algorithm for the all-pairs shortest-paths problem and then improve its running time to O(V 3 log V). Method 3: Applies the Floyd-Warshall algorithm (negativeweight edges allowed). Adjacency matrix: O(V 3 ) time. Method 4: Applies Johnson's algorithm for sparse graphs (negative-weight edges allowed). Adjacency list: O(V 2 log V + VE) time. 44

45 Johnson's Algorithm for Sparse Graphs Idea: All edge weights are nonnegative Dijkstra's algorithm is applicable (+Fibonacci-heap priority queue: O(V 2 lg V + VE)). Reweighting: If there exists some edge weight w < 0, reweight w to 0. must satisfy two properties: 1. u, v V, shortest path from u to v w.r.t. w shortest path from u to v w.r.t.. 2. e E, (e) 0. Preserving shortest paths by reweighting: Given directed G = (V, E) with weight function w: E R, let h: V R. For each edge (u, v) E, define (u, v) = w(u, v) + h(u) - h(v). Let p = <v 0, v 1,, v k >. Then 1. w(p) = (v 0, v k ) iff (p) = (v 0, v k ). 2. G has a negative cycle w.r.t w iff has a negative cycle w.r.t. 45

46 Johnson's Algorithm for Sparse Graphs Idea: All edge weights are nonnegative Dijkstra's algorithm is applicable (+Fibonacci-heap priority queue: O(V 2 lg V + VE)). Reweighting: If there exists some edge weight w < 0, reweight w to 0. must satisfy two properties: 1. u, v V, shortest path from u to v w.r.t. w shortest path from u to v w.r.t.. 2. e E, (e) 0. Preserving shortest paths by reweighting: Given directed G = (V, E) with weight function w: E R, let h: V R. For each edge (u, v) E, define (u, v) = w(u, v) + h(u) - h(v). Let p = <v 0, v 1,, v k >. Then 1. w(p) = (v 0, v k ) iff (p) = (v 0, v k ). 2. G has a negative cycle w.r.t w iff has a negative cycle w.r.t. 46

47 Johnson's Algorithm for Sparse Graphs Idea: All edge weights are nonnegative Dijkstra's algorithm is applicable (+Fibonacci-heap priority queue: O(V 2 lg V + VE)). Reweighting: If there exists some edge weight w < 0, reweight w to 0. must satisfy two properties: 1. u, v V, shortest path from u to v w.r.t. w shortest path from u to v w.r.t.. 2. e E, (e) 0. Preserving shortest paths by reweighting: Given directed G = (V, E) with weight function w: E R, let h: V R. For each edge (u, v) E, define (u, v) = w(u, v) + h(u) - h(v). Let p = <v 0, v 1,, v k >. Then 1. w(p) = (v 0, v k ) iff (p) = (v 0, v k ). 2. G has a negative cycle w.r.t w iff has a negative cycle w.r.t. 47

48 Johnson's Algorithm for Sparse Graphs Idea: All edge weights are nonnegative Dijkstra's algorithm is applicable (+Fibonacci-heap priority queue: O(V 2 lg V + VE)). Reweighting: If there exists some edge weight w < 0, reweight w to 0. must satisfy two properties: 1. u, v V, shortest path from u to v w.r.t. w shortest path from u to v w.r.t.. 2. e E, (e) 0. Preserving shortest paths by reweighting: Given directed G = (V, E) with weight function w: E R, let h: V R. For each edge (u, v) E, define (u, v) = w(u, v) + h(u) - h(v). Let p = <v 0, v 1,, v k >. Then 1. w(p) = (v 0, v k ) iff (p) = (v 0, v k ). 2. G has a negative cycle w.r.t w iff has a negative cycle w.r.t. 48

49 Johnson's Algorithm for Sparse Graphs Idea: All edge weights are nonnegative Dijkstra's algorithm is applicable (+Fibonacci-heap priority queue: O(V 2 lg V + VE)). Reweighting: If there exists some edge weight w < 0, reweight w to 0. must satisfy two properties: 1. u, v V, shortest path from u to v w.r.t. w shortest path from u to v w.r.t.. 2. e E, (e) 0. Preserving shortest paths by reweighting: Given directed G = (V, E) with weight function w: E R, let h: V R. For each edge (u, v) E, define (u, v) = w(u, v) + h(u) - h(v). Let p = <v 0, v 1,, v k >. Then 1. w(p) = (v 0, v k ) iff (p) = (v 0, v k ). 2. G has a negative cycle w.r.t w iff has a negative cycle w.r.t. 49

50 Johnson's Algorithm for Sparse Graphs Idea: All edge weights are nonnegative Dijkstra's algorithm is applicable (+Fibonacci-heap priority queue: O(V 2 lg V + VE)). Reweighting: If there exists some edge weight w < 0, reweight w to 0. must satisfy two properties: 1. u, v V, shortest path from u to v w.r.t. w shortest path from u to v w.r.t.. 2. e E, (e) 0. Preserving shortest paths by reweighting: Given directed G = (V, E) with weight function w: E R, let h: V R. For each edge (u, v) E, define (u, v) = w(u, v) + h(u) - h(v). Let p = <v 0, v 1,, v k >. Then 1. w(p) = (v 0, v k ) iff (p) = (v 0, v k ). 2. G has a negative cycle w.r.t w iff has a negative cycle w.r.t. 50

51 Preserving Shortest Paths by Reweighting 1. Show that w(p) = (v 0, v k ) (p) = (v 0, v k ). Suppose p'= <v 0,, v k >, (p') < (p). w(p') < w(p), contradiction! ( (p) = (v 0, v k ) w(p) = (v 0, v k ): similar!) 2. Show that G has a negative cycle w.r.t w iff has a negative cycle w.r.t. Cycle c = <v 0, v 1,, v k =v 0 > = w(c) + h(v 0 ) - h(v 0 ) = w(c). 51 (c)

52 Preserving Shortest Paths by Reweighting 1. Show that w(p) = (v 0, v k ) (p) = (v 0, v k ). Suppose p'= <v 0,, v k >, (p') < (p). w(p') < w(p), contradiction! ( (p) = (v 0, v k ) w(p) = (v 0, v k ): similar!) 2. Show that G has a negative cycle w.r.t w iff has a negative cycle w.r.t. Cycle c = <v 0, v 1,, v k =v 0 > = w(c) + h(v 0 ) - h(v 0 ) = w(c). 52 (c)

53 Preserving Shortest Paths by Reweighting 1. Show that w(p) = (v 0, v k ) (p) = (v 0, v k ). Suppose p'= <v 0,, v k >, (p') < (p). w(p') < w(p), contradiction! ( (p) = (v 0, v k ) w(p) = (v 0, v k ): similar!) 2. Show that G has a negative cycle w.r.t w iff has a negative cycle w.r.t. Cycle c = <v 0, v 1,, v k =v 0 > = w(c) + h(v 0 ) - h(v 0 ) = w(c). 53 (c)

54 Preserving Shortest Paths by Reweighting 1. Show that w(p) = (v 0, v k ) (p) = (v 0, v k ). Suppose p'= <v 0,, v k >, (p') < (p). w(p') < w(p), contradiction! ( (p) = (v 0, v k ) w(p) = (v 0, v k ): similar!) 2. Show that G has a negative cycle w.r.t w iff has a negative cycle w.r.t. Cycle c = <v 0, v 1,, v k =v 0 > = w(c) + h(v 0 ) - h(v 0 ) = w(c). 54 (c)

55 Choice of the h function We want (u, v) = w(u, v) + h(u) - h(v) 0. By Lemma 24.10, we have for all edges (u, v) E, (s, v) (s, u) + w(u, v). That is, w(u, v) + (s, u) - (s, v) 0 Therefore, we can define h(u) = (s, u) 55

56 Producing Nonnegative Weights by Reweighting 1. Construct a new graph G' = (V', E'), where V' = V {s} E' = E {(s, v): v V} w(s, v) = 0, v V 2. G has no negative cycle G' has no negative cycle. 3. If G and G' have no negative cycles, define h(v) = (s, v), v V'. 4. Define the new weight for : (u, v) = w(u, v) + h(u) - h(v) 0 (since (s, v) (s, u) + w(u, v)). 56

57 The Johnson's Algorithm for APSP Johnson(G) 1. compute G', where V[G']=V[G] {s} and E[G']=E[G] {(s,v):v V[G]}; 2. if Bellman-Ford(G', w, s) = FALSE 3. print the input graph contains a negative-weight cycle; 4. else for each vertex v V[G'] 5. set h(v) to the value of (s,v) computed by the Bellman-Ford algorithm; 6. for each edge (u,v) E[G'] 7. (u,v) w(u, v) + h(u) - h(v); 8. for each vertex u V[G] 9. run Dijkstra(G,, u) to compute (u,v) for all v V[G]; 10. for each vertex v V[G] 11. d uv (u, v)+h(v)-h(u); 12. return D; Steps: Compute the new graph 57

58 The Johnson's Algorithm for APSP Johnson(G) 1. compute G', where V[G']=V[G] {s} and E[G']=E[G] {(s,v):v V[G]}; 2. if Bellman-Ford(G', w, s) = FALSE 3. print the input graph contains a negative-weight cycle; 4. else for each vertex v V[G'] 5. set h(v) to the value of (s,v) computed by the Bellman-Ford algorithm; 6. for each edge (u,v) E[G'] 7. (u,v) w(u, v) + h(u) - h(v); 8. for each vertex u V[G] 9. run Dijkstra(G, \hat{w}, u) to compute (u,v) for all v V[G]; 10. for each vertex v V[G] 11. d uv (u, v)+h(v)-h(u); 12. return D; Steps: Compute the new graph Bellman-Ford 58

59 The Johnson's Algorithm for APSP Johnson(G) 1. compute G', where V[G']=V[G] {s} and E[G']=E[G] {(s,v):v V[G]}; 2. if Bellman-Ford(G', w, s) = FALSE 3. print the input graph contains a negative-weight cycle; 4. else for each vertex v V[G'] 5. set h(v) to the value of (s,v) computed by the Bellman-Ford algorithm; 6. for each edge (u,v) E[G'] 7. (u,v) w(u, v) + h(u) - h(v); 8. for each vertex u V[G] 9. run Dijkstra(G, \hat{w}, u) to compute (u,v) for all v V[G]; 10. for each vertex v V[G] 11. d uv (u, v)+h(v)-h(u); 12. return D; Steps: Compute the new graph Bellman-Ford Reweighting 59

60 The Johnson's Algorithm for APSP Johnson(G) 1. compute G', where V[G']=V[G] {s} and E[G']=E[G] {(s,v):v V[G]}; 2. if Bellman-Ford(G', w, s) = FALSE 3. print the input graph contains a negative-weight cycle; 4. else for each vertex v V[G'] 5. set h(v) to the value of (s,v) computed by the Bellman-Ford algorithm; 6. for each edge (u,v) E[G'] 7. (u,v) w(u, v) + h(u) - h(v); 8. for each vertex u V[G] 9. run Dijkstra(G, \hat{w}, u) to compute (u,v) for all v V[G]; 10. for each vertex v V[G] 11. d uv (u, v)+h(v)-h(u); 12. return D; Steps: Compute the new graph Bellman-Ford Reweighting Dijkstra's (+ Fibonacci heap) 60

61 The Johnson's Algorithm for APSP Johnson(G) 1. compute G', where V[G']=V[G] {s} and E[G']=E[G] {(s,v):v V[G]}; 2. if Bellman-Ford(G', w, s) = FALSE 3. print the input graph contains a negative-weight cycle; 4. else for each vertex v V[G'] 5. set h(v) to the value of (s,v) computed by the Bellman-Ford algorithm; 6. for each edge (u,v) E[G'] 7. (u,v) w(u, v) + h(u) - h(v); 8. for each vertex u V[G] 9. run Dijkstra(G, \hat{w}, u) to compute (u,v) for all v V[G]; 10. for each vertex v V[G] 11. d uv (u, v)+h(v)-h(u); 12. return D; Steps: Compute the new graph Bellman-Ford Reweighting Dijkstra's (+ Fibonacci heap) Recompute the weights. 61

62 The Johnson's Algorithm for APSP Johnson(G) 1. compute G', where V[G']=V[G] {s} and E[G']=E[G] {(s,v):v V[G]}; 2. if Bellman-Ford(G', w, s) = FALSE 3. print the input graph contains a negative-weight cycle; 4. else for each vertex v V[G'] 5. set h(v) to the value of (s,v) computed by the Bellman-Ford algorithm; 6. for each edge (u,v) E[G'] 7. (u,v) w(u, v) + h(u) - h(v); 8. for each vertex u V[G] 9. run Dijkstra(G, \hat{w}, u) to compute (u,v) for all v V[G]; 10. for each vertex v V[G] 11. d uv (u, v)+h(v)-h(u); 12. return D; Steps: Compute the new graph Bellman-Ford Reweighting Dijkstra's (+ Fibonacci heap) Recompute the weights. Time complexity: O(V) + O(VE) + O(E) + O(V 2 log V + VE) + O(V 2 ) = O(V 2 log V + VE). 62

63 Example: Johnson's Algorithm for APSP 63

64 Outline Review Graph algorithms Shortest paths All pairs NP-completeness 64

65 NP-Completeness Topics: Complexity classes Reducibility and NP-completeness proofs Coping with NP-complete problems Reading: Chapter 34 Chapter 35 (35.1, 35.2) 65

66 Outline Review Shortest paths Single source (Bellman-Ford) All pairs NP-completeness Complexity class (introduction) Formal notion of problem Encoding and complexity Language framework Polynomial time verification Reduction and NP-complete 66

67 Complexity classes All of the algorithms we have studied thus far have been polynomial-time algorithms: on inputs of size n, their worst-case running time is O(n k ) for some constant k. It is natural to wonder whether all problems can be solved in polynomial time. The answer is no. For example, there are problems, such as Turing's famous "Halting Problem," that cannot be solved by any computer, no matter how much time is provided. 67

68 Halting Problem Infinite loop can tie up a computer system It would be useful to be able to preprocess a program and its data set by running it through a checking program, checkhalt(p,d), that prints "halts" if algorithm P terminates in a finite number of steps when run with data set D, or "loops forever" if algorithm P does not terminate in a finite number of steps when run with data set D 68

69 Halting Problem Prove no such checking algorithm (program) exists by contradiction: Assume check-halt(p,d) exists Construct algorithm test(d) as follows algorithm test(d) if check-halt(test,d) prints "halts" then while 0=0 print "halts" else stop Contradiction: test(test) loop forever and test(test) terminates after one step 69

70 Complexity classes No polynomial-time algorithm has yet been discovered for an NP-complete problem, nor has anyone yet been able to prove a superpolynomialtime lower bound for any of them. This so-called P NP question has been one of the deepest, most perplexing open research problems in theoretical computer science since it was posed in Clay Mathematics Institute 7 millennium problems (1 million per problem) 70

71 Complexity classes To become a good algorithm designer, you must understand the rudiments of the theory of NP-completeness. If you can establish a problem as NP-complete, you provide good evidence for its intractability. As an engineer, you would then do better spending your time developing an approximation algorithm rather than searching for a fast algorithm that solves the problem exactly. (next lecture) Moreover, many natural and interesting problems that on the surface seem no harder than sorting, graph searching, or shortest paths are in fact NP-complete. Thus, it is important to become familiar with this remarkable class of problems. 71

72 Complexity classes Many natural and interesting problems that on the surface seem no harder than sorting, graph searching, or shortest paths are in fact NP-complete. Longest path vs. shortest path Vertex cover vs. edge cover 3-CNF satisfiability vs. 2-CNF satisfiability Hamiltonian cycle vs. Euler tour 72

73 Outline Review Shortest paths Single source (Bellman-Ford) All pairs NP-completeness Complexity class (introduction) Formal notion of problem Encoding and complexity Language framework Polynomial time verification Reduction and NP-complete 73

74 Formal notion of "problem" We define an abstract problem Q to be a binary relation on a set I of problem instances and a set S of problem solutions. Q I S For example, consider the problem SHORTEST-PATH of finding a shortest path between two given vertices in an unweighted, undirected graph G = (V, E). I = {a triple consisting of a graph and two vertices}. S= {a sequence of vertices in the graph}. The problem SHORTEST-PATH (SPSP) itself is the relation that associates each instance of a graph and two vertices with a shortest path in the graph that connects the two vertices. 74

75 Formal notion of "problem" For simplicity, the theory of NP-completeness restricts attention to decision problems: those having a yes/no solution. We can view an abstract decision problem as a function that maps the instance set I to the solution set {0, 1}. For example, a decision problem PATH related to the shortest-path problem is, "Given a graph G = (V, E), two vertices u, v V, and a nonnegative integer k, does a path exist in G b/w u and v whose length is at most k?" If i = G, u, v, k is an instance of this shortest-path problem, then PATH(i) = 1 (yes) if a shortest path from u to v has length at most k, and PATH(i) = 0 (no) otherwise. 75

76 Formal notion of "problem" Many abstract problems are not decision problems, but rather optimization problems, in which some value must be minimized or maximized. In order to apply the theory of NP-completeness to optimization problems, we must recast them as decision problems. Typically, an optimization problem can be recast by imposing a bound on the value to be optimized. As an example, in recasting the shortest-path problem as the decision problem PATH, we added a bound k to the problem instance. 76

77 Recap: Decision & Optimization Problems Optimization problems: those finding a legal configuration such that its cost is minimum (or maximum). MST: Given a graph G=(V, E), find the cost of a minimum spanning tree of G. TSP: Given a set of cities and that distance between each pair of cities, find the distance of a minimum route starts and ends at a given city and visits every city exactly once. Decision problems: those having yes/no answers. MST: Given a graph G=(V, E) and a bound K, is there a spanning tree with a cost at most K? TSP: Given a set of cities, distance between each pair of cities, and a bound B, is there a route that starts and ends at a given city, visits every city exactly once, and has total distance at most B? NP-completeness is associated with decision problems. 77

78 Outline Review Shortest paths Single source (Bellman-Ford) All pairs NP-completeness Complexity class (introduction) Formal notion of problem Encoding and complexity Language framework Polynomial time verification Reduction and NP-complete 78

79 Encodings If a computer program is to solve an abstract problem, problem instances must be represented in a way that the program understands. An encoding of a set S of abstract objects is a mapping e from S to the set of binary strings. For example, we are all familiar with encoding the natural numbers N = {0, 1, 2, 3, 4,...} as the strings {0, 1, 10, 11, 100,...}. Using this encoding, e(l7) = In the ASCII code, e(a) = Even a compound object can be encoded as a binary string by combining the representations of its constituent parts. Polygons, graphs, functions, ordered pairs, programs--all can be encoded as binary strings. 79

80 Encodings Thus, a computer algorithm that "solves" some abstract decision problem actually takes an encoding of a problem instance as input. We call a problem whose instance set is the set of binary strings a concrete problem. We say that an algorithm solves a concrete problem in time O(T(n)) if, when it is provided a problem instance i of length n = i, the algorithm can produce the solution in at most O(T(n)) time. A concrete problem is polynomial-time solvable, therefore, if there exists an algorithm to solve it in time O(n k ) for some constant k. We can now formally define the complexity class P as the set of concrete decision problems that are solvable in polynomial time. 80

81 Encodings We can use encodings to map abstract problems to concrete problems. Given an abstract decision problem Q mapping an instance set I to {0, 1}, an encoding e : I {0, 1} * can be used to induce a related concrete decision problem, which we denote by e(q). If the solution to an abstract-problem instance i I is Q(i) {0, 1}, then the solution to the concrete-problem instance e(i) {0, 1} * is also Q(i). As a technicality, there may be some binary strings that represent no meaningful abstract-problem instance. For convenience, we shall assume that any such string is mapped arbitrarily to 0. Thus, the concrete problem produces the same solutions as the abstract problem on binary-string instances that represent the encodings of abstract-problem instances. 81

82 Encodings Binary strings correspond to problem instances (output 1) Binary strings correspond to problem instances (output 0) Binary strings do not correspond to 0 0 problem instance 1 82

83 Encodings We would like the efficiency of solving a problem should not depend on how the problem is encoded. Unfortunately, it depends quite heavily. For example, suppose that an integer k is to be provided as the sole input to an algorithm, and suppose that the running time of the algorithm is Θ(k). If the integer k is provided in unary--a string of k 1's--then the running time of the algorithm is O(n) on length-n inputs, which is polynomial time. If we use the more natural binary representation of the integer k, however, then the input length is n = log k. In this case, the running time of the algorithm is Θ(k) = Θ(2 n ), which is exponential in the size of the input. Thus, depending on the encoding, the algorithm runs in either polynomial or superpolynomial time. 83

84 Encodings The encoding of an abstract problem is therefore quite important to our understanding of polynomial time. We cannot really talk about solving an abstract problem without first specifying an encoding. Nevertheless, in practice, if we rule out "expensive" encodings such as unary ones, the actual encoding of a problem makes little difference to whether the problem can be solved in polynomial time. For example, representing integers in base 3 instead of binary has no effect on whether a problem is solvable in polynomial time, since an integer represented in base 3 can be converted to an integer represented in base 2 in polynomial time. 84

85 Encodings A function f : {0, 1} * {0, 1} * is polynomial-time computable if there exists a polynomial-time algorithm A that, given any input x {0, 1} *, produces as output f(x). For some set I of problem instances, we say that two encodings e 1 and e 2 are polynomially related if there exist two polynomial-time computable functions f 12 and f 21 such that for any i I, we have f 12 (e 1 (i)) = e 2 (i) and f 21 (e 2 (i)) = e 1 (i). That is, the encoding e 2 (i) can be computed from the encoding e 1 (i) by a polynomial-time algorithm, and vice versa. Lemma 34.1 (P. 974) Let Q be an abstract decision problem on an instance set I, and let e 1 and e 2 be polynomially related encodings on I. Then, e 1 (Q) P if and only if e 2 (Q) P. 85

86 Encodings In order to be able to converse in an encoding-independent fashion, we shall generally assume that problem instances are encoded in any reasonable, concise fashion. To be precise, we shall assume that the encoding of an integer is polynomially related to its binary representation, a finite set is polynomially related to its encoding as a list of its elements, enclosed in braces and separated by commas. (ASCII is one such encoding scheme.) With such a "standard" encoding in hand, we can derive reasonable encodings of other mathematical objects, such as tuples, graphs, and formulas. To denote the standard encoding of an object, we shall enclose the object in angle braces. Thus, G denotes the standard encoding of a graph G. 86

87 Encodings As long as we implicitly use an encoding that is polynomially related to this standard encoding, we can talk directly about abstract problems without reference to any particular encoding, knowing that the choice of encoding has no effect on whether the abstract problem is polynomial-time solvable. Henceforth, we shall generally assume that all problem instances are binary strings encoded using the standard encoding, unless we explicitly specify the contrary. We shall also typically neglect the distinction between abstract and concrete problems. 87

88 Recap: problem and encoding Problem: Sorting, shortest path, abstract problem Q=(I,S) S={0,1} abstract decision problem I={0,1}* concrete decision problem language over Σ={0,1} 88

89 Questions? 89