Bird s-eye view Reductions designing algorithms establishing lower bounds establishing intractability classifying problems Desiderata. Classify problems according to computational requirements. Linear: min/max, median, Burrows-Wheeler transform,... Linearithmic: sort, convex hull, closest pair, Quadratic: Cubic: Exponential: rustrating news. Huge number of fundamental problems have defied classification. Algorithms in Java, 4 th Edition Robert Sedgewick and Kevin Wayne Copyright 008 April 0, 008 7:44:0 AM Bird s-eye view Reduction Desiderata. Classify problems according to computational requirements. Desiderata'. Suppose we could (couldn't) solve problem X efficiently. What else could (couldn't) we solve efficiently? Def. Problem X reduces to problem Y if you can use an algorithm that solves Y to help solve X. instance I (of X) Algorithm for Y solution to I Algorithm for X Cost of solving X = total cost of solving Y + cost of reduction. Give me a lever long enough and a fulcrum on which to place it, and I shall move the world. Archimedes perhaps many calls to Y on problems of different sizes 4
Reduction Reduction Def. Problem X reduces to problem Y if you can use an algorithm that solves Y to help solve X. Def. Problem X reduces to problem Y if you can use an algorithm that solves Y to help solve X. instance I (of X) Algorithm for Y solution to I instance I (of X) Algorithm for Y solution to I Algorithm for X Algorithm for X Ex 1. [element distinctness reduces to sorting] o solve element distinctness on N integers: Sort N integers. Scan through consecutive pairs and check if any are equal. Cost of solving element distinctness. N log N + N. Ex. [-collinear reduces to sorting] o solve -collinear instance on N points in the plane: or each point, sort other points by polar angle. - scan through consecutive triples and check if they are collinear Cost of solving -collinear. N log N + N. 5 6 Reduction: design algorithms Def. Problem X reduces to problem Y if you can use an algorithm that solves Y to help solve X. Design algorithm. Given algorithm for Y, can also solve X. designing algorithms establishing lower bounds establishing intractability classifying problems Ex. Element distinctness reduces to sorting. -collinear reduces to sorting. PER reduces to topological sort. [see digraph lecture] h-v line intersection reduces to 1d range searching. [see geometry lecture] Mentality. Since I know how to solve Y, can I use that algorithm to solve X? programmer s version: I have code for Y. Can I use it for X? 7 8
Convex hull reduces to sorting Shortest path on graphs and digraphs Sorting. Given N distinct integers, rearrange them in ascending order. Convex hull. Given N points in the plane, identify the extreme points of the convex hull (in counter-clockwise order). Proposition. Undirected shortest path (with nonnegative weights) reduces to directed shortest path. 9 5 10 4 15 15 10 1514 86154 988818 4190745 1546464 89885444 4441 s 5 1 6 1 t convex hull sorting Proposition. Convex hull reduces to sorting. Graham scan algorithm. 9 10 Shortest path on graphs and digraphs Shortest path with negative weights Proposition. Undirected shortest path (with nonnegative weights) reduces to directed shortest path. Caveat. Reduction is invalid in networks with negative weights (even if no negative cycles). 9 5 a 7 v -4 t 10 4 15 15 10 s 5 1 6 1 t 7-4 s 7 v -4 t Replace each undirected edge by two directed edges. 9 reduction creates negative cycles 9 5 s 10 10 5 15 10 15 15 4 4 10 15 1 5 1 t Remark. Can still solve shortest path problem in undirected graphs (if no negative cycles), but need more sophisticated techniques. 1 1 11 reduces to weighted non-bipartite matching (!) 1
Primality testing Primality testing PRIME. Given an integer x (represented in binary), is x prime? COMPOSIE. Given an integer x, does x have a nontrivial factor? PRIME. Given an integer x (represented in binary), is x prime? COMPOSIE. Given an integer x, does x have a nontrivial factor? Proposition. PRIME reduces to COMPOSIE. Proposition. COMPOSIE reduces to PRIME. public static boolean isprime(biginteger x) { if (iscomposite(x)) return false; else return true; } public static boolean iscomposite(biginteger x) { if (isprime(x)) return false; else return true; } PRIME 14757955896764191 PRIME 14757955896764191 prime prime COMPOSIE 14757955896764197 COMPOSIE 14757955896764197 composite composite 1 14 Caveat Some reductions PRIME. Given an integer x (represented in binary), is x prime? COMPOSIE. Given an integer x, does x have a nontrivial factor? Proposition. COMPOSIE reduces to PRIME. Proposition. PRIME reduces to COMPOSIE. bipartite matching shortest paths (nonnegative) arbitrage A possible real-world scenario. System designer specs the APIs for project. Programmer A implements iscomposite() using isprime(). Programmer B implements isprime() using iscomposite(). Infinite reduction loop! PRIME whose fault? median finding element distinctness closest pair sorting Euclidean MS convex hull maximum flow LP shortest paths (no neg cycles) COMPOSIE Voronoi LP (standard form) 15 16
Bird's-eye view Goal. Prove that a problem requires a certain number of steps. Ex. Ω(N log N) lower bound for sorting. designing algorithms establishing lower bounds establishing intractability classifying problems 1514 86154 988818 4190745 1546464 89885444 4441 argument must apply to all conceivable algorithms Bad news. Very difficult to establish lower bounds from scratch. Good news. Can establish Ω(N log N) lower bound for Y by reducing sorting to Y. assuming cost of reduction is not too large 17 18 Linear-time reductions Linear-time reductions Def. Problem X linear-time reduces to problem Y if X can be solved with: linear number of standard computational steps one call to Y Def. Problem X linear-time reduces to problem Y if X can be solved with: linear number of standard computational steps one call to Y Ex. Almost all of the reductions we've seen so far. Q. Which one was not a linear-time reduction? Establish lower bound: If X takes Ω(N log N) steps, then so does Y. If X takes Ω(N ) steps, then so does Y. instance I (of X) Algorithm for Y Algorithm for X solution to I Mentality. If I could easily solve Y, then I could easily solve X. I can t easily solve X. herefore, I can t easily solve Y. 19 0
Lower bound for convex hull Sorting linear-time reduces to convex hull act. In quadratic decision tree model, any algorithm for sorting N integers requires Ω(N log N) steps. allows quadratic tests of the form: xi < xj or (xj - xi) (xk - xi) - (xj ) (xj - xi) < 0 Proposition. Sorting linear-time reduces to convex hull. Sorting instance. X = { x 1, x,..., x N } Convex hull instance. P = { (x 1, x 1 ), (x, x ),..., (x N, x N ) } Proposition. Sorting linear-time reduces to convex hull. [see next slide] f(x) = x 1514 86154 988818 4190745 1546464 89885444 4441 (x, x ) (x 1, x 1 ) sorting convex hull a quadratic test Implication. Any ccw-based convex hull algorithm requires Ω(N log N) ccw's. Region {x : x x} is convex all points are on hull. Starting at point with most negative x, counter-clockwise order of hull points yields integers in ascending order. 1 Lower bound for -COLLINEAR Lower bound for -COLLINEAR -SUM. Given N distinct integers, are there three that sum to 0? -SUM. Given N distinct integers, are there three that sum to 0? -COLLINEAR. Given N distinct points in the plane, are there that all lie on the same line? recall Assignment -COLLINEAR. Given N distinct points in the plane, are there that all lie on the same line? 1514-86154 988818-4190745 1546464 89885444-4441 Proposition. -SUM linear-time reduces to -COLLINEAR. [see next slide] in certain restricted model of computation -sum -collinear act. Any algorithm for -SUM requires Ω(N ) time. Implication. No sub-quadratic algorithm for -COLLINEAR. your N log N algorithm was pretty good 4
-SUM linear-time reduces to -COLLINEAR -SUM linear-time reduces to -COLLINEAR Proposition. -SUM linear-time reduces to -COLLINEAR. -SUM instance: X = { x 1, x,..., x N } -COLLINEAR instance: P = { (x 1, x 1 ), (x, x ),..., (x N, x N ) } Lemma. If a, b, and c are distinct, then a + b + c = 0 if and only if (a, a ), (b, b ), (c, c ) are collinear. f(x) = x Proposition. -SUM linear-time reduces to -COLLINEAR. -SUM instance: X = { x 1, x,..., x N } -COLLINEAR instance: P = { (x 1, x 1 ), (x, x ),..., (x N, x N ) } Lemma. If a, b, and c are distinct, then a + b + c = 0 if and only if (a, a ), (b, b ), (c, c ) are collinear. (, 8) hree points (a, a ), (b, b ), (c, c ) are collinear iff: (1, 1) (-, -7) - + + 1 = 0 (a - b ) / (a - b) = (b - c ) / (b - c) (a - b)(a + ab + b ) / (a - b) = (b - c)(b + bc + c ) / (b - c) (a + ab + b ) = (b + bc + c ) a + ab - bc - c = 0 (a - c)(a + b + c) = 0 a + b + c = 0 slopes are equal factor numerators a-b and b-c are nonzero collect terms factor a-c is nonzero 5 6 More lower bounds Establishing lower bounds: summary Establishing lower bounds through reduction is an important tool element distinctness (N log N lower bound) -sum (N lower bound) in guiding algorithm design efforts. Q. How to convince yourself no linear-time convex hull algorithm exists? Hard way. Long futile search for a linear-time algorithm. Easy way. Reduction from sorting. sorting closest pair -collinear dihedral rotation sorting convex hull convex hull Voronoi Euclidean MS min area triangle Q. How to convince yourself no subquadratic -COLLINEAR algorithm exists. Hard way. Long futile search for a subquadratic algorithm. Easy way. Reduction from -SUM. -SUM -COLLINEAR 7 8
Bird's-eye view Desiderata. Prove that a problem can't be solved in poly-time. designing algorithms establishing lower bounds establishing intractability classifying problems EXPIME-complete. Given a fixed-size program and input, does it halt in at most k steps? Given N-by-N checkers board position, can the first player force a win (using forced capture rule)? rustrating news. Extremely difficult and few successes. input size = lg k 9 0 -satisfiability -satisfiability is intractable Literal. A boolean variable or its negation. Clause. An or of distinct literals. Conjunctive normal form. An and of clauses. x i or x i C j = (x 1 x x ) Φ = (C 1 C C C 4) Q. How to solve an instance of -SA with n variables? A. Exhaustive search: try all n truth assignments. Q. Can we do anything substantially more clever? -SA. Given a CN formula Φ consisting of k clauses over n literals, does it have a satisfying truth assignment? yes instance ( x 1 x x ) (x 1 x x ) ( x 1 x x ) ( x 1 x x 4) ( x x x 4) x1 x x x4 ( ) ( ) ( ) ( ) ( ) no instance ( x 1 x x ) (x 1 x x ) ( x 1 x x ) ( x 1 x x 4) ( x x x 4) Conjecture (P NP). No poly-time algorithm for -SA. "intractable" Applications. Circuit design, program correctness,... Good news. Can prove problems "intractable" via reduction from -SA. 1
Polynomial-time reductions Polynomial-time reductions Def. Problem X poly-time reduces to problem Y if X can be solved with: Polynomial number of standard computational steps. One call to Y. Def. Problem X poly-time reduces to problem Y if X can be solved with: Polynomial number of standard computational steps. One call to Y. Ex. All reductions we've seen. Establish intractability. If -SA poly-time reduces to Y, then Y is intractable. instance I (of X) Algorithm for Y Algorithm for X solution to I Mentality. If I could solve Y in poly-time, then I could also solve -SA. I can t solve -SA. herefore, I can t solve Y. 4 Integer linear programming Graph -colorability ILP. Minimize a linear objective function, subject to linear inequalities, and integer variables. -COLOR. Given a graph, is there a way to color the vertices red, green, and blue so that no adjacent vertices have the same color? Proposition. -SA poly-time reduces to ILP. [by example] ( x1 x x) (x1 x x) ( x1 x x) ( x1 x x4) ( x x x4) minimize subject to the constraints C1 + C + C + C4 + C5 (1 - x1) C1 x C1 x C1... all xi and Cj = { 0, 1 } CN formula satisfiable iff min = 5 C1 = 1 iff clause 1 is satisfied add inequalities for each clause yes instance Interpretation. Boolean variable xi is true iff integer variable xi = 1. 5 6
Graph -colorability Graph -colorability -COLOR. Given a graph, is there a way to color the vertices red, green, and blue so that no adjacent vertices have the same color? -COLOR. Given a graph, is there a way to color the vertices red, green, and blue so that no adjacent vertices have the same color? no instance yes instance Applications. Register allocation, Potts model in physics, 7 8 -satisfiability reduces to graph -colorability -satisfiability reduces to graph -colorability Proposition. -SA poly-time reduces to -COLOR. Given -SA instance Φ, we construct an instance G of -COLOR that is -colorable if and only if Φ is satisfiable. Construction. (i) Create one vertex for each literal and vertices,, and N. (ii) Connect,, and N in a triangle and connect each literal to N. (iii) Connect each literal to its negation. (iv) or each clause, attach a 6-vertex gadget [details to follow]. Claim. If graph G is -colorable then Φ is satisfiable. Consider assignment where corresponds to false and to true. (ii) [triangle] ensures each literal is true or false. false true false true N neither N neither x 1 x 1 x x x x... x n x n x 1 x 1 x x x x... x n x n 9 40
-satisfiability reduces to graph -colorability -satisfiability reduces to graph -colorability Claim. If graph G is -colorable then Φ is satisfiable. Consider assignment where corresponds to false and to true. (ii) [triangle] ensures each literal is true or false. (iii) ensures a literal and its negation are opposites. Claim. If graph G is -colorable then Φ is satisfiable. Consider assignment where corresponds to false and to true. (ii) [triangle] ensures each literal is true or false. (iii) ensures a literal and its negation are opposites. (iv) [gadget] ensures at least one literal in each clause is true. next slide (x 1 x x ) B false true x 1 x x N neither 6-node gadget x 1 x 1 x x x x... x n x n true false 41 4 -satisfiability reduces to graph -colorability -satisfiability reduces to graph -colorability Claim. If graph G is -colorable then Φ is satisfiable. Consider assignment where corresponds to false and to true. (ii) [triangle] ensures each literal is true or false. (iii) ensures a literal and its negation are opposites. (iv) [gadget] ensures at least one literal in each clause is true. Claim. If Φ is satisfiable then graph G is -colorable. at least one in each clause Color nodes corresponding to false literals and to true literals. herefore, Φ is satisfiable. next slide (x 1 x x ) B if all literals in clause are false, then not -colorable (contradiction) (x 1 x x ) B x 1 x x x 1 x x 6-node gadget true false 4 44
-satisfiability reduces to graph -colorability -satisfiability reduces to graph -colorability Claim. If Φ is satisfiable then graph G is -colorable. Color nodes corresponding to false literals and to true literals. Color vertex below one vertex, and vertex below that. Claim. If Φ is satisfiable then graph G is -colorable. Color nodes corresponding to false literals and to true literals. Color vertex below one vertex, and vertex below that. Color remaining middle row vertices. (x 1 x x ) B (x 1 x x ) B x 1 x x x 1 x x 45 46 -satisfiability reduces to graph -colorability -satisfiability reduces to graph -colorability Claim. If Φ is satisfiable then graph G is -colorable. Color nodes corresponding to false literals and to true literals. Color vertex below one vertex, and vertex below that. Color remaining middle row vertices. Color remaining bottom vertices or as forced. Works for all gadgets, so graph is -colorable. (x 1 x x ) B Proposition. -SA poly-time reduces to -COLOR. Given -SA instance Φ, we construct an instance G of -COLOR that is -colorable if and only if Φ is satisfiable. Construction. (i) Create one vertex for each literal and vertices,, and N. (ii) Connect,, and N in a triangle and connect each literal to N. (iii) Connect each literal to its negation. (iv) or each clause, attach a 6-vertex gadget. x 1 x x Consequence. -COLOR is intractable. 47 48
More poly-time reductions from -satisfiability Establishing intractability: summary -SA reduces to -COLOR -SA Establishing intractability through poly-time reduction is an important tool in guiding algorithm design efforts. -COLOR DM VEREX COVER Dick Karp '85 uring award Q. How to convince yourself that a new problem is intractable? Hard way. Long futile search for an efficient algorithm (as for -SA). Easy way. Reduction from a know intractable problem (such as -SA). EXAC COVER PLANAR--COLOR CLIQUE HAM-CYCLE hence, intricate reductions are common SUBSE-SUM INDEPENDEN SE SP HAM-PAH -SA -COLOR PARIION INEGER PROGRAMMING KNAPSACK BIN-PACKING Conjecture: no poly-time algorithm for -SA. (and hence none of these problems) 49 50 Implications of poly-time reductions Implications of poly-time reductions 51 5
Implications of poly-time reductions designing algorithms establishing lower bounds establishing intractability classifying problems 5 54 Classify problems Classify problems Desiderata. Classify problems according to difficulty. Linear: can be solved in linear time. Linearithmic: can be solved in linearithmic time. Quadratic: can be solved in quadratic time. ractable: can be solved in poly-time. Intractable: seem to require exponential time. Desiderata. Classify problems according to difficulty. Linear: can be solved in linear time. Linearithmic: can be solved in linearithmic time. Quadratic: can be solved in quadratic time. ractable: can be solved in poly-time. Intractable: seem to require exponential time. Ex. Sorting and convex hull are in same complexity class. Sorting linear-time reduces to convex hull. Convex hull linear-time reduces to sorting. linearithmic Ex. PRIME and COMPOSIE are in same complexity class. PRIME linear-time reduces to COMPOSIE. COMPOSIE linear-time reduces to PRIME. tractable, but nobody knows which class 55 56
Classify problems Cook s theorem Desiderata. Classify problems according to difficulty. Linear: can be solved in linear time. Linearithmic: can be solved in linearithmic time. Quadratic: can be solved in quadratic time. P. Set of problems solvable in poly-time. Importance. What scientists and engineers can compute feasibly. NP. Set of problems checkable in poly-time. Importance. What scientists and engineers aspire to compute feasibly. ractable: can be solved in poly-time. Intractable: seem to require exponential time. Ex. -SA and -COLOR are in the same complexity class. -SA poly-time reduces to -COLOR. -COLOR poly-time reduces to -SA. probably intractable Cook's theorem. All problems in NP poly-time reduces to -SA. "NP-complete" Cook's theorem (stay tuned) 57 58 Implications of Cook s theorem Implications of Karp + Cook -COLOR -COLOR reduces to -SA -SA DM VEREX COVER Stephen Cook '8 uring award -COLOR -COLOR reduces to -SA -SA reduces to -COLOR -SA DM VEREX COVER EXAC COVER PLANAR--COLOR CLIQUE HAM-CYCLE EXAC COVER PLANAR--COLOR CLIQUE HAM-CYCLE SUBSE-SUM INDEPENDEN SE SP HAM-PAH SUBSE-SUM INDEPENDEN SE SP HAM-PAH PARIION INEGER PROGRAMMING PARIION INEGER PROGRAMMING KNAPSACK BIN-PACKING All of these problems (any many more) poly-time reduce to -SA. KNAPSACK BIN-PACKING all of these problems are NP-complete; they are manifestations of the same really hard problem 59 60
Summary Reductions are important in theory to: Establish tractability. Establish intractability. Classify problems according to their computational requirements. Reductions are important in practice to: Design algorithms. Design reusable software modules. - stack, queue, sorting, priority queue, symbol table, set, - graph, shortest path, regular expression Determine difficulty of your problem and choose the right tool. - use exact algorithm for tractable problems - use heuristics for intractable problems 61