REDUCTIONS BBM ALGORITHMS DEPT. OF COMPUTER ENGINEERING ERKUT ERDEM. Bird s-eye view. May. 12, Reduction.

Size: px

Start display at page:

Download "REDUCTIONS BBM ALGORITHMS DEPT. OF COMPUTER ENGINEERING ERKUT ERDEM. Bird s-eye view. May. 12, Reduction."

Rosaline Eaton
5 years ago
Views:

BBM 0 - ALGORITHMS DEPT. OF COMPUTER ENGINEERING ERKUT ERDEM REDUCTIONS May., 0 Bird s-eye view Desideraa. Classify problems according o compuaional requiremens.

Frusraing news. Huge number of problems have defied classificaion. Ackwledgemen:.The$course$slides$are$adaped$from$he$slides$prepared$by$R.$Sedgewick$ and$k.$wayne$of$princeon$universiy.

Wha else could (could ) we solve efficienly? Reducion Def. Problem X reduces o problem Y if you can use an algorihm ha solves Y o help solve X.

1 BBM 0 - ALGORITHMS DEPT. OF COMPUTER ENGINEERING ERKUT ERDEM REDUCTIONS May., 0 Bird s-eye view Desideraa. Classify problems according o compuaional requiremens. complexiy order of growh examples linear linearihmic N N log N min, max, median, Burrows-Wheeler ransform,... soring, convex hull, closes pair, farhes pair,... quadraic N? exponenial c N? Frusraing news. Huge number of problems have defied classificaion. Ackwledgemen:.The$course$slides$are$adaped$from$he$slides$prepared$by$R.$Sedgewick$ and$k.$wayne$of$princeon$universiy. Bird s-eye view Desideraa. Classify problems according o compuaional requiremens. Desideraa'. Suppose we could (could ) solve problem X efficienly. Wha else could (could ) we solve efficienly? Reducion Def. Problem X reduces o problem Y if you can use an algorihm ha solves Y o help solve X. insance I (of X) Algorihm for Y soluion o I Algorihm for X Cos of solving X = oal cos of solving Y + cos of reducion. Give me a lever long eugh and a fulcrum on which o place i, and I shall move he world. Archimedes perhaps many calls o Y on problems of differen sizes preprocessing and posprocessing 3

2 Reducion Def. Problem X reduces o problem Y if you can use an algorihm ha solves Y o help solve X. Reducion Def. Problem X reduces o problem Y if you can use an algorihm ha solves Y o help solve X. insance I (of X) Algorihm for Y soluion o I insance I (of X) Algorihm for Y soluion o I Algorihm for X Algorihm for X Ex 1. [elemen disincness reduces o soring] To solve elemen disincness on N iems: Sor N iems. Check adjacen pairs for equaliy. Cos of solving elemen disincness. N log N + N. cos of soring cos of reducion Ex. [3-collinear reduces o soring] To solve 3-collinear insance on N poins in he plane: For each poin, sor oher poins by polar angle or slope. - check adjacen riples for collineariy cos of soring Cos of solving 3-collinear. N log N + N. cos of reducion 6 REDUCTIONS Reducion: design algorihms Designing algorihms Esablishing lower bounds Classifying problems Def. Problem X reduces o problem Y if you can use an algorihm ha solves Y o help solve X. Design algorihm. Given algorihm for Y, can also solve X. Ex. Elemen disincness reduces o soring. 3-collinear reduces o soring. CPM reduces o opological sor. [shores pahs lecure] h-v line inersecion reduces o 1d range searching. [geomeric BST lecure] Baseball eliminaion reduces o maxflow. Burrows-Wheeler ransform reduces o suffix sor. Menaliy. Since I kw how o solve Y, can I use ha algorihm o solve X? programmer s version: I have code for Y. Can I use i for X? 8

3 Convex hull reduces o soring Soring. Given N disinc inegers, rearrange hem in ascending order. Convex hull. Given N poins in he plane, idenify he exreme poins of he convex hull (in counerclockwise order). Graham scan algorihm Graham scan. Choose poin p wih smalles (or larges) y-coordinae. Sor poins by polar angle wih p o ge simple polygon. Consider poins in order, and discard hose ha would creae a clockwise urn p convex hull Proposiion. Convex hull reduces o soring. Pf. Graham scan algorihm. cos of soring Cos of convex hull. N log N + N. cos of reducion soring Shores pahs on edge-weighed graphs and digraphs Proposiion. Undireced shores pahs (wih nnegaive weighs) reduces o direced shores pah. Shores pahs on edge-weighed graphs and digraphs Proposiion. Undireced shores pahs (wih nnegaive weighs) reduces o direced shores pah. s 3 6 s 3 6 Pf. Replace each undireced edge by wo direced edges. s 3 11

4 Shores pahs on edge-weighed graphs and digraphs Proposiion. Undireced shores pahs (wih nnegaive weighs) reduces o direced shores pah. Shores pahs wih negaive weighs Cavea. Reducion is invalid for edge-weighed graphs wih negaive weighs (even if negaive cycles). s 7 s s 7 cos of shores pahs in digraph cos of reducion reducion creaes negaive cycles Cos of undireced shores pahs. E log V + E. Remark. Can sill solve shores-pahs problem in undireced graphs (if negaive cycles), bu need more sophisicaed echniques. 13 reduces o weighed n-biparie maching (!) 1 Some reducions involving familiar problems REDUCTIONS compuaional geomery d farhes pair combinaorial opimizaion undireced shores pahs (nnegaive) Designing algorihms Esablishing lower bounds Classifying problems median convex hull biparie maching direced shores pahs (nnegaive) arbirage elemen disincness soring maximum flow shores pahs ( neg cycles) d closes pair d Euclidean MST baseball eliminaion Delaunay riangulaion linear programming

5 Bird's-eye view Goal. Prove ha a problem requires a cerain number of seps. Ex. In decision ree model, any compare-based soring algorihm requires Ω(N log N) compares in he wors case. a < b b < c a < c a b c a < c b a c b < c a c b c a b b c a c b a Bad news. Very difficul o esablish lower bounds from scrach. argumen mus apply o all conceivable algorihms Good news. Spread Ω(N log N) lower bound o Y by reducing soring o Y. Linear-ime reducions Def. Problem X linear-ime reduces o problem Y if X can be solved wih: Linear number of sandard compuaional seps. Consan number of calls o Y. Ex. Almos all of he reducions we've seen so far. Esablish lower bound: If X akes Ω(N log N) seps, hen so does Y. If X akes Ω(N ) seps, hen so does Y. Menaliy. If I could easily solve Y, hen I could easily solve X. I can easily solve X. Therefore, I can easily solve Y. assuming cos of reducion is oo high Elemen disincness linear-ime reduces o closes pair More linear-ime reducions and lower bounds Closes pair. Given N poins in he plane, find he closes pair. Elemen disincness. Given N elemens, are any wo equal? Proposiion. Elemen disincness linear-ime reduces o closes pair. Pf. Elemen disincness insance: x1, x,..., xn. Closes pair insance: (x1, x1), (x, x),..., (xn, xn). Two elemens are disinc if and only if closes pair = 0. Elemen disincness lower bound. In quadraic decision ree model, any algorihm ha solves elemen disincness akes Ω(N log N) seps. Implicaion. In quadraic decision ree model, any algorihm for closes pair akes Ω(N log N) seps. allows quadraic ess of he form: xi < xj or (xi xk) (xj xk) < 0 soring soring d convex hull elemen disincness (N log N lower bound) Delaunay riangulaion d closes pair d Euclidean MST 3-sum 3-sum (conjecured N lower bound) 3-collinear 3-concurren dihedral roaion min area riangle 1 0

6 Esablishing lower bounds: summary Esablishing lower bounds hrough reducion is an imporan ool in guiding algorihm design effors. Q. How o convince yourself linear-ime convex hull algorihm exiss? A1. [hard way] Long fuile search for a linear-ime algorihm. A. [easy way] Linear-ime reducion from soring. Designing algorihms Esablishing lower bounds Classifying problems REDUCTIONS 1 Classifying problems: summary Desideraa. Problem wih algorihm ha maches lower bound. Ex. Soring, convex hull, and closes pair have complexiy N log N. Desideraa'. Prove ha wo problems X and Y have he same complexiy. Firs, show ha problem X linear-ime reduces o Y. Second, show ha Y linear-ime reduces o X. Conclude ha X and Y have he same complexiy. soring convex hull even if we don' kw wha i is! Cavea SORT. Given N disinc inegers, rearrange hem in ascending order. CONVEX HULL. Given N poins in he plane, idenify he exreme poins of he convex hull (in counerclockwise order). Proposiion. SORT linear-ime reduces o CONVEX HULL. Proposiion. CONVEX HULL linear-ime reduces o SORT. Conclusion. SORT and CONVEX HULL have he same complexiy. A possible real-world scenario. Sysem designer specs he APIs for projec. Alice implemens sor() using convexhull(). Bob implemens convexhull() using sor(). Infinie reducion loop! Who's faul? well, maybe so realisic 3

7 Ineger arihmeic reducions Ineger muliplicaion. Given wo N-bi inegers, compue heir produc. Brue force. N bi operaions. Ineger arihmeic reducions Ineger muliplicaion. Given wo N-bi inegers, compue heir produc. Brue force. N bi operaions problem arihmeic order of growh ineger muliplicaion a b M(N) ineger division a / b, a mod b M(N) ineger square a M(N) ineger square roo a M(N) ineger arihmeic problems wih he same complexiy as ineger muliplicaion Q. Is brue-force algorihm opimal? 6 Hisory of complexiy of ineger muliplicaion Linear algebra reducions year algorihm order of growh? brue force N Marix muliplicaion. Given wo N-by-N marices, compue heir produc. Brue force. N 3 flops. 16 Karasuba-Ofman N Toom-3, Toom- N 1.6, N 1.0 column j j 166 Toom-Cook N 1 + ε 0,1 0, 0,8 0,1 0, 0,3 0,1 0,1 0,16 0,11 0,3 0,6 171 Schönhage Srassen N log N log log N 007 Fürer N log N log*n?? N number of bi operaions o muliply wo N-bi inegers row i 0, 0,3 0, 0,6 0,1 0 0,7 0, 0 0,3 0,3 0,1 0, 0, 0 0, , 0, 0,8 0, 0,1 0, = i 0,7 0, 0,7 1, 0,36 0,1 0,33 0,7 0,1 0,1 0,13 0, used in Maple, Mahemaica, gcc, crypography, = 0.7 Remark. GNU Muliple Precision Library uses one of five differen algorihm depending on size of operands. 7 8

8 Linear algebra reducions Marix muliplicaion. Given wo N-by-N marices, compue heir produc. Brue force. N 3 flops. problem linear algebra order of growh marix muliplicaion A B MM(N) marix inversion A 1 MM(N) deerminan A MM(N) sysem of linear equaions Ax = b MM(N) LU decomposiion A = L U MM(N) leas squares min Ax b MM(N) numerical linear algebra problems wih he same complexiy as marix muliplicaion Q. Is brue-force algorihm opimal? Hisory of complexiy of marix muliplicaion year algorihm order of growh? brue force N 3 16 Srassen N Pan N Bini N Schönhage N. 18 Romani N Coppersmih-Wigrad N Srassen N.7 18 Coppersmih-Wigrad N Sroher N Williams N.377?? N + ε number of floaing-poin operaions o muliply wo N-by-N marices 30 Birds-eye view: revised Desideraa. Classify problems according o compuaional requiremens. complexiy order of growh examples linear N min, max, median,... linearihmic N log N soring, convex hull, closes pair, farhes pair,... M(N)? ineger muliplicaion, division, square roo,... MM(N)? marix muliplicaion, Ax = b, leas square, deerminan,... NP-complee probably N b 3-SAT, IND-SET, ILP,... Summary Reducions are imporan in heory o: Esablish racabiliy. Esablish inracabiliy. Classify problems according o heir compuaional requiremens. Reducions are imporan in pracice o: Design algorihms. Design reusable sofware modules. - sacks, queues, prioriy queues, symbol ables, ses, graphs - soring, regular expressions, Delaunay riangulaion - MST, shores pah, maxflow, linear programming Deermine difficuly of your problem and choose he righ ool. - use exac algorihm for racable problems - use heurisics for inracable problems Good news. Can pu many problems i equivalence classes. 31 3

Algorithms. Algorithms 6.5 REDUCTIONS. introduction designing algorithms establishing lower bounds classifying problems intractability

Algorithms. Algorithms 6.5 REDUCTIONS. introduction designing algorithms establishing lower bounds classifying problems intractability Algorithms ROBERT SEDGEWICK KEVIN WAYNE 6.5 REDUCTIONS Algorithms F O U R T H E D I T I O N ROBERT SEDGEWICK KEVIN WAYNE introduction designing algorithms establishing lower bounds classifying problems