2. ALGORITHM ANALYSIS

Transcription

1 2. ALGORITHM ANALYSIS computatioal tractability survey of commo ruig times 2. ALGORITHM ANALYSIS computatioal tractability survey of commo ruig times Lecture slides by Kevi Waye Copyright 2005 Pearso-Addiso Wesley Last updated o 11/3/17 5:41 AM A strikigly moder thought Brute force As soo as a Aalytic Egie exists, it will ecessarily guide the future course of the sciece. Wheever ay result is sought by its aid, the questio will arise By what course of calculatio ca these results be arrived at by the machie i the shortest time? Charles Babbage (1864) Brute force. For may otrivial problems, there is a atural brute-force search algorithm that checks every possible solutio. Typically takes 2 time or worse for iputs of size. Uacceptable i practice. how may times do you have to tur the crak? Aalytic Egie 3 4

2 Polyomial ruig time Polyomial ruig time Desirable scalig property. Whe the iput size doubles, the algorithm should slow dow by at most some costat factor C. Def. A algorithm is poly-time if the above scalig property holds. We say that a algorithm is efficiet if it has a polyomial ruig time. Justificatio. It really works i practice! I practice, the poly-time algorithms that people develop have low costats ad low expoets. Breakig through the expoetial barrier of brute force typically exposes some crucial structure of the problem. There exist costats c > 0 ad d > 0 such that, for every iput of size, the ruig time of the algorithm is bouded above by c d primitive computatioal steps. choose C = 2 d Exceptios. Some poly-time algorithms do have high costats ad/or expoets, ad/or are useless i practice. 120 Map graphs i polyomial time Q. Which would you prefer vs l? Mikkel Thorup Departmet of Computer Sciece, Uiversity of Copehage Uiversitetsparke 1, DK-2100 Copehage East, Demark mthorup@diku.dk Abstract Che, Grigi, ad Papadimitriou (WADS 97 ad STOC 98) have itroduced a modified otio of plaarity, where two faces are cosidered adjacet if they share at least oe poit. The correspodig abstract graphs are called map graphs. Che et.al. raised the questio of whether map graphs ca be recogized i polyomial time. They showed that the decisio problem is i NP ad preseted a polyomial time algorithm for the special case where we allow at most 4 faces to itersect vo Neuma (1953) Nash (1955) Gödel (1956) Cobham (1964) Edmods (1965) Rabi (1966) i ay poit if oly 3 are allowed to itersect i a poit, we get the usual plaar graphs. Che et.al. cojectured that map graphs ca be recogized i polyomial time, ad i this paper, their cojecture is settled affirmatively. 5 6 Worst-case aalysis Types of aalyses Worst case. Ruig time guaratee for ay iput of size. Geerally captures efficiecy i practice. Dracoia view, but hard to fid effective alterative. Exceptios. Some expoetial-time algorithms are used widely i practice because the worst-case istaces seem to be rare. Worst case. Ruig time guaratee for ay iput of size. Ex. Heapsort requires at most 2 log 2 compares to sort elemets. Probabilistic. Expected ruig time of a radomized algorithm. Ex. The expected umber of compares to quicksort elemets is ~ 2 l. Amortized. Worst-case ruig time for ay sequece of operatios. Ex. Startig from a empty stack, ay sequece of push ad pop operatios takes O() primitive computatioal steps usig a resizig array. Average-case. Expected ruig time for a radom iput of size. Ex. The expected umber of character compares performed by 3-way radix quicksort o uiformly radom strigs is ~ 2 l. simplex algorithm Liux grep k-meas algorithm Also. Smoothed aalysis, competitive aalysis,

3 Why it matters 2. ALGORITHM ANALYSIS computatioal tractability survey of commo ruig times 9 Big O otatio Notatioal abuses Upper bouds. f() is O(g()) if there exist costats c > 0 ad 0 0 such that f() c g() for all 0. Ex. f() = f() is O( 2 ). f() is also O( 3 ). choose c = 50, 0 = 1 f() is either O() or O( log ). Typical usage. Isertio sort makes O( 2 ) compares to sort elemets. Alterate defiitio. f() is O(g()) if sup f() g() < 0 c g() f() Equals sig. O(g()) is a set of fuctios, but computer scietists ofte write f() = O(g()) istead of f() O(g()). Ex. Cosider g 1() = 5 3 ad g 2() = 3 2. We have g 1() = O( 3 ) = g 2(). Thus, g 1() = g 2(). Domai. The domai of g() is typically the atural umbers { 0, 1, 2,. Sometimes we restrict to a subset of the atural umbers. Other times we exted to the reals. No-egative fuctios. Whe usig big O otatio, we assume that the fuctios ivolved are (asymptotically) o-egative. Bottom lie. OK to abuse otatio; ot OK to misuse it

4 Big Omega otatio Big Theta otatio Lower bouds. f() is Ω(g()) if there exist costats c > 0 ad 0 0 such that f() c g() for all 0. Ex. f() = f() is both Ω( 2 ) ad Ω(). f() is either Ω( 3 ) or Ω( 3 log ). Typical usage. Ay compare-based sortig algorithm requires Ω( log ) compares i the worst case. choose c = 32, 0 = 1 Meaigless statemet. Ay compare-based sortig algorithm requires at least O( log ) compares i the worst case. 0 f() c g() Tight bouds. f() is Θ(g()) if there exist costats c1 > 0, c2 > 0, ad 0 0 such that c1 g() f() c2 g() for all 0. Ex. f() = f() is Θ( 2 ). f() is either Θ() or Θ( 3 ). choose c1 = 32, c2 = 50, 0 = 1 Typical usage. Mergesort makes Θ( log ) compares to sort elemets. betwee ½ log2 ad log2 0 c2 g() f() c1 g() Useful facts Propositio. If f(), the f() is Θ(g()). g() = c > 0 Pf. By defiitio of the it, there exists 0 such that for all 0 Thus, f() 2 c g() for all 0, which implies f() is O(g()). Similarly, f() ½ c g() for all 0, which implies f() is Ω(g()). Propositio. If 1 2 c f() g() 2c f(), the f () is O(g()) but ot Θ(g()). g() = 0 Asymptotic bouds for some commo fuctios Polyomials. Let f() = a 0 + a a d d with a d > 0. The, f() is Θ( d ). Pf. Logarithms. Θ(log a ) is Θ(log b ) for ay costats a, b > 0. Logarithms ad polyomials. For every d > 0, log is O( d ). Expoetials ad polyomials. For every r > 1 ad every d > 0, d is O(r ). Pf. a 0 + a a d d d = a d > 0 d r = 0 o eed to specify base (assumig it is a costat) 15 16

5 Big O otatio with multiple variables Upper bouds. f(m, ) is O(g(m, )) if there exist costats c > 0, m 0 0, ad 0 0 such that f(m, ) c g (m, ) for all 0 ad m m 0. Ex. f(m, ) = 32m m f(m, ) is both O(m ) ad O(m 3 ). f(m, ) is either O( 3 ) or O(m 2 ). 2. ALGORITHM ANALYSIS computatioal tractability survey of commo ruig times Typical usage. Breadth-first search takes O(m + ) time to fid a shortest path from s to t i a digraph with odes ad m edges. 17 Liear time: O() Liear time: O() Liear time. Ruig time is proportioal to iput size. Computig the maximum. Compute maximum of umbers a 1,, a. max a 1 for i = 2 to { if (a i > max) max a i Claim. Computig the maximum of umbers takes O() time. Merge. Combie two sorted lists A = a 1, a 2,, a with B = b 1, b 2,, b ito sorted whole. i = 1, j = 1 while (both lists are oempty) { if (a i b j ) apped a i to output list ad icremet i else(a i b j )apped b j to output list ad icremet j apped remaider of oempty list to output list Claim. Mergig two lists, each of legth, takes O() time. Pf. After each compare, the legth of output list icreases by

6 Liearithmic time: O( log ) Quadratic time: O( 2 ) O( log ) time. Arises i divide-ad-coquer algorithms. Sortig. Mergesort ad heapsort are sortig algorithms that perform O( log ) compares. Largest empty iterval. Give time stamps x 1,, x o which copies of a file arrive at a server, what is largest iterval whe o copies of file arrive? O( log ) solutio. Sort the time stamps. Sca the sorted list i order, idetifyig the maximum gap betwee successive time-stamps. Ex. Eumerate all pairs of elemets. Closest pair of poits. Give a list of poits i the plae (x 1, y 1 ),, (x, y ), fid the pair that is closest to each other. O( 2 ) solutio. Try all pairs of poits. mi (x 1 - x 2 ) 2 + (y 1 - y 2 ) 2 for i = 1 to { for j = i+1 to { d (x i - x j ) 2 + (y i - y j ) 2 if (d < mi) mi d Remark. Ω( 2 ) seems ievitable, but this is just a illusio. [see Chapter 5] Cubic time: O( 3 ) Polyomial time: O( k ) Ex. Eumerate all triples of elemets. Set disjoitess. Give sets S 1,, S each of which is a subset of 1, 2,,, is there some pair of these which are disjoit? O( 3 ) solutio. For each pair of sets, determie if they are disjoit. foreach set S i { foreach other set S j { foreach elemet p of S i { Idepedet set of size k. Give a graph, are there k odes such that o two are joied by a edge? O( k ) solutio. Eumerate all subsets of k odes. foreach subset S of k odes { check whether S is a idepedet set if (S is a idepedet set) report S is a idepedet set k is a costat determie whether p also belogs to S j if (o elemet of S i belogs to S j ) report that S i ad S j are disjoit Check whether S is a idepedet set takes O(k 2 ) time. = k Number of k-elemet subsets = O(k 2 k / k!) = O( k ). poly-time for k=17, but ot practical ( 1)( 2) ( k + 1) k(k 1)(k 2) 1 k k! 23 24

7 Expoetial time: 2 O ( k ) Subliear time: o() Idepedet set. Give a graph, what is maximum cardiality of a idepedet set? O( 2 2 ) solutio. Eumerate all subsets. S* φ foreach subset S of odes { check whether S is a idepedet set if (S is largest idepedet set see so far) update S* S Search i a sorted array. Give a sorted array A of umbers, is a give umber x i the array? O(log ) solutio. Biary search. lo 1, hi while (lo hi) { mid (lo + hi) / 2 if (x < A[mid]) hi mid - 1 else if (x > A[mid]) lo mid + 1 else retur yes retur o f() Defiitio. f() is o(g()) if. g() =