CSE 417: Algorithms and Computational Complexity

Transcription

1 Time CSE 47: Algorithms ad Computatioal Readig assigmet Read Chapter of The ALGORITHM Desig Maual Aalysis & Sortig Autum 00 Paul Beame aalysis Problem size Worst-case complexity: max # steps algorithm takes o ay iput of size Best-case complexity:mi # steps algorithm takes o ay iput of size Average -case complexity: avg # steps algorithm takes o iputs of size The complexity of a algorithm associates a umber T(), the best/worst/average-case time the algorithm takes, with each problem size. Mathematically, T: N + fi R + that is T is a fuctio that maps positive itegers givig problem size to positive real umbers givig umber of steps. 3 4 Why Worst-Case Aalysis? T() Appropriate for time-critical applicatios, e.g. avioics Ulike Average-Case, o debate about what the right defiitio is Aalysis ofte easier Result is ofte represetative of "typical" problem istaces Problem size Of course there are exceptios 5 6

2 Time O-otatio etc Give two fuctios f ad g:nfir f() is O(g()) iff there is a costat c>0 so that f() is evetually always c g() f() is W(g()) iff there is a costat c>0 so that f() is evetually always c g() f() is Q(g()) iff there is are costats c ad c >0 sothat evetually always c g() f() c g() log log T() Problem size 7 8 Examples is O( ) also O( 3 ) for all is W ( ) also W () for all 6 Therefore also is Q( ) is ot O() also ot W( 3 ) Note: I do t use otatio f()=o(g()) Workig with O-W-Q otatio Claim: For ay a, b> log a is Q(log b ) log a =log a b log b so lettig c=log a b we get that clog b log a clog b Claim: For ay a ad b>0, (+a) b is Θ( b ) (+a) b () b for a = b b = c b for c= b so (+a) b is O( b ) (+a) b (/) b for a = -b b =c for c = -b so (+a) b is W( b ) 9 0 Aalysis aalysis overview We have looked at type of complexity aalysis worst-case, best-case, average-case types of fuctio bouds O, W, Q These two cosideratios are orthogoal to each other oe ca do ay type of fuctio boud with ay type of complexity aalysis Alg A differet ruig time for each iput strig Type of Aalysis Fuctio mappig iput legth to ruig time Type of Boud T() grows like log Nice formula approximatig rutime of A Usually we represet the fuctio i the middle usig a recurrece relatio rather tha explicitly

3 Geeral algorithm desig paradigm Fid a way to reduce your problem to oe or more smaller problems of the same type Whe problems are really small solve them directly Example Mergesort o a problem of size at least Sort the first half of the umbers Sort the secod half of the umbers Merge the two sorted lists o a problem of size do othig 3 4 Cost of Merge Recurrece relatio for Mergesort Give two lists to merge size ad m Maitai poiter to head of each list Move smaller elemet to output ad advace poiter m +m Worst case +m- comparisos I total icludig other operatios let s say each merge costs 3 per elemet output ceilig roud up T()=T( / )+T( / )+3 T()= floor roud dow for Ca use this to figure out T for ay value of T(5)=T(3)+T()+3x5 =(T()+T()+3x3)+(T()+T()+3x)+5 =((T()+T()+3x)++9)+(++6)+5 = =4 T()= 3 log Best case mi(,m) comparisos 5 6 Isertio Sort For i= to do j i while(j> & X[ j ] > X[ j-]) do swap X[ j ] ad X[ j-] i.e., For i= to do Isert X[i] i the sorted list X[],...,X[i-] Recurrece relatio for Isertio Sort Let T (i) be the worst case cost of creatig list that has first i elemets sorted out of. We wat to kow T () The isertio of X[i] makes up to i- comparisos i the worst case T (i)=t (i-)+i- for i> T ()=0 sice a list of legth is always sorted Therefore T ()=(-)/ 7 8 3

4 Solvig recurrece relatios e.g. T()=T(-)+f() for T(0)=0 solutio is T()= i = fi () Isertio sort: T (i)=t (i-)+i- so T ()= i= ( i - ) =(-)/ Arithmetic Series S= (-) S= (-)+(-)+(-3) S= {- terms} S=(-) so S=(-)/ Works geerally whe f(i)=a i+b for all i Sum = average term size # of terms 9 0 Quicksort Quicksort(X,left,right) if left < right split=partitio(x, left, right) Quicksort(X, left, split-) Quicksort(X, split+, right) Partitio - two figer algorithm Partitio(X, left,right) choose a radom elemet to be a pivot ad pull it out of the array, say at left ed maitai two figers startig at each ed of the array slide them towards each other util you get a pair of elemets where right figer has a smaller elemet ad left figer has a bigger oe (whe compared to pivot) swap them ad repeat util figers meet put the pivot elemet where they meet Partitio - two figer algorithm Partitio(X,left,right) swap X[left], X[radom(left, right)] pivot X[left]; L left; R right while L<R do while (X[L] pivot & L right) do L L+ while (X[R] > pivot & R left) do R R- if L>R the swap X[L],X[R] swap X[left],X[R] retur R I practice ofte choose pivot i fixed way as middle elemet for small arrays media of st, middle, ad last for larger arrays media of 3 medias of 3 (9 elemets i all) for largest arrays four figer algorithm is better also maitai two groups at each ed of elemets equal to the pivot swap them all ito middle at the ed of Partitio equal elemets are bad cases for two figers 3 4 4

5 Quicksort Aalysis Quicksort Aalysis Average Case Partitio does - comparisos o a list of legth pivot is compared to each other elemet If pivot is i th largest the two sub-problems are of size i- ad -i If pivot is always i the middle get T()=T(/)+- comparisos T() = log better tha Mergesort If pivot is always at the ed get T()=T(-)+- comparisos T() = (-)/ like Isertio Sort 5 Recall Partitio does - comparisos o a list of legth If pivot is i th largest the two sub-problems are of size i- ad -i Pivot is equally likely to be ay oe of st through th largest T() = + T(i ) + T( i) i= ( ) 6 Quicksort aalysis Quicksort aalysis T() = + ( T(i ) + T( i) ) i= T() + T() T( -) = - + T() = (-) + T() + T() T( -) ( + )T( + ) = ( + ) + T() + T() T() ( + )T( + )-T() = T() + ( + )T( + ) = ( + )T() + T( + ) T() = (+ )(+ ) 7 T() Let Q() = + Q( + ) Q() + + Q() ( ) = H l =. 38log 3 (Recallthat l = T().38log /x dx) 8 Gestalt Aalysis of Quicksort Quicksort executio Look at elemets that eded up i positios j < k of the fial sorted array The expected # of comparisos i Qsort = the expected # of j < k such that the j th ad k th elemets were compared = sum j < k Pr[j th ad k th elts were compared] j k

6 Gestalt Aalysis of Quicksort Look at elemets that ed up i positios j < k of the fial sorted array What is the chace that they were compared to each other durig the course of the algorithm? They started off together i the same sub-problem They eded up i differet sub-problems The oly time they might have bee compared to each is whe they were split ito separate subproblems Gestalt Aalysis of Quicksort The oly time they might have bee compared to each is whe they were split ito separate sub-problems The oly way they could be split i a step is if the pivot was a elemet that eded up betwee j th ad k th i the fial sorted array The pivot could be j th or k th Those are the oly cases whe they are compared Chaces of that happeig is out of (k -j+) equally likely possibilities 3 3 Total cost of Quicksort Total expected cost k> jk-j+ The cotributio for each j is at most loge 3 4 Total log e =.38 log 33 6