Randomized Algorithms

Transcription

1 // Copyright he McGrw-Hill Compies, Ic. Permissio required for reproductio or disply. Rdomized Algorithms Copyright he McGrw-Hill Compies, Ic. Permissio required for reproductio or disply. Copyright he McGrw-Hill Compies, Ic. Permissio required for reproductio or disply.

2 // Copyright he McGrw-Hill Compies, Ic. Permissio required for reproductio or disply. Copyright he McGrw-Hill Compies, Ic. Permissio required for reproductio or disply. Quicsort Divide d coquer Quicsort -elemet rry:. Divide: Prtitio the rry ito two subrrys roud pivot x such tht elemets i lower subrry x elemets i upper subrry. x x x. Coquer: Recursively sort the two subrrys. 3. Combie: rivil. Key: Lier-time prtitioig subroutie.

3 // Prtitioig subroutie PARIIONA, p, q A[p.. q] x A[p] pivot = A[p] Ruig time i p = O for for j p + to q elemets. do if A[ j] x the i i + exchge A[i] A[ j] exchge A[p] A[i] retur i Ivrit: x x x? p i j q Pseudocode for quicsort QUICKSORA, p, r if p < r the q PARIIONA, p, r QUICKSORA, p, q QUICKSORA, q+, r Iitil cll: QUICKSORA,, Worst-cse of quicsort Iput sorted or reverse sorted. Prtitio roud mi or mx elemet. Oe side of prtitio lwys hs o elemets. rithmetic series Worst-cse recursio tree = + + c 3

4 // Worst-cse recursio tree = + + c c c c h = Worst-cse recursio tree = + + c c c c = + = Best-cse lysis For ituitio oly! If we re lucy, PARIION splits the rry evely: = / + = lg sme s merge sort Wht if the split is lwys 9 :? 9 Wht is the solutio to this recurrece? Alysis of lmost-best cse c 9 c log / c c c c c O leves c c c 4

5 // Alysis of lmost-best cse c 9 c log log / c c c c lg Lucy! c O leves c c c c log c log /9 + O More ituitio Suppose we lterte lucy, ulucy, lucy, ulucy, lucy,. L = U/ + U = L + lucy ulucy Solvig: L = L/ + / + = L/ + = lg Lucy! How c we me sure we re usully lucy? Rdomized quicsort IDA: Prtitio roud rdom elemet. Ruig time is idepedet of the iput order. No ssumptios eed to be mde bout the iput distributio. No specific iput elicits the worst-cse behvior. he worst cse is determied oly by the output of rdom-umber geertor. Rdomized quicsort lysis Let = the rdom vrible for the ruig time of rdomized quicsort o iput of size, ssumig rdom umbers re idepedet. For =,,,, defie the idictor rdom vrible = if PARIION geertes : split, otherwise. [ ] = Pr{ = } = /, sice ll splits re eqully liely, ssumig elemets re distict. 5

6 // 6 Alysis cotiued = + + if : split, + + if : split, M + + if : split,. Clcultig expecttio ] [ e expecttios of both sides. Clcultig expecttio ] [ Lierity of expecttio. Clcultig expecttio ] [ Idepedece of from other rdom choices.

7 // 7 Clcultig expecttio ] [ Lierity of expecttio; [ ] = /. Clcultig expecttio ] [ Summtios hve ideticl terms. Hiry recurrece ] [ he =, terms c be bsorbed i the. Prove: [] lg for costt >. Use fct: 8 lg lg exercise. Choose lrge eough so tht lg domites [] for sufficietly smll. Substitutio method lg Substitute iductive hypothesis.

8 // Substitutio method Use fct. lg lg 8 Substitutio method lg lg 8 lg 4 xpress s desired residul. Substitutio method lg lg 8 lg 4 lg, if is chose lrge eough so tht /4 domites the. Quicsort i prctice Quicsort is gret geerl-purpose sortig lgorithm. Quicsort is typiclly over twice s fst s merge sort. Quicsort c beefit substtilly from code tuig. Quicsort behves well eve with cchig d virtul memory. 8

9 // Precise Alysis of Quicsort Precise Alysis of Quicsort Wht re lessos lert? Where c it be used? Quicsort Algorithm Give usorted rry x x x x Quicsort Algorithm Quicsort for elemets; Prtitio rry A[..] usig pivot x; this tes + comprisos x x x Recursively A[..-]; Recursively A[+..]; Alysis of Quicsort Let = the verge time te to sort rry of size usig Quicsort If pivot x eds up i positio, = x x x Prob pivot is t pos = / for ll 9

10 // he, we hve the followig recurrece: xpd the summtios Alysis of Quicsort he, we hve the followig recurrece: Get rid of depedece o full history... Alysis of Quicsort he, we hve the followig recurrece: Divide by Alysis of Quicsort he, we hve the followig recurrece: Now telescope Alysis of Quicsort

11 // he, we hve the followig recurrece: Now telescope 3 3 Alysis of Quicsort he, we hve the followig recurrece: Alysis of Quicsort he, we hve the followig recurrece:... 3 O H the Hrmoic series is H Alysis of Quicsort Averge ruig time of Quicsort: O H is the uler costt l γ γ H l O lg l O lg.386 O Alysis of Quicsort

12 // Usig the result Usig similr lysis, we c show Averge height of biry serch tree BS.386 lg Sortig i lier time ry it out yourself 8. Lower boud for sortig he decisio tree model : heorem 9.. Ay decisio tree tht sorts elemets hs height log. Proof: h! l, h log! lg. : 3 : 3 <,,3> : 3 <,,3> : 3 Corollry 9. Hepsort d merge sort re symptoticlly optiml comprisos. <,3,> <3,,> <,3,> <3.,>

13 // 8.4 Bucet sort Alysis BUCK_SORA legth [ A] for i to 3 do isert A[ i] ito list BA[ i] 4 for i to - 5 do sort list B[ i] with isertio sort 6 coctete B[ ], B[ ],..., B[ ] together i order he ruig time of bucet sort is O i. i tig expecttios of both sides d usig lierity of expecttio, we hve [ ] O i i [ O i ] i O[ i ] i We clim tht [ i ] / We defie idictor rdom vribles ij = I {A[j] flls i bucet i} for i =,,, - d j =,,,. thus, i ij. j [ ] i ij j ij i j ij ij i j j j [ ] [ ], ij j j ij i j 3

14 // Idictor rdom vrible ij is with probbility / d otherwise, d therefore [ ij ] Whe j, the vribles ij d i re idepedet, d hece [ ij i ] [ ij ] [ i ]. i [ ] j j j We c coclude tht the expected time for bucet sort is + O-/=. Aother lysis: O [ i ] O [ i ] O O i i i Becuse i Vr [ i ] [ i ] [ i ] p where p= Order Sttistics Vr [ i ] p p i Bsic Probbility heory 4

15 // Order Sttistic i th order sttistic: i th smllest elemet of set of elemets. Miimum: first order sttistic. Mximum: th order sttistic. Medi: hlf-wy poit of the set. Uique, whe is odd occurs t i = +/. wo medis whe is eve. Lower medi, t i = /. Upper medi, t i = /+. For cosistecy, medi will refer to the lower medi. Selectio Problem Selectio problem: Iput: A set A of distict umbers d umber i, with i. Output: the elemet x A tht is lrger th exctly i other elemets of A. C be solved i O lg time. How? We will study fster lier-time lgorithms. For the specil cses whe i = d i =. For the geerl problem. Selectio i xpected Lier ime Rdomized Quicsort: review Modeled fter rdomized quicsort. xploits the bilities of Rdomized-Prtitio RP. RP returs the idex i the sorted order of rdomly chose elemet pivot. If the order sttistic we re iterested i, i, equls, the we re doe. lse, reduce the problem size usig its other bility. RP rerrges the other elemets roud the rdom pivot. If i <, selectio c be rrowed dow to A[.. ]. lse, select the i th elemet from A[+..]. Assumig RP opertes o A[..]. For A[p..r], chge ppropritely. QuicsortA, p, r if p < r the q := Rd-PrtitioA, p, r; QuicsortA, p, q ; QuicsortA, q +, r fi A[p..r] 5 A[p..q ] Prtitio 5 A[q+..r] 5 5 Rd-PrtitioA, p, r i := Rdomp, r; A[r] A[i]; x, i := A[r], p ; for j := p to r do if A[j] x the i := i + ; A[i] A[j] fi od; A[i + ] A[r]; retur i + 5

16 // Rdomized-Select Rdomized-SelectA, p, r, i // select ith order sttistic.. if p = r. the retur A[p] 3. q Rdomized-PrtitioA, p, r 4. q p + 5. if i = 6. the retur A[q] 7. elseif i < 8. the retur Rdomized-SelectA, p, q, i 9. else retur Rdomized-SelectA, q+, r, i Alysis Worst-cse Complexity: As we could get ulucy d lwys recurse o subrry tht is oly oe elemet smller th the previous subrry. Averge-cse Complexity: Ituitio: Becuse the pivot is chose t rdom, we expect tht we get rid of hlf of the list ech time we choose rdom pivot q. Why d ot lg? Averge-cse Alysis Defie Idictor RV s, for. = I{subrry A[p q] hs exctly elemets}. Pr{subrry A[p q] hs exctly elemets} = / for ll = Hece, [ ] = /. Let be the RV for the time required by Rdomized-Select RS o A[p q] of elemets. Determie upper boud o []. Averge-cse Alysis A cll to RS my ermite immeditely with the correct swer, Recurse o A[p..q ], or Recurse o A[q+..r]. o obti upper boud, ssume tht the i th smllest elemet tht we wt is lwys i the lrger subrry. RP tes O time o problem of size. Hece, recurrece for is: mx, O For give cll of RS, = for exctly oe vlue of, d = for ll other. 6

17 // [ ] Averge-cse Alysis mx, O mx, O igexpecttio, wehve mx, O [ mx, ] O [ ] [ mx, ] O [ mx, ] O by lierity of expecttio by q. C.3 by q. 9. Averge-cse Alysis Cotd. if / mx, if / he summtio is expded / [ ] O / If is odd, thru / occur twice d / occurs oce. If is eve, thru / occur twice. hus,wehve [ ] / [ ] O. Averge-cse Alysis Cotd. We solve the recurrece by substitutio. Guess [] = O. [ ] c / / c c c 4 3c c 4 c c c 4 c c c c, 4 c c 4 c/ 4 c/, or c/ c, if c 4. c/ 4 c 4 hus, if we ssume = O for < c/c 4, we hve [] = O. Biry Serch rees 7

18 // Stdrd trees 8 xmple ree structure depeds o the order of isertios ito the iitilly empty tree Height c icrese lierly, but it c lso be i Olog, more precisely log Isert Preorder: 7,, 7, 4,, Postorder: 7,, 4,,, 7 Iorder: 7,,, 4, 7, xmple for Serch, Isertio, Deletio Iterl pth legth Iterl pth legth I: mesure for judgig the qulity of serch tree t. Recursive defiitio:. If t is empty, the It =.. For tree t with left subtree t l d right subtree t r : It := It l + It r + # odes i t. Appretly: It p p iterl ode i t depth p 7 7 8

19 // Averge serch pth legth For tree t the verge serch pth legth is defied by: Dt = It/, = # iterl odes i t Iterl pth: best cse We obti complete biry tree Questio: Wht is the size of Dt i the best worst verge cse for tree t with iterl odes? Iterl pth: worst cse Rdom trees Without loss of geerlity, let {,,} be the eys to be iserted. Let s,, s be rdom permuttio of these eys. Hece, the probbility tht s hs the vlue, Ps = = /. If is the first ey, will be stored i the root. he the left subtree cotis - elemets the eys,, - d the right subtree cotis - elemets the eys +,,

20 // 77 xpected iterl pth legth I : xpecttio for the iterl pth legth of rdomly geerted biry serch tree with odes Appretly we hve: I I I I I I I Behuptug: I.386 log Olog. d hece Proof I * I * I * From the lst two equtios it follows tht * I * I I I * I * I I I I I. 78 Proof By iductio over it is possible to show tht for ll : I H 3 H... is the -th hrmoic umber, which c be estimted s follows: H l g O where g the so-clled uler costt Proof 3 hus, I l 3 g * l g O d hece, I l l 3 g... l * log 3 g... log e log l * log 3 g... log e l.386log 3 g...

21 // Observtios Serch, isertio d deletio of ey i rdomly geerted biry serch tree with eys c be doe, o verge, i Olog steps. I the worste cse, the complexity c be Ω. Oe c show tht the verge distce of ode from the root i rdomly geerted tree is oly bout 4% bove the optiml vlue. However, by the restrictio to the symmetricl successor, the behviour becomes worse. If updte opertios re crried out i rdomly geerted serch tree with eys, the expected verge serch pth is oly Θ. ypicl biry tree for rdom sequece of eys 8 8 Resultig biry tree fter updtes 83