Introducton to Algorthms October 4, 2002 Massachusetts Insttute of Technology 6046J/18410J Professors Erk Demane and Shaf Goldwasser Handout 14 Problem Set 3 Solutons (Exercses were not to be turned n, but we re provdng the solutons for your own nterest) Exercse 3-1 Wth groups of 3, the medan-of-medans s larger than (about) of the elements, and smaller than (about) of the elements Thus, the recursve call s on at most elements The recurrence s then ; solvng t by Akra-Bazz, we get only that So n fact wth groups of 3, selecton may not complete n tme Wth groups of 7, the medan-of-medans s larger than (about) elements Thus, the recursve call s on at most elements The recurrence s then! ", whch we solve as # Exercse 3-2 Suppose we are gven a black-box lnear-tme algorthm to fnd the medan We wll descrbe an algorthm to select the th element out of unordered elements: frst we use the black-box to fnd the medan, then we partton around t, yeldng a lower half and upper half If &%' )(, then we are done (the medan s the element we want) If +* %' )(, we recursvely select the th element from the lower half If, %' )(, we recursvely select the -/ %' 0( th element from the upper half (whch s the th element of the entre array) A recurrence for the runnng tme of ths procedure s 1 2 3, whch we solve as 1, as desred Exercse 3-3 We defne an ndcator random varable 46567 8 to be 1 f : -;! : =<, and 0 otherwse Then the number of collsons s the sum of random varables: > 5? @ 8 4A567 8, and we want to know what ts expectaton s Usng lnearty of expectaton, we get BDCEGF 4A567 8IHJ F B3 4L567 8NM 5? @ 8 5? @ 8 Because we are assumng unform hashng, F 5? @ 8P : -; : -< MR : -; S : -< M UT0 )V for every par ;XW are Y[Z\I] such pars, so the expected number of collsons s T0^ V Exercse 3-4 < There We prove correctness of RADIX-SORT by nducton on the dgt postons, from least- to most-sgnfcant When we say the numbers are sorted by ther last dgts, we mean that they are sorted f one only consders ther values modulo _a (where _ s the base we re usng) Upon ntalzaton, the numbers are trvally sorted by ther last 0 dgts Now suppose the numbers are sorted by ther last dgts, and we then we run COUNTING-SORT by ther bt st dgts Consder the last ct dgts of any two elements ; and <, where ;d fe _ahgj,k<d fe _flgj (we need to prove that ; now follows < ) If the mt st dgt of ; s larger than that of <, then ndeed the sort we just performed put ; after < If ther nt st dgts are equal, then ;d fe _aò,x<d fe _p, so by the nductve assumpton ; appeared after < before we sorted by the T st dgts By the stablty of the sortng subroutne, ; stll follows < Therefore the elements are sorted by ther last mt dgts By nducton, after sortng by the most sgnfcant dgt, the elements are completely sorted
Z 2 Handout 14: Problem Set 3 Solutons Problem 3-1 RADIX-SORT on larger ranges (a) The th dgt s % ; ( d fe To see ths, note that dvdng by and takng the floor s just shftng rght the base- representaton of ; by I T dgts, and takng that value d fe s just keepng the least sgnfcant remanng dgt If we can compute n T0 tme, then an algorthm for computng the th dgt s straghtforward Alterately, we can get the dgts n sequence by computng T \ R6R6R, whch yelds T0 tme per dgt (b) Usng base 10, the numbers have dgts Each COUNTING-SORT call takes T tme, so the runnng tme of RADIX-SORT s (c) If we use base, the numbers have dgts, and by part (a) we can compute each dgt n 3 T0 tme Therefore each COUNTING-SORT call takes m # tme, so the runnng tme of RADIX-SORT s # Ths method extends to numbers n the range R6R6R I T for any constant Problem 3-2 Majorty and pluralty elements (a) If a majorty element exsts, then t must by the medan of the elements n the array To see ths, note that f the array were n sorted order, and the medan were some other element, then there would not be enough room on ether sde of the medan for all the copes of the majorty element Therefore we smply SELECT the medan ;, and then pass over the array to count the number of tmes ; appears If t appears more than tmes, we output ;, otherwse we output (because ; s not a majorty element, and no other element can be ether) The runnng tme of ths algorthm s for SELECT, and 3 to pass over the array agan, for total (b) By the above reasonng, any T0 _ -pluralty element must have rank _ for some n T R6R6R _ Therefore, our algorthm works as follows: for each n T R6R R _, SELECT the _ th order statstc ;, then pass over the entre array and output ; f ; appears at least _ tmes Each teraton of algorthm takes 3 tme, and there are _ teratons, for a runnng tme of 3 _ Problem 3-3 Set operatons (a) We use any unversal hash famly we have seen n lecture Frst, choose a random, and construct a hash table wth slots Then nsert every element of nto the hash table usng Next, for each element (where bw ; ), search for ;3, so output (snce both ; n the hash table If t exsts, then W
R Handout 14: Problem Set 3 Solutons 3 and are n and sum to ; ) If ;, then t only occurs once because s a set, so there s no other element such that ; If all the searches fal, then output, because there s no par of elements n that sum to ; Constructng the hash table takes 3 expected tme, because each hash operaton takes 3 T 2+ 3 T T0 + 3 T expected tme and we can sum the tmes by lnearty of expectaton The runnng tme of the search stage s agan, because each search takes T0 expected tme (b) Ths soluton agan uses a unversal hash famly We descrbe an algorthm to determne f ; ths can be used once n each drecton to determne whether the sets are equal Frst choose a random Next, buld a hash table of sze and hash all of s elements nto t Then, for each, search for n the hash table If all the searches succeed, then, otherwse not Buldng the hash table takes 3 expected tme, and so do all the searches Problem 3-4 (a) If Unversal and 2-unversal hashng s 2-unversal, then for every par of dstnct keys ; < and for every R6R R IV T o '; : -<! M T0 V \ The only way for ; and < to collde s f they both take on the same value Then : -; : =< M > @ o '; : -< M #V" V \ kt0 V, so s unver- sal by defnton ; < (b) We set VD, the number of keys to be 2, and wll have 2 hash functons Here s the famly : 0 0 \ 1 0 It s easy to see that s unversal, because ; and < collde only under, whch s selected wth probablty T0 T0 )V However, s not 2-unversal, because : '; : =< never equals the par 6T or T 6T for any (c) The adversary can n fact force a collson We modfy the above hash famly by addng a key as follows: ; < 0 0 1 \ 1 0 1 Note that ths famly s stll unversal: ; and only collde under \, and < and never collde (ths s allowed by the defnton of unversalty) Here s how the adversary can force a collson: frst we choose a random hash functon, whch s ether,
4 Handout 14: Problem Set 3 Solutons or \ The adversary gves us ; to hash, and learns the value o '; If o '; 1DT, the adversary knows that \, and can force a collson by makng us hash If : -;, then and the adversary forces a collson by makng us hash < (d) Wth a 2-unversal hash famly, the adversary cannot force a collson wth probablty better than T0 V Essentally, knowng : -; gves the adversary no nformaton about : =< for any other key < We can prove ths formally usng condtonal probabltes Suppose we choose a random hash functon, then the adversary forces us to hash some key ; and learns the value : -; Then the adversary gves us a key <, hopng to cause a collson By defnton of 2-unversal, we have for any ; < wth ; W < : : =< : -; : -;! M o -< o '; : -; M T0 V \ T0 )V : -; M T0 V R Therefore, no matter whch ; the adversary frst chooses, and whch : -; value t learns, the probablty of any partcular < colldng wth ; s only T0 V Problem 3-5 Weghted 3-medan (a) Frst, we compute by addng all the weghts Then we sort the ; elements Fnally, for T R6R6R I, we mantan the runnng sum > 5p5 by addng the weght of the prevous element ; wth each teraton As soon as (whch must happen before, ), we output ; as the weghted 3-medan element (b) Assume the elements are gven to us n an array, and ther weghts n an array Every tme we move elements wthn, we wll make dentcal moves n, to mantan the correspondence between elements and ther weghts We use a dvde-and-conquer algorthm to essentally perform a bnary search for the proper ; The algorthm mantans a wndow M, wth the nvarant that all elements wth ranks through are contaned n R6R M, and the weghted 3-medan ; s somewhere n!r R M It also mantans two sums and, where s the total weght of all elements whose ranks are less than (e, > 5p5 ), and s the total weght of all elements whose ranks are greater than At the start of the algorthm, M T I M and (wth these values, the nvarants are clearly true) The algorthm then proceeds as follows: When searchng wndow M wth and, we frst fnd the medan ; of R6R M and partton R6R M around ; We then compute to be the total weght of all elements less than ; n the wndow!r6r M, and to be the total weght of all elements greater than ; n the wndow There are three cases: If!" and #, then ; return t If,%, then we let '&!R6R M (that s, all elements n the wndow smaller than ; s the weghted 3-medan and we and loop on the lower half of )
Handout 14: Problem Set 3 Solutons 5 If,, then we let &!R6R M (all elements n the wndow greater than ; and loop on the upper half of ) Gven the nvarant, the correctness of the frst case s apparent (ths case also acts as the base case, f the wndow shrnks down to one element) It s also clear that the fnal two cases preserve the nvarant We now analyze the runnng tme of the algorthm: each teraton takes tme lnear n the sze of the wndow to fnd the medan, partton, and compute and Each teraton halves the sze of the wndow, therefore a recurrence for the runnng tme s, whch solves to Ths completes the proof