Divide-and-conquer algorithms 1

Transcription

1 * 1 Multipliation Divide-and-onquer algorithms 1 The mathematiian Gauss one notied that although the produt of two omplex numbers seems to! involve four real-number multipliations it an in fat be done with just three: and sine " # $ %&( This speeds up the omputation but only by a onstant fator whih to our big-) way of thinking is negligible Can something more substantial be salvaged from Gauss observation? The seemingly modest improvement in omputation time beomes very signifiant if this same trik is applied reursively Let s move away from omplex numbers and see how this helps with regular multipliation Suppose * and are two -bit integers As a first step towards multiplying them split eah of them into their left and right halves whih are -0 bits long: For instane if * 1 3 * 1 * 1 1 *21 *43 <;$=>;8; then <;$= *? 687:*21 607: 1 <;8; and * 3 87: * 1 3 *43 and 3 The produt is We will ompute *2 via the expression on the right-hand side The additions take linear time as do the multipliations by powers of two (whih are left-shifts) The signifiant operations are the four -0 -bit multipliations * 1 1 * 1 3 * 3 1 * 3 3 ; these we an handle by four reursive alls If A denotes the time taken to multiply -bit numbers by this method then A an be written as a reurrene relation BDC A A -0 ) We will soon examine general strategies for solving suh equations In the meantime this partiular one works out to ) whih is disappointing beause it is no better than the traditional grade-shool multipliation tehnique So we would like to speed up our reursive algorithm somehow and now Gauss trik omes to mind Although the expression for *2 seems to demand four -0 -bit multipliations H as before just& three will do: * 1 1 *21 *43 E1 3 sine *F1(3 *43GE1 *21 *F3 E1 3 *F1(E1 *F3I3 running time is BKJ A A -0 ) * 3 1 L 1 Copyright 2004 S Dasgupta CH Papadimitriou and V * 3 3 * 3 3 and The improved 1

2 Q J d d Figure 11 A divide-and-onquer algorithm for integer multipliation funtion multiply(* ) * Input: Two -bit numbers and Output: Their produt M; if : return *? * 1 * 3 leftmost rightmost NO-06P bits of * 1 3 leftmost rightmost NO-06P bits of QR multiply * 1 1 multiply QTS * 3 3 multiply Q RV * W 1 * return Q S Q R Q Q S 87: At first glane this seems only a onstant fator better than the previous attempt But the improvement ours at every level ofryx the ZY[ reursion and this ompounding effet leads to a dramatially lower time bound of ) The algorithmi strategy we have been using (see Figure 11) is alled divide-and-onquer: it takles a problem by seleting subproblems reursively solving them and then gluing together these partial answers All the work is done in the seleting and gluing and in the base ase of the reursion The running time of our algorithm follows from its pattern of reursive alls whih form a tree struture as in Figure 12 Let s try to understand the shape of \^]8_ this tree `^a At eah suessive level of reursion the subproblems get halved in size At the level the subproblems \b]8_ get down to size one and so the reursion ends Therefore the height of the tree is The branhing fator is three eah problem reursively produes three smaller ones with the result that at depth Jd in the tree there are subproblems eah of size -0 For eah subproblem a linear amount of work is done in seleting further subproblems and gluing together answers Therefore the total time spent at depth in the tree is J d )<e dff )hgji lk Gm n= <\b]8_ At the J0oqpYrs very top level when this works oqpyr sout S to ) At the bottom when it is ) whih an be rewritten as ) (hek!) oqpyr s S Between these two endpoints the work done inreases geometrially from ) to ) J by a fator of -0 per level The sum of any inreasing geometri series is within a onstant fator simply the last term of oqpyr the s S series: suh is the RYX rapidity ZY[ of the inrease Therefore the overall running time is ) whih is about ) In the absene of Gauss trik the reursion C oqpyr s tree would have the same height but with a branhing fator of four There would be G leaves and therefore the running time would be at least this muh In divide-and-onquer algorithms the number of subproblems translates into the branhing fator of the reursion tree; small hanges in this oeffiient an 2

3 A P ƒ e y f A P Figure 12 The first few levels of reursion of divide-and-onquer integer multipliation size t size ttu6v size tuxw have a big impat on running time One pratial note: it generally does not make sense to reurse all the way down to one bit For most proessors 16 or 32-bit multipliation is a single operation so by the time the numbers get into this range they should be handed over to the built-in proedure 2 Reurrene relations Claim If A! A large number of divide-and-onquer algorithms onform to a generi pattern: they takle a problem of size by reursively solving subproblems of size - and then ombining these answers in ) Iy 2zx {}= ~ x time for some Their running time is therefore A NO- ) y We now find a losed-form solution to this general reurrene so that we no longer have to solve it expliitly in eah new instane NO- A ) Iy for some positive onstants?zx then q ) y \b]8_ ) Gy oqpyrzˆ6 ) } if W y if {} y if y This single theorem tells us the running times of most of the divide-and-onquer proedures we are likely to use To prove the laim let s start by assuming for onveniene that is a power of ; this will not influene the final bound and will allow us to ignore the rounding effet in NO- P The size of the subproblems dereases by a fator of with eah level of reursion and therefore reahes the \b]8_š base ase `^a after levels This is the height of the reursion tree Its branhing fator is so the d d level of the tree is made up of subproblems eah of size - The total work done at this level is d )<e dff y ) y d (the leaves) these numbers form a geometri series y To determine the sum of the series there are three ases we need to onsider If the ratio is less than one then the series is dereasing in whih ase the first term is dominant If the ratio is more than one the series is inreasing and the last term As goes = \b]8_š from (the root) to with ratio - ) Gy 3

4 - Q y Œ ) y oqpyrzˆ \^]8_ ) oqpyrzˆ6 x is dominant Finally it ould be that the ratio is exatly one in whih ase all ) terms of the series are equal to ) %y These ases translate diretly into the three ontingenies of the theorem 3 Matrix multipliation The produt of two In general Œ Œ R d Œ d matries and is a third d$ matrix is not the same as Œ ; matrix multipliation is not ommutative The formula above implies an ) S ŽŒ with `^a entry algorithm for matrix multipliation: there are entries to be omputed and eah takes linear time For quite a while this was widely believed to be the best running time possible and it was even proved that no algorithm whih used just additions and multipliations ould do better It was therefore a soure of great exitement when in 16 Strassen announed a signifiantly more effiient algorithm based upon divide-and-onquer Matrix multipliation is partiularly easy to break into subproblems beause it an be performed blokwise To see what this means arve into four -0 š ž Ÿ š Ÿ -0 bloks and also Œ : Then their produt an be expressed in terms of these bloks and is exatly as if the bloks were single elements ŽŒ š ž Ÿ š Ÿ š ž ž Ÿ We now have a divide-and-onquer strategy: to ompute the size- produt ŽŒ reursively ompute eight size--0 produts ž ž and then do a few ) - time additions The total running time is desribed by the reurrene relation whih omes out to ) S A Wª A -0 the same as for the default algorithm However an improvement in the time bound is possible and as with integer multipliation it relies upon algebrai triks It turns out that Œ an be omputed from just seven subproblems ) Q~R QS ž QI«ž QZ ž Q ž QT ŽŒ š QTZ Q«Q QTS Q«Q Q~R Q QR QZ QTS Q Ÿ 4

5 ±»» This translates into a running time of A A -0 ) oqpyr s whih by the result of the previous setion is ) ) G X R 4 Mergesort The problem of sorting a list of numbers lends itself immediately to a divide-and-onquer strategy: split the list into two halves reursively sort eah half and then merge the two sorted sublists 4%$$ funtion mergesort( 4 ) F%$$ Input: An array of numbers 4 Output: A sorted version of this array {; if return merge(mergesort( 4%$$6-06µ$ ) mergesort( 4± -06µ else return W; $$ 4 )) The orretness of this algorithm is self-evident %$$ as long%$$ as a orret merge subroutine is speified If we are given two sorted %$$ arrays } * and how do we effiiently merge them into a single sorted array ±¹? Well the very first element of is either * or whihever is smaller The rest of an then be onstruted reursively %$$ %$$ funtion = merge(* %$$ ) if : return G= %$$ if : return * if * : $$ return * >º merge * else: %$$ return º merge * %$$ ± $$ This kind of tail reursion an be unraveled into a purely iterative algorithm as shown in Figure 41 It performs a onstant number of operations for eah element of the ombined array and therefore takes linear time ) in all The overall running time of mergesort is then whih works out to ) Medians \b]8_ A xa -0 The median of a list of numbers is its 0th perentile: half the numbers are bigger than it and half are smaller In other words it ± C»¼;0¼;$=zJ0= is the middle element when the numbers are arranged in order For instane the median of is If the list has even length there are two hoies for the middle element and the median is defined to be their average The purpose of the median is to summarize a set of numbers by a single typial value The mean or average is also ommonly used for this but it has the tremendous disadvantage that )

6 * ƒ Ä Figure 41 The merge proedure %$$ %$$ funtion merge(* ) Input: Two sorted arrays * Output: A sorted array ½?¾ ½? <; for ½FÀ <; to if ½2 {! ±½2À ±½?¾ ½ ¾ ½ ¾ ; else return ±½2À ±½? ½? ½2 ; %$$ %$$ %$$ and ontaining the ombined elements of * Positions in the two input arrays : or * ±½?¾ ±½? : it an be ompletely thrown off by a single large or small number For instane the mean of a list of a hundred 1 s is (rightly) 1 as is the median However if just one of these numbers gets aidentally hanged to the mean shoots up above 100 while the median is unaffeted Computing \b]8_ the median of numbers is easy: just sort them The only problem is that this takes ) time whereas we would ideally like something linear We have reason to be hopeful beause sorting is doing far more work than we really need we just want the middle element and don t are about the relative ordering of the rest of them Often a problem beomes learer and easier to solve when we onsider a more general version of it In this ase the generalization we will onsider is seletion SELECTION Input: A list of numbers Á ; an integer `^a Output: The smallest element of Á Â; For instane if the minimum of Á is sought whereas if setting appropriately all the perentiles of Á an be found  Á it is the maximum By Here s a divide-and-onquer approah to seletion For any number Ä imagine splitting list Á into three ategories: elements smaller than Ä those equal to Ä (there might be dupliates) and those greater than Ä Call these Á Å(Æ Á Æ and ÁTÇ(Æ respetively By heking against the sizes of these subarrays we an quikly determine whih of them holds the desired element: selet Á ~ The three sublists Á Å(Æ q selet ÁTÅ(Æ Á Æ selet Á Ç(Æ Ê Á Å(Æ 0Ë Á Æ q È if > ÁTÅ(Æ if ÁÅ(Æ {È if Á Å(Æ È $É ÁÅ(Æ Á ¼ Æ Á Ì Æ Á Ç(Æ an be omputed from Á in linear time and in fat this omputation an be done in plae that is without alloating new memory for them using three pointers (an you figure out this neat trik?) One they are obtained exatly one of the three senarios an hold and so the original array Á effetively shrinks to one of size at most ÍÏÎ6ÐFÑ Ìx ÓÒ ÁÅ(Æ ÁÇ(Æ 6

7 ; A R C Á C The hoie of Ä is ruial It should be8 n piked8 quikly and it should shrink the array substantially the ideal situation being Á~Å(Æ ÁÇ(Æ If we ould always guarantee this situation we would get a running time of A A -0 whih is linear as desired But this requires piking Ä to be the median whih is what we are trying to do in the first plae! Instead our method of hoosing Ä is muh simpler: we just pik it randomly from Á The running time of our algorithm depends on the random hoies of Ä It is possible that due to persistent bad luk we keep piking Ä to be the smallest element of the array (or the largest element) and thereby shrink the array by only one element eah time This worst ase senario takes ) steps but is extremely unlikely to our Equally unlikely is the best possible ase in whih eah randomly hosen Ä just happens to be the median so that as above the running time is ) Where in this spetrum from ) to ) does the average running time lie? Fortunately it lies very lose to the best-ase time To distinguish» `^a 0» between `^a luky and unluky hoies of Ä we will all Ä good if it lies any- to perentile of the array that it is hosen from The following property where in follows from the definition of perentile Property When Ä is hosen randomly from an array it has a probability ; -0 of being good of its size (why?) This is very promising but how many times on average do we need to pik Ä before we get a good one? There is an equivalent question whih might be more familiar: on average how many times must we toss a fair oin before getting heads? Let be the expeted number of tosses We ertainly need at least one toss and if it s heads we re done Otherwise (and this ours with probability -0 ) we need to repeat In other words <; so J Therefore after every two reursive alls on average the array will shrink to - of its be the expeted running time Then A good Ä auses the array to shrink to at most J - size Let A whih means A in an average of ) A J - I CI R ) ) 6 is linear In short on any input our algorithm returns the orret answer steps 7