rus i O(log log Λ = log log ) time usig a optimal umber of processors. Chaudhuri, Hagerup, ad Rama's algorithm [2] rus i O(log= loglog) time o the CRC

Transcription

1 Selectio Algorithms for Parallel Disk Systems 1 Saguthevar Rajasekara Dept. of CISE, Uiv. of Florida, Gaiesville, FL Abstract. With the wideig gap betwee processor speeds ad disk access speeds, the I/O bottleeck has become critical. Parallel Disk Systems (PDS) have bee itroduced to alleviate this bottleeck. I this paper we preset determiistic ad radomized selectio algorithms for parallel disk systems. The algorithms to be preseted, i additio to beig asymptotically optimal, have small uderlyig costats i their time bouds ad hece have the potetial of beig practical. 1 Itroductio Give a sequece of keys ad a iteger i, 1» i», the problem of selectio is to idetify the ith smallest of the keys. This importat compariso problem has bee extesively studied. Numerous asymptotically optimal sequetial algorithms have bee discovered. Asymptotically optimal algorithms have bee preseted for varyig parallel models as well. Floyd ad Rivest's sequetial algorithm [9], i additio to beig asymptotically optimal, is simpler tha the optimal determiistic algorithm of Blum et al. [7]. Reischuk's radomized algorithm for the parallel compariso tree model takes O(1) time usig processors ad hece is clearly optimal. Meggido's algorithm fids the maximum i O(1) time usig parallel compariso tree processors. Both of these algorithms are based o the idea of [9]. A costat time radomized algorithm for maximum selectio for the CRCW PRAM has bee give by Rajasekara ad Se [18]. It is easy to obtai a radomized logarithmic time algorithm for selectio usig CRCW PRAM processors (see e.g., [10]). A determiistic logarithmic time asymptotically optimal algorithm is also kow [11]. Cole's algorithm for the CRCW PRAM 1 This research was supported i part by ansfaward CCR ad a EPA Grat R log 1

2 rus i O(log log Λ = log log ) time usig a optimal umber of processors. Chaudhuri, Hagerup, ad Rama's algorithm [2] rus i O(log= loglog) time o the CRCW PRAM usig log log = log CRCW PRAM processors. Optimal algorithms have also bee desiged for models such as the mesh, the hypercube, meshes with buses, etc. For a survey of parallel selectio algorithms, the reader is referred to [16]. We preset two selectio algorithms for the PDS. The PDS have bee proposed with the wideig gap betwee processor speeds ad disk access speeds i mid. I essece a PDS ca be thought of as a computer with may disks. It is assumed that i oe I/O operatio we ca fetch a block of data from each of the disks i parallel. The first algorithm we preset for the PDS is radomized ad the secod algorithm is determiistic. The umber of parallel I/O read operatios eeded for either is O N, where N is the umber of iput keys, D is the umber of disks, ad B is the block size. Thus the algorithms are asymptotically optimal. Due to the small uderlyig costats, the algorithms have the potetial of beig practical as well. May problems such as sortig, graph problems, etc. have bee studied o the PDS. Ay sortig algorithm ca clearly be used to perform selectio. The kow lower boud o the umber of passes through the data for sortig o the PDS is Ω log(n=b) log(m=b), where N is the umber of iput keys, M is the iteral memory size, ad B is the block size. I practice oe ca assume that N is a polyomial i M ad M is a polyomial i B ad hece this lower boud simply meas a costat umber of passes through the data. The PDS model we use is the oe suggested by Vitter ad Shriver i their pioeerig paper [20]. Several asymptotically optimal sortig algorithms have bee proposed for the PDS. All these algorithms, though theoretically importat, have rather large costats i their time bouds. Recetly, Rajasekara has proposed a sortig algorithm called the (l; m)-merge sort (LMM) [15]. A implemetatio of this algorithm o the PDS makes» 2 log(n=m) o more tha log(mif(m=b); p Mg) +1 passes through the data. It has bee show that whe D is large, LMM performs better tha the disk-striped merge sort (DSM) algorithm that is used i practice. A iterestig questio is if we ca perform selectio o the PDS i time better tha sortig time. This paper aswers this questio i the affirmative. I Sectio 2 2

3 we itroduce the PDS model. I Sectios 3 ad 4 we preset our radomized ad determiistic algorithms, respectively. Sectio 5 cocludes the paper. 2 Parallel Disk Systems Several models for parallel disks have bee ivestigated i the literature. The model employed i this paper is the oe itroduced i oe of the pioeerig papers of Vitter ad Shriver [20]. I this model there are D distict ad idepedet disk drives. The disks ca simultaeously trasmit a block of data. A block cosists of B records. If M is the iteral memory size, the oe usually requires that M 2. For the algorithms preseted i this paper, achoice of M =3 suffices. Of this, amout of memory is used to prefetch data i order to overlap computatio ad data access. From hereo, we use M to deote. The problem of disk sortig was first studied by Aggarwal ad Vitter i their foudatioal paper [5]. I the model they cosidered, each I/O operatio results i the trasfer of D blocks each block havig B records. A more realistic model was evisioed i [20]. Several asymptotically optimal algorithms have bee give for sortig o this model. Nodie ad Vitter's optimal algorithm [13] ivolves solvig certai matchig problems. Aggarwal ad Plaxto's optimal algorithm [4] is based o the Sharesort algorithm of Cypher ad Plaxto. Vitter ad Shriver gave a optimal radomized algorithm for disk sortig [20]. All these results are highly otrivial ad theoretically iterestig. However, the uderlyig costats i their time bouds are high. I practice the simple disk-striped mergesort (DSM) is used [6], eve though it is ot asymptotically optimal. DSM has the advatages of simplicity ad a small costat. Data accesses made by DSM is such thatatay I/O operatio, the same portios of the D disks are accessed. This has the effect of havig a sigle disk which ca trasfer records i a sigle I/O operatio. A M -way mergesort is employed by this algorithm. To start with, iitial rus are formed i oe pass through the data. At the ed the M disk has N=M rus each of legth M. Next, rus are merged at a time. Blocks of ay ru are uiformly striped across the disks so that i future they ca be accessed i parallel utilizig the full badwidth. Each phase of mergig ivolves oe pass through 3

4 the data. There are DSM is log(n=m) log(m=) by the algorithm is N log(n=m) log(m=) phases ad hece the total umber of passes made by. I other words, the total umber of I/O read operatios performed. The costat here is just log(n=m) log(m=) The kow lower boud o the umber of passes for parallel disk sortig is Ω log(n=m) log(m=b) If oe assumes that N is a polyomial i M ad that B is small (which are readily satisfied i practice), the lower boud simply yields Ω(1) passes. All the above metioed optimal algorithms make oly O(1) passes. So, the challege i the desig of parallel disk sortig algorithms is i reducig this costat. If M =2, the umber of passes made by DSM is 1 + log(n=m), which ideed ca be very high. Recetly, several works have bee doe that deal with the practical aspects. Pai, Schaffer, ad Varma [14] aalyzed the average case performace of a simple mergig algorithm, employig a approximate model of average case iputs. Barve, Grove, ad Vitter [6] have preseted a simple radomized algorithm (SRM) ad aalyzed its performace. The aalysis ivolves the solutio of certai occupacy problems. The expected umber Read SRM of I/O read operatios made by their algorithm is such that Read SRM» N + N log(n=m) log kd ψ log D log log log D 1+ k log log D log log D + 1+logk log log D + O(1) The algorithm merges R = kd rus at a time, for some iteger k. Whe R =Ω(Dlog D), the expected performace of their algorithm is optimal. However, i this case, the iteral memory eeded is Ω(BD log D). They have also compared SRM with DSM through simulatios ad show that SRM performs better tha DSM. Recetly, Rajasekara [15] has preseted a algorithm (called (l; m)-merge sort (LMM)) which is asymptotically optimal uder the assumptios that N is a polyomial i M ad B is small. The algorithm is as simple as DSM. LMM makes less umber of passes through the data tha DSM whe D is large. The selectio algorithms to be preseted are as simple as the DSM. Data accesses are such that i ay I/O operatio, the same portios of the disks are accessed. Alive keys (i.e., keys that have ot yet bee elimiated) i ay stage are uiformly striped across the disks.! (1). 4

5 3 A Radomized Selectio Algorithm I this sectio we preset a radomized selectio algorithm for the PDS. The umber of I/O read operatios made by the algorithm is O N with high probability. Almost all the selectio algorithms proposed i the literature, be they determiistic or radomized, sequetial or parallel, are based o samplig. For example, Floyd ad Rivest's radomized algorithm [9] is based o radom samplig. The algorithm cosists of the followig steps. 1) Select a radom sample S of s elemets from the iput set X; 2) Sort the sample S ad fid two elemets `1 ad `2 from S such that the ith smallest elemet of X will have a value i betwee `1 ad `2 ad also the umber of keys from X that have a value i betwee `1 ad `2 is 'small'; 3) Elimiate keys of X that do ot have avalue i the rage [`1;`2]; ad4)perform a appropriate selectio i the set of remaiig keys. Samplig techiques have bee repeatedly used to develop selectio algorithms for a variety of parallel models of computig. Though these algorithms employ samplig as a commo theme, they have model-depedet iovatios ad employ additioal techiques. Our radomized algorithm is also based o the above theme. A Samplig Lemma. Let Y be a sequece of umbers from a liear order ad let S = fk 1 ;k 2 ;:::;k s g be a radom sample from Y. Also let k 0 1 ;k0 2 ;:::;k0 s be the sorted order of this sample. If r i is the rak of k 0 i i Y, the followig lemma provides a high probability cofidece iterval for r i. (The rak of ay elemet k i Y is oe plus the umber of elemets <ki Y.) Lemma 3.1 For every ff>0, Prob jr i i s j > p 3ff p s p log < ff. A proofoftheabove lemma ca be foud i [17]. We say a radomized algorithm uses e O(g()) amout of ay resource (like time, space, etc.) if there exists a costat c such that the amout of resource used is o more tha cffg() with probability 1 ff o ay iput of legth ad for ay fixed ff (see e.g., [10]). Similar defiitios apply to eo(g()) ad other such `asymptotic' fuctios. 5

6 By high probability we mea a probability of 1 ff for ay fixedff 1( beig the iput size of the problem at had). Let B(; p) deote a biomial radom variable with parameters ad p. Oe of the most frequetly used facts i aalyzig radomized algorithms is Cheroff bouds. These bouds provide close approximatios to the probabilities i the tail eds of a biomial distributio. Let X stad for the umber of heads i idepedet flips of a coi, the probability of a head i a sigle flip beig p. X is also kow to have a biomial distributio B(; p). The followig three facts (kow as Cheroff bouds) will be used i the paper (ad were discovered by Cheroff [3] ad Aglui & Valiat [1]): for ay 0 <ffl<1, ad m>p. Prob [X m]» p m m e m p ; Prob [X (1 + ffl)p]» exp( ffl 2 p=3); ad Prob [X» (1 ffl)p]» exp( ffl 2 p=2); Now we are ready to describe our algorithm. Let N = M c. I practice c ca be assumed to be a costat. I today's PC market, M is of the order of megabytes ad the disk space is of the order of gigabytes. So it is perhaps safe to assume that c is o more tha 3. To begi with each key is alive ad = N. Algorithm RSelect Iput is a sequece of N keys ad the output is the ith smallest key of the sequece. = N. repeat Step 1. Let be the umber of alive keys. If» M the goto Step 6. Each alivekey is icluded i the sample S with probability M 2. The expected umber of sample keys is M. We ca show that the actual umber of keys 2 i S is M + eo(m). Cout the umber s of sample keys. 2 Step 2. Sort S ad pick two keys `1 ad `2 from S whose raks i S are i s ffi ad i s + ffi, respectively, for ffi p 3ffs log, for ay fixed ff 1. 6

7 forever Step 3. Compute the umber 1 of alive keys that are less tha `1 ad the umber 2 of alive keys that have avalue i the rage [`1;`2]. Step 4. If i< 1,ori> 1 + 2, or 2 > M 0:4, goto Step 1. Step 5. Ay alive key whose value lies outside the rage [`1;`2] dies. Set i = i 1 ad = 2. Step 6. Sort the alive keys ad output the ith smallest key. Aalysis. I Step 1, the umber of sample keys has a biomial distributio B(; M 2 ). A applicatio of Cheroff bouds shows that s = M + eo(m). Also Step 1 takes O( 2 ) I/O read operatios. Sice S is kept i the iteral memory, Step 2doesot ivolve ay I/O operatios. I Step 3, a applicatio of Lemma 3.1 implies that the umber 2 of keys survivig Step 5 is O e ps p log = O e M. As a result it follows that, the umber of iteratios 0:4 of the repeat loop is O(c). e I Step 4, we ca show that the probability ofexecutig the goto statemet is very small. Step 5 ivolves O( ) I/O read operatios. Note that the umber of survivig keys from oe iteratio of the repeat loop to the ext decreases by a factor of M 0:4 with high probability. Thus it follows that the total umber of I/O read operatios made by the etire algorithm is e O N. Thus we get the followig Theorem. Theorem 3.1 Selectio from out of N keys ca be performed o the PDS usig O e N I/O read operatios provided that N = M c for some costat c. 2 4 A Determiistic Selectio Algorithm I this sectio we preset our determiistic selectio algorithm for the PDS. The umber of I/O read operatios performed by the algorithm is O N. The uderlyig costat is small ad hece this algorithm has the potetial of beig practical. 7

8 Samplig has also domiated as a techique useful i the desig of determiistic selectio algorithms. For example, Blum et al.'s algorithm [7] partitios the iput such that there are 5 elemets i each part, fids the media of each part, fids the media M of these medias, splits the iput ito two groups (those that are» M ad those that are greater tha M), idetifies the group that has the key to be selected, ad fially performs a selectio i the group that cotais the key to be selected. The medias of the 5-elemet parts ca be thought of as a sample of the iput keys ad hece the media M of these medias ca be expected to be a approximate media of the iput keys. Cosider a collectio X of keys ad cosider the problem of fidig the ith smallest key of X. We use the followig strategy to idetify two elemets of X such that they will bracket the ith smallest elemet of X ad also the umber of keys of X that have avalue i betwee these two keys is ot large. Partitio the collectio X = R 0 such that there are M keys i each part. Sort each part. From each part retai those keys that are at a distace of p M from each other. That is, keep the keys whose raks are p M;2 p M;3 p M;:::. Thus the umber of keys i the retaied set R 1 is p M. Now group the elemets of R 1 such that there are M elemets i each part, sort each part, ad retai oly every p Mth elemet i each part. Call the retaied set R 2. Proceed to obtai R i 's i a similar fashio (for i 3) util we reach a stage whe jr j j»m. If = M c, the clearly, j =2c 2. This process ca be represeted by a tree of degree p M. Each leaf has M iput elemets. All the elemets i the leaves costitute R 0. The root has R j. Let the root be i level j. Let its childre be i level j 1, ad so o. The leaves are at level 0. There are p M childre to the root. Each suchchild has M elemets. p M elemets are passed o from each child to its paret. I geeral each ode i the tree has M elemets. p M elemets from out of these will go to its paret. Each ode p M childre. PickfromR j two elemets `1 ad `2 whose raks are i jr j j ffi ad i jr jj +ffi, respectively. Without loss of geerality assume that jr j j = M. The, jr i j = M( p M) j i. Cosider a elemet x whose rak i R j is q. The the rak of q i R j 1 will be i the rage [q p M;q p M + p M( p M 1)]. This rak is also i the iterval [q p M;q p M + M]. I.e., there is a ucertaity ofm i the rak of x i R j 1. Each child of the root cotributes 8

9 p M 1 ß p M to this ucertaity. I geeral the ucertaity i the rak of x i Ri is cotributed to by each ode i level i. Note that there are M (j i)=2 odes at level i. Each such ode cotributes p M 1 to the ucertaity. Thus, if U(i) is the maximum possible rak of x i R i, the U(i) satisfies: p U(i)» M U(i +1)+M (j i+1)=2 which solves to U(i)» M (j i)=2 U(j) +(j i)m (j i+1)=2. Whe i = 0, we get U(0)» M j=2 U(j)+jM (j+1)=2. I other words, U(0)» qm c 1 +(2c 2) p M where c = log log M. As a result, if we pick ffi to be (2c 2+ffl) p M, for ay ffl>0, the rak of `1 i R 0 will be i the iterval " i (2c 2+ffl) p ;i ffl # p M M Also, the rak of `2 i R 0 willlieitheiterval " i +(2c 2+ffl) p ;i+(4c 4+ffl) # p M M Puttig together, we realize that the ith smallest elemet ofr 0 will have a value i the iterval [`1;`2] ad also the umber of keys of R 0 that have a value i the iterval [`1;`2] isomore tha (6c 6+2ffl) p M. Call the above process of startig from R 0 ad obtaiig R 1 ;R 2 ;:::;R j a stage of samplig. Note also that the umber of I/O read operatios eeded for a stage is O where jr 0 j =. Oe could see that a stage of samplig correspods to oe iteratio of the repeat loop of RSelect. Similar samplig techiques have bee employed by Muro ad Paterso [12]. Now we preset a detailed descriptio of our algorithm. Let K = k 1 ;k 2 ;:::;k N be the iput. Say we are iterested i fidig the ith smallest key. To begi with each key is alive ad = N. 9

10 Algorithm DSelect Iput is a sequece of N keys ad the output is the ith smallest key of the sequece. = N. repeat forever Step 1. If» M goto Step 3. Perform a stage of samplig i the collectio of alive keys. As a result, obtai two keys `1 ad `2 that will bracket the key to be selected. Step 2. Sca through the alive keys ad kill the keys that have a value outside the rage [`1;`2]. Cout theumber of keys survivig this step. Let this umber be. Step 3. Sort the alive keys ad output the ith smallest elemet. Aalysis. Clearly, theumberofalivekeys reduces by a factor of Ω p M log M log from oe iteratio to the ext of the repeat loop. Therefore, if N = M c, the umber of iteratios of the repeat loop is O(c). Also, the umber of I/O read operatios eeded i ay iteratio of the repeat loop is O as has bee discussed before. Sice the umber of alive keys decreases by a factor of Ω p M log M log from oe iteratio to the ext, the total umber of I/O read operatios is oly O N. We arrive atthefollowig Theorem. Theorem 4.1 We ca perform selectio from out of N give keys o the PDS usig O N I/O read operatios whe N = M c for some costat c. 2 5 Coclusios We have preseted two selectio algorithms for the PDS. Both are asymptotically optimal. The uderlyig costats i the time bouds are small ad hece the algorithms have the potetial of beig practical. 10

11 Refereces [1] D. Aglui ad L. G. Valiat, Fast Probabilistic Algorithms for Hamiltoia Paths ad Matchigs, Joural of Computer ad System Scieces 18, 1979, pp [2] S. Chaudhari, T. Hagerup, ad R. Rama, Approximate ad Exact Determiistic Parallel Selectio, Proc. 18th Aual Symposium o Mathematical Foudatios of Computer Sciece, [3] H. Cheroff, A Measure of Asymptotic Efficiecy for Tests of a Hypothesis Based o the Sum of Observatios, Aals of Mathematical Statistics 23, 1952, pp [4] A. Aggarwal ad C. G. Plaxto, Optimal Parallel Sortig i Multi-Level Storage, Proc. Fifth Aual ACM Symposium o Discrete Algorithms, 1994, pp [5] A. Aggarwal ad J. S. Vitter, The Iput/Output Complexity of Sortig ad Related Problems, Commuicatios of the ACM, 1988, 31(9): [6] R. Barve, E. F. Grove, ad J. S. Vitter, Simple Radomized Mergesort o Parallel Disks, Techical Report CS , Departmet of Computer Sciece, Duke Uiversity, October [7] M. Blum, R. W. Floyd, V. Pratt, R. L. Rivest, R. E. Tarja, Time Bouds for Selectio, Joural of Computer ad System Scieces 7, 1973, pp [8] R. Cole, A Optimally Efficiet Selectio Algorithm, Iformatio Processig Letters 26, 1988, pp [9] R. W. Floyd ad R. L. Rivest, Expected Time Bouds for Selectio, Commuicatios of the ACM 18(3), 1975, pp [10] E. Horowitz, S. Sahi, ad S. Rajasekara, Computer Algorithms, W. H. Freema Press, [11] J. J'a Já, Parallel Algorithms: Desig ad Aalysis, Addiso-Wesley Publishers,

12 [12] J. I. Muro ad M. S. Paterso, Selectio ad Sortig with Limited Storage, Theoretical Computer Sciece 12, 1980, pp [13] M. H. Nodie, J. S. Vitter, Large Scale Sortig i Parallel Memories, Proc. Third Aual ACM Symposium o Parallel Algorithms ad Architectures, 1990, pp [14] V. S. Pai, A. A. Schaffer, ad P. J. Varma, Markov Aalysis of Multiple-Disk Prefetchig Strategies for Exteral Mergig, Theoretical Computer Sciece, 1994, 128(2): [15] S. Rajasekara, A Framework For Simple Sortig Algorithms O Parallel Disk Systems, Proc. 10th Aual ACM Symposium o Parallel Algorithms ad Architectures, [16] S. Rajasekara, Sortig ad Selectio o Itercoectio Networks, DIMACS Series i Discrete Mathematics ad Theoretical Computer Sciece 21, 1995, pp [17] S. Rajasekara ad J.H. Reif, Derivatio of Radomized Sortig ad Selectio Algorithms, i Parallel Algorithm Derivatio ad Program Trasformatio, Edited by R. Paige, J.H. Reif, ad R. Wachter, Kluwer Academic Publishers, 1993, pp [18] S. Rajasekara ad S. Se, Radom Samplig Techiques ad Parallel Algorithms Desig, i Sythesis of Parallel Algorithms, edited by J. H. Reif, Morga-Kaufma Publishers, 1993, pp [19] R. Reischuk, Probabilistic Parallel Algorithms for Sortig ad Selectio, SIAM Joural of Computig, 1985, 14(2): [20] J. S. Vitter ad E. A. M. Shriver, Algorithms for Parallel Memory I: Two-Level Memories, Algorithmica, 1994, 12(2-3):