A NOTE ON COARSE GRAINED PARALLEL INTEGER SORTING

Size: px

Start display at page:

Download "A NOTE ON COARSE GRAINED PARALLEL INTEGER SORTING"

Anne Marshall
5 years ago
Views:

1 Chater 26 A NOTE ON COARSE GRAINED PARALLEL INTEGER SORTING A. Cha ad F. Dehe School of Comuter Sciece Carleto Uiversity Ottawa, Caada K1S 5B6 æ {acha,dehe}@scs.carleto.ca Abstract Keywords: We observe that for = ç, which is usually the case i ractice, there exists a very simle, determiistic, otimal coarse graied arallel iteger sortig algorithm with 24 commuicatio rouds (6 -relatios ad 18 -relatios), Oè=è memory er rocessor ad Oè=è local comutatio. Exerimetal data idicates that the algorithm has very good erformace i ractice. BSP, coarse graied arallel algorithms, iteger sortig INTRODUCTION Goodrich [4] reseted a determiistic sortig algorithm for the BSP [5] ad closely related CGM model [2, 3]. Give Oèè data items stored o a rocessor BSP/CGM, Oè=è data items er rocessor, these items ca be log sorted i Oè è commuicatio rouds (h-relatios), for h = æè=è, logèh+1è with Oè log è local comutatio, usig Oè=è memory er rocessor. For = ç æ log, æ é 0, Oè è = Oè1è. That is, for this case, the algorithm logèh+1è requires Oè1è commuicatio rouds. We are iterested i the roblem of sortig Oèè itegers i the rage 1;::: ; c, for fixed costat c, stored o a rocessor BSP/CGM, = data items er rocessor. The sort algorithm i [4] is based o Cole s merge sort [1]. The Oè log è local comutatio i [4] is due to a costat umber of local æ Research artially fuded by the Natural Scieces ad Egieerig Research Coucil of Caada. 261

2 262 HIGH PERFORMANCE COMPUTING SYMPOSIUM 1999 sorts. Hece, by alyig radix sort for the iteger case, it is easy to obtai Oè=è local comutatio without icreasig the umber of commuicatio rouds. I this aer we observe that for = ç, which is usually the case i ractice, there exists a very simle, determiistic, otimal BSP/CGM iteger sortig algorithm with 24 commuicatio rouds (6 -relatios ad 18 -relatios), Oè=è memory er rocessor ad Oè=è local comutatio. Exerimetal data idicates that the algorithm has very good erformace i ractice THE ALGORITHM We first reset a BSP/CGM iteger sortig algorithm for ç 2,which serves as our base case, ad the exted it to the case ç. The iteger sortig algorithm for ç 2, described ext, follows the well kow determiistic samle sort method combied with radix sort for the sequetial sortig stes. We are makig the algorithm descritios fairly detailed i order to allow a aalysis that icludes estimates of costat factors. Algorithms 1: Sortig itegers o a rocessors BSP/CGM with ç 2. Iut: itegers i the rage 1;::: ;Oè c è, for fixed costat c, stored o a rocessor BSP/CGM, = itegers er rocessor. ç 2. Outut: The itegers are ermuted ito sorted order. 1. Each rocessor sorts locally its = itegers, usig radix sort. 2. Each rocessor selects from its locally sorted itegers a samle of itegers with raks i, 2 0 ç i ç, 1. We refer to these selected itegers as local samles. All local samles are set to rocessor P 1. (Note that P 1 is to receive i total 2 itegers. This is ossible sice ç 2.) 3. P 1 sorts the 2 local samles received i Ste 2 (usig radix sort) ad selects a samle of itegers with raks i, 0 ç i ç, 1. We refer to these selected itegers as the global samles. The global samles are the broadcast to all rocessors. 4. Based o the received global samles, each rocessor P i artitios its = itegers ito buckets B i;1 ;::: ;B i; where B i; are the local itegers with value betwee the è, 1è-th ad -th global samle. 5. I oe (combied) h-relatio, every rocessor P i, 1 ç i ç, seds B i; to rocessor P, 1 ç ç. Let R be the set of itegers received by rocessor P, 1 ç ç,adletr i = R i. 6. Every rocessor P i, 1 ç i ç, locally sorts R i usig radix sort.

3 Coarse Graied Parallel Iteger Sortig A global balacig shift oeratio which distributes all itegers evely amog the rocessors without chagig their order is erformed as follows: Every rocessor P i, 1 ç i ç, seds r i to P 1. Processor P 1 calculates for each P a array A of umbers idicatig how may of its itegers have to be moved to the resective rocessors. I oe h-relatio, every A is set to P, 1 ç i ç. The balacig shift is the erformed i a subsequet sigle h-relatio accordig to the A values. ed of Algorithm Theorem 1 Algorithm 1 sorts itegers i the rage 1;::: ;Oè c è, for fixed costat c, storedoa rocessor BSP/CGM, = itegers er rocessor, ç 2, usig 6 commuicatio rouds (2 -relatios ad 4 2 -relatios), Oè è local memory er rocessor, ad Oè è local comutatio. Proof. Correctess: The algorithm follows the well kow determiistic samle sort method combied with radix sort for the sequetial sortig stes. Let S i; be the set of itegers o P i, at the ed of Ste 1, with rak betwee 2 ad è +1è, 2 0 ç ç, 1. The mai observatio is that each R k cotais at most 3 sets S i;. Comlexity: The local memory ad local comutatio are bouded by the sequetial local radix sorts. For the commuicatio, we observe that Stes 2 ad 3 require oe 2 -relatio, each, Ste 5 requires oe -relatio, ad Ste 7 requires two 2 -relatios ad oe -relatio. 2 We ow describe the algorithm for the case ç. The basic idea is to artitio the rocessors ito grous G 1 ;::: ;G of rocessors, each. The mai oeratio cosists of ermutig the itegers such that all itegers stored i G i are smaller tha all itegers stored i G for all i é. We ca the aly Algorithm 1 to each grou. Agai, we are makig the descritio of Algorithm 2 fairly detailed i order to allow a aalysis that icludes estimates of costat factors. Algorithms 2: Sortig itegers o a rocessors machie whe ç. Iut: itegers i the rage 1;::: ; c, for fixed costat c, stored o a rocessor BSP/CGM, = itegers er rocessor. ç. Outut: The itegers are ermuted ito sorted order. 1. Grou the rocessors ito grous G 1 ;::: ;G of rocessors, each. For each grou, aly Algorithm 1 to sort the itegers withi the grou.

4 264 HIGH PERFORMANCE COMPUTING SYMPOSIUM Each rocessor seds its smallest iteger to rocessor P 1. We will refer to these items as the local miima. (Note that P 1 is to receive local miima i total, ad that this is ossible sice ç.) 3. P 1 sequetially sorts all local miima ad selects the itegers with rak i, 0 ç i ç, referred to as global slitters. These global slitters are broadcast to all rocessors (usig 2 -relatios). 4. Based o the received global slitters, each rocessor P i artitios its = itegers ito buckets B 0 i;1 ;::: ;B0 where B0 cotais the i; i; local itegers with value betwee the è, 1è-th ad -th global slitter. 5. Every rocessor P i, 1 ç i ç, seds B 0 to a rocessor i grou G i;, 1 ç ç. Let R 0 be the set of itegers received by rocessors i grou G, 1 ç ç,adletr 0 = R0. The routig schedule for this oeratio is determied as follows: First, withi each grou G i the size, t i;, of each iteger set set to grou G, 1 ç ç, is comuted. All t i;, 1 ç i; ç, are set to oe leadig rocessor er grou. Furthermore, withi each grou, the sizes of all B 0 are also set to the i; leadig rocessor. Each leadig rocessor ca the comute a routig schedule for its grou ad broadcast it to the rocessors i its grou. 6. Each grou G, 1 ç ç, sorts R 0 usig Algorithm A global balacig shift oeratio which distributes all itegers evely amog the rocessors without chagig their order is erformed aalogous to Ste 6 of Algorithm 1 but with a two hase scheme aalogous to the routig schedule comutatio i Ste 5 of Algorithm 2. ed of Algorithm Theorem 2 Algorithm 2 sorts itegers i the rage 1;::: ; c, for fixed costat c, storedoa rocessor BSP/CGM, = itegers er rocessor, ç, usig 24 commuicatio rouds (6 -relatios ad 18 -relatios), Oè è local memory er rocessor, ad Oè è local comutatio. Proof. Correctess: For each grou G i of rocessors, we have 0 = itegers ad 0 = rocessors. Therefore, 0 = = 0 = ç = 02,ad Algorithm 1 is alicable for each grou. Let S 0 be the set of itegers stored at i P i after Ste 1. The secod mai observatio is that each R 0 cotais at most 3 sets S 0. i Comlexity: The local memory ad local comutatio are bouded by the sequetial local radix sorts. For the commuicatio, we observe that Ste 1

5 Coarse Graied Parallel Iteger Sortig 265 Figure 26.1 Comutatio Times (I Secods) For Differet Numbers Of Data Items. The Three Curves Rereset Cofiguratios Of 2, 4, Ad 8 Processors, Resectively. requires 2 -relatios ad 4 -relatios, Ste 2 requires 1 -relatio, Ste 3 requires 2 -relatios, Ste 5 requires 1 -relatio ad 3 -relatios, Ste 6 requires 2 -relatios ad 4 -relatios, ad Ste 7 requires 1 -relatios ad 3 -relatios IMPLEMENTATION AND EXPERIMENTS We have imlemeted Algorithm 1 i MPI ad tested it o a multirocessor Petium latform ruig LINUX. The commuicatio betwee the rocessors is erformed through a Etheret switch. It is iterestig to observe that, eve o this low cost architecture, our algorithm shows good erformace. Figures 26.1, 26.2, ad 26.3 show the comutatio, commuicatio, ad total times (i secods), resectively, for differet umbers of data items. The three curves i each figure rereset cofiguratios of 2, 4, ad 8 rocessors, resectively. As exected, the commuicatio times are fairly similar. The comutatio times ad total ruig times are almost liear, ad we observe close to liear seedu. Note that, liear seedu is archived eve with a reasoably small workload. (May algorithms achive liear seedus oly for very high workloads.) I summary, we observe that the algorithm shows very good erformace i ractice.

6 266 HIGH PERFORMANCE COMPUTING SYMPOSIUM 1999 Figure 26.2 Commuicatio Times (I Secods) For Differet Numbers Of Data Items. The Three Curves Rereset Cofiguratios Of 2, 4, Ad 8 Processors, Resectively. Figure 26.3 Total Ruig Times (I Secods) For Differet Numbers Of Data Items. The Three Curves Rereset Cofiguratios Of 2, 4, Ad 8 Processors, Resectively.

7 26.4 CONCLUSION Coarse Graied Parallel Iteger Sortig 267 We have described a very simle, determiistic, otimal BSP/CGM iteger sortig algorithm that assumes = ç, which is usually the case i ractice. Our algorithm descritio is fairly detailed i order to allow a aalysis that icludes estimates of costat factors. The algorithm requires 24 commuicatio rouds (6 -relatios ad 18 -relatios), Oè=è memory er rocessor ad Oè=è local comutatio. For theoretical iterest, it is easy to see that the algorithm ca be geeralized to ru with Oè1=æè rouds for = ç æ, æé0. Exerimetal data idicates that the algorithm has very good erformace i ractice. Refereces [1] R. Cole, Parallel merge sort, SIAM J. Comut., 17(4), , [2] F. Dehe, A. Fabri, ad A. Rau-Chali, Scalable Parallel Geometric Algorithms for Coarse Graied Multicomuters, i Proc. ACM 9th Aual Comutatioal Geometry, , 1993 [3] F. Dehe, X. Deg, P. Dymod, A. Fabri ad A.A. Kokhar, A radomized arallel 3D covex hull algorithm for coarse graied arallel multicomuters, i Proc. ACM Sym. o Parallel Algorithms ad Architectures, [4] M.T. Goodrich, Commuicatio Efficiet Parallel Sortig, i Proc. 28th Aual ACM Sym. o Theory of Comutig (STOC 96), [5] L.G. Valiat, A Bridgig Model for Parallel Comutatio. Commuicatios of the ACM, Vol. 33, , 1990.

Sorting on Clusters of SMPs (Extended Abstract)

Sorting on Clusters of SMPs (Extended Abstract) Sortig o Clusters of SMPs (Exteded Abstract) David R. Helma Joseh JáJá Istitute for Advaced Comuter Studies & Deartmet of Electrical Egieerig, Uiversity of Marylad, College Park, MD 20742. fhelma, joseh