Partitioning for Parallelization Using Graph Parsing

Transcription

1 Prtitioning for Prlleliztion Using Grph Prsing C. L. MCrery Deprtment of Computer Siene n Engineering Auurn University, Al e-mil: mrery@eng.uurn.eu (205)

2 List of Figures Figure 3.1 Figure 3.2 Figure 3.3 Figure 3.4 Figure 3.5 Figure 3.6 Figure 3.7 Figure 3.8 Figure 3.9 Figure 3.10 Figure 3.11 Figure 3.12 Figure 4.1 Figure 4.2 Figure 4.3 Figure 4.4 A Grph n Its Trnsitive Reution Clns Types of Clns Proution exmples Cnonil proutions Cnonil prse Quotient grph-liner Quotient grph-primitive Cln hrteriztions Grph n prse tree An ritrrily lrge primitive Deomposing primitive Ajent inepenent lns Prllel vs. ggregte ost eisions Prtitioning Deision Liner sequene of inepenent lns

3 Prtitioning for Prlleliztion Using Grph Prsing C. L. MCrery* Deprtment of Computer Siene n Engineering Auurn University Introution Progrmming prllel systems for effiient exeution is iffiult hllenge tht urrently onsumes gret el of effort n time. Current progrmming prties n tools re lmost entirely rhiteture speifi, n the effiieny of progrm is primrily epenent on how omptile the progrm is with the hrwre. The softwre investment is wste when trnsition to new rhiteture or onfigurtion is require. Progrmming tools re neee to reue the lor-intensive tsk of progrmming n re-progrmming prllel systems. This pper presents grph-theoreti sheme for prtitioning n sheuling omputtion for effiient prllel exeution. The tehnique prses grph into hierrhy of sugrphs known s lns tht n e ientifie s le or unle to support prllel exeution. The tehnique is known s ln-se grph eomposition n will e referre to s CGD. Unlike stnr grph prtitioning tehniques, CGD onsiers the struture of the entire grph n ses prlleliztion eisions on this struture. Other prtitioning n sheuling tehniques generlly fous on only setion of the grph like the ritil pth or group of tightly onnete neighors to fin lol optimiztions. *This reserh is sponsore y NSF grnt no. CCR Exploiting Prllelism: Automti vs. Mnul The prolem of eveloping progrms for prllel systems hs een pprohe from two opposite iretions. One solution is the progrmmer-se pproh where environments n

4 lnguges re eing evise to i the progrmmer in reting his own prllel progrm. New progrms re written in extensions of existing lnguges or in speil prllel lnguges. The progrmmer retes prllel lgorithms tht re speilize to suit the hrwre onfigurtion n nnot esily e porte to other prllel systems. Muh of the effort of this pproh is ple in tuning the progrm to get goo spee-up on only one system. At the opposite en of the spetrum is the use of utomti tools to etet potentil prllelism n extrt it from existing sequentil oe. If the onversion n rete effiient prllel oe, the effort require to evelop suh systems will yiel gret pyoffs. The vntges in terms of lor svings re ovious. The gretest rwk is tht the onversion is epenent on the existing lgorithm. Humn insight to speify lternte formultions of the lgorithm tht offer greter opportunities for prllelism is not ville. Proly the est solutions lie in mile groun where prllel progrmmers intert with tools to evelop prllel progrms. The tools n pinpoint t epenenies n query the user out their removl. They oul lso store informtion out the hrteristis of the hrwre tht must e known in orer to evelop effiient progrms. Temporl n sptil onerns oul e remove from the user n their responsiility ple with the tools. This pper presents solution to eveloping one of those tools: the utomti prtitioning tool. 2. Tsks for Automti Prlleliztion of Seril Coe When utomtilly onverting sequentil progrm to oe suitle for exeution on multiproessor system, there re four non-trivil prolems to e solve: the ientifition, through progrm epenene nlysis, of oe frgments whih n e exeute in prllel; the elimintion of non-essentil epenenes to mximize the numer of prllelizle oe frgments; the ggregtion of suh frgments into proesses tht optimize overll performne (in terms of oth ommunition n proessor time); the ssignment of proesses to proessors (sheuling).

5 The fous of the pper is on metho to omplish the lst two tsks: prtitioning n sheuling. Prtitioning Speeup is the rtio of the running time of progrm on single proessing unit to the running time on prllel omputer. The iel speeup for prllel omputtion on multiproessor with n noes is n (liner speeup). Liner speeup is virtully impossile to otin, however, ue to two prolems: (1) limite ville prllelism in the omputtion n (2) overhe. When the result of one sttement is require for nother sttement, the sttements must e exeute sequentilly, n epeneny is si to exist etween the sttements. Overhe osts re inurre through inter-proessor ommunition, memory ontention, synhroniztion, sheuling, n other system tivities. As higher egrees of prllelism re introue into omputtion, the expettion is tht the totl proessing time will e erese. Empiril tests hve shown tht threshol exists eyon whih n inrese in the numer of proessors tully inreses the proessing time require. The prolem is tht two onfliting gols re t work: high egree of prllelism n low ommunition osts. On the one hn, inresing the numer of proessors llows more instrutions to e exeute in the sme mount of time, ut when more proesses re working on the sme omputtion, itionl ommunition is require etween the proesses. This key issue in the retion of n effiient prllel progrm is tht of hoosing the proper grin size. The grnulrity of prllel progrm is efine to e the verge size of sequentil unit of omputtion in the progrm, with no inter-proessor synhroniztions or ommunitions [22]. Prtitioning estlishes the grnulrity of omputtion. The optiml grnulrity is epenent on rhiteturl hrteristis. Systems with few powerful proessors n istriute systems with expensive inter-proessor ommunition will require orser grin thn systems with mny, ut less powerful proessors with fst inter-proessor ommunition. The gol of prtitioning is to minimize the elpse time from the eginning of the progrm exeution to the en. The prtitioning tool filittes this proess y etermining the optiml grin size for the lgorithm s pplie to prtiulr rhiteture.

6 3. Cln-se Grph Deomposition Mny prtitioning tehniques hve een propose, ut most hve een se on heuristis tht mke prtitioning eisions with informtion only from some sugrph. Cln-se grph eomposition etermines the struture of the entire unerlying epeneny grph efore ttempting to formulte the pproprite grin. Its fountions re from the work on 2-strutures y Ehrenfeuht n Rosenerg[5,6,7] n is relte to the work of Buer n Mohring[4]. This setion explins the tehnique. Progrm Depenene Grph A progrm epenene grph (PDG) is irete grph where the verties represent loks of sttements n the eges represent essentil ontrol or t epenene[8]. The strongly onnet omponent grph or onenstion grph of the PDG n e use to represent the epenenes in the progrm. In this interprettion, single grph vertex oul moel progrm loop, n the omponent orrespons to strongly onnete omponent of the PDG. Loops n lso e unrolle n grph verties n represent single or multiple loop itertions. Ege weights will e ssigne to estimte the mgnitue of ommunition etween grph verties n vertex weights will estimte the exeution requirements of the lok. The intention is to leve the interprettion of the PDG flexile so tht the prtitioning tehnique n e pplie in vriety of settings. The only requirement of the eomposition is tht the input e irete yli grph (DAG). Clnse grph eomposition (CGD) is prse of the PDG into hierrhy of sugrphs. Clns The entrl fous of the prtitioning sheme is the ln. Let G e DAG. A suset X! G is ln iff for ll x, y " X n ll z " G - X, () z is n nestor of x iff z is n nestor of y, or () z is esennt of x iff z is esennt of y. An lternte esription of ln epits it s suset of verties where every element not in the suset is relte in the sme wy (i.e. nestor, esennt or neither) to eh memer in the suset. Trivil lns inlue singleton sets n the entire grph. In Figure 3.1, sets {2,3},

7 {2,3,4,5,6}, {1,2,3,4,5,6}, n {2,3,4,5,6,7} re the nontrivil lns. C={2,3} is ln sine vertex 1 is n nestor of eh element of C n 5 n 7 re esennts of eh element of C. The set {2,3,4} is not ln sine 6 is esennt of only vertex 4. Clns re known in the literture y mny nmes[20]. Spinr n Muller use the term moules [20], n Buer n Mohring [4] ll them utonomous sets. e e G G t Figure 3.1 A grph n its trnsitive reution A ln C is lssifie s one of three types. It is (i) primitive if the only lns in C re the trivil lns; (ii) inepenent if every sugrph of C is ln; or (iii) liner if for every pir of verties x n y in C, x is n nestor or esennt of y. Inepenent lns re sets of isolte verties. Figure 3.2 is n exmple of primitive ln. Liner lns re sequenes of one or more verties v i,v i+1,...,v j-1,v j where for i < k, v i is n nestor of v k. (See Figure 3.2). Grph-grmmrs String grmmrs re speil se of grph-grmmrs. A string is isomorphi to liner grph, n in proution of the string grmmr, the replement string is onnete to the host string in nturl wy. More speifilly, the preeessor of the host eomes the preeessor of the first element of the replement string n the suessor of the host eomes the suessor of the lst element of the replement string. For exmple, if the proution rule --> e, is pplie to the

8 string xy, the string xey is proue. x eomes the preeessor of n y eomes the suessor of e Figure 3.2 Clns In sequentil grph rewriting system or grph-grmmr, grphs re generte from some initil grph y proutions where the mother grph, sugrph of the host grph, is reple y nother grph, the ughter grph. The min prolem of grph-grmmrs is speifying wht eges shoul e e to onnet the ughter grph to the host grph n etermining how eges inient to the mother grph shoul e moifie in the erive grph. The speifition of the eges onneting the ughter grph to the host grph is lle the emeing rule. For CGS, the grph-grmmr efine here, the reonnetion rule or emeing rule is lle hereity. All host n ughter grphs re Hsse grphs. An emeing is lle hereitry when the mother grph onsists of single vertex n eh vertex in the ughter grph of proution is onnete to the host grph in extly the sme wy s its mother. More formlly, let the mother

9 grph e vertex u. For eh vertex v in the set of soure verties in the ughter grph, (w,v) is n ege in the resultnt grph whenever (w,u) is n ege in the host grph. For sink verties, t, of the ughter grph, (t,w) is n ege in the resultnt grph whenever (t,w) is n ege in the host grph. A grph-grmmr is system G = (z, P, H) where z is vertex lle the xiom or strt grph, P is set of proutions n H is n emeing rule. The prtiulr grph-grmmr of the work in this pper is lle the ln genertion system, CGS. For CGS, the xiom is single vertex; P is set of pirs (v,d) where v is ny grph vertex n D is primitive, inepenent or liner ln; n H is the hereitry rule of reonnetion. Let us ll proution of this system CGS-proution. Applitions of CGS-proutions preserve the properties of Hsse grphs. Furthermore, the ughter grph eomes ln in the resultnt grph. Figure 3.3 shows exmple proutions with the mother vertex ientifie s vertex 3. () Inepenent ln () Primitive ln () Liner ln Cnonil proutions Figure 3.3 Types of lns A sequene of proutions is lle nonil if (i) proution (v,d) with liner ughter grph is never followe iretly y nother proution (,D') where " D n D' is liner, n (ii) proution (v,d) with inepenent ughter grph is never followe iretly y nother proution (,D') where " D n D' is inepenent.

10 Note tht requiring proutions to e nonil oes not reue the set of possile grphs tht re generte. Figure 3.4 emonstrtes this property. Cnonil proutions re importnt euse ny Hsse grph n e generte y unique sequene of nonil proutions, n the uniqueness les to hope of eomposing ny Hsse grph into its originl omponents. Figure 3.5 emonstrtes three ifferent prse trees for the liner grph with four verties. The first orrespons to the sequene of proutions: (v, x->y), (x,->), (y, ->). The seon omes from the sequene: (v, x->), (x,y->), (y,->). The thir whih is the nonil proutions is the single proution, (v, ->->->) The nonil requirement gurntees tht the prse must e tht shown in Figure 3.5() G m = 3 D () G G m = 3 D G () Figure 3.4 Proution exmples

11 e e g h g h m = f f m = e f G D G D G () Two jent inepenent proutions m = g h f g h f G D G () Cnonil proution Figure 3.5 Cnonil proutions Quotient grph The onept of quotient grph is importnt for CGS euse it permits the lssifition of omplex lns into the tegories of liner, primitive or inepenent. Let C e ln n with {C 1,C 2,..C k } prtition of C where eh C i is ln of C. The quotient grph of C enote C/C 1..C k, is the grph with verties C 1,C 2,.C k. Pir(C i,c j ) is n ege of C/C 1..C k iff x " C i, y " C j, n (x,y) is n ege of C. In Figure 3.6, the lns tht prtition C re C 1 ={}, C 2 = {,,}, C 3 = {e}, n the quotient grph is liner. The lns tht prtition of Figure 3.7 re {,}, {}, {}, n

12 {e} n the quotient grph is primitive. Every ln n e ientifie s liner, primitive or inepenent oring to the lssifition of its quotient grph. liner grph L L L L L L L () () () Figure 3.6 Cnonil prse Theorem 3.1 When the originl grph is Hsse grph, the grph generte y CGS is lso Hsse grph. Proof: The proof follows iretly s onsequene of the emeing sheme. Sine new eges re onnete only to soures n sinks of the ughter grph, no short ut eges re e n the grph remins Hsse grph. Theorem 3.2 Any Hsse grph n e generte y unique nonil sequene of proutions from CGS. Proof: Grphs re speil se of 2-strutures n this theorem is speil se of theorem 7.11 in [6].

13 Theorem 3.3 For every Hsse grph there is unique eomposition into quotient grphs tht re ientifie s liner, primitive or inepenent. Proof: See pges [6]. e C 1 C C 2 3 C C/C C C Figure 3.7 Quotient grph - liner Prsing Theorem 3.3 gurntees the existene of unique prse tree from the lns of grph. The lns tht eome omponents of the prse tree re irete sugrphs C with the following properties: 1. Any nestor of one soure element of C is n nestor of ll elements of C. 2 Any esennt of one sink of C is esennt of ll elements of C. 3 All hilren of non-sink elements of C must e ontine in C. 4 All prents of non-soure elements of C must e ontine in C. These four properties ssure tht ll elements of C re onsiere s ientil in their epenene reltionship to ll elements not in C n provies n lternte hrteriztion of the efinition of ln. Properties 1 n 2 suggest tht groups of verties with the sme prents n groups of verties with the sme hilren shoul e ientifie, respetively, s potentil soures n sinks of lns. Properties 3 n 4 prohiit entry to n exit from the ln t points other thn

14 soures or sinks. Theorem 3.4 Let S e the prtition of the verties of grph G into sets S were x,y " S iff the set of prents of x is the sme s the set of prents of y n let M e the prtition of verties of G into sets M where x,y " M iff the set of hilren of x is the sme s the set of hilren of y. Let D(S) enote the union of S n the set of esennts of S, n let A(S) enote the union of S with the set of nestors of S. A sugrph C of G is ln iff (i) C = D(S') # A(M') for some S'! S " S n M'! M " M. (ii) D(S') - (D(M') $ A(M)) = %, n (iii) A(m') - (D(S') $ A(S')) = %. Proof: See [15] Property (i) efines sugrph whose soure elements, S', hve the sme set of prents n whose sink elements, M', hve the sme set of hilren. If this were not the se, the ln property woul e violte. Properties (ii) n (iii) gurntee tht there is no illegl entry or exit from the ln. (ii) prohiits ses where there is n ege from ln element tht is not sink to non-ln element, n (iii) gurntees tht no ege enters the interior of the ln. In Figure 3.8 onsier S = {,,e} " S, n M= M' = {i,j} " M. S' = {,}. The nestor n esennt sets re: A(S') = {,,,}; A(M') = {,,,,f,g,h,i,j} D(S') = {,,f,g,h,i,j,k,l} D(M') = {i,j,k,l} The potentil ln is C = D(S') # A(M') = {,,f,g,h,i,j} Sine properties (ii) n (iii) hol, C is ln. If the eges (e,g) n (h,m) re e to the grph, properties (ii) n (iii) re violte: m is then in D(S') ut not in (D(M') $ A(M)) n e is then in A(M') ut not (D(S') $ A(S')). All lns n e foun y omputing C from theorem 3.4(i) using ll pirs (S, M) of sets from S n M. This woul pir s mny s n 2 sets, mny of whih oul not form ln. To reue the options, prtil orering is efine on the elements from M n S, n only those pirs where elements of S re nestors of M re onsiere. Liner lns n e etete s the ontention of two or more lns. Two lns, C 1 n C 2, omine to form liner ln C 3 iff the set of hilren of the sink of C 1 is the sme set s the soure of C 2 n the set of prents of the soure of C 2 is extly the sink of C 1. Inepenent lns hve

15 more thn one onnete omponent, n eh omponent forms its own suln. An exmple grph n prse is shown in Figure 3.9. Internl verties on the tree re lele, I for inepenent, P for primitive, n L for liner. C C 1 2 e C 3 C 4 C C/C C C C Figure 3.8 Quotient grph - primitive e f g h i j k l m Figure 3.9 Cln hrteriztions

16 Deomposing Primitives When prtitioning for prllel exeution, primitive lns represent speil hllenge. They o not fll into the ler-ut tegories of verties tht n e exeute simultneously nor verties tht must exeute sequentilly. In ition, primitive lns n e ritrrily lrge s shown in Figure One proeure for further reution of primitive lns is the ition of eges of weight zero[17]. For primitive ln P, eges from the soure verties to the union of their hilren or from sink verties to the union of their prents. The ition of eges gurntees tht the previous soures (sinks) form n inepenent ln whih is linerly onnete to the rest of the ln. This proess n e pplie reursively. Often, single ege ugmenttion n le to full eomposition (see Figure 3.11) The prse tree of ny ompletely eompose grph is iprtite tree where the internl verties re lssifie s either liner or inepenent. For onnete grph, the root vertex is lwys liner. See Figure L I 1 P G Fig Grph n prsetree

17 4n-2 4n+1 4n-1 4n-3 4n Figure 3.11 An ritrrily lrge primitive

18 L I I I L I e f g h (e) Prse tree of eompose primitive Figure 3.12 Deomposing primitive () Liner onnetion () liner L g h e f g e h f () inepenent e f g h () Primitive: ugment with zero-weight eges (,e) (,f) (,) (,) e f g h

19 4. Grph Deomposition s Mens of Prtitioning for Prllel Exeution Cln-se grph eomposition hs severl properties tht mke it very useful in the prlleliztion of omputtion. A ln is olletion of omputtionl elements with ientil nee to ommunite with other lns. Grouping y lns therefore yiels sustntil reution in ommunition overhe. Clns re lssifie s liner, primitive or inepenent. The liner lns of PDG orrespon to progrm segments tht must exeute sequentilly, while the oe segments represente y inepenent lns my exeute in prllel. Primitive lns re eompose further into liner or inepenent lns. The ln struture erive from PDG is unique, n forms hierrhy tht n e regre s prse tree of the PDG. The levels of the PDG-prse tree orrespon to ifferent egrees of prlleliztion tht n e pplie to prtiulr progrm. When ost moel for prtiulr rhiteture is pplie to the PDG-prse tree, the optiml grins n e etermine for tht rhiteture. At eh inepenent ln in the prse tree, eision must e me s to whether the hilren will e exeute in prllel or ggregte to exeute sequentilly on single noe. Cost metris No single prllel rhiteture hs yet emerge s the stnr. Multiproessor systems my hve few fst proessors suh s the Cry vetor proessors. The system might e n rry multiproessor (SIMD), or shre memory synhronous moel (MIMD-SM) or istriute memory system (MIMD-DM). Eh of these moels hs ifferent performne hrteristis, ifferent types of inter-proessor ommunition overhe, ifferent sheuling requirements, n ifferent requirements for synhroniztion. In short, eh system requires its own unique moel of performne hrteriztion or ost metri. CGD nlyzes the struture of the PDG with no regr for the trget rhiteture n its ost

20 metri. When it is neessry to mke prtitioning eisions, the ost metri must eome prt of the proess. The next setion will emonstrte the pplition for ost moels in generl n will give s n exmple the eision formuls tht orrespon to the ipsc system of prllel omputers. The ipsc multiproessor omputing system is istriute memory synhronous multi-proessor system with hyperue interonnetion topology. Assoite with eh noe is iret onnet moule (DCM) tht hnles the inter-proessor ommunition. Although the noes re onfigure s hyperue topology, the DCMs set up virtul links etween ny pir of noes n the ost of pssing messges etween ny pir of noes is essentilly the sme s pssing messges etween iretly onnete neighors [19]. For this reson, the ipsc hyperue n e viewe s ompletely onnete. To pply ost metri to the prse tree, it is neessry to mke some simplifying ssumptions out the ommunition n exeution requirements of the system: (1) Communition n exeution hppen onurrently. (2) The totl time to exeute more thn one proess on proessor is the sum of the proess exeution times. (3) Communition times re inepenent of the ommuniting noe lotions. (4) The prllel exeution time of proesses strting t the sme time is the mximum of the iniviul exeutions. (5) Inresing the mount of informtion to e sent from noe oes not hnge the ost of the ommunition. These simplifying ssumptions out the unerlying ommunition metri re not unrelisti for the ipsc systems. The performne mesurements in [19] prove tht (1) n (3) re true for these systems. (2) n (4) re ssumptions generlly gree upon y reserhers of prllel systems[3, 11,13]. The lst ssumption shoul e moifie for more etile pplitions of the ost metri. However, it is very resonle ssumption for the ipsc system, espeilly when messges re smll. The performne mesurement work [19] shows tht the gretest ommunition ost is in the strt-up overhe. All messges uner 100 ytes hve the sme trnsmission osts. For messges up to 1000 ytes, fully 50% of the ommunition ost is overhe. Unless messges re extremely lrge, the fifth ssumption is resonle.

21 Applition of Cost metri to the Prse Tree The prse tree shows proesses tht my exeute in prllel s well s those tht must exeute sequentilly. It is the jo of prtitioning to etermine when to use the opportunities to exeute in prllel n when to group possile prllel tsks into single grin. The performne hrteristis s esrie y the ost metri re require to mke the prtitioning eisions. e e e e Inepenent Liner Inepenent Figure 4.1 Ajent inepenent lns To etermine whih inepenent lns shoul e exeute in prllel, inspet the prent (liner) vertex in the prse tree n exmine the jent hilren pirwise. Imgine the sugrph of Figure 4.1. By ompring the ost of ggregtion with the ost of prlleliztion, eision on the metho of exeution n e me. In the se where the inepenent lns ontin 2 verties, there re four possiilities for the jent hilren of liner vertex with seven sheuling hoies: 1. oth the left n right hilren n e exeute in prllel with () verties n ple on one proessor, n on nother or () n on the sme proessor, n on nother 2 the left hil n e exeute in prllel n the right hil ggregte with () vertex ple on the sme proessor s n or () vertex on the sme proessor s n 3 the left hil n e ggregte n the right hil exeute in prllel with () vertex ple on the sme proessor s n

22 or () vertex on the sme proessor s n. 4 oth hilren n e ggregte n ll four verties,,, n ple on the sme proessor. The seven ses re represente in Figure 4.2. The smll squres represent the eision to prllelize n the soli ovls the eision to ggregte. The she ovls represent the sheuling hoies where ll verties within the ovl re to e exeute on the sme proessor. The pproprite metri n e pplie to the grph t this point. The exmple equtions in Figure 4.2 re se on the ipsc ost moel. 1 mx (mx (, +e ) +, mx (, + e ) + ) 1 mx (mx (, +e ) +, mx (, + e ) + ) 2 mx (, + e ) mx (, + e ) mx (, e + ) mx (, e + ) Figure 4.2 Prllel vs. ggregte ost eisions

23 Let x give the ost of exeuting PDG vertex x, n let e x represent ost of the ommuniting the results of proess x to ny other proess. Uner the ssumptions, the formultions of the omine exeution times of the jent inepenent verties re given next to the igrms. In 1, for exmple, n exeute onurrently n pss informtion efore n n exeute. The proessor ontining n exeutes, wits for the results from, if neessry, n then exeutes for totl time of mx(, + e ) +. Similrly the proessor ontining n exeutes for totl time of mx(,+ e ) +. By ompring these seven vlues the est lol eision n e me. The lultions for the ses n e revise to fit the trget rhiteture. Our formuls re merely exmples of how to rrive t the proper eisions. When one of the hilren of liner prse tree vertex is singleton n the other inites the inepenene of two verties, only three hoies hol: (i) prllelize the two n ple the first on the sme proessor s the single vertex; (ii) prllelize the two n ple the seon on the sme proessor s the single vertex; or (iii) ple ll three verties on the sme proessor. The formuls re similr to the ones for the seven ses. When either hil hs more thn three verties, heuristi proeures must e use to etermine the proper ggregtion/prlleliztion eision, for onsiering ll ses is unresonle. Let (i), e the ost of trnsmitting the informtion proue y i, x(i) e the exeution time of vertex i, n the S(i) e the sum of i's exeution time plus ommunition time. The heuristi lgorithm fins lower oun on the ost n ggregtes proesses when those whose ggregtions o not inrese the overll proessing time ove the lower oun. First onsier the se of the liner vertex with two inepenent hilren n the sitution where there re mny proesses in the left hil of the liner vertex. Sort the left-hil's proesses in non-eresing orer y S(i) n sy S(i 1 ) & S(i 2 ) &... &S(i k ). If there is extly one right hil r, vertex i 1 shoul e sheule on the sme proessor s r. If the sum of ll the exeution times is no greter thn S(i 1 ), ll proesses shoul e ggregte. Otherwise the lower oun for proessing the remining verties is L = mx(x(i 1 ),S(i 2 )) Aggregte the proesses in orer i 3...i 3+q until the sum of the exeution times plus the mximum ommunition time eomes greter thn L. Assign n itionl proessor for the

24 next vertex group n repply the ggregtion riteri until ll verties re sheule. Figure 4.3 illustrtes this sitution. When there is more thn one right hil n the eision is me to prllelize the right hil into more thn one group, the lower oun eomes S(i 1 ). The proess n e reformulte for lternte ost moels. liner inepenent h lef h e 15 5 f loweroun = g e f g Figure 4.3 Prtitioning eision Liner lns re not limite to the onnetion of only two sets of inepenent lns, ut n represent the sequentil epenenies of ny numer of lns s illustrte y Figure 4.4. In this se, the ggregtion/prllelize eisions on pirwise jent lns my le to onflit. If jent eisions gree on the ommon (mile) ln, the jent onfigurtions re referre to s stle. The sequene of eisions is moelle y multistge grph where the noes in stge i represent the eision spe for stge i n the ege weights from stge i to stge i+1 give the ost

25 of the orresponing eisions. The shortest pth through the grph gives the orret eision t eh stge [18]. Sine the leves inherit their osts from the progrm DAG, the internl vertex lultions egin from ottom up y etermining the lest ost onfigurtion for jent pirs of inepenent verties. Eh eision orrespons to n ege in the prent (liner) ln. () Sequene of inepenent lns () Prsetree Figure 4.4 Liner sequene of inepenent lns 5. Comprison of Prtitioning Methos Mny prtitioning tehniques hve een propose. Some tehniques fil to ount for ommunition overhe[3], trget only one lss of prolem[9][25], or minimize ommunition without regr to exeution time[24]. Aitionl strtegies ttempt to mp the grph of prtiulr lgorithm or set of lgorithms onto the physil lyout of some rhiteture with primitive or no ommunition mesurements[2]. Tehniques tht nlyze the PDG usully exmine some pertinent sugrph suh s the ritil pth[22,13,21] or neighorhoos of noes[1.]. None of the shemes of whih we re wre nlyze the entire grph exept ln eomposition. Creting the hierrhy of lns esries the struture of the grph tht is neessry for prlleliztion n provies equte informtion for mking wise eisions. A stuy is urrently unerwy to ompre the results of ln-se prtitioning with some of the other more promising tehniques. The first phse of the stuy will sheule PDGs from

26 vriety of rel progrms. The PDGs tht hve een gthere represent oth very regulr omputtions suh s FFT n some irregulr omputtions. Preliminry results show lns in positive light.

27 Biliogrphy 1. Bxter, J., n Ptel, J., "The LAST Algorithm: A Heuristi-Bse Stti Tsk Allotion Algorithm," Proeeings of the 1989 Interntionl Conferene on Prllel Proessing, pp Bermn, F., "Experiene with n Automti Solution to the Mpping Prolem", in The Chrteristis of Prllel Algorithms, y L. H. Jmieson, D. B. Gnnon, n R. J. Douglss, es.,mit Press, Cmrige, Bokri, S., "On the Mpping Prolem", IEEE Trnstions on Computers, Mrh, Buer, H., n Mohring, R.H., "A Fst Algorithm for hte Deomposition of Grphs n Posets", Mthemtis of Opertions Reserh, Vol. *, No. 2, My 1983, pp Ehrenfeuht, A. n Rozenerg, G., "Theory of 2-Strutures, Prt I: Clns, Bsi Sulsses, n Morphisms," Theoretil Computer Siene (1990), pp Ehrenfeuht, A. n Rozenerg, G., "Theory of 2-Strutures, Prt II: Representtion Through Lele Tree Fmilies," Theoretil Computer Siene (1990), pp Ehrenfeuht, A. n Rozenerg, G., "Primiitive is Hereitry for 2-strutures," Theoretil Computer Siene (1990), pp Ferrnte, J., Ottenstein, K. J., n Wrren, J. D., "The Progrm Depenene Grph n Its Use in Optimiztion.", ACM Trnstions on Progrmming Lnguges n Systems, 9: 3 (July 1987), pp Fortes, J. A. B. n Molovn, D. I., "Prllelism Detetion n Trnsformtion Tehniques Useful for VLSI Algorithms", Journl of Prllel n Distriute Computing, 2 (1985), pp Gersoulis, A. n Yng, T. On the Grnulrity n Clustering of Direte Ayli Tsk Grphs. LCSR-TR-153, Dept. of Computer Siene, Rutgers University. Sept. 1990, pp Gions, P., "A More Prtil PRAM Moel", Interntionl Computer Siene Institute, Berkeley, CA. Tehnil Report TR , Jmieson, L. H., Gnnon, D. B. n Douglss, R. J., eitors, The Chrteristis of Prllel Algorithms, MIT Press, Cmrige, Kim, S.J. n Browne, J.C., "A Generl Approh to Mpping of Prllel Computtions Upon Multiproessor Arhitetures", Proeeings of the 1988 Interntionl Conferene on Prllel Proessing, Vol. III, pp Leiserson, C., n Mggs, B., "Communition Effiient Prllel Grph Algorithms", Proeeings of the Interntionl Conferene on Prllel Proesing, 1986, pp MCrery, C.L., "An Algorithm for Prsing Grph Grmmr", Ph.D. Thesis, Deprtment of Computer Siene, Univeristy of Coloro, MCrery, C. L. n Gill, D. H., "Automti Determintion of Grin Size for Effiient Prllel Proessing," Communitions of the ACM Vol. 32, No. 8, Sept. 1989, MCrery, C. L. n Gill, D. H., "Effiient Exploittion of Conurreny Using Grph Deomposition," Proeeings of the 1990 Interntionl Conferene on Prllel Proessing 18. MCrery, C. L., Gill, D. H. n Zhu, Y., "Shortest Pth Evlution for Hierrhil Grin

28 Aggregtion", Auurn Univ. Teh Rpt. CSE MCrery, C., MArle, M., n MCrery, J., "Hyperue Communition Performne", Auurn University Dept. of Computer Siene n Engineering Tehnil Report CSE Muller, J.H., n Spinr, J., "Inrementl Moulr Deomposition", Journl of the ACM, Vol. 36, No. 1, Jn pp Ppimitriou, C. H.,n Ynnkkis, M., "Towrs n Arhiteture-Inepenent Anlysis of Prllel Algorithms", Proeeings of the 1988 Symposium on Theory of Computing, pp Srkr, V., Prtitioning n Sheuling Prllel Progrms for Multiproessors, MIT Press, Cmrige, Strmm, B. n F. Bermn, (Otoer 1990), "Performne Preition--How Goo is Goo?," Proeeings of the Thir Symposium on the Frontiers of Mssively Prllel Computtion, pp Towsley, D., "Alloting Progrms Contining Brnhes n Loops Within Multiple Proessor System," IEEE Trnstions on Softwre Engineering, Otoer 1986,SE-12: Zhu, Y. n MCrery, C., Optiml n Ner Optiml Tree Sheuling for Prllel Systems, Fourth IEEE symposium on Prllel n Distriute Proessing., De. 1992, pp Zim, H., Superompilers for Prllel n Vetor Computers, ACM Press, New York, 1990.

29

30