ATsunami is a series of waves generated in a

Transcription

1 The Internatonal Journal on Advances n ICT for Emergng Regons (0) : 3-0 On the Performance of the Parallel Implementaton of the Shallow Water Model on Dstrbuted Memory Archtectures K.Ganeshamoorthy, D.N.Ranasnghe, K.P.M.K.Slva and R.Wat Abstract Ths paper presents a study of the mpact of memory archtectures, dstrbuted memory (DM) and vrtual shared memory (VSM), n the soluton of parallel numercal algorthms on a mult-processor cluster. A parallel mplementaton of the shallow water equatons to model a Tsunam s chosen as the case study. Data s parttoned nto sub-domans, namely a four by three grd scheme and an eght by s grd scheme whch are used for the parallel mplementaton of ths model. There are four versons of the parallel algorthm for each grd scheme: dstrbuted memory wthout threads, dstrbuted memory wth threads, vrtual shared memory wthout threads, and vrtual shared memory wth threads. These four parallel versons have been mplemented on a hgh performance cluster, connected to the Nordugrd. Eperments are realzed usng the Message Passng Interface (MPI) lbrary, the C/Lnda, and the Lnu pthreads. Subect to the avalablty of memory, the vrtual shared memory verson wthout threads performs best, and as the task s scaled up, the threaded verson becomes effcent n both DM and VSM mplementatons. A Tsunam s characterzed as a shallow water wave. Shallow water waves are dfferent from wnd generated waves, the waves many of us have seen at the beach. Wnd generated waves usually have a perod of fve to twenty seconds and a wavelength of about one hundred to two hundred meters. A tsunam can have a perod n the range of ten mnutes to two hours and a wavelength n ecess of 500 km. It s because of ther long wavelengths that a tsunam behaves as shallow water waves. A wave s characterzed as a shallow water wave when the rato between the water depth and ts wavelength gets very small []. Inde Terms MPI, Lnda, mult processors, Shallow water equatons, tsunam model. I. INTRODUCTION ATsunam s a seres of waves generated n a body of water by an mpulsve dsturbance that vertcally dsplaces the water column [], []. Earthquakes, landsldes, volcanc eruptons, eplosons can generate a tsunam. A tsunam can savagely attack coastlnes, causng devastatng property damage and loss of lfe. Manuscrpt receved March 6, 009. Accepted July 7th, 009. Ths research was funded by the Natonal Scence Foundaton, Sr Lanka (Grant No. RG/005/FR/07) and by SPIDER, the Swedsh programme of ICT for developng Regons. K.Ganeshamoorthy s wth the Department of Computaton & Intellgent Systems, Unversty of Colombo School of Computng, 35, Red Avenue, Colombo-7, Sr Lanka. (e-mal: ganesh@webmal.cmb.ac.lk). D.N.Ranasnghe and K.P.M.K.Slva are also wth the Department of Computaton & Intellgent Systems, Unversty of Colombo School of Computng, 35, Red Avenue, Colombo-7, Sr Lanka. (emal: dnr@ucsc.cmb.ac.lk, mks@ucsc.cmb.ac.lk) R.Wat s wth the Internatonal Scence Programme and Department of Informaton Technology, Uppsala Unversty, Uppsala, Sweden. (rchard.wat@sp.uu.se) Fg.. Typcal med mode programmng model [9]. The shallow water equatons on a rotatng sphere serve as a prmary test problem for numercal methods used n modelng global atmospherc flows [3]. They descrbe the behavour of a shallow homogeneous ncompressble and nvscd flud layer. They present the maor dffcultes found n the horzontal aspects of three-dmensonal global atmospherc modelng. Thus, they provde a frst test to weed out potentally non-compettve schemes wthout the effort of buldng a complete model. However, because they do not represent the complete atmospherc system, the shallow water equatons are only a frst test. Ultmately schemes whch look attractve based on these tests must be appled to the complete baroclnc problem. The estence of a standard test set for the shallow water equatons wll encourage the contnued eploraton

2 4 K.Ganeshamoorthy, D.N.Ranasnghe, K.P.M.K.Slva and R.Wat of alternatve numercal methods and provde the communty wth a mechansm for udgng the merts of numercal schemes and parallel computers for atmospherc flow calculatons [3]. The current state-of-the-art n tsunam modelng stll requres consderable qualty control, udgment, and teratve, eploratory computatons before a scenaro s assumed relable. Ths s why the effcent computaton of many scenaros for the creaton of a database of pre-computed scenaros that have been carefully analyzed and nterpreted by a knowledgeable and eperence tsunam modeler s an essental frst step n the development of a relable and robust tsunam forecastng and hazard assessment capablty. Usng more advanced parallel algorthms, t may become techncally feasble to eecute realtme model runs for gudance as an actual event unfolds. However, ths s not currently ustfed on scentfc grounds; an operatonal real-tme model forecastng capablty must awat mproved and more detaled characterzaton of earthquakes n real-tme, and verfcaton that the real-tme tsunam model computatons are suffcently robust to be used n an operatonal, real-tme model [4]. Shallow water equatons have been wdely used to study the Tsunam phenomenon [5] [3], as a model of the basc flud dynamcs of the ocean. Solvng partal dfferental equatons numercally for real-lfe problems are computatonally demandng, therefore, utlzng super-computers/clusters effcently s mportant n order to acheve computatonal effcency [6]. Strateges to mprove the accuracy and overall qualty of model predctons have been and contnue to be of great nterest to numercal model developers but n addton to accuracy, the utlty of a numercal model s greatly affected by the algorthm s effcency [7]. Parallel computng provdes a feasble and effcent approach to solve very large-scale predcton problems but any redstrbuton of the data s a potentally tme-consumng task for parallel archtectures [8]. Fg.. shows the memory herarchy that ests n most nodes of a modern cluster envronment [9], where many nodes are lnked together by a hgh-speed network; and nsde each node there may be two or more processors; along wth each processor, memory access s ether to a hgh speed memory unt cache or the low speed man memory. The am of ths paper s to study the mpact of memory archtectures assocated wth dstrbuted memory and vrtual shared memory n the etracton of multple levels of parallelsm, for the soluton of numercal algorthms. The obectve here s to evaluate the effects of the programmng model on the scalablty of ths shallow water model. Computng the wave propagaton n the tsunam model, where the entre ocean s the soluton doman, s challengng, both due to the huge amount of computaton needed and due to the fact that dfferent physcs apples n dfferent regons [0]. The Message Passng Interface, MPI [], and C/Lnda [] are alternatve paradgms for communcaton between global nodes n the dstrbuted memory envronment and n the vrtual shared memory envronment, respectvely. In the med mode programmng model, both MPI and C/ Lnda are used for communcaton between global nodes whle nsde each MPI process and wthn each C/Lnda process POSIX threads [3] are used n order to etract further parallelsm. Ths paper s organzed as follows: In the secton, Related Work, we present the related work for our study of research. The secton on Lnear Long Wave Theory descrbes the lnear long wave theory used to smulate the tsunam model. In Algorthm Desgn, we descrbe our desgn prncples. Implementaton s presented net. Epermental results from both memory archtectures on hgh performance cluster are presented and are dscussed n Evaluaton. Fnally, n Concluson we make the concludng remarks. A. II. Related Work Parallel smulators for Tsunam Hybrd tsunam smulators that allow dfferent sub-domans to use dfferent mathematcal models, spatal dscretzatons, local meshes, and seral codes have been proposed by Xng Ca [0]. Boussnesq water wave equatons gven below are used for ths purpose. η + η 6 3.( H + αη) φ + εh H. φ H = 0 ( ) φ α ε φ ε φ + φ. φ + η H. H + H = 0 6 where η and φ are prmary unknowns denotng, respectvely, the water surface elevaton and velocty potental. The water depth H s assumed to be a functon of the spatal coordnates and y. In equatons () and () the weak effect of dsperson and nonlnearty s controlled by the two dmensonless constants ε and α respectvely. The wdely used lnear shallow water equatons can be derved by choosng ε = α = 0. ( ) December 009 The Internatonal Journal on Advances n ICT for Emergng Regons 0

3 K.Ganeshamoorthy, D.N.Ranasnghe, K.P.M.K.Slva and R.Wat 5 η +. ( H φ) = 0 ( 3) φ + η = 0 ( 4) The equatons from () to (4) are used to model the tsunam. The equatons () and () are resolved by unstructured meshes and fnte element dscretzaton, whereas structured meshes and fnte dfferences are commonly used for equatons (3) and (4). Such a parallelzaton strategy s most easly realzed by usng sub-domans, such that the entre spatal doman Ω s decomposed nto a set of overlappng sub-domans {Ω s } p In generc s=. settng, where a partal dfferental equaton (PDE) s epressed as L Ω (u)=f Ω, the Schwarz algorthm conssts of an teratve processes generatng u 0, u, u,..., u k as a seres of appromate solutons. Durng Schwarz teraton k, each sub-doman frst ndependenly updates ts local soluton through LΩ ( U k ) = k s fω ( 5) s s k where f Ω refers to a rght-hand sde whch s s restrcted wthn Ω s and depends on the latest global appromaton u k-. Then, the new global soluton u k s composed by sewng together the sub-doman local solutons u k, uk, uk,..., 3 uk p Equaton (5) thus opens for the possblty of usng dfferent local solvers n dfferent sub-domans. Takng the dea of addtve Schwarz one step further, dfferent mathematcal models n dfferent subdomans can be appled. Therefore, dfferent seral codes may be deployed regon wse. The proposed hybrd parallel tsunam smulator s mplemented usng obect-orented technques and s based on an estng advanced C++ fnte element solver named class Boussnesq applcable for unstructured meshes, and a legacy F77 fnte dfference code applcable for unform meshes. The resultng hybrd parallel tsunam smulator thus has full fleblty and etensblty [0]. B. Parallel computaton of a hghly nonlnear Boussnesq equaton model through doman decomposton Applcatons of the Boussnesq equatons cover a broad spectrum of ocean and coastal problems of nterest, from wnd wave propagaton n ntermedate and shallow water depths to the study of tsunam wave propagaton across large ocean basns. In general, mplementatons of the Boussnesq wave model to calculate free surface wave evoluton n large basns are computatonally ntensve, requrng large amount of CPU tme and memory. To facltate such etensve computatons, a parallel Boussnesq model has been developed by the Kharl et al. [], usng the doman decomposton technque n conuncton wth MPI. The parallel Boussnesq model developed s based on ts seral counterpart. The governng equatons consst of the two-dmensonal depth-ntegrated contnuty equaton: H + 6.( Hu ) ì. H ( ç çh + h ) z S + ( ç h) z T 0 ( 6) and the horzontal momentum equaton: u + η + where S =.u, T =.(hu ) + /, h s depth, η s free surface elevaton. Equatons (6) and (7) dffer from the equatons gven by We et al. [4] n the ncluson of the tme dervatves of the depth (h, h )to account for temporal bottom profle changes that occur durng landslde/earthquake, whch s one of several possble sources of tsunam. The parallel approach has had three mportant aspects, doman decomposton, communcaton, and parallel solver of the trdagonal system of the smultaneous lnear equatons. Three dfferent doman szes have been consdered:( ), ( ), and ( ). The overall performance of the model has been very good. The effcency of the model decreases as the number of ( )and ( ) processors ncreases whch s apparent n the case of domans. The rate of the effcency decrease s faster for smaller doman. Ths s due to the rato of arthmetc operaton tme to communcaton tme decreasng faster for domans wth smaller number of nodes. The performance of the model mproves as the number of grds ncreases; a favourable feature of a parallel model whch s ntended for smulaton on ever-ncreasng doman szes. Thus, ths parallel model provdes a future opportunty for large waveresolvng smulatons n the near shore, wth global domans of many mllons of grd ponts, coverng O(00km ) and greater basns. Further, real-tme smulaton wth Boussnesq equatons becomes a possblty. C. Implct Parallel FEM Analyss of Shallow Water Equatons Jang Chunbo et al. [5] have solved the shallow water equatons (SWEs) as the governng equaton to model a rver flow. SWEs are mplemented on clustered workstatons. For the parallel computaton, the mesh s automatcally parttoned usng the = ( u. u ) + g η + z S + z T η S + ηt ( T + ηs) + ( z η)( u. ) T + ( z η )( u. ) S + ( T + ηs) = 0 ( 7) December 009 The Internatonal Journal on Advances n ICT for Emergng Regons 0

4 6 K.Ganeshamoorthy, D.N.Ranasnghe, K.P.M.K.Slva and R.Wat geometrc mesh parttonng method. The governng equatons are then dscretzed mplctly to form a large sparse lnear system, whch s solved usng a drect parallel generalze mnmum resdual algorthm (GMRES). The shallow water equatons (8) and (9) used here are obtaned by ntegratng the conservaton of mass and momentum equatons assumng a hydrostatc pressure dstrbuton n the vertcal drecton. η q + = 0 (8) q q η + q = gh + H ρ where η s the water elevaton, H s the water depth, q = uh, u s the mean horzontal velocty, g s the gravtatonal acceleraton, v s the eddy vscosty, ρ s the water densty, τ s s the surface shear stress, and τ b s the bottom shear stress. The fnte element method for trangular elements s used for the spatal dscretzaton. Three knds of fnte element meshes are used to smulate the velocty feld n the two numercal eamples, flow around a crcular cylnder and flow n the Yangtze Rver. MPI has been used for the communcaton between nodes. The computed results agree well wth the observed results. The good speedup and effcency for the parallel computaton show that the parallel computng technque s a good method to solve large-scale problems [5]. III. q s b ( ) q τ τ + v +,, =, (9) Lnear Long Wave Theory There are many dfferent numercal methods for computng shallow water equatons on a sphere. Therefore, a standard test sute of seven problems for evaluatng numercal methods for the shallow water equatons n sphercal geometry was proposed by Wllamson et al [3] and accepted by the modelng communty n order to compare newly proposed methods. The shallow water equatons are wdely used as a prototype to study phenomena lke wave-vorte nteractons that occur n more complcated models of large scale atmosphere/ ocean dynamcs [5]. Consder the sea to be a volume of ncompressble water on a rotatng sphere, wth Corols force, fu. The horzontal coordnates are and, y the vertcal coordnate z, whch s zero at the mean sea surface and postve upwards. The sea bed s located at z = - H, and the surface s located at z = h. The lnear shallow water equatons [6], [7], [8] conssts of the contnuty equaton + [ u( h + H )] + [ v( h + H )] = 0 and the conservaton of horzontal momentum u u u + u + v fv = g (0) () v v v + u + v + fu = g () u and v are the velocty components n - and y- drecton, s the surface elevaton and g s the acceleraton due to gravty. Defnng the equatons n terms of the dscharge flues U = uh, V = vh, leads to dscretzaton that always satsfy the conservaton of mass. Then the conservaton of horzontal momentum can be wrtten as U U + t h H + + v + UV fv = g h + H UV V + fu = g h H y h H ( h + H ) (3) ( h + H ) (4) The fnte dfference appromatons usng centered dfferences n space and a leap-frog tme dscretzaton, are based on a staggered grd correspondng to an Arakawa C-grd [9], [0], wth the contnuty equaton centered on the pont, y, t and the equatons of moton centered k + / on the ponts +/, y, t and, y, t k +/ k respectvely. Wrtng D h + H, and usng upwnd dfferences for the convecton terms to mantan stablty t follows that k k k 3 U, U, U U + +, λ, + λ, + λ3, (5) D k k k D 3 D D, +, +, where wth up-wndng, U 0 < 0 λ = 0, and smlarly for the other terms. The dfference equatons are defned on a rectangular grd n terms of sphercal polar coordnates. An accurate representaton of tsunam runnng up the shore mples a grd spacng of no more that 00 meters n a regon of about 4 km out from the shore. As ths fne grd s not reasonable over the whole ocean a successon of overlappng grds s necessary near the coast. A data decomposton scheme s appled =, = k,, 3,, + λ, =, λ, =, λ 3, = 0 λ λ December 009 The Internatonal Journal on Advances n ICT for Emergng Regons 0

5 K.Ganeshamoorthy, D.N.Ranasnghe, K.P.M.K.Slva and R.Wat 7 for the parallel soluton of the shallow water equatons. In data decomposton, we keep the sequental formulaton of the problem, but dstrbute the data and operatons among the processors. The scalablty of several data decomposton algorthms for fnte dfference atmospherc and ocean models have been analyzed by several authors [], []. Rectangular physcal or computatonal doman The physcal or computatonal doman used n ths paper s rectangular n shape. In the doman decomposton method, the rectangular doman s dvded nto several smaller rectangular subdomans, where the number of sub-domans s equal to the number of processors used. Wth four processors, for eample, there are three possble ways of decomposng the doman nto equal-area parts as depcted n Fg.. The best decomposton depends not only on the archtecture of the system beng used, but more mportantly on the memory lmtatons on each node, especally n commodty clusters, as such, an assgnment lke two by two s recommended Fg.. Three possble ways of decomposng a rectangular doman. The numbers n the sub-domans represent the processor number. Several strateges est wthn the data decomposton paradgm for dvdng domans nto sub-domans. In the two-dmensonal grd, the computatonal doman s decomposed both n and y coordnate drectons. In many cases, the computaton s proportonal to the volume of a sub-doman and the communcaton s proportonal to the surface area. In such cases, a logcal strategy s to partton the doman n such a way that t mnmzes the surface area of each sub-doman relatve to ts volume. Ths keeps the computaton-to-communcaton rato hgh. In ths study, two-dmensonal decomposton s chosen. Ths nvolves assgnng each sub-doman to a processor and solvng the equatons for that subdoman on the respectve processor. Wth the two dmensonal decomposton, no global nformaton s requred at any partcular grd pont and nterprocessor communcatons are requred only at the boundares of the sub-domans. The nner-border of a sub-doman requres the outer-border of the adacent sub-doman durng a tme-step because of the spatal dscretzaton [6], [7], [8]. IV. Algorthm Desgn In our work, the doman decomposton method s used to parallelze the tsunam model. In ths method, the parallel algorthm s very smlar to the seral algorthm wth some addtonal routnes added to facltate the communcaton between processors. Usng ths method, all the processors nvolved n the parallel calculatons bascally perform the same computatonal operatons. The only dfference s n the data beng processed n each processor. Fg. 3. (A) Overvew of the unstructured grd and assgnment of worker nodes to sub-domans. (B) Arrows ndcate the nnerworker communcaton. An mportant aspect n decomposng the doman s the load balancng,.e. all processors should have equal or almost equal amount of data to be processed. If the number of grd ponts s dvsble by the number of processors, the grd ponts n each processor s smply the rato of the number of grd ponts to processors. If t s not, the remander s dstrbuted across the frst m processors, where m s the remander. The load should be balanced n both and y drectons. Fg. 4. Lnear shallow water wave profles at three dfferent tmesteps (ts) calculated usng 48 processors. December 009 The Internatonal Journal on Advances n ICT for Emergng Regons 0

6 8 K.Ganeshamoorthy, D.N.Ranasnghe, K.P.M.K.Slva and R.Wat Two grd schemes, a node (4 3) grd system and a 48 node (8 6) grd system, are used to parallelze the tsunam model wth a doman sze of (900 6). Portons of the domans are assgned to each of the worker nodes, as llustrated n Fg. 3 where each sub-doman s labeled wth a processor (worker) number. Snapshots of the free surface evoluton are shown n Fg. 4. Data for each subdoman s stored wth each processor ncludng water depth, coordnates, and ntal dsturbance. Ths data s read by each of the workers n a preprocessng stage.e., pror to ntatng the tmentegraton loop. Snce the model updates the soluton eplctly usng local data, each processor works ndependently of the others but requres data from neghbourng workers to update soluton along sub-doman boundares. The echange of data between processors occurs several tmes per tme-step. There are two key features to ths echange: Frst, only data along the boundares between processors s echanged. Second, each processor s only requred to communcate wth at most four other processors. The doman decomposton s performed wth ths second feature n mnd to avod communcatons between more than four other processors. Communcaton occurs between two adacent processors durng message passng. In passng the data from one processor to another, an effcent and safe communcaton must be developed. To effcently echange the data between adacent processors, the data s frst stored n a contguous memory pror to eecutng the sendng processes. At the same tme contguous memores of the same sze as used n the sendng processes are created to receve the data from the sendng processes. At ths pont, the data s ready for sendng and recevng processes. TABLE I Run Tme In Seconds (wot wthout thread; wt wth thread) Nodes MPI C/Lnda wot wt wot wt In ths model, message passng occurs four tmes per tme-step and the MPI functon, ``MPI_ Sendrecv, and the C/Lnda operatons n and out are used to perform the message passng between global nodes. Ths corresponds to eght messages per tme- step per processor ndependent of the number of processors beng used. Many more messages are sent durng pre-processng, but these are gnored for run-tme analyss purposes snce tme ntegraton s by far the most tme consumng element of the program. The parallel algorthm s outlned below.. Decompose rectangular doman nto load balanced rectangular sub-domans where the wdth and length of a rectangular sub-doman are W/N w appromately, and L/N l respectvely, where W and L are the wdth and length of doman, and N = N w *N s the number of processors to be used n the (N w N ) grd scheme.. Specfy parallel language related parameters, such as locatons, neghbours, sub-doman szes, and fle names, of processors. Locaton of k th processor s (r k, c k ), where r k = k/gdm + and c k = k - (r k - )* gdm + wth gdm and gdm beng the dmensons of the grd. In the four by three grd scheme, gdm = 3 and gdm = 4, and n the eght by s scheme, gdm=6 and gdm=8. For k = 0,,, 3, 4, , (gdm*gdm -), defne north k, south k, east k, and west k to be the neghbours located n the sde of north, south, east and west of the k th processor, then: north k = k - gdm f r k > south k = k + gdm f r k <gdm east k = k + f c k < gdm west k = k - f c k > 3. Input data and ntal condtons: (a) (b), reads the water depth, coordnates, and ntal dsturbance from the tet fle assgned n step., echanges subdoman boundary data from east k to west k, from west k to east k, from north k to south k, and form south, to northk, n oder. 4. Set parameters and coeffcents used at the open sea boundary. 5. Repeat the followng steps for the pre-defned tme-steps: (a) (b) (c) (d), echanges subdoman boundary data from east k to west k., echanges subdoman boundary data from north k to south k Computaton of the equaton of contnuty. Settng of the open sea boundary condton. December 009 The Internatonal Journal on Advances n ICT for Emergng Regons 0

7 K.Ganeshamoorthy, D.N.Ranasnghe, K.P.M.K.Slva and R.Wat 9 (e) (f), echanges subdoman boundary data from west k to east k., echanges subdoman boundary data from south k to north k. 6. Gather computed results among all processors. 7. Compute the output on processor, where both are equal to null. V. Im p l e m e n tat o n Data s parttoned by sub-domans, where a four by three grd scheme and an eght by s grd scheme are used for the parallel mplementaton of ths model. In each of the grd schemes, there are four parallel varatons: () dstrbuted memory wthout threads; () dstrbuted memory wth threads; (3) vrtual shared memory wthout threads; (4) vrtual shared memory wth threads. Implementatons use the Message Passng Interface (MPI) lbrary [], the C/Lnda [] and the pthread lbrary [3]. The med-mode programmng model whch uses thread programmng n the shared memory layer and message passng programmng n the dstrbuted memory layer s a method commonly used to utlze the herarchy of memory resources effcently [9]. In our med-mode programmng model, MPI s used for the data communcaton between the global nodes, and wthn each MPI process, pthreads are used. In the vrtual shared memory med-mode approach, C/Lnda s used for the data communcaton between the global nodes, and wthn each C/Lnda process, pthreads are used. These four algorthms have been mplemented on the Monolth cluster of Nordugrd [3] whch conssts of a hgh speed backbone nterconnectng mult processor nodes. The Monolth cluster has 396 nodes, all of whch are 686 archtecture wth.0ghz Intel(R) XEON(TM) processors, dual processor nodes and 048MB per node man memory and 5KB cache. The operatng system s Lnu verson.4.34-cap-smp. VI. Eva l u at o n Table I shows the tmng values for the eght scenaros arsng from the four parallel varatons of the algorthm mentoned above. Consder the four by three grd scheme, wth both threadng and non-threadng. The parallel algorthms for the vrtual shared memory ehbt better performance than the algorthms for dstrbuted memory. Though the vrtual shared memory mplctly passes messages, the replcaton management subsystem has been optmzed n C/Lnda compared to MPI [0], to yeld better performance. In the four by three grd scheme, n both scenaros for dstrbuted and vrtual shared memory, nonthreadng algorthms ehbt better performance than threadng algorthms. One possble eplanaton for ths s that each node keeps nne, two dmensonal floatng pont type arrays of sze equal to the sub doman sze. Because of the memory lmtatons t s not possble to declare local two-dmensonal arrays for threads, causng all threads n a node to concurrently use global arrays for ther own computatons. Now consder the eght by s grd scheme. Here too, for both threadng and non-threadng, the parallel algorthms for the vrtual shared memory ehbt better performance than the algorthms for dstrbuted memory. In contrast to the prevous nstance, both algorthms for dstrbuted and vrtual shared memory, the threadng algorthms ehbt better performance than non-threadng algorthms. Ths s because that the sub doman sze allocated to each node s half sze of sub doman sze of the smaller grd scheme of four by three sze. The local two dmensonal floatng pont type array allocaton for threads n a node s now possble compared to the four by three grd system. Therefore, snce all threads of a node work ndependently, the threadng parallel algorthm shows better performance than non-threadng parallel algorthm. Among the parallel varatons not usng threads n both memory archtectures, the four by three grd scheme shows better performance than eght by s grd scheme. Ths s due to the rato of computaton tme to communcaton overhead decreasng faster for domans wth smaller sze. However, when the task s scaled up, say up to eght by s grd system, owng to the smaller sub-doman szes algnng wth the avalable memory n the node, the threaded versons become more effcent n both MPI and C/Lnda mplementatons. In both threadng and non-threadng envronments, the C/Lnda verson ehbts better performance than the MPI verson. VII. Co n c l u s o n Ths paper has presented eght dfferent parallel mplementatons of a tsunam model based on the shallow water equatons. Each of these mplementatons use a med-mode programmng model from thread based shared memory, to December 009 The Internatonal Journal on Advances n ICT for Emergng Regons 0

8 0 K.Ganeshamoorthy, D.N.Ranasnghe, K.P.M.K.Slva and R.Wat dstrbuted memory and fnally to vrtual shared memory. Owng to node memory lmtatons, scalablty ssues become paramount, and threadng becomes a sgnfcant bottleneck f suffcent node memory s not avalable, offsettng the mddleware advantages. Wth suffcent node memory however, C/Lnda wth threads outperforms MPI wth threads, ndcatng the effectveness of etractng parallelsm over vrtual shared memory, dstrbuted memory and shared memory multple levels for ths class of problems. Ac k n o w l e d g m e n t Ths work was conducted at the Department of Computaton and Intellgent Systems, Unversty of Colombo School of Computng. Ths research was performed usng computatonal resources at the Natonal Supercomputer Centre, Lnkopng Unversty, Sweden, and of the computatonal resources of Research Laboratory, Unversty of Colombo School of Computng. Re f e r e n c e s [] Wkpeda [] K. I. Stanggang and P. Lynett, Parallel computaton of a hghly nonlnear Boussnesq equaton model through doman decomposton, Internatonal ournal for numercal methods n fluds, ISSN 07-09, vol. 49, no., 005, pp [3] D. L. Wllamson, J. B. Drake, J. J. Hack, R. Jakob, and P. N. Swarztrauber, A standard Test Set for numercal Appromatons to the Shallow Water Equatons n Sphercal Geometry, Journal of Computatonal Physcs, 0(), pp. -4, 99. [4] T. Vasly, G. Frank, and M. Hal, Tsunam Forecastng and Hazard Assessment Capabltes, Pacfc Dsaster Center (PDC) & Mau Hgh Performance Computng Center (MHPCC) Tsunam Modelng, Natonal Oceanc and Atmospherc Admnstraton Center for Tsunam Research ( 009) [5] J. D. Paul, and S. Rck, Shallow Water Equatons wth a Complete Corols Force and Topography, Physcs of Fluds, September 005. [6] B. Ragnhld, Nested Parallelsm n OpenMP wth Applcaton to Adaptve Mesh Refnement, Unversty of Bergen, February 003. [7] F. S. Brett and C. P. John, Parallel Implementaton of an Eplct Fnte-volume Shallow-water Model, 6th ASCE Engneerng Mechancs Conference, July 6-8, 003, Unversty of Washnton, Seattle. [8] S. L. Johnsson and C. Ho, Algorthms for Matr Transposton on Boolean N-Cube Confgured Ensemble Archtectures, SIAM J. Matr Analyss and Applcaton, vol.9(3), July 988, pp [9] W. Meng-Shou, A. Srnvas, and A. K. Rcky, Med Mode Matr Multplcaton, Proceedngs of the IEEE Internatonal Conference on Cluster Computng, IEEE Computer Socety Washngton, DC, USA, 00. [0] C. Xng and P. L. Hans, Makng Hybrd Tsunam Smulators n a Parallel Software Framework, LNCS, Appled Parallel Computng. State of the Art n Scentfc Computng, vol. 4699/008, pp [] MPI Forum mcs.anl.gov/mp/, 009. [] Lnda User Gude, Scentfc Computng Assocates IC., One Century Tower, 65 Church Street, New haven, CT USA, September, 005. ( about/nde.html, 009) [3] Pthreads. threads.html, 009. [4] G. We, J. T. Krby, S. T. Grll, R. Subramanya, A fully nonlnear Boussnesq model for surface waves, Part I, Hghly nonlnear unsteady waves, Journal Flud Mechancs995; 94:7-9. [5] J. Chunbo, L. Ka, L. Nng, and Z. Qngha, Implct Parallel FEM Analyss of Shallow Water Equatons, Tsnghua Scence & Technology, vol. 0, Issue 3, June 005, pp [6] atmos_spectral_shallow/shallow.pdf, 009. [7] [8] [9] Arakawa A and Lamb V, Computatonal desgn of the basc dynamcal processes of the UCLA general crculaton model, Methods n Computatonal Physcs, vol. 7, Academc Press, 977, pp [0] C. Goto and Y. Ogawa, Dept. of Cvl Engneerng, Tohoku Unversty, Translated for the Tsunam Inumdaton Modellng Echange Proect, by N. Shuto. Numercal method of tsunam smulaton wth the leap-frog scheme, UNESCO Intergovernmental Oceangraphc Commsson, Manuals and Gudes, 35, 99. [] S. Roar, Scalablty of Parallel Grd Pont Lmted Area Atmospherc Models I & II, Manuscrpt, Department of Mathematcs Scences, Norwegan Unversty of Scence and Technology, Trodhem, Norway, 996. [] S. Thomas, J. Cote, A. Stanforth, I. Le, and R. Skaln, A Sem-Implct Sem-Lagrange Shallow-Water Model for Massvely Parallel Processors, Proceedngs of the 6th ECMWF Workshop on the Use of Parallel Processors n Meteorology, November 994, pp [3] The Grd mddleware proect n the Nordc countres. December 009 The Internatonal Journal on Advances n ICT for Emergng Regons 0