a tree-dmensonal settng. In te presented approac, we construct te ne mes by renng an exstng coarse mes and updatng te nodes of te ne mes accordng to t

Size: px

Start display at page:

Download "a tree-dmensonal settng. In te presented approac, we construct te ne mes by renng an exstng coarse mes and updatng te nodes of te ne mes accordng to t"

Charla Morris
5 years ago
Views:

1 Parallel Two-Level Metods for Tree-Dmensonal Transonc Compressble Flow Smulatons on Unstructured Meses R. Atbayev a, X.-C. Ca a, and M. Parascvou b a Department of Computer Scence, Unversty of Colorado, Boulder, CO 80309, U.S.A., frakm, cag@cs.colorado.edu b Department ofmecancal and Industral Engneerng, Unversty oftoronto, Toronto, Canada, M5S 3G8, marus@me.utoronto.ca We dscuss our prelmnary experences wt several parallel two-level addtve Scwarz type doman decomposton metods for te smulaton of tree-dmensonal transonc compressble ows. Te focus s on te mplementaton of te parallel coarse mes solver wc s used to reduce te computatonal cost and speed up te convergence of te lnear algebrac solvers. Results of a local two-level and a global two-level algortm on a multprocessor computer wll be presented for computng steady ows around a NACA0012 arfol usng te Euler equatons dscretzed on unstructured meses. 1. INTRODUCTION We are nterested n te numercal smulaton of tree-dmensonal nvscd steady-state compressble ows usng two-level Scwarz type doman decomposton algortms. Te class of overlappng Scwarz metods as been studed extensvely n te lterature [11], especally, te sngle level verson of te metod [6,9]. It s well-known, at least n teory, tat te coarse space plays a very mportant role n te fast and scalable convergence of te algortms. Drect metods are often used to solve te coarse mes problem eter redundantly on all processors or on a subset of processors [3]. Ts presents a major dculty n a fully parallel mplementaton for 3D problems, especally wen te number of processors s large. In ts paper, we propose several tecnques to solve te coarse mes problem n parallel, togeter wt te local ne mes problems, usng two nested layers of precondtoned teratve metods. Te constructon of te coarse mes s an nterestng ssue by tself. We take a derent approac tan wat s commonly used n te algebrac multgrd metods n wc te coarse mes s obtaned from te gven ne mes { not te gven geometry. In our twolevel metods to be presented n ts paper, we construct bot te coarse and te ne mes from te gven geometry. To better t te boundary geometry, tenemesnodes may not be on te faces of te coarse mes tetraedrons. In oter words, te coarse space and ne space are not nested. Ts does not present a problem as long as te proper nterpolaton s dened [2]. As a test case, we consder a symmetrc nonlftng ow over a NACA0012 arfol n

2 a tree-dmensonal settng. In te presented approac, we construct te ne mes by renng an exstng coarse mes and updatng te nodes of te ne mes accordng to te boundary geometry of te gven pyscal doman. Suc approac s easy to mplement snce te same computer code can be used on bot te ne and te coarse level, and only a mnmal addtonal programmng s requred to construct te restrcton and prolongaton operators. Moreover,tgves a natural partton of te ne mes from te partton of te coarse mes. In te tests, te system of Euler equatons s dscretzed usng te backward derence approxmaton n te pseudo-temporal varable and a nte volume metod n te spatal varables. Te resultng system of nonlnear algebrac equatons s lnearzed usng te Defect Correcton (DeC) sceme. At eac pseudo-temporal level, te lnear system s solved by a restrcted addtve Scwarz precondtoned FGMRES metod [10], and te coarse mes problem s solved wt an nner level of restrcted addtve Scwarz precondtoned FGMRES metod. 2. GOVERNING EQUATIONS AND BOUNDARY CONDITIONS Let R 3 be a bounded ow doman wt te boundary consstng of two parts: a wall boundary ; w and an nnty boundary ; 1. Let be te densty, ~u (u v w) T te velocty vector, e te total energy per unt volume, and p te pressure. We consder, ~u, e and p as te unknowns at pont (x, y, z), and te pseudo-temporal varable t. Set U ( u v w e) T @x : An nvscd compressble ow ns descrbed by te Euler equatons U t + ~ r ~F 0 were F ~ (F G H) T s te ux vector wt te Cartesan components dened as on page 87 of [7]. Te equaton (1) s closed by te equaton of state for a perfect gas p ( ;1) (e ; k~uk 22), were s te rato of specc eats and kk 2 2 s te 2-norm n R 3.We specfy te ntal condton Uj t0 U 0, were U 0 s an ntal approxmaton to a steady-state soluton, and te followng boundary condtons. On te wall boundary ; w, we mpose a no-slp condton for te velocty ~u ~n 0, were ~n s te outward normal vector to te wall boundary. On te nnty boundary ; 1,we mpose unform free stream condtons 1, ~u ~u 1,andp 1 1(M1 2 ), were M 1 s te free stream Mac number. We seek a steady-state soluton, tat s, te lmts of, ~u, e and p as t!1. 3. DISCRETIZATION In ts secton, we present an outlne of te dscretzaton of te Euler equatons for more detals, see [5]. Let be a tetraedral mes n, and N be te number of mes ponts. For te pseudo-temporal dscretzaton, we use a rst-order backward derence sceme. For te spatal dscretzaton of (1), we use a nte volume sceme n wc control volumes are centered on te vertces of te mes. For upwndng, we use Roe's approxmate Remann solver wc as te rst order spatal accuracy. Second order accuracy s aceved by te MUSCL tecnque [13] wc uses pecewse lnear nterpolaton at te nterface between control volumes. For 1 2 ::: N and n 0 1 :::,letu n denote te value of te dscrete soluton at pont (x y z ) and at te pseudo-temporal level n and set U n (U n 1 U n 2 ::: UN) n T. (1)

3 Let U 0 U 0 (x y z )and (U )( 1 (U ) ::: N (U )) T, were (U ) denotes te descrbed second order approxmaton of convectve uxes r ~ F ~ at pont (x y z ). We dene te local tme step sze by t n C CFL (C + k~u n k 2 ) were C CFL > 0 s a preselected number, s a control volume centered at node, s ts caracterstc sze, C s te sound speed and ~u n s te velocty vector at node. Ten, te proposed sceme as a general form (U n+1 ; U n )tn + (U n+1 ) ::: N n 0 1 ::: : (2) We note, te nte volume sceme (2) as te rst order approxmaton n te pseudotemporal varable and te second order approxmaton n te spatal varable. On ; w, no-slp boundary condton s enforced. On ; 1, a non-reectveverson of te ux splttng of Steger and Warmng [12] s used. We apply a DeC-Krylov-Scwarz type metod to solve (2) tat s, we use te Defect Correcton sceme as a nonlnear solver, te restarted FGMRES algortm as a lnear solver, and te restrcted addtve Scwarz algortm as te precondtoner. At eac pseudo-temporal level n, te equaton (2) represents a system of nonlnear equatons for te unknown varable U n+1. Ts nonlnear system s lnearzed by te Defect Correcton (DeC) sceme [1] formulated as follows. Let ~ (U ) be te rst-order approxmaton of convectve uxes r ~ F ~ obtaned n way smlar to tat of (U ), and ~ (U ) denote ts Jacoban. Suppose tat, for xed n, an ntal guess U n+1 0 s gven (say U n+1 0 U n ). For s 0 1 :::, solve foru n+1 s+1 te followng lnear system D n ~ U n+1 0 U n+1 s+1 ; U n+1 s ;D n U n+1 s ; U n ; U n+1 s were D n dag (1tn ::: 1 1tn N ) s a dagonal matrx. Te DeC sceme (3) preserves te second-order approxmaton n te spatal varable of (2). In our mplementaton, we carry out only one DeC teraton at eac pseudo-temporal teraton, tat s, we use te sceme D n ~ ~U n ~U n+1 ; U ~ n ; ~U n n 0 1 ::: U ~ 0 U 0 : (3) 4. LINEAR SOLVER AND PRECONDITIONING Let te nonlnear teraton n be xed and denote A D n ~ ~ U n : (4) Matrx A s nonsymmetrc and ndente n general. To solve (4), we use two nested levels of restarted FGMRES metods [10] one at te ne mes level and one at te coarse mes level nsde te addtve Scwarz precondtoner (AS) to be dscussed below.

4 4.1. One-level AS precondtoner To accelerate te convergence of lnear teratons n te FGMRES algortm, we use an addtve Scwarz precondtoner. Te metod splts te orgnal lnear system nto a collecton of ndependent smaller lnear systems wc could be solved n parallel. Let be subdvded nto k non-overlappng subregons 1, 2, :::, k. Let 0 1, 0, ::: 2,0 be overlappng extensons of k 1, 2, :::, k, respectvely, andbe also subsets of. Te sze of overlap s assumed to be small, usually one mes layer. Te node orderng n determnes te node orderngs n te extended subregons. For 1 2 ::: k,letr be a global-to-local restrcton matrx tat corresponds to te extended subregon 0, and let A be a \part" of matrx A tat corresponds to 0.Te AS precondtoner s dened by AS 1 R T A ;1 R : For certan matrces arsng from te dscretzatons of ellptc partal derental operators, an AS precondtoner s spectrally equvalent to te matrx of a lnear system wt te equvalence constants ndependent of te mes step sze, altoug, te lower spectral equvalence constant as a factor 1H, were H s te subdoman sze. For some problems, addng a coarse space to te AS precondtoner removes te dependency on 1H, ence, te number of subdomans [11] One-level RAS precondtoner It s easy to see tat, n a dstrbuted memory mplementaton, multplcatons by matrces R T and R nvolve communcaton overeads between negborng subregons. It was recently observed [4] tat a slgt modcaton of R T allows to save alf of suc communcatons. Moreover, te resultng precondtoner, called te restrcted AS (RAS) precondtoner, provdes faster tan te orgnal AS precondtoner convergence for some problems. Te RAS precondtoner as te form RAS R 0T A ;1 1 R were R 0 T corresponds to te extrapolaton from. Snce t s too costly to solve lnear systems wt matrces A,we use te followng modcaton of te RAS precondtoner: 1 R 0T B ;1 1 R were B corresponds to te ILU(0) decomposton of A.We call M 1 te one-level RAS precondtoner (ILU(0) moded) Two-level RAS precondtoners Let H be a coarse mes n, and let R 0 be a ne-to-coarse restrcton matrx. Let A 0 be a coarse mes verson of matrx A dened by (4). Addng a scaled coarse mes component to(5),we obtan 2 (1 ; ) 1 R 0 T B ;1 R + R T 0 A ;1 0 R 0 (6) (5)

5 were 2 [0 1] s a scalng parameter. We call M 2 te global two-level RAS precondtoner (ILU(0) moded). Precondtonng by M 2 requres solvng a lnear system wt matrx A 0,wc s stll computatonally costly f te lnear system s solved drectly and redundantly. In fact, te approxmaton to te coarse mes soluton could be sucent for a better precondtonng. Terefore, we solve te coarse mes problem n parallel usng agan a restarted FGMRES algortm, wc we call te coarse mes FGMRES, wta moded RAS precondtoner. Let H be dvded nto k subregons H 1, H 2, :::, H k wt te extented counterparts 0 H 1, 0 H 2, :::, 0 H k. To solve te coarse mes problem, we usefgmres wt te onelevel ILU(0) moded RAS precondtoner (R0 ) 0 T B ;1 0 R 0 (7) were, for 1 2 ::: N, R 0 s a global-to-local coarse mes restrcton matrx, (R0 ) 0 T s a matrx tat corresponds to te extrapolaton from H, and B 0 s te ILU(0) decomposton of matrx A 0,apartofA 0 tat corresponds to te subregon 0.After H r coarse mes FGMRES teratons, A ;1 0 n (6) s approxmated by A ~ ;1 0 poly l ( 0 1 A 0 ) wt some l r, were poly l (x) s a polynomal of degree l, and ts explct form s often not known. We note, l maybe derent at derent ne mes FGMRES teratons, and t depends on a stoppng condton. Terefore, FGMRES s more approprate tan te regular GMRES. Tus, te actual precondtoner for A as te form ~ 2 (1 ; ) 1 R 0 T B ;1 T R + R 0 ~A ;1 0 R 0 : (8) For te ne mes lnear system, we also use a precondtoner obtaned by replacng A ;1 0 n (6) wt 0 1 dened by (7): 3 1 (1 ; ) R 0 T B ;1 R + R T 0 (R 0 0 )T B ;1 R 0 0 R 0 : (9) We call M 3 a local two-level RAS precondtoner (ILU(0) moded) snce te coarse mes problems are solved locally, and tere s no global nformaton excange among te subregons. We expect tat M 3 works better tan M 1 and tat ~ M 2 does better tan M 3. Snce no teoretcal results are avalable at te present, we test te descrbed precondtoners M 1, ~ M2,andM 3 numercally. 5. NUMERICAL EXPERIMENTS We computed a compressble ow over a NACA0012 arfol on te computatonal doman wt te nonnested coarse and ne meses. Frst, we constructed an unstructured coarse mes H ten, te ne mes was obtaned by renng te coarse mes twce. At eac renement step, eac coarse mes tetraedron was subdvded nto 8 tetraedrons. After eac renement, te boundary nodes of te ne mes were adjusted to te geometry of te doman. Szes of te coarse and ne meses are gven n Table 1.

6 Table 1 Coarse and ne mes szes Coarse Fne Fne/coarse rato Nodes 2, , Tetraedrons 9, , Nonlnear resdual level RAS local 2-level RAS 10-1 global 2-level RAS Total number of lnear teratons Number of lnear teratons level RAS local 2-level RAS global 2-level RAS Nonlnear teraton Fgure 1. Comparson of te one-level, local two-level, and global two-level RAS precondtoners n terms of te total numbers of lnear teratons(left pcture) and nonlnear teratons (rgt pcture). Te mes as 32 subregons. For parallel processng, te coarse mes was dvded, usng METIS [8], nto 16 or 32 submeses wt nearly te same number of tetraedrons. Te ne mes partton was obtaned drectly from te correspondng coarse mes partton. Te sze of overlap bot n te coarse and te ne mes partton was set to one, tat s, two negborng extended subregons sare a sngle layer of tetraedrons. In (8) and (9), R T 0 was set to a matrx of a pecewse lnear nterpolaton. Multplcatons by R T 0 and R 0, solvng lnear systems wt M 1, M2 ~,andm 3, and bot te ne and te coarse FGMRES algortm were mplemented n parallel. Te experments were carred out on an IBM SP2. We tested convergence propertes of te precondtoners dened n (5), (8), and (9) wt N c N f, were N c and N f are te numbers of nodes n te coarse and ne meses, respectvely. We studed a transonc case wt M 1 set to 0:8. Some of te computaton results are presented n Fgures 1 and 2. Te left pcture n Fgure 1 sows resdual reducton n terms of total numbers of lnear teratons. We see tat te algortms wt two-level RAS precondtoners gve sgncant mprovements compared to te algortm wt te one-level RAS precondtoner. Te mprovement n usng te global two-level RAS precondtoner compared to te local twolevel RAS precondtoner s not very muc. Recall, tat n te former case te nner FGMRES s used wc could ncrease te CPU tme. In Table 2, we present a summary from te gure. We see tat te reducton percentages n te numbers of lnear teratons drop wt te decrease of te nonlnear resdual (or wt te ncrease of te nonlnear teraton number). Ts s seen even more clear n te rgt pcture n Fgure 1. After

7 Table 2 Total numbers of lnear teratons and te reducton percentages compared to te algortm wt te one-level RAS precondtoner (32 subregons). One-level RAS Local two-level RAS Global two-level RAS Resdual Iteratons Iteratons Reducton Iteratons Reducton 10 ; % % 10 ;4 1, % % 10 ;6 1,953 1,397 28% 1,245 36% 10 ;8 2,452 1,887 23% 1,758 28% Nonlnear resdual 1-level RAS / 16 subregons level RAS / 32 subregons local 2-level RAS / 16 subregons 10-1 local 2-level RAS / 32 subregons Total number of lnear teratons Nonlnear resdual 1-level RAS / 16 subregons level RAS / 32 subregons global 2-level RAS / 16 subregons 10-1 global 2-level RAS / 32 subregons Total number of lnear teratons Fgure 2. Comparson of te one-level RAS precondtoner wt te local two-level RAS (left pcture) and te global two-level RAS precondtoner (rgt pcture) on te meses wt 16 and 32 subregons. approxmately 80 nonlnear teratons, te tree algortms gve bascally te same number of lnear teratons at eac nonlnear teraton. Ts suggests tat te coarse mes may not be needed after some number of ntal nonlnear teratons. In Fgure 2, we compare te algortms on te meses wt derent numbers of subregons, 16 and 32. Te left pcture sows tat te algortms wt te one-level and local two-level RAS precondtoners ntally ncrease te total numbers of lnear teratons as te number of subregons was ncreased from 16 to 32. On te oter and, we see n te rgt pcture n Fgure 2 tat te te ncrease n te number of subregons almost dd not aect te convergence of te algortm wt te global two-level RAS precondtoner. Tese results suggest tat te algortm wt te global two-level RAS precondtoner s well scalable to te number of subregons (processors) wle te oter two are not. In bot pctures we observe te decrease n te total number of lnear teratons to te end of computatons. Ts s due to te fact tat only 4 or 5 lnear teratons were carred out at eac nonlnear teraton n bot cases, wt 16 and 32 subregons (see te rgt pcture n Fgure 1), wt lnear systems n te case of 32 subregons solved just one teraton faster tan te lnear systems n te case of 16 subregons.

8 6. CONCLUSIONS Wen bot te ne and te coarse mes s constructed from te doman geometry, t s farly easy to ncorporate a coarse mes component nto a one-level RAS precondtoner. Te applcatons of te two-level RAS precondtoners gve a sgncant reducton n total numbers of lnear teratons. For our test cases, te coarse mes component seems not needed after some ntal number of nonlner teratons. Te algortm wt te global two-level RAS precondtoner s scalable to te number of subregons (processors). Szes of ne and coarse meses sould be well balanced, tat s, f a coarse mes s not coarse enoug, te applcaton of a coarse mes component could result n te CPU tme ncrease. REFERENCES 1. K. Bomer, P. Hemker, and H. Stetter, Te defect correcton approac, Comput. Suppl, 5 (1985), pp. 1{ X.-C. Ca, Te use of pontwse nterpolaton n doman decomposton metods wt non-nested meses, SIAM J. Sc. Comput., 16 (1995), pp. 250{ X.-C. Ca, W. D. Gropp, D. E. Keyes, R. G. Melvn, and D. P. Young, Parallel Newton-Krylov-Scwarz algortms for te transonc full potental equaton, SIAM J. Sc. Comput., 19 (1998), pp. 246{ X.-C. Ca and M. Sarks, Arestrcted addtve scwarz precondtoner for general sparse lnear systems, SIAM J. Sc. Comput., (1999). To appear. 5. C. Farat and S. Lanter, Smulaton of compressble vscouse ows on a varety of MPPs: computatonal algortms for unstructured dynamc meses and performance results, Comput. Metods Appl. Mec. Engrg., 119 (1994), pp. 35{ W. D. Gropp, D. E. Keyes, L. C. Mcnnes, and M. D. Tdrr, Globalzed Newton{Krylov{Scwarz algortms and software for parallel mplct CFD, Int. J. Hg Performance Computng Applcatons, (1999). Submtted. 7. C. Hrsc, Numercal Computaton of Internal and External Flows, Jon Wley and Sons, New York, G. Karyps and V. Kumar, A fast and g qualty multlevel sceme for parttonng rregular graps, SIAM J. Sc. Comput., 20 (1998), pp. 359{ D. K. Kausk, D. E. Keyes, and B. F. Smt, Newton{Krylov{Scwarz metods for aerodynamcs problems: Compressble and ncompressble ows on unstructured grds, n Proc. of te Elevent Intl. Conference on Doman Decomposton Metods n Scentc and Engneerng Computng, To appear. 10. Y. Saad, A exble nner-outer precondtoned GMRES algortm, SIAM J. Sc. Stat. Comput., 14 (1993), pp. 461{ B. F. Smt, P. E. Bjrstad, and W. D. Gropp, Doman Decomposton: Parallel Multlevel Metods for Ellptc Partal Derental Equatons, Cambrdge Unversty Press, J. Steger and R. F. Warmng, Flux vector splttng for te nvscd gas dynamc wt applcatons to nte-derence metods, J. Comp. Pys., 40 (1981), pp. 263{ B. Van Leer, Towards te ultmate conservatve derence sceme V: a second order sequel to Godunov's metod, J. Comp. Pys., 32 (1979), pp. 361{370.

. Introducton. The system of unsteady compressble Naver-Stokes (N.-S.) equatons s a fundamental system n ud dynamcs. To be able to solve the system qu

. Introducton. The system of unsteady compressble Naver-Stokes (N.-S.) equatons s a fundamental system n ud dynamcs. To be able to solve the system qu VARIABLE DEGREE SCHWARZ METHODS FOR THE IMPLICIT SOLUTION OF UNSTEADY COMPRESSIBLE NAVIER-STOKES EQUATIONS ON TWO-DIMENSIONAL UNSTRUCTURED MESHES Xao-Chuan Ca y Department of Computer Scence Unversty of