τ-extrapolation on 3D semi-structured finite element meshes

Transcription

1 τ-extrapolation on 3D semi-structured finite element meshes European Multi-Grid Conference EMG 2010 Björn Gmeiner Joint work with: Tobias Gradl, Ulrich Rüde September, 2010

2 Contents The HHG Framework τ-extrapolation Scalability 2

3 The HHG Framework 3

4 Combining finite element and multigrid methods FE mesh may be unstructured. What nodes to remove for coarsening? Not straightforward! Why not start from the coarse grid? The Hierarchical Hybrid Grids (HHG) concept Benjamin Bergen*: prototype Tobias Gradl: tuning, extensions and adaptivity * Dissertation in Erlangen, ISC award in Currently at Los Alamos Labs. 4

5 Advantages Properties of the HHG approach Multigrid is straightforward Very memory efficient Massive performance benefits on current computer architectures Subserves parallelization unknowns are possible Limitation Coarse input grid needed Adaptivity (ongoing work by Tobias Gradl) 5

6 τ-extrapolation 6

7 Overview Achieving higher accuracy by the combination of two grids and error expension 1 For 2-D unstructured triangular meshes it was shown: One step of multigrid τ-extrapolation for piecewise linear C 0 finite element methods is equivalent to using quadratic elements 2 A lack of regularity causes larger discretization errors at the interfaces between two structured regions 3 How severe are these discretization errors on semi-structured meshes in praxis? 1 A. Brandt, Multigrid Techniques: 1984 Guide with Applications to Fluid Dynamics, GMD-Studien, Bonn (1984) 2 M. Jung, U. Rüde, Implicit Extrapolation Methods for Multilevel Finite Element Computations, SIAM J. Sci. Comput., Vol 17 (1996) 3 H. Blum, Asymptotic Error Expansion and Defect Correction in the Finite Element Method, Institut für Angewandte Mathematik, Heidelberg (1990) 7

8 Motivation: The (local) truncation error PDE with a differential operator N: N(u) = f Restriction to a grid by restriction R: R(N(u)) = R(f ) Adding a discretized differential operator applied to the solution N k (ˆR(u)): N k (ˆR(u)) = R(f ) + N k (ˆR(u)) R(N(u)) }{{} local truncation error τ h Can we estimate the local truncation error in order to improve our approximation? 8

9 Full Approximation Storage Formulation Solve the coarse grid equation N k 1 (w k 1 ) = R(f k N k (u k )) }{{} defect d k 1 +N k 1 (ˆR(u k )). and correct the solution on the fine grid by (w k 1 ˆR(u k )). Rearranging the right hand side yields N k 1 (w k 1 ) = R(f k ) + N k 1 (ˆR(u k )) R(N k (u k )). }{{} relative trunction error τh 2h Using τh 2h as an approximation of the local truncation error τ h = N k 1 (ˆR(u)) R(N(u)) for the coarse grid. 9

10 τ-extrapolation on Structured Tetrahedral Grids Correction Scheme τ extrapolation Discretization Error Regular Tetraeder Level of Refinement 10

11 Two structured regions connected by its faces I I 3 I Problem: u = sinh(x)cos(y)cos(z) 11

12 Correction Scheme on a line 0 x x Discretization Error Discretization Error x Axis x Axis I I 3 I 12

13 τ-extrapolation on a line 0.5 x x Discretization Error Discretization Error x Axis x Axis I I 3 I 13

14 Numerical Experiments on the European Supercomputer Blue Gene/P in Jülich (Jugene) Compute node: 4-way SMP processor Processortype: 32-bit PowerPC 450 core 850 MHz Processors: Overall peak performance: 1 Petaflops Main memory: 2 Gbytes per node (aggregate 144 TB) 14

15 A complexer geometry (coarsest mesh) Top view Side view 15

16 Discretization Errors Struct. Regions: Utilized number of cores: Problem: u = sin(x)sin(y)sin(z) Smoother: µ(6, 0) GS Coarse grid solver: CG 16

17 Struct. Regions: Discretization Errors Utilized number of cores: Problem: u = sin(x)sin(y)sin(z) Smoother: µ(6, 0) GS Coarse grid solver: CG Levels Unknowns Correction Scheme τ-extrapolation

18 Scalability 17

19 Scalability of HHG on Blue Gene/P (Jugene) Speedup normalized to 512 cores strong scaling ideal Cores Figure: Strong Scaling behavior of HHG on PowerPC 450 cores of a Blue Gene/P at Jülich. This test case was performed solving unknowns. 18

20 E-Scalability Multigrid is optimal but not E-scalable! This means the parallel efficiency decreases with increasing number of processors, while keeping the number of grid points per processor constant. :-( The problem: What to do with the coarsest grids? A simple strategy: Truncate the coarsest grids and then just approximate on the coarsest grid! But is it applicable for very many processors and thus quite fine coarsest grids?!? 19

21 Scalability of HHG on Blue Gene/P (Jugene) Cores Struct. Regions Unknowns CG Time

22 Scalability of HHG on Blue Gene/P (Jugene) Cores Struct. Regions Unknowns CG Time

23 Scalability of HHG on Blue Gene/P (Jugene) Cores Struct. Regions Unknowns CG Time

24 Scalability of HHG on Blue Gene/P (Jugene) Cores Struct. Regions Unknowns CG Time

25 Observations τ-extrapolation seems to work on 3d structured FE meshes We see no limitation for the scalability of geometric MG on FE up to 262k cores Outlook Understanding the effects of τ-extrapolation at the faces, edges and vertices on semi-structured meshes Finding a solution to keep O(h 4 ) order consistency with increasing size of the structured regions 21

26 Thank you for you attention! Any questions? The development of HHG was funded by the Elite Network of Bavaria within the International Doctorate Program Identification, Optimization and Control with Applications in Modern Technologies KONWIHR 22