Multigrid Algorithms for Three-Dimensional RANS Calculations - The SUmb Solver

Transcription

1 Multigrid Algorithms for Three-Dimensional RANS Calculations - The SUmb Solver Juan J. Alonso Department of Aeronautics & Astronautics Stanford University CME342 Lecture 14 May 26, 2014

2 Outline Non-linear multigrid algorithm - Review FAS multigrid + modified Runge-Kutta scheme Software and parallel implementation SUmb solver and results Unsteady algorithms Future

3 Non-linear Multigrid Algorithm Solve the non-linear equation in a mesh with spacing, h by finding a correction to the current iterate with some algebra

4 Non-linear Multigrid Algorithm and after smoothing the right hand side we can transfer the equation, solution, and residual to a coarser mesh or After relaxation in the coarse mesh, the coarse grid correction is which can be interpolated to the finer mesh and the procedure repeated until convergence (possibly recursively)

5 Modified Runge-Kutta Time Stepping Semi-discrete NS equations Convective and dissipative residuals Modified Runge-Kutta scheme

6 Modified Runge-Kutta Time Stepping Modified Runge-Kutta scheme Coefficients for Q and D are created separately to Minimize CPU time Improve convergence properties Time accuracy is lost...do we care?

7 FAS Multigrid Algorithm Volume/Area-weighted solution coarsening Coarse grid residual forcing term Modified R-K in coarse grid Coarse grid driven by residuals transferred from fine mesh We impose boundary conditions on every mesh 1st order artificial dissipation on coarser meshes

8 FAS Multigrid Algorithm restriction prolongation restriction prolongation Full-coarsened multigrid V or W cycles of arbitrary depth Bi-linear or tri-linear interpolation of solution to finer meshes

9 Additional Convergence Acceleration Schemes Local time stepping Implicit Residual Smoothing Enthalpy Damping Others... Block-Jacobi preconditioner Low-speed preconditioning J-coarsening or semi-coarsening

10 FAS Multigrid Algorithm C ****************************************************************** C * * C * TRANSFERS THE SOLUTION TO A COARSER MESH * C * * C ******************************************************************. DO N=1,4 JJ = 1 DO J=2,JL,2 JJ = JJ +1 II = 1 DO I=2,IL,2 II = II +1 WWR(II,JJ,N) = (DW(I,J,N)*VOL(I,J) +DW(I+1,J,N)*VOL(I+1,J). +DW(I,J+1,N)*VOL(I,J+1) +DW(I+1,J+1,N) *VOL(I+1,J+1))/. (VOL(I,J)+VOL(I+1,J)+VOL(I,J+1)+ VOL(I+1,J+1)) END DO END DO END DO. C ****************************************************************** C * * C * COMPLETE THE CALCULATION OF THE MGRID FORCING TERMS * C * ON FIRST ENTRY TO ANY COARSER MESH DURING THE CYCLE * C * TRANSFERS THE SOLUTION TO A COARSER MESH * C * * C ****************************************************************** C DO 32 J=2,JL DO 32 I=2,IL WR(I,J,N) = FCOLL*WR(I,J,N) -DW(I,J,N) 32 CONTINUE C C ADD THE MULTIGRID FORCING TERMS TO THE RESIDUALS C 33 DO 34 J=2,JL DO 34 I=2,IL DW(I,J,N) = DW(I,J,N) +WR(I,J,N) 34 CONTINUE 40 CONTINUE Implementation follows description of FAS algorithm closely Most of the code is unaware of what grid level it is working on Pointer kept for current, finer, and coarser grid levels

11

12 Is this good enough? Two-dimensional Euler results converge with an average residual contraction ratio of ~ 0.65 (almost textbook multigrid) Three-dimensional Euler calculations also converge with an average contraction ratio ~ 0.75 Turbulent NS calculations with integration to the wall usually converge at ~ What has happened to our textbook multigrid?

13 Parallel Implementation For cell-centered schemes such as the one in SUmb, parallel implementations are straightforward Chop up the domain and distribute it evenly among processors (using Metis and a graph representation of the multiblock mesh) Whole blocks get assigned to processors Possibility of block splitting to improve load balancing Double halo approach to exchange information

14 Parallel Implementation

15 Parallel Implementation - Main Issues In our algorithm, bandwidth is important since we must communicate At the end of each stage in the mod R-K sequence At all multigrid levels Double level halo is necessary on each communication Double precision messages (8 words = 5 cons. vars + 2 turbulence vars + pressure) Non-blocking receives and sends used throughout the code Not the most stringent requirement

16 Parallel Implementation - Main Issues In our algorithm, latency is VERY important, since its importance grows particularly at the coarse mesh levels Fine meshes: 95% of communication cost is bandwidth Coarsest meshes: 65% of communication cost is latency Only way to avoid latency cost is to communicate less often. We have tried this and have found that the improvement in parallel performance is more than offset by the degradation in multigrid convergence. Bottomline: for good multigrid convergence in parallel you will need a high-performance network...100baset switches will not cut it beyond 8 procs.

17 Parallel Implementation - Main Issues In the literature, there are options for vertical and horizontal multigrid How often does one communicate at the coarser levels of the multigrid sequence? Only on the fine mesh? At all mesh levels? Sometimes at the coarse levels? Most researchers now agree that: A parallel multigrid implementation that mimics the convergence history of a serial implementation is the best approach Parallel performance hit is taken (but overcome by improvements in performance) Some uses of multigrid as preconditioner can ameliorate the parallel performance impact

18 Multigrid Semi-coarsening / J-coarsening There are indications that something can be done about the lack of convergence of multigrid methods in RANS calculations Encouraging 2D results first shown by N. A. Pierce and reproduced by Darmofal Results shown in this talk from N. A. Pierce s PhD Thesis N.A.Pierce, Preconditioned Multigrid Methods for Compressible Flow Calculations on Stretched Meshes, Christ Church, University of Oxford, D remains an elusive challenge

19 Why does multigrid not converge well for RANS? Stiffness comes in from a variety of sources Propagative speed disparity Cell stretching Flow alignment Turbulence models

20 Euler Convergence, Full Multigrid Euler solutions with full multigrid converge reasonably well, although performance can be improved

21 Convergence behaves appropriately for larger meshes and higher Mach numbers, although it can always be improved

22 Euler / NS Multigrid Convergence Analysis Standard Full Coarsened Multigrid Standard Full Coarsened Multigrid + Block-Jacobi Precon

23 Euler / NS Multigrid Convergence Analysis J-Coarsened Multigrid with Block-Jacobi Precon Standard full coarsening has problems with All convective modes High-x / Low-y acoustic modes Adding Block-Jacobi Precon High-x / Low-y acoustic modes J-Coarsening (semicoarsening) appears to be able to damp all modes. Does it?

24 NS Multigrid Convergence Test Cases RANS solutions can be made to converge as efficiently (contraction ratios ~ 0.75) as Euler calculations!!!

25 NS Multigrid Convergence Test Cases Good convergence is a combination of techniques that provide damping across the whole spectrum of modes. What about for 3D?

26 NS Multigrid Convergence Test Cases In theory, the same can be proven for 3D calculations In practice, this has only been shown for VERY smooth meshes (no wing tips, no tip gaps, no bad stretching) More works remains to be done

27 Conclusions Multigrid has been one of the most successful algorithms for compressible fluid flow (of course, also for elliptic equations) It can be parallelized efficiently It can be used for both steady and unsteady computations (dual-time stepping) Performance degrades with high degrees of mesh stretching Work remains to be done to obtain textbook multigrid convergence