High Performance Computing

Transcription

1 High Performance Computing ADVANCED SCIENTIFIC COMPUTING Dr. Ing. Morris Riedel Adjunct Associated Professor School of Engineering and Natural Sciences, University of Iceland Research Group Leader, Juelich Supercomputing Centre, Germany SHORT LECTURE 15 Computational Fluid Dynamics November 9 th, 2017 Room Ág-303

2 Review of Lecture 14 Molecular Systems & Libraries Modelling behaviour & dynamics of molecules (molecular model of a protein) Molecular Docking Molecular Dynamics (docking of a small molecule ligand to a protein receptor to produce a complex) (force field) modified from [1] Wikipedia on Molecular Modelling [2] Wikipedia on Molecular Mechanics (Iterative simulation method calculating forces and solving the equations of motion based on the accelerations obtained from the new forces) modified from [3] Wikipedia on Molecular Dynamics 2/ 27

3 HPC-A[dvanced] Scientific Computing Second Part Consists of techniques for programming large-scale HPC Systems Approach: Get a broad understanding what HPC is and what can be done Goal: Train general HPC techniques and bits of domain-specific applications Domain-specific Science & Engineering A Domain-specific CFD Applications High Performance Computing (a field of constant changes) HPC A Course Domain-specific Engineering Applications Domain-specific Science & Engineering N 3/ 27

4 Outline of the Course 1. High Performance Computing 2. Parallelization Fundamentals 3. Parallel Programming with MPI 4. Advanced MPI Techniques 5. Parallel Algorithms & Data Structures 6. Parallel Programming with OpenMP 7. Hybrid Programming & Patterns 8. Debugging & Profiling Techniques 9. Performance Optimization & Tools 10. Scalable HPC Infrastructures & GPUs 11. Scientific Visualization & Steering 12. Terrestrial Systems & Climate 13. Systems Biology & Bioinformatics 14. Molecular Systems & Libraries 15. Computational Fluid Dynamics 16. Finite Elements Method 17. Machine Learning & Data Mining 18. Epilogue + additional practical lectures for our hands-on exercises in context 4/ 27

5 Outline Computational Fluid Dynamics Terminology & Motivation Naviar-Stokes Method Lattice-Boltzmann Method Large Eddy Turbulence Model Modelling Methodology Selected Libraries & Methods Fire Dynamics Simulator OpenFOAM HemeLB ifluids Short Lecture only shows subsets of libraries & applications in domains Selected promises from previous lecture(s): Lecture 5: Lecture 15 will give in-depth details on parallel CFD algorithms, tools, and application areas Lecture 5: Lectures will provide more details on complexities integrating technical & domain code Lecture 7: Lectures will provide more examples of using stencil-based methods as part of applications Lecture 11: Lecture 15 will show further methods for steering used in computational fluid dynamics (CFD) Lecture 11: Lectures will provide scientific visualizations for different HPC application domain fields Lecture 14: Lecture 15 will give further details on computational fluid dynamics (CFD) techniques & codes 5/ 27

6 Computational Fluid Dynamics 6/ 27

7 Parallel Algorithm Programming Revisited Pro: Network communication is relatively hidden and supported Contra: Programming with MPI/OpenMP requires parallelization methods Not easy: Write technical code well integrated in problem-domain code Three Key Parts, the recipe Example: Race Car Simulation 1. Apply a good parallelization method (e.g. domain decomposition) 2. Write manually good MPI (or/and OpenMP) code for (technical) communication between processors (e.g. across 1024 cores, then more cores, ) 3. Integrate well technical code with problem-domain code (e.g. computational fluid dynamics & airflow) Modified from [4] Caterham F1 team 7/ 27

8 Computational Fluid Dynamics Terminology & Motivation Fluid dynamics stands for the science of fluid motion in order to understand liquids, gases, etc. Computational Fluid Dynamics (CFD) use numerical methods to analyze fluid flows Fluid Dynamics as branch of Fluid Mechanics Alternative to Experimental fluid dynamics & Theoretical Fluid Dynamics Computational Fluid Dynamics (CFD) use numerical methods (cf. Lecture 12) Interaction of fluids with surfaces defined by certain boundary conditions & equations Usage Detailed product development (e.g. cars, aircraft) Conceptual studies of new designs Troubleshooting & Redesign Various Approaches [5] Wikipedia on CFD E.g. Naviar-Stokes equations, Lattice-Boltzmann method, Large-Eddy, etc. 8/ 27

9 Computational Fluid Dynamics Methods Overview [6] S. Orszag et al. Lecture 16 provide more details on the finite element method (FEM) used by CFD simulations 9/ 27

10 Computational Fluid Dynamics Navier-Stokes Method Navier-Stokes equations are used in CFD simulations to describe the motion of fluid substances Equations describe the physics of a wide variety of elements in scientific and engineering domains Origin of unique name modified from [7] Wikipedia on Navier-Stokes Named after Claude-Louis Navier and George Gabriel Stokes Solving equations Solution is a velocity (not a position) Result is a velocity/flow field Description of the velocity of the fluid at a given point in space and time Modeling CFD examples Ocean currents (cf. assignments) Water flow in a pipe Air flow around a wing (equations are nonlinear partial differential equations - PDEs, cf. Lecture 12) (classical mechanics solutions are rather trajectories of positions of a certain particle, here fluid velocity is in focus) 10 / 27

11 Computational Fluid Dynamics Lattice Boltzmann Method The Lattice Boltzmann Method (LBM) is used in CFD simulations to perform fluid simulations Discrete Boltzmann equations are solved to simulate the flow of fluids including collision models Alternative to Navier-Stokes Different set of equations Modeling approach Fluid consisting of particles Particles have a finite number of discrete velocity values Particles perform consecutive propagation and collision processes Performed over discrete lattice mesh Modeling complex CFD examples Airflow around a vehicle Blood flow in a brain modified from [8] Wikipedia on LBM (traditional CFD methods solve the conversation equations of macroscopic properties like mass, momentum, and energy) (Research accoustic impact of headlights) (Collision step) (Streaming step) [6] S. Orszag et al. 11 / 27

12 Computational Fluid Dynamics Large Eddy Turbulence Large Eddy Simulation (LES) is a mathematical model for turbulence used in CFD simulations LES operates on the Navier-Stokes equations to reduce the range of length scales of the solution Simulating Turbulence modified from [9] Wikipedia on LES [10] Wikipedia on Turbulence Flow characterized by chaotic property changes (chaotic from chaos theory) Low momentum diffusion High momentum convection Rapid variation of pressure and velocity (in space and time) LES enables complex simulations Solution is a filtered velocity field (filtered Naviar-Stokes equations) Small length and time scales enable computational simulation 12 / 27

13 Stencil-based Iterative Methods Revisited Stencil-based iterative methods update array elements according to a fixed pattern called stencil The key of stencil methods is its regular structure mostly implemented using arrays in codes Method is often used in computational science as part of scientific and angineering applications Simulation sciences & numerical methods Stencil-based iterative methods Applicable with exceptions with other methods: Finite element method (selected codes on regular grids can use stencil codes) Selected application examples Computational Fluid Dynamics (CFD) codes Partial differential equations (PDE) solver Jacobi method Gauss-Seidel method Image processing [11] Wikipedia on stencil code 13 / 27

14 Computational Fluid Dynamics Modeling Methodology Solve practical fluid flow problems Modeling & simulating reality but with specific scientific detailed elements Practical Preprocessing Steps 1. Geometry (physical bounds) of the problem space is defined 2. Volume occupied by the fluid is divided into discrete cells (aka mesh ) 3. Physical modeling is defined (e.g. equations of motion) 4. Boundary conditions (e.g. specify fluid behaviour & boundary properties) Computational simulation Equations are solved iteratively (cf. Lecture 7) via time-steps over space Postprocessing Steps Further simulation output data analysis and/or visualization [5] Wikipedia on CFD Lecture 16 provide more details on advanced domain decomposition methods reffered to as mesh 14 / 27

15 [Video] Computational Fluid Dynamics [12] YouTube Video, CFD 15 / 27

16 Selected Libraries & Methods 16 / 27

17 Computational Fluid Dynamics (CFD) Algorithms Revisited Scientific case: Understanding fire and smoke dynamics E.g. CFD of evacuation networks in underground traffic nodes Numerical representation of reality [13] Civil Security & Traffic Group Numerically well resolved CFD simulations require massive computational resources Parallel computing significantly decreases computing time Smart domain decompositions 17 / 27

18 Selected Libraries & Methods FDS Fire Dynamics Simulator (FDS) is a CFD simulation package solving simplified forms of the Navier- Stokes equations and enables large-eddy simulations (LES) for low-speed flows (i.e. smoke/fire) Selected Facts Open source package Includes visualization tool SmokeView Parallelization MPI (cf. Lecture 3) & OpenMP (cf. Lecture 6) Hybrid programming (cf. Lecture 7) Examples Smoke and heat transport from fires in underground stations (e.g. fire safety / civil engineering) modified from [14] FDS Web page [13] Civil Security & Traffic Group Lecture 16 provide more details on how smoke can be simulated using adaptive mesh refinement 18 / 27

19 Selected Libraries & Methods OpenFOAM Open source field operation and manipulation (OpenFOAM) is a C++ toolbox of numerical solvers and pre-/post processing utilities for the solution of CFD problems used in HPC simulations Selected Facts Free and open source software & maintained by OpenFOAM foundation Use of object oriented programming (e.g. C++ classes, etc.) Parallelization Several methods for Domain decomposition (cf. Lecture 2) Distributed memory with MPI (cf. Lecture 3) Examples Dispersion of stack exhaust above a pitched roof (e.g. environmental consulting) modified from [15] OpenFOAM Web page (Parallelization is inherent in the OpenFOAM design) (using the LES mode within OpenFOAM) [16] Lohmeier Consulting Engineers 19 / 27

20 Selected Libraries & Methods HemeLB HemeLB uses the Lattice-Boltzmann method and is a CFD simulation software to model the flow of blood on highly scalable supercomputers (e.g. blood velocity and pressure fields) Selected Facts Fluid particles are tracked (microscopic level) across a discret set of permissible velocities Parallelization Only nearest neighbor interactions are considered, use of advanced torus network topologies (cf. Lecture 4) Use of distributed memory with MPI (cf. Lecture 3) Examples modified from [17] HemeLB Web page (Could provide radiologists with Estimates of flow rates, pressures in the vascular structures to enable better treatment of patients) Patient brain blood-flow with magnetic resonance angiogram blueprint 20 / 27

21 Computational Steering Different Methods Revisited Steering requires an active manipulation channel Influencing HPC simulations by waiting for requests (e.g. not a good practice since HPC simulation may wait) Influencing HPC simulations by collecting requests (e.g. good practice since HPC simulation decides update) Only one channel vs. one channel per n remote users Application area examples Often iterative methods (e.g. time steps in simulations) Steer airflow in constructions (e.g. passengers in aircraft) Steer nbody simulations (e.g. star cluster simulations) Steer fluid simulations (e.g. boat acceleration in water) Parameter space exploration in HPC simulations in order to focus on regions of interests in science 21 / 27

22 Selected Libraries & Methods ifluids ifluids is a parallel software for CFD simulations in general and for indoor air flow simulations in particular being also augmented with computational steering capabilities and online visualizations Selected Facts [18] P. Wenisch et al. Simulation progresses & airflow can be interactively changed (e.g. geometries) Parallelization Use of MPI (cf. Lecture 3) Enables interactive parallel steering and online visualization (cf. Lecture 11) Examples Exploring capabilities of a ventilation system in a surgery room (i.e. airflow) 22 / 27

23 [Video] Selected Library example [19] YouTube Video, LES 23 / 27

24 Lecture Bibliography 24 / 27

25 Lecture Bibliography (1) [1] Wikipedia on Molecular Modelling, Online: [2] Wikipedia on Molecular Mechanics, Online: [3] Wikipedia on Molecular Dynamics, Outline: [4] Caterham F1 Team Races Past Competition with HPC, Online: [5] Wikipedia on Computational Fluid Dynamics, Online: [6] Steven Orszag et al., Lattice Boltzmann Methods for Fluid Dynamics, Online: [7] Wikipedia on Navier-Stokes, Online: [8] Wikipedia on Lattice Boltzmann Methods, Online: [9] Wikipedia on Large eddy simulations, Online: [10] Wikipedia on Turbulence, Online: 25 / 27

26 Lecture Bibliography (2) [11] Wikipedia on stencil code, Online: [12] YouTube Video, Computational Fluid Dynamics (CFD), Online: [13] Civil Security & Traffice Group, Online: [14] FDS Web page, Online: [15] OpenFOAM Web page, Online: [16] Lohmeyer Consulting Engineers, Online: [17] HemeLB, Science2020, Online: [18] P. Wenisch et al., Computational Steering: Interactive Flow Simulation in civil Engineering, inside, Vol. 5(2),2007 [19] YouTube Video, Large-Eddy Simulation (LES), Online: 26 / 27

27 27 / 27