High-Speed Flow Dynamic Research Through High-Order Numerical Simulation on ARCHER

Transcription

1 Daresbury Laboratory Science & Technology Facilities Council High-Speed Flow Dynamic Research Through High-Order Numerical Simulation on ARCHER Dr. Jian Fang Scientific Computing Department Science and Technology Facilities Council Daresbury Laboratory, UK

2 Daresbury Laboratory Science & Technology Facilities Council Outlines Background CFD Code Shock-wave/turbulent boundary layer interactions Flow Mechanisms Next-Gen Code for UKCTRF Conclusions Acknowledgements

3 Background Shock-Wave/Turbulent Boundary Layer Interaction

4 Background <u> Mean flow field, overall performance. Small amount of data, very efficient. Highly relied on turbulence model RANS Computational Fluid Dynamics Fully temporal/spatial resolved flow field. High-accuracy, Lots of data. Very expensive, Not ready for industrial applications. Highly relied on HPC resource and CFD code. DDES DES LES DNS x

5 CFD Code ASTR (Advanced flow Simulator for Turbulence Research) High-order FDM on generalized mesh High-order Dealiasing Compact Central scheme High-order Low-Dissipative Shock-Capturing Scheme 3 rd -order Runge-Kutta time scheme Modern Fortran 90 Parallelized by using MPI and Hybrid MPI-OpenMP Collective HDF5 I/O Tested in ARCHER, Tianhe, Hector, Blue Genes Mesh Generation Software Initial Field grid.h5 flowini3d.h5 PP Solver Tecplot/Origin/ Excel Results To analyze flowfield.h5 meanflow.h5 budget.h5 supply.h5 flowstate.dat monitor.dat tecfield.plt

6 Benchmarks 1.0 Present Result Experiment of Norberg Cp θ Present Results PIV of Parnaudeau et al. y/d Flow Passing a Cylinder x/d=1.06 x/d=1.53 x/d= v'v' Impinging Shock-Wave/Flat-Plate Boundary Layer Interaction Turbulent Channel Flow Tip-Leakage Vortex Well resolved wall-turbulence

7 Performance on ARCHER Test Case: 3D Taylor-Green Vortex Mach:0.1, Re=1600 Mesh size: 16.8M= ARCHER Nodes Cores CPU Time I/O Time Computing Time HDF5 I/O Time Time Cores

8 Performance on ARCHER Test Case: DNS of Turbulent Boundary Layer Mach:0.2, Re δ =7500 Mesh size: 71.2M= ARCHER Nodes Cores CPU Time I/O Time KNL Nodes Cores HyperThread MPI OpenMP CPU Time Comuting Time HDF5 I/O Time Perfect Acceleration Cputime Cores

9 Shock-wave/turbulent boundary layer interactions Cases studied Impinging Shock-Wave/Flat-Plate Boundary Layer Interaction Transonic Flow Passing a Cylinder Transonic Airfoil Compression Corner 3D Single-Fin Flow Supersonic Backward Step

10 Impinging Shock-wave/flat plate boundary layer interaction Mach=2.25;Impinging Angle: α=33.2 Re δ =41167, Re θ =3148 Impinging Shock Ma=2.25 α=33.2 Comput. Domain Reflected Shock Leading Edge Blow & Suction IM JM KM x y z (10 4 ) x min+ y 1+ z + Lz

11 Impinging Shock-wave/flat plate boundary layer interaction Shutts et al Bookey et al DNS DNS y/δ u + VD u + =y + u + =1/κ ln(y + ) u/u y + Velocity Profile RMS Velocity Fluctuation 3.0 Present DNS 2.7 Wu and Moin, Pirozzoli, et al., 2010 u ' Erm & Joubert, DRMS w ' DRMS Purteld et al., v ' DRMS y/δ RMS Velocity Fluctuation u ' DRMS w ' DRMS v ' DRMS Present DNS Wu and Moin, 2009 Spalart, 1981 Pirozzoli et al, 2010 Purteld et al., y + RMS Velocity Fluctuation Intensity Profile DNS.vs. PIV Humble, R.A., et al., Exp. Fluids, 2007, 43: p* Experiment of Dupont at Ma=2.3, α=32.4 Ο Experiment of Dupont at Ma=2.3, α=33.2 Ο Present Case S X* Wall pressure

12 Flow Mechanisms Turbulent Kinetic Energy Turbulent structures under zero and adverse pressure gradient Jian Fang, Zhaorui Li, and Lipeng Lu, An Optimized Low-Dissipation Monotonicity- Preserving Scheme for Numerical Simulations of High-Speed Turbulent Flows, Journal of Scientific Computing, 2013, Vol. 56, pp

13 Expansion-Compression Corner Expansion-Compression Corner Mach=2.9;Deflection Angle: β=25 Re δ =20000/40000/80000 Mesh size: / / Density Schlieren Pressure Field

14 Expansion-Compression Corner PW/P Cf EC CC x Wall Pressure Case 1 Case 2 Case 3 Experiment at Re δ =40700 Experiment at Re δ =67600 Experiment at Re δ =80000 Experiment at Re δ = LES of El-Askary 2011 LES of Knight et al EC CC x Skin friction Case 1 Case 2 Case 3 LES of El-Askary 2011 LES of Knight et al Experiment at Re δ = Experiment at Re δ =80000 y/δ RMS Velocity Fluctuation Case 1 Case 2 Case 3 Experiment at Re= at x=2δ Experiment at Re=80000 <u> y Mean Velocity Profiles in the Undisturbed + BL Case 1 Case 2 Case Purteld et al., 1981 Erm & Joubert, 1991 Wu and Moin, 2009 Pirozzoli, et al., y/δ u + vd RMS Velocity Fluctuation Case 1 at Re θ =1571 Case 2 at Re θ =3171 Case 3 at Re θ =6140 Murlis et al. (1982) at Re θ =791 Murlis et al. (1982) at Re θ =1368 Erm et al. (1985) at Re θ =617 u + vd =y+ u + vd =1/0.41y Case 1 Case 2 Case Purtell et al., 1981 Spalart, Wu and Moin, Pirozzoli et al, y + Turbulence Velocity Intensities in the Undisturbed BL 6140

15 Flow Mechanisms Formation of Görtler vortices Gortler Vortex Large-scale Görtler Vortex Iso-surface of ω colored with u x Jian Fang, Yufeng Yao, Alexandr A. Zheltovodov, Zhaorui Li, and Lipeng Lu, Direct numerical simulation of supersonic turbulent flows around a tandem expansion-compression corner, Physics of Fluids, 2015, 27,

16 3D SWTBLI Hypersonic Single Fin Mach=5.0;Deflection Angle: β=23 Re= /m, Re δ = Mesh: Fin Sketch map of the domain

17 3D SWTBLI Turbulent Coherent Structures & Shock Surface Near Wall Velocity Streaks

18 3D SWTBLI Separation Line Angle ϕ 1 of Experiment Reattachment Line Angle γ 1 of Experiment Secondary Separation Line Angle ϕ 2 of Experiment Separation Line Angle ϕ 1 of LES Reattachment Line Angle γ 1 of LES Secondary Separation Line Angle ϕ 2 of LES p W /p R1 R2 LES Results at x=83 mm LES Results at x=93 mm LES Results at x=123 mm LES Results at x=153 mm LES Results at x=183 mm Experiment Measure at x=83 mm Experiment Measure at x=93 mm Experiment Measure at x=123 mm Experiment Measure at x=153 mm Experiment Measure at x=183 mm β Characteristic Angles 0-3 S2 S z/x Wall Pressure

19 Flow Mechanisms Instantaneous Vorticity Fluctuations Zone 1: Undisturbed Wall Turbulence Zone 2: Separated Free-Shear Turbulence Zone 3: Jet Flow Zone 4: Regenerated Wall Turbulence Zone 5: Low-Turbulence Region Jian Fang, Yufeng Yao, Alexandr A. Zheltovodov, and Lipeng Lu, Investigation of Three-Dimensional Shock Wave/Turbulent-Boundary-Layer Interaction Initiated by a Single Fin, AIAA Journal, Vol. 55, No. 2:

20 Next-Gen Code for UKCTRF Geometry Complexity Swirler Piston Igniter Spray Interface Flame front Gas/Liquid interface Droplets surface Scalevariety Turbulence Flame Shock-wave Droplets

21 Next-Gen Code for UKCTRF HAMISH Code First developed at CUED. To solve the compressible Navier-Stokes equations with species mass fraction equations and chemical reactions. Finite volume method using a 2 nd -order spatial scheme. Runge-Kutta time stepping with sub-steps. Morton-code based unstructured dataset. Octree-based AMR. MPI+domain decomposition paralization.

22 Next-Gen Code for UKCTRF AMR in HAMISH (h-refinement) M-Code Tree data structure Binary tree/quadtree/octree M-Code+Tree

23 Next-Gen Code for UKCTRF Temperature Density

24 Next-Gen Code for UKCTRF HAMISH with Fixed Mesh of 6400 cells Temperature at t=0.001 HAMISH with Adaptive Mesh of 3676 cells

25 Next-Gen Code for UKCTRF HAMISH with Fixed Mesh of 6400 cells Temperature at t=0.001 HAMISH with Adaptive Mesh of 3676 cells

26 Next-Gen Code for UKCTRF trebal 2% discon 2% sortem 2% enthpy 2% strcel 1% OTHER 5% mccomp 19% Profile by Function More than 60% of the computation is cost for M- Code, Octree, AMR related subroutines. adaptm 12% mcxyzc 10% steppr 25% mclowr 1% mcrtrv 1% mcci2o 9% mccxyz 2% ocfind 5% mcglvl 2% MCCOMP MCXYZC MCCI2O OCFIND STEPPR ADAPTM Compares two Morton codes in their entirety Converts xyz coordinates into a Morton code at the specified level Converts an encoded integer array to an octal string Searches the local Octree using a given Morton code Time stepping of the solution, including calculating RHS Adapts the spatial mesh

27 Conclusions Thanks to ARCHER that a series of DNS/LES of SWTBLI flows can be done and the quality of the results is proved to be good. New flow structures and mechanisms are discovered based on the analysis of the data. AMR solver could be a game changer, although it is hard in term of coding. Future Plans Improve performance of ASTR in terms of I/O and vectorization Improve the parallel performance and load balance of HAMISH Add more functionalities to HAMISH.

28 Daresbury Laboratory Science & Technology Facilities Council Acknowledgements UKTC (EPSRC EP/L000261/1) UKCTRF (EPSRC EP/K026801/1) Hartree Centre for using their machines EPSRC ARCHER Leadership

29 Daresbury Laboratory Science & Technology Facilities Council Thank you very much