Efficient use of OpenFOAM in industry

Transcription

1 Elmer Technologies: Efficient use of OpenFOAM in industry Author: Oskar Elmgren

2 Elmer Technologies Built on motorsport experience Specializing in product and technology development Simulation and prototype production expertice We want to help change the world!...and make it more efficient in the process

3 Presentation Targets Setup Overview of a efficient, fully parallel, setup Preprocessing Calculation Postprocessing Benchmarks Scalability 1-20 cores, heterogenous setup Typical case wall time Observations Challenges, thoughts and suggestions

4 Elmer Technologies' Network Setup Company Network Interface Computer Compute Node 1 Compute Node 3 1Gbit Router Compute Node 2 Compute Node 4

5 Our Interface Computer Hardware 8GB memory 3TB SATA hard disks nvidia graphics card Multiple Ethernet cards Software Linux; opensuse OpenFOAM source installation

6 Compute Node Setups Hardware Desktop CPU (Intel i5, 4core) (AMD Phenom II X6, 6core) 16GB memory (2x1600MT/s) (2x1333MT/s) 1TB SATA harddisk Ethernet (Intel chipset) Software Linux; opensuse, passwordless rsh access OpenFOAM NAT accessed Case files NAT accessed

7 Computation Process 1. Copy base case files to new directory 2. Copy computation model file (.stl) 3. Run compute script 4. Check convergence, etc. 5. Postprocess

8 Compute Script 1. blockmesh 2. decomposepar 3. snappyhexmesh -parallel 4. boundary condition script 5. potentialfoam -parallel 6. simplefoam -parallel

9 Boundary Script Needed if meshing is done in parallel Custom code; Python script Read target boundary conditions Read case boundary conditions Compare Add missing boundary conditions to parallel mesh

10 Result Checking (CFD in general) Check convergence of result variables Is the result reasonable? Do changes produce correct results? Ultimately: TEST!

11 Postprocessing ParaView in parallel (on interface machine) paraview compiled with mpi option parallel OpenFOAM case reader ParaView in client-server mode (for larger cases) same as above utilizes compute nodes' processors and memory (very fast, will not run out of memory)

12 Capability of Our Setup Race car model difficult flow 24.5M cells static k-omega SST turbulence model 3000 iterations, 29h

13 Scalability Test GEV model well behaved flow 1M & 5M cells static k-omega SST turbulence model 900 iterations

14 Scalability Data "small" case 1M cells computation time [min] snappyhexmesh 80 simplefoam Total time core i5 2core i5 4core i5 8core i5 2computers 2core i5 2computers 4core i5 2computers 1core X6 6core X6 12core X6 multilayer 20core core core X6 12core X6 multilayer2 20core 1-1

15 More Scalability Data! "large" case 5M cells computation time [min] snappyhexmesh simplefoam Total time core i5 1core i5 4core i5 2core i5 2computers 8core i5 2computers 4core i5 2computers 20core core X6 20core 1-3

16 Performance Observations Efficient heterogenous decomposition difficult Multi-level decomposition seems interesting Strange behavior to be expected, with large performance difference between Ethernet and memory bandwidth

17 Other Observations Hardware Intel i5 unreliable, but fast. Also needs HD cooler Realtek network hardware should be avoided Debugging problems might be difficult Software No user interface, poor meshing utility Dedicated machines strongly advised Supprisingly few bugs/issues

18 Cost Interface machine +700 Compute node ~700 Network router, cables ~150 example setup 3650 Employee system setup 1week-2months case setup 1-40h

19 Conclusions CFD computations can be done very cheaply Open source world is a challenge to enter help = google often very poor/no documentation Open source software is a safe investment copes very well with future needs scales to more, or larger cases

20 Thank You! (and links)

21 Science is like sex; it has practical uses, but that's not why we do it. -Richard Feynman