OpenFOAM Wind Simulation Tutorial

Transcription

1 H2020-EINFRA VI-SEEM VRE for regional Interdisciplinary communities in Southeast Europe and the EasternMediterranean OpenFOAM Wind Simulation Tutorial Author(s): Status Version: Distribution - Type: Neki Frasheri (UPT) Draft a Date: September 11, 2017 Internal Abstract: Tutorial on preparation for execution of OpenFOAM for wind simulation over mountainous terrain, application supported by Polytechnic University of Tirana.. Copyright by the VI-SEEM Consortium The VI-SEEM Consortium consists of: GRNET Coordinating Contractor Greece CYI Contractor Cyprus IICT-BAS Contractor Bulgaria IPB Contractor Serbia NIIF Contractor Hungary UVT Contractor Romania UPT Contractor Albania UNI BL Contractor Bosnia-Herzegovina UKIM Contractor FYR of Macedonia UOM Contractor Montenegro RENAM Contractor Moldova (Republic of) IIAP-NAS-RA Contractor Armenia GRENA Contractor Georgia BA Contractor Egypt IUCC Contractor Israel SESAME Contractor Jordan VI-SEEM consortium

2 D5.1Detailed technical implementation plan for VRE services and tools Page 2 of 19 The VI-SEEM project is funded by the European Commission under the Horizon 2020 e- Infrastructures grant agreement no This document contains material, which is the copyright of certain VI-SEEM beneficiaries and the European Commission, and may not be reproduced or copied without permission. The information herein does not express the opinion of the European Commission. The European Commission is not responsible for any use that might be made of data appearing herein. The VI-SEEM beneficiaries do not warrant that the information contained herein is capable of use, or that use of the information is free from risk, and accept no liability for loss or damage suffered by any person using this information.

3 D5.1Detailed technical implementation plan for VRE services and tools Page 3 of 19 Document Revision History Date Issue Author/Editor/Contributor Summary of main changes 11/09/2017 a Neki Frasheri First version.

4 D5.1Detailed technical implementation plan for VRE services and tools Page 4 of 19 Preface In the last decade, a number of initiatives were crucial for enabling high-quality research - by providing e-infrastructure resources, application support and training - in both South East Europe (SEE) and Eastern Mediterranean (EM). They helped reduce the digital divide and brain drain in Europe, by ensuring access to regional e-infrastructures to new member states, states on path to ascension, and states in European Neighborhood Policy area in total 14 countries in SEE and 6 in EM. This VI-SEEM project brings together these e-infrastructures to build capacity and better utilize synergies, for an improved service provision within a unified Virtual Research Environment (VRE) for the inter-disciplinary scientific user communities in the combined SEE and EM regions (SEEM). The overall objective is to provide user-friendly integrated e-infrastructure platform for regional cross-border Scientific Communities in Climatology, Life Sciences, and Cultural Heritage for the SEEM region; by linking compute, data, and visualization resources, as well as services, models, software and tools. This VRE aspires to provide the scientists and researchers with the support in full lifecycle of collaborative research: accessing and sharing relevant research data, using it with provided codes and tools to carry out new experiments and simulations on large-scale e-infrastructures, and producing new knowledge and data - which can be stored and shared in the same VRE. Climatology and Life Science communities are directly relevant for Societal Challenges. The driving ambition of this proposal is to maintain leadership in enabling e- Infrastructure based research and innovation in the region for the 3 strategic regional user communities: supporting multidisciplinary solutions, advancing their research, and bridging the development gap with the rest of Europe. The VI-SEEM consortium brings together e-infrastructure operators and Scientific Communities in a common endeavor. The overall objective is to provide user-friendly integrated e-infrastructure platform for Scientific Communities in Climatology, Life Sciences, and Cultural Heritage for the SEEM region; by linking compute, data, and visualization resources, as well as services, software and tools. The detailed objectives of the VI-SEEM project are: 1. Provide scientists with access to state of the art e-infrastructure - computing, storage and connectivity resources - available in the region; and promote additional resources across the region. 2. Integrate the underlying e-infrastructure layers with generic/standardised as well as domain-specific services for the region. The latter are leveraging on existing tools (including visualization) with additional features being co-developed and co-operated by the Scientific Communities and the e-infrastructure providers, thus proving integrated VRE environments. 3. Promote capacity building in the region and foster interdisciplinary approaches. 4. Provide functions allowing for data management for the selected Scientific Communities, engage the full data management lifecycle, link data across the region, provide data interoperability across disciplines. 5. Provide adequate user support and training programs for the user communities in the SEEM region. 6. Bring high level expertise in e-infrastructure utilization to enable research activities of international standing in the selected fields of Climatology, Life Sciences and Cultural Heritage.

5 D5.1Detailed technical implementation plan for VRE services and tools Page 5 of 19 The VI-SEEM project kicked-off in October 2015 and is planned to be completed by September It is coordinated by GRNET with 15 contractors from Cyprus, Bulgaria, Serbia, Hungary, Romania, Albania, Bosnia-Herzegovina, FYR of Macedonia, Montenegro, Moldova (Republic of), Armenia, Georgia, Egypt, Israel, Jordan. The total budget is The project is funded by the European Commission's Horizon 2020 Programme for Excellence in Science, e-infrastructure. The project plans for Polytechnic University of Tirana (UPT), Albania, include realization of wind simulation over mountainous terrain characteristic for SEE countries. The selected software for such simulations is open source package OpenFOAM. In this document the procedures for preparation for execution of OpenFOAM models are presented. The work was supported by the parallel system in the Faculty of Information Technology of Polytechnic Univertsity of Tirana, and parallel systems HPC and AVITOHOL of Institute of Information and Communication Technologies IICT-BAS of Bulgarian Academy of Sciences in Sofia.

6 D5.1Detailed technical implementation plan for VRE services and tools Page 6 of 19 Table of contents 1. Introduction Input Data Preparation Input Data Modification Running of OpenFOAM solver Visualization of OpenFOAM results...18 References [1] OpenFOAM - the open source CFD toolbox guide (html) [2] OpenFOAM - the open source CFD toolbox guide (pdf) [3] Utkarsh Ayachit. The ParaView Guide submit=download&version=v5.3&type=data&os=all&downloadfile=paraviewguide pdf [4] Neki Frasheri, Emanouil Atanassov. Scalability Issues for Wind Simulation using OpenFOAM. COST Action IC rd Workshop NESUS2016, 6-7 October 2016, Sofia, Bulgaria [5] N.Frasheri, E. Atanassof. Interprocess Communication Issues with OpenFOAM for Wind Simulation. BalkanCom'17 First International Balkan Conference on Communications and Networking, Tirana, Albania, May 30-June 2, 2017 [6] N. Frasheri, D. Minarolli. H2020 Project VI-SEEM a Regional Virtual Research Environment. 8-th International Conference on Information Systems and Technology Innovations: Fostering the As-A-Service Economy, Tirana, Albania, June 23-24, 2017 [7] E. Leblebici, I. H. Tuncer. Coupled Unsteady OpenFOAM and WRF Solutions for an Accurate Estimation of Wind Energy Potential. VII European Congress on Computational Methods in Applied Sciences and Engineering, M. Papadrakakis, V. Papadopoulos, G. Stefanou, V. Plevris (eds.) Crete Island, Greece, 5 10 June 2016 List of Figures Figure 1Shape of 3D body modeling the atmosphere...12 Figure 2 DEM image used for wind simulation...14 Figure 3 Deformed bottom surface of 3D body...15 Figure 4 Visualization of cavity example results...18 Figure 5 Some results for West-East wind, low resolution DEM...18 Figure 6 Some results of field potential for North-South wind. Left: Nort-Souuth component, Center: West-East component, Right: vertical component...19 List of Tables

7 D5.1Detailed technical implementation plan for VRE services and tools Page 7 of 19 Executive Summary What is the focus of this Tutorial The focus of this tutorial is presentation of OpenFOAM (Open Source Field Operation and Manipulation C++ libraries) data preparation for wind simulation over mountainous terrain. There are proper tutorials [1,2] for using of OpenFOAM in solving a wide range of problems in computational fluid dynamics (CFD). The user must get acquainted with the use of this complex package through proper tutorials. Steps used for the analysis of the scalability of OpenFOAM for wind simulation over the terrain of Albania are described in this document. Preparation of digital terrain model data Digital terrain model data (DEM) used in this work is obtained from US Geological Service (USGS), based on Shuttle Radar Topography Mission (SRTM) carried out by space shuttle Endeavour in February 11-22, 2000, which images of GEOTIFF format are offered free in the USGS repositories with resolution of 3 arc, approximately with pixel size 100m*100m in equator, and 70m*100m in latitudes 40 degrees. Processing of DEM data started with the combination of image tiles using a software for remote sensing imagery processing (GlobalMapper in our case), and exporting the combined image in USGS DEM ASCII format. In a second step the package GDAL was used to transform DEM data in a simpler format SURFER ASCII. A specific software DEM2FOAM was developed in UPT to deform following the terrain the 3D mesh array generated by pre-processing package blockmesh of OpenFOAM. Running of OpenFOAM software After preparation of DEM data in SURFER format, the execution of OpenFOAM followed the standard steps with one addition: - generation of rectangular 3D mesh using blockmesh - additional: deformation of nodes coordinates using DEM data and DEM2FOAM software - optional: splitting of 3D mesh in chunks for parallel processing with decomposepar - running of OpenFOAM solver icofoam (in sequential or in parallel) - optional: joining processed chunks in a single array with reconstructpar - visualization of results using parafoam (OpenFOAM version of paraview) The results are composed by 3D arrays containing field potential and flux in mesh nodes for a number of time intervals as defined in standard input data of OpenFOAM.

8 D5.1Detailed technical implementation plan for VRE services and tools Page 8 of Introduction OpenFOAM (Open Source Field Operation and Manipulation) is an open source package of C++ libraries for solving of a wide range of problems in computational fluid dynamics (CFD) based in Navier Stokes equations [1,2], including both static and dynamic cases. Wind simulation over rugged terrain represents a typical case of gas dynamics that may be solved with OpenFOAM. Moreover part of OpenFOAM packages are parallelized with MPI and may be run in parallel systems. Preparation of OpenFOAM input data is described in details in available tutorials, and the package itself comes with a large number of ready to run examples that may be used as starting point. In our case the incompressible flow example was used, based on the solver module icofoam. IcoFoam module is parallelized with MPI that permits its running in HPC systems. Specific modules of OpenFOAM are used to prepare the parallelized input data and combine related results in a single set of files. Because of the dynamism of the phenomena solutions are obtained sequentially in regular time steps and stored in separate directories. Visualization of results is done using ParaFoam software, which is a customized version of general purposed ParaView software. ParaFoam package includes client-server modules that may be used to visualize in local results from a HPC remote system.

9 D5.1Detailed technical implementation plan for VRE services and tools Page 9 of Input Data Preparation The preparatory module blockmesh is used to digitize 3D volumes composed by a number of prismatic bodies. The problem in wind simulation over rugged terrain is modeled with a simple parallelepiped, which lower horizontal face would represent the earth surface and need to be deformed following the terrain. This deformation must be distributed through the whole 3D mesh and decreased gradually until becomes zero in the upper face of the volume. Execution environment is based in OpenFOAM directory in users home directory with path: /home/username/openfoam/username-version/run/ where: username the login name of the user version the version number of OpenFOAM Under /run/ directory specific directories for different models may be created. Input data files are distributed in several standard directories in related model directory: 0/ 0/p 0/U constant/ constant/polymesh/ constant/polymesh/blockmeshdict constant/transportproperties system/ system/controldict system/fvschemes system/fvsolution All these files are simple text ASCII. We used data files for the cavity incompressible flow example found in OpenFOAM directory: /opt/openfoam-2.3.0/tutorials/incompressible/icofoam/cavity/ and modified part of them to create our models for the wind simulation. The file 0/p contains boundary initial conditions for the potential field: boundaryfield { movingwall { type zerogradient; outputwall { type fixedvalue; value uniform -1.0; inputwall { type fixedvalue; value uniform 1.0; fixedwall { type zerogradient;

10 D5.1Detailed technical implementation plan for VRE services and tools Page 10 of 19 freewall { type zerogradient; Here we have potential field values +1 and -1 for input and output faces of the 3D body. The file 0/U contains boundary initial conditions for the velocity field: boundaryfield { outputwall { type fixedgradient; gradient uniform (0 0 0); inputwall { type fixedgradient; gradient uniform (0 0 0); fixedwall { type fixedvalue; value uniform (0 0 0); freewall { type fixedgradient; gradient uniform (0 0 0); Here we have initial velocity field values zero in input and output faces of the 3D body. The constant/polymesh/bloclkmeshdict file contains data on corners (vertices) of the 3D body and its faces are defined using numbers of respective vertices, and at last the boundary initial conditions: vertices ( ( 0 0 0) // vertice 0 coordinates ); blocks ( ) // vertice 1 coordinates ( ) // vertice 2 coordinates ( ) // vertice 3 coordinates ( ) // vertice 4 coordinates ( ) // vertice 5 coordinates ( ) // vertice 6 coordinates ( ) // vertice 7 coordinates // parallelipiped vertices and number of nodes in each of XYZ axes ( hex ( ) ( ) simplegrading (1 1 1) );

11 D5.1Detailed technical implementation plan for VRE services and tools Page 11 of 19 boundary ( outputwall { type wall; faces ( ( ) ); // top inputwall { type wall; faces ( ( ) ); //bottom fixedwall { type wall; faces ( ( ) ); // back freewall { type wall; faces ( ( ) // front ( ) // right ( ) ); // left Vertice numbers are used to define the 3D block and its faces, right oriented to point from inside to outside (figure 1): Z 7 6 Y X 0 1 Figure 1Shape of 3D body modeling the atmosphere After running the digitizing module blockmesh, new files with coordinates of all elements nodes and faces are created in the directory constant/polymesh, including the points file with the number of nodes and coordinates of each node. It is this file that is modified to deform the 3D mesh following the terrain DEM data, considering the bottom face {1,5,4,0 to represent the terrain surface.

12 D5.1Detailed technical implementation plan for VRE services and tools Page 12 of 19 Other important input data include the Reynolds number and the time step control. Reynolds number defines the viscosity of the fluid, and for gases it has very high values like 10^5, defined (in bold) in the file constant/transportproperties in the line: nu nu [ ] ; Time step control for the digitizing in time domain is included in the file system/controldict with lines similar with: starttime 0; endtime 100; deltat 1.0; writeinterval 10; The solver calculates both potential and velocity fields in 3D mesh nodes for time moments starting in starttime until endtime with step deltat, all in seconds. There is a link between the digitizing spatial steps and the temporal step. Decreasing of spatial steps must be followed by decreasing of the temporal step. If the ratio: ( temporal step ) / (spatial step ) increases, increases the risk of degeneration of the iterative process and the OpenFOAM solver may abort due to the increase of errors, reflected in the value Courant Number greater than 1 (one), displayed by the solver after processing of each iteration. This constraint increases difficulties for solving of high resolution models for the same time interval, decreasing of the spatial steps implies decreasing of temporal step by similar factor, increasing the number of iterations, as result the runtime of the solver and the disk space required for the results.

13 D5.1Detailed technical implementation plan for VRE services and tools Page 13 of Input Data Modification The Digital Terrain Model (DEM) is obtained from US Geological Service (USGS), based on Shuttle Radar Topography Mission (SRTM) carried out by space shuttle Endeavour in February 11-22, DEM data are composed of image tiles in GEOTIFF format, and are offered free in the USGS repositories. The resolution of images is 3 arc per pixel, approximately with pixel size 100m*100m in equator, and 70m*100m in latitudes 40 degrees. DEM tiles must be combined in a single image using any remote sensing software like GLOBALMAPPER. Example of DEM image for the territory of Albania is given in figure 2: Figure 2 DEM image used for wind simulation Processing of DEM data started with the combination of image tiles using a software for remote sensing imagery processing (GlobalMapper in our case), in our case combined image was exported the in USGS DEM ASCII format. In a second step the package GDAL was used to transform DEM data in a simpler format SURFER ASCII, for example: gdal_translate -of GSAG alb.dem alb.grd where the output file head is similar to: DSAA // number of pixels // corner latitudes // corner longitudes // altitudes in meters // pixel values in meters // Three cases of DEM images were used, with different resolutions: pixels 3601 x 4801 pixels 360 x 480 resolution 100 m/pixel resolution 1,000 m/pixel pixels 36 x 48 resolution 10,000 m/pixel

14 D5.1Detailed technical implementation plan for VRE services and tools Page 14 of 19 A specific software DEM2FOAM was developed in UPT to deform the file points following the terrain the 3D mesh array generated by pre-processing package blockmesh of OpenFOAM. DEM2FOAM works based on configuration data from a text file:./dem2foam config_file Where the structure of config_file is similar with following: Model test # identification string inpfilename points # name of input file demfilename dem-file # name of DEM file outfilename points.new # name of output file ( points ) dem_xori 0 # origine coordinates X dem_yori 0 # origine coordinates Y dem_xstep 750 # DEM digitalization X step dem_ystep 1000 # DEM digitalization Y step dem_nx 750 # number of DEM points in X dem_ny 1000 # number of DEM points in Y demxyz z # coordinate to be modified An example of the deformed 3D body bottom surface, visualized with ParaFoam, is given in figure: Figure 3 Deformed bottom surface of 3D body After calculation the new points.new file must be renamed with the original name points, because the OpenFOAM solver requires this file named points in its directory.

15 D5.1Detailed technical implementation plan for VRE services and tools Page 15 of Running of OpenFOAM solver After preparation of DEM data in SURFER format in the working directory, the execution of OpenFOAM followed the standard steps with one addition: - generation of rectangular 3D mesh using blockmesh - additional: deformation of nodes coordinates using DEM data and DEM2FOAM software - optional: splitting of 3D mesh in chunks for parallel processing with decomposepar - running of OpenFOAM solver icofoam (in sequential or in parallel) - optional: joining processed chunks in a single array with reconstructpar - visualization of results using parafoam (OpenFOAM version of paraview) The results are composed by 3D arrays containing field potential and flux in mesh nodes for a number of time intervals as defined in standard input data of OpenFOAM. The program DEM2FOAM must be copied in the constant/polymesh directory together with the DEM file and the config file where the input / output files names are declared. Scripts for sequential execution of icofoam were: # digitalization blockmesh # terrrain correctio cd constant/polymesh mv points points.copy./dem2foam config # solver icofoam # visualiization parafoam Parallel execution requires an additional data file:./system/decomposepardict, which includes data in splitting the 3D body in chunks, similar with: numberofsubdomains 8; // number of chunks method simple; simplecoeffs { n ( ); // chunks per axis X,Y,Z delta 0.001; In this example the 3D body will be split in 8 similar chunks making 4 cuts in X axis and 2 cuts in Y axis. The parallel execution may be carried out with 8 processes using a maximum number of 8 cores. The typical script for the parallel execution of the solver is similar with the sequential one, including commands for pre-parallelization of input data and post-parallelization of results:

16 D5.1Detailed technical implementation plan for VRE services and tools Page 16 of 19 # digitalization blockmesh # terrrain correctio cd constant/polymesh mv points points.copy./dem2foam config # pre paralellization decomposepar # solver in paralle with 8 processes mpiexec -np 8 icofoam -parallel # post parallelization reconstructpar # visualization parafoam Launching of this script should be done following usual rules for MPI execution. The solver icofoam will store results separately for each process in subdirectories named processor#, containing output directories for partial results, and it is the reconstructpar that combines partial results for the whole 3D body and predefined time intervals. The solver calculates both potential and velocity fields in 3D mesh nodes for time moments starting in starttime until endtime with step deltat in seconds. Results for each writeinterval are stored in the working directory, each of them in a separate directory named by the respective time moment, containing field arrays in files named as: p phi U => volume potential field (scalar) => surface potential field (scalar) => volume velocity field (vectorial) Graphical visualization of results may be done using the package ParaFoam [3]. Some results from experiments to evaluate the scalability of OpenFOAM while running wind simulation in parallel systems are presented in [4,5,6]. The work was supported by the parallel system in the Faculty of Information Technology of Polytechnic Univertsity of Tirana, and parallel systems HPC and AVITOHOL of Institute of Information and Communication Technologies IICT-BAS of Bulgarian Academy of Sciences in Sofia.

17 D5.1Detailed technical implementation plan for VRE services and tools Page 17 of Visualization of OpenFOAM results Visualization of results is done using ParaFoam, a customized version of the general purpose spatial data visualization package ParaView. ParaFoam automatically reads results from the working directory and permits the user to display all fields for any selected time in several modes, including volume and planar isolines and field lines. The user needs to use the ParaView tutorials [1,2,3] to learn how to visualize the data. Different modes of visualization for a modified case of cavity example from the OpenFOAM tutorial are as in figure: Figure 4 Visualization of cavity example results In this case the 3D body is a regular cube, which bottom is deformed. Some results from wind simulation tests are in figure: Figure 5 Some results for West-East wind, low resolution DEM In the figure there are potential field and field lines in three vertical planes North-South, and for three horizontal planes in altitudes 500m, 1000m, and 1500m.

18 D5.1Detailed technical implementation plan for VRE services and tools Page 18 of 19 Medium resolution results of potential field for Nort-to-South wind in altitude 1000m are as in the figure: Figure 6 Some results of field potential for North-South wind. Left: Nort-Souuth component, Center: West-East component, Right: vertical component In case of running OpenFOAM in remote machine, a client-server version of ParaView is available: server side: client side: module pvserver must be executed depending on the mode of execution of OpenFOAM serial or parallel serial: pvserver - use-offscreen-rendering parallel: mpirun -np # pvserver use-offscreen-rendering ParaView menu File -> Connect opens a dialog pops up where the IP of the server machine may be added and the related port number (11111), afterwards the server selected and connected. After connection open the file *.Foam.

19 D5.1Detailed technical implementation plan for VRE services and tools Page 19 of Conclusions OpenFOAM is a software package for solving Navier Stokes equations for a wide range of problems from gas dynamics to stress analysis. As such, it can be used for isolated wind simulation models; while there are tentatives to couple OpenFOAM with weather forecast package WRF [7]. During our experiments [4,5,6] we analyzed the scalability of the software in parallel systems. Some conclusions are hereunder. Being a general purposed package, OpenFOAM complex of input data is not very suitable for the narrow scope of wind simulation when considering the fact that wind is a dynamic process dependant on variable boundary conditions and coupled with a multitude of atmospheric phenomena. Eventually it may be used also to integrate in wind simulation models the effect of the air temperature, which remains a task for the future. The critical problem with OpenFOAM for wind simulation over wide terrain areas remains the scalability in the runtime and memory requirements. While only the solver icofoam is parallelized, other modules are not blockmesh, decomposepar, reconstructpar. During the experiments these modules presented runtime and memory requirements comparable with icofoam [4,5,6]. And the physical central memory defines the size of virtual memory required by each of modules. Results for a 3D dynamic model as wind simulation are stored in the external storage for a suite of time intervals, occupying huge disk space. While executed in parallel, this space is doubled because icofoam stores chunks of results for each parallel process, while reconstructpar combines them in a single set of file taking its part of disk space. Keeping results for long suites of short time intervals are necessary to analyze wind fluctuations and turbulence, also necessary for high resolution models. All this increases runtime and memory requirements. In case of the terrain of Albania it was possible to run models with resolution up to 100m steps of the spatial mesh, while most of valleys are narrow with widths of the range of 1,000m. Detailed analysis of such requirements seems to be rare in the literature. Using OpenFOAM for wind simulations over rugged terrain is feasible but requires careful planning of simulation cases and their parallelization, starting from small models in order to evaluate the runtime in concrete HPC systems. It is certain that in small HPS systems only low resolution models may be executed.