CLAW FORTRAN Compiler source-to-source translation for performance portability
|
|
|
- Ophelia O’Brien’
- 5 years ago
- Views:
Transcription
1 CLAW FORTRAN Compiler source-to-source translation for performance portability XcalableMP Workshop, Akihabara, Tokyo, Japan October 31, 2017 Valentin Clement Image: NASA
2 Summary Performance portability problem in COSMO Single source code Portability Performance Portability of code CLAW FORTRAN Compiler Single column abstraction Future project Valentin Clement 5 th XcalableMP Workshop - 31st of October
3 GPU machine in Switzerland - Piz Kesch (MeteoSwiss) Production + R&D Each rack is composed of 12 Compute nodes: 2x Intel Haswell E5-2690v3 2.6 GHz 12-core CPUs (total of 24 CPUs) 256 GB 2133 Mhz DDR4 (total of 3TB) 8x Dual NVIDIA TESLA K80 GPU (total of 96 cards GPUs) Valentin Clement 5 th XcalableMP Workshop - 31st of October
4 GPU machine in Switzerland - Piz Daint (CSCS) Cray XC40/XC50 Intel Xeon E v3 2.60GHz, 12 cores, 64GB RAM NVIDIA Tesla P100 Valentin Clement 5 th XcalableMP Workshop - 31st of October
![Performance portability problem - COSMO Radiation 1.2-30% Execution time [s] 0.9 0.6 0.](/docs-images/84/89560689/images/5-2.jpg "3-35% 0 Execution on CPU Haswell E5-2690v3 12 cores Code restructured for CPU Code restructured for GPU with OpenACC Execution on GPU 1/2 NVIDIA Tesla")
5 Performance portability problem - COSMO Radiation % Execution time [s] % 0 Execution on CPU Haswell E5-2690v3 12 cores Code restructured for CPU Code restructured for GPU with OpenACC Execution on GPU 1/2 NVIDIA Tesla K80 Valentin Clement 5 th XcalableMP Workshop - 31st of October
6 Performance portability problem - COSMO Radiation CPU structure DO k=1,nz CALL fct() DO j=1,nproma! 1st loop body DO j=1,nproma! 2nd loop body DO j=1,nproma! 3rd loop body GPU structure!$acc parallel loop DO j=1,nproma!$acc loop DO k=1,nz CALL fct()! 1st loop body! 2nd loop body! 3rd loop body!$acc end parallel Valentin Clement 5 th XcalableMP Workshop - 31st of October
7 How to keep a single source code for everyone Massive code base ( to >1mio LOC) Several architecture specific optimization survive Most of these code base are CPU optimized Not suited for next generation architecture Not suited for massive parallelism Few or no modularity Valentin Clement 5 th XcalableMP Workshop - 31st of October
8 What kind of code base are we dealing with? Global/local area weather forecast model >10 around the world Monster FORTRAN monolithic code Without much modularity So far we investigate: COSMO (Local area model consortium) - Several institution ICON - DWD (German Weather Agency) - Will replace COSMO IFS Current Cycle + FVM - ECMWF - Member state usage Valentin Clement 5 th XcalableMP Workshop - 31st of October
9 What kind of code base are we dealing with? Example of three code base we investigated so far: COSMO ICON IFS Valentin Clement 5 th XcalableMP Workshop - 31st of October
10 COSMO Mode - loc Climate and local area model used by Germany, Switzerland, Russia Language files blank comment code Fortran C/C++ Header C Python Bourne Again Shell Bourne Shell XML CMake make CUDA SUM: Valentin Clement 5 th XcalableMP Workshop - 31st of October
11 ECMWF IFS - loc European Centre for Medium-range weather forecasts - Global Model Language files blank comment code Fortran C/C++ Header Perl SUM: *only source without external modules Valentin Clement 5 th XcalableMP Workshop - 31st of October
12 DWD ICON - loc New German Global model with option to used it as local area model Language files blank comment code Fortran C HTML Fortran Java C/C++ Header Python SUM: Valentin Clement 5 th XcalableMP Workshop - 31st of October
13 Portability of code Current code base Optimized for one architecture Often with old optimizations still in place Not really modular Global fields injected everywhere Future? Modular standalone package that can be plug In different model For different architecture Abstract data layout where it can be done Abstract model specific data, everything passed as arguments. Valentin Clement 5 th XcalableMP Workshop - 31st of October
14 Performance portability - current code Loop structure better for CPUs DO ilev = 1, nlay DO icol = 1, ncol tau_loc(icol,ilev) = max(tau(icol,ilev,igpt) trans(icol,ilev) = exp(-tau_loc(icol,ilev)) DO ilev = nlay, 1, -1 DO icol = 1, ncol radn_dn(icol,ilev,igpt) = trans(icol,ilev) * radn_dn(icol,ilev+1,igpt) DO ilev = 2, nlay + 1 DO icol = 1, ncol radn_up(icol,ilev,igpt) = trans(icol,ilev-1) * radn_up(icol,ilev-1,igpt) ilev -> dependency icol -> no dependency nlay size ~= 100 ncol size >= Valentin Clement 5 th XcalableMP Workshop - 31st of October
15 Performance portability - current code Sometimes using nproma block optimization for better vectorization!$omp parallel default(shared) DO igpt = 1, ngptot, nproma CALL physical_parameteriztaion( )! Code from previous silde!$omp end parallel Valentin Clement 5 th XcalableMP Workshop - 31st of October
16 Performance portability - GPU structured code Loop structure better for GPUs DO icol = 1, ncol DO ilev = 1, nlay tau_loc(icol,ilev) = max(tau(icol,ilev,igpt) trans(icol,ilev) = exp(-tau_loc(icol,ilev)) DO ilev = nlay, 1, -1 radn_dn(icol,ilev,igpt) = trans(icol,ilev) * radn_dn(icol,ilev+1,igpt) DO ilev = 2, nlay + 1 radn_up(icol,ilev,igpt) = trans(icol,ilev-1) * radn_up(icol,ilev-1,igpt) ilev -> dependency icol -> no dependency nlay size ~= 100 ncol size >= Valentin Clement 5 th XcalableMP Workshop - 31st of October
17 Performance portability - next architecture - What is the best loop structure/data layout for next architecture? - Do we want to rewrite the code each time? - Do we know exactly which architecture we will run on?? Valentin Clement 5 th XcalableMP Workshop - 31st of October
18 What is the CLAW FORTRAN Compiler? Source-to-source translation for FORTRAN code Based on XcodeML/F IR Using OMNI Compiler front-end and back-end Contribution to it via GitHub Transformation of AST Different transformation applied based on target Promotion of scalar and arrays Insertion of iteration Insertion of OpenACC and OpenMP directives Valentin Clement 5 th XcalableMP Workshop - 31st of October
19 CLAW FORTRAN Compiler under the hood Based on the OMNI Compiler FORTRAN front-end & back-end Source-to-source translator Open source under the BSD license Available on GitHub with the specifications.f90.f90 FPP F_Front CX2X F_Back CLAWFC Valentin Clement 5 th XcalableMP Workshop - 31st of October
20 Single column abstraction Separation of concerns Domain scientists focus on their problem (1 column, 1 box) CLAW compiler produce code for each target and directive languages Achieve modularity Standalone physical parameter Modular from model specificity Valentin Clement 5 th XcalableMP Workshop - 31st of October
21 RRTMGP Example - A nice modular code CPU structured F2003 radiation code From Robert Pincus and al. from AER University of Colorado Compute intensive part are well located in kernel module. Code is non-the-less CPU structured with horizontal loop as the inner most in every iteration. Valentin Clement 5 th XcalableMP Workshop - 31st of October
22 RRTMGP Example - original code - CPU structured Loop over spectral intervals Loop over vertical dimension Loop over horizontal dimension SUBROUTINE lw_solver(ngpt, nlay, tau, )! DECLARATION PART OMITTED DO igpt = 1, ngpt DO ilev = 1, nlay DO icol = 1, ncol tau_loc(icol,ilev) = max(tau(icol,ilev,igpt) trans(icol,ilev) = exp(-tau_loc(icol,ilev)) DO ilev = nlay, 1, -1 DO icol = 1, ncol radn_dn(icol,ilev,igpt) = trans(icol,ilev) * radn_dn(icol,ilev+1,igpt) DO ilev = 2, nlay + 1 DO icol = 1, ncol radn_up(icol,ilev,igpt) = trans(icol,ilev-1) * radn_up(icol,ilev-1,igpt) radn_up(:,:,:) = 2._wp * pi * quad_wt * radn_up(:,:,:) radn_dn(:,:,:) = 2._wp * pi * quad_wt * radn_dn(:,:,:) END SUBROUTINE lw_solver Valentin Clement 5 th XcalableMP Workshop - 31st of October
23 RRTMGP Example - Single column abstraction SUBROUTINE lw_solver(ngpt, nlay, tau, ) Only dependency on these iteration spaces! DECLARATION DECL: Fields PART don t OMITTED have the horizontal dimension (demotion) DO igpt = 1, ngpt DO ilev = 1, nlay tau_loc(ilev) = max(tau(ilev,igpt) trans(ilev) = exp(-tau_loc(ilev)) DO ilev = nlay, 1, -1 radn_dn(ilev,igpt) = trans(ilev) * radn_dn(ilev+1,igpt) DO ilev = 2, nlay + 1 radn_up(ilev,igpt) = trans(ilev-1) * radn_up(ilev-1,igpt) radn_up(:,:) = 2._wp * pi * quad_wt * radn_up(:,:) radn_dn(:,:) = 2._wp * pi * quad_wt * radn_dn(:,:) END SUBROUTINE lw_solver Valentin Clement 5 th XcalableMP Workshop - 31st of October
24 RRTMGP Example - CLAW code Algorithm for one column only SUBROUTINE lw_solver(ngpt, nlay, tau, )!$claw parallelize! model dimension info located in config DO igpt = 1, ngpt DO ilev = 1, nlay tau_loc(ilev) = max(tau(ilev,igpt) trans(ilev) = exp(-tau_loc(ilev)) DO ilev = nlay, 1, -1 radn_dn(ilev,igpt) = trans(ilev) * radn_dn(ilev+1,igpt) DO ilev = 2, nlay + 1 radn_up(ilev,igpt) = trans(ilev-1) * radn_up(ilev-1,igpt) radn_up(:,:) = 2._wp * pi * quad_wt * radn_up(:,:) radn_dn(:,:) = 2._wp * pi * quad_wt * radn_dn(:,:) END SUBROUTINE lw_solver Dependency on the vertical dimension only Valentin Clement 5 th XcalableMP Workshop - 31st of October
25 RRTMGP Example - CLAW transformation Original code (Architecture agnostic) CLAWFC.f90 A single source code Specify a target architecture for the transformation Specify a compiler directives language to be added CPU OpenMP.f90 MIC OpenMP.f90 GPU OpenACC.f90 Automatically transformed code clawfc --directive=openacc --target=gpu -o mo_lw_solver.acc.f90 mo_lw_solver.f90 clawfc --directive=openmp --target=cpu -o mo_lw_solver.omp.f90 mo_lw_solver.f90 clawfc --directive=openmp --target=mic -o mo_lw_solver.mic.f90 mo_lw_solver.f90 Valentin Clement 5 th XcalableMP Workshop - 31st of October
26 CLAW - One column - OpenACC - local array strategy Data analysis for generation of OpenACC directives Potentially collapsing loops Generate data transfer if wanted Adapt data layout Promotion of scalar and array fields with model dimensions Detect unsupported statements for OpenACC Insertion of do statements to iterate of new dimensions Insertion of directives (OpenMP/OpenACC) Valentin Clement 5 th XcalableMP Workshop - 31st of October
27 RRTMGP Example - GPU w/ OpenACC SUBROUTINE lw_solver(ngpt, nlay, tau, )! DECL: Fields promoted accordingly to usage!$acc data present( )!$acc parallel!$acc loop gang vector private( ) collapse(2) DO icol = 1, ncol, 1 DO igpt = 1, ngpt, 1!$acc loop seq DO ilev = 1, nlay, 1 tau_loc(ilev) = max(tau(icol,ilev,igpt) trans(ilev) = exp(-tau_loc(ilev))!$acc loop seq DO ilev = nlay, 1, (-1) radn_dn(icol,ilev,igpt) = trans(ilev) * radn_dn(icol,ilev+1,igpt)!$acc loop seq DO ilev = 2, nlay + 1, 1 radn_up(icol,ilev,igpt) = trans(ilev-1)*radn_up(icol,ilev-1,igpt)!$acc loop seq DO igpt = 1, ngpt, 1!$acc loop seq DO ilev = 1, nlay + 1, 1 radn_up(icol,igpt,ilev) = 2._wp * pi * quad_wt * radn_up(icol,igpt,ilev) radn_dn(icol,igpt,ilev) = 2._wp * pi * quad_wt * radn_dn(icol,igpt,ilev)!$acc end parallel!$acc end data END SUBROUTINE lw_solver Valentin Clement 5 th XcalableMP Workshop - 31st of October
28 CLAW - One column - OpenACC - local array strategy Example of different strategy easy to test with an automatize workflow: 1.Privatize local arrays Make local arrays private (unsupported for allocatable arrays) 2.Promote arrays Reduce allocation overhead Valentin Clement 5 th XcalableMP Workshop - 31st of October
29 RRTMGP lw_solver - Original vs. CLAW CPU/OpenMP RRTMGP lw_solver comparison of different kernel version / Domain size: 100x100x42 Piz Kesch (Haswell E5-2690v3 12 cores vs. 1/2 NVIDIA Tesla K80) PGI Reference: original source code on 1-core 8 8.1x 6 5 Speedup x 0.5x Original code One column abstraction CLAW CPU/OpenMP (12cores) Valentin Clement 5 th XcalableMP Workshop - 31st of October
30 RRTMGP lw_solver - CLAW CPU vs. CLAW GPU Comparison of different kernel version / Domain Size100x100x42 Piz Kesch (Haswell E5-2690v3 12 cores vs. 1/2 NVIDIA Tesla K80) PGI Reference: CLAW CPU/OpenMP 12-cores 6 6.1x 5 4 Speedup x 0 CLAW CPU/OpenMP (12cores) CLAW GPU/OpenACC (0.5K80) Valentin Clement 5 th XcalableMP Workshop - 31st of October
31 RRTMGP lw_solver - CLAW vs. GridTools 6 Comparison of different kernel version / Domain Size100x100x42 Piz Kesch (Haswell E5-2690v3 12 cores vs. 1/2 NVIDIA Tesla K80) PGI Reference: CLAW CPU/OpenMP 12-cores 5 4 CLAW (Cray 8.4.4) CLAW (PGI 16.3) GridTools (GNU + NVCC) 4.6x 5.0x Speedup 2 3.3x 1 1.0x 0 0.6x 0.5x CLAW (Cray 8.4.4) CLAW (PGI 16.3) GridTools (GNU + NVCC) CLAW (Cray 8.4.4) CLAW (PGI 16.3) GridTools (GNU + NVCC) Haswell E5-2690v3 12 cores 1/2 NVIDIA Telse K80 Valentin Clement 5 th XcalableMP Workshop - 31st of October
32 IFS-CloudSC - one column version CloudMircophysics Scheme Take less than a day to create a one column version Can play with it and apply different strategy OpenACC privatization of local arrays OpenACC promotion of local arrays Valentin Clement 5 th XcalableMP Workshop - 31st of October
33 IFS-CloudSC - one column version: early results Comparison of different kernel version / Domain Size: 16000x137 Piz Daint (Haswell E5-2690v3 12 cores vs. NVIDIA Tesla P100) Cray Reference: OpenMP 12-cores Speedup x 2.9x x 0 OpenMP / Block CLAW GPU/OpenACC (private) CLAW GPU/OpenACC (promote) Valentin Clement ECMWF / UKMO / STFC 33
34 Portability with performance ECMWF IFS Operational DyCore data layout: nproma, level, block IFS Physical Parameterizations data layout: nproma, level ECMWF FVM data layout: jlevel, jnode Valentin Clement 5 th XcalableMP Workshop - 31st of October
35 Portability with performance ECMWF IFS Operational DyCore data layout: nproma, level, block IFS Physical Parameterizations data layout: nproma, level model cfg IFS Physical Parameterizations data layout: level ECMWF FVM data layout: jlevel, jnode IFS Physical Parameterizations data layout: level, jnode model cfg Valentin Clement 5 th XcalableMP Workshop - 31st of October
36 Portability with performance Transformed code might be the same More parallelism on outer loop for GPU is better Independent from data layout Might have to introduce copy Spending small time copying to new data layout might worth in overall performance Valentin Clement 5 th XcalableMP Workshop - 31st of October
37 Portability with performance Changement of data layout with copy % Original layout Oprmized layout Oprmized layout w/ copy Different layout Valentin Clement 5 th XcalableMP Workshop - 31st of October
38 CLAW collaboration with the OMNI Compiler Project Only viable FORTRAN source-to-source framework Only one currently maintained Very responsive people Accept Pull Requests 61 issues opened as today -> 51 closed 61 PR as today -> 57 closed Open source takes some effort but it is rewarding! Valentin Clement 5 th XcalableMP Workshop - 31st of October
39 ENIAC Project ( ) Enabling ICON model on heterogenous architecture Port to OpenACC GridTools for stencil computation (DyCore) Looking at performance portability in FORTRAN code Enhance CLAW FORTRAN Compiler capabilities Move physical parameterization to single column Getting more numbers :-) Apply transformation for x86, XeonPhi and GPUs Valentin Clement 5 th XcalableMP Workshop - 31st of October
40 EuroExa Project Machine will be hosted at STFC in UK ARM processor node Loaded with FPGA ECMWF will investigate single column abstraction Specific transformation for ARM processor Mabye automatic offloading to FPGA (FORTRAN to C translation) Valentin Clement 5 th XcalableMP Workshop - 31st of October
41 Only a problem of MeteoSwiss? Valentin Clement 5 th XcalableMP Workshop - 31st of October
42 Possible future collaboration Valentin Clement 5 th XcalableMP Workshop - 31st of October
43 CLAW FORTRAN Compiler - Resources CLAW FORTRAN Compiler developer s guide Valentin Clement 5 th XcalableMP Workshop - 31st of October
44 Valentin Clement 5 th XcalableMP Workshop - 31st of October
CLAW FORTRAN Compiler Abstractions for Weather and Climate Models
CLAW FORTRAN Compiler Abstractions for Weather and Climate Models Image: NASA PASC 17 June 27, 2017 Valentin Clement, Jon Rood, Sylvaine Ferrachat, Will Sawyer, Oliver Fuhrer, Xavier Lapillonne valentin.clement@env.ethz.ch
Deutscher Wetterdienst
Porting Operational Models to Multi- and Many-Core Architectures Ulrich Schättler Deutscher Wetterdienst Oliver Fuhrer MeteoSchweiz Xavier Lapillonne MeteoSchweiz Contents Strong Scalability of the Operational
Deutscher Wetterdienst
Accelerating Work at DWD Ulrich Schättler Deutscher Wetterdienst Roadmap Porting operational models: revisited Preparations for enabling practical work at DWD My first steps with the COSMO on a GPU First
The CLAW project. Valentin Clément, Xavier Lapillonne. CLAW provides high-level Abstractions for Weather and climate models
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss The CLAW project CLAW provides high-level Abstractions for Weather and climate models Valentin Clément,
Adapting Numerical Weather Prediction codes to heterogeneous architectures: porting the COSMO model to GPUs
Adapting Numerical Weather Prediction codes to heterogeneous architectures: porting the COSMO model to GPUs O. Fuhrer, T. Gysi, X. Lapillonne, C. Osuna, T. Dimanti, T. Schultess and the HP2C team Eidgenössisches
Porting COSMO to Hybrid Architectures
Porting COSMO to Hybrid Architectures T. Gysi 1, O. Fuhrer 2, C. Osuna 3, X. Lapillonne 3, T. Diamanti 3, B. Cumming 4, T. Schroeder 5, P. Messmer 5, T. Schulthess 4,6,7 [1] Supercomputing Systems AG,
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss. PP POMPA status.
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss PP POMPA status Xavier Lapillonne Performance On Massively Parallel Architectures Last year of the project
NVIDIA Update and Directions on GPU Acceleration for Earth System Models
NVIDIA Update and Directions on GPU Acceleration for Earth System Models Stan Posey, HPC Program Manager, ESM and CFD, NVIDIA, Santa Clara, CA, USA Carl Ponder, PhD, Applications Software Engineer, NVIDIA,
Using OpenACC in IFS Physics Cloud Scheme (CLOUDSC) Sami Saarinen ECMWF Basic GPU Training Sept 16-17, 2015
Using OpenACC in IFS Physics Cloud Scheme (CLOUDSC) Sami Saarinen ECMWF Basic GPU Training Sept 16-17, 2015 Slide 1 Background Back in 2014 : Adaptation of IFS physics cloud scheme (CLOUDSC) to new architectures
INTRODUCTION TO OPENACC. Analyzing and Parallelizing with OpenACC, Feb 22, 2017
INTRODUCTION TO OPENACC Analyzing and Parallelizing with OpenACC, Feb 22, 2017 Objective: Enable you to to accelerate your applications with OpenACC. 2 Today s Objectives Understand what OpenACC is and
GPU Developments for the NEMO Model. Stan Posey, HPC Program Manager, ESM Domain, NVIDIA (HQ), Santa Clara, CA, USA
GPU Developments for the NEMO Model Stan Posey, HPC Program Manager, ESM Domain, NVIDIA (HQ), Santa Clara, CA, USA NVIDIA HPC AND ESM UPDATE TOPICS OF DISCUSSION GPU PROGRESS ON NEMO MODEL 2 NVIDIA GPU
Physical parametrizations and OpenACC directives in COSMO
Physical parametrizations and OpenACC directives in COSMO Xavier Lapillonne Eidgenössisches Departement des Innern EDI Bundesamt für Meteorologie und Klimatologie MeteoSchweiz Name (change on Master slide)
Porting the ICON Non-hydrostatic Dynamics and Physics to GPUs
Porting the ICON Non-hydrostatic Dynamics and Physics to GPUs William Sawyer (CSCS/ETH), Christian Conti (ETH), Xavier Lapillonne (C2SM/ETH) Programming weather, climate, and earth-system models on heterogeneous
Using EasyBuild and Continuous Integration for Deploying Scientific Applications on Large Scale Production Systems
Using EasyBuild and Continuous Integration for Deploying Scientific Applications on Large HPC Advisory Council Swiss Conference Guilherme Peretti-Pezzi, CSCS April 11, 2017 Table of Contents 1. Introduction:
Accelerating Financial Applications on the GPU
Accelerating Financial Applications on the GPU Scott Grauer-Gray Robert Searles William Killian John Cavazos Department of Computer and Information Science University of Delaware Sixth Workshop on General
Stan Posey, NVIDIA, Santa Clara, CA, USA
Stan Posey, sposey@nvidia.com NVIDIA, Santa Clara, CA, USA NVIDIA Strategy for CWO Modeling (Since 2010) Initial focus: CUDA applied to climate models and NWP research Opportunities to refactor code with
GPUs and Emerging Architectures
GPUs and Emerging Architectures Mike Giles mike.giles@maths.ox.ac.uk Mathematical Institute, Oxford University e-infrastructure South Consortium Oxford e-research Centre Emerging Architectures p. 1 CPUs
Experiences with CUDA & OpenACC from porting ACME to GPUs
Experiences with CUDA & OpenACC from porting ACME to GPUs Matthew Norman Irina Demeshko Jeffrey Larkin Aaron Vose Mark Taylor ORNL is managed by UT-Battelle for the US Department of Energy ORNL Sandia
GPU Developments in Atmospheric Sciences
GPU Developments in Atmospheric Sciences Stan Posey, HPC Program Manager, ESM Domain, NVIDIA (HQ), Santa Clara, CA, USA David Hall, Ph.D., Sr. Solutions Architect, NVIDIA, Boulder, CO, USA NVIDIA HPC UPDATE
Can Accelerators Really Accelerate Harmonie?
Can Accelerators Really Accelerate Harmonie? Enda O Brien, Adam Ralph Irish Centre for High-End Computing Motivation There is constant demand for more performance Conventional compute cores not getting
The challenges of new, efficient computer architectures, and how they can be met with a scalable software development strategy.! Thomas C.
The challenges of new, efficient computer architectures, and how they can be met with a scalable software development strategy! Thomas C. Schulthess ENES HPC Workshop, Hamburg, March 17, 2014 T. Schulthess!1
S Comparing OpenACC 2.5 and OpenMP 4.5
April 4-7, 2016 Silicon Valley S6410 - Comparing OpenACC 2.5 and OpenMP 4.5 James Beyer, NVIDIA Jeff Larkin, NVIDIA GTC16 April 7, 2016 History of OpenMP & OpenACC AGENDA Philosophical Differences Technical
An OpenACC GPU adaptation of the IFS cloud microphysics scheme
805 An OpenACC GPU adaptation of the IFS cloud microphysics scheme Huadong Xiao 1, Michail Diamantakis and Sami Saarinen Research Department 1 Visiting Scientist, National Meteorological Information Center,
OpenACC (Open Accelerators - Introduced in 2012)
OpenACC (Open Accelerators - Introduced in 2012) Open, portable standard for parallel computing (Cray, CAPS, Nvidia and PGI); introduced in 2012; GNU has an incomplete implementation. Uses directives in
Accelerator programming with OpenACC
..... Accelerator programming with OpenACC Colaboratorio Nacional de Computación Avanzada Jorge Castro jcastro@cenat.ac.cr 2018. Agenda 1 Introduction 2 OpenACC life cycle 3 Hands on session Profiling
A Simulation of Global Atmosphere Model NICAM on TSUBAME 2.5 Using OpenACC
A Simulation of Global Atmosphere Model NICAM on TSUBAME 2.5 Using OpenACC Hisashi YASHIRO RIKEN Advanced Institute of Computational Science Kobe, Japan My topic The study for Cloud computing My topic
Our Workshop Environment
Our Workshop Environment John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center Copyright 2018 Our Environment Today Your laptops or workstations: only used for portal access Bridges
PLAN-E Workshop Switzerland. Welcome! September 8, 2016
PLAN-E Workshop Switzerland Welcome! September 8, 2016 The Swiss National Supercomputing Centre Driving innovation in computational research in Switzerland Michele De Lorenzi (CSCS) PLAN-E September 8,
Carlos Osuna. Meteoswiss.
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss DSL Toolchains for Performance Portable Geophysical Fluid Dynamic Models Carlos Osuna Meteoswiss carlos.osuna@meteoswiss.ch
Optimizing Weather Model Radiative Transfer Physics for the Many Integrated Core and GPGPU Architectures
Optimizing Weather Model Radiative Transfer Physics for the Many Integrated Core and GPGPU Architectures John Michalakes NOAA/NCEP/Environmental Modeling Center (IM Systems Group) University of Colorado
IFS RAPS14 benchmark on 2 nd generation Intel Xeon Phi processor
IFS RAPS14 benchmark on 2 nd generation Intel Xeon Phi processor D.Sc. Mikko Byckling 17th Workshop on High Performance Computing in Meteorology October 24 th 2016, Reading, UK Legal Disclaimer & Optimization
COMP Parallel Computing. Programming Accelerators using Directives
COMP 633 - Parallel Computing Lecture 15 October 30, 2018 Programming Accelerators using Directives Credits: Introduction to OpenACC and toolkit Jeff Larkin, Nvidia COMP 633 - Prins Directives for Accelerator
OpenACC/CUDA/OpenMP... 1 Languages and Libraries... 3 Multi-GPU support... 4 How OpenACC Works... 4
OpenACC Course Class #1 Q&A Contents OpenACC/CUDA/OpenMP... 1 Languages and Libraries... 3 Multi-GPU support... 4 How OpenACC Works... 4 OpenACC/CUDA/OpenMP Q: Is OpenACC an NVIDIA standard or is it accepted
CMSC 714 Lecture 6 MPI vs. OpenMP and OpenACC. Guest Lecturer: Sukhyun Song (original slides by Alan Sussman)
CMSC 714 Lecture 6 MPI vs. OpenMP and OpenACC Guest Lecturer: Sukhyun Song (original slides by Alan Sussman) Parallel Programming with Message Passing and Directives 2 MPI + OpenMP Some applications can
Running the FIM and NIM Weather Models on GPUs
Running the FIM and NIM Weather Models on GPUs Mark Govett Tom Henderson, Jacques Middlecoff, Jim Rosinski, Paul Madden NOAA Earth System Research Laboratory Global Models 0 to 14 days 10 to 30 KM resolution
Profiling and Parallelizing with the OpenACC Toolkit OpenACC Course: Lecture 2 October 15, 2015
Profiling and Parallelizing with the OpenACC Toolkit OpenACC Course: Lecture 2 October 15, 2015 Oct 1: Introduction to OpenACC Oct 6: Office Hours Oct 15: Profiling and Parallelizing with the OpenACC Toolkit
RAMSES on the GPU: An OpenACC-Based Approach
RAMSES on the GPU: An OpenACC-Based Approach Claudio Gheller (ETHZ-CSCS) Giacomo Rosilho de Souza (EPFL Lausanne) Romain Teyssier (University of Zurich) Markus Wetzstein (ETHZ-CSCS) PRACE-2IP project EU
Miwako TSUJI XcalableMP
Miwako TSUJI AICS 2014.10.24 2 XcalableMP 2010.09 2014.03 2013.10.25 AKIHABARA FP2C (Framework for Post-Petascale Computing) YML + XMP(-dev) + StarPU integrated developed in Japan and in France Experimental
CSCS Proposal writing webinar Technical review. 12th April 2015 CSCS
CSCS Proposal writing webinar Technical review 12th April 2015 CSCS Agenda Tips for new applicants CSCS overview Allocation process Guidelines Basic concepts Performance tools Demo Q&A open discussion
INTRODUCTION TO COMPILER DIRECTIVES WITH OPENACC
INTRODUCTION TO COMPILER DIRECTIVES WITH OPENACC DR. CHRISTOPH ANGERER, NVIDIA *) THANKS TO JEFF LARKIN, NVIDIA, FOR THE SLIDES 3 APPROACHES TO GPU PROGRAMMING Applications Libraries Compiler Directives
ECE 574 Cluster Computing Lecture 18
ECE 574 Cluster Computing Lecture 18 Vince Weaver http://web.eece.maine.edu/~vweaver vincent.weaver@maine.edu 2 April 2019 HW#8 was posted Announcements 1 Project Topic Notes I responded to everyone s
OpenStaPLE, an OpenACC Lattice QCD Application
OpenStaPLE, an OpenACC Lattice QCD Application Enrico Calore Postdoctoral Researcher Università degli Studi di Ferrara INFN Ferrara Italy GTC Europe, October 10 th, 2018 E. Calore (Univ. and INFN Ferrara)
Our Workshop Environment
Our Workshop Environment John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center Copyright 2017 Our Environment This Week Your laptops or workstations: only used for portal access Bridges
Status of the COSMO GPU version
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Status of the COSMO GPU version Xavier Lapillonne Contributors in 2015 (Thanks!) Alon Shtivelman Andre Walser
NVIDIA Think about Computing as Heterogeneous One Leo Liao, 1/29/2106, NTU
NVIDIA Think about Computing as Heterogeneous One Leo Liao, 1/29/2106, NTU GPGPU opens the door for co-design HPC, moreover middleware-support embedded system designs to harness the power of GPUaccelerated
An Extension of XcalableMP PGAS Lanaguage for Multi-node GPU Clusters
An Extension of XcalableMP PGAS Lanaguage for Multi-node Clusters Jinpil Lee, Minh Tuan Tran, Tetsuya Odajima, Taisuke Boku and Mitsuhisa Sato University of Tsukuba 1 Presentation Overview l Introduction
INTRODUCTION TO ACCELERATED COMPUTING WITH OPENACC. Jeff Larkin, NVIDIA Developer Technologies
INTRODUCTION TO ACCELERATED COMPUTING WITH OPENACC Jeff Larkin, NVIDIA Developer Technologies AGENDA Accelerated Computing Basics What are Compiler Directives? Accelerating Applications with OpenACC Identifying
Our Workshop Environment
Our Workshop Environment John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center Copyright 2016 Our Environment This Week Your laptops or workstations: only used for portal access Bridges
The Design and Implementation of OpenMP 4.5 and OpenACC Backends for the RAJA C++ Performance Portability Layer
The Design and Implementation of OpenMP 4.5 and OpenACC Backends for the RAJA C++ Performance Portability Layer William Killian Tom Scogland, Adam Kunen John Cavazos Millersville University of Pennsylvania
OpenACC Standard. Credits 19/07/ OpenACC, Directives for Accelerators, Nvidia Slideware
OpenACC Standard Directives for Accelerators Credits http://www.openacc.org/ o V1.0: November 2011 Specification OpenACC, Directives for Accelerators, Nvidia Slideware CAPS OpenACC Compiler, HMPP Workbench
Comparing OpenACC 2.5 and OpenMP 4.1 James C Beyer PhD, Sept 29 th 2015
Comparing OpenACC 2.5 and OpenMP 4.1 James C Beyer PhD, Sept 29 th 2015 Abstract As both an OpenMP and OpenACC insider I will present my opinion of the current status of these two directive sets for programming
Porting the ICON Non-hydrostatic Dynamics to GPUs
Porting the ICON Non-hydrostatic Dynamics to GPUs William Sawyer (CSCS/ETH), Christian Conti (ETH), Gilles Fourestey (CSCS/ETH) IS-ENES Workshop on Dynamical Cores Dec. 14-16, 2011, Carlo V Castle, Lecce,
PP POMPA (WG6) News and Highlights. Oliver Fuhrer (MeteoSwiss) and the whole POMPA project team. COSMO GM13, Sibiu
PP POMPA (WG6) News and Highlights Oliver Fuhrer (MeteoSwiss) and the whole POMPA project team COSMO GM13, Sibiu Task Overview Task 1 Performance analysis and documentation Task 2 Redesign memory layout
Piz Daint: Application driven co-design of a supercomputer based on Cray s adaptive system design
Piz Daint: Application driven co-design of a supercomputer based on Cray s adaptive system design Sadaf Alam & Thomas Schulthess CSCS & ETHzürich CUG 2014 * Timelines & releases are not precise Top 500
arxiv: v1 [hep-lat] 12 Nov 2013
![arxiv: v1 [hep-lat] 12 Nov 2013](/thumbs/87/96002319.jpg "arxiv: v1 [hep-lat] 12 Nov 2013")
Lattice Simulations using OpenACC compilers arxiv:13112719v1 [hep-lat] 12 Nov 2013 Indian Association for the Cultivation of Science, Kolkata E-mail: tppm@iacsresin OpenACC compilers allow one to use Graphics
Illinois Proposal Considerations Greg Bauer
- 2016 Greg Bauer Support model Blue Waters provides traditional Partner Consulting as part of its User Services. Standard service requests for assistance with porting, debugging, allocation issues, and
OpenACC Course Lecture 1: Introduction to OpenACC September 2015
OpenACC Course Lecture 1: Introduction to OpenACC September 2015 Course Objective: Enable you to accelerate your applications with OpenACC. 2 Oct 1: Introduction to OpenACC Oct 6: Office Hours Oct 15:
A case study of performance portability with OpenMP 4.5
A case study of performance portability with OpenMP 4.5 Rahul Gayatri, Charlene Yang, Thorsten Kurth, Jack Deslippe NERSC pre-print copy 1 Outline General Plasmon Pole (GPP) application from BerkeleyGW
Improving the Energy- and Time-to-solution of COSMO-ART
Joseph Charles, William Sawyer (ETH Zurich - CSCS) Heike Vogel (KIT), Bernhard Vogel (KIT), Teresa Beck (KIT/UHEI) COSMO User Workshop, MeteoSwiss January 18, 2016 Summary 2 Main Objectives Utilise project
OpenACC Fundamentals. Steve Abbott November 13, 2016
OpenACC Fundamentals Steve Abbott , November 13, 2016 Who Am I? 2005 B.S. Physics Beloit College 2007 M.S. Physics University of Florida 2015 Ph.D. Physics University of New Hampshire
GPU Acceleration of the Longwave Rapid Radiative Transfer Model in WRF using CUDA Fortran. G. Ruetsch, M. Fatica, E. Phillips, N.
GPU Acceleration of the Longwave Rapid Radiative Transfer Model in WRF using CUDA Fortran G. Ruetsch, M. Fatica, E. Phillips, N. Juffa Outline WRF and RRTM Previous Work CUDA Fortran Features RRTM in CUDA
OpenACC. Part I. Ned Nedialkov. McMaster University Canada. October 2016
OpenACC. Part I Ned Nedialkov McMaster University Canada October 2016 Outline Introduction Execution model Memory model Compiling pgaccelinfo Example Speedups Profiling c 2016 Ned Nedialkov 2/23 Why accelerators
Lecture: Manycore GPU Architectures and Programming, Part 4 -- Introducing OpenMP and HOMP for Accelerators
Lecture: Manycore GPU Architectures and Programming, Part 4 -- Introducing OpenMP and HOMP for Accelerators CSCE 569 Parallel Computing Department of Computer Science and Engineering Yonghong Yan yanyh@cse.sc.edu
Optimised all-to-all communication on multicore architectures applied to FFTs with pencil decomposition
Optimised all-to-all communication on multicore architectures applied to FFTs with pencil decomposition CUG 2018, Stockholm Andreas Jocksch, Matthias Kraushaar (CSCS), David Daverio (University of Cambridge,
OpenACC compiling and performance tips. May 3, 2013
OpenACC compiling and performance tips May 3, 2013 OpenACC compiler support Cray Module load PrgEnv-cray craype-accel-nvidia35 Fortran -h acc, noomp # openmp is enabled by default, be careful mixing -fpic
Addressing the Increasing Challenges of Debugging on Accelerated HPC Systems. Ed Hinkel Senior Sales Engineer
Addressing the Increasing Challenges of Debugging on Accelerated HPC Systems Ed Hinkel Senior Sales Engineer Agenda Overview - Rogue Wave & TotalView GPU Debugging with TotalView Nvdia CUDA Intel Phi 2
Progress on GPU Parallelization of the NIM Prototype Numerical Weather Prediction Dynamical Core
Progress on GPU Parallelization of the NIM Prototype Numerical Weather Prediction Dynamical Core Tom Henderson NOAA/OAR/ESRL/GSD/ACE Thomas.B.Henderson@noaa.gov Mark Govett, Jacques Middlecoff Paul Madden,
Programming paradigms for GPU devices
Programming paradigms for GPU devices OpenAcc Introduction Sergio Orlandini s.orlandini@cineca.it 1 OpenACC introduction express parallelism optimize data movements practical examples 2 3 Ways to Accelerate
Evaluating the Performance and Energy Efficiency of the COSMO-ART Model System
Evaluating the Performance and Energy Efficiency of the COSMO-ART Model System Joseph Charles & William Sawyer (CSCS), Manuel F. Dolz (UHAM), Sandra Catalán (UJI) EnA-HPC, Dresden September 1-2, 2014 1
GPU. OpenMP. OMPCUDA OpenMP. forall. Omni CUDA 3) Global Memory OMPCUDA. GPU Thread. Block GPU Thread. Vol.2012-HPC-133 No.
GPU CUDA OpenMP 1 2 3 1 1 OpenMP CUDA OM- PCUDA OMPCUDA GPU CUDA CUDA 1. GPU GPGPU 1)2) GPGPU CUDA 3) CPU CUDA GPGPU CPU GPU OpenMP GPU CUDA OMPCUDA 4)5) OMPCUDA GPU OpenMP GPU CUDA OMPCUDA/MG 2 GPU OMPCUDA
OpenACC Course. Office Hour #2 Q&A
OpenACC Course Office Hour #2 Q&A Q1: How many threads does each GPU core have? A: GPU cores execute arithmetic instructions. Each core can execute one single precision floating point instruction per cycle
Accelerating Harmonie with GPUs (or MICs)
Accelerating Harmonie with GPUs (or MICs) (A view from the starting-point) Enda O Brien, Adam Ralph Irish Centre for High-End Computing Motivation There is constant, insatiable demand for more performance
Titan - Early Experience with the Titan System at Oak Ridge National Laboratory
Office of Science Titan - Early Experience with the Titan System at Oak Ridge National Laboratory Buddy Bland Project Director Oak Ridge Leadership Computing Facility November 13, 2012 ORNL s Titan Hybrid
KernelGen a toolchain for automatic GPU-centric applications porting. Nicolas Lihogrud Dmitry Mikushin Andrew Adinets
P A R A L L E L C O M P U T A T I O N A L T E C H N O L O G I E S ' 2 0 1 2 KernelGen a toolchain for automatic GPU-centric applications porting Nicolas Lihogrud Dmitry Mikushin Andrew Adinets Contents
COSMO Dynamical Core Redesign Tobias Gysi David Müller Boulder,
COSMO Dynamical Core Redesign Tobias Gysi David Müller Boulder, 8.9.2011 Supercomputing Systems AG Technopark 1 8005 Zürich 1 Fon +41 43 456 16 00 Fax +41 43 456 16 10 www.scs.ch Boulder, 8.9.2011, by
Portability of OpenMP Offload Directives Jeff Larkin, OpenMP Booth Talk SC17
Portability of OpenMP Offload Directives Jeff Larkin, OpenMP Booth Talk SC17 11/27/2017 Background Many developers choose OpenMP in hopes of having a single source code that runs effectively anywhere (performance
Programming NVIDIA GPUs with OpenACC Directives
Programming NVIDIA GPUs with OpenACC Directives Michael Wolfe michael.wolfe@pgroup.com http://www.pgroup.com/accelerate Programming NVIDIA GPUs with OpenACC Directives Michael Wolfe mwolfe@nvidia.com http://www.pgroup.com/accelerate
Evaluation of Asynchronous Offloading Capabilities of Accelerator Programming Models for Multiple Devices
Evaluation of Asynchronous Offloading Capabilities of Accelerator Programming Models for Multiple Devices Jonas Hahnfeld 1, Christian Terboven 1, James Price 2, Hans Joachim Pflug 1, Matthias S. Müller
The Rise of Open Programming Frameworks. JC BARATAULT IWOCL May 2015
The Rise of Open Programming Frameworks JC BARATAULT IWOCL May 2015 1,000+ OpenCL projects SourceForge GitHub Google Code BitBucket 2 TUM.3D Virtual Wind Tunnel 10K C++ lines of code, 30 GPU kernels CUDA
Early Experiences with the OpenMP Accelerator Model
Early Experiences with the OpenMP Accelerator Model Canberra, Australia, IWOMP 2013, Sep. 17th * University of Houston LLNL-PRES- 642558 This work was performed under the auspices of the U.S. Department
Using GPUs for ICON: An MPI and OpenACC Implementation
Using GPUs for ICON: An MPI and OpenACC Implementation William Sawyer, Jeff Poznanovic, Leonidas Linardakis (MPI-M), Christian Conti (ETH), Gilles Fourestey Earth System Models on Heterogeneous Multicore
PERFORMANCE PORTABILITY WITH OPENACC. Jeff Larkin, NVIDIA, November 2015
PERFORMANCE PORTABILITY WITH OPENACC Jeff Larkin, NVIDIA, November 2015 TWO TYPES OF PORTABILITY FUNCTIONAL PORTABILITY PERFORMANCE PORTABILITY The ability for a single code to run anywhere. The ability
Lattice Simulations using OpenACC compilers. Pushan Majumdar (Indian Association for the Cultivation of Science, Kolkata)
Lattice Simulations using OpenACC compilers Pushan Majumdar (Indian Association for the Cultivation of Science, Kolkata) OpenACC is a programming standard for parallel computing developed by Cray, CAPS,
CPU+GPU Programming for ASUCA. Production Weather Model
Unified CPU+GPU Programming for the ASUCA Production Weather Model Michel Müller Research Assistant, Aoki Laboratory michel@sim.gsic.titech.ac.jp Supervised by Prof. Dr. Takayuki Aoki Tokyo Institute of
Directive-based Programming for Highly-scalable Nodes
Directive-based Programming for Highly-scalable Nodes Doug Miles Michael Wolfe PGI Compilers & Tools NVIDIA Cray User Group Meeting May 2016 Talk Outline Increasingly Parallel Nodes Exposing Parallelism
OPENMP GPU OFFLOAD IN FLANG AND LLVM. Guray Ozen, Simone Atzeni, Michael Wolfe Annemarie Southwell, Gary Klimowicz
OPENMP GPU OFFLOAD IN FLANG AND LLVM Guray Ozen, Simone Atzeni, Michael Wolfe Annemarie Southwell, Gary Klimowicz MOTIVATION What does HPC programmer need today? Performance à GPUs, multi-cores, other
CURRENT STATUS OF THE PROJECT TO ENABLE GAUSSIAN 09 ON GPGPUS
CURRENT STATUS OF THE PROJECT TO ENABLE GAUSSIAN 09 ON GPGPUS Roberto Gomperts (NVIDIA, Corp.) Michael Frisch (Gaussian, Inc.) Giovanni Scalmani (Gaussian, Inc.) Brent Leback (PGI) TOPICS Gaussian Design
Dynamical Core Rewrite
Dynamical Core Rewrite Tobias Gysi Oliver Fuhrer Carlos Osuna COSMO GM13, Sibiu Fundamental question How to write a model code which allows productive development by domain scientists runs efficiently
Advanced OpenACC. Steve Abbott November 17, 2017
Advanced OpenACC Steve Abbott , November 17, 2017 AGENDA Expressive Parallelism Pipelining Routines 2 The loop Directive The loop directive gives the compiler additional information
Trends in HPC (hardware complexity and software challenges)
Trends in HPC (hardware complexity and software challenges) Mike Giles Oxford e-research Centre Mathematical Institute MIT seminar March 13th, 2013 Mike Giles (Oxford) HPC Trends March 13th, 2013 1 / 18
OpenACC 2.6 Proposed Features
OpenACC 2.6 Proposed Features OpenACC.org June, 2017 1 Introduction This document summarizes features and changes being proposed for the next version of the OpenACC Application Programming Interface, tentatively
Optimising the Mantevo benchmark suite for multi- and many-core architectures
Optimising the Mantevo benchmark suite for multi- and many-core architectures Simon McIntosh-Smith Department of Computer Science University of Bristol 1 Bristol's rich heritage in HPC The University of
Our Workshop Environment
Our Workshop Environment John Urbanic Parallel Computing Scientist Pittsburgh Supercomputing Center Copyright 2017 Our Environment This Week Your laptops or workstations: only used for portal access Bridges
GPU Consideration for Next Generation Weather (and Climate) Simulations
GPU Consideration for Next Generation Weather (and Climate) Simulations Oliver Fuhrer 1, Tobias Gisy 2, Xavier Lapillonne 3, Will Sawyer 4, Ugo Varetto 4, Mauro Bianco 4, David Müller 2, and Thomas C.
OpenACC. Introduction and Evolutions Sebastien Deldon, GPU Compiler engineer
OpenACC Introduction and Evolutions Sebastien Deldon, GPU Compiler engineer 3 WAYS TO ACCELERATE APPLICATIONS Applications Libraries Compiler Directives Programming Languages Easy to use Most Performance
OpenMP 4.0. Mark Bull, EPCC
OpenMP 4.0 Mark Bull, EPCC OpenMP 4.0 Version 4.0 was released in July 2013 Now available in most production version compilers support for device offloading not in all compilers, and not for all devices!
GPU Programming. Alan Gray, James Perry EPCC The University of Edinburgh
GPU Programming EPCC The University of Edinburgh Contents NVIDIA CUDA C Proprietary interface to NVIDIA architecture CUDA Fortran Provided by PGI OpenCL Cross platform API 2 NVIDIA CUDA CUDA allows NVIDIA
Lecture 1: Gentle Introduction to GPUs
CSCI-GA.3033-004 Graphics Processing Units (GPUs): Architecture and Programming Lecture 1: Gentle Introduction to GPUs Mohamed Zahran (aka Z) mzahran@cs.nyu.edu http://www.mzahran.com Who Am I? Mohamed
It s a Multicore World. John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist
It s a Multicore World John Urbanic Pittsburgh Supercomputing Center Parallel Computing Scientist Moore's Law abandoned serial programming around 2004 Courtesy Liberty Computer Architecture Research Group
OpenACC Fundamentals. Steve Abbott November 15, 2017
OpenACC Fundamentals Steve Abbott , November 15, 2017 AGENDA Data Regions Deep Copy 2 while ( err > tol && iter < iter_max ) { err=0.0; JACOBI ITERATION #pragma acc parallel loop reduction(max:err)