CEA and RIKEN AICS Collaboration

Transcription

1 CEA and RIKEN AICS Collaboration Yutaka Ishikawa RIKEN AICS 16:25 16:55 First French Japanese German Workshop on Programming and Computing for Exascale and beyond, 5 th April 2017, Tokyo

2 Outline of Talk An Overview of FLAGSHIP 2020 and development of post-k system CEA Collaboration Concluding Remarks 20017/04/05 2

3 FLAGSHIP2020 Project Missions Building the Japanese national flagship supercomputer, post K, and Developing wide range of HPC applications, running on post K, in order to solve social and science issues in Japan I/O Network Maitenance Servers Portal Servers Login Servers Hierarchical Storage System Hardware and System Software Post K Computer RIKEN AICS is in charge of development Fujitsu is vendor partnership Applications 9 High priority issues from a social and national viewpoint Promising creation of world Leading achievement Promising strategic use of post K computer 20017/04/05 3

4 9 Social and scientific priority issues Selected from the following point of view: High priority issues from a social and national viewpoint Promising creation of world Leading achievement Promising strategic use of post K computer Category Priority issues Life science Disaster prevention and global climate problem Energy problem Industrial applications Basic science 1 Innovative drug discovery infrastructure through functional control of biomolecular systems 2 Integrated computational life science to support personalized and preventive medicine 3 Development of integrated simulation systems for hazard and disaster induced by earthquake and tsunami 4 Advancement of meteorological and global environmental predictions utilizing observational Big Data 5 Development of new fundamental technologies for high-efficiency energy creation, conversion/storage and use 6 Accelerated Development of Innovative Clean Energy Systems 7 Creation of new functional devices and high-performance materials to support next-generation industries 8 Development of Innovative Design and Production Processes that Lead the Way for the Manufacturing Industry in the Near Future 9 Elucidation of the fundamental laws and evolution of the universe 20017/04/05 4

5 An Overview of Co-design in the Post K development Node and Storage Architecture #SIMD, SIMD length, #core, #NUMA node cache (size and bandwidth) network (topologies, latency and bandwidth) memory technologies specialized hardware Node interconnect, I/O network System Software Operating system for many core architecture Communication libraries (low level layer, MPI, PGAS) File I/O (Asynchronous I/O, buffering/caching) Programming Environment Programming model and languages Math libraries, domain specific libraries 9 social & scientific priority issues and their R&D organizations have been selected from the following point of view: High priority issues from a social and national viewpoint Promising creation of world Leading achievement Promising strategic use of post K computer Program 1 GENESIS MD for proteins Target Application Brief description 2 Genomon Genome processing (Genome alignment) 3 GAMERA 4 NICAM+LETK Earthquake simulator (FEM in unstructured & structured grid) Weather prediction system using Big data (structured grid stencil & ensemble Kalman filter) 5 NTChem molecular electronic (structure calculation) 6 FFB Large Eddy Simulation (unstructured grid) 7 RSDFT an ab-initio program (density functional theory) 8 Adventure Computational Mechanics System for Large Scale Analysis and Design (unstructured grid) 9 CCS-QCD Lattice QCD simulation (structured grid Monte Carlo) 20017/04/05 5

6 An Overview of post-k Hardware Manycore architecture based on ARM+SVE+Fujitsuʼs extensions 6D mesh/torus Interconnect 3-level hierarchical storage system Silicon Disk Magnetic Disk System Software Multi-Kernel: Linux with Light-weight Kernel File I/O middleware for 3-level hierarchical storage system and application Application-oriented file I/O middleware MPI+OpenMP programming environment Highly productive programing language and libraries Storage for archive I/O Network Maintenance Servers Portal Servers Login Servers Hierarchical Storage System 20017/04/05 6

7 CPU Architecture ARMv8-A + SVE (Scalable Vector Extension) FP64/FP32/FP16 Fujitsuʼs extensions Inter core barrier Sector cache Hardware prefetch assist 20017/04/05 7

8 McKernel developed at RIKEN Partition resources (CPU cores, memory) Full Linux kernel on some cores System daemons and in-situ non HPC applications Device drivers Light-weight kernel(lwk), McKernel on other cores HPC applications McKernel is loadable module of Linux McKernel supports Linux API McKernel runs on Intel Xeon and Xeon phi Fujitsu FX10 and FX100 (Experiments) will be McKernel is deployed to the Oakforest PACS supercomputer, 25 PF in peak, at JCAHPC organized by U. of Tsukuba and U. of Tokyo Batch job queues for McKernel has not been turned on System daemons TCP stack Dev. Drivers Interrupt In situ non HPC application Linux VFS File Sys Driers Partition Complex Mem. Mngt. General scheduler Memory Thin LWK Very? simple memory management HPC Applications Linux API (glibc, /sys/, /proc/) Partition Process/Thread management Core Core Core Core Core Core 20017/04/05 8

9 How to deploy McKernel Linux Kernel+Loadable LWK, McKernel Linux Kernel is resident, and daemons for job scheduler and etc. run on Linux McKernel is dynamically reloaded (rebooted) for each application No hardware reboot App A, requiring LWK-withoutscheduler, Is invoked App B, requiring LWK-with-scheduler, Is invoked Finish App C, using full Linux capability, Is invoked Finish Finish 20017/04/05 9

10 FWQ: Fixed Work Quanta FWQ Benchmark Linux with isolcpus McKernel /04/05 10

11 GeoFEM (University of Tokyo) ICCG with Additive Schwartz Domain Decomposition - weak scaling Up to 18% improvement Figure of merit (solved problem size normalized to execution time) Linux IHK/McKernel Results using the same binary Acknowledgement: Kengo Nakajima, University of Tokyo, for providing GeoFEM. This result is on Oakforest PACS supercomputer, 25 PF in peak, at JCAHPC organized by U. of Tsukuba and U. of Tokyo k 32k 64k 128k Number of physical cores 20017/04/05 11

12 CCS-QCD (University of Tsukuba) Lattice quantum chromodynamics code - weak scaling Up to 38% improvement MFlop/sec/node /04/05 0 Linux IHK/McKernel Results using the same binary Acknowledgement: Ken ichi Ishikawa, Hiroshima University, providing CCS QCD. This result is on Oakforest PACS supercomputer, 25 PF in peak, at JCAHPC organized by U. of Tsukuba and U. of Tokyo k 32k 64k 128k Number of physical cores 12

13 minife (CORAL benchmark suite) Conjugate gradient - strong scaling Up to 3.5X improvement (Linux falls over.. ) Linux IHK/McKernel 3.5X Total CG MFlops Results using the same binary /04/ k 32k 64k Number of physical cores Oakforest PACS supercomputer, 25 PF in peak, at JCAHPC organized by U. of Tsukuba and U. of Tokyo 13

14 Programming Language CEA Collaboration Christophe Calvin, Marc Pérache, Patrick Carribault, Julien Jaeger, Julien Bigot Mitsuhisa Sato, Hitoshi Murai, Jinpil Lee, Atsushi Hori Runtime Environment Jacques-Charles Lafoucrière, Gilles Wiber, Yutaka Ishikawa, Masamichi Takagi, Balazs Gerofi, Takahiro Ogura Energy-aware batch job scheduler Matthieu Hautreux, Francis Belot, Atsuya Uno Large DFT calculations and QM/MM Thierry Deutsch, Luigi Genovese, Takahito Nakajima, Takahito Nakajima Application of High Performance Computing to Earthquake Related Issues of Nuclear Power Plant Facilities Evelyne Foerster, Gauthier Folzan, Alberto Frau, Muneo Hori, Hiroki Motoyama, Kohei Fujita KPIs (Key Performance Indicators) Jean-Philippe Bourgoin, Jean-Philippe Nominé, Didier Juvin, Shigeo Okaya, Miwako Tsuji, Mitsuhisa Sato, Kenji Morishita 20017/04/05 14

15 CEA Collaboration: Programming Language Collaborators Christophe Calvin, Marc Pérache, Patrick Carribault, Julien Jaeger, Julien Bigot Mitsuhisa Sato, Hitoshi Murai, Jinpil Lee, Atsushi Hori Objective and Collaboration Topics Supporting a wide range of applications productivity PGAS (Partitioned Global Address Space) model for the next generation manycore parallel systems provides light-weight one-sided communication and low overhead synchronization semantics. Background CEA: MPC (MultiProcessor Communications) RIKEN: XcalableMP (XMP), PVAS (Partitioned Virtual Address Space), and PIP (Processes in a Process) 20017/04/05 15

16 CEA Collaboration: Programming Language Collaborators Christophe Calvin, Marc Pérache, Patrick Carribault, Julien Jaeger, Julien Bigot Mitsuhisa Sato, Hitoshi Murai, Jinpil Lee, Atsushi Hori Objective and Collaboration Topics Supporting a wide range of applications productivity PGAS (Partitioned MPC Global as MPI Address Space) model for the next generation manycore parallel systems XMP available on ATOS/Bull supercomputer MPC available on ARM architecture provides light-weight one-sided communication and low overhead synchronization semantics. List of benchmarks and mini-app to be evaluated Background implementation for XMP prototype Benchmarks implemented with XMP CEA: MPC (MultiProcessor on target architectures Communications) Study on a unified API for inter XMP nodes communication Benchmarks implemented with XMP- MPC on target architectures Benchmarks implemented with integrated environment on target architectures RIKEN: XcalableMP (XMP), PVAS (Partitioned Virtual Address Space), and PIP (Processes in a Process) 20017/04/05 16

17 CEA Collaboration: Runtime Environment Collaborators Christophe Calvin, Marc Pérache, Patrick Carribault, Julien Jaeger, Julien Bigot Mitsuhisa Sato, Hitoshi Murai, Jinpil Lee, Atsushi Hori Objective and Collaboration Topics Improving (performance) portability of applications Defining a standard of the runtime environment settings (including libraries, OS parameters and OS kernels) Finding optimal settings in terms of application performance Contributing to the OpenHPC community Background CEA: SELFIE (profiling tool) and PCOCC (virtualization tool) EasyBuild, a software build and installation framework, is used to manage open-source packages OS: Linux CPU: Intel Xeon, Intel Xeon Phi, ARM Network: InfiniBand, Omni-Path, Fujitsu Tofu, Bull BXI RIKEN: Linux with IHK/McKernel (Light-weight OS Kernel) 20017/04/05 17

18 CEA Collaboration: Runtime Environment Collaborators Christophe Calvin, Marc Pérache, Patrick Carribault, Julien Jaeger, Julien Bigot Mitsuhisa Sato, Hitoshi Murai, Jinpil Lee, Atsushi Hori Objective and Collaboration Topics Improving portability of applications with performance 1 st version of configuration standard, libraries, kernel parameters and kernels 2 nd version of configuration standard, Defining a standard of the runtime environment settings (including libraries, kernel libraries, kernel libraries, kernel parameters and kernels) CEA tests McKernel on CEA s machines RIKEN Background investigates EasyBuild CEA and RIKEN provide the current user demands parameters and kernels Finding optimal settings in terms of application performance. CEA: SELFIE (profiling tool) and PCOCC (virtualization tool) RIKEN: McKernel (Light-weight OS Kernel) 3 rd version of configuration standard, parameters and kernels 4 th version of configuration standard, libraries, kernel parameters and kernels 20017/04/05 18

19 Concluding Remarks The system software stack for post-k is being designed and implemented with the leverage of international collaborations, CEA, DOE Labs, and JLESC (NCSA, INRIA, ANL, BSC, JSC, RIKEN) The software stack developed at RIKEN is open source It also runs on Intel Xeon and Xeon phi RIKEN would like to contribute to OpenHPC 20017/04/05 19