NEC SX Series and Its Applications to Weather and Climate Modeling

Transcription

1 The 4th International Workshop on Next Generation Climate Models for Advanced High Performance Computing Facilities NEC SX Series and Its Applications to Weather and Climate Modeling March 13, 2002 Hiroshi Takahara (NEC( Corporation) David Parks (HNSX Supercomputers Inc) NEC Corporation Advancement of of HPC Technology FLOPS 1 Peta Tera Highly parallel vector processing Multi-layered highly parallel processing / Distributed global computing High performance 1-chip vector / massively parallel processing Future system Earth Simulator Giga Parallel processing cluster Micro processor Server PC Supercomputer Applications Structural & thermal analysis CFD Crash Weather/climate Ocean Biochemistry

2 Vector Processor vs Scalar Processor IA-64 Architecture Itanium(800MHZ) Max 16way AzusA Max 64GB Shared Memory SX-6 6 Profiles and Highlights Proven PVP Architecture Empowered by Leading Technologies 1. 64GF SMP nodes scale up to 8TF (128 nodes) Well-accepted SMP Multi-Node architecture since SX-4 (1995) The SX-5 is rated as the most powerful node architecture Balanced 8 CPU/ 64GF/64GB/ 256GB/s per node transfer rate with memory 8GB/s inter-node cross-bar switch up to 1TB/s per system Personal Supercomputer SX-6i 2. Established SX architecture with rich ISV applications Binary Compatible with the SX-4 and SX-5 : running at production sites Full availability of SUPER-UX and its programming environment 130 ISV APs, covering major industry APs, are available 3. Leading technologies built in one chip vector processor 8GF per chip with a 0.15 micron design rule High density packaging

3 SX-6 6 single node system High performance supercomputer Ultra-high bandwidth shared memory subsystem Maximum 8 processors, 8 Gigaflops each Maximum 64 Gigabyte memory Maximum 64 Gigaflops per node SX-6 6 multi node system Maximum 128 nodes Maximum 1024 CPUs, max 8 TFLOPS Internode crossbar switch 8 GB/s interconnection bandwidth per node 1 TB/s maximum interconnection bandwidth per system

4 SX Series in Meteorology / Environmental Science SX Series at Worldwide Major Meteorological Institutions Danish Meteorological Institute(DMI) Institute for Atmospheric Physics in Germany(IAP) Deutsches Klimarechenzentrum (DKRZ) Czech Hydrometeorological Institute(CHMI) Europe Asia Istituto Nazionale di Geofisica e Vulcanologia (INGV) Meteorological Service Singapore (MSS) Swiss Center for Scientific Computing (CSCS) Interdisciplinary Center for Mathematical and Computational Modeling, Warsaw University(ICMW) Ѓњ Korea Meteorological Administration(KMA) Japan National Institute of Environmental Studies (NIES) Japan Marine Science and Technology Center (JAMSTEC) Frontier Research System for Global Change Bureau of Meteorology (BOM)/CSIRO Australia IRI Lamont Doherty North America Atmospheric Environment Service(AES) Instituto Nacional De Pesquis Espaciais (INPE) South America SX Series Solutions Application-driven solutions Data-centric solutions Data management / reduction Visualization Ease of use Partnership

5 Application Solutions Enhanced availability of applications Performance enhancement capitalizing on hardware architecture Establishment of customer-specific application environments Weather codes MM5, IFS, ARPS, ECHAM, Oceanic codes POP, MOM... Climate codes NCAR CSM... AFES (Earth Simulator) Vector vs Scalar(Climate App.) 64CPUs 48CPUs 32CPUs

6 Example: the Earth Simulator Processor node #0 Interconnection network (16GB/s * 2) #15 Cluster #0 #1 Shared memory Shared memory Shared memory Vector processor Vector Vector processor processor Vector Vector processor processor Vector processor Vector processor Vector processor #0 #1 #0 #1 #7 #0 #1 #7 Vector processor #7 HDD #39 #1 Next-generation supercomputer in Japan Tape MT library archives Specifications Total number of processors 5,120 Peak performance / processor 8 Gflops Total number of nodes 640 Peak performance / node 64 Gflops Total peak performance 40 Tflops Shared memory 16 GB Total main memory 10 TB Optimization for AFES model for the ES AFES model (funded by NASDA) Original Developers: CCSR (Center for Climate Sys. Res., Univ. of Tokyo ) NIES (National Inst. for Environmental Studies ) Redesigning / full-optimization tailored to the ES Earth Simulator Research and Development Center Target application on the Earth Simulator Atmospheric global circulation model based on the spectral method Parallelization with MPI/Microtasking Fortran95-compatible with expandability and coupling with oceanic and other models

7 Cost profile of AFES ( % ) for a 1-day 1 simulation on the ES (160 nodes * 8 CPUs per node =1280 CPUs) ( MPI Root Process ) Whole Legendre Transformation DYMAMICS 74% ( FFT + LT + others ) PHYSICS 15% ( Radiation + others ) Total ELAPSE = sec Performance of Legendre Transformation (Kernel for the Spectral Method) LT backward LT forward T1279 forward MFLOPS backward MFLOPS

8 Effective Performance of the AFES Climate Code on the ES with the Kuo's Cumulus Convection Scheme for a T1279L96 Resolution Model World-beating14.5TFLOPS sustained performance with 320 nodes (2560 CPUs/ 20TFLOPS) 70.8% 74.3% 75.3% Effect of Load Balancing (with the A-S A S cloud dynamics model) - T1279L96 model resolution with 160 MPI processes Load balancing : ON / OFF South Pole Equator North Pole Balanced Original Balanced Original

9 Data-centric Solutions Mission Statement: NEC will provide the means to facilitate analysis, visualization, and extraction of information from a vast amount of data. Even with multi-terabyte local disk sub-systems and multi-petabyte archives, I/O can become a bottleneck in high performance computing. Jeanette Jenness, LLNL, ASCI-Project, 1998 Data extraction / hierarchical data Server-driven, image-based visualization Data Management Tools Approaches for reduced transfer of HPC data De facto standard meta data format for accessibility NetCDF, GRIB, HDF... Selected access to a huge volume of data ECMWF - Meteorological Archival and Retrieval System (MARS) and its interfacing Query-based retrieval and extraction of data - Open array databases for geophysical data ESTEDI initiative (European Spatio-Temporal Data Infrastructure for High-Performance Computing) htttp:// Data extraction select q[ *:*, 100:150, *:*, 30 ] from WeatherModel as q

10 RVSLIB: NEC s s Real-time Visual Simulation Library Network (LAN / WAN) RVSLIB/Client Image Display GUI in Java Rendering Animation Terminal Workstation/PC Compressed Image Data Tracking Steering Scenario File Simulation code linked with RVSLIB library RVSLIB/Server Visualization Steering of Solver Creation of Image Animation File Computing Server Supercomputer/Workstation RVSLIB/Server: SX, WS RVSLIB/Client: PC,WS (Java) Monitoring of an ongoing simulation (tracking) Alteration of simulation parameters (steering) while continuing the simulation - Constant and reduced data transfer rate between server and client Reduced Cost Efficient NW Usage In interactive and batch modes Bureau of Meteorology (Australia) Atmospheric simulation -- Relative humidity around Australia represented by isosurfaces

11 Ease of use Mission Statement: NEC will continue to strive for enhanced ease of use in HPC by capitalizing on emerging Internet and Web technologies. SX-6 6 Enhanced Software Technology Large Scale System Support MPI provides high-performance interfaces for the maximum scalability Ease of Operation: MasterScope for the entire system operations Network Internet SX-6 END USERS (BROWSERS) WSCE SX-GFS istorage FC-AL AzusA WebSuperComputing Environments provide Web based AP interfaces, facilitating - Visualization - Manipulation and tracking/ Tape Librarysteering of APs Global File System : SAN features for SX - common file system for the entire multi-node system - up to 80% of local storage performance - IA-64 AzusA supports the SX-GFS on Linux

12 Grid Computing GRID computing = Global computing PC Supercomputer Storage device Observation equipment IT-Based Laboratory Project Japanese original global/grid-computing project

13 Vision - Global Computing Environments Requirements for Grid Services Collaboration Tools Data Mgmt Tools... Distributed Simulation One Single Machine Never Fits For All... NEC s s involvement in GRID computing Porting of GLOBUS on the SX Series Vector computers with distinguished performance characteristics from conventional scalar computers ---> key node system that makes up the GRID Giga-bit interconnected labs --- operational use since April 2001 Porting of CORBA to SX (NEC Europe) Development of hierarchical MPI balancing between wide-area communication with TCP/IP and high-speed communication inside the SX Series supercomputer --> Institute of Laser Engineering, Osaka University

14 Partnership Mission Statement: NEC will recognize the interdisciplinary nature of the Meteo community and continue to participate in cutting-edge collaboration projects to promote the evolution of meteorological applications. Vendor s involvement in code development and optimization becoming an integral part with the growing complexity of applications and the market evolution Growing expectations for vendors to serve as an intermediary between computational science and meteorology Coordination of collaborative projects Climate simulation with NCAR and CRIEPI (Japan) Impacts of global warming on evolution of typhoon (NCAR CCM3) PRISM (PRogramme for Integrated earth System Modeling) ---> European version of the Earth Simulator Development of physical and technical interfaces among different models --- coupler models and their optimization MAP (Mesoscale Alpine Program) High resolution model (Canadian MC2) on the SX at CSCS for comparative field / computer experiments Earth Simulator /Frontier Research System for Global Change Re-designing and optimization of applications tailored to the ES Project coordination capitalizing on the ES --> Government-led projects aimed at the establishment of application environments as well as full exploitation of the ES

15 AFES Simulation Code on the ES T1279L96 resolution : horizontal grid spacing10km Precipitation with T42L24 model

16 Precipitation with T106L24 model Precipitation with T639L24 model

17 Computing Capabilities Application Solutions Ease of Use Data-centric Solutions