Post-K Supercomputer with Fujitsu's Original CPU, A64FX Powered by Arm ISA

Transcription

1 Post-K Superomputer with Fujitsu's Original CPU, A64FX Powered by Arm ISA Toshiyuki Shimizu Nov. 15th, 2018 Post-K is under development, information in these slides is subjet to hange without notie 0

2 Agenda Bakground K omputer and Post-K Post-K goals and approahes Fujitsu new Arm CPU A64FX Post-K system System software Configuration Performane disussion Summary and development status 1

3 Agenda Bakground K omputer and Post-K Post-K goals and approahes Fujitsu new Arm CPU A64FX Post-K system System software Configuration Performane disussion Summary and development status 2

4 K omputer, PRIMEHPC, and Post-K FUJITSU PRIMEHPC FX10 1.8x CPU perf. Easier installation Japan s National Projets Development HPCI strategi apps program App. review FS projets PRIMEHPC FX100 4x(DP) / 8x(SP) CPU & Tofu2 High-density pkg & lower energy Tehnial Computing Suite (TCS) Handles millions of parallel jobs OS: Lower OS jitter w/ FEFS: super salable file system assistant ore MPI: Ultra salable olletive ommuniation libraries Operation of K omputer Post-K Post-K omputer development K omputer, PRIMEHPC FX Many appliations are running and being developed for siene and various industries System software TCS supports hardware with newly developed tehnologies Post-K RIKEN and Fujitsu are working together for Post-K OS is running and design verifiation is proeeding as sheduled 3

5 Post-K goals and approahes Post-K goals High appliation performane and good power effiieny Good usability and better aessibility for users Keeping appliation ompatibility while advaning from predeessors Our approahes Develops high performane and salable, ustom CPU ores Performane Wider SIMD & mathematial a. primitives, high memory BW Salability Salable interonnet Tofu Power effiieny The best devie teh, power ontrol funtions, optimal resoures Adopts Arm ISA, binary ompatibility Maintains performane balane and supports advaned features 4

6 Agenda Bakground K omputer and Post-K Post-K goals and approahes Fujitsu new Arm CPU A64FX Post-K system System software Configuration Performane disussion Summary and development status 5

7 Fujitsu new Arm CPU: A64FX A64FX approah Spes and tehnologies Tofu interonnet D Power management A64FX summary 6

8 A64FX: Approah High performane Arm CPU Develops original CPU ore supporting Arm SVE (Salable Vetor Extension) Targeting high performane servers Floating point alulations High memory bandwidth Low power onsumption Salable performane and onfiguration Extension for emerging appliations Half preision (FP16) & INT16/8 partial dot produt support High BW/Cal. is the key for Real apps. 7

9 A64FX: Spes & tehnologies CPU generations and key parameters CPU SPARC64 VIIIfx SPARC64 IXfx SPARC64 XIfx A64FX 1 st system w/ CPU K FX10 FX100 Post-K Si teh. (nm) Core perf. (GFLOPS) SVE 57~ # of ores Chip perf. (TFLOPS) ~ Memory BW (GB/s) HBM 1024 B/F (Bytes/FLOP) Combination of tehnologies SVE inreases ore performane is salable arhiteture to inrease # of ores HBM enables high bandwidth 8

10 A64FX tehnologies: Core performane High al. throughput of Fujitsu original CPU ore w/ SVE 512-bit wide SIMD x 2 pipelines and new integer funtions (GOPS) 500 Core peak performane INT8 partial dot produt >460 >230 >57 > bit 32-bit 16-bit 8-bit Multiply and add (Element size) INT8 partial dot produt 8bit 8bit 8bit 8bit A0 A1 A2 A3 X X X X B0 B1 B2 B3 C 32bit 9

11 A64FX tehnologies: Salable arhiteture Core Memory Group () 12 ompute ores and an assistant ore for OS daemon, I/O, et. Shared L2 ahe Dediated memory ontroller onfiguration Four s maintain ahe oherene w/ on-hip diretory Threads binding within a allows linear speed of up to 48 ompute ores A64FX pakage onfiguration ore ore ore ore ore ore ore ore ore ore ore ore ore Tofu ontroller PCIe ontroller X-Bar onnetion HBM2 HBM2 L2 ahe 8MiB 16-way Memory ontroller HBM2 Network on hip HBM2 HBM2 Network on hip 10

12 A64FX: L1D ahe unompromised BW 128B/yle sustained BW even for unaligned SIMD load Read port0 L1D$ Read data0 64B/yle Read port1 Read data1 64B/yle Combined Gather doubles gather (indiret) load s data throughput, when target elements are within a 128-byte aligned blok for a pair of two regs, even & odd Mem. Regs 128B flow-1 flow flow-4 flow-3 8B Maximizes BW to 32 bytes/y. 12

13 Tofu Network Router 2 lanes 10 ports A64FX: Tofu interonnet D Integrated w/ rih resoures Inreased TNIs ahieves higher injetion BW & flexible omm. patterns Inreased barrier resoures allow flexible olletive omm. algorithms Memory bypassing ahieves low lateny Diret desriptor & ahe injetion Data rate Link bandwidth TofuD spe Gbps 6.8 GB/s HBM2 NOC HBM2 TNI0 TNI1 TNI2 PCle A64FX Injetion bandwidth Put throughput 40.8 GB/s Measured 6.35 GB/s PingPong lateny 0.49~0.54 µs HBM2 HBM2 TNI3 TNI4 TNI5 TofuD 13

14 A64FX: Power monitor and analyzer Energy monitor (per hip) Node power via Power API*1 (~mse) Averaged power of a node, (ores, an L2 ahe, a memory) et. Energy analyzer (per ore) Power profiler via PAPI*2 (~nse) Fine grained power analysis of a ore, an L2 ahe and a memory *1: Sandia National Laboratory *2: Performane Appliation Programming Interfae A64FX #0 Core L2 ahe Memory Core ativity Energy Analyzer Power al. Core L2 Cahe Memory L2 ahe ativity Power al. Memory ativity Power al. #1 #2 #3 Energy Monitor Node 12 Cores L2 ahe HBM2 Assistant ore Tofu PCIe Power al. for Tofu Power al. for PCIe 14

15 A64FX: Power Knobs to redue power onsumption Power knob limits units ativity via user APIs Performane/W would be optimized by utilizing Power knobs, Energy monitor & analyzer Feth Issue Dispath Reg-read Exeute Cahe and Memory Commit L1 I$ Branh Preditor Deode & Issue RSA RSE0 RSE1 PGPR PPR EAGA EXC EAGB EXD EXA EXB PRX Feth Port Store Port Write Buffer L1D$ CSE PC Control Registers Frequeny redution EXA pipeline only RSBR PFPR FLA FLB FLA pipeline only L2$ Deode width: 4 to 2 Tofu ontroller HBM2 Controller HBM2 PCI Controller 52 ores HBM2 B/W adjust (units of 10%) 15

16 A64FX: Summary Arm SVE, high performane and effiieny DP performane >2.7 TFLOPS, Memory BW 1024 GB/s, Triad 12x ompute ores 1x assistant ore HBM2 HBM2 PCle Controller C C N O C Tofu Interfae C C HBM2 HBM2 :Core Memory Group NOC:Network on Chip A64FX ISA (Base, extension) Armv8.2-A, SVE Proess tehnology 7 nm Peak DP performane >2.7 TFLOPS SIMD width 512-bit # of ompute ores 48 Memory apaity 32 GiB (HBM2 x4) Memory peak bandwidth 1024 GB/s PCIe Gen3 16 lanes High speed interonnet TofuD integrated 16

17 Agenda Bakground K omputer and Post-K Post-K goals and approahes Fujitsu new Arm CPU A64FX Post-K system System software Configuration Performane disussion Summary and development status 17

18 Software development: System software RIKEN and Fujitsu are developing software staks for Post-K Fujitsu Tehnial Computing Suite / RIKEN developing system software Management software System management for high availability & power saving operation Job management for higher system utilization & power effiieny Post-K appliations File system FEFS Lustre-based distributed file system LLIO NVM-based file I/O aelerator Linux OS / MKernel (Lightweight kernel) Post-K system hardware Programming environment XalableMP MPI (Open MPI, MPICH) OpenMP, COARRAY, Math. libs. Compilers (C, C++, Fortran) Debugging and tuning tools Post-K Under development w/ RIKEN 18

19 FEFS vs Lustre, LLIO vs Burst buffer Post-K supports superior performane file system, FEFS + LLIO FEFS Lustre Storage network Tofu, InfiniBand, Ether, OPA InfiniBand, Ether, OPA Injetion BW per node 40.8 GB/s 12.5 GB/s (IB EDR) MDS salability Yes Yes (ver. 2.4~) Interoperability x86, Arm (A64FX) x86, Power, Arm? FEFS + LLIO* Lustre + Burst buffer** Loal temporary file a. Yes No Cahe file a. Yes Yes Shared file within a job. Yes Yes Expliit funtion all Not required (by mount point) Required (Soure modifiation) *Lightweight Layered IO-Aelerator **DDN IME 19

20 LLIO (Lightweight Layered IO-aelerator) overview Maximizing appliation file I/O performane Easy aess to User Data: File Cahe of Global File System Higher Data Aess Performane: Temporary Loal FS (in a proess) Higher Data Sharing Performane: Temporary Shared FS (among proesses) Job A Job B Assistant Cores File I/O Node App. Node App. Node App. Node App. Node App. Compute Cores 1 st level file1 Loal file2 Cahe File3 Shared Temporary Cahe Loal file systems 2 nd level file4 Global file system (Lustre based) 20

21 Post-K system onfiguration Salable design CPU CMU BoB Shelf Rak Unit # of nodes Desription CPU 1 Single soket node with HBM2 & Tofu interonnet D CMU 2 CPU Memory Unit: 2x CPU BoB 16 Bunh of Blades: 8x CMU Shelf 48 3x BoB Rak 384 8x Shelf 23

22 CMU: CPU Memory Unit A64FX CPU x2 QSFP28 x3 for Ative Optial Cables Single-side blind mate of signals & water ~100% diret water ooling Water Water AOC AOC QSFP28 (Z) QSFP28 (Y) QSFP28 (X) Eletrial signals 24

23 Agenda Bakground K omputer and Post-K Post-K goals and approahes Fujitsu new Arm CPU A64FX Post-K system System software Configuration Performane disussion Summary and development status 25

24 Performane inrease normalized by SPARC64 XIfx Preliminary performane evaluation results Over 2.5x faster in HPC & AI benhmarks than SPARC64 XIfx Arm SVE, Fujitsu s miroarhiteture, and high bandwidth A64FX hip performane measurements & arhitetural ontributions HPC Throughput tests Appliation kernels AI 9.4x Memory BW 512-bit SIMD Combined gather L1$ BW L2$ BW INT8 partial dot produt 2.5TF 840 GB/s 3.0x 2.8x 3.4x 2.5x Baseline: SPARC64 XIfx DGEMM STREAM Triad Fluid dynamis Atomosphere Seismi wave propagation Convolution FP32 Convolution low preision 26

25 Agenda Bakground K omputer and Post-K Post-K goals and approahes Fujitsu new Arm CPU A64FX Post-K system System software Configuration Performane disussion Summary and development status 27

26 Summary of Post-K High appliation performane and good power effiieny High memory bandwidth & wider SIMD Arm SVE support Salable vetor extension, state of art Arm instrution set arhiteture Binary ompatibility advanes and expands the power of eosystem by inorporating HPC tehnologies and appliations Optimizing for new superomputer standards Low power onsumption in many aspets Being fousing on the existing appliations and emerging appliations Post-K inheriting the strong and proven miroarhiteture with HPC system software will be more powerful with Arm and its eosystem 28

27 System development status: Post-K Fujitsu is urrently developing Post-K, suessor to K omputer, with RIKEN OS running and design verifiation is underway as sheduled RIKEN announed Post-K early aess program to begin around Q2/CY Post-K prototype rak

28 30