HPC as a Driver for Computing Technology and Education

Transcription

1 HPC as a Driver for Computing Technology and Education Tarek El-Ghazawi The George Washington University Washington D.C., USA

2 NOW- July 2015: The TOP 10 Systems Rank Site Computer Cores Rmax [Pflops] % of Peak Power [MW] MFlops /Watt National Super Computer Center in Guangzhou, China DOE / OS Oak Ridge Nat Lab USA DOE / NNSA L Livermore Nat Lab USA RIKEN Advanced Inst for Comp Sci, Japan DOE / OS Argonne Nat Lab, USA 6 Swiss CSCS 7 KAUST, Saudi Tianhe-2 NUDT, Xeon 12C 2.2GHz + IntelXeon Phi (57c) + Custom Titan, Cray XK7, AMD (16C) + Nvidia Kepler GPU (14c) + Custom Sequoia, BlueGene/Q (16c) + custom K computer Fujitsu SPARC64 VIIIfx (8c) + Custom Mira, BlueGene/Q (16c) + Custom Piz Daint, Cray XC30, Xeon 8C + Nvidia Kepler (14c) + Custom Shaheen II, Cray XC30, Xeon 16C + Custom 3,120, , ,572, , , , , TACC, USA 9 10 Forschungszentrum Juelich (FZJ), Germany DOE / NNSA LLNL, USA Stampede, Dell Intel (8c) + Intel Xeon Phi (61c) + IB JuQUEEN, BlueGene/Q, Power BQC 16C 1.6GHz+Custom Vulcan, BlueGene/Q, Power BQC 16C 1.6GHz+Custom 204, , , (422) Software Comp HP Cluster USA 18,

3 HPC is a Top National Priority! Executive Order from the White House Establishment of a National Strategic Computing Initiative (NCSI) 29 July

4 National Strategic Computing Initiative Five strategic themes of the NSCI: 1) Create systems that can apply exaflops of computing power to exabytes of data 2) Keep the United States at the forefront of HPC capabilities 3) Improve HPC application developer productivity 4) Make HPC readily available 5) Establish hardware technology for future HPC systems 4 4

5 Future/Investments - International Exascale HPC Programs Country Funding Year(s) Remarks European Union 700M Private-Public Partnership commitment through European Tech Platform for HPC (ETP4HPC) 143.4M in M dedicated FP7 Exascale projects India $2B Led by IISc (Indian Institute of Science) and ISRO (Indian Space Research Organization). Targeting a 132 ExaFLOP/s machine $750M C-DAC (Center for Development of Advanced Computing) to set up 70 supercomputers over 5 years Japan $1.38B Post-K computer to be installed at RIKEN; Tentatively based on Extreme SIMD chip PACS-G China - Due to U.S./DoC ban will use Chinese 5 Tarek El-Ghazawi, parts GWU to upgrade current #1 system 5

6 Why is HPC Important? Critical for economic competitiveness (Highlighted by Minster Daoudi) because of its wide applications (through simulations and intensive data analyses) Drives computer hardware and software innovations for future conventional computing Is becoming ubiquitous, i.e. all computing/information technology is turning into Parallel!! Is that why it is turning into an international HPC muscle flexing contest? 6

7 Why is HPC Important? (1)Competitiveness Design Build Test Design Model Simulate Build 7

8 Molecular Dynamics HIV-1 Protease Why is HPC Important? Competitiveness Inhibitor Drug Gene Sequence Alignment Simulation for 2ns: 2 weeks on a desktop 6 hours on a supercomputer HPC Application Examples Phylogenetic Analysis: 32 days on desktop 1.5 hrs supercomputer Car Crash Simulations Understanding Fundamental Structure of Matter 2 million elements simulation: 4 days on a desktop 25 minutes on a supercomputer Requires a billionbillion calculations per second 8

9 Why is HPC Important? (2) HPC of Today is Conventional Computing for Tomorrow The ASCI Red Supercomputer 9000 chips for 3 TeraFLOPs in 1997 Intel 80 Core Chip 1 Chip and 1 TeraFLOPs in

10 3- Why is HPC Important?- HPC Concepts are becoming Ubiquitous Sony PS3 Samsung S6 8 Cores Uses the Cell Processors! HPC is Ubiquitous! All Computing is becoming HPC, Can we become bystanders? The Road Runner: Was Fastest Supercomputer in 08 Tile64: A 64 CPU Chip- Can be in your future laptop! Uses Cell Processors! 10

11 How Did we Get Here - Supercomputers in recent History Computer Processor # Pr. Year Tianhe-2 (MilkyWay-2) Titan TH-IVB-FEP Cluster, Intel Xeon E C 2.200GHz, TH Express-2, Intel Xeon Phi 31S1P Cray XK7, Opteron 16 Cores, 2.2GHz, Nvidia K20X R max (TFlops) till now 33, ,600 K-Computer, Japan SPARC64 VIIIfx 2.0GHz, ,510 Tianhe-1A, China Intel EM64T Xeon X56xx (Westmere-EP) 2930 MHz (11.72 Gflops) + NVIDIA GPU, FT C ,566 Jaguar, Cray Cray XT5-HE Opteron Six Core 2.6 GHz ,759 Roadrunner, IBM PowerXCell 8i 3200 MHz (12.8 GFlops) ,026 BlueGene/L - eserver Blue Gene Solution, IBM BlueGene/L - eserver Blue Gene Solution, IBM PowerPC MHz (2.8 GFlops) PowerPC MHz (2.8 GFlops) BlueGene/L beta-system IBM PowerPC MHz (2.8 GFlops) Earth-Simulator / NEC NEC 1000 MHz (8 GFlops) IBM ASCI White,SP POWER3 375 MHz (1.5 GFlops) IBM ASCI White,SP POWER3 375MHz (1.5 GFlops) Intel ASCI Red Intel IA-32 Pentium Pro 333 MHz (0.333 GFlops)

12 How Did we Get Here - Supercomputers in recent History See: 12

13 How Did we Get Here - Supercomputers in recent History PetaFLOPS Performance Vector Machines Massively Parallel Processors MPPs with Multicores and Heterogeneous Accelerators TeraFLOPS Discrete Integrated HPCC End of Moore s Law in Clocking! Time 13

14 NOW- July 2015: The TOP 10 Systems Rank Site Computer Cores Rmax [Pflops] % of Peak Power [MW] MFlops /Watt National Super Computer Center in Guangzhou, China DOE / OS Oak Ridge Nat Lab USA DOE / NNSA L Livermore Nat Lab USA RIKEN Advanced Inst for Comp Sci, Japan DOE / OS Argonne Nat Lab, USA 6 Swiss CSCS 7 KAUST, Saudi Tianhe-2 NUDT, Xeon 12C 2.2GHz + IntelXeon Phi (57c) + Custom Titan, Cray XK7, AMD (16C) + Nvidia Kepler GPU (14c) + Custom Sequoia, BlueGene/Q (16c) + custom K computer Fujitsu SPARC64 VIIIfx (8c) + Custom Mira, BlueGene/Q (16c) + Custom Piz Daint, Cray XC30, Xeon 8C + Nvidia Kepler (14c) + Custom Shaheen II, Cray XC30, Xeon 16C + Custom 3,120, , ,572, , , , , TACC, USA 9 10 Forschungszentrum Juelich (FZJ), Germany DOE / NNSA LLNL, USA Stampede, Dell Intel (8c) + Intel Xeon Phi (61c) + IB JuQUEEN, BlueGene/Q, Power BQC 16C 1.6GHz+Custom Vulcan, BlueGene/Q, Power BQC 16C 1.6GHz+Custom 204, , , (422) Software Comp HP Cluster USA 18,

15 How to Make Progress Launch a competitive funding cycle or a large national project Pose a system challenge ~ 33.8 PFLOPS/17.8 Mwatt provides about 2GF/Watt To get to Exascale using same total power we need 200GF/Watt Pose an application challenge(s) Let the community compete for government funding with innovative ideas 15

16 Challenges - The End of Moore s Law The phenomenon of exponential improvements in processors was observed in 1979 by Intel co-founder Gordon Moore The speed of a microprocessor doubles every months, assuming the price of the processor stays the same Wrong, not anymore! The price of a microchip drops about 48% every months, assuming the same processor speed and on chip memory capacity Ok, for Now The number of transistors on a microchip doubles every months, assuming the price of the chip stays the same Ok, for Now 16

17 No faster clocking but more Cores? Source: Ed Davis, Intel 17

18 ccelerators and Dealing with the Moore s Law Challenge Through Parallelism Fab. Process Freq # Cores Peak FP Performance Peak Power DP Flops/W Memory nm GHz SPFP GFlops DPFP GFlops W BW GB/s Memory type PowerXCell 8i XDR Nvidia Kepler K40 Intel Xeon Phi 7120P Intel Xeon 12- core 2.7 GHz E5-2697v2 AMD Opteron 6370P Interlagos GDDR (244 threads) GDDR DDR DDR Xilinx XC7VX1140T Xilinx XCUV * 5.0* Altera Stratix V GSB

19 Accelerators/Heterogeneous Computing FPGAs Cell GPUs Phi Microprocessor Application Speedup SAVINGS Cost Power Size DNA Match x 779x 253x DES Breaker x 3439x 1116x El-Ghazawi et. al. The Promise of HPRCs. IEEE Computer, February

20 A General Execution Model for Heterogeneous Computers µp Transfer of Control Input Data GPU Accelerator CELL B.E. PC FPGA Clearspeed Intel Xeon Phi Output Data Transfer of Control 20

21 Challenges for Accelerators 1. Application must lend itself to the rule, and different accelerators suit diffent type of computations 2. Programmer partitions the code across the CPU and accelerator 3. Programmer co-schedules CPU and accelerator, and ensures good utilization of the expensive accelerator resources 4. Programmer explicitly transfers data between CPU and accelerator 5. Accelerators are fast as compared to the link, and overhead that can render the use of the accelerator useless or harmful 6. Multiple programming paradigms are needed 7. New accelerator means learning/porting to a new programming interface 8. Changing the ratio of CPUs to accelerators requires also substantial programming unless accelerators are vituralized 21

22 Challenges for Advancing or for Exascale DoE ASCAC Subcommittee Report Feb Energy Efficiency 2. Interconnect Technology 3. Memory Technology 4. Scalable System Software 5. Programming Systems 6. Data Management 7. Exascale Algorithms 8. Algorithms for Discovery, Design & Decision 9. Resilience and Correctness 10. Scientific Productivity Data movement Tarek and/or El-Ghazawi, programming GWU related 22

23 Exascale Technological Challenges The Power Wall Frequency scaling is no longer possible, power increases rapidly The Memory Wall Gap between processor speed and memory speed is widening The Interconnect Wall Available bandwidth per compute operations is dropping Power needed for data movement is increasing Programmability Wall, Resilience Wall,

24 The Data Movement Challenge Bandwidth density vs. system distance Energy vs. system distance [Source: ASCAC 14] Locality matters a lot, cost (energy and time) rapidly increases with distance Locality should be exploited at short distance, needed more at far distances 24

25 Data Movement and the Hierarchical Locality Challenge 25 25

26 Locality is Not Flat Anymore Chip and System 26 26

27 Locality is Not Flat in Anymore Chip and System 27 27

28 Locality is Not Flat Anymore Chip and System 28 28

29 Locality is Not Flat in Extreme Scale Chip and System Cray XC

30 Locality in Extreme Scale Chip and System Perspectives TTT TILE64 Tile64 Cray XC

31 What Does that Mean for Programmers Exploiting Hierarchical Locality Machine level and Chip level Hierarchical Tiled Data Structures Hierarchical Locality Exploitation with RTS MPI+X 31

32 General Implications Short term programming challenge Golden opportunity for smart programmer New hardware advances needed first and they will influence software May be silicon based, may be nano technologies like carbon nano-tube transistors by IBM (9nm), may keep things the way they are from the software side for a while 32

33 General Implications- Longer Run Long-term hardware technology may move toward Nano-photonics for computing Quantum Computing Many of the new hardware computing innovations may show first as discrete accelerators, then on the chip accelerator, then move closer to the processor internal circuitry ( data path ) 33

34 Longer term The bad news: with the limits of the silicon approached we may see departures from conventional methods of computing which may dramatically change the way we conceive software The good news: history has shown that good ideas from the past get resurrected in new ways 34

35 Conclusions Graduating and intelligent IT workforce can be a golden egg for countries like Morocco You can teach skills but it is imperative to teach and stress concepts in the curriculum Stress Parallelism Stress Locality See the recommendations by IEEE/NSF and SIAM for incorporating parallelism in Computer Science, Computer Engineering, and Computational Science and Engineering Curricula, and add locality For the very long-term There is nothing better than having good foundations in Physics and Math even for CS and CE majors 35

36 Conclusions cont. Integrate teaching soft skills as President Ouaouicha said Communications Entrepreneurism and marketing, individually and in groups Patenting and legal 36