EMERGING TECHNOLOGIES AND ARCHITECTURES FOR LOW-POWER AND HETEROGENEOUS COMPUTE NODE TARGETING EXASCALE LEVEL COMPUTING

Transcription

1 EMERGING TECHNOLOGIES AND ARCHITECTURES FOR LOW-POWER AND HETEROGENEOUS COMPUTE NODE TARGETING EXASCALE LEVEL COMPUTING Denis Dutoit, E. Guthmuller, JP Noël, Y. Thonnart, P. Vivet CEA-Leti Strategic Marketing Manager Advanced Computing

2 TECHNOLOGICAL BACKGROUND Supercomputer Cluster Blade TERA CEA Exascale = Flop/s (Nov. 2018: 0.2x10 18 Flop/s) Technologies? Architectures? European effort? Scope of the talk Compute Node 2

3 OUTLINE Part 1: Compute node architecture evolution Part 2: Architectures and associated technologies: Heterogeneous Architectures & Integration Photonic interposer In-Memory-Computing Part 3: European HPC Exascale effort: EuroHPC European Processor Initiative (EPI) Conclusion 3

4 PART 1: COMPUTE NODE ARCHITECTURE EVOLUTION 4

5 HIGH PERFORMANCE COMPUTING EVOLUTION Compute High Performance Computing Analyze Data in Data out New drivers Requirements Solutions New workloads (deep learning) Massive volume of data (big data) More computing performance (Ops per second) for simple operations (FP16, FP8, INT ). Energy efficiency (Ops per Watt). Increased Bytes per Flops. High bandwidth/low latency access to all data. Heterogeneity In-Memory- Computing Optical Networkon-Chip Starting from high performance compute only, HPC evolves towards: New workloads Massive volume of data TERA CEA 5

6 CHALLENGES FOR ADVANCED COMPUTING < 10x energy efficiency improvement every 4 years PERFORMANCE 100 EFLOPS 10 EFLOPS x10 every 4 years 1 EFLOPS 100 PFLOPS ENERGY PER OPERATION * 10 PFLOPS 2 nj/flop 1 PFLOPS 200 pj/flop 100 TFLOPS 20 pj/flop 10 TFLOPS 2 pj/flop 1 TFLOPS 0.2 pj/flop /10 every 4 years * assuming 20 MWatt supercomputer

7 TECHNOLOGY & ARCHITECTURE SOLUTIONS FOR ADVANCED COMPUTING End of Dennard s scaling End of Amdahl s law End of Moore s law PERFORMANCE 100 EFLOPS Happy scaling Many-core Heterogeneous architectures Disruptive architectures 10 EFLOPS 1 EFLOPS 100 PFLOPS ENERGY PER OPERATION * 10 PFLOPS 2 nj/flop 1 PFLOPS 200 pj/flop 100 TFLOPS 20 pj/flop 10 TFLOPS 2 pj/flop 1 TFLOPS 0.2 pj/flop * assuming 20 MWatt supercomputer

8 COMPUTE NODE ARCHITECTURE EVOLUTION End of Dennard s scaling End of Amdahl s law End of Moore s law Happy scaling Many-core Heterogeneous architectures Disruptive architectures Far Mem. NIC Data Centric Interconnect CPU Generic processing Cache Bus Cache Cache NoC + LLC Cache Cache Close Mem. Close Mem Coherent Link Close Mem Close Mem In Memory Computing Memory NIC (Network InterConnect) Memory NIC HW accelerator 8

9 ARCHITECTURES AND ASSOCIATED TECHNOLOGIES 2 - Photonics 1 Heterogeneous Integration Far Mem. NIC Data Centric Interconnect Generic processing In Memory Computing Part 2: Architectures and associated technologies Close Mem. Close Mem Coherent Link Close Mem Close Mem HW accelerator 3 - In-Memory- Computing 9

10 52 ND EDITION OF THE TOP500 LIST (NOVEMBER 11 TH, 2018) Top#1 today: Flop/s Peak It is 1/5 of Exascale level of performance Users: #1-#2: US (Department of Energy) #3-#4: China (National Supercomputing center) #5: Switzerland (National Supercomputing center) Processor design & technology: Chip Design Manuf. IBM POWER9 NVIDIA Volta GV100 Sunway SW26010 Intel Xeon E5 10

11 EUROPEAN HPC EXASCALE EFFORT How to bring back Europe into the race? Supercomputing infrastructure Processor design Part 3: European HPC Exascale effort 11

12 PART 2: ARCHITECTURES AND ASSOCIATED TECHNOLOGIES 12

13 1 - HETEROGENEOUS ARCHITECTURE & INTEGRATION: OUTLINE 1 Heterogeneous Integration 13

14 SCALABLE VECTOR EXTENSION: FUJITSU & ARM Source Fujitsu HotChips 2018 < Generic processing is going towards ultra-high memory bandwidth 14

15 GPU: NVIDIA TESLA V100 + IBM POWER9 Source IBM Source NVIDIA Computing performance from GPU High-speed link between components < Integration technologies for heterogeneous compute 15

16 ADVANCED PACKAGING TECHNOLOGIES Advanced Packaging Integration SiP Multi-Chip-Module 3D 2.5D Die stacking Interconnect density: 100µm x 100µm Interconnect density: 10µm x 10µm Interconnect density: 10µm x 10µm Source: AMD EPYC 7260, 4-chiplet chip Source: Micron High-Bandwidth-Memory System-in-Package 3D Integrated-Circuit (3D IC) 16

17 MULTI-CHIP-MODULE: INTEGRATION WITH CHIPLETS AMD Zen architecture integrates upto 4 chiplets on a substrate; for scalable solution and more than reticle size silicon area in a chip. AMD Zen 2 architecture integrates upto 9 chiplets on a substrate ( ) EPYC 7260, 4-chiplet chip ( ) EPYC Rome, 9-chiplet chip ( ) AMD Zen2 architecture; 17

18 3D DIE STACKING: HBM MEMORY High memory density and bandwidth for small footprint Starting from GPU but now is used on high performance chips (2015) Hynix HBM1 on AMD Fuji 18

19 2.5D INTERPOSER: HBM INTEGRATION FOR MEMORY BANDWIDTH NEC Aurora SX-10+ First product with 6x HBM2: 1.2 TB/s total memory bandwidth 2.45 TFLOPS ~0.5 Byte/Flops 19

20 LETI S DEMONSTRATOR ROADMAP FOR HETEROGENEOUS COMPUTING IP design HIGH PERFORMANCE COMPUTING Heterogeneous integration SERVER Optical link on Photonic Interposer Chiplet on Active Interposer Processor design EUROPEAN PROCESSOR For HPC, Servers & ehpc EUROPEAN EXASCALE & POST-EXASCALE HPC Compute Node on Photonic Interposer Neuromorphic Quantum computing LOGIC-ON-LOGIC 3D Interconnect MICRO-SERVER Multicore Architecture 23 partners officially in the EPI consortium For HPC, Servers & ehpc 120M funding 20

21 3D NETWORK ON CHIP LOGIC-ON-LOGIC 3D Interconnect 21

22 3D NOC DEMONSTRATOR FROM LETI ISSCC 2016 < Power efficient 3D interconnect, a first step towards 3D-based computing architectures Architecture Design Technology 3D Network-on-Chip: heterogeneous and homogenous multi-cores 3D Plug: High throughput Low latency DFT TSV middle (AR 1:8), µ-bumps, 50µm x 40µm pitch Face2Back stacking, Die2Die assembly Demonstrator 3D NoC 0.32 pj/b Molding Bottom die Package substrate Top die 3D cross section 22

23 CHIPLET ON ACTIVE INTERPOSER SERVER Chiplet on Active Interposer 23

24 ACTIVE INTERPOSER PARTITIONING FOR MANY-CORE «Active» Interposer : which added value? Heterogeneous 3D - Advanced tech node for computation within chiplets - Mature tech node for communication/power/dft/etc Chip-to-Chip Interconnect - Hierarchical NoC, for energy efficient communications System IOs - On Interposer, for off-chip memory accesses Power Management - Chiplet power supply, without any external passives And most of all... preserve (active) interposer cost! Target low logic density (eg < 10%) to preserve interposer yield & cost Source Vivet, ISVLSI'15 24

25 ACTIVE INTERPOSER FROM LETI (INTACT) Symposium DIC 2015 ISVLSI 2015 < 96 cores compute fabric with 6 chiplets stacked on an active interposer System Architecture Design Technology Cache Coherent Compute Fabric with: 96 cores (MIPS32), 3 levels of caches, integrated power management Performance targets 100 GOPS 10 GOPS/Watt 25 Watts total Heterogeneous 3D partitioning with: 28nm FDSOI chiplets (x6) Low power compute fabric Wide voltage range (0.6V 1.2V) Body biasing for logic boost & leakage ctrl 65nm active interposer Power unit (Switched Cap DC-DC conv.) Interconnect (Network-on-Chip) Test, clocking, thermal sensors, etc TSV Ø 10µm Height 100µm µ-bumps Ø 10 µm Pitch 20 µm 25

26 4 L3 CACHE TILES 64<->512 bits SER/DES 64<->512 bits SER/DES 4 CPU CLUSTERS L3 ctrl Network Interfaces Circuit IOs (250 signals) CHIPLET ARCHITECTURE & MAIN FEATURES Future 3D interco. Future 3D interco. Future 3D interco. (N1) Coherent L1<->L2 2D-Mesh Interconnect 32-bit scalar core with MMU 16 KB I & D L1 caches CPU 0 Bottom Die Active Silicon Interposer 32-bit scalar core with MMU 16 KB I & D L1 caches CPU 1 (N0) 5 Crossbars 256 KB shared distributed L2 cache 1 MB L3 cache Low freq/high bandwidth 32-bit scalar core with MMU 16 KB I & D L1 caches CPU 2 32-bit scalar core with MMU 16 KB I & D L1 caches CPU 3 Peripherals (ICU, UART, SPI) Local clock generators (N2) Adaptive L2->L3 2D-Mesh Interconnect Top Dies : TSARLET Chiplets 4 Clusters of 4 Cores Die-2-Die Interconnects (N1) Passive Links within Interposer Top Clock Generators PVT+timing fault sensors DFT & Memory BISTs Config. Registers (N3) L3->DRAM 2D-Mesh Interconnect South Bridge UART SPI Circuit Config. North Bridge System+Mem IOs Hierarchical 3D NoCs (N2-N3) 2D NoC within Chiplets 2D NoC within Interposer 3D NoC vertical link 3D Plug PE2 L1$ PE3 L1$ L2$ PE0 L1$ PE1 L1$ Circuit IO Interface PE2 L1$ PE3 L1$ L2$ PE0 L1$ PE1 L1$ 3D Plug Clk, Rst, Test L3$ L3$ L3$ L3$ Die 3D Plug PE2 L1$ PE3 L1$ PE2 L1$ PE3 L1$ L2$ L2$ PE0 L1$ PE1 L1$ FUSE mem PE0 L1$ PE1 L1$ 3D Plug Technology and main features FDSOI 28nm (STMicroelectronics) Technology LVT option, 10 Metal Layers 4.0 mm x 5.6 mm = 22 mm2, Area 395 Million Transistors Primary (2D) 249 signals, 237 powers, Circuit IOs 200µm pitch 3D Circuit 2618 signals implemented up to IOs Metal 10 20µm pitch Flip-Chip, 4 layers substrate, Package 19x19= mm pitch [0.5v - 1.3v] for core logic Power [-2v - +2v] for body biasing Supplies 1.8v for circuit IOs Clocking PVT Sensors Thermal Sensors FLLs for local clock generation, Timing fault sensors for worst path 8 PVT Sensors based on 7 Ring Oscillators 1 absolute for thermal reference, 4 PVT sensors for gradient Substrate Chiplet main innovations : Cache Coherent Shared Memory Fully Scalable architecture for future integration in 3D using inteporser Fault Tolerant and adaptive L3-caches FDSOI 28 nm technology, Ultra Large Voltage Range, Energy Efficiency [Guthmuller, ESSCIRC 2018] 26

27 ACTIVE INTERPOSER ARCHITECTURE Active interposer, integrating 6 chiplets, offering 96 cores DC-DC converters - Switched Capacitances - Poly + MIM + MOM - Power Density 0.31 W/mm2 - Vin 1.8V, Vout [0.6V 1.2 V] - Power efficiency up to 80% no external passives [G. Pillonnet, 3DIC 2015] Off-Chip Links - LVDS IOs for L3$ access - Delay calibration per lane - 4*Tx/Rx 32bits@300MHz DDR - Total Bandwidth 19 GByte/s 3D cross-section - TSV for IO signals and DC-DC converter power suplies µ-bumps on the interposer ( signals, power) C4 bumps for package connection CMOS 65nm 15 x 13 = 200 mm 2 15 Millions de transistors 0.6 % complexity of total complexity 3D Communication Links - L1/L2/L3 NoC interconnects - Passive links (L1 refill) - Active links (L2 & L3 refill) Interposer infrastructure - Configuration interface - Thermal Sensors - Stress Sensors - IEEE 1687 DFT (IJTAG) [J. Durupt, ETS 2016] 27

28 MULTI-CHIP-MODULE AND ACTIVE INTERPOSER HIGH PERFORMANCE COMPUTING Heterogeneous integration 28

29 ExaNoDe H2020-FETHPC-2014 Starting date: 01/10/2015 Duration: 42 months ExaNoDe designs core technologies for an integrated heterogeneous compute node. ExaNoDe will deliver an HW/SW integrated prototype comprising: Technology and design solutions for an interposer-based computing device targeting HPC applications, Integration of devices in a Multi- Chip-Module (MCM), System and middleware SW stack. COORDINATING ORGANISATION CEA - Commissariat à l Energie Atomique et aux énergies alternatives, France OTHER PARTNERS Arm Limited, UK ETH Zürich, Switzerland FORTH, Greece Fraunhofer ITWM, Germany Scapos AG, Germany University of Manchester, UK Bull SAS (Atos Group), France Virtual Open Systems, France Barcelona Supercomputing Centre, Spain Forschungszentrum Jülich, Germany Kalray SA, France CNRS - Centre National de la Recherche Scientifique, France

30 ExaNoDe HW Prototype: Integration Hierarchy Node: Xilinx MPSoC FPGA for ARM core and reconfigurable HW Chiplet for HW accelerator (CNN) Chiplet SoC Multi-Chip-Module Chiplet SoC MPSoC FPGA MPSoC FPGA Accelerator Chiplet SoC Chiplet SoC DMA Multi-Chip-Module MPSoC FPGA Coherence island MPSoC FPGA Silicon Interposer Organic Substrate Silicon Interposer Organic Substrate Daughter Board DDR DDR DDR DDR Mezzanine Board (ExaNeSt project)

31 Multi-Chip-Module Objective & Challenge: Coarse grain heterogeneous integration Warpage Architecture: Laminate substrate Two FPGA bare dies and one silicon interposer Cu/Ni lid Design: Interposer routing to FPGA and decoupling capacitors: Chiplet Interposer

32 2 - PHOTONICS 2 - Photonics 32

33 Si photonic subsystem Large scale circuit integration Architecture & circuit design KEY TECHNOLOGIES FOR CHIP-TO-CHIP PHOTONIC COMMUNICATION IN POST-EXASCALE COMPUTING Heterogeneous computing system on large-scale silicon interposer Full WDM link integration Die assembly on interposer Optical NoC topology Integrated lasers Fine-grain thermal control Circuit-switched routing Optical IO Fiber coupling TSV & microbump IOs Thermal dissipation Mechanical stress Generic E/O chiplet for communication Routing, flow-control & arbitration Tx/Rx electro-optical drivers Dense integration Autonomous thermal control Integration in computing fabric 33

34 OPTICAL LINK ON PHOTONIC INTERPOSER HIGH PERFORMANCE COMPUTING Heterogeneous integration Optical link on Photonic Interposer 34

35 LETI S OPTICAL NETWORK-IN-PACKAGE NOCS 2015 ISSCC 2018 < Optical network-in-package to interconnect microprocessors and memories System Architecture Design Technology Cache Coherent Compute Fabric with: 96 cores (MIPS32), Optical NoC. Design challenges demonstrated on silicon: Thermal control of WDM devices E/O co-design of drivers Optimized short-range optical WDM system Reference design with 6 chiplets, 96 cores, 8 transceivers available. Preliminary integration of E/O transceiver and photo-diode Photonic interposer 35

36 DEMONSTRATION OF A THERMALLY TUNED WDM ELECTRO-OPTICAL LINK CMOS+Si-Photonics 3D stack Optical fiber array 1Tbps/mm² bandwidth density Tight technology integration of E/O ring modulators within a 3D stack Integrated thermal tuning robust to compute fabric heating [Y. Thonnart, ISSCC2018] Chip-on-board integration Y. Thonnart & al. ISSCC

37 Si-Photonics Architecture Packaging LETI S SI-PHOTONICS ROADMAP FOR POST-EXASCALE COMPUTING TSV for CMOS TSV for Si-Pho WDM link E/O Micro rings ONoC Thermal tuning Target demonstrator core cache-coherent processor Generic E/O chiplets 8-node optical NoC 576 Gbit/s aggregated bandwidth 384 microring resonators ~10 ns electro-optical latency 37

38 3 - IN MEMORY COMPUTING 3 - In-Memory- Computing 38

39 BREAK THE MEMORY WALL! BUT HOW? "memory wall" 11 TB/s; BF ratio = TB/s; BF ratio = TB/s; BF ratio = 0.37 Source Fujitsu HotChips 2018 is nowaday the main limitation for high performance computing it s time to consider data centric architectures toward in-memory computing 39

40 HOW TO DEFINE IN-MEMORY COMPUTING UNIT? in-memory computing = memory with computing R =row R =row R =row k datax datay dataz Computing means selected data during a computing operation is in-situ processed (w/o external processing R =row W =row R =row R =row k data_op(x,y,z) datax data_op(x,y,z) datay dataz Computing means in-situ processed data can be directly written-back Source Leti data_op(x,y,z) 40

41 row decoder row selector HOW TO INTERACT WITH IN-MEMORY COMPUTING UNIT? in-memory computing = memory able to execute microinstructions in-situ memory unit multi-row selection IMC unit bitcell array bitcell array FSM IO (read/write) instruction decoding FSM IO (read/write) processing unit FSM+IO (compute) Source Leti R/W DATA_IN DATA_OUT SYSTEM BUS R/W DATA_IN DATA_OUT SYSTEM BUS 41

42 Compilation Interpretation Analysis EXPECTED GAIN: PRELIMINARY RESULTS FROM LETI Exploration Simulation Platform Algorithm C-code + intrinsic LLVM IR Execution Trace Performance Preliminary evaluation results: Execution time speed-up factor from 10x to 10000x Energy reduction factor from 3x to 29x 42

43 NON VOLATILE MEMORY (NVM) LANDSCAPE PCM MRAM OXRAM/CBRAM Everspin 64Mb DDR3 STT-RAM embedded RRAM 2Mb Avalanche Technologies STT- MRAM 32Mb 128kbit CBRAM 8-bit controller with embedded RRAM 43

44 NEW ARCHITECTURE PARADIGMS WITH NVM Current system 1-Smart SSDs 2-Main memory replacement 3-Cache replacement 4-Post-Exascale Architectures 25 % 75 % Non-Volatile Memories invade logic. Data movement energy reduced. 100 % In-Memory Computing Execution time Memory stalls Memory bound architecture Balanced architectures Disruptive architecture Compute CPU CPU CPU CPU CPU CPU CPU CPU SoC L1 $ L1 $ L2/L3 $ L1 $ L1 $ L2/L3 $ L1 $ L1 $ L2/L3 $ L1 $ L1 $ NVM NVM Ext. Mem. DRAM DRAM DRAM NVM DRAM I/O Disk/SSD NVM Disk/SSD Disk/SSD Disk/SSD Other nodes 44

45 IT S NOT OVER EXASCALE LEVEL COMPUTING IS ALSO: Low power design: Not possible to run all cores at (Vmax, Fmax) due to heat dissipation Low voltage operation, Power domains, Probes, actuators, control. Security: HPC compute nodes are open to external world (access to data) Root of trust, Secure boot, Isolation for secure services, Chip monitoring. Variable precision: Fixed floating point arithmetic reaches its limits: undesirable calculation effects, too heavy for new workloads, New arithmetic formats: variable precision, UNUM s e f sign exponent fraction 45

46 EUROPEAN HPC EXASCALE EFFORT 46

47 EUROHPC: A NEW LEGAL AND FUNDING ENTITY FOR DEPLOYING IN EUROPE A WORLD-CLASS SUPERCOMPUTING INFRASTRUCTURE The EuroHPC Joint Undertaking represents a public investment of EUR 1 billion between the European Union and 25 participating European countries. Source: 47

48 NEXT STEPS EuroHPC will initially operate from 2019 to 2026 EuroHPC will support activities through procurement and open Calls in 2019 and : EuroHPC foresees the initial co-investment with Member States of about EUR 1 billion: To acquire two pre-exascale machines and several petascale systems by 2020, For R&I actions covering the full HPC ecosystem : Further funds would allow a full coverage of the HPC strategy: Acquisition in of two exascale systems, at least one of them with European technology, one post-exascale system. Source: 48

49 European Processor Initiative European independence in High Performance Computing Technologies EU Exascale machine based on EU processor by 2023 Based on a solid, long-term economic model 49 16/11/2018 PN European Processor Initiative Subject to change

50 European Commission (EC) expectations & EPI value proposal EPI expected impacts (as per EC request) Get a world class processor for the Exascale machines supplied by EuroHPC in 2023 Develop a sustainable economic model Technology drive High Performance Computing needs for Exascale machines and beyond Connected mobility & Advanced Driver Assistance Systems (ADAS) computing needs beyond 2023 Servers, Cloud, Edge Low Power CPU needs 50 16/11/2018 PN European Processor Initiative Subject to change Business drive

51 Multiple expertises and excellent combo AUTOMOTIVE EXPERTS INDUSTRY EXPERTS HPC & RESEARCH 51 16/11/2018 PN European Processor Initiative Subject to change

52 CONCLUSION era Heterogeneous integration Heterogeneous Architecture & Integration > era Disruptive architectures Optical Network-on-Chip In-Memory-Computing European HPC strategy 52

53 THANK YOU QUESTIONS? D. Dutoit, E. Guthmuller, JP Noël, Y. Thonnart, P. Vivet Leti, technology research institute Commissariat à l énergie atomique et aux énergies alternatives Minatec Campus 17 rue des Martyrs Grenoble Cedex France