Exploiting Dark Silicon in Server Design. Nikos Hardavellas Northwestern University, EECS

Transcription

1 Exploiting Dark Silicon in Server Design Nikos Hardavellas Northwestern University, EECS

2 Moore s Law Is Alive And Well 90nm 90nm transistor (Intel, 2005) Swine Flu A/H1N1 (CDC) 65nm 45nm 32nm 22nm 16nm Device scaling continues for at least another 10 years 2

3 Moore s Good Days Law Ended Is Alive Nov. And 2002 Well [Yelick09] New Moore s Law: 2x cores with every generation 3

4 # Cores Exponential Growth of Core Counts Teraflops TILE Year So, are 1000-core chips a viable architecture? 4 MIT-RAW Sun Rock Larrabee Cell Magny-Cours Niagara Beckton Xeon Istanbul Core 2 QuadAgena Barcelona Core i7 UltraSPARC IV Toliman Power4 Pentium-D Pentium Pro Pentium P-II P-III P-4 Xeon Brisbane Itanium Turion X2 Core 2 Duo 1 Denmark

5 Relative Performance Performance Expectations vs. Reality Speedup under Moore's Law Speedup under Physical Constraints Year of Technology Introduction Physical constraints limit speedup 5

6 Number of Cores Area vs. Power Envelope Cache Size (MB) Good news: can fit 100 s cores. Bad news: cannot power them all 6 Area (310mm) Power (130W)

7 Number of Cores Pack More Slower Cores, Cheaper Cache Area (310mm) Power (130W) 1 GHz, 0.27V 2.7 GHz, 0.36V 4.4 GHz, 0.45V 5.7 GHz, 0.54V 6.9 GHz, 0.63V 8 GHz, 0.72V 9 GHz, 0.81V Cache Size (MB) The reality of The Power Wall: a power-performance trade-off 7 VFS

8 Number of Cores Pin Bandwidth Constraint Area (310mm) Power (130W) 1 GHz, 0.27V 2.7 GHz, 0.36V 4.4 GHz, 0.45V 5.7 GHz, 0.54V 6.9 GHz, 0.63V 8 GHz, 0.72V 9 GHz, 0.81V Bandwidth (1 GHz) Bandwidth (2.7GHz) Cache Size (MB) Bandwidth constraint favors fewer + slower cores, more cache 8 VFS

9 Performance (GIPS) Example of Optimization Results Power + BW: ~5x loss BW: ~2x loss Area (max freq) Power (max freq) Bandwidth, VFS Area+Power, VFS Area+P+BW, VFS Cache Size (MB) Jointly optimize parameters, subject to constraints, SW trends Design is first bandwidth-constrained, then power-constrained 9

10 Number of Cores Core Counts for Peak-Performance Designs Max EMB Cores Embedded (EMB) General-Purpose (GPP) Physical characteristics modeled after UltraSPARC T2 (GPP) ARM11 (EMB) Year of Technology Introduction Designs > 120 cores impractical for general-purpose server apps B/W and power envelopes + dataset scaling limit core counts 10

11 Scaling Factor Application Dataset Scaling Year of Technology Introduction OS Dataset Scaling (Muhrvold's Law) Transistor Scaling (Moore's Law) TPC Dataset (Historic) Application datasets scale faster than Moore s Law! Big Caches 11

12 Scaling Factor Pin Bandwidth Scaling Transistor Scaling (Moore's Law) Pin Bandwidth Year of Technology Introduction Off-chip bandwidth scales slowly (#pins, off-chip clock) Big Caches 12

13 Scaling Factor Supply Voltage Scaling Transistor Scaling (Moore's Law) Supply Voltage (ITRS) Year of Technology Introduction Supply voltage scaling is SLOW! Dark Silicon 13

14 Watts / Chip Chip Power Scaling Year of Technology Introduction Max Power (air cooling + heatsink) Chip Power (ITRS) Chip power does not scale! 14

15 Range of Operational Voltage [Watanabe et al., ISCA 10] Shrinking range of operational voltage hampers voltage-freq. scaling 15

16 Mitigating Bandwidth Limitations: 3D-stacking [Philips] [Loh et al., ISCA 08] [Amcor Tech] Delivers TB/sec of bandwidth; use as large in-package cache 16

17 Performance (GIPS) Performance Analysis of 3D-Stacked Multicores Cache Size (MB) Area (max freq) Power (max freq) Bandwidth, VFS Area+Power, VFS Area+P+BW, VFS Chip becomes power-constrained 17

18 Die Size (mm 2 ) Exponentially Large Die Area Left Unutilized Max Die Size DB2-TPCC DB2-TPCH Apache Trendline (exp.) Year of Technology Introduction Dark Silicon!!! Should we waste it? 18

19 Example of a Specialized Multicore Chip ILP OoO Core ILP OoO Core SIMD SIMD Many Threads SIMD SIMD Many Threads Reconfigurable DSP Crypto TCP Many custom cores on chip; power only the most useful ones 19

20 Core Specialization Existing general designs OoO for ILP, in-order MT for memory-latency-bound, SIMD for data-parallel, systolic arrays Customizable cores Tensilica Xtensa (custom ISA and datapath, operation fusion) Reconfigurable logic Generality of implemented operations Target specific application Common macro-operations General ISA Trade-offs in performance, power, programmability, generality Wide range of heterogeneity and specialization meanings 20

21 First-Order Core Specialization Model 720p HD H.264 encoder (high-definition video encoder) Several optimized implementations exist Commercial ASICs, FPGAs, CMP software Wide range of computational motifs Frames per sec Energy per frame (mj) ASIC 30 4 CMP Performance gap with ASIC Energy gap with ASIC IME x 707x FME x 468x Intra x 157x CABAC x 261x 21 [Hameed et al., ASPLOS 10]

22 Speedup Performance of Specialized Multicores Year of Technology Introduction Ideal-P + 3D mem Ideal-P EMB + 3D mem EMB GPP + 3D mem GPP Specialized multicores deliver 2x-12x higher performance 22

23 Number of Cores Core Counts for Specialized Multicores Year of Technology Introduction 23 Max EMB Cores EMB + 3D mem EMB GPP + 3D mem Only few cores need to run at a time; large die area allow many cores Power constraints? Yield?

24 Taming Power and Bandwidth : Nanophotonics Optical Interconnect Split chip into chiplets, spread in space Ease cooling and power delivery, high yield; photonics for bandwidth 24

25 Nanophotonic Components Rings selectively couple optical energy of a specific wavelength Ge-doped 64 wavelengths DWDM, 3 ~ 5μm waveguide pitch, 10Gbps per link ~100 Gbps/μm bandwidth density!!! [Batten et al., HOTI 08] 25

26 Technology: Off-chip Channel Material Material Silicon Waveguide Optical Loss Propagation Speed Pitch (density) 0.3 db/cm* 0.286c 20um Optic Fiber 0.2 db/km 0.676c 250um Optical fiber is low-loss, high speed Enables further spreading out chiplets. BW density was a challenge (fiber pitch size is large) 26 * J. Cardenas et al., Optics Express 2009 Fiber: low optical loss, high speed, flexibility eases assembly

27 Technology: Dense Off-Chip Coupling Dense optical fiber array. [Lee et al., OSA / OFC/NFOEC 2010] <1dB loss, 8 Tbps/mm demonstrated. Tapered couplers solved bandwidth problem, demonstrated Tbps/mm 27

28 Galaxy Overall Architecture Electrical cluster Chiplet 3 Chiplet 0 Chiplet 2 Cross-chiplet assemblies share an optical bus, forming optical crossbars (FlexiShare) Laser Source Optical fiber couplers dst Chiplet 1 Chiplet 0 src Chiplet 3 Chiplet 4 28

29 Large-Scale Interconnects 200mm 2 die, 64 routers per chiplet, 9 chiplets, 16cm fiber Supports > 1K cores! 29

30 Conclusions Physical constraints and software pragmatics limit core counts and performance Emerging/exotic technologies may solve some problems 3D-memory for bandwidth Nanophotonics for bandwidth, power, yield Need to reduce wasted energy per unit of work Heterogeneity, only power the few cores needed Need to innovate across software/hardware stack Programmability, tools are a great challenge Scaling forces caches to grow exponentially Address data management both at cache and software 30

31 Thank You! Acknowledgements: Y. Pan, J. Kim, G. Memik, M. Ferdman, B. Falsafi 31