Energy Efficiency and Resilience in Future ICs

Size: px

Start display at page:

Download "Energy Efficiency and Resilience in Future ICs"

Rudolf Bradley
6 years ago
Views:

1 Energy Efficiency and Resilience in Future ICs Andrew B. Kahng UCSD VLSI CAD Laboratory CSE and ECE Departments University of California, San Diego

2 Outline The Power Gap Low-Power Design Beyond Low-Power: Resilience Conclusion 2

3 Part I: Power Crisis The Power Gap Low-Power Design Beyond Low-Power: Resilience Conclusion

4 Value From Semiconductor Scaling Value is enabled by integration Greater utility Less cost Scaling enables new products Density: more functions per chip Device: better performance Device Density Product Challenge: Maintain Scaling of Value Variability larger design guardband Cost technology and design risk Leakage waste of increasingly expensive energy 4

Scaling of Density Layout density increase Capacitance density increase (nf/mm 2 ) 25 20 15 10 5 Capacitance density + 10% / year P dynamic CV 2 8008 8080 4004 8086 Intel 386 286 Intel Pentium Intel

5 Scaling of Density Layout density increase Capacitance density increase (nf/mm 2 ) Capacitance density + 10% / year P dynamic CV Intel Intel Pentium Intel Intel Pentium 4 Intel Pentium III Intel Pentium II 2,600,000,000 Intel Itanium 2 Intel Itanium 3,000,000, Core Intel Xeon Dual-Core Intel Itanium NVIDIA GF100 NVIDIA GT200 AMD RV700 NVIDIA G80 transistors 10,000,000,000 1,000,000, ,000,000 10,000,000 1,000, ,000 10,000 1,000 5

6 Scaling of Product Video Audio Voice MPEG1 Extraction JPEG MPEG4 MPEG2 Extraction Compression Sentence Translation Dolby-AC3 MPEG Word Recognition 100 GOPS Voice Auto Translation Graphics 2D Graphics Communication Recognition Modem FAX 3D Graphics 10Mpps 100Mpps VoIP Modem SW Defined Radio Face Recognition Voice Print Recognition Moving Picture Recognition Required performance for multimedia processing (GOPS: Giga Ops/Sec) 2007 ITRS Consumer-Stationary SOC Driver: 220 TFlops on a single chip by

7 Scaling of Device To meet the performance scaling Mobility enhancement Vdd slowly lowering I ds C ox V dd V th C ox ( 1/ t ox ) increasing t ox lowering V th lowering Gate leakage Scaling of transistor intrinsic speed of highperformance logic (ITRS 2009, 13%/year) Subthreshold leakage 7

8 Scaling of Device 8

9 The Power Gap Capacitance Frequency Tox, Vth Functions Density Product (CV/I) Device P dynamic Power Crisis P leakage Higher VDD? Quadratic P dynamic increase VDD? Lower VDD? Lower Vth Exponential P leakage increase 9

Power Limits the Technology Roadmap ITRS MPU clock frequency roadmap (ABKGroup since 2001 ITRS) Normalized frequency 20 18 16 14 12 10 8 6 4 Frequency (GHz) 100 10 High-Performance Device Intrinsic

10 Power Limits the Technology Roadmap ITRS MPU clock frequency roadmap (ABKGroup since 2001 ITRS) Normalized frequency Frequency (GHz) High-Performance Device Intrinsic Speed (1 / ) Normalized to % / year (2001 ITRS) before ITRS 2007 ITRS ( = CV / I) 13% / year (2009 ITRS) 2011 ITRS (tentative/planned) 8% / year (2011 ITRS) (tentative / planned) View ITRS MPU Model 10

11 The Power Gap (roadmapped by UCSD since 2001) Low-Power Design Beyond Low-Power: Resilience Conclusion

12 Digression: Concept of Timing Slack Many power optimizations convert positive timing slack into power reductions: smaller transistors, area, power, But, this is not easy! Transistors in positive-slack T arrival T required cells can have higher V th, larger L gate, more variation, CLK Slack = T required T arrival CLK 12

Even If We Slow Down Frequency Scaling Normalized Freq. 3.5 3 2.5 2 1.5 1 Cap. scaling: 2 Tr.

scaling: 4% / year 1995 2005 2015 2025 100,000 200 180 160 140 120 100 80 60 40 20 0 Normalized Cap.

13 Even If We Slow Down Frequency Scaling Normalized Freq Cap. scaling: 2 Tr.density / 2 years 2 L gate Freq. scaling: 4% / year , Normalized Cap. Roadmap of lowpower techniques! Clock Gating Multi-Vth Multi-Core Arch L gate Bias* Power Gating* Adaptive Body Bias MPU Power (W) 10,000 1, Practical Power Limit Multi-Vdd DVFS* * Work at UCSD

14 Clock Gating P dynamic Reduction Dynamic Power Reduction Cumulative efficiency (%) 9000% 8000% 7000% 6000% 5000% 4000% 3000% 2000% 1000% 0% Clock gating 2834% 100% 300% 270% 270% 525% Technology node (nm) 8503% Leakage Power Reduction Cumulative efficiency (%) 35000% 30000% 25000% 20000% 15000% 10000% 5000% 0% 32400% 9000% 100% 100% 300% 300% Technology node (nm) %

15 Multi-Vth P leakage Reduction Critical Timing Path Dynamic Power Reduction Cumulative efficiency (%) 9000% 8000% 7000% 6000% 5000% 4000% 3000% 2000% 1000% 0% Clock gating 2834% 100% 300% 270% 270% 525% Technology node (nm) 8503% Low-V th High-V th Leakage Power Reduction Cumulative efficiency (%) 35000% 30000% 25000% 20000% 15000% 10000% 5000% 0% 9000% 100% 100% 300% 300% Multi-Vth Technology node (nm) 32400% %

100 8503% 2834% 50 0 Leakage Power Reduction Factor Cumulative efficiency (%) 35000% 30000% 25000% 20000% 15000% 10000% 5000%

16 Multi-Core Architecture P dynamic Reduction Dynamic Power Reduction Cumulative efficiency (%) 9000% 8000% 7000% 6000% 5000% 4000% 3000% 2000% 1000% 0% Clock gating Multi-Vth 100% 300% 270% 270% 525% Multi-cores 150 Technology node (nm) % 2834% 50 0 Leakage Power Reduction Factor Cumulative efficiency (%) 35000% 30000% 25000% 20000% 15000% 10000% 5000% 0% 32400% 16200% 9000% 100% 100% 300% 300% Technology node (nm) M. Domeika, drdobbs.com, Dec. 27,

17 Gate Length Biasing P leakage Reduction Dynamic Power Reduction Cumulative efficiency (%) 9000% 8000% 7000% 6000% 5000% 4000% 3000% 2000% 1000% 0% Multi-cores Clock gating Multi-Vth 100% 300% 270% 270% 525% Technology node (nm) % 2834% 50 0 Leakage Power Reduction Cumulative efficiency (%) 35000% 30000% 25000% 20000% 15000% 10000% 5000% 0% 16200% 9000% 100% 100% 300% 300% Body biasing Power gating L gate biasing 150 Technology node (nm) %

18 Transistor Gate-Length Biasing UCSD 2003 Leakage and Delay vs. Gate Length Bias Impact Exponential (I sub ) Leakage reduction Variability reduction Linear Performance reduction Apply very small biases (+2nm, +4nm, etc.) just before tapeout Delay 18

Transistor Gate-Length Biasing UCSD 2003 Transistor on non-critical path: target CD 70nm Transistor on near setupcritical path: target CD 66nm Transistor on

19 Transistor Gate-Length Biasing UCSD 2003 Transistor on non-critical path: target CD 70nm Transistor on near setupcritical path: target CD 66nm Transistor on setup-critical path: target CD 65nm Challenging global optimization over millions of gates, with complex timing constraints (Spent two years developing a leading industry tool ) 19

20 Transistor Gate-Length Biasing UCSD 2003 Leakage and Delay vs. Gate Length Bias Impact Exponential (I sub ) Leakage reduction Variability reduction Linear Performance reduction Apply very small biases (+2nm, +4nm, etc.) just before tapeout Chip-scale optimization: trade timing slack for leakage power everywhere possible UCSD-patented flow currently offered in TSMC s Green Power Trim service Energy savings for just AMD/ATI Radeon GPUs: >> 10 9 watt-hours Delay 20

21 Power Gating P leakage Reduction Dynamic Power Reduction Cumulative efficiency (%) 9000% 8000% 7000% 6000% 5000% 4000% 3000% 2000% 1000% 0% Multi-cores Clock gating Multi-Vth 100% 300% 270% 270% 525% Technology node (nm) % 2834% 50 0 Leakage Power Reduction Cumulative efficiency (%) 35000% 30000% 25000% 20000% 15000% 10000% 5000% 0% 100% 100% 300% 300% Body biasing Power gating L gate biasing % Technology node (nm) % 16200%

22 Power Gating Typical operation modes: active, idle Power gating: cut off leakage path during predefined idle modes Typical operation with power gating: VDD VSS_INT VSS idle active Logic block active idle active PG_ENABLE Current sleep Without power gating With power gating wake up More than 10x leakage saving during idle mode 22

23 New: Runtime Power Gating Joint with Tajana Rosing and Rick Strong, UCSD Clock gating and power gating address dynamic and leakage power during idle mode But in active mode, cycles are still wasted E.g., execution time spent waiting for memory access What If. Circuit design (wakeup logic) to enable faster, more flexible wakeup of one or more cores? shorter power gating intervals runtime power gating 23

24 Token-Based Power Gating Architectural power gating: Send tokens to control core power gating About tokens: Sent by cache or memory controller Received by core Stamped with system cycle in which it was generated Has estimated request wait latency Sets minimum core wakeup latency Managed by token controller Memory Joint work with Tajana Rosing and Rick Strong, UCSD L2$ miss L1$ miss Core PG-L1-Miss PG-L2-Miss Time 24

25 Token-Based Power Gating Joint work with Tajana Rosing and Rick Strong, UCSD About token controller: Manages token properties Queries cores for performance information Maintains peak current constraint by managing core wakeup latencies Maximizes energy savings, e.g., by balancing wakeup latencies for parallel apps, or increasing wakeup latencies for underutilized cores Memory L2$ miss L1$ miss Core PG-L1-Miss PG-L2-Miss Time 25

26 System Model and Tool Flow Multi-core assumptions 4 cores Private L1 (32KB-2way-0.5ns), L2 caches (2MB-2way-9ns) MESI cache coherence protocol Memory: Size = 2GB Latency = 40ns Core Type: In-order EV4 ISA: ALPHA64 Frequency: 2GHZ Width: 2 OS: Vanilla-Linux Tools Joint work with Tajana Rosing and Rick Strong, UCSD M5 Full-System Simulator (Spec2006, Parsec2.0, Splash2.0 benchmarks) McPAT used to generate power numbers 26

Different memory hierarchy levels have different latencies L1 hit

5ns // L2 hit latency = 9ns // Memory Latency = 40ns Lowering the

27 Wakeup Latencies vs. Energy Saving Joint work with Tajana Rosing and Rick Strong, UCSD Different memory hierarchy levels have different latencies L1 hit latency = 0.5ns // L2 hit latency = 9ns // Memory Latency = 40ns Lowering the core wakeup latency (10ns 5ns 2ns) can make power gating for smaller idle periods more attractive PGT (for both L1 and L2 misses) vs. PGTL2 (for only L2 misses) ~40% energy saving 27

28 And More: ABB / Multi-VDD / DVFS / NTC /... Dynamic Power Reduction Cumulative efficiency (%) 9000% 8000% 7000% 6000% 5000% 4000% 3000% 2000% 1000% 0% Clock gating Multi-Vth Multi-cores 100% 300% 270% 270% 525% Technology node (nm) % 50 Multi-VDD DVFS, AVS RTL-Opt Data gating DC-DC efficiency 8503% 0 Near-threshold computing Leakage Power Reduction Cumulative efficiency (%) 35000% 30000% 25000% 20000% 15000% 10000% 5000% 0% Body biasing Power gating L gate biasing 9000% 100% 100% 300% 300% Technology node (nm) % 16200%

DVFS: Dynamic Voltage/Frequency Scaling DVFS enables Operation at multiple power-performance points Adaptation to different operating conditions or modes Observation 1: DVFS changes only voltage and

29 DVFS: Dynamic Voltage/Frequency Scaling DVFS enables Operation at multiple power-performance points Adaptation to different operating conditions or modes Observation 1: DVFS changes only voltage and frequency, not the design itself is a fixed design always optimal? Observation 2: Lifetime energy changes with scenario (R * X) is scenario-oblivious design always optimal? Different duty cycle (R) Lifetime e.g., talk mode e.g., standby mode Different frequency scaling (X) X = Joint work with Rakesh Kumar and John Sartori, UIUC High performance mode clock frequency Low performance mode clock frequency 29

benefits are different in each module optimal use of replication = knapsack formulation Multi mode

30 DVFS Suboptimality #1 Joint work with Rakesh Kumar and John Sartori, UIUC No single design can work well in all modes: jack of all trades, master of none What If: selective replication Replication benefits are different in each module optimal use of replication = knapsack formulation Multi mode design Selective replication design Example: CTL module has 12% energy savings through replication 30

31 DVFS Suboptimality #1 Joint work with Rakesh Kumar and John Sartori, UIUC No single design can work well in all modes: jack of all trades, master of none What If: selective replication Replication benefits are different in each module optimal use of replication = knapsack formulation avg 9% energy savings High Performance Replica ITLB IRF SPU DTLB Multi-mode design IN mode 1 0 OUT IFU H MUX EXU LSU TLU H MUX Low Performance Replica mode mode IFU L instructio n cache FPRF FFU H FFU L MUX data cache TLU L High Perf. Replica Low Perf. Replica 31

32 DVFS Suboptimality #2 Joint work with Rakesh Kumar and John Sartori, UIUC Lifetime energy is not optimal Energy high performance Why this happens Design A Design B low Delay performance Design A saves energy in high perf mode. Design B saves energy in low perf mode. Neither is optimal for lifetime energy. E = P hi R + P lo (1- R) operating point Timing slack (ns) Freq. voltage Path A Path B 1.0GHz 0.95V MHz 0.60V How to optimize for multiple performance modes? 32

33 Context-Driven Multi-Mode Design Joint work with Rakesh Kumar and John Sartori, UIUC Goal: Find a minimum lifetime energy design Conventional design flow sets constraints (frequency, voltage) before implementation (but the min-energy constraints are unknown!) What If: context-driven multi-mode design: design to the scenario rather than to constraints avg 8% energy savings power consumption 33

34 The Power Gap (roadmapped by UCSD since 2001) Low-Power Design (Lgate biasing, runtime power gating, scenario-aware and replication-based DVFS) Beyond Low-Power: Resilience Conclusion

35 New Mindset Better-than-worst-case (typical case) design Dynamic reliability (error) management Living with variations Further Energy Reductions Resilient Design limit of Worst-Case design Typical-Case + error-tolerance Voltage scaling ( )

36 Types of Resilience Error Acceptance Error Tolerance Error Avoidance Key Ideas Allow errors Approximate computation for accuracy insensitive applications Detect and correct errors dynamically Error detection FF + architectural correction schemes No error allowed DVFS + canary circuits UCSD Works Approximate arithmetic design Recovery driven design Design Dependent Ring Oscillator 36

37 The Power Gap Low-Power Design Beyond Low-Power: Resilience Error Tolerance Error Acceptance Error Avoidance Recovery-Driven Design Approximate Arithmetic Logic Design-Dependent Ring Oscillator Conclusion

38 Recovery-Driven Design [HPCA10] [DAC10] Motivation #1: If the design uses an error-tolerance mechanism, then the design process should be modified accordingly Motivation #2: Error rate demands the use of functional information Joint work with Rakesh Kumar and John Sartori, UIUC 38

39 Recovery-Driven Design [HPCA10] [DAC10] Joint work with Rakesh Kumar and John Sartori, UIUC Low-power methodology for error-tolerant designs Minimize power for a target error rate Slack redistribution with functional information Voltage Scaling Path Optimization Power Reduction reduce voltage until the error rate exceeds a target optimize frequently exercised, negative slack paths reducing power w/o affecting error rate 39

Recovery-Driven Design: Experimental Results Path toggling extraction and error rate estimation accurate fast (20X) Power comparison across design techniques 25% power savings w/ 2% error 22%

40 Recovery-Driven Design: Experimental Results Path toggling extraction and error rate estimation accurate fast (20X) Power comparison across design techniques 25% power savings w/ 2% error 22% power savings w/ Razor flip flop Power Consumption (W) Conventional P&R Tight P&R PCT Slack Optimizer Power Optimizer % 0.25% 0.50% 1.00% 2.00% 4.00% 8.00% Error Rate (%) 40

Resilient Overhead Reduction (Ongoing Work) Resilient overhead: For the resilience, design overheads are required i.e., additional circuit and operations (pipeline flush) New tradeoffs in resilient design Pros.

41 Resilient Overhead Reduction (Ongoing Work) Resilient overhead: For the resilience, design overheads are required i.e., additional circuit and operations (pipeline flush) New tradeoffs in resilient design Pros. tradeoff ET register avoid over design, voltage scaling further Error tolerant (ET) registers Razor flip flop Cons. cost for ET registers, recovery overhead Goal: Minimize the cost function (power) using the tradeoffs Approach: Find optimal assignment of registers (error-tolerant or normal) 41

42 The Power Gap Low-Power Design Beyond Low-Power: Resilience Error Tolerance Error Acceptance Error Avoidance Recovery-Driven Design Approximate Arithmetic Logic Design-Dependent Ring Oscillator Conclusion

Error Avoidance Adjust Vdd,Freq. according to delay margin No error recovery mechanism not required Estimated delay DVFS controller Monitoring How to Circuit feedback delay margin?

43 Error Avoidance Adjust Vdd,Freq. according to delay margin No error recovery mechanism not required Estimated delay DVFS controller Monitoring How to Circuit feedback delay margin? Delay Vdd, Freq Actual Circuit Delay constraint guardband Conventional approaches Inverter-based RO: Critical paths have different sensitivity to process variations Critical path RO: Replicating critical path with long interconnect costs area Time Error avoidance system Worst case design 43

Monitor for Resilience: DDRO 1 Delay. V Delay nom th 1 Delay.

L gate Delay nom path (A+B) Problem: Measure real-time performance variation in an

with similar delay sensitivity to variations (Lgate, Vth, Tox, V, T, ) as actual critical

critical paths having similar sensitivities to reduce number of RO Low area overhead

44 Monitor for Resilience: DDRO 1 Delay. V Delay nom th 1 Delay. V Delay nom th 1 Delay. V Delay nom th Gate A Gate B 1 Delay. L gate Delay nom 1 Delay. L gate Delay nom path (A+B) Problem: Measure real-time performance variation in an adaptive system Approach: Select gates to form designdependent ring oscillators (DDROs) with similar delay sensitivity to variations (Lgate, Vth, Tox, V, T, ) as actual critical paths Potential Benefits: Specific to path s rising or falling transition Can cluster critical paths having similar sensitivities to reduce number of RO Low area overhead Automated design flow, standard cells only 1 Delay. V Delay th nom DDRO 1 Delay. L gate Delay nom Critical path 1 Delay. L Delay gate nom 44

DDRO Synthesis Flow Gate sensitivities Critical path

V Delay nom th Critical path Cluster 1 Cluster

L gate Delay nom Cluster 2 For each cluster,

SOI test chip ARM Cortex M3 DDRO Delay sensitivity

45 DDRO Synthesis Flow Gate sensitivities Critical path sensitivities 1 Delay. V Delay nom th Critical path Cluster 1 Cluster critical paths DDRO error 1 Delay. L gate Delay nom Cluster 2 For each cluster, synthesize a DDRO using integer linear program 45nm SOI test chip ARM Cortex M3 DDRO Delay sensitivity Error (%) Synthesis result INV. RO CPRO DDRO Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Average 45

46 Monte Carlo Simulation Results (30 samples) Estimated delay (ns) Without within-die variation modeling Estimated delay (ns) Estimated delay (ns) 1.2 Estimation error : -1.4 % ~ 3.7 % 1.2 Estimation error : -2.0 % ~ 4.1 % 1.2 Estimation error : -4.3 % ~ 7.1 % DDRO Critical path RO Inv. RO Actual delay (ns) Actual delay (ns) With within-die variation modeling Actual delay (ns) Estimated delay (ns) Estimated delay (ns) Estimated delay (ns) 1.2 Estimation error : -0.5 % ~ 3.7 % 1.2 Estimation error : -1.3 % ~ 3.6 % 1.2 Estimation error : -1.7 % ~ 5.1 % DDRO Critical path RO Inv. RO Actual delay (ns) Actual delay (ns) Actual delay (ns) 46

47 The Power Gap (roadmapped by UCSD since 2001) Conclusion Low-Power Design (Lgate biasing, runtime power gating, scenario-aware and replication-based DVFS) Beyond Low-Power: Resilience (recovery-driven design, approximate arithmetic, design-dependent RO)

48 The Elephant Big picture for power and resilience spans Software and applications Architectures Interconnects Memories Circuits and devices Technology Fundamental limits Different animal from recent talk topics: Design for Manufacturability, Technology Roadmap, 22nm Chip Implementation, 3D PDN Pathfinding,! 48

49 What I Spend My Time On Connecting and building: dots, bridges, big pictures, The IC Design-Manufacturing Interface The ITRS roadmap IC physical design (clustering, placement, interconnect design, ) NOC modeling and optimization (ORION2.0, trace-driven optimizations,...) Utilities for teaching (math drill generator, automatic editor, ) Current projects MARCO Gigascale Systems Research Center: Physical Architecture Components: Models, Roadmaps and Integrations = system-level impacts of 3D, new memories, new design optimizations UC Discovery: Integrated Modeling, Process, and Computation for Technology (IMPACT) Center (Design-Manufacturing Interface) SRC (with UIUC): New Directions in Architecture and Design of Scalable Energy Constrained SoCs SRC (with UCLA): New Directions in Design-Aware Manufacturing NSF (with UCLA): Research on Benchmarking and Robustness of VLSI Sizing Optimizations Qualcomm: Power Delivery Pathfinding for 3D Through-Silicon Stacking STMicroelectronics: Across-Field Variation Mapping From Silicon Measurements, and Design-Driven DoseMap Flow 49

50 Thank You! 50

OUTLINE Introduction Power Components Dynamic Power Optimization Conclusions

OUTLINE Introduction Power Components Dynamic Power Optimization Conclusions 04/15/14 1 Introduction: Low Power Technology Process Hardware Architecture Software Multi VTH Low-power circuits Parallelism