The Future of Electrical I/O for Microprocessors. Frank O Mahony Intel Labs, Hillsboro, OR USA

Transcription

1 The Future of Electrical I/O for Microprocessors Frank O Mahony frank.omahony@intel.com Intel Labs, Hillsboro, OR USA 1

2 Outline 1TByte/s I/O: motivation and challenges Circuit Directions Channel Directions Tool Directions 470Gb/s Prototype 2 2

3 Microprocessor Bandwidth Needs As CPU core count increases, I/O bandwidth (BW) requirements will increase for all segments Current system bandwidth requirements (Y2010) Client BW = ~50GB/s Server BW = ~100GB/s High-end Server BW = ~200GB/s Server Example High-End Server Example 3

4 Microprocessor Bandwidth Trends Bandwidth (GB/s)1000 3X/2yrs. 2X/2yrs Bandwidth Drivers: CPU Memory CPU CPU CPU Peripheral CPU I/O bridge High-end microprocessors are expected to need ~1TB/s during coming decade 4 4

5 Microprocessor I/O Power Current system I/O power efficiency is 20-40pJ/bit System BW I/O Pwr. Eff. I/O Pwr Client ~50GB/s 20pJ/bit 8W Server ~100GB/s 20pJ/bit 16W High-End Server ~200GB/s 20pJ/bit 32W If I/O power efficiency doesn t improve during the next decade, then: 1TB/s x 20pJ/bit = 160W 5

6 I/O Energy Efficiency Trends 100 Energy Eff. (pj/bit) 10-20%/year Ref: R. Palmer, ISSCC Year 2010 Issue: ~20% per year power reduction while bandwidth increasing 40-70% per year 6

7 Energy Efficiency and Channel Loss Tradeoff 100 Energy Eff. (pj/bit) 10 Based on transceivers reported in nm CMOS Channel Symbol rate (db) Power efficiency is strongly correlated to channel loss Simply scaling per-pin BW will not meet power budget Low power interfaces should be wider not faster 7 7

8 Density Channel/Interconnect Density Conventional package/socket density does not scale with process Width of interfaces is limited by routing congestion Time C4 bump area Circuit area Flip-chip Package µp C4 pitch << Pkg. pin pitch 8

9 Problem Statement Summary Bandwidth needs are quickly approaching 1TB/s Energy efficiency is not scaling as aggressively as bandwidth The channel limits our ability to increase per-pin data rate and/or increase the width of an interface 9

10 How Will Electrical I/O scale to 1TB/s? 1. Co-design the interconnects and I/O circuitry to meet bandwidth, scalability and power efficiency demands 2. Scale the channel by transitioning to new channel configurations and materials 3. Use accurate, statistical link design tools to identify balanced architectures. 10

11 Outline 1TByte/s I/O: motivation and challenges Circuit Directions Channel Directions Tool Directions 470Gb/s Prototype 11 11

12 Power Eff. (mw/gb/s) Low Active Power Techniques Channel Symbol rate (db) Power Optimized Links Simple equalization Low TX swing Sensitive RX sampler Low-power clocking 12 12

13 Minimize analog circuit complexity Lowest power links find ways to simplify equalization and clocking circuitry to reduce power Equalization examples: Constrain equalization range by known channel characteristics Continuous-time linear Rx equalizer RX C PAD L R TERM Ref: G. Balamurugan, JSSC 4/

14 Power Management: Scalable supplies Adapt supply to frequency, process, temperature (f,p,t) Digital: Power V SUPPLY2 f Analog: Power V SUPPLY I bias Removes excess circuit BW and headroom V SUPPLY V REF - + V SUPPLY V REG (f,p,t) VR TX V REG (f,p,t) C REG RX Regulated supply ring VCO Data link with adaptive supply 14 14

15 Power Management: Scalable supplies TX Driver TX Ser/Pre Energy Eff. (pj/bit) TX Clk RXFE RX Clk 0 5Gb/s 0.68V 10Gb/s 0.85V 15Gb/s 1.05V 20Gb/s 1.2V Power efficiency improves with adaptive supply/biasing Refs: G. Balamurugan, JSSC 4/08 and B. Casper, ISSCC

16 Aggressive Power Management Normalized Bandwidth Demand Don t spend power doing nothing! Rapidly adapt to bandwidth demand Requires fast, granular bandwidth adaptation Time 16

17 Aggressive Power Management Conventional (fixed Bandwidth) Wasted Energy Normalized Bandwidth Demand Don t spend power doing nothing! Rapidly adapt to bandwidth demand Requires fast, granular bandwidth adaptation Time 17

18 Aggressive Power Management Conventional (fixed Bandwidth) Energy Savings Normalized Bandwidth Demand Adaptive Bandwidth 0.0 Don t spend power doing nothing! Rapidly adapt to bandwidth demand Requires fast, granular bandwidth adaptation Time 18

19 Device Variation in Scaled CMOS Device manufacturing tolerances are improving but area scaling still causes higher variation Fundamental power/area to variation tradeoff is not acceptable 1 c2 σvt 2 Weff Leff Need circuit architectures that fundamentally change this tradeoff. Ref: K. Kuhn, IEDM

20 Mitigating Device Variation Calibration greatly improves the power/variation tradeoff Receiver offset calibration Duty cycle correction Adaptive equalizers Clock recovery (or deskew) Simple calibration doesn t alleviate all variation issues (e.g. PSRR) + -Voffset Circuit derivatives (gm, ro) are not calibrated by offset calibration PSRR is not calibrated 20 20

21 INL (ps) DNL (ps) Mitigating Device Variation Calibration greatly improves the power/variation tradeoff Receiver offset calibration Duty cycle correction Adaptive equalizers Clock recovery (or deskew) Simple calibration doesn t alleviate all variation issues (e.g. PSRR) Possible solutions: Dynamic calibration (e.g. auto-zero) Redundancy/reconfigurability Better correct by design circuits Coarse Phase gen. Φ 1 Φ N selφ DSM Ref: P. Hanumolu, JSSC 2/08. Φ LPF Φ OUT 21 21

22 Outline 1TByte/s I/O: motivation and challenges Circuit Directions Channel Directions Tool Directions 470Gb/s Prototype 22 22

23 Channel scaling Circuit innovation alone will probably not be enough to reach the 1TB/s target the channel needs to scale too! Better signal integrity: Improved electrical characteristics mean less power in clocking and equalization Higher density: More lanes allow each lane to operate at lower data rate better power efficiency 23 23

24 Channel vs. Equalization tradeoffs: Backplane example 0dB -20dB S 21-40dB -60dB 5mm Stubbedvia BP Drilledvia BP -80dB 0GHz 5GHz 10GHz 15GHz Ref: B. Casper, CICC

25 Improve channel signal integrity 0dB 19 Flex -20dB S 21-40dB -60dB 5mm Stubbedvia BP Drilledvia BP -80dB 0GHz 5GHz 10GHz 15GHz CPU Socket Flex Connector 19 Flex Cable 25 25

26 Approx. Contact/Routing Pitch (µm) High density channels 1000 Contact pitch Routing pitch

27 Approx. Contact/Routing Pitch (µm) High density channels 1000 Contact pitch Routing pitch

28 Approx. Contact/Routing Pitch (µm) High density channels 1000 Contact pitch Routing pitch

29 Approx. Contact/Routing Pitch (µm) High density channels 1000 Contact pitch Routing pitch

30 Approx. Contact/Routing Pitch (µm) High density channels 1000 Contact pitch Routing pitch

31 Outline 1TByte/s I/O: motivation and challenges Circuit Directions Channel Directions Tool Directions 470Gb/s Prototype 31 31

32 What is the Right Link Architecture? TX RX Clock Jitter? Signal Swing? Equalization? ISI? Xtalk? Modulation (PAM)? Data Rate? Interface width? Clock Jitter? Sensitivity? Equalization? Designers need the ability to quickly and accurately compare architecture options 32 32

33 Empirical Approach Simulate system with random data This doesn t provide adequate accuracy (BER<10-12 ) 33 33

34 Full System Statistical Analysis Channel & co-channel responses RX sampling jitter TX jitter Statistical Signaling Analysis Equalization Modulation RX input referred noise Specify high-level architecture and block characteristics Enables fast evaluation of link sensitivities 34 34

35 Max. Data Rate (BER=10-12 ) Maximum Data Rate Comparison: Backplane vs. Flex 45Gb/s Flex 30Gb/s 15Gb/s Drilled-via BP Stubbed-via BP TX FIR taps DFE taps Statistical system analysis provides designers with real performance tradeoffs and brick walls 35 35

36 Max. Data Rate (BER=10-12 ) Maximum Data Rate Comparison: Backplane vs. Flex 45Gb/s Flex 30Gb/s 15Gb/s Drilled-via BP Stubbed-via BP TX FIR taps DFE taps Statistical system analysis provides designers with real performance tradeoffs and brick walls 36 36

37 Outline 1TByte/s I/O: motivation and challenges Circuit Directions Channel Directions Tool Directions 470Gb/s Prototype 37 37

38 47x10Gb/s, 1.4pJ/bit Interface (45nm CMOS) IL-VCO diea dieb IL-VCO txbundle_a[1] txbundle_a[0] rxbundle_a[2] rxbundle_a[1] rxbundle_a[0] fclk_a 9 10 fclk_b rxbundle_b[1] rxbundle_b[0] txbundle_b[2] txbundle_b[1] txbundle_b[0] 38 38

39 Bundled Architecture Conventional: Independent clocking clk data Deskew Deskew Deskew Deskew Deskew RX sampler RX sampler RX sampler RX sampler RX sampler 39

40 Bundled Architecture Clocking innovation Bundle clocking Conventional: Independent clocking clk Deskew data RX sampler clk Deskew Deskew Deskew Deskew RX sampler RX sampler RX sampler RX sampler Bundle Deskew 40

41 Bundled Architecture Clocking innovation Bundle clocking Conventional: Independent clocking Optimized: Bundle clocking clk data clk data Deskew RX sampler RX sampler Deskew Deskew RX sampler RX sampler Bundle Deskew RX sampler RX sampler Deskew RX sampler RX sampler Deskew RX sampler RX sampler 41 Bundled clocking reduces I/O power

42 Probability of correct seeding Fast RX Power States 1 rxbundle_b[0] rxlane[9:0] Wake-up time [ns] RX bundle power reduced by 93% in standby All RX lanes return to reliable operation in <5ns 42

43 Silicon Area Compression Conventional: I/O floor plan Power Ground I/O layout I/O signals 43

44 Silicon Area Compression Floor plan optimization minimize I/O area Conventional: I/O floor plan Power Ground I/O layout I/O signals 44 I/O circuitry

45 Silicon Area Compression Floor plan optimization minimize I/O area Conventional: I/O floor plan Power Ground Optimized: Bundle layout I/O layout I/O signals 45 I/O circuitry

46 1302µm Lane[9:5] 10 TL pairs Interface Floorplan IL-VCO + Drv Active I/O circuitry Lane[4:0] txbundle_a[1] txbundle_a[0] rxbundle_a[2] rxbundle_a[1] rxbundle_a[0] 2864µm Die edge Active circuit area is reduced with TL routing. 46

47 Interface Configuration 500µm LGA connector diea HDI/Flex bridge 5 signals/mm dieb Microstrip Stripline1 Stripline2 Package Socket PCB Within-bundle lanes matched to <100µm Dense LGA connector minimizes breakout area Bundles share the same routing layer 2X density on stripline layers due to reduced Xtalk 47

48 Silicon and Interconnect Prototypes 0.5m flex interconnect 3m twinax cable 48

49 Electrical Interconnect Scaling Challenges 100 Power Eff. (pj/bit) 10 This work: 45nm CMOS data rate Channel Symbol rate (db) (Based on transceivers reported in nm CMOS) 49

50 Power Efficiency (pj/bit) I/O Power Efficiency Measurements Link data rate = 10Gb/s Channel Length (cm) HDI * LCP flex 32AWG - twinax *high density interconnect (HDI) 50

51 Summary Bandwidth needs are quickly approaching 1TB/s Extending electrical I/O to 1TB/s requires balance between power, data rate, density and cost Evaluate alternate channel configurations and materials Recent results indicate that electrical will be up to the task for in-box I/O 51 51