Monolithic Integration of Energy-efficient CMOS Silicon Photonic Interconnects

Transcription

1 Monolithic Integration of Energy-efficient CMOS Silicon Photonic Interconnects Vladimir Stojanović Integrated Systems Group Massachusetts Institute of Technology

2 Manycore SOC roadmap fuels bandwidth demand 64-tile system ( cores) - 4-way SIMD GHz TFlops on one chip - Need 5-10 TB/s of off-chip I/O - Even higher on-chip bandwidth 2 cm 2 cm Intel 48 core -Xeon 2

3 DRAM DIMM DRAM DIMM DRAM DIMM System Bottlenecks Manycore system cores CPU CPU CPU Cache/ MC Cache/ MC Interconnect Network Cache/ MC Bottlenecks due to energy and bandwidth density limitations Interconnect Network 3

4 Energy-cost [pj/b] Wire and I/O scaling On-chip wires On-chip wires copper resistivity Best electrical links I/O Loss ~20-25dB Chip2Chip Backplane Loss ~10dB Data-rate [Gb/s] Increased wire resistivity makes wire caps scale very slowly Can t get both energy-efficiency and high-data rate in I/O 4

5 Package pin count Bandwidth, pin count and power scaling 256 cores *> half pins for power supply Need 16k pins in 2017 for HPC* 2 TFlop/s signal 20 Gb/s/link 2,4 cores 1 Byte/Flop 5

6 Monolithic CMOS-Photonics in Computer Systems Supercomputers Si-photonics in advanced CMOS and DRAM process NO costly process changes Embedded apps 6

7 Many architectural studies show promise [Shacham 07] [Petracca 08] [Joshi 09] [Pan 09] [Vantrease 08] [Psota 07] [Kirman 06] [Koka 08-10] [Batten 08] 7 [Kurian 10]

8 Optimization requires full system insight DSENT Electrical and optical link and network models Cache & Core Energy & Area Developed cross-layer modeling framework 8 Kurian, Chen 2011

9 Register Mux Register Mux Demux Register Demux Register Start at the link level: Jointly optimize circuits and photonic devices PLL or Opt. Clk Pre-Driver in PLL or Opt. Clk in Mod-Driver Pre-Driver Mod-Driver Phase Adjust Receiver Front-end PLL or Phase Opt. Clk Adjust PLL or Opt. Clk Samplers & Receiver Samplers & Monitoring Front-end Monitoring + + Dense WDM 128 wavelengths/waveguide - >1Tb/s per waveguide Need 1000 s of transceivers on die with < 100fJ/bit cost at > 10Gb/s! - Optimized modulator circuits/devices - Optimized receiver circuits/photo-detector - Optimized thermal tuning 9

10 Need to optimize carefully Register Mux Register Mux Demux Register Demux Register 512 Gb/s aggregate throughput PLL or Opt. Clk in PLL or Opt. Clk Pre-Driver Mod-Driver Pre-Driver Mod-Driver in Phase Adjust PLL or Phase Opt. Clk Adjust PLL or Opt. Clk Receiver Samplers & Receiver Samplers & Front-end Monitoring Front-end Monitoring + + Laser energy increases with data-rate Limited Rx sensitivity assuming 32nm CMOS Modulation more expensive -> extinction ratio / insertion loss trade-off Tuning costs decrease with data-rate Moderate data rates most energy-efficient 10 Georgas CICC 2011

11 DWDM link efficiency optimization >10x Electrical bump-pitch limited to <1Tb/s/mm 2 Package pin limit 0.05 Tb/s/mm 2 Optimize for min energy-cost Bandwidth density dominated by circuit and photonics area (not coupler pitch) 10x better than electrical bump limited 200x better than electrical package pin limit 11

12 Mem Scheduler Photonic DRAM Network Organization Laser in CPU Super DIMM DRAM cube 1 Important Concepts MC 1 DRAM cube 4 - Power/message switching (only to active DRAM chip in DRAM cube/super DIMM) MC K Dwr Drd cmd Drd Dwr ( cube 1, die 8) - Vertical die-to-die coupling (minimizes cabling - 8 dies per DRAM cube) die-die switch cmd Dwr Drd ( cube 1, die 1) -Command distributed electrically (broadcast) - Data photonic (single writer multiple readers) Super DIMM K DRAM cube 4 MC 16 Processor die Modulator bank Receiver/PD bank Tunable filterbank Through silicon via Through silicon via hole 12 Beamer ISCA 2010

13 Optimizing DRAM with photonics P1 P4 Floorplan Beamer ISCA

14 Laser Power Guiding Effectiveness Enables capacity scaling per channel and significant savings in laser energy 14 Beamer ISCA 2010

15 ATAC On-Chip network Example 1000 core die 64 clusters connected via optical broadcast 15

16 Average Energy over Splash2 benchmarks Ring tuning very expensive Non-gated laser very expensive 16

17 Including the cores gives the full picture Energy dominated by cores/caches Faster network saves overall energy (leakage and clock) Need aggressive clock-gating and supply/retention scaling

18 Execution time also matters 18

19 Feedback to device designers Waveguide losses up to 2dB/cm o.k. 19

20 Conclusions Biggest gains if photonics both on-chip and off-chip Core-to-MC network MC-to-DRAM bank network immediate 10x gains Need comprehensive modeling framework to see the full picture Link-level tight interaction of circuits and photonics through good models System-level Include all system components cores, network, caches, memory

21 Acknowledgments Krste Asanović, Rajeev Ram, Miloš Popović, Christopher Batten, Ajay Joshi Anant Agarwal, Li-Shiuan Peh, Lionel Kimerling, Jurgen Michel, Dimitri Antoniadis Jason Miller, Jeff Shainline Jason Orcutt, Chen Sun, Ben Moss, Jonathan Leu, Michael Georgas, Stevan Urosević, Owen Chen, George Kurian, Yong-Jin Kwon, Scott Beamer Dr. Jag Shah and Dr. Charles Holland, DARPA FCRP IFC, NSF Trusted Foundry, Intel Corporation, APIC