Intra Optical Data Center Interconnection: Session 1: Component & Module Focus

Transcription

1 Intra Optical Data Center Interconnection: Session 1: Component & Module Focus Co-Organizer/Presider/Session Chair: Dr. Ioannis Tomkos Networks and Optical Communications group NOC

2 Session 1 Speakers Martin Zirngibl, Vice President and Technical Fellow, Finisar Before joining Finisar recently, Martin headed the IP routing research organization at Alcatel- Lucent s Bell Labs, which he joined in 1990 as a researcher (initially performing pioneering work in fiber amplifiers and photonic integrated circuits). Since 1998, Martin has held various managerial positions at Bell Labs and drove early research and development in 100G that led to a significant industry disruption in WDM long-haul. Martin holds a PhD in applied physics from ETH Lausanne, Switzerland. Brad Booth, Principal Engineer, Microsoft Brad Booth is heading up the 25/50G Ethernet Consortium and the Consortium for On-Board Optics (COBO). At Microsoft, he leads the development of hyperscale interconnect strategy for Microsoft s cloud datacenters. He is also the founder and past Chairman of the Ethernet Alliance. Brad was previously a Distinguished Engineer in the Office of the CTO at Dell Networking and he has also held senior strategist and engineering positions at Applied Micro, Intel, and PMC-Sierra. Toshiki Tanaka, Research Manager, Fujitsu Laboratories Ltd 2 Toshiki joined Fujitsu in 1997 and has been engaged since then in research and development of dense WDM optical transmission systems and high capacity optical transceivers with higher order modulation. Since 2016, he has engaged in developing 100G and beyond 100G optical transceivers for short reach application at Fujitsu Optical Components Limited.

3 History behind ODCI event The optical networking scientific community recognized early the huge potential that optical technologies have to solve the scaling problems in data centre networks. At OFC 2011 the Datacom, Computercom and Short Range and Experimental Optical Networks subcommittee (now track DSN6 in the OFC 2016 technical programme) was formed Since that time, the topic of optical communications for data centres has become mainstream and many other major conferences have adapted their coverage accordingly. Before the NGON 2015 event, there were a few sporadic talks about ODCI included in the NGON conference program (mostly focusing on inter-odci) In NGON 2015, we introduced for the first time a special workshop focusing on ODCI, with emphasis on intra-odci technologies/solutions Due to the success on the 2015 workshop and the tremendous market potential of ODCI solutions, IIR decided to expand the NGON conference by including a parallel track on ODCI 3

4 Feature issue at Fibre Systems Magazine (Spring 2016): DATA CENTRE OPTICS Super-sizing the data centre Warehouse-scale data centres are stretching the limits of current networking technologies The article described how advanced optics can remove performance bottlenecks to enable the data centres of the future This is what we will discuss further today! 4

5 Outline Session 1: Component & Module Focus Architectures and characteristics of intra-dc networks Transceiver solutions for optical interconnects in intra DC networks Session 2: Debating Intra-DC solutions and Photonic Integration approaches Photonic integration approaches for intra-dc networks: InP vs. SiP Photonic integration approaches for intra-dc networks: Monolithic vs. hybrid 5

6 What is a (conventional) Data Center A data center is a facility used to house computer systems (aka servers) and associated components, such storage systems, that are networked together The equipment may be used to store and process the (big) data providing relevant applications directly to the DC s operator customers 6

7 Intra-Data Center Interconnection Networks (intra-dcns) Data Center Content switches & Load balance Internet The Data Center Networks are usually based on a fat-tree topology using commodity switches ToR switches Servers Core switches Aggregate Switches Rack 10 Gbps 10 Gbps Rack Rack Rack Access Layer: Top-of-Rack (ToR) switches are used to connects the servers in a rack (e.g. 1Gbps) Aggregate Layer: Used to interconnect the ToR switches (e.g. 10Gbps) Core Layer: Used to interconnect the access switches (e.g. 100Gbps) 7

8 Need for Big (Warehouse-scale) Data Centers Computing and Storage are moving from PC-like clients to large Internet service providers that use the cloud There is the need for more powerful and efficient warehouse data centers to host the cloud computing applications 8

9 How Big are these Mega Data Centers? 9 Wembley Stadium:172,000 square ft

10 Traffic Characteristics of Data Centers (Source: Cisco global cloud index: ) Global DC and cloud IP traffic growth DC traffic destinations IP traffic over data center networks has reached almost 5 Zettabytes in 2015 Cloud DC traffic is growing 33% CAGR Most traffic remains within the DC BW>73% of total network traffic 10

11 Data Centers Power Consumption Data centers consumed 330 Billion KWh in 2007 and is expected to reach 1012 Billion KWh in 2020 [Source: How Clean is Your Data Center, Greenpeace,

12 Future Data Center Needs Current Data Center Networks: What the business need (Terabit Data Centers): [Source: Scaling Networks in Large Data Centers, Facebook ] Current data center networks cannot affordably satisfy the bandwidth requirements of upcoming applications Bandwidth intensive applications Data Centers based on low cost commodity hardware 12

13 Facebook s fabric based data center For building-wide connectivity, fb created four independent planes of spine switches, each scalable up to 48 independent devices within a plane. Each fabric switch of each pod connects to each spine switch within its local plane. Together, pods and planes form a modular network topology. To connect all these switches an enormous number of (low-cost & low-power) transceivers are required! 13

14 Data Growing Faster than Technology Typical silicon devices (electronics) cannot follow the growth of the data => Optical interconnects based on new paradigms are needed! 14

15 Energy and cost targets for interconnects DISTANCE TARGET ENERGY TARGET COST TARGET Inter DC Rack to Rack Board to Board Module to Module Chip to Chip Core to Core on chip few km up to 1-300m 0.1-1m 5-30cm 1-5cm <2cm multi km <10PJ/b <1PJ/b <1PJ/b <0.5PJ/b <0.1PJ/b 0.01PJ/b <$ ~$100 ~$10 ~$5 ~$1 ~$0.01 Electronic interconnects suffer from many limitations, while Photonic based interconnects show much better promise 15

16 Optical interconnect links Optical Links can provide Tbps Data rates But still we are going to need E/O, O/E, transceivers and high performance switch fabrics 16 [Source: Intel, The 50Gbps Si Photonic Link]

17 Options for realizing different 40 Tb/s interconnect scenarios - I The need to handle the increasing volumes of data traffic drives the need for everincreasing communication port densities. This increased density leads to smaller surface areas available to dissipate the heat generated and therefore requires decreased power consumption per port. 17 As an example, in the Table, we present the required data rate per fibre and the number of links for different 40 Tb/s interconnect scenarios. OIF Next Generation Interconnect Framework, April 2013

18 Options for realizing different 40 Tb/s interconnect scenarios - II Given that a bundle of a hundred or less optical fibres could be considered acceptable for an optical conduit, it is easy to understand that in order to achieve 40Tb/s interconnect capacity, a combination of parallel optics, higher baud rates, increased bits per symbol, WDM or/and polarization multiplexing, space-division multiplexing using MCFs, etc. should/could be implemented. The use of parallel optics, advanced modulation formats and WDM are regarded as likely choices to increase the interconnect bit rate and density. MCFs could be used to increase port densities by a significant factor and could be an alternative to VCSEL/MMF technology. However, at some point the bandwidth limit either in the EO/OE or on the system host chip is reached, where the only other option to further increase the bandwidth is with optical switching. 18

19 Future Data Center Networks will require high performance optical switches & WDM Terabit Optical Switch WDM Links Sources: Cisco, Petabit Optical Switch for Data Center Networks, Facebook, Scaling Networks in Large Data Centers Future Data Center Networks We need high-throughput, scalable, energy efficient Data Centers that can sustain the exponential increase of the network traffic 19 [Source: Kachris, Bergman, Tomkos, Optical Interconnects for future Data Center Networks, Elsevier publication]

20 Roadmap of Ethernet speeds Ethernet roadmap, where the year in which the standard was/will be completed is shown 20

21 Relative size and power consumption of optical interfaces today and tomorrow 21

22 Advanced transceiver concepts for intra-odci: possible mod formats? Some possible potential advanced modulation formats that could be utilized up to 400 GbE for intra DCI connectivity are: NRZ OOK Duobinary modulation Pulse amplitude modulation with 4 levels (PAM-4) Discrete multi-tone (DMT) modulation (OFDM) Others? 22

23 PAM-4 based transceiver The 4-levels PAM signal is generated by a PAM-4 encoder and is fed into the digital to analog converter (DAC). Then the signal is electrically amplified by a linear driver (DRV). As a final step, at the transmitter s site, the output of the driver is modulated using an electro-absorption modulated laser (EML) or a directly modulated laser (DML). At the receiver s side, the signal is directly detected with a simple PIN diode and then it is amplified by an electrical amplifier. Finally, the received electrical signal may be processed by digital signal processing (DSP). 23

24 PAM versus DMT 24 Power consumption assuming 28nm CMOS technology: PAM4: 1.5W DMT: 3.5W

25 Issues to be discussed/debated 25 Material system: InP vs. SiP vs.??? Integration approaches: monolithic vs. hybrid? Packaging approaches? Wavelength of operation: 850nm vs. 1310nm vs. 1550nm? Laser type: VCSELs vs. DFBs vs.??? Direct vs. external modulation? The road to 400G and then to 800G/1600G? Modulation formats: PAM vs. DMT vs. QAM? Direct vs. coherent detection? Extend of use of DSP? Optical switching in the DC? Others???

26 Session 1 Speakers Martin Zirngibl, Vice President and Technical Fellow, Finisar Before joining Finisar recently, Martin headed the IP routing research organization at Alcatel- Lucent s Bell Labs, which he joined in 1990 as a researcher (initially performing pioneering work in fiber amplifiers and photonic integrated circuits). Since 1998, Martin has held various managerial positions at Bell Labs and drove early research and development in 100G that led to a significant industry disruption in WDM long-haul. Martin holds a PhD in applied physics from ETH Lausanne, Switzerland. Brad Booth, Principal Engineer, Microsoft Brad Booth is heading up the 25/50G Ethernet Consortium and the Consortium for On-Board Optics (COBO). At Microsoft, he leads the development of hyperscale interconnect strategy for Microsoft s cloud datacenters. He is also the founder and past Chairman of the Ethernet Alliance. Brad was previously a Distinguished Engineer in the Office of the CTO at Dell Networking and he has also held senior strategist and engineering positions at Applied Micro, Intel, and PMC-Sierra. 26 Toshiki Tanaka, Research Manager, Fujitsu Laboratories Ltd Toshiki joined Fujitsu in 1997 and has been engaged since then in research and development of dense WDM optical transmission systems and high capacity optical transceivers with higher order modulation. Since 2016, he has engaged in developing 100G and beyond 100G optical transceivers for short reach application at Fujitsu Optical Components Limited.

27 Intra Optical Data Center Interconnection Session 2: Debating Intra-DC solutions and Photonic Integration approaches Co-Organizer/Presider/Session Chair: Dr. Ioannis Tomkos Networks and Optical Communications group NOC

28 Session 2 Speakers & Panelists - I Chris Pfistner, Vice President, Datacom Product Line Management, Lumentum Chris joined the company in October 2015, bringing over 20 years of experience in Marketing, Sales, and Product Line Management in the global fiber optic module and systems market. Prior to Lumentum, he managed Finisar s product management team for optical transceivers and earlier he built the transceiver business at NeoPhotonics. He was also a co-founder of Terawave, and held marketing and product management positions at AFC and Pirelli. Chris holds Ph.D. and MS. degrees in Applied Physics from the University of Berne, in Switzerland. Brad Booth, Principal Engineer, Microsoft Brad Booth is heading up the 25/50G Ethernet Consortium and the Consortium for On- Board Optics (COBO). At Microsoft, he leads the development of hyperscale interconnect strategy for Microsoft s cloud datacenters. He is also the founder and past Chairman of the Ethernet Alliance. Brad was previously a Distinguished Engineer in the Office of the CTO at Dell Networking and he has also held senior strategist and engineering positions at Applied Micro, Intel, and PMC-Sierra. 28

29 Session 2 Speakers & Panelists - II James Regan, CEO, Effect Photonics James has over 30 years of experience in the photonic component business, in product development, marketing, sales and general management in building successful businesses within large companies (Nortel, JDSU) and start-ups (Agility Communications). Silvio Abrate, Head of Applied Photonics, ISMB Silvio Abrate is head of the Applied Photonics research group at ISMB and manager of the PhotonLab research facility, held in cooperation with Politecnico di Torino. Mauro Macchi, Director SP EMEAR, Cisco Mauro s 20+ years career in telecom industry includes Engineering and Product Management roles in Pirelli, Cisco and Juniper Networks. He is currently leading EMEAR Business Overlay team for IP, Optical and Data Center technologies. 29

30 Why Photonic Integration? Why integration? Look back at the electronics! Pictures taken at: Whirlwind, MIT, 1952 EAI 580 patch panel, Electronic Associates, 1968 Today s state of computing is based on: Integration and scaling of the logic functions (CMOS electronics) Integration and scaling of the interconnects (PCB technology & assembly) 30 For optical interconnects, this resembles: Electro-optical integration and scaling of transceiver technology Integration of optical connectivity and signal distribution

31 What is possible with InP today? Fully integrated monolithic 8-channel OS- and AO-OFDM Tx InP PIC fabricated! EU project ASTRON Tx PIC presentation at ECOC 2016! ASTRON designed, fabricated and characterized fully integrated monolithic 8-channel OS-/AO- OFDM InP transmitter PIC, for the first time, integrating 8 IQ modulators and the other passive Tx building blocks (8- port AWG, 1x8 splitter/combiner) on a single PIC 31 OS-OFDM Tx PIC AO-OFDM Tx PIC

32 Comparison of InP and SiPh technologies The development of the SiPh technology has helped to drive large-scale manufacturing of PICs at low costs, since they can leverage highly developed fabrication processes from the microelectronics industry. However, some analysts claim that InP platforms can, depending on yields, have production costs equal to or lower than SiPh, for the production volumes expected for telecom and DC applications. The table summarizes the pros and cons of InP and silicon photonics for PIC manufacturing Silicon photonics also has also the added advantage compared to InP that it can be integrated with electronic Ics, using 2.5D and 3D packaging, thus saving cost, footprint, and power. 32

33 Factors affecting PIC costs The PIC market is growing at a phenomenal rate as it provides significant improvements in system size, power consumption, reliability and cost. Many factors can affect the projected costs of a new technology, among which are: the scale of production (e.g. annual production volume), the manufacturing location (e.g. the difference between producing in the USA and East Asia), the cost and size of wafers, the maturity of the manufacturing process, and most importantly the production yields achievable for each technology 33

34 Relative cost per PIC The simple relation between achievable number of PICs per wafer and PIC size: Current InP SotA It is obvious that the decreasing number of available PICs per wafer, with increasing PIC size, is accompanied by increasing costs. The relative cost with decreasing number of PICs per wafer, for different substrate sizes and while assuming 100%(!) on-wafer yield is also shown. The cost calculation refers to a single MZ modulator fabricated on a 3-inch InP wafer as a reference (i.e. on-wafer cost of a single MZM = 1). This cost will be reduced by a factor of 2 if a 4-inch InP substrate is used instead of a 3-inch one, and even more with larger substrate sizes. 34 SiP can support larger wafers compared to InP and therefore, just for that, can result in lower cost PICs

35 Evolution of chip complexity for InPand SiPh- based ICs In the figure we can observe, the evolution of chip complexity for InPbased IC (blue) and SiPh-based IC without laser (red) and with heterogeneously integrated lasers (green) The problem of laser integration on silicon stems from the fact that silicon has an indirect bandgap and hence is a very inefficient light emitter. A possible energy-efficient and costeffective solution is wafer bonding of III V materials that can be wafer bonded to the silicon photonic chip to co-fabricate lasers that are lithographically aligned to the silicon waveguide circuit. 35

36 Integration approaches Photonic integration strategies can be divided into three main categories, each of them having its own pros and cons. In hybrid integration, multiple single-function devices are assembled into a single package, sometimes with associated ICs, and inter-connected to each other by electronic and/or optical couplings internal to the package. Several problems may arise from the use of this integration technology: alignment tolerances of 1-2microns are sometimes necessary; different materials for different components may have different optical, mechanical and thermal characteristics, etc. In semi-hybrid integration, specialized regions are grown in appropriate materials over a common substrate (normally silicon). Finally, in monolithic integration, devices are built into a common substrate, providing significant packaging consolidation, testing simplification, reduction in fibre couplings, improved reliability and maximum possible reduction in space and power consumption per device. In principle significant savings can be expected when moving from hybrid integration to monolithic integration, provided that the process yield can be maintained high. 36

37 Integration Roadmap for SiP Cost-advantages through further integration Cost of optics determines the application scope Silicon photonics versatile and cost-efficient Si: indirect bandgap additional material for laser sources required Today: external laser source must be coupled expensive and high losses Already demonstrated: hybrid III-V on silicon integration (on-chip) DIMENSION breakthrough: processing of III-V devices at silicon wafer level Monolithic Hybrid Semi-Hybrid State-of-the-art demonstrated 37 III-V on silicon integration

38 Directly Modulated Lasers on Silicon

39 DIMENSION approach Combining BiCMOS electronics, photonics and III-V on a new technology platform for monolithic electro-optical integration Integrated devices, with CMOS, photonic and III-V functionality at the cost of silicon volume fabrication Si CMOS wafer at front-end level including silicon photonics Concept IP protected Bonding a III-V photonic membrane onto the first dielectric oxide (ILD1) 39 Metal interconnection of the III-V with the CMOS underneath Structuring of the III-V active photonics

40 Concept and Main Objectives Technology platform for monolithic integration of BiCMOS electronics with Si- and III-V photonics: Bonding or growth of ultra-thin (<500 nm) III-V quantum well stack on the FEOL (Bi)CMOS Cost-effective embedding of high-quality III-V structures in (Bi)CMOS BEOL Efficient optical coupling between silicon and III-V layer based on adiabatic mode conversion with high modal overlap for low power consumption and high-speed modulation Laser feedback and passive optical structures in silicon layer DIMENSION implementation of III-V materials in between the front-end-of-line and the back-end-of-line of a BiCMOS process 40

41 Issues to be discussed/debated 41 Material system: InP vs. SiP vs.??? Integration approaches: monolithic vs. hybrid? Packaging approaches? Wavelength of operation: 850nm vs. 1310nm vs. 1550nm? Laser type: VCSELs vs. DFBs vs.??? Direct vs. external modulation? The road to 400G and then to 800G/1600G? Modulation formats: PAM vs. DMT vs. QAM? Direct vs. coherent detection? Extend of use of DSP? Optical switching in the DC? Others???

42 Session 2 Speakers & Panelists - I Chris Pfistner, Vice President, Datacom Product Line Management, Lumentum Chris joined the company in October 2015, bringing over 20 years of experience in Marketing, Sales, and Product Line Management in the global fiber optic module and systems market. Prior to Lumentum, he managed Finisar s product management team for optical transceivers and earlier he built the transceiver business at NeoPhotonics. He was also a co-founder of Terawave, and held marketing and product management positions at AFC and Pirelli. Chris holds Ph.D. and MS. degrees in Applied Physics from the University of Berne, in Switzerland. Brad Booth, Principal Engineer, Microsoft Brad Booth is heading up the 25/50G Ethernet Consortium and the Consortium for On- Board Optics (COBO). At Microsoft, he leads the development of hyperscale interconnect strategy for Microsoft s cloud datacenters. He is also the founder and past Chairman of the Ethernet Alliance. Brad was previously a Distinguished Engineer in the Office of the CTO at Dell Networking and he has also held senior strategist and engineering positions at Applied Micro, Intel, and PMC-Sierra. 42

43 Session 2 Speakers & Panelists - II James Regan, CEO, Effect Photonics James has over 30 years of experience in the photonic component business, in product development, marketing, sales and general management in building successful businesses within large companies (Nortel, JDSU) and start-ups (Agility Communications). Silvio Abrate, Head of Applied Photonics, ISMB Silvio Abrate is head of the Applied Photonics research group at ISMB and manager of the PhotonLab research facility, held in cooperation with Politecnico di Torino. Mauro Macchi, Director SP EMEAR, Cisco Mauro s 20+ years career in telecom industry includes Engineering and Product Management roles in Pirelli, Cisco and Juniper Networks. He is currently leading EMEAR Business Overlay team for IP, Optical and Data Center technologies. 43