Doug Coleman Manager, Technology & Standards Distinguished Associate Corning Optical Communications

Transcription

1 Optical Trends in the Data Center Doug Coleman Manager, Technology & Standards Distinguished Associate Corning Optical Communications

2 Higher speeds Higher density Higher reliability Lower capex Lower opex Green Data Center Environment

3 Data Centers Going Through Data Rate Increase YESTERDAY and TODAY TODAY and TOMORROW Nx10G Core switch 10 GbE non-blocking Nx40/100G Core switch 100 GbE non-blocking Metro/Campus 10G connectivity LR/ER Metro/Campus 100 G connectivity LR/ER WAN connectivity through 40 G WAN connectivity through 100 G DWDM 10G TOR switch 40/100G TOR switch 48 G Rack Capacity 480 G Rack Capacity 2-4 X10 GbE uplinks 2-4 X 40/100 GbE uplinks SFP+ MMF or SMF QSFP/CFP/CFP2 (MMF/SMF) 1G Servers to TOR switch 48 x 1 GbE Copper CAT5, 100 m 10G Servers to TOR switch 48 x 10 GbE Copper or Fiber

4 VCSELs Drives MMF Value Proposition 850 nm Vertical Cavity Surface Emitting Laser (VCSEL) Highly Efficient Manufacturing Process (>100,000 / wafer) Ease of Packaging into Transceiver TOSA Lowest Transceiver Price 10G Serial x Serial 2:1 ($SM/$MM) 40G Serial x Parallel 5:1 ($SM/$MM) 100G Serial x Parallel l 25:1 ($SM/$MM) IEEE 802.3ba 10:1 ($SM/$MM) IEEE 802.3bm SFP+ LD TOSA ROSA

5 Data Center Environment: Enterprise Source: Corning Optical Communications

6 Standard Specified Distances 850 nm Ethernet Distance (m) 10G 40G 100G** 100G*** OM OM4 400 / 550* * Engineered Length ** 802.3ba10G/lane *** 802.3bm 25G/lane 850 nm Fibre Channel Distance (m) 4G 8G 16G 32G OM OM

7 Migration to OM4 Fibre Channel (32G) and Ethernet (100G) utilized OM4 to define distance objectives Transmission and Cable Standards recommend OM4

8 Do We Need OM5? Typical Dispersion Curve Emerging high data rate 16G are limited due to excessive chromatic dispersion penalty associated with 850mm transceiver line widths

9 Do We Need OM5? Legacy 850nm transceiver technology has minimal value with increased EMB Source: VI Systems Chromatic Dispersion Limited Effective Modal Bandwidth (EMB) OM3 >2.0 GHz-km OM4 >4.7 GHz-km

10 Do We Need OM5 Fiber? New multimode fiber At ECOC Workshop 4, "Technologies for Short-Reach Optical Interconnect, " Avago suggested the development of a new multimode fiber to improve distance. Target wavelength is in the 920 1,000nm range. New fiber would be designed to support longer distance through waveguide design that minimizes the chromatic dispersion impairment.

11 Data Center Trends: Electronics & Connectivity Optical Electronics and Connectivity Focus Low Cost Low Power High Density Increase data rates, transceiver size reduction, server consolidation/virtualization, multi-core processors, line card density, embedded optics, cloud computing, increased interconnect density

12 Evolution of VSR Computing Optical Interconnects Source: Finisar 2012

13 Market Trends Toward 100GE Embedded optics adoption in data center Reduction of signal trace length at higher data rates requires close proximity of optics with ASIC Enabling 5x 10x denser I/O than edge mounted pluggable optics Source: Avago

14 Market Trends Toward 100GE Source: Avago Embedded optics adoption in data center Reduction of signal trace length at higher data rates requires close proximity of optics with ASIC Optical connector Ribbon Fiber Optical opening Electrical connector

15 MMF Transceivers Trend SFP+ Source: Avago 850nm MM VCSEL Pluggable Transceivers Continue dominating the DC demand from 10GE to 40GE and to 100GE due to power, density, and cost 850nm MM VCSEL Embedded Optics Getting more adoption in DC and HPC especially when moving to 25G per lane QSFP+ 12x10G Embedded Optics

16 Receiver / Photo-Detector Single-Mode Silicon Photonics Single Chip 4x28Gbps Optical Transceiver RXs TXs Digital core Source: Luxtera Transmitter / Modulator LAMP Packaged Light Source 8 Fiber array I/Os 4 for transmit and 4 for receive 5mm x 6mm die size Single CW Laser Split 4 ways

17 Evolution of VSR Computing Optical Interconnects Source: Luxtera

18 Will VCSEL Products Be Displaced by Si Photonics? Multimode transceivers and board-mountable-optical assemblies (BOAs) are cheaper to manufacture due to high VCSEL yield, lower cost optics and high tolerance MM alignment Volume for short reach MM < 100 m in data centers is over 80% of total and will continue to grow as data centers grow, and as server-to-tor, and server-to-server interconnects evolve from electrical to optical VCSELs have fundamentally lower power consumption (pj/bit) for short reach MM interconnects, allowing higher density VCSELs are smaller in size relative to InP laser + Si MZ configurations allowing higher bandwidth density Source: Finisar

19 The Need for Speed: 10/40/100G

20 Server Virtualization Drives Higher Data Rates Multiple applications running in parallel on one server. (eg., apps/server) servers consolidated Increases utilization efficiency to =< 90% Multi-core processors (4,8,16..50) PCIe2: 8 5G 8b/10b PCIe3: 16 8G 128/130b Increased Memory Drive utilization of high BW optical connectivity to mitigate system bottlenecks and support required increased I/O speeds.

21 Number of Servers per Rack Increasing 48 - Servers / Rack 60+ Servers / Rack

22 1310nm Silicon Photonics on 50um MMF Source: Intel 1310nm optimized 50um MMF supports 300m at 25Gb/s High BW SiP links server CPUs to storage units within the rack Trading Monolith Servers for disaggregation Allows servers to be replaced easily and independently. Optical interconnect boasts peak transfer rates of f16tb/ 1.6Tb/s 8-64F MXC Connector Lensed connector Enables 10x the immunity to dust

23 Legacy Data Center Collapsed Architecture: EDGE, Aggregation, g and Core Switch Consolidation into MDA

24 Data Center Flat Architecture: Top of Rack EDGE Switch Source: FCI Source: Blade Networks Technologies 4 QSFP+ 40G ports 48 SFP+ ports 10G or 1G 48 server connections

25 Top of Rack EDGE Switch/Server Interconnect: 10G SFP+ Active Optical Cables Direct Attached SFP+ Transceiver Distribution to Edge Interconnect Edge to Server Interconnect Performance Attributes: Low Cost Optical components selected to designed distance Reduced manufacturing and testing costs Small diameter, Low Weight & Flexible Distance Capability (=>10m) No cleaning connector concerns No connector insertion loss concerns No mismatch transceiver concerns SFP+ AOC Source: Cisco

26 Top of Rack EDGE Switch/Server Interconnect Source: Cis sco Twinax Harness Sou urce: INF-8438i 16 QSFP+ 40G ports Each QSFP+ port can handle four 1/10G server connections 64 x 1/10G server connections MTP to LC Harness

27 QSFP+ MM Transceivers QSFP+ QSFP+ Source: Avago QSFP+ SFP+ 3x increase in density at ½ power dissipation per 10G lane in comparison with SFP+

28 Data Center Trend: Flat Architecture Two Tier Hierarchy Architecture Core to Edge to Server Non-blocking Core to Edge Over-subscription Edge to Server Facilitates east-west layer 2 switch to server traffic flow Low network latency Ease VM movement Transparent Interconnection of Lots of Links (TRILL) management increases overall network bandwidth and availability Spanning Tree Replaced Sou rce: Fulcrum Mic crosystems Source: Mellanox

29 Data Center Trend: Flat Architecture DC Fabric Concepts: Leaf, Spine & Pod Source: Cisco

30 Spine and Leaf Architectures Cross-Connect Or Interconnect EDGE Fabric A Fabric B

31 Cross-Connect and Interconnect Design for Spine and Leaf Architectures Connectivity with Cross-Connect Structured Cabling Connectivity with Interconnect Structured Cabling

32 Data Center Trend: Network Monitoring Network layer data must first be extracted in order to apply the analysis tools SPAN (mirroring) Ports (active) Port TAP (passive for optical) What is monitoring looking for? Security threats Performance issues Optimization (I/O bottlenecks)

33 Data Center Trend: Optical Tap Network Security & Performance OPEX savings through improved operations Better tracking & adherence to SLAs enables move to cloud infrastructures

34 Configuration Options Configuration B Integrated MTP/LC Configuration C Integrated MTP/MTP Configuration A Non-Integrated LC/LC

35 Data Center Fabrics SAN HPC LAN

36 Ethernet Speedmap Name Speed Date Standard Ratified 10Mb/s Ethernet 10 Mb/s Mb/s Ethernet 100Mb/s 1995 Gigabit Ethernet 1Gb/s Gigabit Ethernet 10 Gb/s 2002 Sour ce: Ethernet Alli ance 40 Gigabit Ethernet 40 Gb/s Gigabit Ethernet 100 Gb/s 2010 / Gigabit Ethernet 400 Gb/s 2016

37 40GbE Roadmap PMD Name Electrical Reach Medium Date Interface Standard Ratified 40GBASE-CR4 XLAUI-4 7 m Twinax GBASE-SR4 XLAUI-4 100/150 m OM3/OM GBASE-LR4 XLAUI-4 10 km OS1/OS GBASE-FR XLAUI-4 2 km OS1/OS GBASE-ER4ER4 XLAUI-4 40 km OS1/OS GBASE-T??? 30 m CAT GBASE-SR* SR* XLAUI m OM GBASE-LR* XLAUI-1 2 km OS1/OS Sour ce: Ethernet Alli ance

38 40GeSR4 Extended Reach Parallel Optics 40GeSR4 QSFP Transceiver OM3/OM4: 300 m / 400 m (Standard Connectivity) 330 m / 550 m (Corning Connectivity) Corning 2012 modeling showed 12% data center lengths >100 m Internal testing demonstrated 1250m with random transceiver and OM4 fiber. Economic solution for OM3/OM4 distances beyond 802.3ba (100/150m) when compared to OS2 connectivity/electronics Commercially available now Source: Avago

39 40GBASE-SR-BD Duplex MMF Transmission Performance Duplex OM3/OM4 Connectivity/Polarity i i QSFP Transceiver Form Factor 100m/150m with OM3/OM4

40 OM3/OM4 Cabling for 1/10/40/100G

41 IEEE 802.3bm 40 and 100G Fiber Optic Cables Task Group Task Group will develop guidance in accordance with approved distance objectives Define a 40 Gb/s PHY for operation over at least 40 km of SMF Duplex fiber, single wavelength, expected Define a 100 Gb/s PHY for operation up to at least 500 m of SMF Options under consideration include: 4x25G parallel optics (8F), duplex fiber wave division multiplexing (WDM), and duplex fiber Pulse Amplitude Modulation (PAM) (Will not happen no consensus) Define a 100 Gb/s PHY for operation up to at least 100 m of OM4 4x25G parallel optics (8F) expected Define a 100 Gb/s PHY for operation up to at least 20 m of OM4 4x25G parallel optics (8F) expected (Will not happen based on minimal economical value) Expected Standard completion date: March 2015

42 100GbE Roadmap PMD Name Electrical Reach Medium Date Interface Standard Ratified 100GBASE-CR10 CAUI-10 7m Twinax GBASE-SR10 CAUI /150 m OM3/OM GBASE-LR4 CAUI km OS1/OS GBASE-ER4 CAUI km OS1/OS GBASE-CR4 4X25G 5 m Twinax GBASE-KR4 4X25G 1m Improved FR GBASE-KP4 4X25G 1m >Improved FR GBASE-SR4 CAUI-4 70/100 m OM3/OM GBASE-LR4 CAUI-4 10 km OS1/OS Sour ce: Ethernet Alli ance

43 Market Trends Toward 100GE 100GE (4x25G) Pluggable Form Factor 10x10G 4x25G or 10x10G 4x25G SM/MM CFP 10x10G SM/ MM 4x25G 4x25G or 10x10G CXP 10x10G10G MM 4x25G or 10x10G CFP2 SM/MM 4x25G 4x25G CFP4 SM/ QSFP28 MM Source: Avago Year

44 IEEE High Speed Ethernet Industry Connections Source: USConec Interim Meeting Geneva September 2012 Scope focused on building consensus related to the next speed of Ethernet for wireline applications, Straw polls showed strong support for 400G as next generation Ethernet t Study Group Adopted November 2013 Objectives: At least 100 m over MMF At least 500 m over SMF At least 2 km over SMF At least 10 km over SMF

45 Ethernet Speeds T 1.6TbE 16X100G TbE 10X100G Key: Ethernet Speeds Speed (b b/s) 400G 100G 40G 10G 1G 100GbE 10X10G 40GbE 4X10G 100GbE 4X25G CAUI-4 4x25.8G 400GbE 16X25G CDAUI-16 16x25.8G 400GbE 8X50G CDAUI-8 8X50G 400GbE 4X100G 100GbE 1X100G nx100g Ethernet Electrical Interfaces Hollow Symbols = predictions Source: Sco ott Kipp, Etherne et Alliance 4x G -XLAUI 10X G CAUI-10, CPPI

46 IEEE Working Group Standards Status July 15, 2013 SA = Subscribers Access Networks BPE = Backplane Ethernet IEEE Std Books (sections) Published 28-Dec-12 IEEE Std (Ethernet MIB) Approved 14-Jun-13 IEEE OM Approved Nov-97

47 Fibre Channel speedmap Product Naming Throughput (MBps) Line Rate (GBAUD) T11 Spec Technically Completed (Year) Market Availability (Year) 1GFC GFC GFC FC 8GFC GFC GFC GFCp X GFC TBD GFC TBD GFC TBD GFC TBD 2025 Market Demand Market Demand Market Demand Market Demand 1TFC TBD 2028 Market Demand FC used throughout all applications for Fibre Channel infrastructure and devices, including edge and ISL interconnects. Each speed maintains backward compatibility at least two previous generations (I.e., 8GFC backward compatible to 4GFC and 2GFC) Line Rate: All GFC speeds listed above are single-lane serial stream I/O s. All GFCp speeds listed above are multi-lane I/Os Dates: Future dates estimated

48 Fibre Channel FC-PI6 32G Approved Objectives 32G activity started: 2010 Expected completion: 2014 OM3/OM4: 70 m/100 m SMF 10 km SFP+ form factor SFP+ 70m OM3 100m OM4 So ource: Avago

49 FC-PI6 Project - 32G Optical Objectives Same external connectors LC and SFP+ Cable length of 100m on OM4 cables Duplex fiber 850nm Serial transmission FEC Power goal is less power per port than comparable 40GE port in 2014/15 timeframe Auto-Negotiation 8/16/32G Products ship in 2015 Source: Brocade SAN Director

50 Use Case #2 128GFC Port Breakout A 128GFC port can be broken out into 4x32GFC, 16GFC or 8GFC ports 16X128GFC line card yields 64 duplex MMF circuits QSFP+ transceiver from factor

51 InfiniBand Roadmap Source: IBTA

52 InfiniBand HPC New Business Vertical for fiber with => 56G Dominated By Twinax Copper (3-5m) up to 40G Expect =>100G to be optical connectivity AOC (SiP / MMF) Pluggable Modules Embedded Optics Passive Twinax 1-2m Spine-Leaf Architecture ToR 1:1 Over-Subscription 100G Server I/ Source: Mellanox

53 Thank you

54 Contact Info Doug Coleman th Street NW Hickory, NC