within IEEE Alan Flatman LAN Technologies

Transcription

1 Ethernet Developments within IEEE Alan Flatman Principal Consultant Principal Consultant LAN Technologies

2 Focal Points Bandwidth Growth Technology Choice What s Next? Market Trends Copper vs. Fibre

3 5 Generations of Ethernet Moore s Law ISP = Internet Service Providers CP = Content Providers HPC = High Performance Computing DCs = Data Centres 10G 40/100G ISP nets, Servers, DCs, Backbones ISP/CP nets, HPC, DCs, BBs 1G Backbones, Servers, PCs 10M PCs/Servers/ Peripherals 100M Backbones, Servers, PCs x2 speed every 24 months

4 Higher Speed Networking Scenarios

5 Market Drivers for Higher Speed Campus Backbones Corporate Backbones Corporate Data Centres ISP Aggregation High Performance Computing (HPC) Broadband Aggregation Internet exchanges ISP Backbones Content Providers Year HSE Market Adopters

6 Global Subscriber Access Traffic Million ns TB/yea ar 300 IPTV 250 Internet Phone M IPTV subscribers assumed in 2011 with 4 hours/day from 2007 Quote from Deutsche Telecom traffic is expected to grow faster than Moore s Law, with a 10-fold increase in the next 4 years, a 100-fold increase in the next 8 years, making today traffic in 2007 just 1% of the traffic expected in 2015!

7 Bandwidth Supply & Demand Terabit Ethernet? Gigabit Ethernet it/s) 40 Gigabit Ethernet Bit Ra ate (Mbi Core Networking Doubling ~18 months 10 Gigabit Ethernet Gigabit Ethernet 100 Server I/O Doubling ~24 months

8 Ethernet Switch Port Shipments Worldwide Ethernet Switch Volume; Dell Oro (2006) 10G 1G 100M 10M very slow start! Switch Ports (millions)

9 Optical Fibre Ethernet Price vs. Speed Source: Dell Oro (2007) abit d per Giga ous Speed tive Price er Previo Relat Ov Price Parity 100M 1G 10G Price Parity

10 10GBASE-T Deployment early 90nm chips consumed watts (2006) a 24-port switch would need 300 watts of power! a 40nm chip sampled in Feb 2010 with power consumption reduced to 4 watts (Broadcom) switch products announced by Cisco Mar 2010 commodity x.86 servers are now shipping with 10GBASE-T ports as standard many data centres are now 10GBASE-T ready

11 Higher Speed Ethernet (HSE) Study Group CFI Task Force sanctioned TF Review WG Ballot Sponsor Ballot Standard J A S O N D 2006 J F M A M J J A S O N D 2007 J F M A M J J A S O N D 2008 J F M A M J J A S O N D 2009 J F M A M J 2010 project sanctioned Dec 2007 after long study phase decided to specify 40G+100G via SMF, MMF, Copper 8 technical solutions defined for different markets far more complex than any existing Ethernet project well managed group routinely hit every milestone IEEE 802.3ba approved 17 June 2010, now published

12 HSE Naming 40/100GBASE-xyz family of specs: x = media = K backplane copper C cable assembly copper S short reach, 100m optics L long reach, 10km optics E extended long reach, 40km optics y = coding = R 64B/66B block encoding z = # lanes = 4 or 10 copper number of lanes or λs optics 1 not required (serial)

13 HSE Media & Reach Objectives 40G 100G 1m Backplane 7m Copper 100m MMF 10km SMF 40km SMF

14 HSE Technical Approaches 100GBASE-ER4 100GBASE-LR4 100GBASE-SR10 100GBASE-CR10 40GBASE-LR4 40GBASE-SR4 40GBASE-CR4 40GBASE-KR4 4λx25G based on ITU G DWDM 4λx25G based on ITU G DWDM 850nm; 100m OM3, 150m OM4 10x10G over 7m twinax cable assembly 4λx10G based on ITU G CWDM 850nm; 100m OM3, 150m OM4 4x10G over 7m twinax cable assembly 4x10GBASE-KR over 1m backplane ITU G.694.1: Widely Spaced DWDM Grid for LAN Applications ITU G.694.2: CWDM Grid for LAN Applications

15 40G Backplane existing 10GBASE-KR 10G backplane adopted as the basis of 4-lane solution for 40GBASE-KR4: ohm PCB tracks, 2 connectors & <1m total length 2. backplane Ethernet auto-neg enhanced to support 40G

16 ux Mux/Demu 4λx25G Optical M Optical Multiplexers for 100G SMF

17 Parallel Lanes for MMF low cost 850nm 10G VCSEL/PIN arrays parallel/ribbon fibres + MPO connectors 40G duplex link = fibres 100G duplex link = fibres Tx to Rx lanes mapped automatically 100m OM3 fibre (1.5dB connector loss) 150m OM4 fibre (1.0dB connector loss)

18 Data Centre Switch-to-Switch Link Lengths % 97% Source: IEEE802.3ba flatman_01_ ks ive % Lin OM3 OM4 SMF Cumulati Length (m)

19 MPO Connectors & Assignments Side-by-side Ports TxTxTxTxTxTxTxTxTxTx RxRxRxRxRxRxRxRxRxRx RxRxRxRxRxRxRxRxRxRx Vertically Stacked Ports TxTxTxTxTxTxTxTxTxTx TxTxTxTx RxRxRxRx Single Port RxRxRxRxRxRxRxRxRxRx TxTxTxTxTxTxTxTxTxTx preference for 24-way 40GBASE-SR4 100GBASE-SR10

20 Parallel Lanes for Copper 7m max supported with multi-lane twinax cables 40G duplex link = cables 100G duplex link = cables 28AWG twinax typical cable dimensions: 8mm OD for an 8-pair bundle (40G) 12mm OD for a 20-pair bundle (100G) QSFP MSA or CX4 connector for 40GBASE-CR4 SFF-8642 connector adopted for 100GBASE-CR10 MSA = Multi Source Agreement

21 40GBASE-CR4 & 100GBASE-CR10 Interface Connectors 40GBASE-CR4 100GBASE-CR10 Style 1 (QSFP) Style 2 (CX4) SFF-8642

22 Higher Speed Ethernet Markets Metropolitan Networks 100 Gig needed for ISP/CP links today 10km and 40km SMF links required high equipment costs easily justified but a relatively small volume market High Performance Computing early market for 100 Gig Ethernet 100m links will cover all needs low latency, low power essential larger volume market/cost sensitive Server Environment/Data Centres need 40 Gig from 2012, 100 Gig 2017 majority of server links under 30m low latency, low power essential huge volume market/ v. cost sensitive Enterprise Backbone Networks may need 100 Gig after 2012 SMF essential for links > 150m low latency/power not essential smaller market/not as cost sensitive

23 40GBASE-T growing interest in 40G over twisted pair cabling technical/market analysis presented at Aug 2009 workshop in Penn State, Pennsylvania UTP operation to be considered but PIMF cable would significantly reduce electronics complexity, power consumption, latency, implementation cost 100m reach is technically feasible with Cat 7A but 30-40m may be optimum target for server-switch PIMF = Pairs in Metal Foils

24 40GBASE-T Comparisons Advantages over 40GBASE-CR m reach would enable hot servers and hot switches to be physically distributed to spread the thermal load in a data centre supports auto-negotiation supported by managed/structured cabling Advantages over 40GBASE-SR4 less expensive cabling & electronics supports auto-negotiation

25 Generic Data Centre Model 24:2 switch downlinks to uplinks 2 connections per server Storage Storage Storage Storage Storage Storage

26 x86 Servers by Ethernet Port Speed Based on IDC (2009) Server Forecast & Ethernet t port speed analysis by Intel/Broadcom (2008) millio ns serverr units M 1G 10G 40G 100G

27 Data Centre Server-to-Switch Switch Link Lengths 40GBASE-CR4 100GBASE-CR Source: ks ive % Lin Cu Fibre Cumulati % Length (m)

28 Server-to-Switch Data Centre Links based on 7m Limit it for 40GBE & 100GBE Total Se erver-to-s Switch Link ks (million s) GBE 7-100m Fibre 40GBE 7-100m Fibre GBE 0-7m Copper 40GBE 0-7m Copper 10GBE 0-100m Copper 20 1GBE 0-100m Copper

29 Server-to-Switch Data Centre Links based on 20m Limit it for 40GBE & 100GBE Total Se erver-to-s Switch Link ks (million s) GBE m Fibre 40GBE m Fibre GBE 0-20m Copper 40GBE 0-20m Copper 10GBE 0-100m Copper 20 1GBE 0-100m Copper doubles volume on 7m reach

30 Server-to-Switch Data Centre Links based on 30m Limit it for 40GBE & 100GBE Total Se erver-to-s Switch Link ks (million s) GBE m Fibre 40GBE m Fibre GBE 0-30m Copper 40GBE 0-30m Copper 10GBE 0-100m Copper 20 1GBE 0-100m Copper extra 18.5% of total volume on 20m reach

31 Server-to-Switch Data Centre Links based on 40m Limit it for 40GBE & 100GBE Total Se erver-to-s Switch Link ks (million s) GBE m Fibre 40GBE m Fibre GBE 0-40m Copper 40GBE 0-40m Copper 10GBE 0-100m Copper 20 1GBE 0-100m Copper extra 7% of total volume on 30m reach

32 Server-to-Switch Data Centre Links based on 50m Limit it for 40GBE & 100GBE Total Se erver-to-s Switch Link ks (million s) GBE m Fibre 40GBE m Fibre GBE 0-50m Copper 40GBE 0-50m Copper 10GBE 0-100m Copper 20 1GBE 0-100m Copper extra 3.5% of total volume on 40m reach

33 40GBE & 100GBE Copper Link Shares vs. Length 100% lative G cumul G/100G links 20 hare of 4 server % s 80% 60% 40% 20% 0% Fibre sweet spot 40GBASE-T 100GBASE-T Reach Capability of 40GBASE-T & 100GBASE-T (m)

34 Server-to-Switch Data Centre Links based on 40GBASE-CR4 & 100GBASE-CR10 Fibre Necessary Copper Possible Total Server-to-Switch Links (million s)

35 Server-to-Switch Data Centre Links based on 40GBASE-T with 40m Reach Fibre Necessary Copper Possible Total Server-to-Switch Links (million s)

36 The Case for Higher Speeds over Twisted Pairs significant market opportunity for 40GBASE-T in data centres with reach in the range 30-40m ~50M additional links captured by copper to 2020 equals 80M 40G links captured by copper to GBASE-T also technically feasible but data centre market is longer term than 40GBASE-T probably not worth exceeding 40m link reach 40GBASE-T transceiver cost << 40GBASE-SR4 Cat 7A cabling cost << 8-fibre bundle with MPOs

37 Technologies For Next Generation Optics We are here today!

38 Observations dispersion limit of MMF forces parallel solutions 8 fibres for 10G per lane 20 fibres for 10G per lane 32 fibres for 25G per lane SMF needed for 40G, 100G links exceeding 150m MMF reach expected to shrink at higher speeds price gap between copper and fibre will grow at higher speeds 40G/100G CR links will address short-term needs for top-of-rack connections 40GBASE-T will address longer-term needs for server-to-switch connections in the data centre BUT the solution must be optimised i for 30-40m reach to limit cost, latency & power consumption

39