Ethernet evolution: what s new and what s next

Size: px

Start display at page:

Download "Ethernet evolution: what s new and what s next"

Brandon Stokes
5 years ago
Views:

1 Ethernet evolution: what s new and what s next Greg Hankins <greg.hankins@nokia.com> European Peering Forum Nokia 2016 EPF /09/19

2 I ve been giving presentations about Ethernet developments for over 10 years... Thanks for listening! Cutting edge: 24 x 10 GE switch 2 Nokia 2016

Ethernet evolution over 40+ years Driven by new diverse market requirements Market requirements for Ethernet are changing for different applications based on its success - Speed - Distance - Media -

3 Ethernet evolution over 40+ years Driven by new diverse market requirements Market requirements for Ethernet are changing for different applications based on its success - Speed - Distance - Media - Cost Six new Ethernet speeds are coming to market to address these different requirements - Same amount as in the past 30 years New standards for existing speeds are also in development - Newer technology, lower cost - Automotive and industrial applications - Passive optical networks (PON) Roadmap courtesy of the Ethernet Alliance: 3 Nokia 2016

4 Recent IEEE Ethernet standards 803.3bw-2015: 100BASE-T by-2016: 25 Gb/s Ethernet 803.2bq-2016: 25/40GBASE-T 802.3bp-2016: 1000BASE-T br-2016: Interspersing Express Traffic (IET) IEEE Ethernet Working Group: 4 Nokia 2016

5 IEEE Ethernet Working Group projects There s a lot going on here... P802.3bn: EPON Protocol over Coax (EPoC) Task Force P802.3bs: 200 Gb/s and 400 Gb/s Ethernet Task Force P802.3bt: DTE Power via MDI over 4-Pair Task Force P802.3bu: 1-Pair Power over Data Lines (PoDL) Task Force P802.3bv: Gigabit Ethernet Over Plastic Optical Fiber Task Force P802.3bz: 2.5G/5GBASE-T Task Force P802.3ca: 25 Gb/s, 50 Gb/s, and 100 Gb/s Ethernet Passive Optical Networks Task Force P802.3cb: 2.5 Gb/s and 5 Gb/s Backplane and Copper Cables Task Force P802.3cc: 25 Gb/s Ethernet over Single-Mode Fiber Task Force P802.3cd: 50 Gb/s, 100 Gb/s, and 200 Gb/s Ethernet Task Force P /Cor 1: (IEEE 802.3ce) Multilane timestamping YANG Data Model(s) Study Group Mb/s Single Twisted Pair Ethernet Study Group Multi-Gig Automotive Ethernet PHY Call for Interest (for discussion at the November 2016 Plenary) IEEE Ethernet Working Group: 5 Nokia 2016

Higher speed Ethernet target applications Key application drivers 25 GE 25 GE 50 GE 50 GE 100 GE 100 GE 200 GE 400 GE 200 GE 50 GE 100 GE 25 GE 200 GE 50 GE 400 GE 100 GE 200 GE 25/50/100/200 GE

6 Higher speed Ethernet target applications Key application drivers 25 GE 25 GE 50 GE 50 GE 100 GE 100 GE 200 GE 400 GE 200 GE 50 GE 100 GE 25 GE 200 GE 50 GE 400 GE 100 GE 200 GE 25/50/100/200 GE applications Data center aggregation and core Data center access Server NICs Metro/campus core 50/100/200 GE applications Service provider aggregation and core Data center core Metro core 200/400 GE applications 25 GE 50 GE 100 GE 200 GE 25 GE 50 GE 100 GE 200 GE Service provider core Large data center core Large metro core 6 Nokia 2016

7 Agenda What s new: 2.5 GE and 5 GE (don t have time, grab for an intro) 1. What s new: 25 GE (grab for background info) What s new: 40 GE (no new development, see reference slides in back) 2. What s next: 50 GE 3. What s new: 100 GE 4. What s next: 200 GE 5. What s next: 400 GE What s new: detailed optical developments (don t have time, grab /presentations/2.3_Christian_Urricariet_AusNOG2016.pdf for an overview) 7 Nokia 2016

8 Market drivers for 25 Gb/s signaling and 25 GE Provide a connection speed faster than 10 GE that is optimized for cost, throughput and efficiency Use a single 25 Gb/s signaling lane based on existing 25 Gb/s technology that is used in other Ethernet speeds and in pluggable modules Connections to switch ASICs is limited by SerDes count and bandwidth - Single higher speed 25 Gb/s lanes maximize bandwidth and switch fabric utilization vs. 4 x 10 Gb/s lanes A single higher speed lane per physical port maximizes the number of connections per switch Initially for server to access switch connections, expanded for longer reaches SFP28 pluggable module (same size as SFP and SFP+) Image courtesy of the SFF Committee. 8 Nokia 2016

9 25 GE developments IEEE P802.3cc 25 Gb/s Ethernet over Single-Mode Fiber Task Force started in May GBASE-LR: 1 x 25 Gb/s over 10 km duplex (based on 100GBASE-LR4 with RS-FEC) - 25GBASE-ER: 1 x 25 Gb/s over 40 km duplex (based on ITU-T 4L1-9D1F ER4-lite with RS-FEC) - Optional Energy Efficient Ethernet (EEE) support (fast wake mode only) Generated Draft 1.0 for Task Force review Standard expected in October 2017 Interfaces expected on the market in Task Force web page 9 Nokia 2016

10 25 GE technology reference Physical layer reach 1 m backplane 3 m copper cable 5 m copper cable 30 m twisted-pair 70 m OM3 / 100 m OM4 10 km 40 km Name 25GBASE-KR-S 25GBASE-KR 25GBASE-CR-S 25GBASE-CR 25GBASE-T 25GBASE-SR 25GBASE-LR 25GBASE-LR Standard June 2016 IEEE 802.3by June 2016 IEEE 802.3by June 2016 IEEE 802.3by June 2016 IEEE 802.3bq June 2016 IEEE 802.3by October 2017 IEEE 802.3cc October 2017 IEEE 802.3cc Electrical signaling (Gb/s) 1 x 25 1 x 25 1 x 25 1 x 25 1 x 25 1 x 25 1 x 25 Media signaling (Gb/s) 1 x 25 1 x 25 1 x 25 1 x 25 1 x nm λ 1 x nm λ 1 x nm λ Media type Backplane Twinax copper Twinax copper Cat 8 Duplex MMF Duplex Duplex Module type Market availability Backplane SFP28 SFP28 RJ45 SFP28 SFP28 SFP Nokia 2016

11 25 GE details IEEE Std 802.3bq GBASE-T: 1 x 25 Gb/s over 30 m Cat 8 4-pair twisted-pair copper cabling (ISO/IEC JTC1 SC25 WG3 and TIA TR-42.7) Optional Energy Efficient Ethernet (EEE) support 11 Nokia 2016

12 25 GE details IEEE Std 802.3by GBASE-KR-S: 1 x 25 Gb/s over 1 m Megtron 6 backplane (no FEC, BASE-R FEC) 25GBASE-KR: 1 x 25 Gb/s over 1 m Megtron 6 backplane (no FEC, BASE-R FEC, RS-FEC) 25GBASE-CR-S: 1 x 25 Gb/s over 3 m copper twinax cable (no FEC, BASE-R FEC, RS-FEC) 25GBASE-CR: 1 x 25 Gb/s over 5 m copper twinax cable (RS-FEC) 25GBASE-SR: 1 x 25 Gb/s over 70 m OM3 and 100 m OM4 duplex MMF Auto-negotiation between copper interface types and FECs Passive direct attach cable (DAC) types - CA-N (2 m; no FEC, BASE-R FEC, RS-FEC) - CA-S (3 m; no FEC, BASE-R FEC, RS-FEC) - CA-L (5 m; RS-FEC) Optional Energy Efficient Ethernet (EEE) support Forward Error Correction (FEC) Added latency* (ns) Bit Error Rate (BER) None BASE-R (Clause 74) *The latency added by FEC is insignificant for links > 500 m. 12 Nokia 2016 RS (Clause 91)

13 Agenda 1. What s new: 25 GE 2. What s next: 50 GE 3. What s new: 100 GE 4. What s next: 200 GE 5. What s next: 400 GE 13 Nokia 2016

14 Market drivers for 50 Gb/s signaling and 50 GE A faster base signaling rate is needed to for higher capacities - Rack sizes, rack units and front panels on devices must stay the same size - This means we have to go faster to support higher capacity networks Similar to 25 GE, 50 GE extends existing common network topologies for higher speeds - Reuses existing leaf/spine network architectures - Reuses existing 1 x up / 4 x down architecture The server and data center market requirements vary widely - Variety of CPU architectures, clock speeds, number of cores, and CPUs per system - Different applications have different IO and compute power requirements - Need a variety of low and high speed options to meet bandwidth requirements CFI: 14 Nokia 2016

15 50 Gb/s maximizes bandwidth and efficiency ToR switch ASICs are designed to support 128 I/O lanes Connections to switch ASICs is limited by SerDes count and bandwidth Single higher speed 50 Gb/s lanes maximize bandwidth and switch fabric utilization A single lane per physical port maximizes utilization and total interconnect bandwidth Port speed (GE) Lane speed (Gb/s) Lanes / port Usable ports lane switch capacities Total capacity (Tb/s) Now Future Source: 15 Nokia 2016

50 GE pluggable modules Higher speeds in common sizes New 50 GE pluggable modules in same common form factor sizes SFP56 - Extension of SFF-8402-1 x 50 Gb/s for 1 x 50 GE QSFP56 - Extension of

16 50 GE pluggable modules Higher speeds in common sizes New 50 GE pluggable modules in same common form factor sizes SFP56 - Extension of SFF x 50 Gb/s for 1 x 50 GE QSFP56 - Extension of SFF x 50 Gb/s for 4 x 50 GE breakout - 4 x 50 Gb/s for 1 x 200 GE Enables common higher speed port density combinations : 32 x QSFP+ ports support 4 x 10 GE down and 40 GE up : 32 x QSFP28 ports support 4 x 25 GE down and 100 GE up : 32 x QSFP56 ports support 4 x 50 GE down and 200 GE up Images courtesy of the SFF Committee. 16 Nokia 2016 SFP56 pluggable module (same size as SFP, SFP+, SFP28) QSFP56 pluggable module (same size as QSFP, QSFP+, QSFP28)

17 50 GE developments IEEE P802.3cd 50 Gb/s, 100 Gb/s, and 200 Gb/s Ethernet Task Force started in May GBASE-KR: 1 x 50 Gb/s over 1 m Megtron 6 backplane - 50GBASE-CR: 1 x 50 Gb/s over 3 m copper twinax cable - 50GBASE-SR: 1 x 50 Gb/s over 70 m OM3 and 100 m OM4 duplex MMF - 50GBASE-FR: 1 x 50 Gb/s over 2 km duplex - 50GBASE-LR: 1 x 50 Gb/s over 10 km duplex - Optional Energy Efficient Ethernet (EEE) Standard expected in September 2018 Interfaces expected on the market in Task Force web page 17 Nokia 2016

18 50 GE technology reference Physical layer reach 1 m backplane 1 m backplane 5 m copper cable 5 m copper cable 70 m OM3 / 100 m OM4 2 km 10 km Name 50GBASE-KR 50GBASE-KR2 50GBASE-CR 50GBASE-CR2 50GBASE-SR 50GBASE-FR 50GBASE-LR Standard September 2018 IEEE 802.3cd September G Ethernet Consortium September 2018 IEEE 802.3cd September G Ethernet Consortium September 2018 IEEE 802.3cd September 2018 IEEE 802.3cd September 2018 IEEE 802.3cd Electrical signaling (Gb/s) 1 x 50 2 x 25 1 x 50 2 x 25 1 x 50 1 x 50 1 x 50 Media signaling (Gb/s) 1 x 50 2 x 25 1 x 50 2 x 25 1 x nm λ 1 x nm λ 1 x nm λ Media type Backplane Backplane Twinax copper Twinax copper Duplex MMF Duplex Duplex Module type Backplane Backplane SFP56, QSFP56 TBD SFP56, QSFP56 SFP56, QSFP56 SFP56, QSFP56 Market availability TBD TBD Nokia 2016

19 Agenda 1. What s new: 25 GE 2. What s next: 50 GE 3. What s new: 100 GE 4. What s next: 200 GE 5. What s next: 400 GE 19 Nokia 2016

20 Chasm 100 GE technology adoption lifecycle 2 nd generation 100 GE crossed the chasm 1M 100 GE ports projected in 2016 by Dell Oro 802.3cd 100 GE at 2 x 50 Gb/s 3.2 Tb/s 6.4 Tb/s Innovators Early adopters Early majority Late majority Laggards premium ports/slot CXP and CFP high cost ports/slot CFP lower cost ports/slot QSFP28 and CFP4 OSFP High density low cost ports OSFP and microqsfp 1 st generation 2 nd generation 3 rd generation High density commodity ports Future SFP112 and QSFP112 Pluggable module images courtesy of Finisar. microqsfp image courtesy of the microqsfp MSA. 20 Nokia 2016

21 100 GE pluggable module evolution Graphical view of module form factors XFP (10 GE) CFP4 CFP2 CFP CFP vs. iphone 6 CFP2 vs. iphone 6 CFP4 vs. iphone 6 XFP(10 GE) CFP4 CFP2 CFP CFP2 CFP4 QSFP28* CFP images courtesy of the CFP MSA. QSFP28 image courtesy of the SFF Committee. *Not quite to scale. 21 Nokia 2016 CFP

100 GE pluggable module evolution microqsfp Designed for high density, shorter reach and lower power requirements - Provides QSFP functionality in an SFP-sized form factor supporting up to >3.

22 100 GE pluggable module evolution microqsfp Designed for high density, shorter reach and lower power requirements - Provides QSFP functionality in an SFP-sized form factor supporting up to >3.5 W - Can support DAC, MMF and cabling Features four electrical channels - 25 Gb/s data rates support 100 GE or 4 x 25 GE breakout - Expected to support 50 Gb/s data rates Maximum 1 RU densities of - 48 ports (4.8 Tb/s) - 72 ports in 3 rows (7.2 Tb/s) Increase in port density of - 4 x SFP x QSFP Images courtesy of the microqsfp MSA. 22 Nokia 2016

23 100 GE developments IEEE P802.3cd 50 Gb/s, 100 Gb/s, and 200 Gb/s Ethernet Task Force started in May GBASE-KR2: 2 x 50 Gb/s over 1 m Megtron 6 backplane - 100GBASE-CR2: 2 x 50 Gb/s over 3 m copper twinax cable - 100GBASE-SR2: 2 x 50 Gb/s over 70 m OM3, 100 m OM4 and 150 m WBMMF duplex MMF - 100GBASE-DR2: 2 x 50 Gb/s over 500 m parallel (TBD) - 100GBASE-FR2: 2 x 50 Gb/s over 2 km duplex (TBD) - Optional Energy Efficient Ethernet (EEE) Standard expected in September 2018 Interfaces expected on the market in Task Force web page 23 Nokia 2016

24 100 GE IEEE technology reference Physical layer reach 1 m backplane 1 m backplane 3 m copper cable 5 m copper cable 7 m copper cable 70 m OM3 / 100 m OM4 70 m OM3 / 100 m OM4 / 150 m WBMMF 100 m OM3 / 150 m OM4 500 m 2 m 10 km 40 km Name 100GBASE- KP4 100GBASE- KR4 100GBASE- KR2 100GBASE- CR2 100GBASE- CR4 100GBASE- CR10 100GBASE- SR4 100GBASE- SR2 100GBASE- SR10 100GBASE- DR2 100GBASE- DF2 100GBASE- LR4 100GBASE- ER4* Standard June 2014 IEEE 802.3bj Sep 2018 IEEE 802.3cd Sep 2018 IEEE 802.3cd June 2014 IEEE 802.3bj June 2010 IEEE 802.3ba Feb 2015 IEEE 802.3bm Sep 2018 IEEE 802.3cd June 2010 IEEE 802.3ba Sep 2018 IEEE 802.3cd Sep 2018 IEEE 802.3cd June 2010 IEEE 802.3ba June 2010 IEEE 802.3ba Electrical signaling (Gb/s) 4 x 25 2 x 50 2 x 50 4 x x 10 4 x 25 2 x x 10 2 x 50 2 x x x 10 Media signaling (Gb/s) 4 x 25 NRZ or PAM-4 2 x 50 2 x 50 4 x x 10 4 x nm λs 2 x nm λ 900 nm λ 10 x nm λs 2 x 50 2 x 50 4 x nm λs 4 x nm λs Media type Backplane Backplane Twinax Twinax Twinax Parallel MMF (MPO12) Duplex MMF Parallel MMF (MPO24) Parallel (MPO12) Duplex Duplex Duplex Module type Backplane Backplane QSFP28 CFP2, CFP4, QSFP28 CXP, CFP2, CFP4, QSFP28 CFP2, CFP4, CPAK, QSFP28 QSFP28 CFP, CFP2, CFP4, CPAK, CXP, QSFP28 QSFP28 QSFP28 CFP, CFP2, CFP4, CPAK, QSFP28 CFP, CFP2 Market availability None known *100GBASE-ER4-lite (ITU-T application code 4L1-9D1F) has different optical specifications and uses the RS-FEC. 24 Nokia 2016

25 100 GE MSA technology reference Physical layer reach 70 m OM3 / 100 m OM4 500 m 2 km 2 km 2 km 2 km 10 km 40 km Name SWDM4 PSM4 10x10-2km CWDM4 CLR4 OpenOptics 10x10-10km 10x10-40km Standard Future SWDM4 Alliance September G PSM4 MSA March x10 MSA November 2015 CWDM4 MSA Group March G CLR4 Alliance March 2014 OpenOptics MSA August x10 MSA August x10 MSA Electrical signaling (Gb/s) 4 x 25 4 x x 10 4 x 25 4 x 25 4 x x x 10 Media signaling (Gb/s) 4 x nm λs 4 x nm λs 10 x nm λs 4 x nm λs 4 x nm λs 4 x nm λs 10 x nm λs 10 x nm λs Media type Duplex MMF Parallel (MPO12) Duplex Duplex Duplex Duplex Duplex Duplex Module type QSFP28 CFP4, QSFP28 CFP CFP2, CFP4, QSFP28 QSFP28 QSFP28 CFP CFP Market availability TBD TBD Non-IEEE media signaling standards that comply to the IEEE electrical signaling interface. MSA means Multisource Agreement. 25 Nokia 2016

26 Agenda 1. What s new: 25 GE 2. What s next: 50 GE 3. What s new: 100 GE 4. What s next: 200 GE 5. What s next: 400 GE 26 Nokia 2016

27 Market drivers for 200 GE Similar to 25 GE and 50 GE, 200 GE extends existing common network topologies for higher speeds - Reuses existing leaf/spine network architectures - Reuses industry investment in 50 Gb/s technology - Reuses existing 1 x up / 4 x down architecture Network backbone links need to scale to higher, cost effective speeds to match growing server speeds at the edge - Servers migrating from 10 GE to 25 GE, 40 GE and 50 GE Investment in MMF cabling infrastructure can support 40 GE to 100 GE to 200 GE capacity upgrades - Same {40,100, 200}GBASE-SR4 technologies with MPO12 connector - 4 x 850 nm λs TX, 4 x 850 nm λs RX CFI: 27 Nokia 2016

28 200 GE maximizes aggregation bandwidth Total capacity (Tb/s) 100 GE ports, 25 Gb/s lanes 200 GE ports, 50 Gb/s lanes 400 GE ports, 50 Gb/s lanes Less than current port densities Greater than current port densities ToR switch port densities Current leaf/spine network architectures are designed around 32 x 100 GE ToR switches 200 GE interfaces in a leaf/spine switch provide higher port density at a lower ASIC bandwidth than 400 GE Source: 28 Nokia 2016

MMF and cabling Features eight electrical channels - 25 Gb/s and 50 Gb/s data rates - Supports 200 GE and 400 GE - 1 x or 4 x breakout Maximum 1

29 200 and 400 GE pluggable module evolution QSFP-DD (Quad Small Form-factor Pluggable - Double Density) Designed for high speed and high density requirements Slightly deeper form factor than QSFP28 supporting >7 W (target up to 10 W) - Backwards compatible with QSFP28 - Can support DAC, MMF and cabling Features eight electrical channels - 25 Gb/s and 50 Gb/s data rates - Supports 200 GE and 400 GE - 1 x or 4 x breakout Maximum 1 RU density of 36 x QSFP-DD (14.4 Tb/s) 2 x 1 stacked integrated cage and connector QSFP-DD QSFP28 Images courtesy of the QSFP-DD MSA. 29 Nokia 2016

30 200 GE developments Copper and fiber optic cables IEEE P802.3cd 50 Gb/s, 100 Gb/s, and 200 Gb/s Ethernet Task Force started in May GBASE-KR4: 4 x 50 Gb/s over 1 m Megtron 6 backplane - 200GBASE-CR4: 4 x 50 Gb/s over 3 m copper twinax cable - 200GBASE-SR4: 4 x 50 Gb/s over 70 m OM3 and 100 m OM4 parallel MMF - Optional Energy Efficient Ethernet (EEE) Working towards Draft 1.0 Standard expected in September 2018 Interfaces expected on the market in Task Force web page 30 Nokia 2016

31 200 GE developments Fiber optic cables IEEE P802.3bs 400 Gb/s Ethernet Task Force adopted 200 GE objectives in May 2016, and became the IEEE P802.3bs 200 Gb/s and 400 Gb/s Ethernet Task Force - 200GBASE-DR4: 4 x 50 Gb/s over 500 m parallel - 200GBASE-FR4: 4 x 50 Gb/s over 2 km duplex - 200GBASE-LR4: 4 x 50 Gb/s over 10 km duplex - Optional Energy Efficient Ethernet (EEE) Generated Draft 1.5 for Task Force review Standard expected in December 2017 (no change in standard schedule due to 200 GE) Interfaces expected on the market in Task Force web page 31 Nokia 2016

32 200 GE technology reference Physical layer reach 1 m backplane 3 m copper cable 70 m OM3 / 100 m OM4 500 m 2 km 10 km Name 200GBASE-KR4 200GBASE-CR4 200GBASE-SR4 200GBASE-DR4 200GBASE-FR4 200GBASE-LR4 Standard September 2018 IEEE 802.3cd September 2018 IEEE 802.3cd September 2018 IEEE 802.3cd December 2017 IEEE 802.3bs December 2017 IEEE 802.3bs December 2017 IEEE 802.3bs Electrical signaling (Gb/s) 4 x 50 4 x 50 8 x 25 4 x 50 8 x 25 4 x 50 8 x 25 4 x 50 8 x 25 4 x 50 8 x 25 Media signaling (Gb/s) 4 x 50 4 x 50 4 x nm λs 4 x nm λs 4 x nm λs 4x nm λs Media type Backplane Twinax Parallel MMF (MPO12) Parallel (MPO12) Duplex Duplex Module type Backplane QSFP56, QSFP-DD QSFP56, QSFP-DD QSFP56, QSFP-DD QSFP56, QSFP-DD QSFP56, QSFP-DD Market availability Nokia 2016

33 Agenda 1. What s new: 25 GE 2. What s next: 50 GE 3. What s new: 100 GE 4. What s next: 200 GE 5. What s next: 400 GE 33 Nokia 2016

34 400 GE developments IEEE P802.3bs 400 Gb/s Ethernet Task Force started in March 2014 Adopted 200 GE objectives in May 2016, and became the IEEE P802.3bs 200 Gb/s and 400 Gb/s Ethernet Task Force - 400GBASE-SR16: 16 x 25 Gb/s over 70 m OM3, 100 m OM4 and 150 m WBMMF (TBD) parallel MMF - 400GBASE-DR4: 4 x 100 Gb/s over 500 m parallel - 400GBASE-FR8: 8 x 50 Gb/s over 2 km duplex - 400GBASE-LR8: 8 x 50 Gb/s over 10 km duplex - Electrical interfaces: 25 Gb/s (NRZ) and 50 Gb/s (PAM-4) - Optional Energy Efficient Ethernet (EEE) Generated Draft 1.5 for Task Force review Standard expected in December 2017 (no change in standard schedule due to 200 GE) First interfaces expected to be available in Task Force web page 34 Nokia 2016

35 400 GE pluggable module evolution estimates Each module increases density, while reducing cost and power Electrical signaling (Gb/s) 1 st generation 2 nd generation 3 rd generation 16 x 25 8 x 50 8 x 50 4 x 100 Ethernet at terabit speeds becomes feasible Module type CFP8 (~CFP2 size) CFP16 (~CFP4 size), OSFP, QSFP-DD CFP16, Future QSFP112 Market availability estimate CFP8 image courtesy Finisar. 35 Nokia 2016

36 400 GE technology reference Physical layer reach 70 m OM3 / 100 m OM4/WBMMF 500 m 2 km 10 km Name 400GBASE-SR16 400GBASE-DR4 400GBASE-FR8 400GBASE-LR8 Standard December 2017 IEEE 802.3bs December 2017 IEEE 802.3bs December 2017 IEEE 802.3bs December 2017 IEEE 802.3bs Electrical signaling (Gb/s) 16 x 25 8 x x 25 8 x x 25 8 x x 25 8 x 50 Media signaling (Gb/s) 16 x nm λs 4 x nm λs 8 x nm λs 8 x nm λs Media type Parallel MMF (MPO32) Parallel (MPO12) Duplex Duplex Module type CFP8, OSFP, QSFP-DD CFP8, OSFP, QSFP-DD CFP8, OSFP, QSFP-DD CFP8, OSFP, QSFP-DD Market availability Nokia 2016

37 Ethernet evolution futures Ethernet continues to evolve to meet new and diverse market requirements Different new speeds are needed for different new applications Old 10 x performance for 3 x cost model doesn t work anymore as we get to higher speeds - 10 ME 100 ME 1 GE 10 GE 100 GE Current best technical and economic solutions are multiples of the highest lane rate - 2 x 50 Gb/s for 100 GE - 4 x 50 Gb/s for 200 GE - 8 x 50 Gb/s for 400 GE New technology based on 50 Gb/s is the basis of a new generation of speeds 37 Nokia 2016

38 Ethernet evolution summary 2.5 GE and 5 GE is coming soon for higher speed Cat 5e/6 applications 10 GE is widely deployed in every part of the network 25 GE is available now for server and ToR aggregation 40 GE is widely deployed in data center networks 50 GE is in development for server and ToR aggregation 100 GE has transitioned to 2nd generation technology with CFP2, CFP4 and QSFP28 - New interfaces based on 25 Gb/s and 50 Gb/s signaling - New OSFP and microqsfp pluggable modules Gb/s serial signaling for longer reaches is still a challenge 200 GE is in development for server and ToR aggregation - New QSFP-DD pluggable modules 400 GE is in development for core networks Ethernet at Terabit speeds is still unfeasible in the near future, but we ll get there eventually (2020+) 38 Nokia 2016

39 References IEEE P802.3cc 25 Gb/s Ethernet over Single-Mode Fiber Task Force - IEEE P802.3cd 50 Gb/s, 100 Gb/s, and 200 Gb/s Ethernet Task Force - IEEE P802.3bs 200 Gb/s and 400 Gb/s Ethernet Task Force - CDFP MSA - CFP MSA - COBO - microqsfp MSA - QSFP-DD MSA - SFF Committee Nokia 2016

40 Questions? Acknowledgements: Scott Kipp, Brocade Samuel Liu, Nokia Steve Trowbridge, Nokia Christian Urricariet, Finisar Some older and reference slides are next Nokia 2016

MPO cable assemblies High density parallel fiber cabling 40 GE and 100 GE short reach pluggable modules use a Multifiber Push-On (MPO) cable assembly to interconnect network devices - Also called MTP

SC, etc - Keyed connectors maintain correct transmit/receive orientation 40GBASE-SR4 uses a 12-fiber OM3 or OM4 MMF MPO12 cable - 8 fibers used, left 4 for transmit and right 4 for receive - 4 middle

42 MPO cable assemblies High density parallel fiber cabling 40 GE and 100 GE short reach pluggable modules use a Multifiber Push-On (MPO) cable assembly to interconnect network devices - Also called MTP by US Conec Widely available in a variety of high density multimode fiber (MMF) and single-mode fiber () cabling options for data centers - MPO to MPO - MPO cassette for patch panels with into LC, SC, etc - Keyed connectors maintain correct transmit/receive orientation 40GBASE-SR4 uses a 12-fiber OM3 or OM4 MMF MPO12 cable - 8 fibers used, left 4 for transmit and right 4 for receive - 4 middle fibers are unused 100GBASE-SR10 uses a 24-fiber OM3 or OM4 MMF MPO24 cable - 20 fibers used, top middle 10 for receive and bottom middle 10 for transmit - 2 fibers on each end are unused MPO12 connector 10 x receive 10 x transmit 100GBASE-SR10 CXP 42 Nokia 2016

43 40 GE transmission Multimode and single-mode fiber Multimode ribbon fiber - Used for distances of 100 m on OM3 and 150 m on OM4 MMF - Data is sent using multiple 850 nm lasers transmitting over multiple parallel fibers - MPO cables provide multiple separate transmit and receive strands of multimode fiber in a ribbon cable assembly Single-mode duplex fiber - Used for distances of 2 km, 10 km and 40 km on standard duplex - WDM component in the pluggable module multiplexes four transmit λs over one strand of fiber and four receive λs over the other strand of fiber in the 1310 nm CWDM band for 40GBASE-LR4 and 40GBASE-ER4-40 Gb/s serial transmit over one strand of fiber and receive over the other strand of fiber on one 1550 nm λ is used for 40GBASE-FR 4 x 10 Gb/s over parallel MMF 40GBASE-SR4 4 x 10 Gb/s over duplex 40GBASE-LR4 and 40GBASE-ER4 43 Nokia 2016

44 100 GE transmission Multimode and single-mode fiber Multimode ribbon fiber - Used for distances of 100 m on OM3 and 150 m on OM4 MMF - Data is sent using multiple 850 nm lasers transmitting over multiple parallel fibers - MPO cables provide multiple separate transmit and receive strands of multimode fiber in a ribbon cable assembly Single-mode duplex fiber - Used for distances of 2 km, 10 km and 40 km on standard duplex - WDM component in the pluggable module multiplexes all transmit λs over one strand of fiber and all receive λs over the other strand of fiber 10x10 MSA standards use 10 x 10 Gb/s λs in the 1550 nm DWDM band IEEE standards use 4 x 25 Gb/s λs in the 1310 nm CWDM band 10 x 10 Gb/s over parallel MMF 100GBASE-SR10 10 x 10 Gb/s over duplex 10x10-2km, 10x10-10km and 10x10-40km 4 x 25 Gb/s over duplex 100GBASE-LR4 and 100GBASE-ER4 44 Nokia 2016

40 Gb/s QSFP+ modules overview Quad Small Form-factor Pluggable+ Created for high density interfaces primarily short reach interfaces for data center

Ethernet, Fibre Channel and InfiniBand applications - 40 GE uses 4 x 10 Gb/s bidirectional channels Supports a variety of copper and fiber 40 GE

45 40 Gb/s QSFP+ modules overview Quad Small Form-factor Pluggable+ Created for high density interfaces primarily short reach interfaces for data center applications - Small compact form factor enables low power consumption and high density - Also used for longer reach 40 GE Used for a variety of Ethernet, Fibre Channel and InfiniBand applications - 40 GE uses 4 x 10 Gb/s bidirectional channels Supports a variety of copper and fiber 40 GE interfaces - Breakout from 40 GE to 4 x 10 GE Same faceplate size as an XFP but slightly shorter 40GBASE-SR4 40GBASE-LR4 Images courtesy of Finisar. 45 Nokia 2016

40 GE QSFP+ pluggable modules Data center server and access: native 40 GE and 4 x 10 GE

100 m OM3/OM4 7 m passive copper cable 100 m OM3/ 150 m OM4 10 km 40 km Pluggable module

twinax (QSFP+ to 4 x SFP+) Parallel MMF (MPO to 4 x Duplex LC) Integrated twinax (QSFP+ to

IEEE 802.3ae June 2010 IEEE 802.3ba June 2010 IEEE 802.3ba June 2010 IEEE 802.3ba February 2015 IEEE 802.

46 40 GE QSFP+ pluggable modules Data center server and access: native 40 GE and 4 x 10 GE breakout Aggregation and core: native 40 GE Physical layer reach 10 m passive copper cable 100 m OM3/OM4 7 m passive copper cable 100 m OM3/ 150 m OM4 10 km 40 km Pluggable module 10GSFP+Cu 10GBASE-SR 40GBASE-CR4 40GBASE-SR4 40GBASE-LR4 40GBASE-ER4 Media Integrated twinax (QSFP+ to 4 x SFP+) Parallel MMF (MPO to 4 x Duplex LC) Integrated twinax (QSFP+ to QSFP+) Parallel MMF (MPO12) Duplex (LC) Duplex (LC) Standard July 2009 SFF-8431 June 2002 IEEE 802.3ae June 2010 IEEE 802.3ba June 2010 IEEE 802.3ba June 2010 IEEE 802.3ba February 2015 IEEE 802.3bm 40GBASE-SR4, 40GBASE-LR4, and 40GBASE-ER4 QSFP+ images courtesy of Finisar. 46 Nokia 2016

47 40 GE technology reference Physical layer reach 1 m backplane 7 m copper cable 30 m twisted-pair 100 m OM3 / 150 m OM4 2 km 10 km 40 km Name 40GBASE-KR4 40GBASE-CR4 40GBASE-T 40GBASE-SR4 40GBASE-FR 40GBASE- LR4 40GBASE-ER4 Standard June 2010 IEEE 802.3ba June 2010 IEEE 802.3ba June 2016 IEEE 802.3bq June 2010 IEEE 802.3ba March 2011 IEEE 802.3bg June 2010 IEEE 802.3ba February 2015 IEEE 802.3bm Electrical signaling (Gb/s) 4 x 10 4 x 10 4 x 10 4 x 10 4 x 10 4 x 10 4 x 10 Media signaling (Gb/s) 4 x 10 4 x 10 4 x 10 4 x nm λs 1 x nm λ (RX) 1550 nm λs (TX, RX) 4 x nm λs 4 x nm λs Media type Backplane Twinax Cat 8 Parallel MMF (MPO12) Duplex Duplex Duplex Module type Backplane QSFP+ RJ45 CFP, QSFP+ CFP CFP, QSFP+ QSFP+ Market availability None known CFP 2010 QSFP Note: there are a few 40 GE MSA standards that are not shown. 47 Nokia 2016

1 st generation 100 GE limitations Fundamental 1st generation technology constraints limit higher 100

uses 10 x 10 Gb/s lanes (CAUI-10) Optical signaling on the media - 100GBASE-SR10: 10 x 10 Gb/s

complexity and power consumption 2nd generation modules based on 4 x 25 Gb/s electrical signaling are

48 1 st generation 100 GE limitations Fundamental 1st generation technology constraints limit higher 100 GE density and lower cost Electrical signaling to the CFP Gb/s Attachment Unit Interface (CAUI) uses 10 x 10 Gb/s lanes (CAUI-10) Optical signaling on the media - 100GBASE-SR10: 10 x 10 Gb/s parallel - 10x10 MSA: 10 x 10 Gb/s λs - 100GBASE-LR4 and 100GBASE-ER4: 4 x 25 Gb/s λs CFP module size, complexity and power consumption 2nd generation modules based on 4 x 25 Gb/s electrical signaling are available now Image courtesy of Finisar. 48 Nokia cm GE Electrical signaling Optical signaling

49 1 st generation vs 2 nd generation 100 GE signaling 1 st generation 100 GE 10 x 10 Gb/s electrical and 4 x 25 Gb/s optical 2 nd generation 100 GE 4 x 25 Gb/s electrical and optical Diagram source: 49 Nokia Gb/s electrical signaling and 10:4 gearbox adds complexity, cost, space, and consumes power

50 CFP comparison Images courtesy of the CFP MSA. 50 Nokia 2016

100 GBE - WHAT'S NEW AND WHAT'S NEXT

Here is more rough stuff on the ALTO ALOHA network. Memo sent by Bob Metcalfe on May 22, 1973. 100 GBE - WHAT'S NEW AND WHAT'S NEXT Greg Hankins UKNOF25 UKNOF25 2013/04/18 40 Gigabit