40G BASE-T Cabling Infrastructure Allan Nielsen Standards Manager
40GBASE-T Transmission with bi-directional 4x10G nearby cable Switch 40GBASE-T signal spectrum up to 1.600 MHz Alien Crosstalk (AXT) Shield 10 Gbit/s NIC/LOM bi-directional magnetics 40 Gbit/s DSP 10 Gbit/s 10 Gbit/s DSP 40 Gbit/s 10 Gbit/s NEXT FEXT Return Loss
Market data and forecasts Allan Nielsen
1000000 Bandwidth Supply & Demand 40G Servers 100000 100 Gigabit Ethernet Bit Rate (Mbit/s) 10000 1000 Core Networking Doubling ~18 months 40 Gigabit Ethernet 10 Gigabit Ethernet Gigabit Ethernet 100 Server I/O Doubling ~24 months 1995 2000 2005 2010 2015 2020
1 10 100 1000 10000 100000 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 1 10 100 1000 10000 100000 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 CFI IEEE Std BASE-T Roadmap 10BASE-T 100BASE-T 1000BASE-T 10GBASE-T 100GBASE-T Server I/O Doubling ~24 months were these projects too early? CFI 40GBASE-T
x.86 Servers by Ethernet Port Speed 12 Based on IDC 2010 server forecast and hays_01_0407 ratios of Ethernet port speed 40G Servers 10 8 6 4 2 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 millions server units 100M 1G 10G 40G 100G
Server-to-Switch Data Centre Links based 30m Reach for 40GBE & 7m Reach for 100GBE Total Server-to-Switch Links (millions) 25 20 15 10 5 Based on Flatman contribution to Aug 2009 Penn State Workshop: The Market Opportunity for Higher Speed Ethernet over Copper (updated with IDC 2010 Server Forecast) 100GBE 7-100m Fibre 40GBE 30-100m Fibre 40GBE 7-30m Copper 100GBE 0-7m Copper 40GBE 0-7m Copper 10GBE 15-100mCopper 10GBE 0-15m Copper 1GBE 0-100m Copper 0 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020
Data Centre Link Shares based on Cumulative Volumes 2010 to 2020 1000BASE-T 14% Backplane 27% 10GBASE-CX4 16% 10GBASE-T 16% 40GBASE-T (30m reach) 11% 40/100G Twinax 5% Total Fibre 11%
Data Centre Cabled Link Shares based on Cumulative Values 2010 to 2020 (assuming fibre links cost 3x average copper link) 10GBASE-CX4 17% 1000BASE-T 14% Total Fibre 35% 10GBASE-T 17% 40GBASE-T (30m reach) 12% 40/100G Twinax 5%
BSRIA Global Data Center November 2011 38 %
Market Drivers 2015 Forecast of Data Center Speeds and Storage
Data Center Speeds Global traffic Source: CISCO Visual Networking Index, Global IP Traffic Forecast 2010
Data Center Speeds So what is a PetaBit: Mega = 1.000.000 Bit Giga = 1.000.000.000 Bit Tera = 1.000.000.000.000 Bit Peta = 1.000.000.000.000.000 Bit Exa = 1.000.000.000.000.000.000 Bit Zetta = 1.000.000.000.000.000.000.000 Bit Yotta = 1.000.000.000.000.000.000.000.000 Bit One Byte = 8 bit
Business share of global traffic Source: CISCO Visual Networking Index, Global IP Traffic Forecast 2010
Data Center Speeds Idle / Active profiles Average Busy-hour BW & Average Consumed BW tend to be used quite extensively for Traffic Engineering calculations (determining how much Bandwidth Capacity is required to satisfy a given Service Group (pool) of subscribers Source: ARRIS
Data Center Speeds - Forecast Segments 2011 2015 Servers: 10G 40G internet 40G 200G Storage 2 VServers Users: 40G 100G internet 100G 1.000G Storage 1G internet 1G 10G Storage Expected growth rates: Server to internet 40% per year Server to storage 50% per year
Data Center Storage More storage data is created every two years than all previous years combined Virtualization causes the need for networked storage of all varieties (SAN and NAS) New applications and devices are driving more data access and higher bandwidths 20% of storage is touched by Clouds, 10% is stored in Clouds 2015 Global Storage Forecast Source: IDC's Digital Universe Study, sponsored by EMC, June 2011
Global iphone sales Total of 218.245.000 iphones sold globally since its introduction Each iphone are offered a 5GByte storage in icloud
Storage need for icloud With 218.245.000 sold iphones being offered a free 5GB Cloud, Apple needs to establish a storage of at least: 218.245.000 X 5 GB = 1.091.225.000 GB = 1 ExaByte Storage This is ONLY for one application for one product from one manufacturer of smart phones.
Data transfer times in seconds Size of Data Latency of Latency of Latency of Latency of Latency of Latency of to 1GbE 10GbE 40GbE 100GbE 400GbE 1TbE Exchange 1 Gigabyte 8 0.8 0.2 0.08 0.02 0.008 10GB 80 8 2 0.8 0.2 0.08 100GB 800 80 20 8 2 0.8 1 Petabyte 8000 800 200 80 20 8 10 PB 80000 8000 2000 800 200 80 100 PB 800000 80000 20000 8000 2000 800 1 ExaByte 8000000 800000 200000 80000 20000 8000
icloud transfer times 1 ExaByte By 1G Ethernet = 1 X 8.000.000 Seconds = 8.000.000 = ~93 Days By 10G Ethernet = 1 X 800.000 Seconds = 800.000 = ~9,3 Days By 40G Ethernet = 1 X 200.000 Seconds = 200.000 = ~56 Hours By 100G Ethernet = 1 X 80.000 Seconds = 80.000 = ~22 Hours By 1T Ethernet = 1 X 8.000 Seconds = 8.000 = ~2,2 Hours Needless to say NAS and SAN s needs higher speeds Mirroring of storage more than doubles the need
Data Center Storage - Forecast Source: IDC Worldwide Enterprise Storage Systems 2011-2015
Standards progress Allan Nielsen
IEEE 802.3 A Call For Interest (CFI) planning group have had several meetings, to prepare a CFI for 40GBASE-T. Target date for CFI is July 2012 Supported by uptake in shipments of 10G BASE-T cards Supported by arrival of 10G LAN on Motherboard (LOM) Supported by Switching manufacturers Supported by Cabling manufacturers Technical feasibility checked Economical feasibility checked
ISO/IEC JTC1 SC25 Initiative for creating the next generation of cabling: 1st meeting held in Madrid, February 2012: Two proposals have been made for Next generation cabling Category 6B RJ45 connectors and F/UTP cable 1.600 MHz or 2.000 MHz maximum frequency 50 metre Channel 2 Connector Channel Shielded Only (Alien Crosstalk and Coupling Attenuation) Category 7B Cat 7A connectors and S/FTP cable 2.000 MHz maximum frequency As above but with improved performance on NEXT, FEXT.
CENELEC TC215 WG1 CENELEC have decided to harmonize with ISO/IEC 11801, when the international group have finalized their work.
ANSI TIA/EIA TR.42 TR.42 decided at their meeting held in West Palm Beach, FL in January 2012, that they would develop a next generation cabling under TIA568C.2: Category 8 Connectors and F/UTP cable 40 metre Permanent Link 50 metre Channel Maximum frequency 2.000 MHz Maximum 2 connectors channel Improved performance on TCL and ELTCL.
Anticipated timeline - 40G BASE-T SP-3-4426-AD1 draft 0.1 Call For Interest Technical stable Document Study Group End Industry Ballot Document Technical stable D2.0 ISO/IEC 11801 ed. 3.1? Technical stable Document? FDIS Document 2012 2013 2014 2015
Major industry challenges Choice of interface Laboratory measurement standard 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 7 8 6 I 3 I 4 I 5 I 6 3 4 5 NOTE: the pin designations 3 I, 4 I, 5 I and 6 I for Category 7 and 7 A correspond to 3, 4, 5, and 6 for Category 5, 6 and 6 A NOTE: the pin designations correspond to those of IEC 60603-7 series interfaces Choice of cable type Field testers