10GbE and Beyond. Jeff Cain. Joint Techs Council February 13, 2007

Transcription

1 10GbE and Beyond Jeff Cain Joint Techs Council February 13, 2007 FOE_2007_Oberstar 2006 Cisco Systems, Inc. All rights reserved. 1

2 Outline 10GbE Current state of 10GbE Future Network drivers, Form Factors, Projections, Challenges Beyond 10GbE - 100GbE Market Drivers Why 100GbE? 100GbE Standard 100GbE Technologies Summary Wrap Up & Questions 2

3 10GbE FOE_2007_Oberstar 2006 Cisco Systems, Inc. All rights reserved. 3

4 A quick look back 10GbE costs are still high against standard benchmark of 3-4x for 10x Rate increase Form factor choice is dictated by system constraints: Power/Thermal Electrical Interface Size Xenpak & X2 will represent ~ 90%+ of Cisco volume for next two years Xenpaks shipping since 2003 X2s started in late 04 4

5 Transitions Distance Increased Choice of Flavors IEEE PMDS CX4 15m 30m T SR T 75m 100m SX LX LX4 LR LRM 220m 300m 10 km Distance ER 40 km ZX ZR 80 km WDM WDM km Time Standard/ Product Available 5

6 Transceiver PMD Mix GbE was ~ 5% of 2006 (GbE + 10GbE) Fiber ports 13xx lasers increase PMDs (LX4, LR) share over 850 nm sources (SR) Cisco 10GbE dominant form factor was still Xenpak 6

7 New 10GbE Standards 10GBaseLRM 220m FDDI grade MMF Single laser solution comparable to LX4 First standard to utilize Equalizer Dispersion Compensation Make silicon do the work for the optics This won t be the last time EDC is used to make up channel issues Modules starting to hit marketplace now 7

8 New 10GbE Standards 10GBaseT 10GbE over unshielded twisted pair Cat6A cable is target for 100m operation Phys now becoming reality NIC adoption this calendar year Thermal challenges abound with first generation 8

9 Network Architecture 2005 & Today Enterprise & Data Center Access Desktop Wiring Closet Media Cat5 copper 100m /100 10/100/ /100/1000 MMF (FDDI/ OM1) 220m to 550m GbE 10GbE Distribution MMF/ SMF 550m/ 10km GbE 10GbE 10GbE Core SMF Distribution MMF/ SMF 550m/ 10km MMF OM2 100m to 300m 10GbE GbE 10GbE 10GbE Access Data Center Server MMF/ Copper 100m 10/100/ /100/ GbE 9

10 10G Transceiver Drivers & Influence Bandwidth Demand Datacenter Videos need to be stored somewhere Enterprise web applications drive increased server interconnect and storage interconnect using Ethernet Wiring Closet (Campus Buildings) Bandwidth design goals for distribution/core over-subscription ratio Product Availability - Most switch products now using/transitioning to 10G uplinks vs. 1G Per Port Costs 10G Price premium is an inhibitor today This is particularly true for MMF & SMF links up to 10Km 10

11 Server 10GbE Connection Projections (Based on Blade Server Adoption) Data Center Driving Demand for Higher Speed Ethernet 3 # Servers (M) ~3M ~3M blade servers = ~500K blade 2 X 10GbE uplinks per per chassis = 1M 1M 10GbE ports Year Source: IDC 11

12 10GbE PMD Challenges for Optics - Today Lower COST Lower SIZE / Higher PORT DENSITY Higher THERMAL / Lower POWER CONSUMPTION 12

13 10GbE Optical Transceiver Evolution Transceiver Power (W) 1 4 Year XFP X2 SFP+ Xenpak Max 10 GbE ports per line card Higher Port Counts (4 48) & Lower Power (6W 1W) 13

14 SFP+ Board/ Host SERDES/CDR + EDC SFI 1 x 8.5G 1x x x Optic Module LD + TOSA TIA + ROSA SFP+ MOTIVATION Maximize port density per blade, minimize power, optimize cost PERFORMANCE Smallest 10GbE form factor Power Class 1: < 1 W ( multi-cage designs) Class 2 : 1.5W Rx interface: Linear Rx interface: Supports 10GBASE-LRM products with host-based EDC Limiting Rx interface: supports all other PMD types COST Most electronics out of optics to host board IC IC integration lowers costs; IC on host board enables quad/octal Serdes Higher Density, Lower Power, Lower Cost 14

15 Cisco 10 GbE Mix Forecast 2007 & Beyond 1GbE-10GbE Transceivers Unit Mix 100% 80% 60% 40% 10GbE 1GbE 20% 0% GbE MMF Transceiver Cost Curves Relative cost to 1GbE SX

16 10GbE Summary Market growth driven by use of 10GbE in datacenters 13xx nm PMDs (10G-LRM, LX4, LR) increasingly significant in mix 850 nm PMDs (10G-SR) still significant in mix distance good enough for datacenter Form Factors: SFP+: Routers XFP Switching X2, SFP+ Density, Power, Cost Enable significant 10GbE market adoption by lowering the cost of links <10Km Project ~50% share of Cisco 10GbE ports in

17 Beyond 10GbE to 100GbE FOE_2007_Oberstar 2006 Cisco Systems, Inc. All rights reserved. 17

18 Driving Bandwidth Requirements Consumers Increasing penetration to home Increasing bandwidth needs Content Increasing bandwidth requirements Networking Pulling it all together Service Providers Other Data Centers High Performance Computing 18

19 Network Impact 19

20 Applications Consumer & Business Driven All Devices Networked PCs, Desktop Phones, Cell Phones, Radios, TVs, Security Cameras, Vehicles, Sensors, Actuators, Doors, portable gadgets Everything on Ethernet Wired, Wireless.. Everything over IP Data, voice, video, wireless 20

21 Content Providers Personalized Video content is a killer application Google, Yahoo!, MSN, & YouTube deliver personalized content via high BW Yahoo! (Source: Adam Bechtel, Yahoo!, The Need for 100GE, Designcon 2006 Management Forum Panel) Ethernet everywhere 10 GbE used in data center, metro and WAN Q106: already using 6 x 10 GbE and 4 x 10 GbE LAG Yahoo! Asia Pacific (source: Chris Choi, Yahoo! Asia Pacific) 40+ Gbps aggregate bandwidth need to support Major League Baseball streaming in Asia Pacific YouTube (Source: Colin Corbett, YouTube, Peering of Video, NANOG 37) Personal video site, >50 million videos viewed per day ( >20 Gbps traffic) ~20% monthly traffic growth, all unicast traffic Initiatives for online DVD rentals by NetFlix and Amazon 21

22 Market Drivers - Increased Bandwidth Content Providers Video on demand Consumer & Broadband Access Service Providers & IXCs Data Center High Performance Computing 22

23 High Performance Computing Average Gigaflops per second (top 500, '93 - '05) K to 110K GF/s- HSSG Deployment 10GE Std GE Std 7000 GF/s- 10GE Deployment 585 GF/s- GE Deployment Year Source - Top500.org Historical Observations 4 years between standard and deployment GE captures 50% of market in 3 years ( ) 10GE enters top500 in 2006 (included in Gigabit Ethernet # s) 12x increase in Avg Gigaflops: 10x increase in Ethernet Interconnect Forecast 10x increase needed at 84,000 Gigaflops (2010) 23

24 Use of parallel nx10gbps links Effectiveness of load distribution depends on Hash algorithm Number of flows/size of flows Diversity of hash input (variability in SRC/DEST IP address) Flow-based hash mechanism CANNOT guarantee equal distribution of load or QOS Un-equal load distribution can results in under-utilization of available capacity May potentially cause artificial congestion and packet loss 10x10Gbps is not the same as 1x100Gbps from a real throughput perspective How many parallel 10Gbps links needed to match usable BW of a 100Gbps link? Depends on traffic characteristics 24

25 Why 100GbE now? The need for a higher speed bandwidth link beyond 10Gb is already emerging today Standards are required to gain broad market acceptance for any high speed Ethernet beyond 10GbE Standards are time consuming Realistically, estimate new standard completion by late 09 Need a solution consistent with the technology avail in the timeframe when the standard is completed 100Gb is well aligned to future ASIC switching speed capability Given limited industry resources, it is better to focus on 100GbE for 09 release 25

26 Possible Timeline for 100GbE Standard TASK FORCE Formed WG Ballot HSSG Formed PAR Approved TF Review LMSC Ballot STD!!! J U L S E P N O V J A N M A R M A Y J U L S E P N O V J A N M A R M A Y J U L S E P N O V J A N M A R M A Y J U L S E P N O V J A N M A R M A Y J U L S E P N O V Objectives Drafted D 1 D 2 D 3 D 4 D 5 PAR Drafted Last New Proposal Last Feature Last Technical Change 26

27 Standard Considerations Define small number of PMDs Avoid the proliferation as with 10GbE Optimize Standard for Technology Advances / Time to Market / Cost 27

28 Progress! Six objectives were adopted by the Study Group: Support full-duplex operation only Preserve the 802.3/Ethernet frame format at the MAC Client service interface Preserve current Std minimum and maximum FrameSize Support a speed of 100 Gb/s at the MAC/PLS service interface Support at least 10km on SMF Support at least 100 meters on OM3 MMF This set of objectives could be considered enough to move forward with a project request 28

29 Defining interfaces Standards goal is to leverage existing technologies while being flexible enough to easily incorporate future emerging technologies MAC PCS PMA PMD 10G 20G 25G 50G 100G? Need a universal PCS and a single common interface to optics module 29

30 Observations about interface rates Data Rate (Gbps) Network and Router I/F Speeds (Gbit/s) 140M TDM Transport 565M OC48 POS OC3 OC192 OC192 OC12 OC48 1GE Ethernet Year of Deployment OC768 10GE OC Going back over last 25 years, SONET TDM led introduction of high speed BW technology into network Over time, the gap between introduction of high speed BW into the SONET TDM and Ethernet network has shrunk Beyond 10G, Ethernet data interfaces are likely to be the push for higher BW optical technology 30

31 Study Group PMD objectives Support at least 10km on SMF single mode fiber support considered a requirement actual distance will be a trade-off between reach, cost, complexity Support at least 100 meters on OM3 MMF parallel optical technology will likely be the lowest cost implementation belief that the cost benefits will outweigh the operational challenges of the ribbon fiber 31

32 10GbE GbE Summary Bandwidth bottlenecks are real and increasing Standards are required to gain broad market acceptance for any high speed Ethernet beyond 10GbE Today, SONET OC-768 is the only standard beyond 10Gb 100Gb is better aligned to future ASIC switching speed capability and network bandwidth needs than lower rates Given pace of standards and the development resources needed for new PMDs it is best to focus on 100GbE for 09 release 32

33 Thank you! Questions? 33