DATA CENTER OPTICS ECOC MARKET FOCUS, SEPTEMBER 24, 2013

Transcription

1 DATA CENTER OPTICS ECOC MARKET FOCUS, SEPTEMBER 24, 2013 Lisa Huff, CDCP Principal Analyst, Discerning Analytics Chief Technology Analyst, DataCenterStocks.com

2 AGENDA Data Center Networking Trends Data Center Optics Fiber versus Copper in the Data Center Vertical Market Summary 2

3 DATA CENTER NETWORKING TRENDS Higher speed networks Server trends Data center power and cooling 3

4 DATA CENTERS DEFINED Facility Purpose Typical Data Rates TR Cross-Connections & Switches 100 Mbps 1Gbps Typical # of Racks Typical Business < 5 All Small DC (Reduced) Medium DC (Basic) Servers, CPU, UPS, Switches Servers, CPU, UPS, POE, Local SAN, Switches 1 Gbps 5-20 Small Financial Hospitals 1-10 Gbps Banks Drug Co. Universities Large DC (Distributed) Servers, CPU, UPS, POE, Local & Remote SANs, Switches, Asset Management 1-40Gbps > 100 Large Financial Retail Insurance Large Research Labs Co-location Facilities 4

5 LAN AND DATA CENTER GROWTH DRIVERS Government Initiatives Data Policy Directive (DPD) European Union (E.U.) Basel Capital Accord (2006) HIPAA (U.S.), PIPEDA (Canada) Sarbanes-Oxley Act (US) HITECH (US) Broadband Initiative (US) Redundancy to protect against catastrophic loss Web 2.0/Internet Data Centers Cloud Computing Both corporate and consumer Enterprise Storage Growth Million Terabytes Source: DataCenterStocks.com 5

6 DATA CENTER NETWORK ARCHITECTURE 10G 40G 100G Multiple 1G 10G 40G 100M/1G 1G 10G 6

7 TOP OF RACK (TOR) SWITCHING Source: Extreme Networks 7

8 DATA CENTER NETWORK TRENDS END OF ROW (EOR) SWITCHING Source: Brad Hedlund, Cisco Systems 8

9 DATA CENTER NETWORK TRENDS ETHERNET NETWORK ARCHITECTURE Top of Rack (ToR) End of Row (EoR) 9

10 DATA CENTER NETWORK TRENDS LAN DATA RATE PROGRESSION 8,000 7,000 Ethernet Optical Modules and AOCs Revenue 6,000 Millions of Dollars 5,000 4,000 3,000 2,000 1, Year 10G 40G 100G Source: Discerning Analytics, LLC 10

11 DATA CENTER CABLING TRENDS Higher speed networks Server trends Data center power and cooling 11

12 X86 SERVER FORECAST BY DATA RATE Milions of Ports Year 100M 1G 10G 40G 100G Source: DataCenterStocks.com 12

13 SERVER TRENDS - MARKET PENETRATION BY DATA RATE Ethernet Speed (Mbps) Time to Reach (years) Million Port Sale (after Ratification) % Market Penetration , , , ,

14 SERVER TRENDS FORM FACTORS BLADE SERVERS Servers on a card, containing processors, memory, integrated network controllers, an optional Fibre Channel host bus adapter (HBA) and other input/output (IO) ports. Rack Server Blade Server 14

15 SERVER TRENDS Percentage of Servers 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Year Pedestal Rack Blade 15

16 SERVERS VIRTUALIZATION IMPACT ON THE NETWORK Virtualization is the creation of a virtual (rather than actual) version of something, such as an operating system, a server, a storage device or network resources Drastically reduces the number of servers needed in the data center Impacts the amount of cabling needed Impacts the network architecture need faster server-to-server connections Source: CA Associates,

17 SERVER TRENDS BANDWIDTH DENSITY (Gbps/sqft or /rack) Server virtualization Pushing network connection data rates up Changing network traffic patterns from hierarchal to horizontal Server Form Factor Blades Source: HP 17

18 DATA CENTER CABLING TRENDS Higher speed networks Server trends Data center power and cooling 18

19 DATA CENTER TRENDS - POWER CONSUMPTION Source: Oracle/Sun Microsystems 19

20 DATA CENTER HEAT LOAD TRENDS kw/ft 2 kw/rack ASHRAE, Datacom Equipment Power Trends and Cooling Applications, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc., 20

21 MODULAR DESIGNS BASED ON COOLING REQUIREMENTS Hot/Cold aisle containment POD with APC in-row cooling Hot/Cold aisle containment POD with Liebert XDV overhead spot cooling Source: Oracle/Sun Microsystems 21

22 DATA CENTER TRENDS COLD AISLE CONTAINMENT 22

23 COOLING TRENDS MOVING CABLING TO ABOVE CABINETS 23

24 COOLING TRENDS STRUCTURED CABLING 24

25 AGENDA Data Center Networking Trends Data Center Optics Fiber versus Copper in the Data Center Vertical Market Summary 25

26 GIGABIT ETHERNET OPTICAL VARIANTS Ethernet Variant 1000-BASE- Min. Reach (m) Form Factors Media Wavelength SX 300 GBIC, SFP LOMF 850nm z Standard IEEE LX 5000 GBIC, SFP SMF 1310nm ZX 70,000 GBIC, SFP SMF 1550nm Defacto GBIC SFP Source: Avago Technologies 26

27 10-GIGABIT ETHERNET OPTICAL VARIANTS Ethernet Variant 10GBASE- Data Rate (Gb/s) Min. Reach (m) Form Factors SR XENPAK, XFP, X2, SFP+ LR 10, pin, XENPAK, XFP, X2, SFP+ Media Wavelength LOMF 850nm SFM 1310nm LX4 4x XENPAK, X2 FDDI-grade MMF 850nm ER 10 40,000 XENPAK, XFP, X2 SMF 1550nm ZR 80,000 XFP, SFP+ SMF 1550nm LRM 220 XFP, SFP+ FDDI-grade MMF 1310nm Standard IEEE ae-2002/-2005 aq-2006/ pin XENPAK X2 XFP SFP+ 27 Source: Finisar

28 40-GIGABIT ETHERNET OPTICAL VARIANTS Ethernet Variant 40GBASE- Data rate (Gb/s) Min. Reach (m) Form Factors Media Wavelength Standard IEEE SR4 4x10 100/125 QSFP+ LOMF 850nm ba LR4 4x10 10,000 QSFP+, CFP SMF 1310nm FR 40 2,000 CFP SMF 1310/1550nm bg QSFP+ CFP Source: JDSU Source: Reflex Photonics 28

29 100 GIGABIT ETHERNET OPTICAL VARIANTS Ethernet Variant 100GBASE- Data rate (Gb/s) Min. Reach (m) Form Factors Media Wavelength Standard IEEE SR10 10x10 100/150 CXP, CFP LOMF 850nm ba LR4 4x25 10,000 CFP, CFP2, CFP4, QSFP28 SMF 1310nm ER4 4x25 40,000 CFP SMF 1310nm LR10 10x10 2,000 CFP SMF 1550nm Not supported CXP CFP CPAK CFP2 Source: Ethernet Alliance Source: Avago Technologies Source: Oclaro Source: Cisco Systems Source: Avago Technologies 29

30 DATA CENTER ETHERNET OPTICAL DEVELOPMENT STANDARDS IEEE P802.3bm 40 Gb/s and 100Gb/s Fiber Optic Task Force 4x10G 40km SMF development 4x25G 500m SMF development 4x25G 20m and 100m LOMF development Study Groups 400 Gb/s Ethernet established in May 2013 Working on project request 30

31 DATA CENTER ETHERNET OPTICAL ROADMAP Source: Ethernet Alliance, OFC

32 FIBRE CHANNEL IN THE DATA CENTER 32

33 INFINIBAND AND HIGH PERFORMANCE COMPUTING Today s Supercomputer Source: Sun/Oracle 33

34 INFINIBAND ROADMAP Switched fabric communications link primarily used in high-performance computing. Aggregate Data Rates (Gbps) SDR DDR QDR FDR EDR 1X 2.5 (1x2.5) X 10 (4x2.5) X 20 (8x2.5) X 30 (12x2.5) Supercomputer Juropa (Jülich) #10 worldwide Source: FCI 34

35 PRE-TERMINATED FIBER ASSEMBLIES EXAMPLE Source: Brocade Source: Corning 35

36 AGENDA Data Center Networking Trends Data Center Optics Fiber versus Copper in the Data Center Vertical Market Summary 36

37 2012 DATA CENTER CABLING MIX Data Center Media Mix SAN Media Mix 37% 6% 63% 94% Copper Fiber Copper Fiber Source: DataCenterStocks.com 2012 Research 37

38 2017 DATA CENTER CABLING MIX Data Center Media Mix SAN Media Mix 40% 2% 60% 98% Copper Fiber Copper Fiber Source: DataCenterStocks.com 2012 Research 38

39 DATA CENTER LAYOUT Source: 42u.com 39

40 AIRFLOW: HOT AISLE/COLD AISLE CEILING CABLE TRAYS or LADDER RACK 3-4 CABLE TRAYS or LADDER RACK CABLE TRAYS or LADDER RACK Front Cabinet or Rack Cabinet or Rack Cabinet or Rack Rear Rear RAISED FLOOR CABLE TRAYS POWER CABLES PERFED TILES CABLE TRAYS POWER CABLES PERFED TILES 40

41 DATA CENTER COPPER CABLING UNDER RAISED FLOOR 41

42 DATA CENTER COPPER CABLING ABOVE CABINETS 42

43 AIRFLOW: HOT AISLE/COLD AISLE Wire Basket Air flow channel Cat 6 UTP solution 242 cables at 50% Fill Ratio channel Fiber solution 50/125 LOMF MDP 10 cables, Fill Ratio < 1 % *channels include 2 fibers Source: Berk-Tek, A Nexans Company 43

44 DATA CENTER CABLING FIBER VERSUS COPPER COST COMPARISON SUMMARY Connection Type Port Cost Patch Cords Horizontal Cable and Connectivity Installation Labor per Channel Total per Channel Cost Fiber SM FT $7, $ $ $ $ 7, Fiber LOMF FT $ 3, $ $ $ $ 3, Fiber LOMF PT $ 3, $ $ $ 6.25 $ 3, Copper UTP $ 1, $ $ $ $ 2, Copper FTP $ 1, $ $ $ $ 2, Model details: Distribution prices based on 16-port 10-Gig switch blades for Cisco Catalyst 6509 WS-X T-3CXL WS-X G-3CXL w/x2 10GBASE-SR modules Patch cords are 2m CAT6A, OM3 LOMF or SMF Labor rate of $60/hour Port cost computed by total cost of switch blade and transceivers/number of ports 44

45 FIBER VERSUS COPPER COST COMPARISON WITH POWER CONSUMPTION 48-port GigE Blade Blade Watts # Blades/ Chassis WS-X6748-SFP (Fiber) port 10GigE Blade Blade Watts # Blades/ Chassis Total Watts W/port WS-X G- 3CXL WS-X6748-GE- TX (Copper 1) WS-X T- 3CXL WS-X6548-GE- TX (Copper 2) WS-X T- 3CXL Optical solution saves 1.4 kw/rack 1W power savings yields 2.84W total Total possible savings = 4.kW/rack 45

46 FIBER VERSUS COPPER THE CASCADE EFFECT Source: DVL 46

47 FIBER VERSUS COPPER ENERGY SAVINGS ECONOMICS SWITCH PORTS Save about $ /rack/year with optical solution (at $0.12/kWh) ROI would take about 32 years To get a 10 year payback Electricity would have to cost $0.77/kWh Installed optical port needs to be < $ more than an installed copper port Why not analyze a server rack? For further study Quantify airflow inefficiency with copper cabling 47

48 OTHER COMPELLING REASONS TO INSTALL FIBER Latency nanoseconds versus microseconds Lower power consumption at higher data rates Smaller will allow better airflow Higher density Longer reach Less weight in cable trays Not susceptible to EMI 48

49 DATA CENTER FIBER CABLING TYPES PROGRESSION 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% OM2 OM3 OM4 OS2 49

50 AGENDA Data Center Networking Trends Data Center Optics Fiber versus Copper in the Data Center Vertical Market Summary 50

51 VERTICAL MARKET SUMMARY Typical Business Size (sqft) # of Racks Typical Data Rates (Gbps) Networking Technologies Small Financial Manufacturing K12 Schools <1, Ethernet, iscsi Banks Drug Co. Small Hospitals Higher Education 1,000-10, Ethernet, iscsi, FC Large Financial Retail Insurance Large Research Labs Medium Healthcare Web 2.0/IDCs ISPs Co-location >10,000 > Ethernet, iscsi, FC, InfiniBand 51

52 VERTICAL MARKET BREAKDOWN DATA CENTER OPTICS 25% 62% 13% Co-location Internet Enterprise Source: Discerning Analytics, LLC 2012 Research 52

53 SUMMARY Gigabit Ethernet still the workhorse of the typical data center. 10G Ethernet is the largest revenue generator application in the data center for the next five years SFP+ 10GBASE-SR OM3 and OM4 Fiber SFP+ DAC 10GBASE-CR RJ45 10GBASE-T CAT6A/7 Twisted Pair Ethernet networks in the data center will continue to evolve into flatter topologies Higher data-rates will be pushed closer to the server 53

54 APPENDIX Case Study Contributors and References

55 DATA CENTER CASE STUDY WELLSPAN HEALTH Facilities Data centers support 2 hospitals, 10 health centers, 14 imaging centers, 16 laboratory locations, 55 physician offices State-of-the-art 6,000 sq ft. data center in York, PA built in MW, 1000 A Cooled by CRACs Continuously updated 5,100 sq. ft. additional space State-of-the-art 1,400 sq ft. mirror data center in York, PA built in MW, 800 A Cooled by CRACs Continuously updated 55

56 WELLSPAN HEALTH Network Three-tiered Ethernet network: Access, Distribution/Aggregation, Core Access Gigabit Distribution Multiple Gigabit Core Multiple 10G, looking at 40G Topology Main Gigabit-rated cross-connect Transitioning to Top-of-Rack switching using Cisco Nexus 2000 Cisco Nexus 5000 in Distribution/Aggregation and Nexus 7000 in Core Multi-wavelength CWDM/DWDM between buildings Separate FC SAN Looking at FCoE still too expensive 56

57 WELLSPAN HEALTH Cabling Using TIA-942 as much as possible Copper in Access Category 6 Fiber in Distribution/Aggregation Pre-terminated LOMF OM2 and OM3 MPO assemblies with cassettes Fiber in Core SMF between data centers and buildings LOMF and SMF in SAN 57

58 ACKNOWLEDGEMENTS 58

59 RESOURCES STANDARDS ANSI/BICSI , Data Center Design and Implementation Best Practices FC-PI, Fibre Channel Physical Layer Standards IBTA InfiniBand Architecture Specification IEEE 802.3, Standard for Information Technology TIA-942, Telecommunications Infrastructure Standard for Data Centers TIA-568, Commercial Building Cabling for Telecommunications TIA-1179, Healthcare Facility Telecommunications Infrastructure Standard 59

60 RESOURCES ARTICLES, WHITE PAPERS, BLOGS, MARKET RESEARCH Huff, L. (2013), Ethernet Networks Progress to 400G, Connector Supplier Debiec, T. & Huff, L.(2013)Structured Cabling Technology and Market, Connector Supplier Huff, L. (2012) Multi-Gigabit Datacom Network Connectivity, Bishop & Associates, Inc. Huff, L. (2011) The Battle for the Converged Data Center Network, Data Center Knowledge.com Gross, D., & Huff, L. (2011). DataCenterStocks.com Blog. Huff, L. (2011). Optical Components Blog. Huff, L. (2011) Data Center Cabling Copper versus Fiber Webinar Ponemon Institute (2011), Calculating the Cost of Data Center Outages, Ponemon Institute and Emerson Network Power Hewlett Packard (2011) Building Virtualization-Optimized Data Center Networks, Gosai, Birendra (2011) Building the Next Generation Data Center, CA Technologies Web site 60

61 RESOURCES ARTICLES, WHITE PAPERS, BLOGS, MARKET RESEARCH Huff, L. (2010). Data Center Cabling Cost Analysis. DataCenterStocks.com. Huff, L. (2010). DAC Report. DataCenterStocks.com. Huff, L. (2010). 40/100 GigE Markets: 2010 and Beyond. Communications Industry Researchers. Huff, L. (2010). New Revenue Opportunities for Optical Interconnects: A Market and Technology Forecast. Communications Industry Researchers. Cisco (2009), Data Center Top-of-Rack Architecture Design, Cisco Cisco (2007), Cisco Data Center Infrastructure 2.5 Design Guide, Cisco APC (2005), Determining Total Cost of Ownership for Data Center and Network Room Infrastructure, TechRepublic.com 61

62 MULTI-SOURCE AGREEMENTS (MSAS) AND SFF 10G SFP+: SFF G QSFP+: SFF G CXP: InfiniBand Supplement CFP/CFP2/CFP4: CFP MSA 10x10: 10x10 MSA QSFP28: SFF G CDFP 62