Global Networks and Data Centers Briefing. May 27, 2011

Size: px

Start display at page:

Download "Global Networks and Data Centers Briefing. May 27, 2011"

Mae Gordon
5 years ago
Views:

1 Global Networks and Data Centers Briefing May 27, 2011

Global Disaster Recovery Strategy Short to Mid-Term: Leverage Current Assets Long-Term: DR + SaaS growth Target State T ( ~ FY 13 ) Netherlands Metro-based DC Pair (Tier-III) Single-Instance Order

2 Global Disaster Recovery Strategy Short to Mid-Term: Leverage Current Assets Long-Term: DR + SaaS growth Target State T ( ~ FY 13 ) Netherlands Metro-based DC Pair (Tier-III) Single-Instance Order Management (OM/AR) Cisco International Mountain View, CA Early Adopter DC Production New Product/Service Introduction AsiaPAC TBD Single DC (Tier-III) + land Continental hub for SaaS, Unified Communications and software development E E B 2 A sc 1 A Richardson & Allen (TX) Metro-based 40 ms rtt DC Pair (Tier-III) + Carrollton Co-Lo Global hub for business applications Continental hub for SaaS and communications B 2 A sc BC / DR Plan 1 A 1 x Type - A ( Tier - III ) B Type - B ( Tier - II ) Early Adopter DC B (~Uptime Tier-III) B B B B (~Uptime Tier-II) 2 A sc 2 x Type - A at Synchronous Capable Distance B B 1 A B Distributed standalone DCs (Tier-II) B Latency-sensitive software development at lower availability

Technology Leadership Business Strategy Cisco Green Initiative New

Acceleration Automation Virtualization Accelerated Adoption Shared

infrastructure Reduced Time to Capability, TCO Virtualized, Resilient,

3 Technology Leadership Business Strategy Cisco Green Initiative New Opportunities Adoption Curve Management Next-Gen DC Networking Acceleration Automation Virtualization Accelerated Adoption Shared (leveraged) Infrastructure Aligned Architecture application, business, infrastructure Reduced Time to Capability, TCO Virtualized, Resilient, Green Data Center Accelerated, Video Enabled Global Network Cisco on Cisco Global Data Center

4 Current state: traditional DR (San Jose + RTP) SJC RTP (DR) RPO = Recovery Point Objective: Maximum amount of data loss the business can tolerate (time) RTO = Recovery Time Objective: Maximum downtime the business can tolerate (time) Target state: Metro-based Virtual DC (Texas/Richadson Metro + RTP/DR) Operational continuity DC1 (RCDN9) DC2 RTP (DR) NEW ADL = Acceptable Data Loss (like RPO, but only for incidents limited to single facility) ART = Acceptable Recovery Time (like RTO, but only for incidents limited to single facility) Disaster Recovery DC1 (RCDN9) DC2 RTP (DR) RPO = (same as traditional) RTO = (same as traditional)

5 Richardson Data Center

6 Richardson Campus Site Map GDCP Strategy Transformation Growth Resiliency Enablers Intelligent Network Infrastructure New Technology New Services Footprint for Future

7 CD2 Data Hall 1- Station#6 UPS-A Distribution Gallery- Station#7 Build room- Station#5 Generator & fuel tank yard- Station#3 Cooling towers EF1, EF2 located 3 rd fl. CD1 TST- Station#4 Chiller room 7 Data Hall 4 UPS-B Distribution Data Hall 2 RCDN9 lobby- Station#1 Data Hall 3 Electrical control room- Station#2

8 8

9 Data Hall 1 Build room Generator & fuel tank yard Cooling towers TST Chiller room Data Hall 4 Data Hall 2 Data Hall 3 9 Electrical control room

10 Cooling Towers 1000 Ton Capacity (ea) Cooling Tower Pumps Chillers 1000 Ton Capacity (ea) Generators Fuel Tanks 48,000 gals Generators Main Switchboard B Main Switchboard A Paralleling Switchgear B Paralleling Switchgear A Meter/Switchgear

11 Batteries Switchgear A1 Switchgear A2 UPS A1 UPS A2 IT storage room 2 nd floor CUP IT/Facility control rm & office space 11 UPS Switchgear B1 B1 Switchgear B2 Batteries UPS B2

12 Utility - A Utility - B Generators MSB-A MSB-B Tie Breaker UPS-A UPS-B RPP-A A 40% max load Critical Load (Servers) B 40% max load RPP-B

13 13

14 14

15 15

16 16

17 17

18 18

19 TD New UCS - MVDC ONS - DWDM Nexus 7018 Nexus

20 ONS Gear Supports DWDM for MVDC Enables Resiliency for Data Replication Located in DH1 and DH2 for NPRD MVDC

21 Nexus 7018 Provide Scalability Supports new UCS Server Bays Supports new Teradata Supports MVDC

22 UCS clusters NPROD MVDC NPRD UCS PROD (1) Server Bay = 2 clusters or 80 blades

23 TD New Node Support Cisco Data Warehouse Installed in DH 1

24 24

25 25

26 26

27 27

28 28

29 Physical to Virtual Increased Density Improved Power Utilization Lower unit cost Transformation to Cloud (Unified Management)

with SSDs An even bigger step forward for IT capability Performance Energy

31 A tremendous step forward in technology New processor architecture New platform architecture New memory subsystem New I/O subsystem New options with SSDs An even bigger step forward for IT capability Performance Energy Efficiency Virtualization 2010 Cisco and/or its affiliates. All rights reserved. 31

Typical Memory Cisco UCS Memory Xeon 5500 Fixed number of DIMMs can be addressed by the CPU Each DIMM the CPU looks for is made of 4 standard DIMMs Xeon 5500 Typical System Either 12 DIMMs @ 1066MHz

32 Typical Memory Cisco UCS Memory Xeon 5500 Fixed number of DIMMs can be addressed by the CPU Each DIMM the CPU looks for is made of 4 standard DIMMs Xeon 5500 Typical System Either MHz Max 96GB (192 QR) Or MHz Max 144GB at lower performance Intel Xeon 5500 Series with UCS MHz Max 384GB per Blade at full performance Benefit 2x capacity Lower costs Standards DIMMs, CPUs, OS 2010 Cisco and/or its affiliates. All rights reserved. 32

34 UCS uses significantly fewer cables 40% Savings on Cable Infrastructure Traditional blade server, similarly configured to Cisco Unified Computing March 2009

35 Cisco IT s Case Study Summary Traditional Unified Fabric UCS DC efficiency 100% % 130% % 10,000 sq ft, 1 MW Cabling $2.7 million $1.6 million $1.6 million Physical Server Count per Footprint VM Count Density efficiencies ~40% Savings from cabling 12,000 to 28,000 VMs -- In the same size DC! Notes: Assumes pre-ucs average V2P ratio of 10 to 1 and post UCS average ratio of 20 to 1 due to the memory expansion technology. Unified Fabric efficiency gains result from power optimization. UCS efficiency gains result from additional power benefits of UCS. March 2009

37 Component Management Interfaces No. of Chassis Manager Interfaces No. of Chassis Interconnect Interfaces No. of External Switch Interfaces Total Management Interfaces Cisco (320 servers in 40 chassis across 6 racks) 0 HP (320 servers in 20 chassis across 6 racks) 20 (OA) 0 5 (VC Manager) 1 (UCS Manager) 4 (3 rd Party ToR Switch) 1 29 *External switches are required for both Cisco and HP in order for any blade server to communicate with any other blade server Cisco and/or its affiliates. All rights reserved. 37

38 Embedded Management Management abstracted to Fabric Interconnects Chassis have no state just hold blades All blades share same management domain Existing HP Mgmt Architectures California Fabric Manager Architecture VM Manager ilo Licenses Rapid Deployment Disjointed Management Insight Recovery Onboard Admin CMM Insight Dynamics Virtual Connect Enterprise Insight Control Servers Storage Access B 1 Network Embedded Device Management Embedded S/W Multiple Management Solutions Uncoordinated / Complex Control Plane Integration 2010 Cisco and/or its affiliates. All rights reserved. 38

40 VIRTUALIZE, VIRTUALIZE, VIRTUALIZE ESX Farms built using UCS clusters (2 Viffs per cluster) Farms built using 192GB as a standard Provisioning virtual hosts via SRT/CITEIS Migrating SAN to NAS for ESX farms (evaluation) Bare Metal automation - next Storage Provisioning/Virtualization

UCS 3 Rack Design Cisco 3845 for OOB management of fabric interconnects Initial design built out with UCS 6120 s. Place for easy upgrade to UCS 6140 s.

41 UCS 3 Rack Design Cisco 3845 for OOB management of fabric interconnects Initial design built out with UCS 6120 s. Place for easy upgrade to UCS 6140 s. Initial design supports 5 chassis per cluster, increasing to 10 chassis with 6140 s Each chassis provides 80GB/s throughput configured with 3m and 5m twin-ax. Fiber available for longer runs (legacy DC) Anticipated power consumption per chassis: 2.5kw

42 UCS Single Rack Design Initial design supports 5 chassis per cluster, increasing to 10 chassis with 6140 s Initial design built out with UCS 6120 s. Place for easy upgrade to UCS 6140 s. Cisco 3845 for OOB management of fabric interconnects Anticipated power consumption per chassis: 2.5kw Anticipated power consumption per UCS: 6kw to 10kw 2010 Cisco and/or its affiliates. All rights reserved. 42

Nexus Provide Scalability Nexus 7010 = 256 10G ports compared to Catalyst 6509 = 112 10G ports Nexus 7018 = 512 10G ports compared to Catalyst 6513 = 112 10G ports GREEN Catalyst 6500

43 Nexus Provide Scalability Nexus 7010 = G ports compared to Catalyst 6509 = G ports Nexus 7018 = G ports compared to Catalyst 6513 = G ports GREEN Catalyst port 10G line card: 28.6W per port Nexus 70xx Nexus port 10G line card: 23.4W per port Supports UCS Server Bays (PROD/NPRD/DMZ) Supports Data Warehouse Supports Application Migrations

45 We migrated 462 apps to UCS platform to date. By the end of Q2 FY11, the number will be around 487. The Key applications : Cisco Channels Release Management Cisco Network Academy Cisco Tele-presence Cisco Internal Directory Cisco Internal Learning (e-learning) 2010 Cisco and/or its affiliates. All rights reserved. 45

Customer X Pre UCS: HP Superdome 32 shared for prod & test Prod npar: 20 CPU cores, 48 GB RAM Test npar: 2 CPU cores, 8 GB RAM Key Advantages >$600K reduction in hardware/os cost ~75% Reduction in

46 Customer X Pre UCS: HP Superdome 32 shared for prod & test Prod npar: 20 CPU cores, 48 GB RAM Test npar: 2 CPU cores, 8 GB RAM Key Advantages >$600K reduction in hardware/os cost ~75% Reduction in overall power consumed Performance increase as we adopt latest Intel technology Customer X on UCS: Prod: 1 B440M1 server 32 CPU cores, 256 GB RAM Test: GB B200-M1 blade 8 CPU cores, 96 GB RAM Increased CPU cores & Memory using less power Simplified management. UCS & Linux OS management is simpler & more automated Cisco on Cisco. UCS servers capable of hosting previously Unix-only highperformance applications 2010 Cisco and/or its affiliates. All rights reserved. 46

47 Cisco Texas DC2 James Cribari

51 Cisco multi-tenant, high density data center Total area 162,000 Sq Ft Data floor 27,000 Sq Ft (spread over two halls). No raised floor LED lighting in Data Hall; 100kW photovoltaic system 2N Systems: Primary Utilities, Generators, UPS; N+2: Chillers Building hardened to wind speed requirement of 175 mph Goal: Gold LEED Certified Data Center Geothermal wells Active-Active DC (Two 10 Gb dark fiber connections to RCDN-9 DC) 2010 Cisco and/or its affiliates. All rights reserved. 51

52 Category Legacy Design Allen Design Inlet Temperature 68-72F (20-22C) 78F (25.5C) Air separation None Hot/cold air separation based on chimneys Cooling CHW System with CRAH units Airside economizer (free cooling) Density kw/rack Average 10.5 kw/rack Up to 18 kw/rack Power distribution 480V Back of House 208/120V to Rack 4160V Back of House 415/240V to Rack UPS Systems Static Dynamic (flywheel) Certification n/a LEED PUE Target annual ( ) 2010 Cisco and/or its affiliates. All rights reserved. 52

53 Stand-alone DC facility Separated from RCDN9 DC by around 20 miles 5 MW IT load (with additional 5MW possible in future) 200 watt / sf density yielding approximately 25K sf of data center floor Tier 3 per the Uptime Institute 2010 Cisco and/or its affiliates. All rights reserved. 53

54 5.5 MW of critical load capacity (potential expansion to 11 MW in future requiring additional buildout) 2N electrical service 2N UPS dynamic/flywheel system 2N Generators (approx 96 hr run time) 4160V Distribution System 230V cabinet power distribution All IT equipment to be dual corded Power delivered via overhead busway system with in-rack smart CDUs (power strips) 2010 Cisco and/or its affiliates. All rights reserved. 54

55 Hot air discharge containment via chimneys for all cabinets IT equipment operational target of 78 F inlet supply air temperature AHU capacity is based on an average 25 degree delta T across the IT equipment Overhead air delivery Airside economizer system N+2 chillers and cooling towers, N+1 pumps 15 minute thermal storage tank 48 hrs Domestic water storage 2010 Cisco and/or its affiliates. All rights reserved. 55

56 2 diverse data ductbanks No raised access floor No suspended lay-in ceiling exposed structure and equipment Overhead delivery of power and network Building hardened to 175 mph Vestibule Entry, no security turnstiles Layout will be designed to include a Customer Tour route and viewing panels into the Data Center & Building System rooms Target PUE of 1.3 0r lower Office space with approximately 20 workstations and areas to support customer visits 2010 Cisco and/or its affiliates. All rights reserved. 56

57 LEED NC v2.2 target is Gold certification and current indications are that we will achieve this Utilize airside economizer systems Utilize elevated IT equipment inlet temperatures (78F) Implement Cisco Energy Wise to provide efficient facility energy management Utilize higher voltage power distribution Utilize dynamic UPS systems Solar cells on roof space to reduce utilities consumption Ground source heat pump systems for office area HVAC 2010 Cisco and/or its affiliates. All rights reserved. 57

Overhead Delivery / Partial Containment 2010

3300 kw, Tier 3, Chilled Water, 70,000 ft 2

[Reference Design 22] 3300 kw, Tier 3, Chilled Water, 70,000 ft 2 DESIGN OVERVIEW Data Center IT Capacity 1100 kw per Data Hall 3 Independent Data Halls Design Application Range This design can be applied