Data Center Power and Cooling (and More)

Transcription

1 Data Center Power and Cooling (and More) Francisco Luis Roque WCC Lead System Engineer

2 The Road Behind

3 The Road Behind

4 WCC ESX 5i Overview Physical Architecture Virtual Architecture Robert Phillips

5 Relationship Overview Map Host to VM Host to data-store

6 Some Environment Facts Disk use, 43TB on 51 VMFS volumes Raw Device Mappings < 5TB 212 virtual guests (Development and Production) Windows 2003 server, 2008 server, 2012 server Linux, Solaris, and flavors

7 How WCC Benefits from Virtualization Power Savings Cooling savings Shared Storage environment Portability VMotion, and Storage Motion Helps meet DR and business continuity goals Snapshots Facilitate roll-back during upgrades and changes Test Environments are easily constructed

8 ESX Cluster, two locations 4 hosts, Two Resource pools 252 Vm s/templates Pix Cluster two locations Two hosts, Two Resource Pools 16 vms and templates VDI Cluster Single host cluster 5 vms and templates Storage, multiple SAN, Multiple Location FC Fabric attched Solaris/Global 14 Zones Single Host Cluster ESX5i, Stand Alone on Vcenter Essentials Multiple Stand-Alone Hosts Numerous OS DR Site Storage, multiple SAN, FC Fabric attached

9 2540 M M2 SL500, 150 slot LTO3 Tape Library SC6140, 2 cont. 4 tray, 11TB Main Campus SC Datacenter SC2540, 2cont. 1 tray, 11TB SC AMS2100, 5 Trays, 52TB SC-FLX240, 2cont., 7 tray 15TB FC Fabric BE 6140, 2cont., 3 Tray, 23TB Main Campus BC Site BE AMS2100, 5 trays, 35TB Southfield DR site Southfield 2540, 2cont. 1 tray, 5TB FC Fabric Southfield AMS tray, 18TB

10 Host Requirements Processor - Internal Memory - Internal Network Port Internal/External Storage Internal/External Power - External Cooling - External Network Hardware (Switches) - External Storage Network (Switches) - External Power Distribution - External

11 Power Requirements Physical Machine Group 1, Physical 30 Dell PE R810 4 Socket, Quad Core, 32 Gigs Memory, Raid 5, 3 Disk, Dual Redundant Power Supplies (4 Power Supplies) 100 Dell PE R610 2 Socket, Quad Core, 32 gigs Memory, Raid 0, Two disk, Dual Redundant Power Supplies (2)

12 Power Requirements Virtual Machine Group 2 Virtual 4 Dell PE R910 4 Socket, Quad Core, 128 Gigs Memory, Raid 5, 3 Disk, Redundant Power Supplies, (4 Power Supplies)

13 Side by Side Comparison Virtual Summary Physical Summery Kilowatt Hour Cost: $0.04/kWh $0.04/kWh Solution Watts/BTU 30,270watts 103,285.5btu/h 3,900watts 13,307.4btu/h Current 280.8amps 36.2amps Air Flow l/s 5,070.6CFM 234.6l/s 497.1CFM Total Weight 2579kg 5,685.7lbs 276.6kg 609.8lbs Total Cost Per day $0.04 x 300kWh $12.00 $0.04 x 30kWh $ kwh = 100w for 10 hours Savings Converting to Virtual; Watts/BTU -26,370 Watts / -89,978btu/hr Current amps Air Flow -4,573.5 CFM Weight lbs

14 Other Savings Physical 212 Serv. Virtual 4 Serv. 120VAC Power Connections GB Network Connections Storage Network Connections PDU Requirements (40 Plugs each) 11 2 Network Switch Requirements (48 Prts. each) 14 2 Storage Network Switch Requirements 28 4 (32 Ports each) Rack Space Required (2u per server, 42u per rack 11 1 Physical Hardware Notes: 212 Servers 5 Servers R610, 2 Power, 2 Network, 2 Storage R810, 4 Power, 3 Network, 2 Storage R910, 4 Power, 8 Network, 2 Storage

15 Links Dell, Energy Smart Solution Advisor (ESSA) Provides hardware power/cooling specifications and requirements for Dell hardware.

16 The Road Behind

17 Today

18 What uses power in a data center? AC UPS Lights & other building infrastructure IT Gear Servers Storage Network & other IT infrastructure

19 What needs cooling in a data center? Anything that uses power Plus a few extras Humans Environment

20 Electricity comes from The grid Coal, oil Solar, wind UPS Wet cel Dry cel Generator Natural gas (< 150kw) Diesel (>100kw)

21 Cooling comes from Traditional AC Heat Exchangers Environment (but consider humidity)

22 Useful Terms Volts/Amps/Watts, Celsius/Fahrenheit, Dollars Rack Units: physical size kwh/kva: electricity (cf. power factor) 110/208/440, 1 or 3-phase: types of electricity BTU: heat Tons: cooling CFM: airflow PUE/DCiE/CUE: data center efficiency/sustainability TCO/ROI/CBA/CAPEX/OPEX: money

23 Useful Terms TCO/ROI/CBA/CAPEX/OPEX: How to talk to your CIO CAPEX: buy car for $25000 OPEX: $2500/year, over 10 years: $25000 TCO: CAPEX + OPEX = $50000 New job, $10000/year, over 10 years: $ ROI: $ $50000 = $50000, 100% return CBA: If I buy car, I will get 100% ROI over ten years. Therefore, I should buy car

24 Good Power Design Redundant, redundant, redundant monitor test verify preventative maintenance (pm)

25 Cooling Design: Bad Traditional DC, not much planning

26 Cooling Design: Ok Hot /Cold Aisle

27 Cooling Design: Better Hot /Cold Zone Containment

28 Cooling Design: Alternative Esoteric Rack Options and Other Novel Approaches

29 Case Study: WCC Primary Server Room The Problem: Hotspots, some servers running with constant heat alarm, 3 AC s always running.

30 Case Study: WCC Primary Server Room 1. Gather temp/airflow/elec data 2. Test out different designs 3. Determine savings of each 4. Implement new design Bad Solution: New AC 4 Active AC s CAPEX $40k OPEX $5k Good Solution: Containment 2 Active AC s, 1 Standby CAPEX: $20k OPEX: -$5k

31 A Few Good Data Centers WMU in Kalamazoo, MI UMHS in Ann Arbor, MI Yahoo in Buffalo, NY Google in Mons, Belgium NSA in Bluffdale, UT Others

32 A Simple Model Server OS DB Server SAN VM host AC/Power Employee Forest City Users HW App Server Server VM guest IT Infrastructure Manager Plains Suburbs

33 Cloud Computing Completely virtual On demand Infinite resources Multi-tenant Metered IT becomes a utility, like electricity or water

34 Cloud Services IaaS - Infrastructure PaaS - Platform SaaS - Software *aas Anything as a Service Pay per qty; bandwidth and storage costs often separate WCC: Gmail, Ensemble, Merit CyberRange, Duo Security

35 When Clouds Become Storms Service Upgrades/Changes Dependencies Security issues (side channel attacks, etc) Unknown Issues Hurricane Sandy Future solutions: Iris/PoR/RAFT/HAIL, homomorphic encryption Cloud is still in its infancy; ask your vendor what cloud means to them and how they will weather the storms

36

37

38