Maximise Data Center Value

Size: px

Start display at page:

Download "Maximise Data Center Value"

Stephen Beasley
5 years ago
Views:

1 Maximise Data Center Value with Higher Density Infrastructure Dramatic cost savings enabled; Requires solving unique electrical + cooling challenges; 08 December 2015

2 Speaker Genel Müdür 4G İletişim Teknolojileri A.Ş.

Data Center Investments Keep Increasing Continued Infrastructure Growth Number of data centers increasing Average size of data centers also increasing Even though footprint of equipment is decreasing

3 Data Center Investments Keep Increasing Continued Infrastructure Growth Number of data centers increasing Average size of data centers also increasing Even though footprint of equipment is decreasing Driven by Underlying Storage and Compute Demand business intelligence / data mining e-commerce at every scale private cloud better & cheaper bandwidth RESULTS OpEx requirement (or CapEx for co-location) increasing; Budgets not keeping pace; Massive consolidation efforts and shift to co-location;

4 Even When Server Power Draw Decreasing Year Watts / U 2004 (DL360G3) 360 W 2008 (DL360G5) 238 W 2011 (DL360G7) 157 W 2013 (DL360G8) 170 W 2015 (DL360G9) 105W Source: Data Center Pulse. Wiersma, Jan. Where is the Rack Density Trend Going? ; HP QuickSpecs Combining major CPU improvements in power/mips better underlying h/w architecture e.g. High-efficiency PSUs e.g. Move to SSDs e.g. Improved front-side bus architectures (RAM) standard 1U / 2U servers are actually less power-hungry per U.

Panel Board / Distribution Board Owner operated = CapEx Co-location facility = OpEx

5 Customers Face Immense Upfront Costs: Infrastructure + Real Estate Air Handlers Switchgear Transformers Chillers / Economizers Generators CRAC / CRAH UPS RPP / Panel Board / Distribution Board Owner operated = CapEx Co-location facility = OpEx Before a single CPU is powered on, incredible investments must be made; Land and Building

6 Real Estate Prices Increasing Prohibitively In certain regions, no viable candidate sites available for data center development; e.g. Hong Kong (Google, other multinationals) Some companies convert commercial buildings into data center (limits space and usability) Real Estate Costs + Infrastructure Costs # of Cabinets Per rack investments are massive and fixed ; = fixed investment per rack Therefore, massive returns from increasing cabinet power density.

7 Increased Power Density Enables Consolidation Increase to 11KVA Add 7.5 KVA Avg load /rack = 3.5 KVA Rack 1 Rack 2 Rack 3 Rack 4 Rack 5 Rack 6 Rack 7 Rack KVA Reduce Racks Free Space

but Typically Requires Shift to Blades @ 3ph Phase 16A 32A 63A 16A 32A 63A Single phase Single phase Single phase Three phase Three phase Three phase Input Voltage 230V 230V 230V 400V 400V 400V

8 but Typically Requires Shift to 3ph Phase 16A 32A 63A 16A 32A 63A Single phase Single phase Single phase Three phase Three phase Three phase Input Voltage 230V 230V 230V 400V 400V 400V Output Voltage 230V 230V 230V 230V 230V 230V Active power 3.7 kw 7.4 kw 14.5 kw 11 kw 22.1kW 43.6 kw # of servers * 57 52* # of blades n/a 32 (2 chassis) 64 (4 chassis) 48 (3 chassis) 80* (5 chassis) * Could power more, but cannot fit in standard cabinet HP Proliant DL360pG8 1U server, 192 watts at 70% load HP Proliant C7000 w/ 16x Proliant BL460c Gen8, 3710 watts at 70% load

9 415v / 3ph : Dramatically Increases Density

But High Density Racks Face Unique Challenges Weight. Can cabinet handle increased static and dynamic load? Cooling. Can air handlers, chillers, etc. be scaled to exhaust increased heat? Power.

10 But High Density Racks Face Unique Challenges Weight. Can cabinet handle increased static and dynamic load? Cooling. Can air handlers, chillers, etc. be scaled to exhaust increased heat? Power. Adequate UPS and other infrastructure? Understand complexities of branch circuit in cabinet? Known, proven solutions exist for all these challenges. Cost more per cabinet vs. lower density; But far outweighed by cost savings of increased utilization.

11 Proper Feed Sizing for High Density Blade Chassis Prevalence of blade servers increasing every year; Increased confusion regarding power interconnects required to maintain true 2N; Issue compounded when clients solely consider power capacity guidelines of RPP / distribution panel feed; e.g. Cisco UCS U height; ~1800W typical; ~2300W peak; 4x Power Supplies, up to 2500W each; e.g. 415V, 3phase WYE; 32a Supply 23,000VA / ~22,400 Watts; In theory, should support up to ( ) = 9 chassis; Let s try seven (48U rack)

12 Proper Feed Sizing for High Density Blade Chassis First Connect Six Chassis (14.2kVA) Peak = 4.9A per connection; Peak (Failure Mode) = on one plug

13 Proper Feed Sizing for High Density Blade Chassis Even with B-side down, each circuit breaker at 61% load. Very safe.

14 Proper Feed Sizing for High Density Blade Chassis Failure Mode: B-Side power down and some A-side power supplies fail. Still safe.

15 Proper Feed Sizing for High Density Blade Chassis Add 7 th Chassis. Need to share breakers.

16 Proper Feed Sizing for High Density Blade Chassis 14.7A Still safe so far. Two breakers load to 14.7A if B-side power down. 14.7A

17 Proper Feed Sizing for High Density Blade Chassis 14.7A 14.7A Not truly redundant! One bad power supply (during B-side maintenance) can shut down 3 chassis

18 Proper Feed Sizing for High Density Blade Chassis 14.7A 14.7A Not truly redundant! One bad power supply (during B-side maintenance) can shut down 3 chassis

19 Proper Feed Sizing for High Density Blade Chassis 14.7A Not truly redundant! One bad power supply (during B-side maintenance) can shut down 3 chassis

20 Proper Feed Sizing for High Density Blade Chassis 14.7A Not truly redundant! One bad power supply (during B-side maintenance) can shut down 3 chassis

21 Proper Feed Sizing for High Density Blade Chassis 14.7A Not truly redundant! One bad power supply (during B-side maintenance) can shut down 3 chassis

22 Proper Feed Sizing for High Density Blade Chassis 19.6A Not truly redundant! One bad power supply (during B-side maintenance) can shut down 3 chassis

23 Proper Feed Sizing for High Density Blade Chassis Not truly redundant! One bad power supply (during B-side maintenance) can shut down 3 chassis

24 Proper Feed Sizing for High Density Blade Chassis Not truly redundant! One bad power supply (during B-side maintenance) can shut down 3 chassis Violates best practice concept of isolated failure domains.

Proper Feed Sizing for High Density Blade Chassis Example Result: (One Possible Solution) This cabinet provides two redundant 23kVA power feeds.

25 Proper Feed Sizing for High Density Blade Chassis Example Result: (One Possible Solution) This cabinet provides two redundant 23kVA power feeds. Does not necessarily translate into, Can implement 23kVA of load. 7 ~16.5kVA did not work! Apparent maximum = 6 36U (14.1kVA); Wastes 40% of power capacity; Wastes 25% of rack space (assuming 48U cabinet);

26 Thank You!

Server Technology Inc.

Server Technology Inc. Power Efficiency Gains by Deploying 415 VAC Power Distribution in North American Data Centers White Paper STI-100-008 2011-Sept-22 Headquarters Server Technology, Inc. 1040 Sandhill