Power and Cooling in the Data Center:

Transcription

1 Power and Cooling in the Data Center: Hyper-density Trends and Solutions Ken Baker Infrastructure Technologist 2004 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice

2 Modern Datacenter Design Raised floor, forced air cooling through perforated panels Power and network wiring may be under floor or overheard Designed for 5-10 year lifecycles Not designed for additional growth in power and cooling capacity Cooling efficiencies average between 40-50% 2

3 Average Datacenter Density Lawrence Berkley National Labs study found no datacenters in their study that exceeded 1950W /sq meter Historical trends have shown loading in the W /sq meter Customers have not seen a need to invest in support for density until now. 3

4 Energy Density of Datacenters Datacenter capacity is not rising at the same rate as power density in servers Indicates there is a resistance to dense deployments in volume 4

5 Customer Landscape 3000 Sanger Institute Customer Facility Examples Watts per square meter Dreamworks Palo Alto Nacio Cutten Road 365 Main Site6.2 PNNL Los Alamos Verizon Raytheon Oracle Watts Site Site4.2 Site4.1 Site3 Site8.1 Morgan Stanley Goldman Deutsche Bank Wells Fargo BofA Site8.2 Site2 Site1 State Farm Citigroup Site9 Site7 Size of Datacenter

6 Datacenter Trends Four years ago Average servers per rack = 4 6 Average U size was 5 7U Average watts per rack = Today Average servers per rack = 8-24 Average U size is moving down to 2U Average watts per rack = Future Power densities of watts per U are imminent 6

7 Server Trends Generation over generation density growth rates DL360 G2 DL360 G3 246 W 389 W 58% 2p, 4gb, 2hdd, 1pci 1U DL380G2 362 W DL380 G3 581 W 60 % 2p, 4gb, 6hdd, 2pci 2U DL580 G1 456 W DL580 G2 754 W 65% 4p, 8gb, 4hdd, 3pci 4U 7

8 Intel Processor Roadmap 8+ Way Common Platform CSI based 4 Way Gallatin 2M/130nm 2.0GHz Gallatin 4M 4M/130nm 3GHz T case 71 ºC TDP 85 W Potomac 8M/90nm 3.6GHz T case 65 ºC TDP 115 W 667 MHz Tulsa Dual core 8M/90nm 3.4GHz T case 69 ºC TDP 175 W 32bit Whitefield 16M/90nm TDP W DP Prestonia 512K/130nm 2.8GHz Prestonia 2ML3 130nm 3.2 GHz T case 71 ºC TDP 94 W Nocona 1M/90nm 3.6GHz T case 71 ºC TDP 103 W Dempsey DP DC 65nm T case? ºC TDP 150 W Woodcrest 4M (mobile dual) 65nm TDP 70 W 400 MHz 533 MHz 800 MHz 8

9 Impact of Future Dense Server Designs Dense designs drive up the Watts/ square meter requirement A typical rack requires 1.3 square meters of space At bl30p watt densities of watts per cabinet (max theoretical), the watt density for a facility of bl30p s could exceed >26,000 watts per square meter! 9

10 Understanding the Limits of Data Center Designs HP commissioned study Impact of increasing power density What can a facility successfully support? How can we improve the efficiency of a datacenter? Initial data indicates supporting room level power densities above 2000 to 2500 watts per square meter is not practical using classic raised floor designs 10

11 CM750-FL BC Y-Plot at 78 from the raised floor 11 R1C13-C14 max = 78.5 o F

12 CM750 - FL Solved One extra tile Layout PDU 1 PDU 2 PDU 4 PDU 3 Legend: = deactivated tiles = 56% tiles = 25% tiles = 30KW cabinets 12

13 R1C12 - C13 max = 68.3 o F CM750-FL Solved Y-Plot at 78 from the raised floor Note: Maximum allowable rack inlet temperature is 77 o F 13

14 PDU 3 PDU 4 CM1500 FL - BC Layout Top View 14 Extra AHU All floor tiles 25% open AHU1 AHU2 AHU9 AHU3 AHU4 Hot Aisle Hot Aisle Hot Aisle Hot Aisle Hot Aisle Hot Aisle AHU8 R3C1 AHU7 AHU10 AHU6 AHU5 Extra AHU PDU 2 PDU 1

15 CM1500 FL - BC Y-Plot at 78 from the raised floor Max. inlet temperature occurs at R9C1: 84.1 o F R9C1 R10C1 15

16 R1C14 CM1500 FL Solved HE Layout Top View Extra AHU AHU1 R1T14 AHU2 AHU9 AHU3 R9T14 AHU4 R1C1 R2T1 R2C1 PDU1 PDU2 R3T1 PDU4 PDU3 R10C14 AHU8 AHU7 AHU10 AHU6 R9C1 AHU5 Extra AHU 16

17 CM1500 FL Solved HE Y-Plot at 78 from the raised floor Max. inlet temperature occurs at R9C1: 71.8 o F R9C1 R10C1 Note: Maximum allowable rack inlet temperature is 77 o F 17

18 CM3000-FL BC Y-Plot at 78 from the raised floor R4 R5 C13: 95.2 o F R4C13 R5C13 18

19 CM3000-FL - Solved Layout Floor tiles AHU 10 AHU 9 AHU 8 AHU 7 AHU 6 PDU 1 PDU 2 AHU 11 PDU 4 AHU 12 PDU 3 AHU 5 AHU 4 AHU 3 AHU 2 AHU 1 De-activated tiles 25% open tiles All other tiles are 56% open tiles 19

20 CM3000-FL Solved Y-Plot at 78 from the raised floor AHU 10 AHU 9 AHU 8 AHU 7 AHU 6 AHU 11 AHU 12 AHU 5 AHU 4 AHU 3 AHU 2 AHU 1 Highly localized region of 77.3 o F All hot air temperatures observed in the base case have now come down below 68 o F Note: Maximum allowable rack inlet temperature is 77 o F 20

21 CM150 FL - BC Pressure plot at 2 below the raised floor Max of H 2 O observed below the supply of AHU 21

22 Data Center Thermal Modeling Non-intuitive flow patterns (HPL Palo Alto Smart Data Center Project) 5% open vent tile results in cooler inlet temperature Front View 5% open tile 95% open vent tile results in hotter inlet air temperature 95% open tile Front View 22

23 Data Center Thermal Modeling Dreamworks Redwood City (DL360 G3) Solution: This issue was solved by strategically repositioning inlet air vents (the only degree of freedom allowed by the customer) 23

24 Data Center Thermal Modeling Impact of load or cooling changes Modeling has the capability to show the impact of: Turning on new machines AC failure 100 % load Region Size AC shutdown for scheduled maintenance Factoid: For a given data center (at 2250 W /sq meter) with an A /C failure, the reaction time to reduce load is less than ~ 35 seconds to prevent redlining & shutdown 50% load 24

25 Key Findings Use of intuition is counterproductive in managing thermal situations Costs to support datacenter designs above 1000 watts/sq meter can become prohibitive Harvesting of low hanging fruit lies in improving existing efficiencies Thermal modeling can provide significant energy savings and allow for increased densities without spending on new equipment 25

26

27 Co-produced by: