Data center cooling evolution. The hottest way to save energy Lucio Bustaffa: Product Development Manager BlueBox Group
Data Center overview What s happening in the web? 2
Data Center overview What s happening in the web? 3
Data Center overview How much energy is consumed in Data Centers? 4
Data Center overview How is energy used? 5
Data Center overview How is energy used? 6
Data Center overview How is heat load generated? 7
Data Center overview A Data Center is not intended for human occupation and comfort Comfort Cooling Heat removal 8
Data Center overview What s the right temperature in a Data Center? Common situation coming from an outdated view of Data Centers 9
Data Center overview Why low temperatures? LEGACY DATA CENTERS Strict control on temperature and humidity because of the presence of dated equipment and reams of paper inside 16/20 C 10
Data Center overview Why heat has to be removed? Recommended and allowable environmental specifications fixed for IT equipments 11
Data Center overview Who sets the current parameters and limits? 12
Data Center overview Which are current parameters? 18 C 27 C 13
Data Center overview How did parameters evolved? 14
ENERGY CONSUMPTION FAILURE FACTOR Data Center overview What s the effect? Values represent running equipments 24 x 7 x 365 at indicated temperatures 15
Data Center overview How to reduce cooling energy? COOLING ENERGY Power 20% ENTERING ENERGY = Cooling 30% IT 50% HEAT LOAD 16
Data Center overview How to reduce cooling energy? Increasing temperatures ENTERING ENERGY SAVING COOLING ENERGY = Increasing Cooling equipment efficiency Reducing cooling distribution unefficiency HEAT LOAD Power 20% Cooling <30% IT 50% 17
Data Center overview PUE & ppue (referred to cooling efficiency) PUE = Total facility power IT equipment power IT eq. power + Cooling PPUE = power IT equipment power 18
Data Center overview Average PUE factor 19
Data Center cooling Cooling systems classification Mechanical cooling Cooling systems Freecooling 20
Data Center cooling Mechanical cooling All the cooling systems which provide a vapour compression cycle involving a refrigerant (natural or synthesys) Air based cooling Mechanical cooling Water based cooling Liquid cooling 21
Data Center cooling Freecooling Cooling systems which transfer cooling energy from outdoor to indoor without using vapour compression cycle (mechanical power is still used) Freecooling Air based freecooling Water based freecooling Direct freecooling Indirect freecooling 22
Data Center cooling Perimeter cooling A mechanical cooling system (or a mix: mechanical cooling + freecooling) is based on combination of a cooling generator (positioned outside of the IT area) and several terminal units (positioned in the IT area). Position of indoor units generates a secondary classification Perimeter cooling (room oriented) 23
Data Center cooling Perimeter cooling - Occupied space - Raised floor needed - Cooling density - Air flow mixing - Centralized solution - Flexibility - Modularity - Redundancy - CAPEX - OPEX (room oriented) 24
Data Center cooling In row cooling In row cooling (row oriented) 25
Data Center cooling In row cooling - Occupied space - Need for raised floor - Cooling density - Air flow mixing - Centralized solution - Flexibility - Modularity - Redundancy - CAPEX - OPEX (row oriented) 26
Data Center cooling Containement cooling In row cooling Containement cooling (isle oriented) Perimeter cooling 27
Data Center cooling In rack cooling In rack cooling (single rack oriented) 28
Data Center cooling In rack cooling - Occupied space - Need for raised floor - Cooling density - Air flow mixing - Centralized solution - Flexibility - Modularity - Redundancy - CAPEX - OPEX (single rack oriented) 29
Data Center cooling Liquid cooling Water outlet 40 C Internal fluid Liquid cooling (single server oriented) Exchanger Water inlet 35 C 30
Data Center cooling Liquid cooling - Occupied space - Need for raised floor - Cooling density - Air flow mixing - Centralized solution - Flexibility - Modularity - Redundancy - CAPEX - OPEX Water outlet 40 C Internal fluid Exchanger Water inlet 35 C (single server oriented) 31
Data Center cooling Free cooling Free cooling is something different from a cooling plant or equipment, it is more related to a favorable condition happening when outdoor temperature is lower than the indoor and the difference can be exploited to cool down the indoor air Direct freecooling Freecooling Indirect freecooling Water based freecooling Air based freecooling 32
Data Center cooling Free cooling Direct free cooling: outdoor cold air is directly blown in the Data Center to control the indoor temperature. Fresh air is continuosly introduced in the indoor ambient Exhaust air unit Direct freecooling Cooling media: air Intake air unit Temperature/humidity control. Filtration. Ventilation. 100% back up cooling is needed 33
Data Center cooling Free cooling Indirect free cooling: cold air is used to cool down the recirculated air in the Data Center without introducing fresh air and maintainig isolated the indoor space. Indirect freecooling Water based Cooling media: air-waterair Water pump. Temperature control Partial back up cooling is needed 34
Data Center cooling Free cooling Indirect free cooling: cold air is used to cool down the recirculated air in the Data Center without introducing fresh air and maintainig isolated the indoor space. Indirect freecooling Air based Cooling media: air-air Temperature control Partial back up cooling is needed 35
Data Center cooling What s happening in the Data Center market Many different variables are affecting the cooling technologies in DC. Some of them: DC location DC dimension ASHRAE prescriptions CAPEX & OPEX New construction/refurbishment IT load density Water availability DC cooling technology? Space availability Enterprise/colocator Containe ment Investor desiderata 36
Data Center cooling What s happening in the Data Center market Source: BSRIA cooling technologies report 2015 37
CONTAINEMENT COOLING HAC & CAC Both of them suites either the perimeter cooling or the in row cooling Perfect fot data center retrofitting Maximizes efficiency of cooling units because of the high temperature air entering Guarantees the correct cold air temperature to the servers Avoids air mixing 38
CONTAINEMENT COOLING HAC Perimeter cooling In Row cooling Not focused on cooling Hot air ducting => higher pressure losses Overhead space for ducts Not focused on cooling More efficient, no ducts Most difficult retrofitting 39
CONTAINEMENT COOLING CAC Perimeter cooling In Row cooling Higher cooling load compared to HAC Meet ASHRAE requiremnts More retrofitting oriented Maximum cooling load Higher efficiency More retrofitting oriented Raised floor needed 40
AIR SIDE ECONOMIZER Direct free cooling Efficiency CAPEX OPEX Suitability for retrofitting Contamination Humidity introduction Independency on outdoor air quality 100% back up need 41
AIR SIDE ECONOMIZER Direct free cooling Suites in particular Large Enterprise Data Center. Facebook, Google... - Servers are replaced every 2/3 years - Space and servers are not intended to be rent - The owner of the Data Center has complete control and responsability on it 42
WATER SIDE ECONOMIZERS Indirect water free cooling Efficiency CAPEX OPEX Suitability for retrofitting Contamination Humidity introduction Independency on outdoor air quality 100% back up need Efficiency can be increased with adiabatic cooling 43
WATER SIDE ECONOMIZERS Indirect water free cooling Suites in particular Medium/Large Colocator Data Center. - Servers are replaced less frequently - Space and servers are intended to be rent - No contamination is allowed - Customers need warranty about the safety of their data - A complete back up cooling system is generally needed due to the freecooling intrinsic unefficiency 44
AIR SIDE ECONOMIZERS Indirect air free cooling This solution combines a lot of advantages coming from direct free cooling with the complete flow separation Efficiency decreases a bit if compared to a direct free cooling solution but it allows a very strict control on the indoor air condition in the Data Center Fresh external air flow Recirculated air flow to Data Center Recirculated air flow from Data Center Scavenger air flow 45
AIR SIDE ECONOMIZERS Indirect air free cooling Efficiency CAPEX OPEX Suitability for retrofitting Contamination Humidity introduction Independency on outdoor air quality 100% back up need 46
AIR SIDE ECONOMIZERS Indirect air free cooling Suites in particular Medium/Large Colocator Data Center. - Servers are replaced less frequently - Space and servers are intended to be rent - No contamination is allowed - Customers need warranty about the safety of their data - A partial back up cooling system is generally needed due to the freecooling high efficiency 47
SIMULATION ANALISYS & COMPARISON Data Center model characteristics and lay out Area 146 m² Cooling Load Data Center 400 kw N of Rack 80 / N of rows 4 / Cooling Load for Rack 5 kw Data Center Size 10.2 x 14.4 m x m Rack Size 0.6 x 1.043 m x m Tile Size 0.6 x 0.6 m x m N Cold Aisles 2 / N Hot Aisles 3 / TSP** 38/23 C 48
SIMULATION ANALISYS & COMPARISON Data Center model characteristics and lay out High efficiency water free cooling plant 49
SIMULATION ANALISYS & COMPARISON Data Center model characteristics and lay out 211kW High efficiency water free cooling plant (N+1 redundancy) 18 C 142kW 142kW 211kW 13 C 142kW 211kW 142kW 30 m pipes + manifolds + 10m per CRAC... 2kW hydraulic pump, 3kW for free cooling 50
135kW SIMULATION ANALISYS & COMPARISON Data Center model characteristics and lay out Indirect freecooling plant (N+1 redundancy) 135kW 20318m 3 /h => 100kW (delta T= 15 C) 20318m 3 /h => 100kW (delta T= 15 C) 135kW 135kW 20318m 3 /h => 100kW (delta T= 15 C) 20318m 3 /h => 100kW (delta T= 15 C) 51
SIMULATION ANALISYS & COMPARISON Data Center model characteristics and lay out - London - Frankfurth - Moscow - Stockholm 52
SIMULATION ANALISYS & COMPARISON From DC air flow External air flow flow Scavenger air flow To DC air flow 53
SIMULATION ANALISYS & COMPARISON Scavenger air flow External air flow From DC air flow To DC air flow 54
SIMULATION ANALISYS & COMPARISON From DC air flow Scavenger air flow External air flow To DC air flow 55
SIMULATION RESULTS I n p u t Data Center heat load[kw] Delivery air T [ C] Intake air T[ C] Adiabatic humidifier On-Off (T[ C] ; U.R.) Pressure losses [Pa] Location Abiabatic humidifier active for T>5 C and R.H.< 90% Delivery air temperature regulated according to the external air flow
Energy Consumption[kWh] Energy Consumption[kWh] SIMULATION RESULTS 400000 400000 350000 350000 300000 250000 200000 1_06 1_07 S06 300000 250000 200000 1_06 1_07 S06 150000 S07 P06 150000 S07 P06 100000 P07 100000 P07 50000 Chill.FC 50000 Chill.FC 0 70 80 90 100 0 70 80 90 100 Frankfurth London Rec./Chill. 1_06 1_07 S06 S07 P06 P07 Rec./Chill. 1_06 1_07 S06 S07 P06 P07 % 70 31,32 28,12 49,16 45,94 69,43 69,23 % 70 32,33 28,99 51,43 48,92 67,55 67,33 % 80 25,99 21,72 46,36 43,33 67,44 67,22 % 80 26,29 21,76 48,49 45,65 65,37 65,12 % 90 20,09 14,41 41,70 40,86 64,96 64,68 % 90 20,13 14,19 43,99 43,02 65,67 65,42 % 100 13,76 8,67 39,50 38,43 61,88 61,54 % 100 13,58 7,56 41,65 40,49 62,30 61,99 57
Energy Consumption [kwh] Energy Consumption[kWh] SIMULATION RESULTS 350000 350000 300000 300000 250000 200000 150000 100000 50000 1_06 1_07 S06 S07 P06 P07 Chill.FC 250000 200000 150000 100000 50000 1_06 1_07 S06 S07 P06 P07 Chill.FC 0 70 80 90 100 Moscow 0 70 80 90 100 Stockholm Rec./Chill. 1_06 1_07 S06 S07 P06 P07 % 70 28,32 25,28 44,69 43,50 64,49 64,27 % 80 23,76 19,79 41,14 40,30 62,08 61,84 % 90 17,32 11,31 36,59 35,86 60,83 60,56 % 100 9,90 4,77 33,65 32,57 57,31 56,97 Rec./Chill. 1_06 1_07 S06 S07 P06 P07 % 70 31,51 28,53 47,00 46,23 64,80 64,61 % 80 26,68 22,77 44,11 43,41 62,52 62,32 % 90 19,93 13,64 39,21 38,30 61,49 61,27 % 100 11,86 6,42 36,16 35,05 57,48 57,19 58
FREE COOLING MAP 59
ppue (only cooling referring) Calculation is referred to double plate heat exchanger series IT load= 400kW => IT annual energy consumption: 3.504.000kWh Frankfurth London Mosco w Stockholm ppue Traditional cooling 1,104 1,102 1,090 1,086 ppue Indirect Freecooling 1,064 1,060 1,058 1,055 60
ppue (only cooling referring) Calculation is referred to double plate heat exchanger series IT load= 400kW => IT annual energy consumption: 3.504.000kWh Frankfurth London Mosco w Stockholm ppue Traditional cooling 1,104 1,102 1,090 1,086 ppue Indirect Freecooling 1,064 1,060 1,058 1,055 Reduction in cooling energy demand 38% 41% 36% 36% 61
Thank you for your attention and, above all, thank you for not wasting energy