Saving Water and Operating Costs at NREL s HPC Data Center

Size: px

Start display at page:

Download "Saving Water and Operating Costs at NREL s HPC Data Center"

April Stephens
5 years ago
Views:

com David Sickinger National Renewable Energy Laboratory David.

1 Saving Water and Operating Costs at NREL s HPC Data Center Thomas P. Carter, P.E. Johnson Controls Thomas.P.Carter@jci.com David Sickinger National Renewable Energy Laboratory David.Sickinger@nrel.gov IDEA2017 Conference June 28, A-District Energy & Water Strategies for Sustainable Operation

2 Outline/Agenda Traditional and hybrid choices for heat rejection Description and operation of the thermosyphon cooler (TSC) Description of the National Renewable Energy Laboratory s (NREL s) Energy Systems Integration Facility (ESIF) High Performance Computing (HPC) Data Center and installation of the TSC Initial system modeling Initial results/conclusions. 2

Water-Cooled System Design day is based on the lower WET BULB temperature Evaporative cooling process uses water to improve cooling

3 Air- and Water-Cooled System Options Air-Cooled System Design day is based on DRY BULB temperature Consumes no water (no evaporative cooling) Large footprint/requires very large airflow rates. Water-Cooled System Design day is based on the lower WET BULB temperature Evaporative cooling process uses water to improve cooling efficiency 80% LESS AIRFLOW lower fan energy Lower cost and smaller footprint. Colder heat-rejection temperatures improve system efficiency. However, water-cooled systems depend on a reliable, continuous source of low-cost water. 3

Weather and Load Variations: Opportunities for Hybrid Wet/Dry Solutions Basic principles: Operates wet during peak design periods to save energy (high temperatures and loads) Operates

4 Weather and Load Variations: Opportunities for Hybrid Wet/Dry Solutions Basic principles: Operates wet during peak design periods to save energy (high temperatures and loads) Operates dry during low design periods to save water (lower temperatures and loads) Depending on the design, system might operate either as wet or dry or might be able to operate both wet and dry. 4

5 Basic Hybrid System Concept Dry Heat Out Moist Heat Out Dry Cooler Pump 35.0 C (95 F) 35.0 C (95 F) 32.2C (90 F) 32.2 C (90 F) Dry Sensible Cooler Dry Loop Sized For Water Savings Tower Pump 32.2 C (90 F) Wet Loop Sized For Design Day Thermal Duty 29.4 C (85 F) 35.0 C (95 F) Heat In 29.4 C (85 F) Wet when it s hot, dry when it s not. Condenser Water Pump Process Loop 5

6 Thermosyphon Cooler Out to Tower Process Water In 6

7 NREL Campus Photo by Dennis Schroeder, NREL

8 ESIF HPC Data Center Annualized average power usage effectiveness (PUE) rating of better than 1.06 since opening in 2012 Based on industry/tech trends, committed to direct liquid cooling at the rack o No mechanical chillers o High-power-density racks, more than 60 kw per rack. Holistic approach integrate racks into the data center, the data center into the facility, and the facility into NREL s campus Capture and use data center waste heat: office and lab space (now) and export to campus (future). Two key design parameters: IT cooling supply: 24 C (75 F) on hottest day of year, ASHRAE W2" class IT return water: required 35 C (95 F) to heat facility on coldest day of the year. 8

9 Power Usage Effectiveness Metric PUE is the ratio of the total amount of power used by a computer data center facility to the power delivered to the computing equipment: PUE values cover a wide range for data centers, with an overall average of approximately 1.8. o Data centers focusing on efficiency are achieving PUE values of 1.2 or less. o NREL s HPC Data Center achieved an annualized PUE rating of 1.04 in

Waste Heat (Energy) Reuse Heat energy in the energy recovery water (ERW) loop is available to heat the ESIF s process hot water (PHW) loop through the use of heat exchangers.

10 Waste Heat (Energy) Reuse Heat energy in the energy recovery water (ERW) loop is available to heat the ESIF s process hot water (PHW) loop through the use of heat exchangers. Once heated, the PHW loop supplies: o o o o Active chilled beams to heat office space Air handling units to heat the conference and high-bay spaces Snowmelt loop in the courtyard approaching the ESIF s main entrance District heating loop. 10

11 System Schematic: Original Configuration PUE

12 System Schematic: Current Configuration PUE 1.06 (still) 12

13 System Modification TSC Supply (New) TSC Return (New) ERW Return 14-Inch Valve (New) ERW Supply 13

14 Thermosyphon Cooler Installation TSC (New) Three Towers for Lab/Office Space Four Towers for HPC Data Center Photo by Dennis Schroeder, NREL

System Modeling Program DB & WB Temperatures Operating Cost / Hr Twr Speed = 11.5% Twr Fan kw = 0.09 Twr ACFM = 30,050 Twr Load = 0.3354 Golden, CO Twr Lvg WB = 62.4 % HR Load = 36.9% TSC Speed =54.

15 System Modeling Program DB & WB Temperatures Operating Cost / Hr Twr Speed = 11.5% Twr Fan kw = 0.09 Twr ACFM = 30,050 Twr Load = Golden, CO Twr Lvg WB = 62.4 % HR Load = 36.9% TSC Speed =54.0% TSC Fan kw =3.45 Row = 2331 Twr Lvg DB = 64.7 TSC ACFM =82,277 TSC Load = ,277 TSC Lvg WB =53.7 % HR Load =43.9% WB = 46.4 DB = 57.2 Atmos = GPM = 840. HD (ft) = 37.0 Input KW =7.4 Version: C9 (Note: All Loads are in MW's, All Temperatures are in F) Hourly Annual Total DC Load ,972 Load to Atmos ,748 Bldg Heat Required ,461 Load to Bldg Heat ,223.2 Load to Aux CWS Electrical Energy (kwh) ,293 Water Usage (Gal) ,089 HR System PUE WUE (L/kWh) ERF CDU Load = MW Monday April 7 10AM to 11AM Hourly Costs % Plot Interval ERW Pump $ % 120 $7.00 Hourly TSC Pump $ % TSC Fan $ % $6.00 Operating DB Twr Pump $ % $5.00 Costs Twr Fan $ % WB Water Costs $ % $7.00 Total $ % Max Bldg Load Required = MW DC Load To Bldg = MW % HR Load To Bldg =19.3% 2,185 2,209 2,233 2,257 2,281 2,305 2,329 Hours From Jan. 1st GPM = 188. HD (ft) = 39.0 Input KW =1.8 $4.00 $3.00 $2.00 $1.00 $0.00 Time Water Cost/Hr Total Cost/Hr AHU Load = MW $6.00 $5.00 $4.00 $3.00 $2.00 $1.00 $ ERW Pump TSC Pump TSC Fan Twr Pump Twr Fan Water Costs 76.3 Fan Wall Flow = GPM 67.0 Annual Costs % Elect. Costs $5,411 48% Water Costs $5,950 52% Total Costs $11, % Energy = $0.07/kWh Water = $6.08/kgal Heat Rejection By Device Hourly Annual Bldg Heat TSC Cooling Tower 15

16 Favorable Application Characteristics Any application using an open cooling tower is a potential application for a hybrid cooling system, but certain characteristics will increase the potential for success. Favorable application characteristics: Year-round heat rejection load (24/7, 365 days is best) Higher loop temperatures relative to average ambient temperatures High water and wastewater rates or actual water restrictions Owner s desire to mitigate risk of future lack of continuous water availability (water resiliency) Owner s desire to reduce water footprint to meet water conservation targets. 16

17 Modeling Results: Makeup GPM vs. Dry Bulb Temperature Cooling Tower Make-up GPM Cooling Tower Make-up (l/min) Cooling Tower Make-up Flow Requirements Dry Bulb Temperature ( C) Cooling Tower Only Make-up Hybrid Cooling System Make-up Dry Bulb Temperature ( F) 17

18 Annual Cooling Tower Make-up Volume Modeling Results: Overall Water Use Annual Cooling Tower Make-up Volume Gal 2,500,000 2,000,000 m 3 9,463 7,571 Cooling Tower Only Make-up Hybrid Cooling System Make-up GPM 1,500,000 5,678 1,000,000 3, ,000 1,

19 Controls Allow for Saving Both Water and Operational Costs WECER = Water-to-Energy Cost Equivalence Ratio: WECER = cost of water/cost of electricity WECER = ($/1,000 gal water) / ($/kwh) WECER = kwh/1,000 gal. TSC fan speed = f (WECER and (entering water DB)). 19

20 Modeling Results: Operational Cost Total Hourly Operating Costs Hourly Total Utility Operating Costs Dry Bulb Temperature ( C) $7.00 $6.00 Cooling Tower Only Total Util Costs $5.00 Hybrid Cooling System Total Utility Costs $4.00 $3.00 $2.00 $1.00 $ Dry Bulb Temperature ( F) 20

21 Sample Data: Typical Loads and Heat Sinks 21

22 Early Data: Cumulative Water and Cost Savings 22

23 Conclusions Warm-water liquid cooling has proven very energy efficient in operation. Initial modeling of a hybrid system showed that it was possible to save significant amounts of water while simultaneously reducing total operating costs. System modification was straightforward. Initial data indicate that the system water and operational cost savings are in line with modeling. 23

24 Bibliography Carter, T.; Liu, Z.; Sickinger, D.; Regimbal, K.; Martinez, D Thermosyphon Cooler Hybrid System for Water Savings in an Energy-Efficient HPC Data Center: Modeling and Installation. (LV-17-C005) Presented at the ASHRAE Winter Conference, Las Vegas, NV, January 28 February 1, Additional resources:

25 Thank you! Tom Carter David Sickinger Visit us at Booth 67.

Case Study: Energy Systems Integration Facility (ESIF)

Case Study: Energy Systems Integration Facility (ESIF) at the U.S. Department of Energy s National Renewable Energy Laboratory (NREL) in Golden, Colorado OFFICE HIGH BAY LABORATORIES DATA CENTER 1 Case