Minneapolis Symposium September 30 th, 2015 Fundamentals of CFD and Data Center Cooling Amir Radmehr, Ph.D. Innovative Research, Inc. radmehr@inres.com
Learning Objectives 1. Gain familiarity with Computational Fluid Dynamics (CFD) technique, for simulating the cooling performance of data centers. 2. Understand the factors that influence the airflow motion in the data center and the parameters that affect the cooling of the equipment. 3. Be able to apply simple and cost-effective remedies to eliminate hot spots and control mechanical cooling energy usage. 4. Understand how airflow simulation can be used to design highly efficient (green) data centers. The energy consumption and operating cost of such data centers are substantially less than the traditional data centers.
What is CFD CFD (Computational Fluid Dynamics) is a method of calculating, in great detail, flow fields of any complexity. It gives you the velocity components, pressure, temperature, and other variables at every point in the flow domain. CFD involves placing a large number of points in the domain, setting up equations for the flow variables at these points, and solving these (thousands or millions of) equations. CFD is commonly used in aerospace and automotive applications, combustion chambers, electronics cooling, and chemical industries.
CFD Procedure for Data Centers Construct a computer prototype of the data center Calculate the airflow and temperature distributions Visualize the results Make changes to the model to optimize cooling Implement changes to improve the cooling of the data center, save energy, and reduce operating cost
Ballistic Trajectory of a Projectile
CFD Simulation in a Data Center
Navier Stokes Equations
CFD Grid Grid for a simple geometry Grid for a complex geometry Gas Turbine Blades
Sample CFD Model for a Data Center
Required Data Shape and size of the data center Raised floor and ceiling heights Location and type of cooling units Location and open area of perforated tiles Location and open area of cutouts Location and size of under-floor obstructions Location, orientation, heat load, and airflow of racks Location and size of above-floor obstructions
Results Airflow from perforated tiles Pressure distribution under floor Racks inlet and exhaust temperatures Temperature and airflow patterns in the room CRAC units return temperature
How Accurate Are Our Calculations?
How Accurate Are Our Calculations?
Validation of Results NCEP Data Center Bethesda, MD The measurements were conducted by Dr. Roger Schmidt of IBM and were presented at the 2004 ASHRAE meeting in Nashville, TN
Airflow Rates
Rack Inlet Temperatures
CRAC Return Temperatures
Improve Cooling in an Existing Data Center Model the data center in its current state Identify the causes of cooling problems Model modified layouts Implement changes Prevent equipment failure, save energy, reduce operating cost
Design Highly Efficient Data Centers Make the right decision about Raised floor height Location of the CRAC units Use of return plenum Use of supplemental cooling (overhead ducts, In-row coolers) Layout of the racks Layout of pipes and cable trays Etc.
The Cause of Flow Maldistribution
Initial Validation Measurements by Dr. Roger Schmidt of IBM in a Poughkeepsie data center.
Maldistribution Revisited
Effect of Plenum Height
Effect of Tile Open Area
Use of Perforated Partitions Proposed Locations for Perforated Partitions
Perforated Partitions (80% and 65%)
Above-Floor Two-Part Simulation Strategy Under-Floor Simulation Above-Floor Simulation
A Simple Layout
Rack Inlet Temperatures Heat Load = 88 kw Rack Demand = 10,406 CFM CRAC Flow = 7,500 CFM
Hot Air Recirculation
Increased Cooling Airflow Heat Load = 88 kw Rack Demand = 10,406 CFM CRAC Flow = 10,000 CFM
Rack Inlet Temperatures
Side Recirculation (End Effect)
Create an Air Curtain
Perf Tiles for the Air Curtain
Rack Inlet Temperatures
Effect of the Air Curtain
Partitions at the Ends of the Cold Aisle Heat Load = 88 kw Rack Demand = 10,406 CFM CRAC Flow = 10,000 CFM
Rack Inlet Temperatures
Placement of Partitions
Case Study Simple Solutions for a Complex Problem Heat Load = 720 kw Racks Demand = 32,000 CFM CRACs Flow = 35,000 CFM
Case Study Simple Solutions for a Complex Problem
Airflow Rates from Perforated Tiles
Under-Floor Pressure and Velocity
Airflow Demand vs. Airflow Supplied
Rack Inlet Temperature Distribution
Temperature Distribution at Vertical Planes
Temperature Distribution at 5.5 ft.
Airflow Streams
Airflow Rates Total Airflow Demand = 32,000 CFM Total Airflow Supplied= 35,000 CFM
Modification (Level 1)
Airflow Rates from Perforated Tiles
Rack Inlet Temperature Distribution
Temperature Distribution at 5.5 ft.
Modification (Level 2)
Rack Inlet Temperature Distribution
Temperature Distribution at 5.5 ft.
Alternative Modification (Level 2)
Airflow Streams
Temperature Distribution at 5.5 ft.
Rack Inlet Temperature Distribution
Closing Remarks CFD can be used to simulate the airflow motion and temperature distribution in data centers. Using CFD, the cooling performance of existing data centers can be improved and cooling design of new data centers can be optimized. The accuracy of the CFD results depends on the accuracy of the data provided. In raised-floor data centers, higher plenum height and more restrictive perforated tiles result in more uniform airflow distribution. Simple and cost effective remedies such as closing cable openings or selective use of partitions can tremendously improve the cooling performance of a data center.