Using CFD Analysis to Predict Cooling System Performance in Data Centers Ben Steinberg, P.E. Staff Applications Engineer

Transcription

1 Using CFD Analysis to Predict Cooling System Performance in Data Centers Ben Steinberg, P.E. Staff Applications Engineer

2 Agenda What is CFD? Applications of CFD Applicability to the Data Center How it works Examples

3 What is CFD? CFD stands for Computational Fluid Dynamics Fluid Dynamics is the study of the motion of fluids (liquids and gases) Complex equations that describe fluid flow and heat transfer were developed hundreds of years ago For simple systems, these equations can be solved manually

4 Applications of CFD Computers make solving these equations possible for very complex systems. Developed in the 1960 s for aerospace, defense and nuclear power industries. Usage grew into the automotive industry in 1970 s. CFD is used as a virtual wind tunnel for designing and testing airplanes, cars and ships before a prototype is even built. Even used by hospitals to simulate flows of fluids through the human body.

5 Applicability to the Data Center In 1965, Intel co-founder Gordon Moore predicted that the number of transistors on a chip would double about every 2 years. A musical birthday card costing a few U.S. dollars today has more computing power than the fastest mainframes of a few decades ago. A single rack of servers can now consume as much power and create as much heat as 3 household ovens!

6 Increasing Heat Densities Per Rack Power Density (KW) Max per-rack IT Load Data Center rated rack power capacity '95 '96 '97 '98 '99 '00 '01 '02 '03 '04 '05 Year IT Loads are greatly exceeding rated capacity!

7 Survey Question #1 Do you currently have hot spots in your (or a customer s) data center? Yes No

8 Applicability to the Data Center: Past In general, the heat load of a data center was (and still is) about equal to the power consumption of the IT equipment, in addition to other miscellaneous loads (lighting, solar, etc.) In the past, ensuring that the installed cooling capacity was at least as much as the heat load was enough to cool the IT equipment. Raised floors and overhead ducts were the most common types of air distribution. Perforated tiles in raised floors or supply grilles in overhead ducts were placed as necessary to supply cool air to the equipment inlets.

9 Applicability to the Data Center: Present (and Future) Today s high density loads require so much cooling and airflow (up to 7 tons and 2,400 CFM per rack) that it is difficult to supply the correct quantities of cool air where it is needed (see next slide). Hot spots are created in areas where not enough cool air is supplied. Reactions to hot spots include lowering CRAC set points and adding more CRAC units, both of which can make things worse. Every data center is unique: - Size, shape, type of equipment - Changes very frequently Very difficult to predict performance and level of redundancy.

10 Floor Tile Cooling Ability Typical Capability cfm Perf tile With Effort Requires careful raised floor design, careful CRAC placement, and control of under-floor obstacles (pipes/wiring) Grate tile Extreme Impractical Rack Power (kw) that can be cooled by one tile with this airflow 2 Blade Servers Standard IT Equipment [47.2] [94.4] [141.6] [188.8] [236.0] [283.2] [330.4] [377.6] [424.8] [471.9]

11 How it works Applicable to existing as well as planned installations. All equipment is entered into a 3D computer model - Physical size - Power and cooling capacity -Air flow When the model is complete, the computer solves the equations. - could take hours or days, depending on size and detail. After solving, results can be manipulated to show temperatures, pressures and flows in different locations. Model should be maintained as equipment changes take place.

12 CFD Examples

13 Raised floor cooling CFD example Design Parameters: 4kW per rack / 40 racks total Hot Aisle/Cold Aisle layout Total power 160kW Room size: 38' x 38' x 14' high 2' raised floor w/25% perforated tiles in front of racks and PDUs 5 CRAC units N+1 redundancy at room level - 4 units running at the same time = 160kW CRAC units are standard chilled water 40kW (12 ton) downflow

14 Raised floor cooling CFD example HOT AIR HOT AIR COLD AIR/ PERFORATED TILES COLD AIR/ PERFORATED TILES COLD AIR/ PERFORATED TILES

15 Raised floor cooling CFD example Redundancy: Any unit failure results in loss of cooling to an area N+1 at room level does provide adequate cooling in failure modes Sectional 5-6 from Raised Floor

16 Survey Question #2 What is your (or your customer s) highest rack density today? Less than 1 kw per rack 1-2 kw per rack 3-4 kw per rack 5-6 kw per rack More than 6 kw per rack Not sure

17 Hard floor cooling CFD example Design Parameters: 4kW per rack / 40 racks total Total power 160kW Room size: 38' x 38' x 14' high Raised floor not used for air distribution 6 in-row cooling units N+1 redundancy at row level - 4 units running at the same time = 160kW

18 Hard floor cooling CFD example InRow Architecture NetworkAir IR NetworkAir IR COLD AIR HOT AIR NetworkAir IR COLD AIR HOT AIR NetworkAir IR COLD AIR NetworkAir IR NetworkAir IR

19 Hard floor cooling CFD example InRow Architecture Redundancy: Normal Operating Mode 4 units running No Hot Spots developed in front of IT equipment. Sectional 5-6 from Datacenter Floor

20 Raised floor cooling CFD example Design Parameters: 2kW per rack Not Hot Aisle/Cold Aisle layout Total power 138kW Room size: 53' x 41' x 9' high 2' raised floor w/25% perforated tiles in front of racks 4 CRAC units N+1 redundancy at room level - 3 units running at the same time = 150kW CRAC units are glycol cooled DX 50kW (15 ton) downflow

21 Temperature Distribution Top View, top half of racks

22 Temperature Distribution Side View Cool air from floor plenum is consumed by bottom half of racks; top half is left to ingest hot air from recirculation.

23 Temperature Distribution Top View, top half of racks New CRAC unit is added; note the cooler temperatures around the room.

24 Under floor pressure & flow CFD example 1 foot raised floor height 2kW per rack Analysis of 2 different CRAC layouts

25 Temperatures Scenario 1 Scenario 2

26 Plenum Airflow Raised floor hidden for clarity. Scenario 1 Scenario 2

27 Plenum Pressure Raised floor and IT racks hidden for clarity. Note dead spots at center of recirculations. Scenario 1 Scenario 2

28 Perforated Tile Flow Rates flow rate per tile (cfm) speed (ft/min) Very Non-uniform distribution BACKFLOW here Scenario 1 Scenario 2

29 Conclusions As equipment power densities rise, so does the amount of heat produced. Very difficult to accurately predict performance and redundancy at high densities. Recent advances in computer power and software have made CFD a viable tool to help fix or prevent hot spots from occurring in the data center.

30 Using CFD Analysis to Predict Cooling System Performance in Data Centers QUESTIONS

31