RACK INFRASTRUCTURE EFFECTS ON THERMAL PERFORMANCE OF A SERVER

Transcription

1 RACK INFRASTRUCTURE EFFECTS ON THERMAL PERFORMANCE OF A SERVER By KC Coxe Dell Enterprise Thermal Engineering dell.com/hiddendatacenter

2 EXECUTIVE SUMMARY This paper focuses on quantifying the impacts of a Cable Management Arm and the doors of an IT equipment rack on the airflow through a 1U server. While Dell has historically tested rack-mounted servers and storage equipment in an IT rack, the results were used as a pass/fail criteria for qualifying the thermal design of the server. The impact of individual components of the rack deployment (Doors, CMA, Cabling, etc.) was not, however, individually documented. A combination of customer inquiries, a desire to validate design standards, and a limited body of readily available literature on the topic sparked interest in a methodical approach to documenting these effects. The analysis that follows is based on a Dell PowerEdge 1950III server, a 1U server that uses front-to-back cooling, housed in a Dell PowerEdge 4210 rack. The empirical data collected by Dell in this study to illustrate the interactions between the rack infrastructure and the server showed that the effects of the rack doors and CMA individually were very similar, each in the 3% range, and did not have a concerning impact on the performance of the server being studied. BACKGROUND Best practices for thermal optimization of data center layout and server deployments at the room level have been covered in detail in recent literature. Among the many papers that have been published, the Uptime Institute has published summaries by Sullivan on Hot/Cold Aisle data center layouts [1] and Sullivan et al. on reducing bypass airflow in data centers [2]. The Green Grid has also published a large body of work including a paper edited by Jones that included air management at the data center level as one of their key strategies for improving data center cooling [3]. There is a middle ground between cooling the servers (or other IT equipment) in a data center and the actual data center layout that has infrequently been addressed: the rack-level environment. Within the rack itself is an environment that requires care to manage properly. Jones publication does discuss bypass air within the rack, calling for internal contamination paths to be sealed. These include gaps between servers as well as the zero-u space between the equipment and the side panels. Other factors within the rack, however, can still affect the cooling of the deployed equipment. The focus of this paper will be on quantifying the effects of cable management and perforated rack doors on servers, for which there is a much smaller body of available work. 2

3 Rubenstein [4] published an analysis on the effects of cable management on a 4U server in which he measured a 2% airflow loss in the server due to cabling. Also, Rittal [5] has published a whitepaper on their website exploring the effects of rack door open area on the door s impedance, and concluded that rack doors would have only a very small effect on the equipment in the rack, and that open area greater than 63-64% show little return for the added open area. The additive effects of these factors have not been broken down and applied to a server, or other piece of IT equipment, and that is the motivation for this study. The insights from this study, detailed in the following pages, are that the effects of the rack doors and cabling can be measured and these components can affect the thermal performance of the server. Individually the effects were small, but when compounded, a theoretical airflow loss of up to 6% was identified. As Rubenstein pointed out in his work, the study of these effects against a single piece of equipment may not necessarily be generalized against all hardware. System fan performance, system resistance, cable deployment, and rack selection, among many other factors, can all affect the results of this type of analysis, and results should be expected to vary with different hardware. Dell s mitigation to ensure customers experience minimal if any performance degradation from these effects is qualification of all rackable equipment in a data center environment; this paper helps explain some of the individual contributors to the gross thermal impacts of server deployment. EFFECTS OF RACK DOORS Much of the analysis in this paper will compare airflow vs. pressure characteristics of a Dell PowerEdge 1950III server with the airflow impedance of the rack hardware. The server will be represented as fan performance curve against the impedance curve of the rack. The server fan curve was created on an airflow bench by measuring the air through the server and then adding backpressure using a gate on the flowbench (Dell uses a flowbench designed in accordance with ACMA Standard ). The impedance of a rack door can be overlaid on this plot to estimate the airflow loss due to the added pressure of the rack door on the server. To measure the impedance of a rack door, a Dell PowerEdge 4210 rack door was attached to the flowbench, and the impedance of the door was measured, followed by the impedance of the flowbench fixture without the door. The difference of these two curves can be assumed via superposition to be the impedance of the rack door. Figure 1 shows the rack door impedance. Figure 2 shows the server airflow curve overlaid with the rack door curve. Note that in Figure 2, the rack door curve has been normalized to a per-u basis. The curve for 2 rack doors is simply the single door curve with twice the impedance (assuming air impedances add when the obstructions are in serial in the airflow). 3

4 Figure 1: PowerEdge 4210 Rack Door Impedance As shown in Figure 2, the airflow through the PE1950III without the added backpressure from the doors is approximately 35 CFM. The theoretical operating point when the doors are added upstream and/or downstream of the server in a rack deployment are found on the plot where the PE1950 flow curve crosses the rack door impedance curves. In this case, we would expect approximately a 1 CFM airflow loss due to a pair of rack doors, or about 3%. 4

5 Figure 2: Rack Door Impedance Compared to PE1950III Airflow Curve (Normalized per U) EFFECTS OF CABLING AND CABLE MANAGEMENT ARMS This section of the analysis will quantify the theoretical impact that a cable management arm will have on the airflow through the 1U server being evaluated. In comparison to the Rubenstein study, which approached the problem with the perspective of impedance addition to the system caused by the Cable Management Arm (CMA) and associated cabling, this study will view the server as an air source and examine how much the CMA is expected to reduce the airflow. The airflow measured through the PE1950III in the previous section can be used as the airflow supply curve. Again, approximately 35 CFM was measured through the server with no added backpressure. 5

6 To measure the impedance effects of the cables and management hardware, 4 PE1950III CMA s were configured in a Dell PowerEdge 4210 Rack. The CMA s were attached to rack rails and a duct was constructed around the 4U CMA installation. This duct was then affixed to an AMCA flowbench, which would function as the air source for the measurement. Air was drawn through the CMA and impedance measurements recorded for several conditions including: CMA Only with no cables (Figure 3) CMA + 1 Power, KVM, and 2 Ethernet cables CMA + 2 Power, KVM, and 4 Ethernet cables (Figure 4) Air duct only with CMA removed The Keyboard, Video, Mouse (KVM) system that was used was a single USB interface to the server for keyboard and mouse, plus a VGA connector for video. Both connections are part of a dongle with an Ethernet port that connects to a KVM switch located elsewhere in the rack. Impedance curves were taken of the 4U assembly across a flow rate of CFM (10-70 CFM per U). For conditions with cables installed the ends of the cables that would normally plug into the server were free-floating, but constrained within the 1U height of the server by the CMA tray and the duct (Figure 4). The duct that joined the racked hardware to the flowbench was measured separately, and subtracted from the raw measurements of the hardware via superposition to produce the plotted results. Figure 3: CMA s in measurement duct on flowbench Figure 4: Cabled CMA s in measurement duct on flowbench Impedance results from the CMA study are shown in Figure 5, along with the server airflow curve discussed previously. From this relationship we can see that not only does the CMA affect the air resistance applied to the server, but the number of cables plays a part as well. These results would predict approximately a 4% airflow loss (roughly 1.5 CFM) for the worst measured case of 2 power cords, 4 Ethernet cords, plus KVM. For the more moderate case of 1 power cord, 2 Ethernet cords, plus KVM the predicted airflow loss is 3%, or 1 CFM. 6

7 Figure 5: PE1950 Fan Curve Overlaid with CMA and Cabling Impedance ADDING UP THE EFFECTS While the individual effects on airflow of rack doors and cable management may be small, the analysis can be carried forward to build a model of the entire rack environment. Working under the principal that airflow impedance adds in series when resistances are in series in the airflow, the curves from Figures 2 and 5 can be added to estimate the impedance of: A front door on an IT rack A rear door on an IT rack A CMA plus cables deployed in an IT rack. These 3 items, plus a full infrastructure impedance curve, are shown in Figure 6 with the PE1950III airflow curve. Among the key observations in Figure 6, it is notable that the effects of the rack doors and the cables plus management are roughly equivalent effects in this deployment. Individually, the rack doors and the CMA incur a 3% airflow loss on the server, with the cumulative loss about 6%, or 2CFM in this case. 7

8 Figure 6: Impedance of Rack Impedance Components and Summation of Full Infrastructure SUMMARY In this paper, several factors were explored that can affect IT equipment in a rack to understand how they can individually and cumulatively alter the performance of said equipment. A key observation was made that the rack doors and the cable management each imposed approximately a 3% airflow reduction on the studied 1U server. The 6% cumulative airflow reduction had a small but measurable effect on the temperatures of components within the system. The thermal design of the servers was however sufficiently robust, such that system components met their temperature specifications despite the airflow losses associated with the rack doors and cable management. 8

9 REFERENCES [1] Sullivan, Robert F. Alternating Cold and Hot Aisles Provides More Reliable Cooling for Server Farms. Whitepaper from uptimeinstitute.org/. Downloaded on October 9, [2] Sullivan, Robert F., Lars Strong, and Kenneth G. Brill. Reducing Bypass Airflow is Essential for Eliminating Computer Room Hot Spots. Whitepaper from Downloaded on February 4, [3] Jones, Rich, ed. Seven Strategies to Improve Data Center Cooling Efficiency. The Green Grid Whitepaper #11. Downloaded on February 4, 2009 [4] Rubenstein, B (2008). Cable Management Arm Airflow Impedance Study. Proceedings of 11th Intersociety Conference on Thermal and Thermomechanical Phenomena in Electrical Systems (ITHERM 2008), May 2008, Orlando, FL. pp [5] Rittal Website. White Paper 504; The Realities of Airflow Through Perforated Doors. Downloaded on October 9,