High Voltage Direct Current Power Solutions for Data Centers Professor: Dr. Danny Silver Acadia University Student Name: Jim Nasium Published: April 10 th, 2011
1. Introduction As the demand on data centers increase, the required processing power draws a substantial amount of power. Most data centers still operate on standard 120-240 volts alternating-current (VAC), but research and progress has been made to mitigate the demand for electricity by redesigning the way power is carried to the servers. By distributing electricity to the servers using direct current, significant power savings can be attained. This paper will give a brief history of the modern power grid before delving into the present research and results of using direct current in data centers. The work required to standardize high-voltage direct current (HVDC) in data centers and the power sources best suited to supply this power will follow the research results. It should be noted that a significant portion of the results in this paper are directly associated with research performed at the Lawrence Berkely National Laboratory as part of its High- Performance High-Tech Buildings project 1. 2. History In the 1800 s when electricity was in its infancy, there was a great debate between the viability of direct current versus alternating current. Thomas Edison s direct current grid proved to be challenging to distribute over long distances. Small generating stations were built every few blocks because greater distances proved to have too be inefficient and power losses too high. Early generators were limited to low voltages 2, which resulted in a high current and high losses. Nikola Tesla s Alternating current system proved to be efficiently distributed over long distances, making it the obvious winner. It wasn t long 1 LBNL (2007) 2 EPRI (2006)
before city streets were lined with electric lamps and homes furnished with incandescent bulbs and electric heating. 3. Present Grid The present grid is an elaborate web of power lines feeding every home and industry. While lighting, heating, and motors are still the primary consumer of electricity, electronics are becoming ever more thirsty. Electronics are unique from other devices because they require direct current to operate. This poses an issue that hasn t been considered since the advent of the grid. To feed electronics its direct current, switchedmode power supplies 3 (SMPS) convert alternating current to direct current and drop the voltage to the required level. Consumers recognize these devices as the blocks called power adapters for laptops, or the power supply attached to a power cable on a desktop computer. By 2004, 2.5 billion SMPS were supplying direct current to devices of all shapes and sizes in the United States alone. The power required for these devices totaled 207 billion KWh, or 6% of the total electricity demand in the United States. 4. AC Power in Data Centers Data centers are built for the sole purpose of housing large arrays of servers. These servers are the backbone for almost every large institution today. The power required for a single data center can surpass 15 MW 4. In California, data centers use 14.6 TWh in a year 5 and rising. Electricity enters the facilities at either 600 VAC or 480 VAC before 3 EPRI (2006) 4 EPRI (2006) 5 LBNL (2007)
getting stepped down to either 208 VAC or 120 VAC for use by servers, lighting and more. Even though alternating current is pumped into the building and servers are plugged into standard alternating current, it is not the sole intermediary. Six or more conversions are required between AC and DC from the grid to the server 6. The first conversion takes place at the uninterruptable power supply (UPS). The UPS provides a grace period for the generators to kick in and are in series between the grid and the servers in order to maintain a constant flow of electrons. It is comprised of an inverter, a switch, controls and batteries. Batteries are inherently DC, so the inverter converts the grid s AC to DC for the batteries. 7 The electricity is then converted back to AC for distribution to the racks. At this point, some racks convert the AC to DC for use in the servers, but as discovered at the Bell Data Center in Halifax, this is still not common place. Instead, every server has an internal power supply that converts the electricity back to DC of select voltages for internal components. 4.2 Losses in AC Data Centers Converting electricity between AC and DC is an inefficient process because even the best inverters are only 90-95% efficient 8. Approximately 12-16% of all energy consumed by servers are wasted in the conversion between AC and DC 9. Figure 1 illustrates the severity of the loss within the server itself, since 131 watts of a 450 watt server is lost simply in the conversion between AC and DC. It is the single largest user of power in the server. In a 10-15 MW data center, 2-3 MW may be lost by power conversions alone. 6 LBNL (2007) 7 EPRI (2006) 8 LBNL (2007) 9 EPRI (2006)
The energy that is lost in conversion is lost to the surrounding environment in the form of heat. Removing the heat requires fans within the servers as well as large ventilation and Figure 1 Power for a typical server consuming 450W (Leonard Energy, 2008). Figure 2 Typical data center power distribution (Leonard Energy, 2008). cooling units. Figure 2 shows how the cooling unit consumes almost as much electricity as the servers themselves, at about 33% of all electricity. As mentioned earlier, the UPS requires two conversions between AC and DC, which draws 18% of the power. The heat lost from these conversions contribute to the 33% of electricity required for the chillers. 5. DC Power in Data Centers Technology is emerging that could allow data centers to be designed to run exclusively on DC power. By going DC, data centers could reduce their losses by 10-20%. 10 In the present era of blade-servers, data centers are faced with a growing challenge of powering and cooling these tightly packed machines. Direct current offers a long-term solution to the power density issues related to the emission of heat from power supplies. 10 EPRI (2006)
The results from the Lawrence Berkeley National Laboratory are presented in Tables 1 & 2. Table 1 compares two AC systems to one DC system, all of which containing a top-ofthe-line, high-efficiency UPS, transformer, and server power supplies. Table 2 gives a realistic evaluation because the equipment used in the test are closer to the equipment used in a typical data center. As seen in the tables, a high-efficiency DC data center will show a 7% improvement in efficiency over a high-efficiency AC data center. In comparison, a standard-efficiency DC system showed a 28% improvement over a standard-efficiency AC system. Table 1 High Efficiency AC vs. DC Efficiency Comparison (LBNL, 2007). Table 2 Typical AC vs. DC Efficiency Comparison (LBNL, 2007).
5.2 Rack-Level DC One way to reduce the heat produced within the racks is to have one inverter in each rack that converts 208/120 VAC to 380 VDC for all the servers contained within it. 11 This concept is great for business who are still using AC because it allows for the gradual migration to a DC environment as more racks are installed in the facility. While the racks remove the heat dissipation dilemma within the racks, nearly the same amount of heat must be carried by chillers to the outside environment. Furthermore, the UPS system remains shrouded by an AC environment where multiple AC/DC conversions are required. Eventually the majority of the data center will contain servers designed to operate on DC, at which point it would be a simple process to flip the center to a facility level DC system. 5.3 Facility-Level DC A facility-level DC system is the most efficient design because only one AC/DC conversion is required. In this design, 480 VAC is converted into 380 VDC before reaching the UPS and distributed throughout the facility 12. Potentially, the one inverter could be placed in a weather-proof enclosure and placed outside so less cooling is required within the building. The DC system is potentially more reliable than AC because there is only one inverter to maintain (plus a redundant inverter), as opposed to over the six conversions required in an AC setup. From an electrical engineering standpoint, distributing DC power eliminates harmonics and concerns surrounding AC s power factor in servers. 13 These terms can best be explained as occurrences centered around the 11 LBNL (2007) 12 LBNL (2007) 13 Leonard Energy (2008)
conversion of AC to DC where the quality of AC is diminished as demand from the inverters grow. 6. Integrating Alternative Energy Integrating alternative DC energy sources such as photovoltaic panels, and fuel cells can be integrated into the system without any AC/DC conversions required between the power source and the server. As a result, the clean energy source becomes even more efficient by not requiring the use of an inverter. Traditionally, these alternative power sources would require an inverter to convert the DC to AC 14. The inverters would cost 25% of the installation and would need to be replaced every 5-10 years. With no inverter required, these alternative energy sources become more viable for data centers and can reduce cost, increase reliability, and increase the useable power produced by them. 6.1 Solar Panels Solar panels may be used to supplement the power from the grid. While they would not be viable to provide the sole source of electricity to a data center, they could significantly reduce the building s carbon emissions. Not only would this help the business save its carbon credits, photovoltaic arrays can prove to cut down the energy costs for a building if installed in areas of frequent and intense solar radiation. Figure 3 illustrates the setup of a photovoltaic array 14 EPRI (2006) Figure 3 Photovoltaic integration (EPRI, 2006).
installed without the use of an inverter. Instead, a DC/DC inverter (which far surpasses an AC/DC inverter for efficiency), is implemented to convert the output voltage of the array to the input voltage of the UPS. 6.2 Fuel Cells Bloom Energy recently unveiled a fuel cell that has a net efficiency of over 50%. 15 It creates electricity from gas such as Natural Gas or Biogas as its fuel source. Its current model has a rated output of 100kW at 480 VAC. The modular design allows for easy expansion to multiple units. While it may seem outlandish to think that fuel cells are affordable enough to compete with grid power, over 25 companies have bought Bloom Boxes- including Google and Ebay. As seen in Figure 4, a significant portion of the box is dedicated to Figure 4 Bloom Box (Bloom Energy, 2010) converting the DC power produced by the fuel cells to AC. If a model were available that replaced the AC/DC inverter with a DC/DC inverter, it would increase the net efficiency of the Bloom Box considerably. 7. Advancements Required for DC Implementation While there are servers available at 48 VDC because of the standard 48 VDC telecommunications equipment, there is currently no 380 VDC native servers on the market. 16 Part of the challenge is that there is no standardization agreed upon by the 15 Bloom Energy (2010) 16 LBNL (2007)
industry. After agreeing upon a distribution voltage, a power cord and plug must still be agreed upon. Circuit breakers designed for 380 VDC are scarce because of the lack of demand for them, and very few have been certified by UL. 8. Conclusion Building or retrofitting a data center to run its UPS and servers exclusively on DC has proven to be possible and economically viable. A data center running a HVDC system can save upwards of 10-20% on its electricity bill. Not only is there less of an initial investment because of the fewer parts required in a DC system, but the elimination of AC/DC inverters means there is less work for the chillers. If a data center is already built using AC infrastructure, new racks should be built with DC servers installed. After all the servers are replaced with DC servers, or if a new data center is being built, a facility-level DC system should be considered. If the demand is there, new renewable energy sources (such as the Bloom Box) will rise to the challenge of providing an entirely DC data center. The more demand there is for DC servers, the more manufacturers will produce DC equipment. References: Bloom Energy. (2010). Product Data Sheet. ES-5000 Energy Server. Downloaded from http://www.bloomenergy.com/products/data-sheet/. Page viewed 2011-04-09 Electric Power Research Institute. (2006). An EPRI White Paper. DC Power Production, Delivery and Utilization. Jong & Vaessen, Leonardo Energy. (2007). Briefing Paper. DC Power Distribution for Server Farms. William Tschudi, Lawrence Berkeley National Laboratory. (2007). DC Power for Improved Data Center Efficiency.