Rack Centric - High Voltage Distribution Randy Malik Senior Technical Staff Member Research Triangle Park Raleigh, North Carolina Email: Rmalik@us.ibm.com Data Center Application
Typical Data Center Energy End Use Chiller 100 Units CRAC 35 Units PDU UPS 33 Units Delivered IT Racks 2 10/18/2007
Some Facts about a Typical Data Center Consumption: 10 to 100 times more energy per square foot than a typical office building Electricity Used: 45 billion kwh in 2005 about 1.2% of all retail U.S. Electricity Consumption Power Demand : Expected to double in 10 years Average Utilization : 20 25% of the capacity Overall Efficiency = IT Power / Power Input = 30% Carbon Footprint: Coal Fired Plants = 2 Lbs/KWh Natural Gas = 1 Lb/KWh Diesel = 2.19 Lbs/KWh Source: Emerson Corporation 3 10/18/2007
Energy Consumption by Office and Telecom Equipment in Commercial Buildings 12 10 10.2 8 6 4 2 0 1.1 1.5 0.15 Servers Routers Storage WAN Switches 2.3 LAN Switches 1.6 HUBs Electricity Use TWh/year Source: Roth K. Goldstein, F. Kleinman J(2002) Energy Consumption by Office and Telecommunication Equipment in Commercial Buildings 4 10/18/2007
Data Center Efficiency considering NCPI Losses Data Center Power Path to IT UPS PDU Cabling Switches Power to IT Power to Data Center Cooling IT Equipment Useful Output Power To Lights Secondary Support Generator Switchgear NCPI Data Center Efficiency = Power to IT Data Center Power IN 5 10/18/2007
Power Consumption in a Data Center PDU, 3% IT, 25% UPS, 9% Lighting, 1% Wiring, 1% CRAC, 9% Humidifier, 1% Power Supplies, 23% Chiller, 28% UPS PDU Lighting Wiring CRAC Humidifier Chiller Power Supplies IT 6 10/18/2007
Power Pathway to Data Center Efficiency Create a strategy for enhancing Tech utilization 1.Server consolidation and virtualization 2.Grid Computing 3.High Efficiency Processors 4.High Efficiency Server Power Supplies Size the Power System for Optimum Efficiency 1. Modular Approach Consider alternate distribution methods 1. 600V AC to 208V AC to the racks 2. 480V AC direct to the racks 3. 415V AC to 240V AC to the racks 4. 480V AC to 380VDC within the racks 7 10/18/2007
Efficiency of A typical Data Center 45 40 35 30 Efficiency 25 20 15 10 5 0 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Fraction of the Power Capacity Used Efficiency 8 10/18/2007
Efficiency of UPS & Power System as a function of Load 95 90 85 Efficiency 80 75 70 65 60 5% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % Load of Full Power Rating UPS Power System UPS Source : Eaton Corporation 9 10/18/2007
Power Consumption by a Processor during Active and Idle Mode 150 Modified Power Consumption Power Consumption Watts 100 50 0 Processor Activity Note: Lower Activity by the processor does not mean lower power consumption 10 10/18/2007
A typical Work Load for a Data Center for a 24 hour period Time of Day Work load 45 40 35 30 % Traffic 25 20 15 10 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Source: 2006/2007 Discussions with several NYC Financial Services firms 11 10/18/2007
Electrical Energy Cost of Powering a Data Center Annual Electrical Cost of a 1 MW data center ( $0.10/kWh) Annual Electrical Cost (M$) 2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 NCPI Load IT Load 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Fraction of Power Capacity Used Annual IT Electrical Cost Annual NCPI Electrical Cost Source: APC 12 10/18/2007
Drivers for Rack Centric - High Voltage Distribution Efficiency No UPS Cost High Voltage Drivers No PDU Cooling Modularity 13 10/18/2007
Power Path for IT Racks in a Typical Data Center 9% 3% IT Racks PDU 208 /120V UPS 480V AC IT PDU 208 /120V 14 10/18/2007
Typical Power Distribution for Blade Server Racks Server Blade Chassis System Rack +12V-A AC - DC PSU 3 PSU 1 PSU 3 PSU1 +12V-B +12V-A +12V-B PSU 4 PSU 2 PSU 4 PSU 2 AC Distribution Box PDU - 1 PDU - 2 208V AC 208V AC 15 10/18/2007
High Voltage DC Distribution in a Data Center Rectifier 480V AC Rectifier 380V DC Rectifier Battery Plant Rectifier IT Racks Source: Lawrence Berkley National Laboratory 16 10/18/2007
High Voltage DC Distribution in the Rack 3 Phase 480V AC 480 AC 208/120 AC UPS = 9% PDU = 3% 1% IT Rack 1% 380V 12V Plug in Module Server Blade Chassis 3 Phase 480V AC 3 Phase 480V AC AC DC Rectifier AC DC 380V DC Server Blade Chassis Server Blade Chassis Data Flow Rectifier Server Blade Chassis IT Rack Arrangement 17 10/18/2007
Three Phase Single Stage Series Resonant Cyclo Converter Source: Tomasz Kotula Eaton Development Centre 18 10/18/2007
380V - 12V Regulator Source: Vicor Corporation, Andover, USA Power Density = 1000W/in 3 EMC Filter + Boost + Bulk Cap Chopper up HDD Plug in Module 380V 12V VRD up Memory 380V 12V DC = 100W/in 3 Source: Astec/Artesyn Server Blade 19 10/18/2007
Rack Centric High Voltage Distribution Bus Proposal System Rack Server Blade Chassis Server Blade Chassis 380V DC Bus A AC DC Rectifiers Server Blade Chassis Blade Server 380V DC Bus B Battery Back Up 3 Phase 480V AC1 AC DC Rectifiers AC DC Rectifiers 3 Phase 480V AC2 20 10/18/2007
Lower Overall Cost Higher Overall Efficiency IT, 25% UPS, 9% PDU, 3% Lighting, 1% Wiring, 1% Lighting, Wiring, 1% CRAC, 9% Humidifier, 1% CRAC, 9% IT, 38% Humidifier, 1% Chiller, 28% Power Supplies, 23% Chiller, 28% Power Supplies, 23% UPS PDU Lighting Wiring CRAC Humidifier Chiller Power Supplies IT Lighting Wiring CRAC Humidifier Chiller Power Supplies IT 21 10/18/2007