The End of Redundancy. Alan Wood Sun Microsystems May 8, 2009

Transcription

1 The End of Redundancy Alan Wood Sun Microsystems May 8, 2009

2 Growing Demand, Shrinking Resources By 2008, 50% of current data centers will have insufficient power and cooling capacity to meet the demands of highdensity equipment. ---Gartner * Energy bills traditionally have accounted for less than 10% of an overall IT budget but soon could account for more than half. --- Gartner * IDC estimates that 60% of data centers are already out of power, space, and cooling. *Source: Page 2

3 Page 3 EPA Report to Congress on Server and Data Center Energy Efficiency Data center energy more than doubled from 2000 to The power and cooling infrastructure accounts for 50% of data center total energy consumption. The energy used by the nation s servers and data centers in 2006: > 61 billion kilowatt-hours (kwh) > 1.5% of total U.S. electricity consumption > Total electricity cost of about $4.5 billion > Equal to 5.8 million average US Households Source: EPA Report to Congress on Server and Data Center Energy Efficiency

4 Page 4 Agenda Power costs more than hardware (Lifetime volume server power consumption costs more than the server) What industry is doing to lower power consumption The potential impact on dependability How the dependability community should respond

5 Page 5 When OPEX Exceeds CAPEX Electricity OPEX as % of Server CAPEX Over Server Lifetime of 4 Years 300% Electricity OPEX as a % of Server CAPEX Over Server Lifetime of 4 Years 250% $0.22/kWh Electricity as % of Server Cost 200% 150% 100% Server Costs $3,000 $0.15/kWh $0.12/kWh $0.09/kWh 50% 0% Average Server Watts Consumed

6 Page 6 The Cubing Effect Trend 3: Add Increase in electricity cost at 12% per year 1 US$ (Billions) Trend 2: Add Increase in per server power consumption at 16% a year 2 Trend 1: Increase in number of installed servers growing at 14% a year Baseline: $18.5 Bn 4 By 2012 data center power consumption costs could grow to $250B worldwide demanding proactive energy management solutions 1.U.S. Energy Information Administration ( 2.Sun primary research 3.IDC#34867 U.S. and Worldwide Server Installed Base Forecast (February 2006) 4.IDC Worldwide Server Power and Cooling Expense Forecast

7 Page 7 Data Center Power Flow Utility UPS Substation UPS Subsystem Critical CRAHs/ Fans on UPS Pumps on UPS PDU Computer Equipment Load Transformer Mechanical Yard Battery Chillers Primary Switch Gear Pumps Fans Engine Generator Mechanical Substation Misc Building Heat

8 Page 8 Data Center Voltage Regions Utility UPS Substation UPS Subsystem Critical CRAHs/ Fans on UPS Pumps on UPS PDU Computer Equipment Load Transformer Battery Chillers Primary Switch Gear Pumps Fans Engine Generator Mechanical Substation Misc Building Heat 2,500/12,000 Volt Region 480 Volt Region 110/208/230 Volt Region

9 Page 9 Typical Power Flows in a 15MW Data Center Utility Transformer Primary Switch Gear Engine Generator 15 MW 95% (-800 kw) 14.2 MW 98.5% (-200 kw) 14.2 MW 9 MW 5 MW Legend Efficiency Rating (Actual Power Lost) UPS Substation 95.5% (-400 kw) 95.5% (-250 kw) Mechanical Substation 8.4 MW 0.2 MW UPS Subsystem 86.5% (-1,150 kw) 8 MW Battery 4.75 MW 0.8 MW 0.2 MW Critical CRAHs/ Fans on UPS Pumps on UPS 6.25 MW 2.5 MW 0.75 MW 0.8 MW 0.2 MW 0.5 MW PDU 95.5% (-250 kw) Misc Computer Equipment Load 6 MW Chillers Pumps Fans Building Heat

10 Page 10 Data Center Efficiency Metrics DCIE: Data Center Infrastructure Efficiency PUE: Power Use Efficiency Emissions Factor Power That Reaches the IT Equipment Total Power to the Data Center Total Power to the Data Center Power That Reaches the IT Equipment > CO 2 emission factor = the average emission rate of carbon dioxide for a given source of energy Σ (CO 2 EF for each source)x(kwh of energy delivered by that source) All Sources Σ (Total kwh delivered by the utility)

11 Page 11 Eco in the Data Center Reconciling Conflicting Goals Last Year... AVAILABILITY AND PERFORMANCE MATTER Last Month... ENERGY EFFICIENCY MATTERS The Truth... THEY ALL MATTER This is a complex relationship, not an either/or situation IT equipment serves critical functions

12 Page 12 What is Industry Doing? At the data center level > Economizers, air-side and water-side > Better data center layouts > More efficient power and cooling > Consolidation/Virtualization > Turn off everything not being used > Throttle to match required performance At the server level > Throttle/idle to match required performance > New architectures to allow idle CPUs and memory > Fan and power supply efficiency and settings > New architectures, e.g., flash as a memory tier

13 Page 13 Pods with Hot Aisle Containment Overhead Power Busway In-Row Refrigeration Unit Contained Hot Aisle

14 Page 14 What is Industry Doing? - 2 At the chip level > Lower power, e.g., multi-core, multi-threaded microprocessors > Decreased leakage current > Throttle to match required performance (idle/sleep states) At the OS level > Virtualization > Power-aware resource management and scheduling At the application level > Very limited research

15 Page 15 The Energy Efficient Stack Monitoring, Management, and Services Operating System Virtualization Layer Platform Microprocessor Energy efficiency through giving customers visibility into actual server power consumption, managing power draw across multiple servers, and providing optimization services Energy efficiency through advising hypervisor of application workload requirements and supporting hypervisor directives to localize allocated resources Energy efficiency through control of primary allocation of system resources (CPU, memory, I/O) to multiple guest OSs depending on need Energy efficiency through control of memory DIMMs, I/O links, disks, and intelligence in firmware service processors Energy efficiency through control of clock frequency, cores, threads, instruction pipeline and memory interfaces

16 Page 16 Dependability Impact Data centers operate at higher temperature and humidity Increased power and temperature cycling > From turning off equipment > From throttling performance Lower voltage/power increases soft error vulnerability Many new states to verify > Reduced power states > Software control of hardware Derating, hot spots, wearout,... may all change

17 2008 ASHRAE Guidelines High Temperature: Air conditioning power requirement goes down dramatically, server power draw goes up (server fans run faster) Data Center Operating Temperature 27 C 2008 Data Center Guidelines 25 C Low Humidity: Risk of electrostatic discharge increases 2004 Data Center Guidelines 20 C High Humidity: Growth of conductive anodic filaments in PCBs increases risk of catastrophic failure 18 C Low Temperature: Server power draw remains limited, air conditioning power draw is high 25% 40% RH modulo ** 25% 5 C < Dew Point < 15 C 55% 60% Data Center Relative Humidity Page 17

18 Page 18 Minimize Total Cost Combined OPEX Data Center OPEX Cost of Server Replacement Cost of Air Conditioning Temperature

19 Page 19 Dependability Community Guidance From Industry Don't hurt performance > A few percent at most Don't use much chip area > 10% max from SELSE 2 (includes arrays) And now, don't use any extra power either > And allow throttling And, by the way, our customers still want the same levels of dependability

20 Page 20 The End of Redundancy? Many classical fault-tolerance techniques assume redundancy Active redundant hardware consumes too much power > Standby redundancy consumes too much power today (leakage current) > Even non-powered redundant hardware takes space that equates to power > Fans and power supplies may be exceptions

21 Page 21 How Should the Dependability Community Respond? Focus on techniques that minimize power > Information redundancy (on larger blocks of data) > Self-checking logic, e.g., state machines > Assertion checking Tradeoff power consumption with recovery latency > Retry instead of active redundancy/in-line recovery Consider new architectures > Store data in flash instead of DRAM/disk > Independent electrical zones on chips

22 Page 22 Example Low Power, Self-Checking Logic Uses existing scan chain logic Reference: S. Mitra, et al, Robust System Design with Builtin Soft-Error Resilience, IEEE Computer, Feb., 2005, pp 43-52

23 Page 23 How Should the Dependability Community Respond? - 2 Learn from low-power computing laptops, embedded systems > Laptops have lots of power cycles but still work Do it in software > Reliable protocols, end-to-end checksums, process pairs Power-aware middleware and applications Focus on the entire system

24 Page 24 Newthink vs. Oldthink in System Design Watts Consumed per Computational Unit Performance Optimized Computing Oldthink: Good systems design means moving customer as far to the left as possible Newthink: Good systems design means allowing customer to operate at any point of their choosing on this curve Energy Optimized Computing Time Taken per Computation Unit

25 Page 25 The Good News What a great research opportunity! What a great opportunity to have a positive impact on society! What a great way to recruit environmentally-conscious engineers!