Session 6: Data Center Energy Management Strategies Data Center Energy Use and Opportunities: Applying Best Practices to meet Executive Order Requirements Dale Sartor, PE Lawrence Berkeley National Laboratory August 10, 2016 Rhode Island Convention Center Providence, Rhode Island
Obtaining CEUs and Certificates FEMP is accredited by the International Association for Continuing Education & Training (IACET) and has awarded: 0.2 CEUs for Sessions 1 7, 9 0.1 CEUs for Session 8 To earn CEUs and a certificate of completion, participants must: Attend the entire session Access the assessment between August 19 September 23, 2016 Achieve 80% on the assessment and complete a brief evaluation by logging on to: www.wbdg.org/education/energyexchange2016.php Attendees will receive details for accessing the assessment by email following the Energy Exchange. Note: Participants must register once with the National Institute of Building Sciences Whole Building Design Guide (WBDG.org) to complete the requirements and obtain their certificate. 2
US Data Center Energy Usage Reports (2007 & 2016) ~1.8% U.S. Electricity
Change in Server Shipments Nearly all server shipment growth since 2010 occurred in servers destined for large hyperscale data centers: Hyperscale data centers typically operate more efficiently Growing percentage of overall data center activity
Key Observations Servers are improving in power scaling ability Servers typically operate at 10 50% utilization Increased power scaling reduces average power demand Large reduction in physical server demand within data centers Increase virtualization and consolidation has tempered increase in annual server shipments SPEC workbook data
Energy Use Estimates Savings: 620 billion kwh
Efficiency Scenarios 7
Future Challenges & Opportunities Data center closet clunkers: Improving/removing closet and other poorly operated smaller data centers Beyond 2020: Established efficiency measures (consolidation, power scaling, low PUE) to eventually hit upper limit Computational/storage demand only increasing Computational efficiency growth starting to wane
Executive Order 13693: The New Driver In March 2015, the President issued Executive Order (EO) 13693, Planning for Federal Sustainability in the Next Decade. Specific goals for data centers Promote energy optimization, efficiency, and performance Install/monitor advanced energy meters in all data centers by FY2018 Establish a Power Usage Effectiveness (PUE) target - between 1.2 and 1.4 for new data centers - less than 1.5 for existing data centers Other related goals Reduce building energy 2.5% per year per sq. ft. thru 2025 Increase clean and renewable energy to 25% & 30% by 2020 & 2025 Reduce water consumption 2% per year per sq. ft. thru 2025 Make ENERGY STAR or FEMP designated acquisitions
Implementing Instructions for New Directive Ensuring the agency chief information officer (CIO) promotes data center energy optimization, efficiency, and performance Agencies are to have data center staff trained and certified in data center energy efficiency. All core data centers shall have at least one Data Center Energy Practitioner (DCEP) assigned to manage data center performance Agency CIO should develop, issue and implement policies, procedures and guidance.
Implementing Instructions for New Directive Installing and monitoring advanced energy meters in all data centers by fiscal year 2018 Advanced energy meters installed by agencies as appropriate in all data centers shall be meters that enable the active tracking of power usage effectiveness (PUE) for the data center, as well as promote implementation of Data Center Infrastructure Management (DCIM). All new data centers shall include advanced energy and water meters. Agencies shall evaluate consolidation/closure for existing data centers unable to cost effectively install meters by FY 2018
Implementing Instructions for New Directive Establishing a power usage effectiveness target of 1.2 to 1.4 for new data centers and less than 1.5 for existing data centers Agencies should adopt a cloud first policy when developing new systems. Agencies will evaluate existing data centers unable to achieve 1.5 for alternative solutions. Encourages use of shared service providers and contracted data center services. Procurement preference for data centers with the lowest demonstrated PUE. Power Usage Effectiveness (PUE) is a measure of how efficiently a computer data center infrastructure uses energy.
Other Federal Initiatives Federal Data Center Optimization Initiative (formerly Consolidation Initiative) Reduce the cost of data center hardware, software and operations Shift IT investments to more efficient computing platforms and technologies Better Buildings Data Center Partners (voluntary) Improve efficiency of data center s PUE 1 by at least 20% within 10 years Implement/Share best practices, innovative technologies, and strategies Metering required for tracking Technical & communication assistance
FEMP Technical Assistance FEMP has assembled numerous technical assistance resources, including best practice guides, case studies, technical bulletins, and training on data center energy efficiency Direct technical assistance is available through FEMP to agencies who join as a Better Buildings Data Center Partner For more information visit: https://datacenters.lbl.gov
Benchmark Energy Performance Compare to peers: wide variation Identify best practices, ID opportunities Track performance: Can t manage what isn t measured The relative percentage of energy actually doing computing varies Office Space Conditioning 1% Lighting 2% Other 13% Computer Loads 67% Electrical Room Cooling 4% Cooling Tower Plant 4% Data Center Server Load 51% HVAC - Air Movement 7% Data Center CRAC Units 25% Lighting 2% HVAC - Chiller and Pumps 24%
High Level Metric: PUE Power Utilization Effectiveness (PUE) = Total Power/IT Power
Sample PUEs PUEs: Reported & Calculated PUE EPA ENERGY STAR Average 1.91 Intel Jones Farm, Hillsboro 1.41 T-Systems & Intel DC2020 Test Lab, Munich 1.24 Google 1.16 Leibniz Supercomputing Centre (LRZ) 1.15 National Center for Atmospheric Research (NCAR) 1.10 Yahoo, Lockport 1.08 Facebook, Prineville 1.07 National Renewable Energy Laboratory (NREL) 1.06 Source: Mike Patterson, Intel
Data Center Best Practices 1. Measure and Benchmark Energy Use 2. Identify IT Opportunities, and modify procurement processes to align with the procurement policy 3. Optimize Environmental Conditions 4. Manage Airflow (Air Management) 5. Evaluate Cooling Options 6. Improve Electrical Efficiency 7. Use IT to Control IT
1. Measure and Benchmark Energy Use Use metrics to measure efficiency Benchmark performance Establish continual improvement goals
2. Identify IT Opportunities Specify efficient servers (incl. power supplies) Virtualize Refresh IT equipment Turn off unused equipment Implement acquisition systems to assure efficient products are purchased
3. Optimize Environmental Conditions Follow ASHRAE guidelines or manufacturer specifications Operate near maximum of ASHRAE s recommended range Anticipate servers will occasionally operate in the allowable range Minimize or eliminate humidity control
4. Manage Airflow Implement hot and cold aisles Seal leaks and manage floor tiles Isolate hot and cold air/contain hot or cold aisle Control air flow (save energy with VSD on fans) 95 105ºF vs. 60 70ºF (35 41ºC vs. 16 21ºC) 70 80ºF vs. 45 55ºF (21 27ºC vs. 7 13ºC)
5. Evaluate Cooling Options Use centralized cooling system Maximize central cooling plant efficiency Provide liquid based heat removal Compressorless cooling ( free cooling)
6. Improve Electrical Efficiency Select efficient UPS systems and topography Examine redundancy levels Consider redundancy in the network rather than in the data center (geographical redundancy) Increase voltage distribution and reduce conversions. Redundant Operation Source: LBNL Benchmarking study
7. Use IT to Control IT Energy Evaluate monitoring systems to enhance real time management and efficiency Use visualization tools (e.g., thermal maps) Install dashboards to manage and sustain energy efficiency
Most importantly Get IT and Facilities people talking and working together as a team!
DOE s Center of Expertise Datacenters.lbl.gov Datacenters.lbl.gov
Available Resources Profiling Tool Assessment Tools Best Practices Guide Benchmarking Guide Data Center Programming Guide Technology Case Study Bulletins Report Templates Process Manuals Quick-Start Guide Professional Certification (DCEP)
DOE s Better Buildings Program
Questions Dale Sartor, P.E. Lawrence Berkeley National Laboratory MS 90-3111 University of California Berkeley, CA 94720 DASartor@LBL.gov (510) 486-5988 http://datacenters.lbl.gov/