Measuring and Managing Power Consump2on

Transcription

1 Measuring and Managing Power Consump2on Todd Rosedahl, Chief Engineer IBM/POWER Firmware Development Revolu'onizing the Datacenter Join the Conversa'on #OpenPOWERSummit

2 Mo2va2on! US datacenter energy consump2on 91Billion KWH! MW power plants! State of Minnesota 68Billion KWH! Power Usage Effec2veness! Energy Star, SERT, Regula2ons 4/11/16 2

3 Agenda! System Stack/Ecosystem! OCC Overview! Hardware Block Diagram! Func2onal Details! Measurement! OpenBMC! References 4/11/16 3

4 Ecosystem Enablement Power Open Source So?ware Stack Components Cloud SoVware Standard Opera2ng Environment (System Mgmt) Opera2ng System / KVM Exis2ng Open Source SoVware Communi2es July 2014 Power8 open source firmware stack contributed thru GitHub Toolkits and resources for por2ng and op2mizing, growing repository on website AMESTER now Open Source OpenBMC coming soon Firmware Hardware OpenPOWER Firmware OpenPOWER Technology New OSS Community System KVM/Linux OPAL Management Console ETH BMC Host Boot P8 HW SBE OCC 4/11/16 4

5 4/11/16 5

6 OCC Overview What is OCC?! Hardware/Firmware that controls system power, performance & thermals! 405 processor with 512k dedicated RAM! General Purpose Engines (GPE) to offload the 405 What does OCC do?! Reads/controls system power! Reads/controls chip temperatures! Enables efficient fan control! Provides OT protec2on! Power Capping! Fault Tolerance! Energy saving! Performance boost! Enables sophis'cated measurement and profiling Memory Power Measurement Measures Actuates Reads Writes BMC Communicates Fan Control Actuation OCC Loads GPE Register Reads/Writes Runs on Measures/ Actuates Processor 405 C C VRMs Opal Communicates Loaded and initialized by HostBoot Communicates to other OCC via Sys Mem Processor 405 C C 4/11/16 6 Physical paths not shown

7 Hardware Block Diagram! Main Memory Ini2al Load Temperature Sensing U2liza2on Measurement Bandwidth control OPAL communica2on OCC communica2on! Processor Temperature Sensing Core Frequency Control - PSTATES Chip Voltage Control PSTATES U2liza2on Measurement! BMC Report power/temp Provide Power Cap DCMI compliance Fan Control 4/11/16 7

8 Power/Thermal Management Linux Governors OCC measures/clips for capping/ot Balanced Performance Power save OCC Controls Frequency Power Capping Over Temperature Protec2on Processor Idle States Nap Sleep Winkle Power Capping DCMI commands to set power limit via DCMI BMC can set N and N+1 system power caps. 4/11/16 8

9 OCC Vo2ng Box Compare Power to limits and set Control Vote (every 250us) Compare temp to limit and set Thermal Control Vote (every 16ms) Read current 250us power reading Frequency Voting Box (every 250us) Read current 2ms hojest Core temp Read power every 250us Desired PSTATE from o/s Lowest Pstate (freq) wins Maximum PSTATE Read every 2ms, average all DTS to get core temp and take hottest core temp Core DTS OCC Complex Actual PSTATE PMC (Hardware that actuates V/F slew) Power measurement hardware OPAL VRM Freq Set Get power from OCC in poll response for export 4/11/16 9 BMC Get Temps from OCC in Poll Response for export and for fan control

10 Measurement Out of Band! IPMI Sensors Inlet Temp Proc Core Temps Mem Temps All Power Rails! DCMI Commands Node Power Main Memory IB pathway Tighter workload coupling Performance concerns Jitter concerns Host OS! AMESTER -- IPMI OCC Pass through OOB pathway Measure without affecting system Difficult to correlate to system jobs/events BMC OCC CPU Power/thermal measurement Hardware Node 4/11/16 10

11 AMESTER AMESTER Overview! AMESTER Automated MEasurement for System Temperature and Energy Repor2ng! Uses management network and service processors Does not interfere with opera2ng system or workloads No performance loss Runs without requiring any addi2onal installa2on on the server! Sensor data collec2on - Whole system power measurement Component power on POWER: Processor Vdd, Memory, IO, Storage, Fans Core: temperature, frequency, u2liza2on, memory bandwidth, instruc2ons per second.! 250 µs tracing into 8KB buffer sensor logged every 250usec for 1 second! 2 ms tracing into 8KB buffer 1 sensor logged every 2ms for 8 seconds! Output to CSV file for import into Excel or Matlab Internet POWER8 system P8 chip OCC 4/11/16 11 Service Processor Measurement Hardware 4/11/16 11

12 Sensor Descrip2on 4/11/16 12

13 Sample Sensor list 440 Total Sensors Power, Thermal, Performance, u2liza2on, health 4/11/16 13

14 Simple Example of Sensor Data 4/11/16 14

15 Insights! Visualiza2on is key to rapid prototyping and problem solving! Correla2on of power consump2on with other metrics! Time- alignment of sensor data is crucial for debugging firmware algorithms! Examples Measuring sejling 2me of power capping controller aver workload changes Developing dynamic voltage/frequency scaling algorithms Discovering small non- steady behavior when steady- state behavior was expected 4/11/16 15

16 OpenBMC Collabora2on Power Open Source So?ware Stack Components Cloud SoVware Standard Opera2ng Environment (System Mgmt) Opera2ng System / KVM Firmware OPAL, Host Boot, OCC BMC Exis2ng Open Source SoVware Communi2es New OSS Community Fan Controller Power Supply VRMs Security IPMI DCMI Journaling BMC Core Linux Redfish File Server Energy Management Your Function Here Processor Support Management Console ETH BMC System KVM/Linux OPAL Host Boot P8 HW OCC 4/11/16 16 Why Collaborate on OpenBMC?! Industry Connec'ons! Leading Edge Technology! Funding! Student Job Placement! Research Paper Opportuni'es

17 Open Power Blog link:! hjp://openpowerfounda2on.org/press- releases/occ- firmware- code- is- now- open- source/ GitHub pages: References! OpenPOWER project: hjps://github.com/open- power! OCC: hjps://github.com/open- power/occ! Building OpenPOWER: hjps://github.com/open- power/op- build! Open BMC: hjps://github.com/openbmc/openbmc! AMESTER: hjps://github.com/open- power/amester! OCC Sensor collec2on document: hjps://github.com/open- power/docs/blob/master/occ/ OCC_ipmitool_sensors.pdf 4/11/16 17