ENERGY AWARE COMPUTING

Transcription

1 ENERGY AWARE COMPUTING Luigi Brochard, Lenovo Distinguished Engineer, WW HPC & AI HPC Knowledge June ,

2 Agenda Different metrics for energy efficiency Lenovo cooling solutions Lenovo software for energy aware computing 2017 Lenovo. All rights reserved. 2

3 How to measure Power Efficiency PUE Total Facility Power PUE = IT Equipment Power Power usage effectiveness (PUE) is a measure of how efficiently a computer data center uses its power; PUE is the ratio of total power used by a computer facility] to the power delivered to computing equipment. Ideal value is 1.0 It does not take into account how IT power can be optimised ITUE (IT power + VR + PSU + Fan) ITUE = IT Power IT power effectiveness ( ITUE) measures how the node power can be optimised Ideal value if 1.0 ERE Total Facility Power Treuse ERE = IT Equipment Power Energy Reuse Effectiveness measures how efficient a data center reuses the power dissipated by the computer ERE is the ratio of total amount of power used by a computer facility] to the power delivered to computing equipment. An ideal ERE is 0.0. ERE = PUE Tresuse/IT EqPower; If no reuse, ERE = PUE 2017 Lenovo. All rights reserved. 3

4 Choice of Cooling Air Cooled Standard air flow with internal fans Fits in any datacenter Maximum flexibility Broadest choice of configurable options supported Supports Native Expansion nodes (Storage NeX, PCI NeX) PUE ~2 1.5 ERE ~2 1.5 Choose for broadest choice of customizable options 2017 Lenovo. All Rights Reserved Air Cooled with Rear Door Heat Exchangers Air cool, supplemented with RDHX door on rack Uses chilled water with economizer (18 C water) Enables extremely tight rack placement PUE ~ ERE ~ Choose for balance between configuration flexibility and energy efficiency Direct Water Cooled Direct water cooling with no internal fans Higher performance per watt Free cooling (45 C water) Energy re-use Densest footprint Ideal for geos with high electricity costs and new data centers Supports highest wattage processors PUE ~ 1.1 ERE < < 1 with hot water Choose for highest performance and energy efficiency 4

5 TCO: payback period for DWC vs RDHx New Existing $0.06/kWh Existing Existing $0.12/kWh $0.20/kWh DWC RDHx New data centers: Water cooling has immediate payback. Existing air-cooled data center payback period strongly depends on electricity rate 2017 Lenovo. All Rights Reserved 5

6 idataplex dx360m4 ( ) idataplex rack with 84 dx360m4 servers dx360 M4 nodes, 2xCPUs (130W, 115W), 16xDIMMS (4GB/8GB), 1HDD/2SSD, network card. 85% Heat Recovery, Water 18 C-45 C, 0.5 lpm / node. dx360m4 server dx360m4 Server idataplex Rack 2017 Lenovo. All Rights Reserved 6

7 NextScale nx360m5 WCT ( ) NextScale Chassis 6U/12Nodes, 2 nodes / tray. nx360m5 WCT 2xCPUs (up to 165W), 16xDIMMS (8GB/16GB/32GB), 1HDD/2SSD, 1 ML2 or PCIe Network Card. 85% Heat Recovery, Water 18 C-45 C (and even upto 50 C), 0.5 lpm / node. copper waterloops 2 Nodes of nx-360m5 WCT in a Tray NextScale Chassis Scalable Manifold Rack Configuration nx360m5 with 2 SSDs 2017 Lenovo. All Rights Reserved 7

8 SuperMUC systems at LRZ: Phase 1 and Phase 2 Phase 1 Ranked 28 and 29 in Top500 June 2016 Fastest Computer in Europe on Top 500, June Nodes with 2 Intel Sandy Bridge EP CPUs HPL = 2.9 PetaFLOP/s Infiniband FDR10 Interconnect Large File Space for multiple purpose 10 PetaByte File Space based on IBM GPFS with 200GigaByte/s I/O bw Innovative Technology for Energy Effective Computing Hot Water Cooling Energy Aware Scheduling Most Energy Efficient high End HPC System PUE 1.1 Total Power consumption over 5 years to be reduced by ~ 37% from 27.6 M to 17.4 M 2017 Lenovo. All Rights Reserved Phase 2 Acceptance completed 3096 nx360m5 compute nodes Haswell EP CPUs HPL = 2.8 PetaFLOP/s Direct Hot Water Cooled, Energy Aware Scheduling Infiniband FDR14 GPFS, 10 x GSS26, 7.5 PB capacity, 100 GB/s IO bw 8

9 Lenovo Water Cooling added value Classic Water Cooling Direct Water cooling CPU only - Only 60% of heat goes to water - => 40% still need to be air cooled Inlet water temperature - Upto35 C - => No free cooling all year long/all geo Heat from water is wasted Unproven technology Power of server is not managed 2017 Lenovo. All Rights Reserved Lenovo Water Cooling Direct Water cooling CPU/DIMMS/VRs - 80 to 85% of heat goes to water - => just 10% still need to be air cooled Inlet water temperature - Upto C - => Free cooling all year long in all geo Water is hot enough to be efficiently reused - like with Absorption chiller => ERE <<1 3rd generation Water Cooling - More than nodes installed Power and energy are managed & optimized 9

10 DWC reduces Processor Temperature on Xeon 2697 v4 Conclusion: Direct Water Cooling lowers processor power consumption by about 5% and allows Higher processor frequency. NXT with 2 socket 2697v4, 128 GB 2400 MHz DIMM Inlet Water temperature is 28 C, 2017 Lenovo. All Rights Reserved 10

11 Air and DWC performance DC power on Xeon 2697v4 Conclusion: With Turbo OFF, Direct Water Cooling reduces power by 5% With Turbo ON, it increases performance by 3% and still reduces power by 1% DC energy is measured through aem DC energy accumulator 2017 Lenovo. All Rights Reserved 11

12 Savings from Lenovo Direct Water Cooling Higher TDP processors Reduced server power comsumption Lower processor power consumption (~ 5%) No fan per node (~ 4%) Reduce cooling power consumption With DWC at 45 C, we assume free cooling all year long ( ~ 25%) Additional savings with Energy Aware SW Total savings = ~35-40% Free cooling all yea long => Less chillers => CAPEX savings 2017 Lenovo. All Rights Reserved 12

13 Re-Use of Waste Heat, New buildings in Germany are very good thermally isolated: Standard heat requirement of only 50 W/m2 SuperMUCs waste heat would be sufficient to heat m2 of office space (~10 x) What to do with the waste heat during summer? 2017 Lenovo. All Rights Reserved 13

14 CooLMUC-2: Waste Heat Re-Use for Chilled Water Production ERE=0.3 Lenovo NeXtScale Water Cool (WCT) system technology Water inlet temperatures 50 C All season chiller-less cooling 384 compute nodes 466 TFlop/s peak performance Energy Reuse Effectiveness ( ERE) measures how efficient a data center reuses the power dissipated by the computer 2017 Lenovo. All Rights Reserved SorTech Adsorbtion Chillers based of zeolite coated metal fiber heat exchangers a factor 3 higher than current chillers based on silica gel COP = 60% Total electricity reduced by 50+% ERE = Total Facility Power Treuse IT Equipment Power 14

15 CooLMUC-2: ERE = 0.3 ERE = Total Facility Power Treuse = = 0.32 IT Equipment Power 104 CooLMUC-2 power consumption CooLMUC-2 heat output into warm water cooling loop Cold water generated by absorption chillers (COP ~ 0,5 0,6) Leibniz Supercomputing Centre 15

16 Savins from Direct Water Cooling with Lenovo Server power comsumption Lower processor power consumption (~ 5%) No fan per node (~ 4%) Cooling power consumption With DWC at 45 C, we assume free cooling all year long ( ~ 25%) Additional savings with energy aware SW Total savings = ~35-40% Heat Reuse With DWC at 50 C, additional 30% savings as free chilled water is generated With heat reuse total savings => 2017 Lenovo. All Rights Reserved 50+% 16

17 Lenovo references with DWC ( ) Sites LRZ SuperMUC LRZ SuperMUC 2 LRZ SuperCool2 NTU Enercon US Army Exxon Research NASA Goddard PIK KIT Birmingham U ph1 Birmingham U ph2 MMD UNINET Peking U Nodes Country Germany Germany Germany Singapore Germany Hawai NA NA Germany Germany UK UK Malaysia Norway China Instal date Max. In. Water 45 C 45 C 50 C 45 C 45 C 45 C 45 C 45 C 45 C 45 C 45 C 45 C 45 C 45 C 45 C More than nodes up and running with DWC Lenovo technology 17

18 How to manage/control power and energy Report temperature and power consumption per node / per chassis power consumption and energy per job Optimize Reduce power of inactive nodes Reduce power of active nodes 2017 Lenovo. All Rights Reserved 18

19 Power Management on NeXtScale IMM = Integrated Management Module (Node-Level Systems Management) Monitors DC power consumed by node as a whole and by CPU and memory subsystems Monitors inlet air temperature for node Caps DC power consumed by node as a whole Monitors CPU and memory subsystem throttling caused by node-level throttling FPC = Fan/Power Controller (Chassis-Level Systems Mgmt Monitors AC and DC power consumed by individual power supplies and aggregates to chassis level Monitors DC power consumed by individual fans and aggregates to chassis level Enables or disables power savings for node PCH = Platform Controller Hub (i.e., south bridge) ME = Management Engine (embedded in PCH, runs Intel NM firmware) HSC = Hot Swap Controller (provides power readings) 2017 Lenovo All rights reserved. 19

20 DC power sampling and reporting frequency High Level Software 1Hz IMM/BMC 1Hz NM/ME 200Hz RAPL CPU/memory (energy MSRs) 500Hz Meter 10Hz HSC 1KHz Sensor 2017 Lenovo All rights reserved. 20

21 Power reading improvements on next gen. NeXtScale Targets Better than or equal to +/-3% power reading accuracy down to the node s minimum active power (~40-50W DC). Power granularity <=100mW At least 100Hz update rate for node power readings. 100 readings are returned en masse. In addition, there is a beginning timestamp. Time between each successive readings is maintained at 10mS. ipmi raw oem cmd INA226 (used for metering) FPGA (FIFO) Bulk 12V 2017 Lenovo.. All rights reserved. IMM Rsense ME (Node Manager) SN (used for capping) Node 12V 21

22 Demo at ISC 17 Lenovo HPC Software Stack Lenovo OpenSource HPC Stack Build on top of OpenHPC with xcat Enhanced with Lenovo configuration, plugins and scripts Add in IBM Spectrum Scale, Singularity* Add in Lenovo Antilles and Energy Aware run time** Integrated and supported A ready to use HPC Stack collaborating w/ Greg Kurtzer from Lawrence Berkeley National Lab; 2017 ** : inlenovo. collaboration withreserved. BSC All rights Customer Applications Enterprise Professional Services Installation and custom services, may not include service support for third party software An OpenSource IaaS suite to run and manage optimally and transparently HPC, Big Data and Workflows on a virtualized infrastructure adjusting dynamically to user and datacenter needs through energy policies User / Operator Web Portal xcat Extreme Cloud Admin. Toolkit Energy Awareness Containers Parallel File Systems OS Antilles - Simplified User Job View / Customized Confluent Lenovo EAR Singularity IBM Spectrum Scale & Lustre NFS OFED Lenovo System x Virtual, Physical, Desktop, Server Compute Storage Network OmniPath Infiniband 22

23 Confluent: HeatMap 2017 Lenovo. All rights reserved. 23

24 How to manage/control power and energy Report temperature and power consumption per node / per chassis power consumption and energy per job xcat/confluent Optimize Reduce power of inactive nodes Reduce frequency of active nodes IBM Platform LSF SLURM PBSpro 2017 Lenovo. All rights reserved. 24

25 Demo at ISC 17 Energy Aware Run time: Motivation Power and Energy has become a critical constraint for HPC systems Performance and Power consumption of parallel applications depends on: Architectural parameters Runtime node configuration Application characteristics Input data Select optimal frequency Manual best frequency Difficult to select manually and it is a time consuming process (resources and then power) and not reusable It may change along time It may change between nodes Lenovo. All rights reserved. New architecture Configure application for Architecture X Execute with N frequencies: calculate time and energy New input set 25

26 Energy Aware Run time (EAR) Goals Offer a dynamic and transparent solution to energy awareness : Avoiding having to re-execute applications again and again Easy to use - Without source code modifications - Without historic application information - Supporting standard programming models: MPI, MPI+OpenMP - Using standard libraries and tools as much as possible to be easily portable - Open Source Frequency change based on simple Energy Policies with performance thresholds Minimizing the overhead introduced - Remember we are HPC and we want to save energy! 2017 Lenovo. All rights reserved. 26

27 EAR proposal Automatic and dynamic frequency selection based on: Architecture characterization Application characterization - Outer loop detection (DPD) - Application signature computation (CPI,GBS,POWER,TIME) Performance and power projection Users/System policy definition for frequency selection (configured with thresholds) - MINIMIZE_ENERGY_TO_SOLUTION - Goal: To save energy by reducing frequency (with potential performance degradation) - We limit the performance degradation with a MAX_PERFORMANCE_DEGRADATION threshold - MINIMIZE_TIME_TO_SOLUTION - Goal: To reduce time by increasing frequency (with potential energy increase) - We use a MIN_PERFORMANCE_EFFICIENCY_GAIN threshold to avoid that application that do not scale with frequency to consume more energy for nothing 2017 Lenovo. All rights reserved. 27

28 BQCD use case Berlin quantum chromodynamics program (BQCD) Two input sets generates two different use cases - Input 1 generates a cpu intensive use case, much more sensible to node frequency changes - Input 2 generates a memory intensive use case, less sensible to node frequency changes 2017 Lenovo. All rights reserved. 28

29 BQCD use case BQCD at Lenox MPI+OpenMP code Configured with 16 mpis (4 nodes, 4 ppn) Architecture tested Lenox cluster Intel(R) Xeon(R) CPU E GHz, 18c 500 Exec.Time(sec.) Number of threads adapted to number of cores Making this comparison took 96H of computation x 4 nodes consumed!! 2.3GHz 2.2GHz 2.1GHz CPU 2.0GHz 1.9GHz 1.8GHz MEM BQCD at Lenox 300,00 Avg.Power (W) 250,00 200,00 150,00 100,00 50,00 0,00 2.3GHz 2.2GHz 2.1GHz CPU 2017 Lenovo. All rights reserved. 2.0GHz 1.9GHz 1.8GHz MEM 29

30 BQCD static analysis CPU 1.8Ghz vs. 2.3Ghz Power saving is 34% Performance degradation is 31% Energy variation is 14% Minimal energy is at 2.0GHz Memory 1.8Ghz vs. 2.3Ghz Power saving is 14% Performance degradation is 4% Energy variation is 10% Minimal energy is at 1,8Ghz If we set a maximum performance penalty (i.e 20%) CPU version will run at 2,0GHz Mem. Version will run at 1,8GHz BQCD_CPU: Fi vs. 2.3GHz 40% 35% 30% 25% 20% 15% 10% 5% 0% 2.2GHz 2.1GHz Perf.Degrada on PowerSaving 1.8GHz EnergySaving 16% 14% 12% 10% 8% 6% 4% 2% 0% 2.1GHz Perf.Degrada on 1.9GHz BQCD_MEM: Fi vs. 2.3GHz 2.2GHz 2017 Lenovo. All rights reserved. 2.0GHz 2.0GHz PowerSaving 1.9GHz 1.8GHz EnergySaving 30

31 2017 Lenovo. All rights reserved. 31

32 EAR overall overview Learning Phase: at EAR installation (*) Kernels execution Coefficients computatio n Coefficients DB Store Coefficients DPD detects outer loop Compute power and performance metrics for outer loop Optimal frequency calculation Energy policy Read EAR execution (loaded with application) (*) or every time cluster configuration is modified (more memory per node, new processors...) 2017 Lenovo. All rights reserved. 32

33 Learning Phase 2017 Lenovo. All rights reserved. 33

34 Performance and Power models We use linear models to predict Power and Time at any available frequency fn from a reference frequency Rf based on: HW coefficients (A(Rf,fn), B(Rf,fn), C(Rf,fn), D(Rf,fn), E(fn) and F(Rf,fn)) which represent the parameters of each node in the system, measured in the Learning Phase - At EAR installation or every time cluster configuration is modified Application signature (Power, CPI and GBS) computed while the application is running - Power, CPI and GBS are the basic metrics that characterize an application: we define it as the Application signature Execution of selected kernels with available frequencies on each node of the system Metrics collection: Execution time, Avg.Power, CPI and GBS 2017 Lenovo. All rights reserved. Computation of node coefficients to be used in performance and power models 34

35 Dynamic Application Characterization by Dynamic Periodicity Detection 2017 Lenovo. All rights reserved. 35

36 Dynamic characterization is driven by DPD New Dynamic Pattern Detection -IF Signature changes -IF New Pattern detection Optimal frequency selection based on policy and arguments 2017 Lenovo. All rights reserved. Signature computation: (CPI,GBS,POWER,TIME) Performance/Power projection models applied 36

37 Dynamic Pattern Detection DPD algorithm detects repetitive sequences of events EAR generates a event identifier per dynamic MPI call Using mpi arguments such as type of call, sendbuff, recvbuff, etc HPC applications usually present this repetitive behaviour since they exploit loops DPD receives a new event as input and reports the dpd_status New event 2017 Lenovo Internal. All rights reserved. DPD NEW_LOOP/NEW_ITERATION 37

38 DPD example Events sequence Time 0,4,8,1,2,3,1,2,3,1,2,3,1,2,3,4,8, 1,2,3,1,2,3,1,2,3,1,2,3,4,8 NEW_LOOP Loop_id=1 Loop_size=3 NEW_ITERATION Loop_id=1 Loop_size=3 NEW_LOOP Loop_id=1 Loop_size=3 NEW_ITERATION Loop_id=1 Loop_size=3 END_LOOP NEW_LOOP Loop_id=4 Loop_size= Lenovo Internal. All rights reserved. 38

39 Accumulated 2017 Lenovo Internal. All rights reserved. me Big loop detected Policy is applied F: 2.6Ghz 2.4Ghz Power is reduced Iteration time is similar Frequency BQCD_CPU: Outer loop size detected(mpi rank 0) Acuumulated me BQCD_CPU:Frequency(mpi rank 0) Accumulated me BQCD_CPU:Measured POWER(mpi rank 0) Accumulated me BQCD_CPU:Measured Itera on me(mpi rank 0) Avg.Power (W) Frequency Itera on me (usecs) R A N K Avg.Power (W) M P I Itera on me (usecs) EAR at works on BQCD_CPU BQCD_CPU: Outer loop size detected(mpi rank 8) Acuumulated me BQCD_CPU:Frequency(mpi rank 8) Accumulated me BQCD_CPU:Measured Itera on me(mpi rank 8) Accumulated me M P I Accumulated me BQCD_CPU:Measured POWER(mpi rank 8) R A N K

40 EAR policies MINIMIZE_ENERGY_TO_SOLUTION Goal : Minimize the energy consumed with a limit to the performance penalty User/Sysadmin defines a MAX_PERFORMANCE_PENALTY Based on Signature and node coefficients, EAR computes a performance projection for each available frequency. EAR selects the optimal frequency that minimizes Energy with (performance_penalty <= MAX_PERFORMANCE_PENALTY) MINIMIZE_TIME_TO_SOLUTION Goal: Improve the execution time while guaranteeing a minimum performance improvement that justifies more energy consumption User/Sysadmin defines a MIN_PERFORMANCE_GAIN Based on Signature and node coefficients, EAR computes a performance projection for each available frequency EAR selects the optimal frequency that improves application performance with performance_improvement >= MIN_PERFORMANCE_EFFICIENCY_GAIN 2017 Lenovo. All rights reserved. 42

41 EAR evaluation 2017 Lenovo Internal. All rights reserved. 43

42 Performance and power projection models: BQCD real vs. projected values TIME POWER BQCD_CPU: Real vs. Projecte me Error(Projected vs Real metrics) BQCD_CPU: Real vs. Projected power % % 4% 2% 150 0% 100 2,2GHz 2,1GHz 0 2,3GHz 2,2GHz 2,1GHz Exec.Time(sec.) 2,0GHz 1,9GHz 1,8GHz 2,3GHz Proj.Time(sec.) 2,2GHz 2,1GHz Avg.Power BQCD_MEM: Real vs. Projected me 2,0GHz 1,9GHz 1,8GHz Power (W) ,9GHz 1,8GHz Error Time(bqcd_cpu) Error Power(bqcd_cpu) Error Time(bqcd_cpu) Error Power(bqcd_mem) Proj.Avg.Power Average error is 5% BQCD_MEM: Real vs Projected power 500 2,0GHz Frequency 50 0 Time(sec.) Error Power (W) Time (sec.) 8% ,3GHz 2,2GHz 2,1GHz Exec.Time 2,0GHz 1,9GHz Proj. Exec.Time 2017 Lenovo Internal. All rights reserved. 1,8GHz 2,3GHz 2,2GHz 2,1GHz Avg.power 2,0GHz 1,9GHz 1,8GHz Proj.Avg.Power 45

43 BQCD best static frequency selection CPU 1.8Ghz vs. 2.3Ghz Power saving is 34% Performance degradation is 31% Energy variation is 14% Minimal energy is at 2.0GHz Memory 1.8Ghz vs. 2.3Ghz Power saving is 14% Performance degradation is 4% Energy variation is 10% Minimal energy is at 1.8Ghz BQCD_CPU: Fi vs. 2.3GHz 40% 35% 30% 25% 20% 15% 10% 5% 0% 2.2GHz 2.1GHz Perf.Degrada on 2.0GHz 1.9GHz PowerSaving 1.8GHz EnergySaving BQCD_MEM: Fi vs. 2.3GHz 16% 14% 12% 10% 8% 6% 4% 2% 0% 2.2GHz 2.1GHz Perf.Degrada on 2017 Lenovo Internal. All rights reserved. 2.0GHz PowerSaving 1.9GHz 1.8GHz EnergySaving 46

44 BQCD: EAR MINIMIZE_ENERGY_TO_SOLUTION Default frequency 2.3GHz MAX_PERFORMANCE_DEGRADATION Set to threshold TH=[5%,10%,15%] BQCD_CPU: EAR MIN_ENERGY_TO_SOLUTION 25% 20% 15% 10% 5% CPU use case 0% 5% TH= 5% [2.2Ghz 2.3Ghz] selected TH= 10% 2.2Ghz selected TH= 15% [2.0GHz 2.2GHz] selected MEM use case 10% 15% MAX_PERFORMANCE_DEGRADATION PerfDegrada on PowerSaving EnergySaving BQCD_MEM: EAR MIN_ENERGY_TO_SOLUTION 10% 8% TH= 5% 1.8GHz selected TH= 10% 1.8GHz selected TH= 15% 1.8GHz selected 6% 4% 2% 0% 5% 10% 15% MAX_PERFORMANCE_DEGRADATION PerfDegrada on 2017 Lenovo Internal. All rights reserved. PowerSaving EnergySaving 47

45 BQCD: EAR MINIMIZE_TIME_TO_SOLUTION Default frequency 2.0Ghz MIN_PERFORMANCE_EFFICIENCY_GAIN Set to 70% - 80% BQCD_CPU:MIN_TIME_TO_SOLUTION 10% 8% 6% BQCD_CPU 4% One node is executed at 2.3GHz The rest remains at 2.0GHz 2% 0% 70% 80% MIN_PERFORMANCE_EFFICIENCY_GAIN Performance Gain BQCD_MEM Power increment Energy increment BQCD_MEM: EAR MIN_TIME_TO_SOLUTION All the nodes execute at 2.0GHz 5% 4% 3% 2% 1% 0% 70% 80% MIN_PERFORMANCE_EFFICIENCY_GAIN PerformanceGain 2017 Lenovo Internal. All rights reserved. Power Increment Energy Increment 48

46 EAR distributed design EAR offers a distributed design where each node works locally no centralized information One mpi process per node takes care of: Dynamic Pattern detection EAR metrics collection EAR node state: collecting metrics, evaluating signature, etc EAR policy implementation Per node logging messages: One summary file per node is generated 2017 Lenovo. All rights reserved. 49

47 EAR overhead evaluation Main EAR overhead comes from DPD algorithm Current EAR version invokes DPD at each MPI call DPD consumes 2 usecs per invocation BQCD cpu intensive use case executes 16millions of DPD invocations in 500 seconds Profile information reported by mpi ranks 0,4,8 and 12 We are working in optimizations to reduce the number of DPD calls Mpi rank DPD calls usecs/dpd ,12usecs ,02usecs ,07usecs ,04usecs 50 50

48 EAR Goals.where we are? Our main goal were to offer a dynamic and transparent solution to energy awareness Avoiding having to re-execute applications again and again Easy to use - Without source code modifications - Without historic application information - Supporting standard programming models: MPI, MPI+OpenMP - Using standard libraries and tools as much as possible to be easily portable - Papi for hardware counters (core and uncore) when available and machine configuration - Libcpupower - ibmaem kernel module for node energy (linux distro) - Open Source (on progress) Frequency change based on simple Energy Policies with performance thresholds Minimizing the overhead introduced Overhead has been significantly reduced, but there is still margin for improvement 2017 Lenovo. All rights reserved. 51

49 Conclusion Lenovo has been a leader for years in Energy Aware Computing With end to end solutions to : reduce CAPEX, OPEX and TCO Maximize heat removal by water Maximize temperature for manage systems and infrastructure free cooling all year round efficient use of adsorption chillers with open source and partners solutions And going beyond: Dynamically adjust node frequency in operation Minimize TCO by working on all metrics PUE + ITUE + ERE 2017 Lenovo. All rights reserved. 52

50 Lenovo ISC17 Booth AI demo EAR demo NXS WTC demo Rack with various HW Table 1 Table 2 Open HPC Software Stack Table Lenovo Internal. All rights reserved. Excelero demo Stark HW HW Interactive Show & Tell 53

51