IT Optimization Under Renewable Energy Constraint

Transcription

1 IT Optimization Under Renewable Energy Constraint Gustavo Rostirolla Stephane Caux, Paul Renaud-Goud, Gustavo Rostirolla, Patricia Stolf. IT Optimization for Datacenters Under Renewable Power Constraint (regular paper). Euro-Par, Turin, 27/8/218-31/8/218.

2 DATAZERO Project Jobs submission Environmental observation Simulators Physical systems Real platform Credits: Robin Roche The work presented is supported by the French ANR DATAZERO project ANR-15-CE

3 Agenda Introduction IT Optimization Module Methodology Results Conclusion Future Works Source:

4 Introduction Data centers are known as one of the big players when talking about energy consumption; In 26, were responsible for 61.4 billion kwh in the United States; In 21 about 1.3% of world's electricity; % 18% 2% 47% Devices Networks Data Centers Manufacturing 29% 16% 21% 34% Devices Networks Data Centers Manufacturing Cook, Gary, et al. "Clicking Clean: Who is winning the race to build a Green Internet?." Greenpeace Inc., Washington, DC (217). 4

5 Introduction Shehabi et al United States Data Center Energy Usage Report ~73 billion kwh in 22 In the last years, the use of cloud computing has been the basis of data centers, either in a public or private fashion. 5

6 Introduction This migration to cloud computing increases the concern about power utilization, especially when considering renewable energy sources and its oscillation over time; Gigawatts Annual Additions Capacity World Total 227 Gigawatts Gigawatts Annual additions Capacity World Total 433 Gigawatts Adib, Rana, et al. "Renewables 216 global status report." Global Status Report Renewable Energy Policy Network for the 21st Century (REN21) (216): 272. Tasks submitted by users needs to be executed inside a time interval (release time and due date); But: When? Where? At which speed/frequency? Resource Time Energy 6

7

8 IT Optimization Module Set of Jobs J: Jj=[tj,i;pej,i;memj,i] tj,i=[trelase j,i;tduedate j,i;tdurationj,i] r w d memj,i = Memory requested by task j,i pej,i = Number of processing elements requested by task j,i Set of Machines M: Mi=[npei;memi;Pmin i;pmax i] Computing Node [fi] set of frequencies available Processor 1 Processor 2 memi: Memory available in node Pmin i:power when node is idle αi: Coefficient dependent on the machine Core 1 Core 2 Core 3 Core 4 Core 1 Core 2 Core 3 Core 4 npei: Number of processing elements Memory Pmax i: g(pmin i;fi,l;αi) Power with processing element at 1% Discrete power curves Pavailable(t): power available at instant t Power n Time 8

9 IT Optimization Module Power Metric Task Output: Which task will run where, when, at which frequency; Constraints {Power, CPU, Memory} Translated as a set of scheduling possibilities in the form of a power profile; Associated with metrics (energy, due date violations ) Power (W) Power (W) Power (W) Violations: 2 Energy: 7 t4 t1 t2 t3 t n Time (minutes) Violations: 1 Energy: 6 t4 t1 t2 t3 t n Time (minutes) Violations: Energy: 6 t3 t5 t1 t4 t n Time (minutes) 9

10 IT Optimization Module Greedy Heuristic (Best Fit): Fast scheduling decisions; Easy implementation; Tasks can be sorted by arrival time, due date ; Tasks are scheduled in a local optimal, limited by the power curve received; The combinations of choices locally optimal do not always lead to an overall optimum. Meta-heuristic (Genetic Algorithm): Allows to produce a large number of adapted solutions; Makes it possible to approach an optimum solution; Slow execution time; Difficulties in setting parameters (crossover, mutation rate, population size, selection method). 1

11 IT Optimization Module Genetic Algorithm Workflow Start Selection Generate Initial Population Crossover 2 Variations: Minimize Due Date violations (MPGA) Minimize Due Date and Energy (MPGA-MO) Calculate Fitness of Individuals Mutation Greedy Algorithm to Set Tasks Start Time End Yes Satisfy Stop Criterion No Greedy Algorithm to adjust DVFS Calculate Fitness of Children Elitism for New Generation 11

12 IT Optimization Module Genetic Algorithm: Parents Node 3 Node 3 Node Node 2 T T1 T2 T3 Node Node Node 2 Node 3 T T1 T2 T3 Crossover/Mutation Node 3 Node 3 Node Node Offsprings Node 2 Node 3 T T1 T2 T3 Node Node 2 T T1 T2 T3 Greedy Time, Processor and Frequency Node 3 Processor 1 Frequency: F Start: 1:pm End: 2:pm T Node 3 Processor 2 Frequency: F1 Start: 1:pm End: 2:pm T1 Node Processor 1 Frequency: F Start: 3:pm End: 5:pm T2 Node 2 Processor 1 Frequency: F3 Start: 1:pm End: 2:pm T3 Node Node Node 2 Node 3 Processor 1 Processor 2 Processor 1 Processor 1 Frequency: F Frequency: F1 Frequency: F Frequency: F3 Start: 1:pm Start: 1:pm Start: 1:pm Start: 2:pm End: 2:pm End: 2:pm End: 2:pm End: 3:pm T T1 T2 T3 DVFS: Node Processor 1 Frequency: F1 Start: 1:pm End: 5:pm T1 (a) Node Processor 2 Frequency: F Start: 1:pm End: 2:3pm T2 Node Processor 2 Frequency: F3 Start: 2:3pm End: 5:pm T3 on Processor 1 Processor 2 on Processor 1 Processor 2 T2 T2 T1 T3 T1 T3 off (b) off (c) 12

13 IT Optimization Module Two execution phases for Genetic Algorithm to improve execution time: First phase provides an initial placement of tasks respecting a simplified power curve; Second phase uses the power prediction with all variations to improve this initial placement, allowing the scheduling to take profit of power peaks Power (W) Time Step Refined Simplified 13

14 Evaluation

15 Evaluation Computing Resources: Source: Villebonnet, V. (216). Scheduling and Dynamic Provisioning for Energy Proportional Heterogeneous Infrastructures (Doctoral dissertation, Université de Lyon). Based in Paravance and Taurus (Grid5) 3 Nodes x 2 Processors x 5 Frequencies (1.2 to 2.4 Ghz); + Overhead of turning on/off a node. T. Mudge, Power: A first-class architectural design constraint, Computer, vol. 34, pp ,

16 Evaluation 2 days simulation with DCWorms: 8 7 Profile II Summer Profile 3 power profiles 3 workloads (Google Based) Power (W) , 569 and 129 tasks (known at beginning of execution) Time (seconds) Sun Wind Total Profile I Mixed Profile Profile III Power (W) Power (W) Time (seconds) Sun Wind Total Time (seconds) Sun Wind Total 16

17 Results 17

18 Results Best Fit Due Date Best Fit Arrival Best Fit Size Genetic Algorithm Genetic Algorithm MO Energy Consumption (kwh) Tasks 569 Tasks 129 Tasks 234 Tasks 569 Tasks 129 Tasks 234 Tasks 569 Tasks 129 Tasks Profile I Profile II Profile III Due Date Violations Tasks 569 Tasks 129 Tasks 234 Tasks 569 Tasks 129 Tasks 234 Tasks 569 Tasks 129 Tasks Profile I Profile II Profile III 18

19 Results Power Available Power Consumed Power (W) Power (W) Sample Sample Best Fit Due Date Genetic Algorithm MO 19

20 Conclusion Different algorithms that aims to minimize due date violations while respecting power and resource constraints. Provide scheduling possibilities translated into power profiles with associated metrics for decision modules. Integration between power production and IT load; Just one segment of DataZero, which contains more modules that interacts with IT. 2

21 Future Works Scheduling of mixed workload batch and services; Phases based tasks: Percentage.5 Task Release Duedate Start Time.25 Duration Batch Unknown End Time Service Jun 1 12: Jun 2 : Jun 2 12: Jun 3 : Time Pre-evaluate workload/time available to choose algorithm;. Online scheduling + weather events (quick reaction). 21

22 IT Optimization Under Renewable Energy Constraint Gustavo Rostirolla Stephane Caux, Paul Renaud-Goud, Gustavo Rostirolla, Patricia Stolf. IT Optimization for Datacenters Under Renewable Power Constraint (regular paper). Euro-Par, Turin, 27/8/218-31/8/218.