A Distributed Framework for Carbon and Cost Aware Geographical Job Scheduling in a Hybrid Data Center Infrastructure

Transcription

1 A Distributed Framework for Carbon and Cost Aware Geographical Job Scheduling in a Hybrid Data Center Infrastructure A. Hasan Mahmud, S. S. Iyengar, Fellow, IEEE School of Computing and Information Sciences, Florida International University Summarized by Essien Ayanam CS 788 September 11, 2017

2 Outline Introduction Hybrid Data Center Infrastructure CAGE Experiment Conclusion

3 Introduction Reduce the growing electricity consumption and carbon footprint of IT organization Data Centers Current action to mitigate the issue: installation of massive on-site renewable facilities Dependency: Underlying data center architecture Self Managed Data Center Colocation Data Center Hybrid Data Center

4 Introduction Renewable Energy Self-Managed Data Centers: Costly and dedicated workforce needed to manage the facility Colocation Data Center: Not possible to distinguish the source of energy because they are distributed by a common power infrastructure Hybrid Data Center: Implementing geographical load balancing (GLB) requires consideration of both its self-manage data and colocations GLB decisions need to incorporate both electricity price for cost saving and carbon footprint for sustainability

5 Introduction CAGE (Carbon and Cost Aware Geographical Job Scheduling) Use of a hybrid data center infrastructure Distributed resource management algorithm Based on alternating direction method of multipliers (ADMM) Optimizes GLB decisions to minimize carbon emission, electricity cost and revenue loss while meeting performance requirements. CAGE leverages the geographical variation of carbon usage effectiveness (CUE) of electricity production, renewable energy, electricity cost and cooling efficiency for each location.

6 Hybrid Data Center Infrastructure Focus on the unique sharing of power infrastructure in colocations for optimizing a tenant/organization s GLB decisions Problems and Challenges Lowering revenue cost Reducing electricity cost Minimizing carbon footprint What is the priority for the organization? Reducing carbon footprint vs. cost savings

7 Hybrid Data Center Infrastructure

8 CAGE Distributed job scheduling algorithm Based on the alternating direction method of multipliers (ADMM) Solves convex optimization problems Breaking them into smaller pieces Making them easier to handle Applied ADMM to solve P1 by introducing a set of auxiliary variables b = a (beta = alpha)

9 CAGE

10 CAGE Solution CAGE function P2 Alpha (a) determines the revenue cost due to network propagation delay between load balancers and data centers Beta (b) determines the carbon emission and electricity cost of the data centers

11 CAGE

12 CAGE Enables each load balancer and data center to solve their own subproblems independently Provides an efficient and fast distributed solution for carbon awareness in hybrid data center infrastructure Utilizing 8 data centers and a thousand servers computational time was less than three seconds Expectation of a similar computational time for large scale settings due to the fact that each sub-problem can be solved independently

13 Experiment Data Centers - Considered 3 load balancers, 3 self-managed and 5 colocation data centers Experimental Simulation Normalized the workload using server utilization traces from Microsoft services, Hotmail and MSN, Microsoft Research (MSR), YouTube and FIU university workload traces Collected energy consumption of the various tenants at the 5 colocation data centers Normalized solar energy generation Collected electricity prices from respective utility providers

14 Experiment

15 Experiment CoReUn (Colocation Renewable Unaware) Variant of CAGE Does not consider the availability of renewable energy at colocation locations while making workload scheduling decisions CostMin (Cost Minimization) Minimizes the electricity and revenue cost Ignores Carbon footprint PerfMax (Performance Maximization) Maximizes the delay performance by keeping all servers on at every location

16 Experiment

17 Experiment

18 Experiment

19 Experiment System Experiment 2 Self managed and One Colocation data center Self-managed data center 4 servers; Colocation data center 8 servers, 5 of which belong to two other tenants Locations: VA and IL (Self-Managed); NY (Colocation) 2 Load balancers virtually located at IL and NY Power consumption measured by CloudPOWER power meter Running RUBIS and key-value-store (KVS) benchmarks Wikipedia and Gmail traces as the workload patterns

20 Experiment

21 Conclusion Proposed a novel and distributed geographical workload scheduling algorithm CAGE Reduces the carbon emission of hybrid data infrastructure consisting of both self-managed and colocation data centers Alternating Direction Method of Multipliers was leveraged to design the proposed algorithm Based on simulation and real life experiments CAGE can reduce carbon emission by up to 36% compared to the existing approaches