Building Adaptive Performance Models for Dynamic Resource Allocation in Cloud Data Centers

Size: px

Start display at page:

Download "Building Adaptive Performance Models for Dynamic Resource Allocation in Cloud Data Centers"

Luke Mason
5 years ago
Views:

1 Building Adaptive Performance Models for Dynamic Resource Allocation in Cloud Data Centers Jin Chen University of Toronto Joint work with Gokul Soundararajan and Prof. Cristiana Amza.

2 Today s Cloud Pay for your resources! small, large, extra large instances data stored, I/O rate, data transfered $8 per users

3 Dream: QoS Cloud Pay for your performance! Average query latency < 1 sec Customer: Our workload is usually stable, but there will be a few unpredicted peak times. Cloud Admin: how many resources should we provision dynamically?

4 Cloud Admin: What If Question What is the performance of this application given 2 CPUs, 4G RAM, 300 IOPS?

5 Challenges Performance interference Between consolidated workloads Uncontrolled resource sharing affects performance Increasingly complicated systems Increase in # of resources, e.g. multilevel cache System behavior varies, e.g. prefetching, background operations,... Workload itself causes variance/noise

6 Our Solution Build predictable systems in cloud Partition critical resources CPU, Memory, Network, Storage Answer what if question online Consolidate workloads in cloud Build performance models for each app Dynamically allocate resources Monitor and correct performance models

7 Build Predictable System in Cloud Partition and allocate critical resources dynamically. Switch/Router Net Bandwidth... Storage Array Database Servers CPU, DB Buffer Pool Storage Cache, Storage Bandwidth Applications will have minimum interference from each other!

8 Buffer Pool Size Performance Model for Application Storage Cac Storage Cache Size Avg. Latency High Latency... Low Latency 8

9 Latency Latency Buffer Pool Size Storage Cache Size Buffer Pool Size Storage Cache Size Multilevel Resource Allocation 9

10 Our Goal Challenge of building performance models Large number of configurations to predict performance (CPU, Buffer Pool Size, Network, Storage cache,...) Increasingly complicated systems Build multilevel performance model with Acceptable accuracy In a short amount of time Adapt to system changes

11 Outline Overview of different types of models Analytical models Black box models Our approach: Chorus Experimental results

12 Different Types of Models Analytical Extensive domain knowledge Fast to get results Acceptable accuracy Difficult to adapt Black Box Minimum domain knowledge May take a long time Higher accuracy Easy to adapt

13 Buffer Pool Size Storage Cache Avg. Latency Black Box Performance Models High Latency Low Latency 13

14 Outline Overview of different types of models Our approach: Chorus Exploit incomplete expert knowledge Leverage individual models Automated the modeling process Optimizations Experimental results

15 Exploit Partial Expert Knowledge Analytical Gray Box Black Box

16 Expert Knowledge: Cache Inclusiveness DB Buffer Pool 4 3 LRU Storage Cache LRU I/Os: 6 16

17 Expert Knowledge: Cache Inclusiveness DB Buffer Pool LRU Storage Cache 3 5 LRU I/Os: 6 17

18 Approximate Single Cache Model (LRU) LRU LRU I/Os: 6 18

19 Gray Box Multilevel Cache Model f(cpu, Buffer pool size, storage cache size, storage bandwidth) Gray box model: Ignore the smaller cache size f(cpu, Bigger Cache Size, storage bandwidth) Greatly reduce the # of configurations to predict!

20 Avg. Latency Analytical Gray Box Curve Fitting Model L d (ρ d )= L d(1) ρ d Gray box L d (ρ d ) = α ρ β d Disk Bandwidth 20

21 Outline Overview of different types of models Our approach: Chorus Exploit incomplete expert knowledge: gray box model Leverage individual models Automated the modeling process Optimizations Experimental results

22 Query Latency Buffer Pool Size Disk Bandwidth Build Performance Model 11 days! High Latency Low Latency

23 Leverage Individual Models 5 days! 3 days (manually)! High Latency Query Latency Disk Bandwidth CPU Low Latency Buffer Pool Size Have we simplified the management problem?

24 Iteratively Training Refine if necessary Per Region

25 Optimizations of Chorus Train models from history Train new workload from similar saved workloads models Similarity test; only train regions with low accuracy Prune configurations Find the boundary configurations meeting SLA Cut configurations with less resources than boundary ones

26 Outline Overview of different types of models Our approach: Chorus Optimizations: history and pruning Experimental results Exploit incomplete expert knowledge: gray box model Leverage individual models: ensemble learning Automated the modeling process: iteratively training

27 Evaluation Platform

28 Chorus Composition Gray box models GLR: region based linear model GINV: inverse shape based curve fitting model

29 Chorus Composition Gray box models Analytical models CPU and DISK GLR: region based linear model GINV: inverse shape based curve fitting model Black box models BSVM: support vector machine regression BCR: use average as prediction results per region

30 Evaluate Prediction Accuracy Measured Mean Accuracy: the percent of good predictions e.g. 3/5 = 60%

31 100% 80% 60% 40% 20% 0% Accuracy of Predictions Chorus B SVM B CR G INV G LR Chorus B SVM B CR G INV G LR Chorus B SVM B CR G INV G LR Orion Workload 15% 30% 60% Percentage of Total Training Samples 1 ~ 2 STD 0 ~ 1 STD

32 Orion Workload 100% Composition of Chorus 80% 60% 40% 20% GLR GINV BCR BSVM 0% 15% 30% 60% Percentage of Total Training Samples

33 100% 80% 60% 40% 20% 0% Accuracy of Predictions Chorus B SVM B CR G INV G LR Chorus B SVM B CR G INV G LR Chorus B SVM B CR G INV G LR TPCC Workload 15% 30% 60% Percentage of Total Training Samples 1 ~ 2 STD 0 ~ 1 STD

34 TPCC Workload 100% Composition of Chorus 80% 60% 40% 20% GLR GINV BCR BSVM 0% % 30% 60% Percentage of Total Training Samples

35 20% 0% Accuracy of Predictions 100% 80% 60% 40% Chorus CPU DISK Chorus CPU DISK Chorus CPU TPCW Workload 15% 30% 60% Percentage of Total Training Samples DISK 1 ~ 2 STD 0 ~ 1 STD

36 TPCC Resource Allocation Latency (ms) SLO TPC C TPC C TPC C TPC C Chorus Equal IDEAL* Allocation Scheme

37 Conclusion Key to implement QoS Cloud Build predictable systems Build accurate performance models Our Chorus can build accurate models: Exploit partial domain knowledge: gray box model Leverage individual models: ensemble learning From scratch and from history Allocate resources properly in QoS Cloud

38 Lessons and Future Work QoS Cloud is still difficult Some workload is hard to model with high accuracy May relax application goals, e.g. allow larger variations Feedback loop between admin. and Chorus Chorus suggests new model, and admin verifies it. Admin adds new model, and Chorus verifies it.

39 End. Thanks!

CLOUD applications and services are hosted at large data

CLOUD applications and services are hosted at large data 20 IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. 4, NO. 1, JANUARY-MARCH 2016 Ensemble: A Tool for Performance Modeling of Applications in Cloud Data Centers Jin Chen, Member, IEEE, Gokul Soundararajan, Member,