Evaluating CPU utilization in a Cloud Environment Presenter MSCS, KFUPM Thesis Committee Members Dr. Farag Azzedin (Advisor) Dr. Mahmood Khan Naizi Dr. Salahdin Adam ICS Department, KFUPM 6/9/2017
2 of 37 Outline Introduction Research Objectives Literature Review Present Research Objectiv es Conduct Literatur e Review Select Appropri ate Approac h to achieve objective s Conduct Performa nce Evaluatio n Collect Data Analyze and Conclud e Research Methodology Results and Discussion Conclusion
3 of 37 Introduction A datacenter is a facility composed of networked computers and storage that organizations use to organize, process, store, and disseminate large amounts of data Cloud computing allows users and businesses to access their data and applications from any device and anywhere in the world Virtualization is the fundamental element of cloud computing by which we can deliver resources or data as a service
4 of 37 Introduction Why Virtualization? Underutilized servers (unproductive) Maintenance cost High infrastructure cost High power consumption Disaster recovery Migration challenges Running legacy applications Difficulty in management of computing resources
5 of 37 Introduction Why Virtualization? Virtualization offers many advantages over traditional data centers Reducing fiscal costs Running legacy applications Easing computing resource management Ease of deployment Easing system migration Easing backups and disaster recovery Efficient resource utilization Many organizations are adopting virtualization technology to reduce the cost while maximizing the productivity, flexibility, responsiveness, and efficiency
6 of 37 Introduction What is Virtualization? Virtualization is a technology to run multiple operating systems on a single machine and giving an illusion that each OS is running on real hardware App App App App App App App App App App App Windows Hardware Windows Linux Virtual Machine Monitor Hardware Mac Regular System Virtualized Environment An example of running Windows, Linux, and Mac OSs on the same machine
7 of 37 Introduction Difference between regular & virtualized environment In a regular system, the hardware resources are shared by single operating system In virtualization environments, hypervisors are responsible to manage hardware resources and virtual machines Virtualization technology inserts an additional abstraction layer between the hardware and the operating system System resources such as CPU, main memory, and I/O are virtualized to virtual machine Every guest operating system is in charge of virtual resources and concurrently share and access the hardware resources, which mainly decreases the performance of the system App App App Windows Hardware Regular System App App App Windows Virtual Machine Monitor Hardware Virtualized Environment
8 of 37 Introduction Virtualization Shortcomings Virtual CPUs Virtual Machines configuration Virtual Machines Allocation vcpu4 vcpu3 vcpu2 VM1 vcpu3 vcpu2 VM2 vcpu2 VM2 Virtual Machine Monitor VMn pcpu pcpu pcpu pcpu Virtual CPUs Physical CPU Mapping Virtualized Environment Such shortcomings may lead to system performance degradation
9 of 37 Introduction Virtualization Shortcomings CPU Virtualization & Utilization CPU is the most significant and critical resource among all the available resources in a system CPU utilization is the amount of work handled by a CPU in a given time In virtualized environments, the higher the percentage of the CPU utilization results in the maximum performance CPU utilization is the metric that represents how busy is a processor
10 of 37 Problem Description CPU utilization in virtualized environments CPU installed on the host is only one set, but each VM that runs on the host requires their own CPU. Therefore, CPU needs to be virtualized Under Allocation Balance Allocation Over Allocation Over Allocation In virtualized environments, the higher the percentage of the CPU utilization results in the maximum performance
11 of 37 Problem Description CPU utilization in virtualized environments vcpu-vm allocation Hypervisor vcpu-pcpu mapping The BIOS setting for power management CPU Utilization Resource sharing between physical cores The choice of guest OS Resource sharing between logical cores Main factors that have implications on CPU utilization [23]
12 of 37 Problem Description CPU utilization in virtualized environments The aim is to investigate the best CPU resources allocation and better performance using state-of-theare hypervisors We focused, on hypervisor selection, vcpu-vm allocation, and vcpu-pcpu pinning strategies to find out the variation in CPU utilizations vcpu4 vcpu2 vcpu2 vcpu2 VM1 VM2 VM2 VMn Virtual Machine Monitor pcpu pcpu pcpu pcpu Virtualized Environment
13 of 37 Motivation #01 Virtual CPUs Virtual Machines configuration vcpu4 vcpu3 vcpu2 vcpu3 vcpu2 vcpu2 Virtual Machines Allocation VM1 VM2 VM2 VMn Virtual CPUs Physical CPU Mapping Virtual Machine Monitor pcpu pcpu pcpu pcpu Virtualized Environment We are motivated by the fact that virtualization suffers from drawbacks
14 of 37 Motivation #02 There are a variety of vendors for the virtualization environment Virtual Machine Monitor pcpu pcpu pcpu pcpu Virtualized Environment All virtualization vendors claim that their virtualization hypervisor is the best, however they depend on the used application When applying virtualization technology to an infrastructure environment, which hypervisor among others is better and faster in terms of CPU utilization?
15 of 37 Motivation #03 Researchers evaluate and analyze hypervisors without investigating the best vcpu-vm configuration and vcpu pcpu mapping through which better CPU utilization and performance for each hypervisor can be expected They assigned vcpu tovm based on non-suitable configuration Prior to deploying VM in a cloud environment, which vcpu VM configuration is the best to improve CPU utilization? vcpu4 vcpu3 vcpu2 VM1 vcpu3 vcpu2 VM2 vcpu2 VM2 Virtual Machine Monitor VMn pcpu pcpu pcpu pcpu Virtualized Environment After VM deployment, which vcpupcpu pinning strategies is the best to improve CPU utilization?
16 of 37 Research Objectives
Literature Review 17 of 37 Work Hypervisor Used Architecture Virtualization Type Benchmarking Tool Hypervisor Comparison vcpu-vm Configuration vcpu-pcpu Mapping Hwang et al. [1] S. Varette et al, [14] VMware EXSi, Hyper-V2008, KVM 2.6, Xen Hyper-V, KVM, VirtualBox, VMware CMP CMP HAV FV, HAV Ramspeed, Netperf Randomly Randomly HPL Randomly Randomly Mancas [33] Vmware, KVM Not Mention Not Mention Passmask Randomly Randomly Babu et al [35] Charles David [31] OpenVZ, XenServer, Xen KVM (RHEL 5), Xen 3.1.2 : Authors compared different hypervisors SMP SMP Container, PV, FV HAV, PV Zong et al. [16] Xen 3.4.0 SMP PV Unixbench Randomly Randomly PTS Randomly Randomly Apache TPC-H * Randomly Not Randomly Kourai et al, [10] Xen 4.4.03 SMP PV Tascell * Not Randomly Randomly Sogand et al. [11] VMware EXSi 5.5 NUMA Unknown Vcenter Performace, SSH + Sar * Not Randomly Randomly
18 of 37 Proposed Methodology Present Research Objectives Conduct Literature Review Select Appropriate Approach to achieve objectives Conduct Performance Evaluation Collect Data Analyze and Conclude
19 of 37 Proposed Methodology Architecture
20 of 37 Experimental Design N-Queens problem is a classical combinatorial problem, that have different workloads The problem involves placing N queens on an N x N chessboard such that no queen can attack any other A solution requires that no two queens share the same row, column, or diagonal As the problem size increases, the corresponding possible solutions and the elapsed time to solve the problem is also increasing We tested each hypervisor for different queens size ranges from 4 to 19 For a regular-sized board (8 x 8), there are 92 distinct solutions,
21 of 37 Experimental Design
22 of 37 Results and Discussion For the clarity of presentation, the Results and Discussion is divided into five sections: (1) The Effect of Virtualization Technology on Performance (2) The Effect of Virtual Machines on Performance (3) The Effect of Virtual CPUs on Performance (4) The Significance of Over Allocation on Performance (5) The Effect of Pinning Strategies on Performance Regular System Under Allocation Under Allocation Uniform Configuration Pinning Strategy Hypervisor#01 Balance Allocation Balance Allocation Un-uniform Configuration No Pinning Strategy Hypervisor#02 Over Allocation Over Allocation
Results and Discussion 23 of 37 1. The Effect of Virtualization Technology M M Low High Low High M Low High Ubuntu Hardware Ubuntu Citrix XenServer Hardware Ubuntu KVM Hardware Regular System Virtualized Environment Virtualized Environment
Results and Discussion 24 of 37 1. The Effect of Virtualization Technology N-Queens problem with elapsed time (second) The effect of Virtualization Layer using N-Queens benchmark
Results and Discussion 25 of 37 1. The Effect of Virtualization Technology E: The maximum error with one degree of confidence, (alpha): using two tail distribution, s: standard deviation, n: number of samples The effect of Virtualization Layer using John-the-Ripper benchmark
Results and Discussion 26 of 37 2. The Effect of Virtual Machines on Performance The Effect of Virtual Machines on Performance Under Allocation Balance Allocation Over Allocation
Results and Discussion 27 of 37 2. The Effect of Virtual Machines on Performance <<< Under Allocation >>> 5 3 1 6 4 2 vcpu8 vcpu7 vcpu6 vcpu8 vcpu7 vcpu6 vcpu9 vcpu7 vcpu5 vcpu3 0 vcpu8 vcpu6 vcpu4 vcpu2 vcpu5 vcpu4 vcpu3 vcpu2 vcpu5 vcpu4 vcpu3 vcpu2 vcpu4 vcpu3 vcpu2 vcpu4 vcpu3 vcpu2 vcpu2 vcpu2 vcpu2 vcpu2 VM#3 VM#4 VM#5 VM#6 VM#7 VM#8 VM#1 VM#1 VM#2 VM#1 VM#2 VM#1 VM#2 VM#3 VM#4 Virtual Machine Monitor pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 pcpu7 pcpu8 pcpu9 pcpu10 pcpu11 pcpu12 pcpu13 pcpu14 pcpu15 pcpu16 pcpu17 pcpu18 pcpu19 pcpu20 pcpu21 pcpu22 pcpu23 pcpu24 Virtualized Environment
Results and Discussion 28 of 37 2. The Effect of Virtual Machines on Performance <<< Under Allocation >>> The effect of Virtual Machines on Performance - Under Allocation
Results and Discussion 29 of 37 2. The Effect of Virtual Machines on Performance << Balance Allocation >> The effect of Virtual Machines on Performance - Balance Allocation
Results and Discussion 30 of 37 2. The Effect of Virtual Machines on Performance << Balance Allocation >> Linear Regression model for Balance Allocation
Results and Discussion 31 of 37 2. The Effect of Virtual Machines on Performance < Over Allocation > The effect of Virtual Machines on Performance - Over Allocation
Results and Discussion 32 of 37 2. The Effect of Virtual Machines on Performance < Comparison > The effect of Virtual Machines on Performance: Interval Plot of Elapsed time (sec) for Citrix XenServer and KVM Hypervisors (Problem Size: 19)
Results and Discussion 33 of 37 2. The Effect of Virtual Machines on Performance < Comparison > The effect of Virtual Machines on Performance (Under allocation, Balance Allocation, and Over allocation comparison)
Results and Discussion 34 of 37 3. The Effect of Virtual CPUs on Performance (Comparison) The effect of Virtual CPUs on Performance - Citrix XenServer Total CPU Utilization Hypervisor Level - Citrix XenServer The effect of Virtual CPUs on Performance - KVM Total CPU Utilization Hypervisor Level - KVM
Results and Discussion 35 of 37 4. The Significance of Over Allocation and Pinning Strategies 1 2 vcpu23 vcpu24 1 2 vcpu9 vcpu7 0 vcpu8 vcpu21 9 vcpu22 vcpu20 vcpu9 vcpu7 0 vcpu8 vcpu5 vcpu6 7 8 vcpu5 vcpu6 vcpu3 vcpu4 5 6 vcpu3 vcpu4 vcpu2 3 4 vcpu2 VM#1 VM#2 VM#1 Virtual Machine Monitor pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 pcpu7 pcpu8 pcpu9 pcpu10 pcpu11 pcpu12 pcpu13 pcpu14 pcpu15 pcpu16 pcpu17 pcpu18 pcpu19 pcpu20 pcpu21 pcpu22 pcpu23 pcpu24 Virtualized Environment Virtual Machine Monitor pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 pcpu7 pcpu8 pcpu9 pcpu10 pcpu11 pcpu12 pcpu13 pcpu14 pcpu15 pcpu16 pcpu17 pcpu18 pcpu19 pcpu20 pcpu21 pcpu22 pcpu23 pcpu24 Virtualized Environment
Results and Discussion 36 of 37 4. The Significance of Over Allocation and Pinning Strategies vcpu47 vcpu47 vcpu47 vcpu48 vcpu47 vcpu48 : : : : : : : : : : : : : : : : : : 1 2 vcpu23 vcpu24 1 2 vcpu9 vcpu7 0 vcpu8 vcpu21 9 vcpu22 vcpu20 vcpu9 vcpu7 0 vcpu8 vcpu5 vcpu6 7 8 vcpu5 vcpu6 vcpu3 vcpu4 5 6 vcpu3 vcpu4 vcpu2 3 4 vcpu2 VM#1 VM#2 VM#1 Virtual Machine Monitor pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 pcpu7 pcpu8 pcpu9 pcpu10 pcpu11 pcpu12 pcpu13 pcpu14 pcpu15 pcpu16 pcpu17 pcpu18 pcpu19 pcpu20 pcpu21 pcpu22 pcpu23 pcpu24 Virtualized Environment Virtual Machine Monitor pcpu1 pcpu2 pcpu3 pcpu4 pcpu5 pcpu6 pcpu7 pcpu8 pcpu9 pcpu10 pcpu11 pcpu12 pcpu13 pcpu14 pcpu15 pcpu16 pcpu17 pcpu18 pcpu19 pcpu20 pcpu21 pcpu22 pcpu23 pcpu24 Virtualized Environment
Results and Discussion 37 of 37 4. The Significance of Over Allocation and Pinning Strategies The effect of Virtual CPUs on Performance - Uniform vcpus per VMs CPU Utilization for uniform vcpus The effect of Virtual CPUs on Performance - Non-uniform vcpus per VMs CPU Utilization for non-uniform vcpus
Results and Discussion 38 of 37 The Effect of Hypervisors, Virtual Machines and Virtual CPUs on Performance (Concluding Remarks) Mean Elapsed time of Hypervisors, VMs, and vcpus
Results and Discussion F i r s t S c e n a r i o 39 of 37 The Effect of Pinning Strategies on Performance (Concluding Remarks) S e c o n d S c e n a r i o Normal distribution of Elapsed Time (sec), uniformed vcpu allocation Normal distribution of Elapsed Time (sec), un-uniformed vcpu allocation
40 of 37 Conclusion and Future Work Using CPU-bound operations, the results obtained from this evaluation showed that commercial (Citrix Xenserver) and opensource (KVM) hypervisors showed similar performance in terms of elapsed time and CPU utilization The performance of a system would degrade by running many VMs, improper allocation of vcpus to VMs, or using unsuitable vcpus - pcpus pinning strategies The elapsed time increases, when there is a massive over allocation of vcpus The best performance (elapsed time) was gained when there was only few active VMs with balance allocation and over allocation (over allocation using no pinning strategies) Pinning and no pinning strategies have no effect on under allocation and balance allocation of vcpus-vms or overallocation having same workload is running on each VM
41 of 37 Conclusion and Future Work CPU utilization during live migration of VMs from one host to another is one of the future directions for this work The obtained results from our evaluation experiments can be validated by comparing using other commercial hypervisors (VMware and Hyper V). This will aid the cloud service providers in choosing the best cloud platform CPU utilization for I/O bound operations has not been investigated We suggested that the cloud service providers and researchers should consider the effects of massive over allocation of vcpus, VMs, and vcpus-pcpus mapping when they choose deployment strategies for vcpus-vms configuration and vcpu-pcpu mapping
42 of 37 Acknowledgements I would like to acknowledge the support provided by King Fahd University of Petroleum and Minerals (KFUPM) Information & Computer Science Department My Advisor: Dr. Farag Azzedin Committee Members: Dr. Mahmood Khan & Dr. Salahdin Adam All friends in KFUPM Thank You!
43 of 37
44 of 37
Results and Discussion 45 of 37 3. The Effect of Virtual CPUs on Performance (Citrix XenServer) The effect of Virtual CPUs on Performance - Citrix XenServer
Results and Discussion 46 of 37 3. The Effect of Virtual CPUs on Performance (Citrix XenServer) Total CPU Utilization Hypervisor Level - Citrix XenServer
Results and Discussion 47 of 37 3. The Effect of Virtual CPUs on Performance (KVM) The effect of Virtual CPUs on Performance - KVM
Results and Discussion 48 of 37 3. The Effect of Virtual CPUs on Performance (KVM) Total CPU Utilization Hypervisor Level - KVM
Results and Discussion 49 of 37 F i r s t S c e n a r i o 4. (a) The Significance of Over Allocation << Uniform vcpus per VMs >> The effect of Virtual CPUs on Performance - Uniform vcpus per VMs CPU Utilization for uniform vcpus No Pinning strategy: The hypervisor is free to schedule domain s vcpus on any pcpus Pinning strategy: The hypervisor is free to schedule the Dom0 vcpus on any pcpus and other active VMs vcpus are statically pinned to user define logical CPUs Both strategies have no effect on under allocation and balance allocation of vcpus-vms
Results and Discussion 50 of 37 S e c o n d S c e n a r i o 4. (b) The Significance of Over Allocation << Un-uniform vcpus per VMs >> The effect of Virtual CPUs on Performance - Non-uniform vcpus per VMs CPU Utilization for non-uniform vcpus
Results and Discussion 51 of 37 Elapsed time to solve N-Queen Problem using Citrix XenServer
Results and Discussion 52 of 37 Elapsed time to solve N-Queen Problem using KVM
Types of Hypervisor 53 of 37
54 of 37
59 of 37 CMPs use relatively simple single-thread processor cores to exploit only moderate amounts of parallelism within any one thread, while executing multiple threads in parallel across multiple processor cores. SMP - Symmetric multiprocessing - the case where two or more processors (identical or non-identical) are connected via an interconnect and have equal access to all other system resources like memory, IO ports, etc....
60 of 37 NUMA NUMA - Non-uniform memory access - the case where in an SMP system the processors are not equi-distant from the shared system memory. i.e., a processor may be physically at varying distances from a given set of memory regions.
61 of 37 The cloud service provider assigns resources to each user request aiming to minimize resource allocation and fulfill user requirements