Nested Virtualization and Server Consolidation

Size: px

Start display at page:

Download "Nested Virtualization and Server Consolidation"

Jason Murphy
6 years ago
Views:

1 Nested Virtualization and Server Consolidation Vara Varavithya Department of Electrical Engineering, KMUTNB 1

2 Outline Virtualization & Background Nested Virtualization Hybrid-Nested Virtualization Experiment results Conclusions 2

3 Cloud Computing Cloud computing is the power of new generation data centers Users are accessing and deploying their application from anywhere On demand of user requirements Pay-Per-Use concept Everything as a Service Software as a Service Platform as a Service Infrastructure as a Service 3

4 Cloud Computing (cont.) Virtualization technology is a key feature of Cloud computing Virtualization allow users to quickly and easily create a virtual machine environments Satisfy the need to rapidly deploy and scale application 4

5 What is virtualization? Virtualization is a broad term (virtual CPU, Memory, storage, network, etc.) Virtualization basically allows one computer to do the job of multiple computers, by sharing the resources of a single hardware across multiple environments Virtual Container App. A App. B App. C App. D App. C App. D Operating System Virtualization Layer Nonvirtualized system A single OS controls all hardware platform resources App. A App. B Virtual Container Virtualized system It makes it possible to run multiple Virtual Containers on a single physical platform 5

6 Dynamic binary translation The emulation layer talks to an operating system which talks to the computer hardware The guest OS doesn't see that it is used in an emulated environment Applications OS VMM the interface of HW App. A App. B OS All of the hardware is emulated including the CPU Virtual Machine There is a complete functional ordering between layers Full abstraction of machine (from BIOS to disks, DMA controlers, video ) Virtualization is fully transparent: guest OS unchanged App. C Full virtualization Device Drivers Emulated Device Drivers Host OS

7 App. A App. B OS There is a potential gain in performance, due to specialization of kernel code Virtual Machine Not a really hierarchical ordering between layers Virtualization is similar to HW interface but neither complete nor identical OSs must be modified to become VMaware App. C Para-Virtualization Device Drivers Specialized API Virtual Machine Monitor For example, Main memories are divided for each guest OS. Applications Device Drivers Hypervisor OS his interface is VMM

8 OS-Level Virtualization Sometimes called OS-level or shared OS virtualization, Virtualization layer sit between the operating system and the application programs that run on the operating system Virtual Machine runs applications, or sets of applications written for the operating system but in a controlled environment OS is the same as the host OS, but appears isolated Lowest Overhead

9 Virtualization Software (Hypervisor) Full Virtualization VMware, QEMU, Adeos, Win4Linx, VirtualBox, Virtual PC, Parallels Workstation Para-Virtualization Xen, VMware ESXi server, Win4Lin 9x, IBM s LPARs OS-level Virtualization OpenVZ, FreeBSD Jails, Solaris Containers, Virtouzzo, Linux-Vserver

Intel Virtualization Technology Evolution Vector 3: I/O Focus PCI-SIG Standards for IO-device sharing:! Multi-Context I/O Devices! Endpoint Address Translation Caching!

10 Intel Virtualization Technology Evolution Vector 3: I/O Focus PCI-SIG Standards for IO-device sharing:! Multi-Context I/O Devices! Endpoint Address Translation Caching! Under definition in the PCI-SIG* IOVWG Vector 2: Platform Focus VT-d support for IO-device virtualization! Device DMA remapping! Direct assignment of I/O devices to VMs! Interrupt Routing and Remapping Vector 1: Processor Focus VT-x VT-i Establish foundation for virtualization in the IA-32 and Itanium architectures followed by on-going evolution of support: Micro-architectural (e.g., lower VM switch times) Architectural (e.g., Extended Page Tables) VMM Software Evolution Software-only VMMs! Binary translation! Paravirtualization Past No Support *Other names and brands may be claimed as the property of others Simpler and more Secure VMM through foundation of virtualizable ISAs Today Increasingly better CPU and I/O virtualization performance and functionality as I/O devices and VMMs exploit infrastructure provided by VT-x, VT-i, VT-d VMM software evolution over time with hardware support

11 Intel Xeon Processor E7 platform (Westmere-EX) More Performance More Expandable 10 cores / 20 threads Supports 32GB DDR3 DIMMs (2TB per 4socket system)1 30MB of last level cache More Security & RAS WSMWSM-EX WSMWSM-EX SECURITY More Efficient Intel Advanced Encryption Standard-New Instructions WSMWSM-EX WSMWSM-EX Intel Trusted Execution Technology (TXT) More performance within same max CPU TDP as Xeon 7500 Lower partial active & idle power via Intel Intelligent Power Technology2 RELIABILITY, AVAILABILITY, SERVICEABILITY Support for Low Voltage-DIMMs3 Double Device Data Correction (DDDC) Reduced power memory buffers4 Partial Memory Mirroring Delivers more Performance, Expandability and RAS while improving Energy Efficiency Up to 64 slots per standard 4 socket system x 32GB/DIMM = 2TB Uses similar core and package C6 power states enabled on Intel Xeon 5500/5600 series processors. Requires OS support. Savings dependent on workload and configuration. Example: At 100% SPECpower load it can save ~0.8W for 4GB DIMM DRx8 based on early Intel internal estimates Memory buffer power savings of up to 1.3W active and 3W idle per buffer based on internal Intel estimates. Slightly more savings when used with LV DIMMs 22 * Other names and brands may be claimed as the property of others. Copyright 2012, Intel Corporation. 11

12 Intel Xeon Processor E7 Family and Vmware* vsphere* 5.0 Vmware* vsphere* 5.0 scales to exploit the Intel Xeon processor E7 platform capabilities: 80-cores (160 threads) physical servers 2 TB physical RAM 32-vCPU VMs w/ 1TB virtual RAM 512 VMs, up to 2048 vcpus per physical host VM consolidation on this scale requires hypervisor focus on: Scalability, Energy Efficiency, & Uptime (Reliability) Especially Memory Reliability New Vmware* vsphere* 5.0 memory reliability features require Intel Xeon processor E7 family 27 * Other names and brands may be claimed as the property of others. Copyright 2012, Intel Corporation. 12

Advantages of Virtualization Separate the OS from the hardware Device support is part of the platform Write one device driver rather than N Better for system availability/reliability The key

13 Advantages of Virtualization Separate the OS from the hardware Device support is part of the platform Write one device driver rather than N Better for system availability/reliability The key capabilities for: High Availability computing resources, System Migration Application encapsulated with their OS Easy configuration and management Easy to create/destroy the guest OSs Independent to custom library Better utilize resources 13

14 Evolution of Virtualization App App App App Virtual Machine L3 App App App App App App Virtual Machine Virtual Machine App App Virtual Machine Virtual Machine Virtual Machine Virtual Machine L1 Virtual Machine L1 Single Core Multi-Cores Multi-Cores Single VM Multiple VMs Nested Virtualization 14

15 Virtualization Service Each virtual machine has a perception of real hardware platform Current VMM offer only a subset of virtual computing services Cannot run VMM over VMM, only VM over VMM Single layer services Is it possible to run VMM recursively? 15

16 Nested Virtualization Nested Virtualization App App allows virtual machine(s) Virtual Machine Ln to run inside another virtual machine. App App... Current CPU Virtual Machine Virtual Machine App App virtualization extension limited N to 2 level. Virtual Machine L1 Multi-Cores Virtual Machine L1 16

17 Nested Virtualization 2 types of nested technique Multiple level nested Multiplex in a single level nested L1 Hypervisor Hypervisor L1 L0 Host Hypervisor L0 Host Hypervisor Multiple levels support Multiplexing in a single level 17

18 Why Nested Virtualization User need more management on their own VMs, image configuration, etc. To run on cloud, it is easier to move a group of VMs. Security: encrypt all memory and I/O accesses. Hypervisor is now integrated with OS, i.e., Window 7. 18

19 Advantage of Nested virtualization Group of virtual machines Migration, Suspend, etc. Flexibility resource provisioning Cloud user can create multiple virtual machine on a single cloud instance Users manage their own VM with respective hypervisor Grouping instance reduce 25% of cost. 19

20 Requirementsof Nested Virtualization Running unmodified VMMs, such as VMware, Xen, KVM concurrently. Able to support other hypervisor Security Co-exist with x86 architecture Live migration of hypervisors 20

21 Existing Nested Virtualization IBM z/vm x86 software base Xen Nested Virtualization KVM IBM s Turtles project Support nested virtualization allow VMWare, Xen, Windows virtualization to run over it 21

22 Complexity in Nested Virtualization Nested CPU virtualization nested privilege instructions VMX virtualization Nested MMU virtualization Multi-dimensional paging Nested I/O virtualization Multi-level device assignment In the Turtles project, overhead is 6-14% 22

23 OS-Level Nested Virtualization Combination of hardware assisted virtualization and the OS-level virtualization for supported large-scale server consolidation. Propose hybrid-nested virtualization Study the performance of the hybrid-nested virtualization 23

24 Hybrid-Nested Virtualization Classification of hybrid-nested virtualization (L0 - L1) Full - Full virtualization Full - OS virtualization Para - Full virtualization Para - Para virtualization Para - OS virtualization Others techniques are not valid because of lacking support from hypervisor 24

25 Hybrid-Nested Virtualization Para-virtualization is used as the host hypervisor Based on multiplexing nested virtualization model in single level Partitioned Memory Share Partitioned Memory L1 L1 L1 L1 Hypervisor L1 Hypervisor L1 L0 Host Hypervisor L0 Host Hypervisor Classical Virtualization Nested virtualization for cloud computing 25

26 Experiments Details Performance metrics Computing intensive (HPL) Internet Service application (Apache Benchmark) Experiment model: (a). Classical para-virtualization (b). One OS-level in L1 over Para in (c). two OS-level with n/2 instance in L1 1 2 n L1 L1 L1 L1 L1 L Hypervisor L1 n 1 Hypervisor L1 2 n/2 1 Hypervisor L1 2 n/2 L0 Host Hypervisor L0 Host Hypervisor L0 Host Hypervisor (a) (b) (c) 26

27 Experiment Results 10" 9" 8" 7" HPL'Benchmark' Classical"Virtualiza7on" Nested"on"1VM"Host" Nested"on"2VM"Hosts" GFLOPS' 6" 5" 4" 3" 2" 1" 0" 2" 4" 6" 8" 10" 12" 14" 16" 18" 20" 22" 24" 26" 28" 30" 32" 34" 36" 38" 40" 42" 44" 46" 48" 2750" Number'of'VMs'

28 Experiment Results " Apache'Benchmark Connec&on'per'second' 10000" 1000" 100" Classical Virtualization Nested Virtualization 10" 1" HTML CGI PHP DB10 DB100 DB1000 DB

29 Conclusions Categorized combination of virtualization technology called Hybrid-nested virtualization OS-level virtualization over para-virtualization. Performance better 5% compared with classical virtualization High flexibilities in system management and administration as well as more efficient memory allocation 29

30 Conclusions Nested virtualization on next generation virtualization extension processor. Capability planning in Virtualization environment Management mechanism of multi-layer virtual system 30

31 Thank you Questions? 31

Virtualization. Pradipta De

Virtualization. Pradipta De Virtualization Pradipta De pradipta.de@sunykorea.ac.kr Today s Topic Virtualization Basics System Virtualization Techniques CSE506: Ext Filesystem 2 Virtualization? A virtual machine (VM) is an emulation