Optimizing and Enhancing VM for the Cloud Computing Era. 20 November 2009 Jun Nakajima, Sheng Yang, and Eddie Dong

Size: px

Start display at page:

Download "Optimizing and Enhancing VM for the Cloud Computing Era. 20 November 2009 Jun Nakajima, Sheng Yang, and Eddie Dong"

Miles Washington
5 years ago
Views:

1 Optimizing and Enhancing VM for the Cloud Computing Era 20 November 2009 Jun Nakajima, Sheng Yang, and Eddie Dong

Network, Storage More security requirements Encryption, etc.

2 Implications of Cloud Computing to Virtualization More computation and data processing More Memory More CPU Grid, Cluster, Parallel Computing, HPC More I/O activities Network, Storage More security requirements Encryption, etc. More VMs and power consumption Continue to minimize virtualization overheads and utilize H/W features as much as possible 2

3 Issues and Proposal 64-bit HVM (Hardware-based Virtual Machine) is at par or faster than 64-bit PV (Para-Virtualized) VM on new machines in market Ring0 kernel, Hardware Assisted Paging (HAP, e.g. EPT), Lower VM exit/entry cost More with Direct I/O (e.g. VT-d) and SR-IOV Future processors all have those More H/W virtualization optimizations/features are coming in the future Time to think about new VM architecture for Xen Xen para-virtualization is helpful for HVM guests Simplicity, performance, efficiency, scalability, correctness Dom0 performance should be improved if we run it in HVM on new machines Combine HVM and Para-Virtualization taking advantage of both Hybrid Virtualization ( HVM, PV) Jun Nakajima and Asit K. Mallick, Hybrid-Virtualization Enhanced Virtualization for Linux, in Proceedings of the Linux Symposium, Ottawa, June

4 PV Linux vs. HVM Linux HVM(v2.6.30) 30 PV(v2.6.18) 1000 Pvops(v2.6.30) HVM(v2.6.30) PV(v2.6.18) Pvops(v2.6.30) 0 fork proc exec proc Sh proc 0 The smaller is the better lmbench (above) Kernel build (lower right) 100 Performance regression with 50 pv_ops PV Linux vcpu 2 vcpu HVM PV pv_ops

5 Two Architectures of Hybrid Virtualization Enabled in virtualization on Xen Hybrid HVM Activated on machines with HAP (e.g. EPT) Hybrid Para-Virtualization Unmodified (HVM) Linux 3D* Unmodified (HVM) Linux 3D PV Linux PV Linux Xen API Guest BIOS 0D Guest BIOS* 0D Virtual Platform Virtual Platform* PV Xen API 0D PV Xen API nd: De-privileged (VMX-non root) Ring n HVM (HAP, Ring0, etc.) *: Native ACPI and Platform for Dom0 HVM Container HVM Container Single binary can support HVM, PV, and Hybrid HVM with pv_ops 5

6 Hybrid PV vs. Hybrid HVM Hybrid PV Pros: Does not require qemu-dm. Very quick to boot. Can be more aggressive in terms of para-virtualization Cons: Does not leverage native boot features More changes required (compared with hybrid HVM) Hybrid HVM Pros: Leverage native code, superset of native. Easier to re-use H/W virtualization features. Incremental approach (use Xen API as needed), fewer modifications Cons: HVM can be slower on old machines But fall back to PV anyway Need to modify code for native Hybrid HVM is better option 6

7 HVM Linux vs. Hybrid HVM Linux Example: Local APIC is not used by Hybrid HVM Linux EOI (End of Interrupt) does not cause VM exit. Use Xen API (event channel) MSI, MSI-X handling is para-virtualized in Hybrid HVM Linux MSI Mask/Unmask does not cause VM exit No changes are made to device drivers I/O intensive loads expose those virtualization overheads About 12K interrupts/per sec. (per VCPU) with 10 GbE (Intel Niantic, with Direct VT-d) Interrupt modulation enabled: InterruptThresholdRate = 8000 (default) CPU utilization is >3% higher per VCPU on ordinary HVM Linux EOI and MSI/MSI-X mask/unmask take up >60% of interrupt handling As number of VCPUs increases, overhead increases 7

8 HVM Dom0 Same binary can support both using pv_ops Activate hybrid HVM if machine has advanced virtualization features HVM dom0 is not implemented yet 1/3P Domain 0 (PV) Device Models (qemu-dm) Control Panel (xm/xend) Native Device Drivers PV Xen API Platform BIOS (ACPI) Backend Virtual driver HAP, 3D 0D Ring0, etc. Domain 0 (Hybrid HVM) Device Models (qemu-dm) Control Panel (xm/xend) Native Device Drivers PV Xen API Platform BIOS (ACPI) Backend Virtual driver Xen Hypervisor Xen Hypervisor Direct I/O (e.g. VT-d) 8

9 Enhancing VM in Virtualization Virtualization is better than real Something impossible on native machine does not necessarily mean impossible in virtualization. Opportunities for VM enhancements Example: CPUs change upon live migration Issues with live migration Least common set of CPU features are advertized in machine pool Missing opportunities to gain performance boost (e.g. 30%) Notifications for live migration Sent to kernel or special/performance-sensitive apps prepare for migration to notify of changes to resources on migration E.g. Apps know they will get new CPU features that can enhance performance complete migration E.g. Apps detect new CPU features for performance enhancements, and start using them 9

10 Summary and Current Status Minimize virtualization overheads and utilize new/advanced H/W features in VMs as much as possible for cloud computing Hybrid HVM is superset of HVM Optimized in virtualization, reduce virtualization overheads Allows to extend VM for virtualization Same binary for native (HVM), PV Linux, and Hybrid HVM with pv_ops Prototyped Hybrid HVM Support SMP, MSI/MSI-X Patches sent out for comments 10

Spring 2017 :: CSE 506. Introduction to. Virtual Machines. Nima Honarmand

Spring 2017 :: CSE 506. Introduction to. Virtual Machines. Nima Honarmand Introduction to Virtual Machines Nima Honarmand Virtual Machines & Hypervisors Virtual Machine: an abstraction of a complete compute environment through the combined virtualization of the processor, memory,