Unit 2. VMMs and hypervisors 2966-Network and Services Virtualisation First semester Assistant professor: Katja Gilly Departament: Physics

Transcription

1 Unit 2. VMMs and hypervisors 2966-Network and Services Virtualisation First semester Assistant professor: Katja Gilly Departament: Physics and Computer Architectures

2 Outline Introduction Xen project KVM Vmware Usages in server platforms Usages in client platforms

3 Outline Introduction Xen project KVM Vmware Usages in server platforms Usages in client platforms

4 Introduction System virtual machines are capable of virtualising a full set of hardware resources, including a processor (or processors), memory and storage resources and peripheral devices. Constructed at ISA level Allow multiple environments, or support time sharing. Examples IBM VM/360 VMware Xen KVM OKL4

5 Introduction Main Theorem A virtual machine monitor can be constructed if the set of sensitive instructions is a subset of the set of privileged instructions Proof shows 1. Equivalence by interpreting privileged instructions and executing remaining instructions natively 2. Resource control by having all instructions that change resources trap to the VMM 3. Efficiency by executing all non-privileged instructions directly on hardware A key aspect of the theorem is that it is easy to check

6 Emulation & virtualisation Emulation seems a good way to implement VMM. We can run a guest above emulator. Emulators can manage all hardware resource and arrange sharing resource to the guest However, there are rarely people using emulator as VMM. WHY? Because emulation is quite SLOW! It s not Efficient! How to let it faster? Don t emulate everything. Just emulate some sensitive instructions which will directly access hardware resource. Execute all non-privileged instructions directly on hardware

7 Full-virtualisation Definition: We run the guest without modified. The guest doesn t realise that it is running above VM rather than physical machine. Pro: User can use any what he/she wants to install as guest For those which is hard to patch (ex: Windows, because it s hard to get its source code), user can only install them in full-virtualisation environment Con: For non-virtualisable CPU, running guest without patched critical instruction need use Dynamic Binary Translation in the hypervisor. It costs a lot. Even for the virtualisable or hardware assistant CPU, running an which doesn t find out it's in VM rather in physical machine, it is still lots of computing resource. Using patched guest can avoid these resource wasting. And using full-virtualisation environment cannot gain the performance optimisation.

8 Para-virtualisation Definition: Run the guest which is patched for virtualisation. The guest realises that it is running above VM rather than physical machine. Pro: For non-virtualisable CPU, running guest with patched critical instruction can reduce lots of work for hypervisor. Let guest run faster. Even for the virtualisable or hardware assistant CPU, running an which doesn t find out it's in VM rather in physical machine, it is still lots of computing resource. Using patched guest can avoid these resource wasting. Con: User cannot use any what he/she wants to install as guest For those which is hard to patch (ex: Windows, because it s hard to get its source code), user cannot install them as guest.

9 Several types of VMM According to the category from Popek and Goldberg in 1974, virtual machine monitor can be separate into two major type which are majorly category from where the hypervisor is. Type 1 a.k.a. Bare-metal VMM Type 2 a.k.a. Hosted VMM

10 Bare-Metal VMM

11 Bare-Metal VMM VMM is responsible for scheduling and managing the allocation of HW resources Example: Xen Hyper-V VMware workstation

12 Hosted VMM

13 Hosted VMM VMM is built on top of an existing Installation process is similar to the installation of an APP Let the host to provide device drivers and other low-level services Can patch privileged instructions to VMM calls (traps), or using DBT techniques Example: VMware player KVM Parallels

14 Comparison with native and hosted VMs Virtual Machine Applications Virtual Machine VMM Virtual Machine VMM VMM Host Host Hardware Hardware Hardware Hardware Traditional uniprocessor system Native VM system User-mode Hosted VM system Dual-mode Hosted VM system Non-privileged modes Privileged Mode

15 Outline Introduction Xen project KVM Vmware Usages in server platforms Usages in client platforms

16 Xen Project bio Xen project was created in 2003 at the University of Cambridge Computer Laboratory in what's known as the Xen Hypervisor project Led by Ian Pratt with team members Keir Fraser, Steven Hand, and Christian Limpach. This team along with Silicon Valley technology entrepreneurs Nick Gault and Simon Crosby founded XenSource which was acquired by Citrix Systems in October 2007 The Xen hypervisor is an open source technology, developed collaboratively by the Xen community and engineers (AMD, Cisco, Dell, HP, IBM, Intel, Mellanox, Network Appliance, Novell, Red Hat, SGI, Sun, Unisys, Veritas, Voltaire, and of course, Citrix) Xen is licensed under the GNU General Public License Xen supports Linux 2.4, 2.6, Windows and NetBSD 2.0 Name from next generation virtualization 17

17 Xen Components A Xen virtual environment consists of several modules that provide the virtualisation environment: Xen Hypervisor - VMM Domain 0 Domain Management and Control Domain User, can be one of: Paravirtualised Guest: the kernel is aware of virtualisation Hardware Virtual Machine Guest: the kernel runs natively 18

18 Xen Components Domain 0 Domain Managemen t and Control Domain U Domain U Paravirtual Guest Domain U Paravirtual Guest Paravirtual Guest Domain U Domain U HVM Guest U Domain HVM Guest HVM Guest Hypervisor - VMM 19

19 Xen Components When Xen boots, one of the first things it does is load a Domain 0 (dom0) guest kernel. Domain 0 is the first guest to run, and has elevated privileges. Domain 0 is very important to a Xen system. Xen does not include any device drivers by itself, nor a user interface. These are all provided by the operating system and userspace tools running in the dom0 guest The most obvious task performed by the dom0 guest is to handle devices. This guest runs at a higher level of privilege than others, and so can access the hardware In contrast, other domains are referred to as domain U (domu ) the U stands for unprivileged. However, it is now possible to delegate some of dom0 s responsibilities to domu guests, which blurs this line slightly. 20

20 Xen Components 21

21 Xen Components There are three components to any driver: The split driver The multiplexer The real driver The split driver is typically as simple as it can be. It is designed to move data from the domu guests to the dom0 guest, usually using ring buffers in shared memory. The real driver should already exist in the dom0 operating system, and so it cannot really be considered part of Xen. The multiplexer may or may not. In the example of networking, the firewalling component of the network stack already provides this functionality. In others, there may be no existing operating system component that can be pressed into use. 22

22 Xen Hypervisor - VMM The hypervisor is Xen itself. It goes between the hardware and the operating systems of the various domains. The hypervisor is responsible for: Checking page tables Allocating resources for new domains Scheduling domains. Booting the machine enough that it can start dom0. It presents the domains with a VirtualMachine that looks similar but not identical to the native architecture. Just as applications can interact with an by giving it syscalls, domains interact with the hypervisor by giving it hypercalls. The hypervisor responds by sending the domain an event, which fulfills the same function as an IRQ on real hardware. A hypercall is to a hypervisor what a syscall is to a kernel. 23

23 Xen Hypervisor - VMM Thin, privileged abstraction layer between the hardware and operating systems Defines the virtual machine that guest domains see instead of physical hardware: Grants portions of physical resources to each guest Exports simplified devices to guests Enforces isolation among guests 24

24 Restricting operations with Privilege Rings The hypervisor executes privileged instructions, so it must be in the right place: x86 architecture provides 4 privilege levels / rings Most s were created before this implementation, so only 2 levels are used Xen provides 2 modes: In x86 the applications are run at ring 3, the kernel at ring 1 and Xen at ring 0 In x86 with VT-x, the applications run at ring 3, the guest at ring non-root-0 and Xen at ring root-0 (-1) 25

25 Restricting operations with Privilege Rings Native Paravirtual x HVM x86 The Guest is moved to ring The Hypervisor is moved to ring -1 Applications Guest kernel (dom0 and dom U) Hypervisor 26

26 Domain 0 Domain 0 is a Xen required Virtual Machine running a modified Linux kernel with special rights to: Access physical I/O devices Two drivers are included in Domain 0 to attend requests from Domain U PV or HVM guests Interact with the other Virtual Machines (Domain U) Provides the command line interface for Xen daemons Due to its importance, the minimum functionality should be provided and properly secured Some Domain 0 responsibilities can be delegated to Domain U (isolated driver domain) 27

27 Domain 0 Domain 0 PV Network backend driver Block backend driver HVM Qemu-DM Communicates directly with the local networking hardware to process all virtual machines requests Communicates with the local storage disk to read and write data from the drive based upon Domain U requests Supports HVM guests for networking and disk access requests 28

28 Domain Management and Control - Daemons The Domain Management and Control is composed of Linux daemons and tools: Xm Command line tool and passes user input to Xend through XML RPC (creates and manages guest VMs) Xend Python application that is considered the system manager for the Xen environment (Daemon to communicate with the hypervisor) Libxenctrl A C library that allows Xend to talk with the Xen hypervisor via Domain 0 (privcmd driver delivers the request to the hypervisor) Xenstored Maintains a registry of information including memory and event channel links between Domain 0 and all other Domains Qemu-dm Supports HVM Guests for networking and disk access requests 29

29 Domain U Paravirtualised guests The Domain U PV Guest is a modified Linux, Solaris, FreeBSD or other UNIX system that is aware of virtualisation (no direct access to hardware) No rights to directly access hardware resources, unless especially granted Access to hardware through front-end drivers using the split device driver model Usually contains XenStore, console, network and block device drivers There can be multiple Domain U in a Xen configuration 30

30 Domain U Paravirtualised guests Domain U - PV Console driver XenStore driver Network front-end driver Block front-end driver Similar to a registry Communicates with the Network backend driver in Domain 0 Communicates with the Block backend driver in Domain 0 31

31 Domain U HVM guests The Domain U HVM Guest is a native with no notion of virtualisation (sharing CPU time and other VMs running) An unmodified doesn t support the Xen split device driver, Xen emulates devices by borrowing code from QEMU HVMs begin in real mode and gets configuration information from an emulated BI For an HVM guest to use Xen features it must use CPUID and then access the hypercall page Domain U - HVM Xen virtual firmware Simulates the BI for the unmodified operating system to read it during startup 32

32 Pseudo-Physical to Memory Model In an operating system with protected memory, each application has it own address space. A hypervisor has to do something similar for guest operating systems. Application Virtual Kernel Pseudo-physical Hypervisor Machine The triple indirection model is not necessarily required but it is more convenient from the performance point of view and modifications needed in the guest kernel. If the guest kernel needs to know anything about the machine pages, it has to use the translation table provided by the shared info page (rare) 33

33 Pseudo-Physical to Memory Model There are variables at various places in the code identified as MFN, PFN, GMFN and GPFN PFN (Page Frame Number) It means some kind of page frame number. The exact meaning depends on the context MFN (Machine frame number) Number of a page in the (real) machine s address space GPFN (Guest page frame number) These are page frames in the guest s address space. These page addresses are relative to the local page tables GMFN (Guest machine frame number) This refers to either a MFN or a GPFN, depending on the architecture 34

34 Virtual Ethernet interfaces Xen creates, by default, seven pair of "connected virtual ethernet interfaces" for use by dom0 For each new domu, it creates a new pair of "connected virtual ethernet interfaces", with one end in domu and the other in dom0 Virtualized network interfaces in domains are given Ethernet MAC addresses (by default xend will select a random address) The default Xen configuration uses bridging (xenbr0) within domain 0 to allow all domains to appear on the network as individual hosts 35

35 The Virtual Machine lifecycle Xen provides 3 mechanisms to boot a VM: Booting from scratch (Turn on) Restoring the VM from a previously saved state (Wake) Clone a running VM (only in XenServer) Stop PAUSED Resume Start Pause (paused) Turn on OFF Turn off Turn off Wake RUNNING Migrate Sleep SUSPENDED 36

36 Outline Introduction Xen project KVM Vmware Usages in server platforms Usages in client platforms

37 KVM Overview It is a VMM built within the Linux kernel The name stands for Kernel Virtual Machines It is included in mainline Linux, as of It offers full-virtualisation Para-virtualisation support is in alpha state It works *only* in platforms with hardware-assisted virtualisation Currently only Intel-VT and AMD-V Recently also s390, PowerPC and IA64 Decision taken to achieve a simple design No need to deal with ring aliasing problem, Nor excessive faulting avoidance Nor guest memory management complexity Etc 38

38 Why KVM? Today s hardware is becoming increasingly complex Multiple HW threads on a core Multiple cores on a socket Multiple sockets on a system NUMA memory models (on-chip memory controllers) Scheduling and memory management is becoming harder accordingly Great effort is required to program all this complexity in hypervisors But an operating system kernel already handles this complexity So why no reuse it? KVM makes use of all the fine-tuning work that has gone (and is going) into the Linux kernel, applying it to a virtualised environment Minimal footprint Less than 10K lines of kernel code Implemented as a Linux s module 39

39 How does it work? A normal Linux process has two modes of execution: kernel and user KVM adds a third mode: guest mode A virtual machine in KVM will be seen as a normal Linux process A portion of code will run in user mode: performs I/O on behalf of the guest A portion of code will run in guest mode: performs non-i/o guest code 40

40 How does it work? guest mode With its own 4 rings 41

41 Key features Simpler design: Kernel+Userspace (vs. Hypervisor + Kernel + Userspace) Avoids many context switches Code reuse (today and tomorrow) Easy management of VMs (standard process tools) Supports Qcow2 and Vmdk disk image formats Growable formats (copy-on-write) Advanced guest memory management Increased VM density with KSM (under development) KSM is a kernel module to save memory by searching and merging identical pages inside one or more memory areas Guest s page swapping allowed 42

42 Future trends Para-virtualisation support (Windows & Linux) virtio devices already included in Linux s mainline as of Storage Many similar guests cause a lot of duplicate storage Current solution: baseline + delta images Delta degrades overtime (needs planning) Disk-in-file is overheady 43

43 Outline Introduction Xen project KVM Vmware Usages in server platforms Usages in client platforms

44 VMware In 1998, VMware created a solution to virtualise the x86 platform, creating the market for x86 virtualisation The solution was a combination of binary translation and direct execution on the processor Nonvirtualisable instructions are replaced with new sequences of instructions User level code is directly executed on the processor Each VMM provides each VM with all the services of the physical system, including a virtual BI, virtual devices and virtualised memory management 46

45 VMware 47

46 VMware Basic properties: Separate and hardware break hardware dependencies and Application as single unit by encapsulation Strong fault and security isolation Standard, HW independent environments can be provisioned anywhere Flexibility to choose the right for the right application 48

47 Vmware major products VMware Server A free-of-charge virtualisation-software server suite Run multiple servers on your server Hosted architecture Available for Linux hosts and Windows hosts VMware ESX Server An enterprise-level computer virtualisation product Quality of service High-performance I/O Host-less architecture ( bare-metal ) 49

48 VMware ESX architecture

49 VMware ESX architecture Datacenter-class virtualisation platform used by many enterprise customers for server consolidation Runs directly on a physical server having direct access to the physical hardware of the server Parts: Virtualisation layer (VMM/VMKernel): implements the idealised hardware environment and virtualises the physical hardware devices Resource Manager: partitions and controls the physical resources of the underlying machine Hardware interface components: enable hardware-specific service delivery Service Console: boots the system, initiates execution of the virtualisation layer and resource manager, and relinquishes control to those layers Add Virtual Centre / Lab manager 51

50 VMware ESX architecture 52

51 VMware default deployment Primary method of interaction with virtual infrastructure (console and GUI) Authorises VirtualCenter Servers and ESX Server hosts appropriately for the licensing agreement Virtualisation layer that abstracts the processor, memory, storage, and networking resources of the physical host into multiple virtual machines VI Client from the VirtualCenter Server or ESX Server hosts Organises all the configuration data for the virtual infrastructure environment Centrally manages the VMware ESX Server hosts 53

52 VMware for free VMware provides freeware Server and Workstation virtualisation solutions VMware Server: Is a free desktop application that lets you run virtual machines on your Windows or Linux PC Lets you use host machine devices, such as CD and DVD drives, from the virtual machine Datasheet or FAQ page is available Different Virtual Appliances are provided for free VMware Player: Similar to VMware Server but limited to run pre-built virtual appliances 54

53 Outline Introduction Xen project KVM Vmware Usages in server platforms Usages in client platforms

54 Virtualisation in servers 56

55 Workload Consolidation description Too many servers: Hot and underutilised Server virtualisation consolidates many systems onto one physical platform Higher physical resource utilization Improved utilization: Several users with different requirements can more easily share a virtualised server It allows to run legacy and new operating systems concurrently VM1 App VM1 VMn App App HW0 HWn VMn App Focus in Equivalence prop VMM HW Hardware is more stressed: - More intense IO 57 - More intense mem usage

56 Workload Consolidation pros/cons Pros: Each application can run in a separate environment delivering true isolation Cost Savings: Power, space, cooling, hardware, software and management Ability to run legacy applications in legacy s Ability to run through emulation legacy applications in legacy HW Cons: Disk and memory footprint increase due to multiples s Performance penalty caused by resource sharing management Workload consolidation provides the basis most usages/benefits of virtualisation 58

57 Workload Isolation Virtualisation can improve overall system security and reliability by isolating multiple software stacks in their own VMs Security: intrusions can be confined to the VM in which they occur Reliability: software failures in one VM do not affect the other VMs As a side effect, if the hypervisor or drivers are compromised, the whole VMs can be compromised (equivalent to BI VM1 VMn attack) App App HWn App App VMM HW 59 Focus in Efficiency prop (P&G)

58 Workload migration for dynamic load balancing If a given application needs more resources, it could be easily moved to other physical host with more power Several application replicas can be started in different physical machines and split the workload among them While the applications should be engineered for this behaviour, the deployment is simplified VM1 App VM1 VMn App App VMn App VMM VMM HW0 HWn What happens with the established connections, mem pages, disk? 60

59 Workload migration for disaster recovery High Availability: if an application goes down, it is not necessary to wait for the reboot of the operating system/application Virtualisation allows to immediately put another replica up State-less applications can be immediately restored from snapshots Disk images should exist in external network storage (SAN, NAS) VM1 VMn VM1 VMn App App App App VMM VMM VMM HW0 HW0 HW How often should the VMM store the VM s status? 61

60 Deployment of standard environments Service providers usually offer some standard services Standard images can be provided instantaneously Simplifies deployment procedures: everything is stored in a file that represents the VM Easier backward compatibility (Gold Image 1, 2, 3, etc) Gold Image VM1 VM2 VM3 VMn App App App App App VMM VMM VMM HW0 HW0 HW1 62

61 Test and deployment Development and testing environments A VM with standard tools is distributed amongst developers Releasing new revisions of tools, patches, etc. is very simple Business Agility and Productivity It allows to easily transform environments (Development to test, back to development, etc) Deployment of Patches in controlled environments Allows for testing in production hardware before formal activation App App VMM HW 63

62 Virtual appliances The convergence of virtual machine technology and a new initiative by several tool vendors is giving birth to this new form of software packaging called Virtual Appliance Software appliance that includes necessary components to run in a virtual computing environment (VMware, Xen, Virtual Iron, ) Both, software and virtual appliances, are aimed to eliminate the installation, configuration and maintenance costs associated with running complex stacks of software App. Middleware Virtual Platform Virtual Appliance A software appliance is a software application combined with a tailored operating system Software Appliance Fully pre-installed and pre-configured application and operating system environment Virtual Machine 64

63 Outline Introduction Xen project KVM Vmware Usages in server platforms Usages in client platforms

64 Windows games emulation in Mac Games are very dependant of the platform for which they were programmed Creating games for each platform is too costly Minimal emulation is needed for minimal performance degradation Game App HW VMM HW 66

65 Test and Development also for clients Development and testing environments A VM with standard tools is distributed amongst developers Releasing new revisions of tools, patches, etc. is very simple Business Agility and Productivity It allows to easily transform environments (Development to test, back to development, etc) Deployment of Patches in controlled environments Simplifies testing of P2P applications App App VMM HW 67

66 Desktop infrastructure management Migration to new hardware due to compatibility is too costly Windows Vista slow adoption is an example Gradual upgrade can be implemented Desktops are underutilised (as opposed to previous) Consolidate in the server, run remotely in the client VM1 VM2 VM2 App App App HW VMM VM1 HW (new) Connectivity becomes the critical path App HW 68

67 My corporate PC within my PC Users sharing their own PC with the corporate A VM for own work A VM for corporate usages Can be extended to more profiles (secure, entertainment, etc) Temporary remote employees are given a VM with expiration date VM1 App VM2 App Personal Corp VMM HW (personal) Personal Corp VMM HW (corporation) Connectivity becomes the critical path VMM HW (personal) 69

68 Bibliography Books : The definite guide to Xen Hypervisor. David Chisnall. Prentice Hall Other resources : Lecture slides of Virtual Machine course (5200) in NCTU KVM web page: VMware web page: