Hypervisors & related technologies Arwed Tschoeke Client Center Böblingen 2016 IBM Corporation 2016 IBM Corporation
Agenda Virtualization basics PR/SM and DPM LinuxONE/VM KVM others
Server Virtualization Terms Logical Partition Also called an LPAR, virtual machine, or VM or guest (LinuxONE/VM,KVM) Memory Virtualization Dedicated to an PR/SM LPAR Shared by guests within LinuxONE/VM & KVM Runs an operating system such as Linux, TPF, z/vse, AIX, IBM i, Windows, Solaris Logical Partition or Virtual Machine Computer Memory Logical Partition or Virtual Machine Logical Partition or Virtual Machine Virtual Machines or Guests Hypervisor 2 nd Level Hypervisor Run an Hypervisor inside of an LPAR Provides unique features Example: zinuxone/vm Hypervisor Disk Hardware Hypervisor Virtualization software Divides real computing into logical computers or LPARs Referred to as PR/SM on LinuxONE Ethernet I/O Virtualization Provided by Hypervisor (VMware) I/O owning LPAR (PowerVM, Xen) Direct hardware virtualization (LinuxONE)
Server Virtualization Approaches Hardware Partitioning Bare Metal Hypervisor Hosted Hypervisor Apps OS... Apps OS Apps OS... Apps OS Apps OS... Apps OS Partition Controller Adjustable partitions Hypervisor(FW,OS) Type 1 Hypervisor Host OS Type 2 SMP Server SMP Server SMP Server Server is subdivided into fractions each of which can run an OS Hypervisor provides fine-grained timesharing of all resources Hypervisor uses OS services to do timesharing of all resources Physical partitioning S/370 SI-to-PP and PP-to-SI, Sun Domains, HP npars Logical partitioning HP (PA) vpars Hypervisor software/firmware Hypervisor software runs on runs directly on server a host operating system LinuxONE PR/SM, LinuxONE/VM, KVM VirtualBox POWERVM vmware HP Integrity VM Xen Hypervisor, Hyper-V User Mode Linux Hardware partitioning subdivides a server into fractions, each of which can run an OS Hypervisors use a thin layer of code to achieve fine-grained, dynamic resource sharing Type 1 hypervisors with high efficiency and availability will become dominant for servers Type 2 hypervisors will be mainly for clients where host OS integration is desirable 9 2016 IBM Corporation
Type 1 Hypervisor Goldberg, R. Architectural Principles for Virtual Computer Systems. PhD thesis, National Technical Information Service, February 1973. A Type 1 Virtual Machine Monitor runs as part of the Kernel Extended Machine is the environment running a time-shared program. The EM evolved into a modern OS Process. A Type II Virtual Machine Monitor runs as a standard OS Process
Software Interpretation Goldberg, R. Architectural Principles for Virtual Computer Systems. PhD thesis, National Technical Information Service, February 1973. Non-Virtualized KVM, VMware, Xen, PowerVM, LinuxONE/VM VMware Pre-VMX Qemu
System z CPU Dispatching and Execution Control Logical CPU Program Instruction stream SIE: Start Interpretive Execution instruction Instructions Problem state (aka guest) instructions High-frequency control instructions that require virtualization Low-frequency control instructions that require hypervisor virtualization e.g. Modify Subchannel instruction SIE interception to appropriate hypervisor level Load, Store, Add,... Start Subchannel, Test Subchannel,... LinuxONE/VM, KVM and LPAR hypervisors SIE SIE Hardware Instruction Execution Controls z/vm CPU STATE Description LPAR CPU STATE Description Instruction Interpretation Handling Hardware or Firmware Virtualization Assist Physical CPU Instruction Execution Unit Load Store Add SSCH TSCH
PR/SM and DPM
Linux VM Linux VM Linux VM LinuxONE Virtualization Options At the hypervisor level (2 IBM options: LinuxONE/VM and KVM) At the firmware level Processors, memory and I/O resources are divided into 1 or more Logical Partitions or LPARs What runs in each LPAR appears to be running on bare metal, and is electronically separate from what runs in any other LPAR, as if they were in physically separate machines LPARs are managed by one of two managers: Processor Resource / System Manager (PR/SM ) IBM Dynamic Partition Manager Interaction with the manager is via either: The Support Element (SE) a dedicated workstation used for monitoring and operating the system The Hardware Management Console (HMC) a secure Web application accessed through a browser Linux on bare metal LPAR1 Hypervisor LPAR2 Linux on bare metal LPAR3 PR/SM or IBM Dynamic Partition Manager HMC / SE
PR/SM or LPAR Hypervisor 'Processor Resource/System Manager' (PR/SM) and 'LPAR hypervisor' are commonly used synonymously. However the 'LPAR Hypervisor' is the program itself and 'PR/SM' is the facility of the whole So PR/SM aka LPAR hypervisor is a Type-1 Hypervisor that manages logical partitions: Each partition owns a defined amount of physical storage Strictly no storage shared across partitions No virtual storage management / paging done by LPAR hypervisor Zone relocation lets each partition start at address 0 CPUs may be dedicated to a partition or may be shared by multiple partitions I/O channels may be dedicated to a partition or may be shared by multiple partitions (Multiple image facility, MIF) PR/SM is shipped with LinuxONE (considered as part of the firmware) Separation of logical partitions is considered as good as having each partition on a separate physical machine (Evaluation Assurance Level 5)
How DPM helps in a new Linux Environment LinuxONE and PR/SM require a HW definition Dynamic IO one of the key differentiator of the platform would be nice Having the option to have a GUI-based administration Overcome the prejudice: z is old school and complicated Everything is scripted (that s why we need GUIs ;-) ) short version: No Texteditor required to get started, dynamic IO available
Linux Linux Linux Partition and I/O device management at the HMC Linux partitions or KVM partitions correspond to LPARs under standard PR/SM Supports only Linux and Linux based hypervisors Creation of I/O Configuration Data Set (IOCDS) is under the covers Supports dynamic updates of I/O Hardware and operating system message displays are unchanged Problem determination and maintenance continues to exist on the System Element (SE) On/Off Capacity on Demand (OOCoD) and Customer Initiated Upgrade (CIU) supported for Linux Linux KVM IBM Dynamic Partition Manager HMC / SE Linux
System Requirements IBM LinuxONE Rockhopper or IBM LinuxONE Emperor IBM Dynamic Partition Manager feature code 0016 Two dedicated OSA-Express5S 1000BASE-T Ethernet #0417 features Server can be in standard PR/SM mode or Dynamic Partition Manager mode set at initial IML Only supports Fibre Communication Protocol (FCP) IBM KVM and/or Linux without a hypervisor Does not support Any hypervisor other than IBM KVM ECKD disk IBM zaware GDPS Virtual Appliance 1 Requires firmware level 27 (GA2)
Additional technical information Separate HMCs are NOT required; HMC networks with both IBM Dynamic Partition Manager servers and standard PR/SM servers Only SERVICE or SYSPROG userids can enable/disable IBM Dynamic Partition Manager All alarms exist as before, plus there are 3 or 4 new types of alarms, primarily around utilization being added Improved support over today s ASCII Console support ability to open a console window HMC REST API support everything in the UI can be done via the REST API
LinuxONE/VM
IBM LinuxONE/VM Hypervisor LinuxONE/VM is the product name of a Type-1 Hypervisor LinuxONE/VM virtualizes the architecture: Guests definitions are completely virtual (and do not necessarily be consistent with physical HW) support DASD and FCP Offers the possibility to choose the solution with the largest convenience factor SSI Clustering for increased availability Integration into GDPS
LinuxONE/VM Version 6 Release 3 Making Room to Grow Your Business Product General Availability VM support for zedc Express and 10GbE RoCE Express features Available, CPU Pooling January 14 z13 and VM Enhancements Announcement Feb 13 Base z13 & Crypto support Available March 13 June 26 SMT and Multi-VSwitch Link Scalability Support Aggregation Available Available September 15 RACf enhancements, Prorated Core time January 14 dynamic PDR migration, SIMD support See http://www.vm.ibm.com/zvm630/ Q4/14 VM6.4 GA 2013/14 2015 2016 VM VM VM VM VM VM VM VM VM V 6.1 V 6.2 V 6.3
Preview IBM LinuxONE/VM 6.4 Preview announcement 216-009, dated February 16, 2016 http://www.vm.ibm.com/zvm640/index.html Planned availability date Fourth Quarter 2016 A release born from customer feedback Key components: Enhanced technology for improved scaling and total cost of ownership Increased system programmer and management capabilities New Architecture Level Set (ALS) of z196 and higher
Improved Scalability and TCO LinuxONE/VM Paging enhancements Use of HyperPAV when available to increase bandwidth for paging Increases number of paging I/Os that can be in-flight at once Exploitation for Paging, Spooling, LinuxONE/VM user directory, and minidisk pools that are mapped to LinuxONE/VM data spaces. Guest large page support Enhanced DAT facility for guest use 1 MB pages Decreases memory needed for DAT structures by guest with Enhanced DAT support LinuxONE/VM maps to 4KB pages at the host level. Guest Transactional Execution support Potential efficiency and scaling improvements for guests and guest software that exploits Alternative for serializing a set of operations.
Improved Scalability and TCO Memory scalability improvements Enhanced algorithms to further improve the efficiency of memory management Provide a foundation for future enhancements in scaling and efficiceny Guest Transactional Execution support Potential efficiency and scaling improvements for guests and guest software that exploits Alternative for serializing a set of operations. FlashSystems support for FCP-attached SCSI disks. Removes requirement of a San Volume Controller (SVC) to use FlashSystems for LinuxONE/VM system volumes and EDEVs
System Programmer & Management Capability QUERY SHUTDOWN command Allows better understanding of state of the system Allows for increased programmatical management of the system CP environment variables New framework to allow information to be set and queried for automatic processing Example: Indicate system is being started for Production or DR Test or Actual DR New management queries for SCSI environment. Allows SCSI detailed information to be gathered for emulated devices (EDEVs)
System Programmer & Management Capability Upgrade In Place migration enhancements Upgrade In Place migration was introduced in LinuxONE/VM 6.3 Enhanced to allow migration to LinuxONE/VM 6.4 from LinuxONE/VM 6.2 or LinuxONE/VM 6.3 (but not both at same time in cluster) Supports migration for clustered or non-clustered systems
KVM
KVM for LinuxONE In addition to LinuxONE/VM, IBM supports a Kernelbased Virtual Machine (KVM) offering for LinuxONE that hosts Linux guest virtual machines. The KVM can be installed on LinuxONE processors. The KVM offering co-exists with LinuxONE/VM virtualization environments & Linux. The KVM offering is optimized for the LinuxONE architecture and provides standard Linux and KVM interfaces for operational control of the environment. Enterprises will be enabled to easily integrate Linux servers into their existing infrastructure and cloud offerings. z/os z/vse Linux on z z/os Linux on z Linux on z Linux on z Linux on z z/tpf VM KVM LPARs (PR/SM ) I / O Processors Memory
KVM Overview KVM (Kernel Virtual Machine) is a Linux kernel-based hypervisor Developed and maintained by Avi Kivity / Qumranet, recently acquired by Red Hat KVM turns the Kernel into a hypervisor by loading a kernel module and opening a device node. The main parts of KVM are: Kernel module kvm.ko Hardware specific modules Device node /dev/kvm (to create/run VMs from userspace with a set of ioctl()s) Virtual machines (or guests or domains) appear as normal Linux processes and integrate seamlessly into the rest of Linux A VM has its own memory, that is separated from the user space process Virtual CPUs are not scheduled on it's own (vcpus are realized as Linux threads, and are still scheduled by the Linux Kernel process scheduler) In full virtualization mode it's possible to run multiple unmodified guest OSes in parallel, with each having private virtual hardware (network, disk, graphics etc.) Exploits 'SIE' hardware instruction on LinuxONE
Kernel mode Guest mode Host Process: kvm; kvm-qemu Host Process: virsh; virt-manager (via libvirt) User mode Guest Process Guest Process Qemu/KVM Component Diagram File System & Block devices Physical Drivers Linux Kernel (guest) QEMU Hardware Emulation vcpu0... vcpum iothread KVM (/dev/kvm and kvm.ko) Memory Linux Kernel (host) Hardware disk1 File System & Block devices Physical Drivers... disk2 disk3... pnic1 pnic2 pnic3 pcpu0 pcpu1 pcpu2 pcpun
Others
Virtualization vs. Containers A p A p p A p p p OS A p A p p A p p p OS APP APP Container Container APP Container OS Hypervisor HW Hypervisor HW Infrastructure oriented: coming from servers, now virtualized several applications per server isolation Separation between tenants Service oriented: application-centric solution decomposed DevOps separations between the apps of a tenant
Docker and Containers Docker in general offers different ways to access the Linux Kernel and software resources that allows to constitute and form Containers: libvirt systemd-nspawn lxc and libcontainer The recently introduced 'libcontainer' library Dockers own way to access these resources, like namespaces and cgroups seems to prevail and to become accepted.
Summary
Things to Remember HW-accelerated virtualization integrates into the existing world efficient, secure, scalable Choices: take what ever is suited best What is standard in your IT What provides the best match to the requirements Or both Remember: OR, not XOR
39 2016 IBM Corporation
40 2016 IBM Corporation
41 2016 IBM Corporation
Questions? arwed.tschoeke@de.ibm.com 42 2016 IBM Corporation