Rethinking Machine Types

Size: px

Start display at page:

Download "Rethinking Machine Types"

Kerrie White
5 years ago
Views:

1 Rethinking Machine Types KVM Forum 2015 David Gibson Senior Software Engineer, Virtualization 21 August 2015

2 What's the problem?

3 The qdev model Command line options become virtual devices. Simple... 3 Rethinking Machine Types - KVM Forum 2015

4 ...and then there's machine type Which adds a bunch of other stuff 4 Rethinking Machine Types - KVM Forum 2015

5 pc / q35 i386 pseries ppc64 virt aarch64 Legacy IO PAPR hypercalls GPEX PCI-E host seabios SLOF Flash i440fx / Q35 PAPR PCI Host ARM GIC ACPI PAPR VIO xenpv i386 ppce500 ppc64 mac99 ppc Xen hypercalls Device Tree Apple IO Hub... epapr boot Open Hackware... Apple PCI Host... mac99 ppc cubieboard arm malta mips

6 The trouble with machine type Machine type performs necessary system wide setup But it also.. Adds system devices Even with -nodefaults Behaviour can depend on machine options Or other options (-vga, -usb, -nographics) PROBLEM #1 Machine type behaviour isn't easily discoverable 6 Rethinking Machine Types - KVM Forum 2015

7 VM Hardware Description How does the virtualization stack describe guest hardware? ovirt database vdsm XML-RPC libvirt domain XML QEMU command line Runtime QOM structures Guest Device Tree ACPI Tables PROBLEM #2 This many ways to describe virtual hardware? Really? 7 Rethinking Machine Types - KVM Forum 2015

8 VM Hardware Description Loose versus precise 20G storage 20G SATA disk, on AHCI 20G SATA rev 2.0 disk, on AHCI, rev 1.3, in slot 2, function 1, of PCI host bridge at IO 0xabcd0000 QEMU qdev command ACPI line device tree ovirt OpenStack virt-install libvirt domain XML 8 Rethinking Machine Types - KVM Forum 2015

9 VM Hardware Description Loose versus precise (2) Humans and high-level tools want loose specification..except when they don't QEMU and guest need precise specification Converting loose precise Select default implementations Add standard devices Assign addresses 9 Rethinking Machine Types - KVM Forum 2015

10 Migration Migration destination must have identical hardware to source At least as far as the guest can tell.. Implementation of the devices can change Hosts with different paths to back-end storage Host specific optimization hints libvirt manages migration So it needs precise hardware information PROBLEM #3 libvirt and qemu both have address assignment code 10 Rethinking Machine Types - KVM Forum 2015

11 Hotplug QEMU must keep track of current hardware configuration Must co-ordinate hotplug with guest Including hotplugged (or unplugged) devices Platform specific protocols Combined with migration Destination needs devices hotplugged on source libvirt needs to track hotplugged devices PROBLEM #4 libvirt and qemu track hotplugged devices in parallel 11 Rethinking Machine Types - KVM Forum 2015

12 Recap Problems with VM Hardware Description PROBLEM #1 Machine type behaviour isn't easily discoverable PROBLEM #2 This many ways to describe virtual hardware? Really? PROBLEM #3 libvirt and qemu both have address assignment code PROBLEM #4 libvirt and qemu track hotplugged devices in parallel 12 Rethinking Machine Types - KVM Forum 2015

13 How do we fix it?

14 QEMU What needs to change? Command line processing Hotplug (QMP or HMP) Default devices ( vl.c ) Want a clear split between: Code building qdev tree Code using qdev tree Pretty close already Virtual Machine Construction QEMU qdev tree Machine Type 14 Rethinking Machine Types - KVM Forum 2015 Emulator Runtime..except for machine type

15 QEMU Split machine type MACHINE SCHEMA Construct: Platform essential devices Platform default devices (depending on options) Set up root bus With class and parameters 15 Rethinking Machine Types - KVM Forum 2015 ROOT BUS Subclass of SysBus Checks device dependencies But doesn't try to fix Handles system wide reset Firmware load / setup CPU / memory initial state

16 QEMU Expose hardware description Serialized hardware description format Hardware Description Virtual Machine Construction qdev tree QEMU 16 Rethinking Machine Types - KVM Forum 2015 Emulator Runtime Guest-visible & back-end pieces Allow hardware state to be extracted Simply (no need to walk qtree) Including hotplugged devices Allow specification to be re-inserted Bypass machine construction Bypass machine schema

17 libvirt HV drivers manage precise hardware description Domain Creation Start / Stop HV backends store precise description Migration Creating new VM: Hotplug Domain XML Setup Management qemu driver libvirt Runtime Management Translate XML into qemu options Final VM description extracted (Re-)starting a VM: Virtual Machine Construction qdev tree QEMU 17 Rethinking Machine Types - KVM Forum 2015 Emulator Runtime Can re-generate precise description Use stored precise description But requires guest restart Domain XML becomes loose only

18 The rest of the stack Up the stack Management tools Down the stack Guest Operating Systems Can keep using libvirt like now No change necessary Optionally use new scheme Continue to use ACPI or DT detailed view of HW precise control of HW QEMU already creates this

19 Hardware Description Format What would a consolidated format need? Tree structure Extensible Handle future hardware Separate guest visible and back end information Express bus / bridge layout Work with one without parsing the other Preferably, already exists Less to implement Avoid N+1 standards 19 Rethinking Machine Types - KVM Forum 2015

20 Hardware Description Format What would a consolidated format need? Tree structure Extensible Handle future hardware Separate guest visible and back end information Express bus / bridge layout Work with one without parsing the other Preferably, already exists Less to implement Avoid N+1 standards 20 Rethinking Machine Types - KVM Forum 2015

21 Hardware Description Format libvirt domain XML? XML heirarchy doesn't match bus heirarchy Guest and back-end info mixed Doesn't represent system devices Not clear from XML alone if it is a loose or precise description Needs XML parsing libvirt XML is not well suited to precise hardware description 21 Rethinking Machine Types - KVM Forum 2015

22 Hardware Description Format Better ideas Flattened Device Tree? Used by Linux guests Easy to implement ppc, some ARM & MIPS Easy to parse, existing tools Linearize QOM? Lacks back end information qdev FDT may be complex Some awkward redundancies 22 Rethinking Machine Types - KVM Forum 2015 QEMU already has JSON code Guest and back-end are separate Ties format to QEMU internals Might make future changes harder

23 Getting started How to get from here there 1. Consensus amongst developers (QEMU and libvirt) Is this a good approach? Something like it? 2. Implement the machine type split Has impacts across the tree Enough people with enough time 3. Decide on a hardware description format 4. Implement import / export 5. Work outwards from there 23 Rethinking Machine Types - KVM Forum 2015

24 Questions forum 2015.pdf 24 Rethinking Machine Types - KVM Forum 2015

25 THANK YOU plus.google.com/+redhat facebook.com/redhatinc linkedin.com/company/red-hat twitter.com/redhatnews youtube.com/user/redhatvideos

26 Architecture versus Machine Type Bootup Conventions CPU CPU ARCHITECTURE also known as Firmware Interfaces System Peripherals MACHINE TYPE also known as CPU Architecture System architecture Instruction Set Architecture Sub-architecture Platform 26 Rethinking Machine Types - KVM Forum 2015

27 Flattened Device Trees Background Originated with Open Firmware (IEEE1275) Conveys hardware information firmware OS Bus heirarchy tree plus some special nodes Device properties key value (bytestring) pairs Binding documents Adapted to flattened form for use without full OF Used by Linux for hardware discovery All PowerPC and Microblaze Some ARM, MIPS, and others 27 Rethinking Machine Types - KVM Forum 2015

libvirt integration and testing for enterprise KVM/ARM Drew Jones, Eric Auger Linaro Connect Budapest 2017 (BUD17)

libvirt integration and testing for enterprise KVM/ARM Drew Jones, Eric Auger Linaro Connect Budapest 2017 (BUD17) Overview Enterprise guest requirements QEMU/KVM enterprise guest management libvirt A