Junhong Jiang, Kevin Tian, Chris Wright, Don Dugger

Size: px

Start display at page:

Download "Junhong Jiang, Kevin Tian, Chris Wright, Don Dugger"

Victoria Bates
6 years ago
Views:

1 Updating Xen for the Client Environment Junhong Jiang, Kevin Tian, Chris Wright, Don Dugger

2 Legal Content INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY RELATING TO SALE AND/OR USE OF INTEL PRODUCTS, INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT, OR OTHER INTELLECTUAL PROPERTY RIGHT. Intel may make changes to specifications, product descriptions, and plans at any time, without notice. Intel is a trademark of Intel Corporation in the U.S. and other countries. Copyright 2007, Intel Corporation. All rights are protected. 2

3 Agenda Xen Client Overview Boot Scheme/Verified Launch Device Virtualization Power Management 3

4 Glossary VA Virtual Appliance Xen Client is a platform for implementing VAs UOS User Operating System/User OS/Primary User Partition SOS Service OS/VA Partition VM Virtual Machine/Guest Partition VMM Virtual Machine Monitor/Hypervisor 4

5 Xen Client Xen based virtual appliance infrastructure Limited device virtualization/isolation support Single SOS Linux kernel (Para-virtualized) No direct disk access at run time Permanent storage provided by UOS Exception during boot process Dom0 provides disk access Target platform TXT support Measure and authenticated boot Trust chain from BIOS VT-d Support Virtualizes NIC TPM 5

6 Architecture Overview VMM Xen hypervisor 64 bits Dom0 Para-virtualized Linux. Runs out of RamFS Minimal components Control Panel, device models, default power management policy, virtual appliance specific agents Headless Capable of booting messages UP Service OS Paravirtualized XenLinux Fully virtualized possibility for future Headless Boot up display for SOS in Dom0 Survive even UOS hangs Support single or multiple SOS s (At least one) 6

7 Architecture Overview (Cont.) SOS0 (LAN filtering) LAN Filtering Owns physical NIC Provide VNIF BE driver SMP capable SOS1 (VoIP) Own Audio SMP Capable User Operating System Near native performance Own rest physical resources (CPU, memory, devices) Virtual LAN VNIF front-end (FE) driver SMP Windows XP SP2 (32-bit PAE w/ XD support) Vista (32 and 64-bit) Windows PE Linux 7

8 Xen Client Architecture Overview SOS UOS Dom0 ISV Apps. User Apps. vpro Client Cfg./Launch Verified boot Provision/Install Agent Presence vtpm Network Drivers Other VA Drivers VA Drivers VA Provisioning Agents VA N/W Routing Srvc. VA Storage Services VA Services Agents VA Drivers VA Services Agents VA Provisioning Agents VA Storage Services Other VA Drivers Native Drivers Xen hypervisor Shadow PT PIC/PIT Models Vt-x LT VT-d EM64T Channel ACPI/PM TPM HW LAN/WAN SATA, USB, etc. 8

9 Disk Layout Reserved SOS partition Type 0x71 Pointed to by MBR Contains boot/persistent storage file system Boot FS in SOS partition Contains grub loaded modules Xen kernel image Dom0 vmlinuz Dom0 initrd SOS modules are not loaded by grub Dom0 mounts the boot FS SOS kernel/initrd loaded by domain builder in Dom0 9

10 Disk layout SOS Partition Part 4 (type = 71) Part 3 Part 2 Part 1 SOS partition SOS Boot Block UOS Partition Boot Block UOS usable Disk Physical Disk Master Boot Record Partition Table Boot code Part 4 (type = 71) Part 3 Part 2 Part 1 UOS MBR 0 10

11 Memory Layout Xen hypervisor loaded at fixed memory Affects minimum memory required Dom0 allocated at top of memory First partition loaded SOS s loaded sequentially below Dom0 UOS loaded at low memory Allows 1-1 guest physical to machine physical mapping Needed to execute platform BIOS, especially SMI s Size of UOS dynamically determined Max memory minus fixed size for Xen, Dom0 & SOS s Implications: UOS given contiguous machine physical address Hole in memory for Xen Xen hypervisor memory allocation change 11

12 Physical RAM layout initial boot Max Dom0 initrd Grub Modules Dom0 image 256M Xen VMM image 1M 640K BIOS 0 12

13 Physical RAM layout - runtime Max Dom0 RAM SOS RAM UOS gpfn == mpfn 256M 1M 640K Xen VMM image UOS gpfn == mpfn BIOS 0 13

14 Boot sequence 4 phases 1. Grub loads Xen & Dom0 into memory 2. Dom0 starts 3. SOS starts 4. UOS starts 14

15 Persistent Storage SOS runs out of RAM Needs access to persistent storage on disk SOS run a disk Front End driver Talks to disk Back End driver in Dom0 during boot Unmounts disk from Dom0 Shuts down disk FE from Dom0 Starts disk FE, now talking to UOS Remounts disk 15

16 Verified Launch Dom0 Dom0 (Linux) VM Power Manager User CMFM Domain Builder (xm/libxc equivalent) vtpm Device Model (PCI Config, Serial, ICH,..) Kernel InterVM communication driver TPM Driver Back-end disk driver Standard Linux Drivers (VGA, Chipset NV, Disk) Xen hypervisor SATA, USB, etc. Shared HW User Optimized HW 16

17 Verified Launch SOS Dom0 (Linux) SOS VM Power Manager CMFM Domain Builder (xm/libxc equivalent) vtpm Device Model (PCI Config, Serial, ICH,..) CMFM agent (default) Libxc (user level event channel) BE TPM driver InterVM communication driver BE disk driver VA services InterVM Communication driver Back-end NIC driver TPM Driver Standard Linux Drivers (VGA, Chipset NV, Disk) Front-end Disk Driver vtpm FE Driver Linux NIC Driver Xen hypervisor SATA, USB, etc. Shared HW User Optimized HW 17

18 Verified Launch UOS Dom0 (Linux) SOS (Linux) UOS (Windows) VM Power Manager CMFM vtpm CMFM agent (default) PS agent: Back-end Disk Driver Domain Builder (xm/libxc equivalent) Device Model (PCI Config, Serial, ICH,..) Libxc (user level event channel) Libxc (user level event channel) BE TPM driver TPM Driver Device Model InterVM Communication driver VA services InterVM Communication driver Front-end Disk Driver vtpm FE Driver Back-end NIC driver Linux NIC Driver InterVM Communication driver TPM Driver FE NIC driver VA services Standard Windows Driver Xen hypervisor SATA, USB, etc. Shared HW User Optimized HW 18

19 I/O Device Handling Default is devices handled directly by UOS Near native performance DMA support VT-d provides protection between guests Non-VT-d platforms will depend upon `well behaved guests Special devices paravirtualized through SOS LAN Disk Xen Client Currently UOS owned Xen Client Future PCI IOV w/ secure LBA allows dual ownership Audio Xen Client Future PCI IOV provides secure sharing 19

20 PCI Config Space Virtualization SOS UOS Dom0 Device Models ISV Apps. User Apps. vtpm TPM Driver ICH (Power Control) PCI Config space Network Drivers Other VA Drivers VA Drivers VA Provisioning Agents VA N/W Routing Srvc. VA Storage Services VA Services Agents VA Drivers VA Services Agents VA Provisioning Agents VA Storage Services Other VA Drivers Native Drivers Xen hypervisor Shadow PT PIC/PIT Models Vt-x LT VT-d EM64T Channel ACPI/PM Agent Integrity Boot and Startup Provisioning TPM HW LAN/WAN SATA, USB, etc. 20

21 I/O Device virtualization - PCI Handling Dom0 owns PCI bus enumeration SOS/UOS PCI bus enumeration is virtualized in its host domain. SOS vpci bus enumeration in dom0 UOS vpci bus enumeration in Dom0 Policy of device assignment to UOS, other SOSes setup by management partition Virtualized devices are inserted into PCI hierarchy PCI device physical BAR = vbar in UOS Help for PCI hot plug support and thus docking 21

22 Device assignment PCI hierarchy PV (S0S) Virtual PCI front end/back end frame Qemu based partition (HVM SOS) Using current Piix4 in qemu as virtual PCI hierarchy Attach assigned physical device to virtual PCI bus in qemu, like current VTd effort Qemu maintains vconfig_space to pconfig_space mapping See PCI express, depends on virtual chipset in qemu Native based partition (UOS) See exactly same PCI hierarchy as physical FE becomes the placeholder of detached devices Need a stub to maintain vconfig_space to pconfig_space mapping HVM PCI config space interception Trapped by Xen, and emulated by vpci Device Model Handlers has filter to decide which field can be written through Handlers further issues io/mmio/interrupt assignment based on captured info 22

23 Xen Client: Device Assignment Hide all PCI devices except for the Dom0 assigned devices from Dom0 device drivers Assign PCI devices to guest Attach the assigned device to QEMU vpci bus Assign corresponding VT-d context entry to guest domain Intercept PCI config access in Xen and Qemu appropriately Turn on VMCS IOPORT bit to allow IOPORT access to assigned devices Install P2M entry for MMIO access of the assigned device Xen intercepts physical device interrupts and re-injects to the target guest domain with virq 23

Virtualization Event notification (via evtchn driver) Use at boot time only TPM Driver PV SOS (No PM) Device Driver UOS PM Platform

24 Xen Client PM Architecture Overview VM exit/entry Xen event Virtual PM ops Real PM ops DOM0 Primary UOS VMPM (VM Power Manager) VA Agent ACPI Virtualization SOS Device PM BE FE FE PM Event Coordination ACPI Driver/AML Interpreter DOM0 PM OSPM Driver PCI PM Virtualization Event notification (via evtchn driver) Use at boot time only TPM Driver PV SOS (No PM) Device Driver UOS PM Platform ACPI Component Semi-Virtual Platform Platform ACPI Component TPM HW Registers BIOS Tables Platform Hardware Segregated HW User Optimized HW Shared HW 24

25 Power Management UOS OSPM policy virtualized Policy defined by UOS and enforced by Dom0 UOS controls native devices SOS controls everything else Shared devices CPU Platform UOS initiates PM policy (Suspend, Hibernate, C-state or P-state changes) 25

26 PM - Suspend UOS owns physical PCI bridges It will suspend bridges before SOS can suspend it s devices Implication is that UOS must see virtualized bridges One flat PCI space Could be issue with drivers assuming bus limitations Virtual copy of actual topology Preferable solution Suspend UOS suspends devices Including virtualized devices (as requested by device D3 request) UOS suspends PCI bridges Including any virtualized bridges and conforming to the policy setup in Dom0 (VMPM VM power management) UOS completes suspend and request ACPI suspend request (ICH I/O port write) ICH PM suspend requests suspend of SOSes if needed Should avoid keeping any state in SOS/DOM0 for restart of SOS/DOM0 Wake from Suspend Restarts from Xen in real mode and execute 26

27 Conclusion This is real, download the Alpha release at Questions? Comments? Epithets? 27

I/O virtualization. Jiang, Yunhong Yang, Xiaowei Software and Service Group 2009 虚拟化技术全国高校师资研讨班

I/O virtualization. Jiang, Yunhong Yang, Xiaowei Software and Service Group 2009 虚拟化技术全国高校师资研讨班 I/O virtualization Jiang, Yunhong Yang, Xiaowei 1 Legal Disclaimer INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE,