VT-d Posted Interrupts. Feng Wu, Jun Nakajima <Speaker> Intel Corporation

Size: px

Start display at page:

Download "VT-d Posted Interrupts. Feng Wu, Jun Nakajima <Speaker> Intel Corporation"

Godfrey Webster
6 years ago
Views:

1 VT-d Posted Interrupts Feng Wu, Jun Nakajima <Speaker> Intel Corporation

2 Agenda Motivation Difference btw CPU-based and VT-d Posted Interrupts Architecture Implementation Details Performance Summary 2

3 Motivation Interrupt virtualization efficiency Interrupt migration complexity Big requirement of host vector for different assigned devices 3

4 CPU-based Posted-Interrupt in Xen External interrupt handling External interrupt VCPU0 running on Physical CPU 0 VCPU1 running on physical CPU 1 - No need to kick VCPU1 - Notification Event is handled by VCPU1 in non-root mode - virr is updated by H/W and delivered by virtual interrupt delivery mechanism - VM-Exit is eliminated VM Exit Guest IPI: Notification Event Xen Set bit for guest Vector in Posted-interrupt requests (PIR) of VCPU1 4

Key Data Structures for CPU-based Posted- Interrupt Processing Posted-interrupt notification vector: Send virtual interrupts to guests w/o VM exit If physical vector == Posted-interrupt notification

5 Key Data Structures for CPU-based Posted- Interrupt Processing Posted-interrupt notification vector: Send virtual interrupts to guests w/o VM exit If physical vector == Posted-interrupt notification vector (VMCS field) PIR (Posted-interrupt requests) Set bits for guest vectors in advance ON (Outstanding Notification) If this bit is set, there is a notification outstanding for one or more posted interrupts VMCS PI Desc. Address Virtual-APIC Page PIR ON Posted-interrupt notification vector Posted-interrupt Descriptor 5 5

6 CPU-based Posted-Interrupt in Xen cont d Virtual interrupts from QEMU VCPU1 running on physical CPU 1 - No need to kick VCPU1 - Notification Event is handled by VCPU1 in non-root mode - virr is updated by H/W and delivered by virtual interrupt delivery mechanism - VM-Exit is eliminated Guest IPI: Notification Event QEMU Virtual Devices Virtual interrupts - Virtual MSI - Virtual IOAPIC Xen Set bit for guest Vector in Posted-interrupt requests (PIR) of VCPU1 6

7 What s new for VT-d Posted-Interrupts VCPU1 running on physical CPU 1 - No need to kick VCPU1 - Notification Event is handled by VCPU1 in non-root mode - virr is updated by H/W and delivered by virtual interrupt delivery mechanism - VM-Exit is eliminated Notification Event VT-d Engine with Posted-Interrupt support External Interrupts No VMM overhead AT ALL! Direct-assigned Device 7

8 VT-d Posted-Interrupts Architecture Virtual Processor A Virtual Processor B Virtual Processor C VMCS vapic Page Posted Descriptor VMCS vapic Page Posted Descriptor vapic Page Posted Descriptor VMCS Notification Event Notification Event CPU 0 Interrupt Notification Remapping Table CPU 1 Event IRTE for Interrupt Z IRTE for Interrupt Y CPU 0 Host-Vectors IRTE for Interrupt X CPU 1 Host-Vectors Other Vectors (e.g. Hyp. IPIs) Other Vectors (e.g. Timer) IRTE for Interrupt N Other Vectors (e.g. Hyp. IPIs) Other Vectors (e.g. Timer) Notification Vector for vcpu B Notification Vector for vcpu A External Interrupts Notification Vector for vcpu C 8

9 Xen Implementation Details: Update IRET according to guest s modification to the interrupt configuration (MSI address, data) Interrupt migration during VCPU scheduling 9

10 Xen Implementation: IRTE update VCPU Interrupt configuration update (e.g. MSI/MSIx) QEMU No changes needed for this part Hypercall XEN_DOMCTL_bind_pt_irq! Xen Update the guest information in IRTE - Guest vector - Posted-interrupt descriptor address Changes happen here 10

11 Xen Implementation: VCPU Scheduling Posted-Interrupt Descriptor RUNSTATE_running NV SN PIR [0-255] SN Set NV Notification vector SN Clear NDST New physical CPU of the VCPU NV Host vector for Notification Event SN If set, suppress Notification Event SN Set NV Wakeup vector SN Set NV Wakeup vector RUNSTATE_runnable RUNSTATE_blocked SN Set NV Notification vector 11

12 Summary VT-d Posted-interrupts advantages External interrupts from direct-assigned devices are delivered to guest running in non-root mode directly Improve Interrupt virtualization efficiency, e.g. Less VM-Exits. Simplify interrupt migration Consume less physical interrupts Performance The Specification will be published very soon Can be found in Intel website 12

13 Thank YOU! Q & A Or contact Feng Wu <feng.wu@intel.com> 13

14 14 Back up

VT-d Posted-Interrupts Support Interrupt-remap-table-entry (IRTE) enhanced as follows: - An existing reserved bit claimed to indicated Posted-interrupt (PST) - Software may choose to remap or post

15 VT-d Posted-Interrupts Support Interrupt-remap-table-entry (IRTE) enhanced as follows: - An existing reserved bit claimed to indicated Posted-interrupt (PST) - Software may choose to remap or post each interrupt independently IRTEs with PST set are interpreted per below format - New Fields Descriptor Address: the address of the posted-interrupt descriptor Virtual Vector: the guest vector of the interrupt URG: indicates if the interrupt is urgent - Other fields continue to have the same meaning 15

16 Posted-interrupt Descriptor 16

17 VT-d: Steps for Interrupt Posting Read IRTE Interrupt Remapping N PST == 1 Y Read Posted-interrupt descriptor (locking the cache-line) Set guest virtual in PIR Unlock cache-line N ( ON == 0) && ( URG == 1 SN == 0) Y Unlock cache-line Y Set ON and Generate notification event 17

Intel Virtualization Technology Roadmap and VT-d Support in Xen

Intel Virtualization Technology Roadmap and VT-d Support in Xen Jun Nakajima Intel Open Source Technology Center Legal Disclaimer INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS.