SFO17-410: NEVE: Nested Virtualization Extensions for ARM

Transcription

1 SFO17-410: NEVE: Nested Virtualization Extensions for ARM Jintack Lim, Christoffer Dall, Shih-Wei Li, Jason Nieh, and Marc Zyngier connect.linaro.org

2 Nested Virtualization Kernel Kernel Hypervisor Kernel Hypervisor Hardware

3 Nested Virtualization IaaS hosting private clouds Kernel Kernel Develop and test your hypervisor Hypervisor Kernel Hypervisor Hardware

4 Terminology Nested Nested Kernel Kernel Guest Hypervisor Kernel Host Hypervisor Hardware

5 ARM Virtualization Extensions EL0 EL1 OS Kernel EL2 Hypervisor

6 ARM Nested Virtualization Nested EL0 EL1 OS Kernel Guest Hypervisor EL2 Host Hypervisor

7 ARM Nested Virtualization Nested EL0 EL1 OS Kernel Guest Hypervisor Trap and Emulate EL2 Host Hypervisor

8 ARMv8.0 Nested EL0 EL1 OS Kernel Guest Hypervisor??? and Emulate EL2 Host Hypervisor

9 ARMv8.3 Nested EL0 EL1 OS Kernel Guest Hypervisor Trap and Emulate EL2 Host Hypervisor

10 Performance Evaluation Problem: No ARMv8.3 hardware available. Solution: Use ARMv8.0 hardware with the software modification

11 Emulating v8.3 on v8.0 Nested EL0 EL1 OS Kernel Guest Hypervisor SW Modification EL2 Host Hypervisor ARMv8.0 Hardware

12 Hypercall MicroBenchmark Nested EL0 EL0 EL1 OS Kernel EL1 EL2 Hypercall OS Kernel Hypervisor Return Hypercall Return Guest Hypervisor EL2 Host Hypervisor

13 Hypercall MicroBenchmark ARMv8.3 Nested Cycle counts 2, ,720 Ratio to 1 155x

14 Nested Hypercall Nested EL0 EL1 OS Kernel Hypercall Guest Hypervisor Return EL2 Host Hypervisor

15 Hypercall MicroBenchmark ARMv8.3 Nested Cycle counts 2, ,720 Ratio to 1 155x Trap counts 1 126

16 All Traps are Necessary? Nested EL0 EL1 OS Kernel Hypercall Guest Hypervisor Return EL2 Host Hypervisor

17 NEVE: NEsted Virtualization Extensions for ARM Improves performance of nested virtualization Key Mechanisms 1. Redirects register accesses to memory 2. Redirects register accesses to other registers

18 VNCR_EL2 Virtual Nested Control Register Enable bit BADDR (Base Address)

19 #1 Redirection to Memory Register VNCR_EL2.Enable == 0 Trap! msr x0, TTBR0_EL1 Memory

20 #1 Redirection to Memory Register msr x0, TTBR0_EL1 VNCR_EL2.Enable == 1 VNCR_EL2.BADDR Memory TTBR0_EL1

21 Accessing EL2 State Nested EL0 EL1 OS Kernel Hypercall Guest Hypervisor Return EL2 Host Hypervisor

22 #2 Redirection to Register TTBR0_EL2 VNCR_EL2.Enable == 0 Trap! msr x0, TTBR0_EL2 TTBR0_EL1

23 #2 Redirection to Register VNCR_EL2.Enable == 1 TTBR0_EL2 msr x0, TTBR0_EL2 TTBR0_EL1

24 Hypercall MicroBenchmark ARMv8.3 NEVE Nested Nested Cycle counts 2, ,720 92,385 Ratio to 155x 34x Trap counts

25 lication Workloads lication Description lication Description Kernbench Kernel compile Netperf TCP_RR Network performance Hackbench Scheduler stress Netperf TCP STREAM Network performance SPECjvm2008 Java Runtime Netperf TCP MAERTS Network performance MySQL Database management Apache Web server stress Memcached Key-Value store Nginx Web server stress

26 Experimental Setup Hardware APM X-Gene (ARMv8.0) 8-way SMP 64 GB RAM 10 Gb Ethernet Native//Nested Setup 4-way SMP 12 GB RAM Virt I/O (/nested ) Software K on K v4.10

27 Normalized overhead (lower is better) lication Benchmarks ARMv8.3 ARMv8.3 Nested NEVE Nested Kernbench Hackbench SPECjvm2008 TCP RR TCP STREAM TCP MAERTS Apache Nginx Memcached MySQL

28 Normalized overhead (lower is better) lication Benchmarks ARMv8.3 ARMv8.3 Nested NEVE Nested x86 Nested Kernbench Hackbench SPECjvm2008 TCP RR TCP STREAM TCP MAERTS Apache Nginx Memcached MySQL

29 Conclusion Nested virtualization performance on ARMv8.3 incurs high overhead Due to the exit multiplication problem NEVE enhances performance significantly by reducing number of traps NEVE is included in ARMv8.4