Dune: Safe User- level Access to Privileged CPU Features

Size: px

Start display at page:

Download "Dune: Safe User- level Access to Privileged CPU Features"

Marcus Lindsey
6 years ago
Views:

1 Dune: Safe User- level Access to Privileged CPU Features Adam Belay, Andrea Bi>au, Ali MashAzadeh, David Terei, David Mazières, and Christos Kozyrakis Stanford University

2 A quick review of VirtualizaAon HW Last lecture talked about AMD SVM This lecture: Intel VT- x (conceptually very similar) Key idea: Adds orthogonal Guest and Host CPU modes arch state saved and restored in VMCS HW performs transiaons between modes VM Exit - > trap to hypervisor (enter host mode) VM Enter - > run the guest OS (enter guest mode) 2

3 Normally IDT GDT PGTBL CPL CPU VMCS EPT With VT- X IDT GDT PGTBL CPL CPU (Host Mode) VM Entry VM Exit IDT GDT PGTBL CPL CPU (Guest Mode)

4 Some Key VT- x InstrucAons VMLAUNCH called first Ame to enter guest mode VMRESUME called for subsequent entries to guest mode VMPTRLD sets the VMCS pointer (ordinary memory) The VMCS is accessed with VMREAD and VMWRITE Why is it not okay to modify VMCS memory directly? VMCALL forces a VM exit 4

5 How has Virt. HW Changed? Adams and Agesen s study was > 6 years ago VM exit and VM entry now much faster More hardware support, less need to trap- and- emulate IOMMU - > raw passthrough devices Unrestricted guest mode - > faster boot Nested paging HW is widely available NET RESULT: Be>er performance, hypervisors commodiazed (easy to implement) 5

6 VirtualizaAon HW Support has become Ubiquitous Not just AMD and Intel x86 Available on ARM, Itanium, Power Desktops, servers, notebooks, cell phones 6

7 So what can we do with it? Is it only useful for running virtual machines? Idea behind Dune: Use virtualizaaon HW to give user programs safe access to privilege CPU features 7

8 Outline Review of VirtualizaAon HW Dune Mo.va.on Design EvaluaAon 8

9 The power of privilege Privileged CPU features are fundamental to kernels But other, compelling uses: Speed up garbage collecaon (Azul C4) Page tables provide memory access informaaon Privilege separaaon within a process (Palladium) MMU hardware isolates compartments Safe naave code in web browsers (Xax) System call handler intercepts system calls 9

10 Should we change the kernel? App PTEs Kernel Patch PGTBL Root CPU Problem: stability concerns, challenging to distribute, composability concerns 10

11 What about an Exokernel? App Garbage CollecAon Library OS Exokernel CPU Problem: must replace enare OS stack 11

12 What about a virtual machine? JVM Browser GC Kernel Linux Hypervisor CPU Problem: virtual machines have strict paraaoning 12

13 Dune in a Nutshell Kernel POSIX App Host Mode Guest Mode CPU Provide safe user- level access to privileged CPU features SAll a normal process in all ways (POSIX API, etc) Key idea: leverage exisang virtualizaaon hardware (VT- x) 13

14 Garbage collecaon in Dune PTEs Kernel App PTEs Host Mode Guest Mode Host Page Table CPU Guest Page Table SoluAon: control the page table directly within a process 14

15 Outline Review of VirtualizaAon HW Dune MoAvaAon Design EvaluaAon 15

16 Available CPU features Privilege Modes SYSRET, SYSEXIT, IRET Virtual Memory MOV CRn, INVLPG, INVPCID ExcepAons LIDT, LTR, IRET, STI, CLI SegmentaAon LGDT, LLDT 16

17 Dune architecture Kernel Dune Module Process libdune Host Mode Guest Mode CPU Host mode Normally used for hypervisors In Dune, we run the kernel here Reason: need access to VT- x instrucaons

18 Dune architecture Kernel Dune Module Process libdune Host Mode Guest Mode CPU Guest mode Normally used by the guest OS In Dune, we run ordinary processes here Reason: need access to privileged features

19 Dune architecture Kernel Dune Module Process libdune Host Mode Guest Mode CPU Dune Module (~2500 LOC) Configures and manages virtualizaaon hardware Provides integraaon with the rest of the kernel in order to support a process abstracaon Uses Intel VT- x (could easily add AMD SVM)

20 Dune architecture Kernel Dune Module Process libdune Host Mode Guest Mode CPU libdune (~6,000 LOC) A uality library to help applicaaons manage privileged hardware features Completely untrusted ExcepAon handling, system call handling, page allocator, page table management, ELF loader

21 Providing a process abstracaon Memory management System calls POSIX Signals 21

22 Memory management in Dune Configure the EPT to provide process memory User programs can then directly access the page table Kernel Host- Virtual User Page Table Dune Process Guest- Virtual Guest- Physical Kernel Page Table EPT Host- Physical (RAM) 22

23 System calls in Dune VMCALL SYSCALL Syscall Handler Kernel Process Syscall Handler Host Mode Guest Mode CPU SYSCALL will only trap back into the process Use VMCALL (i.e. a hypercall) to perform normal kernel system calls 23

24 But SYSCALL is sall useful Process (ring 0) Untrusted Code (ring 3) Syscall Handler Isolate untrusted code by running it in a less privileged mode (i.e. ring 3 on x86) Leverage the supervisor bit in the page table to protect memory 24

25 Signals in Dune Signals should only be delivered to ring 0 What happens if process is in ring 3? Possible soluaon: have the Dune module manually transiaon the process to ring 0 Works but slow and somewhat complex Our soluaon: deliver signals as injected interrupts Hardware automaacally switches to ring 0 Can use CLI and STI to efficiently mask signals 25

26 Many implementaaon challenges Reducing VM exit and VM entry overhead Pthread and fork were tricky to integrate with the Linux kernel EPT does not support enough address space Check the paper for details 26

27 Outline Review of VirtualizaAon HW Dune MoAvaAon Design Evalua.on 27

28 EvaluaAon How much overhead does Dune add? What potenaal does Dune create for opamizaaon? What is Dune s performance in end- to- end use cases? 28

29 Overhead analysis Two sources of overhead VMX transiaons EPT translaaons (cycles) Getpid Page fault Page walk Linux 138 2, Dune 895 5,093 86

30 OpAmizaAon analysis Large opportuniaes for opamizaaon Faster system call interposiaon and traps More efficient user- level virtual memory manipulaaon (cycles) ptrace (getpid) trap Appel 1 (TRAP, PROT1, UNPROT) Appel 2 (PROTN, TRAP, UNPROT) Linux 27,317 2, , ,909 Dune 1, ,496 94,854 30

31 End- to- end case studies We built and evaluated three systems ApplicaAon sandbox (~1300 LOC) Constrained the system calls performed by an untrusted binary Garbage collecaon (less than 100 LOC change) Improved dirty page detecaon through direct access to dirty bits Privilege separaaon (~750 LOC) Supported several protecaon domains within a single process through use of mulaple page roots (with TLB tagging) 31

Sandbox: SPEC2000 performance % Slowdown 25 20 15 10 5 0 5 10 15 20 25 vortex gap perlbmk eon parser ammp crafty equake mcf art mesa gcc vpr gzip

32 Sandbox: SPEC2000 performance % Slowdown vortex gap perlbmk eon parser ammp crafty equake mcf art mesa gcc vpr gzip Sandbox Sandbox w/ LGPG Linux w/ LGPG twolf bzip2 Only notable end- to- end effect is EPT overhead Can be eliminated through use of large pages 32

33 Sandbox: ligh>pd performance LighJpd performance (connec.ons per second) Linux Dune Sandbox VMware Player Slight reducaon in throughput (less than 2%) due to VMCALL overhead 33

34 Performance of other use cases Up to 40% improvements in garbage collecaon performance (less than 100 LOC) Privilege separaaon system can context switch between subdomains 3x faster than Linux can switch between processes (750 LOC) 34

35 Conclusions ApplicaAons can benefit from access to privileged CPU features VirtualizaAon hardware allows us to provide such access safely Dune creates new opportuniaes to build and improve applicaaons without kernel changes Dune has modest performance overhead Download Dune at h>p://dune.scs.stanford.edu 35

36 Future Work ARM support, AMD support, 32- bit support, x32 support Passthrough device support (w/ VT- d and SR- IOV) Cool applicaaons C4 Garbage collector? An awesome sandbox? 36

Virtualization. Adam Belay

Virtualization. Adam Belay Virtualization Adam Belay What is a virtual machine Simulation of a computer Running as an application on a host computer Accurate Isolated Fast Why use a virtual machine? To run multiple