Fakultät Informatik Institut für Systemarchitektur, Betriebssysteme THE NOVA KERNEL API. Julian Stecklina

Size: px

Start display at page:

Download "Fakultät Informatik Institut für Systemarchitektur, Betriebssysteme THE NOVA KERNEL API. Julian Stecklina"

Erica Hodge
6 years ago
Views:

1 Fakultät Informatik Institut für Systemarchitektur, Betriebssysteme THE NOVA KERNEL API Julian Stecklina Dresden,

2 00 Disclaimer This is not about OpenStack Compute. NOVA is mainly the work of Udo Steinberg (kernel) and Bernhard Kauer (userland). TU Dresden, The NOVA Kernel API Folie 2 von 26

3 00 Goals not talking about virtualization propaganda, giving a very short overview of NOVA as a whole introducing basic concepts of the kernel API In the end you should be able to pick up the NOVA API manual and make heads or tails of it. TU Dresden, The NOVA Kernel API Folie 3 von 26

4 01 NOVA OS Virtualization Architecture //os.inf.tu-dresden.de/papers_ps/steinberg_eurosys2010.pdf http: TU Dresden, The NOVA Kernel API Folie 4 von 26

5 01 What works, what doesn t Works x86 32-bit SMP VT-x, AMD-V VT-d (Intel IOMMU) SR-IOV grub, syslinux,... Linux, L4,... emulates AHCI, igb,... drivers for AHCI, some Intel NICs,... experimental libvirt support Doesn t work yet Windows Migration Recursive Virtualization 64-bit being user-friendly ;-)... TU Dresden, The NOVA Kernel API Folie 5 von 26

7 02 NOVA Architecture Reduce complexity of hypervisor: hypervisor provides low-level protection domains address spaces virtual machines one VMM per guest in (root mode) userspace, possibly specialized VMMs to reduce attack surface only one generic VMM implement so far TU Dresden, The NOVA Kernel API Folie 7 von 26

8 Demo

9 03 The L4 Influence NOVA cannot deny its roots in the L4 family: task, threads, synchronous IPC recursive mapping of memory TU Dresden, The NOVA Kernel API Folie 9 von 26

10 03 Capability-Based Syscalls operate on capabilities to kernel objects: Protection Domain (PD) ( task ) create pd Execution Context (EC) ( thread ) create ec, ec ctrl Scheduling Context (SC) create sc, sc ctrl Portals (PT) create pt, call, reply Semaphore (SM) create sm, sm ctrl TU Dresden, The NOVA Kernel API Folie 10 von 26

11 03 Capabilities Userspace can create capabilities to objects (by creating kernel objects), delegate capabilities (recursively, just as memory), Capabilities are stored per-pd in capability space in the kernel. A PD uses index into capability space to name capabilities, unforgeable. (Think file descriptors.) TU Dresden, The NOVA Kernel API Folie 11 von 26

12 03 Communication EC (thread) bound to one PD (address space) either thread or vcpu has a special memory region (UTCB) for IPC Portals entry point (instruction pointer) bound to one EC per client/function/... pass data, delegate capabilities from UTCB to UTCB can be called or implicitly used by exceptions (if a thread has the cap) TU Dresden, The NOVA Kernel API Folie 12 von 26

13 03 ECs and SCs There are two kinds of threads: with time global thread or vcpu stick SC to newly created EC causes startup exception when first scheduled without time local thread bind portals to ECs when portal invoked, starts executing at portal EIP caller hands in time to handle the request (no scheduling decision) TU Dresden, The NOVA Kernel API Folie 13 von 26

14 03 Basic Server Scenario EC local (1) call EC global TU Dresden, The NOVA Kernel API Folie 14 von 26

15 03 Basic Server Scenario EC local (1) call EC global TU Dresden, The NOVA Kernel API Folie 14 von 26

16 03 Basic Server Scenario EC local (2) reply EC global TU Dresden, The NOVA Kernel API Folie 14 von 26

17 03 Resource Contention ECs are not reentrant. What happens when a second client wants to call a service? TU Dresden, The NOVA Kernel API Folie 15 von 26

18 03 Resource Contention EC local (1) call EC global TU Dresden, The NOVA Kernel API Folie 15 von 26

19 03 Resource Contention EC local (1) call (2) call EC global TU Dresden, The NOVA Kernel API Folie 15 von 26

20 03 Resource Contention EC local (1) call (2) call EC global TU Dresden, The NOVA Kernel API Folie 15 von 26

21 03 Resource Contention EC local (2) call (3) reply EC global TU Dresden, The NOVA Kernel API Folie 15 von 26

22 03 Resource Contention EC local (4) reply EC global TU Dresden, The NOVA Kernel API Folie 15 von 26

23 03 NOVA s time management With SCs only bound to some threads, it is possible to build (simple) servers without time reservation. How much time should service foo need anyway? fewer things to schedule, contended resources get boosted by clients as needed. TU Dresden, The NOVA Kernel API Folie 16 von 26

24 04 Hardware Support for Virtualization Late Pentium 4 (2004) introduced hardware support for virtualization: Intel VT. (AMD-V is conceptually very similar) root mode vs. non-root mode root mode runs hypervisor non-root mode runs guest situations that Intel VT cannot handle trap to root mode (VM Exit) special memory region (VMCS) holds guest state reduced software complexity Supported by all major virtualization solutions today. TU Dresden, The NOVA Kernel API Folie 17 von 26

25 04 VT-x Problems VMCS (memory region holding guest state) needs to manipulated by VMM, yet cannot be mapped into userspace, have to use privileged VMREAD/VMWRITE instructions to access, reading all content for every VM Exit is expensive. Kernel has to manage VMCS access. TU Dresden, The NOVA Kernel API Folie 18 von 26

26 04 Virtualization on NOVA VM Exits (and normal exceptions) vector through special portals. Portals created with bit field denoting interesting information (Message Transfer Descriptor, MTD) for WRMSR or CPUID we need only general purpose registers for page fault we need complete vcpu state kernel puts this data in handler s UTCB handler produces new MTD on reply Reduce number of expensive VMREAD/VMWRITE in the kernel. TU Dresden, The NOVA Kernel API Folie 19 von 26

27 04 Writing to disk

28 04 Writing to disk Handler Drv IRQ Shared Memory Exc vcpu VMM

29 04 Writing to disk Handler Drv IRQ Shared Memory Exc vcpu MMIO VMM

30 04 Writing to disk Handler Drv IRQ "write data" Exc Shared Memory vcpu MMIO VMM

31 04 Writing to disk Handler Drv IRQ "working on it" Exc Shared Memory vcpu MMIO VMM

32 04 Writing to disk Handler Drv IRQ Shared Memory Exc vcpu VMM

33 04 Writing to disk Handler Drv IRQ Shared Memory Exc vcpu VMM

34 04 Writing to disk Handler Drv IRQ recall Shared Memory Exc vcpu VMM

35 04 Writing to disk Handler Drv IRQ Shared Memory Exc vcpu recall exception VMM

36 04 Writing to disk Handler Drv IRQ Shared Memory Exc inject irq vcpu VMM

37 04 Writing to disk Handler Drv IRQ Shared Memory Exc vcpu VMM

39 05 There is also... Userspace Timer Service Admission Server Device Drivers (IOMMU!)... TU Dresden, The NOVA Kernel API Folie 22 von 26

40 05 Summary The NOVA microhypervisor is a fast capability-based microkernel with virtualization in mind. Supported by: Code at Discuss at TU Dresden, The NOVA Kernel API Folie 23 von 26

41 06 Multiple CPUs Thread-related kernel objects are bound to one CPU: Portals, Execution Contexts, Scheduling Contexts. Semphores work cross-cpu. Communication via Semaphores/recall. Non-donating cross-cpu IPC never really needed. Servers can be CPU-topology aware! TU Dresden, The NOVA Kernel API Folie 24 von 26

43 06 Livelock It s possible to construct helping loops... Ouch! Pager Srv A Client Srv B TU Dresden, The NOVA Kernel API Folie 26 von 26

44 06 Livelock It s possible to construct helping loops... Ouch! Kernel detects loop Random IPC is aborted Pager Client Srv A Srv B TU Dresden, The NOVA Kernel API Folie 26 von 26

VIRTUALIZATION. Dresden, 2011/6/23. Julian Stecklina

VIRTUALIZATION. Dresden, 2011/6/23. Julian Stecklina Department of Computer Science Institute of Systems Architecture, Operating Systems Group VIRTUALIZATION Julian Stecklina (jsteckli@os.inf.tu-dresden.de) Dresden, 2011/6/23 00 Goals Give you an overview