Microkernel Construction

Size: px

Start display at page:

Download "Microkernel Construction"

Lorraine Lawson
5 years ago
Views:

1 Microkernel Construction Interprocess Communication Nils Asmussen 05/03/ / 33

2 Outline Introduction Microkernel vs. Monolithic kernel Synchronous vs. Asynchronous Different Implementations Synchronous IPC in NOVA Asynchronous IPC in NOVA Userspace API 2 / 33

3 Microkernel vs. Monolithic: Syscalls Monolithic kernel: 2 kernel entries/exits Microkernel: 4 kernel entries/exits + 2 context switches Application Application Driver 1 4 Driver 2 3 Monolithic kernel Microkernel Hardware Hardware 3 / 33

4 Microkernel vs. Monolithic: Calls Between Services Monolithic kernel: 2 function calls/returns Microkernel: 4 kernel entries/exits + 2 context switches Network Driver Network Driver Monolithic kernel Microkernel Hardware Hardware 4 / 33

5 Synchronous vs. Asynchronous Synchronous Sender is blocked until receiver is ready Data and control transfer from sender to receiver Asynchronous Data is transferred to temporary location Sender continues execution If receiver arrives, the data is transferred to him Comparison Synchronous is typically simpler and faster (no buffering) Synchronous is less prone to DoS attacks (buffer memory) Asynchronous is typically more flexible Asynchronous allows to do other work while waiting 5 / 33

6 Register IPC Sender Receiver User Stack A sp ip r1 r2 r3 r4... User Stack B Regs EC A (1) save (3) restore Kernel (2) copy EC B CPU 6 / 33

7 User Memory IPC Sender (2) send Data (3) copy Receiver Buffer (1) register EC Kernel bufobj CPU 7 / 33

8 Kernel Memory IPC Sender Receiver Data (1) send Buf EC Kernel (2) copy EC Buf CPU 8 / 33

9 Comparison Register IPC + Very fast Amount of data limited to CPU registers User Memory IPC + Amount of data not limited + No copy to special location first Pagefaults can occur Slower (no direct copy) Kernel Memory IPC + Fast + No pagefaults Amount of data limited Copy to special location first 9 / 33

10 Outline Introduction Synchronous IPC in NOVA Synchronous IPC in General Exception IPC Asynchronous IPC in NOVA Userspace API 10 / 33

11 Introduction to IPC in NOVA NOVA uses synchronous kernel memory IPC to Exchange data Exchange capabilities Asynchronous IPC by semaphores for Signaling Deliver interrupts to user space Synchronous IPC is core-local Asynchronous IPC can be used cross-core 11 / 33

12 Synchronous IPC Uses kernel memory IPC Message buffer is called User Thread Control Block (UTCB) Each EC has exactly one UTCB A UTCB is one page, i.e., 4 KiB large All UTCBs are mapped in kernel space On EC creation, a UTCB is allocated and mapped to a specified address in user space UTCBs are pinned no pagefaults 12 / 33

13 Overview Portal Portal Portal calls calls calls Local Thread Global Thread Global Thread Local Thread Local Thread Sched Context Sched Context Sched Context Kernel schedules 13 / 33

14 Portal Properties Local Thread, that handles the portal Instruction Pointer (address of portal function) Id, delivered to the portal (parameter of portal function) Code example from NRE PORTAL static void portal_echo(void *id) { int main() { Reference<LocalThread> lt = LocalThread::create(); Pt echo(lt, portal_echo); echo.set_id(0x1234); echo.call(); 14 / 33

15 Timeslice Donation and Helping SC L SC H EC 1 calls portal with SC L Timeslice donation: EC 1 EC 2 SC L is donated to EC 3 Portal EC 3 Priority inversion: SC H is blocked by SC L (indirectly with SC M ) Helping: If SC L has no time left, SC H helps EC 3 I.e., EC 3 runs with SC H 15 / 33

16 Timeslice Donation and Helping EC 1 EC 2 Portal SC H Timeslice donation: EC 1 calls portal with SC L SC L is donated to EC 3 Priority inversion: SC H is blocked by SC L (indirectly with SC M ) EC 3 SC L Helping: If SC L has no time left, SC H helps EC 3 I.e., EC 3 runs with SC H 15 / 33

17 Timeslice Donation and Helping EC 1 EC 2 Portal SC H Timeslice donation: EC 1 calls portal with SC L SC L is donated to EC 3 Priority inversion: SC H is blocked by SC L (indirectly with SC M ) EC 3 SC L Helping: If SC L has no time left, SC H helps EC 3 I.e., EC 3 runs with SC H 15 / 33

18 Timeslice Donation and Helping EC 1 EC 2 Portal SC H Timeslice donation: EC 1 calls portal with SC L SC L is donated to EC 3 Priority inversion: SC H is blocked by SC L (indirectly with SC M ) EC 3 SC L Helping: If SC L has no time left, SC H helps EC 3 I.e., EC 3 runs with SC H 15 / 33

19 Timeslice Donation and Helping EC 1 EC 2 Portal Timeslice donation: EC 1 calls portal with SC L SC L is donated to EC 3 Priority inversion: SC H is blocked by SC L (indirectly with SC M ) EC 3 SC L SC H Helping: If SC L has no time left, SC H helps EC 3 I.e., EC 3 runs with SC H 15 / 33

20 UTCB Layout hotspot 0 flags CRD 0... d a t a h e a d e r... untyped word 1 untyped word 0 thread local storage CRD for delegates CRD for translates untyped words typed words 0 16 / 33

21 Syscall: Call Portal Sys_call *s = static_cast<sys_call *>(current->sys_regs()); Kobject *obj = Space_obj::lookup (s->pt()).obj(); Pt *pt = static_cast<pt *>(obj); Ec *ec = pt->ec; if (EXPECT_FALSE (current->cpu!= ec->xcpu)) sys_finish<sys_regs::bad_cpu>(); if (EXPECT_TRUE (!ec->cont)) { current->cont = ret_user_sysexit; current->set_partner (ec); // sets Ec::rcap ec->cont = recv_user; ec->regs.set_pt (pt->id); ec->regs.set_ip (pt->ip); ec->make_current(); ec->help (sys_call); 17 / 33

22 Syscall: Call Portal (2) void Ec::recv_user() { Ec *ec = current->rcap; ec->utcb->save (current->utcb); if (EXPECT_FALSE (ec->utcb->tcnt())) delegate<true>(); ret_user_sysexit(); void Ec::help (void (*c)()) { current->cont = c; if (EXPECT_TRUE (++Sc::ctr_loop < 100)) { Ec *ec = this; while(ec->partner) ec = ec->partner; ec->make_current(); die ("Livelock"); 18 / 33

23 Exception IPC The kernel should have no policy Userland should decide what to do in case of an exception In particular, memory management is done in userland Each EC has an exception portal selector offset At this offset, portals are expected for all exceptions 19 / 33

24 Exception IPC: Details void Ec::handle_exc (Exc_regs *r) { switch (r->vec) { case Cpu::EXC_NM: handle_exc_nm(); return; case Cpu::EXC_PF: if (handle_exc_pf (r)) return; break;... send_msg<ret_user_iret>(); 20 / 33

25 Exception IPC: Details template <void (*C)()> void Ec::send_msg() { Exc_regs *r = &current->regs; Kobject *obj = Space_obj::lookup ( current->evt + r->dst_portal).obj(); Pt *pt = static_cast<pt *>(obj); Ec *ec = pt->ec; if (EXPECT_TRUE (!ec->cont)) { ec->cont = recv_kern;... ec->help (send_msg<c>); void Ec::recv_kern() { Ec *ec = current->rcap; current->utcb->load_exc (&ec->regs); ret_user_sysexit(); 21 / 33

26 Outline Introduction Synchronous IPC in NOVA Asynchronous IPC in NOVA Synchronization Interrupts Userspace API 22 / 33

27 Semaphores A semaphore is a kernel object Properties: Counter Queue of ECs Operations (via syscall): Down Down to zero Up 23 / 33

28 Semaphores: Usecases Synchronization with shared memory (e.g., multithreading) Typically combined with atomic operations Atomic operations in case of no contention System call in case of contention Signaling (e.g., producer-consumer scenarios) Delivery of interrupts to userspace 24 / 33

29 Interrupt Semaphores Object cap space of root PD has semaphore per interrupt Can be delegated to device drivers,... Is up ed by the kernel on IRQ Usage example: Keyboard driver in NRE static void kbhandler(void*) { Gsi gsi(keyboard_irq); while(1) { gsi.down(); Keyboard::Packet data; if(hostkb->read(data)) broadcast(kbsrv, data); 25 / 33

30 Semaphore Operations void Sm::dn (bool zero) { Ec *e = Ec::current; { Lock_guard <Spinlock> guard (lock); if (counter) { counter = zero? 0 : counter - 1; return; enqueue (e); e->block_sc(); void Sm::up() { Ec *e; { Lock_guard <Spinlock> guard (lock); if (!(e = dequeue())) { counter++; return; e->release(); 26 / 33

31 Outline Introduction Synchronous IPC in NOVA Asynchronous IPC in NOVA Userspace API UTCB Frames IPC with C++ shift operators 27 / 33

32 Many Approaches Plain C API C++ shift operators to get/put values from/into UTCB C++ templates generate server and client stubs IDL compiler / 33

33 NOVA Runtime Environment (NRE) Uses C++ shift operators: + No external tool required + No separate language to learn + Rather simple to implement + Much simpler to use than C implementations Need to implement stub functions manually, if desired Need to keep client and server consistent (types, order,... ) Supports multiple frames within one UTCB: Allows nested usages of the UTCB Important for calling library functions 29 / 33

34 NRE UTCB Frames Sender Receiver EC 1 EC 2 UTCB 1 UTCB 2 push pop Frame 2 0 Cap(2) 36 Frame Cap(14) / 33

35 NRE UTCB Frames Sender Receiver EC 1 EC 2 UTCB 1 UTCB 2 push pop Frame 2 0 call() Cap(2) 36 Frame Cap(14) / 33

36 NRE UTCB Frames Sender Receiver EC 1 EC 2 UTCB 1 UTCB 2 push pop Frame 2 0 call() Cap(2) Frame Cap(14) Frame 30 / 33

37 Usage Example Client UtcbFrame uf; uf << 1 << String("foo"); portal.call(uf); int res; uf >> res; Server PORTAL static void myportal(void*) { UtcbFrameRef uf; int i; String s; uf >> i >> s; // handle the request uf << 0; 31 / 33

38 Implementation template<typename T> UtcbFrameRef & operator<<(const T& value) { const size_t words = (sizeof(t) + sizeof(word_t) - 1) / sizeof(word_t); *reinterpret_cast<t*>( _utcb->msg + untyped() * sizeof(word_t)) = value; _utcb->untyped += words; return *this; template<typename T> UtcbFrameRef & operator>>(t &value) { const size_t words = (sizeof(t) + sizeof(word_t) - 1) / sizeof(word_t); value = *reinterpret_cast<t*>( _utcb->msg + _upos * sizeof(word_t)); _upos += words; return *this; 32 / 33

39 Next Time May 17th: Exercise 1 (Kernel entry, exit, syscalls) in room APB E / 33

THREADS ADMINISTRIVIA RECAP ALTERNATIVE 2 EXERCISES PAPER READING MICHAEL ROITZSCH 2

THREADS ADMINISTRIVIA RECAP ALTERNATIVE 2 EXERCISES PAPER READING MICHAEL ROITZSCH 2 Department of Computer Science Institute for System Architecture, Operating Systems Group THREADS ADMINISTRIVIA MICHAEL ROITZSCH 2 EXERCISES due to date and room clashes we have to divert from our regular