Microkernel Construction. Introduction. Michael Hohmuth. Lars Reuther. TU Dresden Operating Systems Group

Transcription

1 Introduction

2 Lecture Goals Provide deeper understanding of OS mechanisms Make all of you enthusiastic kernel hackers Illustrate alternative system design concepts Promote OS research at 2

3 Administration Tuesday, 4 th DS, GRU 370, 2 SWS (1V + 1Ü) Slides / Handouts available at Mailinglist: In summer term: of Microkernel-based (2 SWS) Komplexpraktikum (2 SWS) 3

4 Motivation Monolithic Applications Applications Applications File Network Stacks User Mode Privileged Mode Memory Management Processes Monolithic System Hardware Drivers 4

5 Monolithic Why something different? All system components run in privileged kernel mode No protection between system components Faulty driver can crash the whole system More than 2/3 of today's OS code are drivers No need for good system design Direct access to kernel structures Big and inflexible Not suited for embedded systems Difficult to replace core system components More and more difficult to manage increasing OS complexity 5

6 The Microkernel Vision Small OS kernel Less error prone Small Trusted Computing Base Allows validation System service implemented as user-level servers Flexibility Extensibility Protect individual system components More secure / safe systems 6

7 The Microkernel Vision Allow coexistence of different OS personalities Build customizable systems One OS for embedded systems, PC systems, MP, Enforce reasonable system design Use well defined interfaces to system services No dependencies between system services other than explicitly specified through service interfaces Improved maintainability 7

8 Envisioned System Design Applications Applications Applications File Network Stacks Memory Management System Services Drivers Name Server User Mode Tasks Threads IPC Microkernel I/O Support Privileged Mode Hardware 8

9 Examples Embedded QNX Message passing system Network transparancy Process Manager IPC Interrupt Redirector Network Interface Scheduler Network Manager LynxOS Separation Kernel Combine real-time and secure systems Chorus Device Manager App A App B System Services QNX Microkernel Partitions App A System Services Separation Kernel (Microkernel) Hardware Filesystem Manager Security Policy System Services 9

10 Mach-Based Mach Developed at Carnegie Mellon University (CMU) & OSF Simple, extensible communication kernel Communication channels Memory objects Basis of several systems Single Server UNIX (CMU): Port of 4.3BSD to Mach MKLinux (OSF): Port of Linux to Mach IBM Workplace OS Mac OS X Shortcomings Performance Pushed drivers back into kernel 10

11 Mac OS X Mac OS X Kernel (Darwin) based on Mach/BSD Drivers / BSD services run in kernel mode Classic Carbon Cocoa Application Services Java (JDK) QuickTime BSD Core Services Kernel Environment File System Networking NKE I/O Kit Drivers BSD Mach 11

12 IBM Workplace OS Main Goals: Multiple OS personalities with minimal code duplication Portable to different hardware architectures OS/2 OS/2 Applications Applications OS/400 OS/400 Applications Applications AIX AIX Applications Applications Windows Windows Applications Applications OS/2 Personality DOS Personality OS/400 Personality AIX Personality Windows Personality File Server Network Service Security Power Management Default Pager Device Support Bootstrap Name Service Microkernel ARM PowerPC MIPS IA32 Alpha 12

13 IBM Workplace OS Based on Mach 3.0 Microkernel (CMU / OSF) Never finished (rudiments in last OS/2 version) Failure causes: Underestimated difficulties in creating OS personalities Management errors, forced divisions to adopt new system without having a system Second System Effect : too many fancy features Conclusion: Microkernel worked, but system atop the microkernel did not 13

14 Visions vs. Reality Disappointments Multiple OS personalities Did not work Flexibility Yes, but monolithic kernels became flexible too (kernel modules, ) System design Yes, but monolithic kernels got better too (layered kernel design, modular kernels, ) Maintainability Better, but still very complex Performance 14

15 Visions vs. Reality What holds? Subsystem protection / isolation Code size Fiasco kernel: < lines of code Minimal application: (boot loader + hello world ): ~6.000 loc Linux kernel (2.6.5, i386): 3.2 million loc (drivers: 1.9 million) Customizable Tailored memory management / scheduling / algorithms Adaptable to embedded / real-time / secure / systems 15

16 Challenges Build fast Microkernels L4 microkernel family, Fiasco microkernel Subject of this lecture Provide OS services efficiently Memory Management Synchronization Device Drivers File Communication Interfaces Subject of lecture of Microkernel-based systems (in summer term) 16

17 L4 Microkernel Originally developed by Jochen Liedtke (GMD / IBM Research) 2 nd Generation Microkernel Development continued Uni Karlsruhe and UNSW Sydney (Hazelnut / Pistachio) (Fiasco) Kernel API versions: V2: current stable version (Fiasco) X0: experimental (Fiasco, Hazelnut) X2: experimental, aka L4 V4 (Pistachio, Fiasco) 17

18 L4 Concepts Jochen Liedtke: A microkernel does no real work Kernel provides only inevitable mechanisms No policies enforced by the kernel What is inevitable? Abstractions Mechanisms Threads Communication Scheduling Address Spaces Mapping This should be sufficient for everything 18

19 L4 Address Spaces Recursive Address Spaces: Application Application Application Pager 4 Pager 3 Pager 1 Pager 2 Initial Address Space Physical Memory 19

20 L4 Threads Thread: Abstraction and unit of execution Identified by thread id Consists of Instruction pointer Stack Registers L4 only manages (preserves) IP, SP and registers Entry point, stack allocation (size, location) is managed by user-level applications Several threads in an address space Up to 128 threads (L4 V2) Address Space Code Data Stack Stack Thread Execution Paths 20

21 L4 Communication (IPC) Synchronous communication between two threads To successfully send a message both sender and receiver must agree to participate Timeouts No buffering in the kernel Efficient implementation IPC types send receive from receive call (send and receive from) reply and wait (send and receive) 21

22 L4 IPC Message Types Register message ( short IPC ) Thread A Thread B send( ) receive( ) EBX EDX EBX EDX Fast, no memory reference to transfer data (kernel needs only to access TCBs) 22

23 L4 IPC Message Types Buffer message ( direct long IPC ) Message buffer layout: dword3 +24 dword2 +20 EBX dword1 +16 EDX message send dope message size dope dword0 dwords (19) 0 (5) ~ (8) dwords (19) 0 (5) ~ (8)

24 L4 IPC Message Types Buffer message ( direct long IPC ) Thread A Thread B send(msg, ) receive(msg, ) copy 24

25 L4 IPC Message Types Buffer message ( indirect long / string IPC ) receive address +24 string dope receive size send address message send dope message size dope send size dwords (19) strings (5) ~ (8) dwords (19) strings (5) ~ (8)

26 L4 IPC Message Types Buffer message ( indirect long / string IPC ) Thread A Thread B send(msg, ) receive(msg, ) copy 26

27 L4 IPC Message Types Flexpages write grant flexpage page address (20) ~ (4) size (6) +16 send base message send dope message size dope dwords (19) strings (5) ~ (8) dwords (19) strings (5) ~ (8) receive window page address (20) ~ (4) size (6) ~ 0 27

28 L4 IPC Message Types Flexpages Thread A Thread B send(msg, ) receive(msg, ) map send base copy Flexpages are used to manage address spaces 28

29 L4 IPC Restriction Need to control who can send IPC to whom Security Network IPC / Migration Clans & Chiefs: Chief 1 Chief 2 Thread A Chief 3 Task Thread B Thread C Inflexible & inefficient New approaches IPC redirection Communication spaces / Virtual threads Control which threads can be seen Clan 29

30 L4 Device I/O Support Hardware interrupts: mapped to IPC Special thread id for interrupts Driver receive (irq-id, ) L4 Microkernel Interrupt I/O memory & I/O ports: flexpages 30

31 L4 System Calls (L4 V2) Address Spaces l4_task_new create / delete tasks Threads l4_thread_ex_regs create / modify threads l4_thread_schedule modify scheduling parameter l4_thread_switch switch to a different thread IPC l4_ipc send / receive IPC l4_fpage_unmap unmap flexpage l4_nchief return nearest communication partner 31

32 L 4 Linux init sh Linux Kernel Linux kernel and processes run as user-level L4 tasks Systemcalls / pagefaults are mapped to L4 IPC User Mode Interrupt- Threads Pager L 4 Linux Kernel Server Privileged Mode L4 Microkernel 32

33 DROPS The Dresden Real-Time System Real-Time Filesystem Real-Time Protocol L 4 Linux SCSI/IDE Driver Network Driver Display Driver User Mode Non-Real-Time Real-Time Resource Management Privileged Mode L4 Microkernel 33

34 Virtual Machines Several isolated OS atop a single physical machine Server consolidation, current hardware powerful enough to host several web / database / servers Web Server Domain 1 Database Server Web Server Domain 2 User Mode (L4)Linux (L4)Linux Virtualization Layer (L4)Linux Privileged Mode L4 Microkernel 34

35 µsina Secure Microkernel-based System Architecture VPN Gateway Local Network Network L 4 Linux Encryption / Routing L 4 Linux Network Internet User Mode Privileged Mode Virtualization Layer L4 Microkernel 35

36 Microkernel Lecture Outline Introduction Address spaces, threads, thread switching Kernel entry and exit Thread creation Synchronization Address space management: Mapping database Scheduling Portability Platform optimizations Outlook: Current L4 developments 36

37 Microkernel Tutorials Microkernels Building, installing, and using Fiasco/UX Guide to the Fiasco source code Hacking Fiasco 37

38 Microkernel : The Team Udo Steinberg Marcus Völp Alexander Warg Thanks to for his slides! 38

39 Next lecture: Address spaces and threads Kernel-virtual-address space Threads and thread control blocks (TCBs) TCB lookup Tasks Page tables Thread and task switching FPU switching 39