What we will learn Multi Systems: es &Threads A thread-driven approach is attractive in sensor networks for many of the same reasons that it has been adopted for PDAs, laptops, and servers. In a thread-driven system, an application programmer need not be concerned about indefinitely blocking or being indefinitely blocked by other tasks during execution, except for shared resources, because the scheduler will preemptively time-slice between threads, allowing some tasks to continue execution even though others may be blocked. This automated time-slicing considerably simplifies programming for an application developer. es (classic vs. modern) ST OS MT OS MP OS { Bag of functions & procedures Coroutines weak AUCs Threads es AUCs (AUCs) (AUCs) AUCs (AUCs) Architectural support: or Modes Mode bit: Supervisor or User mode Supervisor mode Can execute all machine instructions Can reference all memory locations User mode Can only execute a subset of instructions Can only reference a subset of memory locations
Architectural support: or Modes Total CPU time percentage Architectural support: or Modes The status register contains information about the status of features of the computer that can be set by the processor while in kernel mode Kernel mode time percentage indicates whether the processor was in kernel or user mode when the last exception occurred indicates whether current status is kernel or user Architectural support: or Modes (MIPS) The processor contains registers useful for handling exceptions but is not accessible in user mode. the status register, cause register, BadVaddr, and EPC (exception program counter). Architectural support: or Modes (MIPS) name number information BadVaddr 8 memory address at which address exception occurred Status 12 interrupt mask and enable bits Cause 13 exception type and pending interrupts EPC(IAR) 14 address of instruction that caused exception
Architectural support: or Modes (MIPS) The status register contains information about the status of features of the computer that can be set by the processor while in kernel mode indicates whether the processor was in kernel or user mode when the last exception occurred 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Mode: 1=user, 0=kernel Enable: 1=on, 0=off indicates whether current status is kernel or user Mode Enable Mode Enable Mode Enable Old Prev Cur Kernels The part of the OS critical to correct operation (trusted software) Executes in supervisor mode The trap instruction is used to switch from user to supervisor mode, entering the OS What happens on a trap/interrupt? 1. Hardware saves program counter (on stack or in a special register) 2. Hardware loads new PC, identifies interrupt 3. Assembly language routine saves registers 4. Assembly language routine sets up stack 5. Assembly language calls C to run service routine 6. Service routine calls scheduler 7. Scheduler selects a process to run next (might be the one interrupted) What happens on a trap? OS runs in privileged mode Some operations are permitted only in privileged (also called supervisor or system) mode Example: access a device like a disk or network card Example: change allocation of memory to processes User programs run in user mode and can t do the operations Programs want the OS to perform a service Operating Systems and 11Distributed Systems Operating Systems and 12Distributed Systems
How system calls work User program enters supervisor mode Must enter via well-defined entry point Program passes relevant information to OS OS performs the service if The OS is able to do so The service is permitted for this program at this time OS checks information passed to make sure it s OK Don t want programs reading data into other System call: Sleep() Program pushes arguments, calls library Library sets up trap, calls OS OS handles system call Control returns to library Library returns to user program 0xffffffff User space Kernel space (OS) 4 3 1 2 Return to caller Trap to kernel Trap code in register Increment SP Call Sleep Push arguments 5 6 Dispatch 9 8 Sys call handler Library (Sleep call) 7 User code Operating Systems and 13Distributed Systems Operating Systems and 14Distributed Systems traps.h #define! TRAP_SLEEP! 0x410 process.h extern void!sleep (void); Supervisor and User Memory ;;; Stub functions for traps..proc _Sleep.global _Sleep _Sleep:!! trap! #0x410!! nop! jr! r31! nop.endproc _Sleep.proc _intrhandler.global _intrhandler _intrhandler:... ;; push regs on stack j! _dointerrupt!.endproc _intrhandler #include <sys/process.h > void main(){!...! Sleep();!... } void dointerrupt (unsigned int cause, unsigned int iar, unsigned int isr,! uint32 *trapargs) { // TRAP_INSTR bit is set: trap instruction. // bit isn't set:system interrupt. if (cause & TRAP_TRAP_INSTR) {... switch (cause) { case TRAP_SLEEP: Suspend (currentpcb); Schedule (); break;... intrreturn (); } User Supervisor User Space Supervisor Space STATUS register.proc! _intrreturn!.global!_intrreturn _intrreturn:! ;; Disable interrupts CAUSE register...;; Reload the regs for new process..endproc _intrreturn
Threads, es, Modern es Implementing the Abstraction P i CPU P j CPU P k CPU Thread P i Executable Memory P j Executable Memory P k Executable Memory OS interface Thread Thread Thread Thread CPU ALU Control Unit OS Address Space P i Address Space P k Address Space P j Address Space Machine Executable Memory What s in a process? The Address Space A process consists of (at least): an address space the code for the running program the data for the running program an execution stack and stack pointer (SP) traces state of procedure calls made the program counter (PC), indicating the next instruction general-purpose processor registers and their values 0xFFFFFFFF address space 0x00000000 stack (dynamic allocated mem) heap (dynamic allocated mem) static data (data segment) code (text segment) SP PC Operating Systems and 19Distributed Systems Operating Systems and 20Distributed Systems
The Address Space Address Space Address Binding Executable Memory Files Other objects Building the Address Space Some parts are built into the environment Files System services Some parts are imported at runtime Mailboxes Network connections Memory addresses are created at compile (and run) time Manager Overview Program Device Manager Devices File Manager Abstract Computing Environment Memory Manager Memory Protection Deadlock Synchronization Scheduler CPU Description Resource Resource Manager Resource Manager Manager Other H/W Libraries Device Manager UNIX Organization File Manager Memory Manager Monolithic Kernel Devices Memory System Call Interface Protection Deadlock Synchronization Scheduler CPU Description Resource Resource Manager Resource Manager Manager Other H/W
Windows NT Organization External View of the Manager User Libraries T T T Subsystem T Subsystem T T T T T Subsystem wait() fork() exec() Application Program CreateThread() CloseHandle() Create() WaitForSingleObject() NT Executive NT Kernel Hardware Abstraction Layer I/O Subsystem UNIX Device Mgr Mgr Memory Mgr File Mgr Device Mgr Mgr Memory Mgr File Mgr Windows or(s) Main Memory Devices Hardware Manager Responsibilities Define & implement the essential characteristics of a process and thread Algorithms to define the behavior Data structures to preserve the state of the execution Define what things threads in the process can reference the address space (most of the things are memory locations) Manage the resources used by the processes/threads Tools to create/destroy/manipulate processes & threads Tools to time-multiplex the CPU Scheduling Tools to allow threads to synchronization the operation with one another Mechanisms to handle deadlock Mechanisms to handle protection User Mode Instructions The Abstract Machine Interface Application Program User Mode Instructions Trap Instruction Abstract Machine Instructions OS open() Supervisor Mode Instructions create() fork()
Machine is Powered up Hardware process progress Tracing the Hardware Load the kernel Initialization Execute a thread Schedule Bootstap Service an interrupt Loader Manager Interrupt Handler P 1 P,2 P n Interrupt Context Switching Initialization 1 7 9 6 4 Executable Memory 2 Manager Interrupt Handler P 1 P 2 8 3 5 P n Descriptors There s a data structure called the process control block (PCB) that holds all this stuff The PCB is identified by an integer process ID (PID) OS keeps all of a process s hardware execution state in the PCB when the process isn t running PC, SP, registers, etc. when a process is unscheduled, the state is transferred out of the hardware into the PCB Note: It s natural to think that there must be some esoteric techniques being used fancy data structures that d you d never think of yourself Wrong! It s pretty much just what you d think of! Descriptors ID Pointer to parent List of children state Pointer to address space descriptor Program count stack pointer (all) register values uid (user id) gid (group id) euid (effective user id) Open file list Scheduling priority Accounting info Pointers for state queues This is (a simplification of) what each of those PCBs looks like inside! 12/31/2006 2007 Gribble, Lazowska, Levy, Zahorjan 31 Exit ( return ) code value 32
What s in a process table entry? Unix Descriptor May be stored on stack management Registers Program counter CPU status word Stack pointer state Priority / scheduling parameters ID Parent process ID Signals start time Total CPU usage File management Root directory Working (current) directory File descriptors User ID Group ID Memory management Pointers to text, data, stack! or Pointer to page table Operating Systems and 33Distributed Systems Linux Descriptor Windows NT Descriptor NT Kernel EPROCESS KPROCESS uint32 KernelTime; uint32 UserTime; Byte state; NT Executive void *UniqueId;
Windows NT Thread Descriptor EPROCESS KPROCESS ETHREAD KTHREAD NT Kernel NT Executive When is a process created? es can be created in two ways System initialization (creatio ex nihilo): one or more processes created when the OS starts up Execution of a process creation system call: something explicitly asks for a new process System calls can come from User request to create a new process (system call executed from user shell) Already running processes User programs System daemons Operating Systems and 38Distributed Systems Creating a in UNIX Creating a in NT Create(); pid = fork(); Table UNIX kernel Win32 Subsystem ntcreate(); ntcreatethread(); NT Executive Descriptor Handle Table Descriptor NT Kernel
Windows NT Handles states Each process has an execution state, which indicates what it is currently doing ready: waiting to be assigned to CPU could run, but another process has the CPU running: executing on the CPU is the process that currently controls the CPU pop quiz: how many processes can be running simultaneously? waiting: waiting for an event, e.g., I/O cannot make progress until event happens Operating Systems and 42Distributed Systems Simple State Diagram Done exception (I/O, page fault, etc.) Blocked Request Running dispatch / schedule interrupt (I/O complete) Ready interrupt (unschedule) Start The PCB revisited (Unix) The PCB is a data structure with many, many fields: process ID (PID) execution state program counter, stack pointer, registers address space info UNIX username of owner scheduling priority accounting info pointers for state queues In linux: defined in task_struct (include/linux/sched.h) over 95 fields!!! Operating Systems and 44Distributed Systems
Refined state model Linux State Transition Diagram 5 Blocked (waiting) 7 Created 1 Ready 3 4 7 Exit 2 Running 6! in one of 5 states " Created " Ready " Running " Blocked " Exit! Transitions between states 1. enters ready queue 2. Scheduler picks this process 3. Scheduler picks a different process 4. waits for event (such as I/ O) 5. Event occurs 6. exits 7. ended by another process Request Wait by parent Done Running zombie Request Schedule Sleeping I/O Request Start Allocate Uninterruptible Sleep I/O Complete Resume Runnable Traced or Stopped Operating Systems and 45Distributed Systems Linux State Transition Diagram Windows NT Thread States TASK_INTERRUPTIBLE The process is suspended (sleeping) until some condition becomes true. Raising a hardware interrupt, releasing a system resource the process is waiting for, or delivering a signal are examples of conditions that might wake up the process (put its state back to TASK_RUNNING). TASK_UNINTERRUPTIBLE Like TASK_INTERRUPTIBLE, except that delivering a signal to the sleeping process leaves its state unchanged. This process state is seldom used. It is valuable, however, under certain specific conditions in which a process must wait until a given event occurs without being interrupted. For instance, this state may be used when a process opens a device file and the corresponding device driver starts probing for a corresponding hardware device. The device driver must not be interrupted until the probing is complete, or the hardware device could be left in an unpredictable state. TASK_STOPPED execution has been stopped; the process enters this state after receiving a SIGSTOP, SIGTSTP, SIGTTIN, or SIGTTOU signal. TASK_TRACED execution has been stopped by a debugger. When a process is being monitored by another (such as when a debugger executes a ptrace( ) system call to monitor a test program), each signal may put the process in the TASK_TRACED state. Two additional states of the process can be stored both in the state field and in the exit_state field of the process descriptor; as the field name suggests, a process reaches one of these two states only when its execution is terminated: Terminated Running Exit CreateThread Reinitialize Initialized Dispatch Activate Wait Waiting Wait Complete Ready EXIT_ZOMBIE execution is terminated, but the parent process has not yet issued a wait4( ) or waitpid( ) system call to return information about the dead process.[*] Before the wait( )-like call is issued, the kernel cannot discard the data contained in the dead process descriptor because the parent might need it. (See the section " Removal" near the end of this chapter.) [*] There are other wait( ) -like library functions, such as wait3( ) and wait( ), but in Linux they are implemented by means of the wait4( ) and waitpid( ) system calls. Wait Complete Transition Select Preempt EXIT_DEAD The final state: the process is being removed by the system because the parent process has just issued a wait4( ) or waitpid( ) system call for it. Changing its state from EXIT_ZOMBIE to EXIT_DEAD avoids race conditions due to other threads of execution that execute wait( )-like calls on the same process (see Chapter 5). The value of the state field is usually set with a simple assignment. For instance: Dispatch Standby p->state = TASK_RUNNING;
Modern processes: concurrency Imagine a web server, which might like to handle multiple requests concurrently While waiting for the credit card server to approve a purchase for one client, it could be retrieving the data requested by another client from disk, and assembling the response for a third client from cached information Imagine a web client (browser), which might like to initiate multiple requests concurrently The CSE home page has 46 src= html commands, each of which is going to involve a lot of sitting around! Wouldn t it be nice to be able to launch these requests concurrently? Imagine a parallel program running on a multiprocessor, which might like to employ physical concurrency For example, multiplying a large matrix split the output matrix into k regions and compute the entries in each region concurrently using k processors Operating Systems and 49Distributed Systems Modern processes: concurrency In each of these examples of concurrency (web server, web client, parallel program): Everybody wants to run the same code Everybody wants to access the same data Everybody has the same privileges Everybody uses the same resources (open files, network connections, etc.) But you d like to have multiple hardware execution states: an execution stack and stack pointer (SP) traces state of procedure calls made the program counter (PC), indicating the next instruction a set of general-purpose processor registers and their values Operating Systems and 50Distributed Systems Modern processes: concurrency Given the process abstraction as we know it: fork several processes cause each to map to the same physical memory to share data see the shmget() system call for one way to do this (kind of) This is like making a pig fly it s really inefficient space: PCB, page tables, etc. time: creating OS structures, fork and copy addr space, etc. Some equally bad alternatives for some of the examples: Entirely separate web servers Manually programmed asynchronous programming (non-blocking I/O) in the web client (browser) Can we do better? Key idea: separate the concept of a process (address space, etc.) from that of a minimal thread of control (execution state: PC, etc.) This execution state is usually called a thread, or sometimes, a lightweight process Operating Systems and 51Distributed Systems Operating Systems and 52Distributed Systems
Modern es and Threads Thrd j in P i Thrd k in P i Single and Multithreaded es P i CPU OS interface Threads: processes sharing memory == address space Thread == program counter / stream of instructions Two examples Three processes, each with one thread One process with three threads 1 2 3 1 & thread information Per process items Address space Open files Child processes Signals & handlers Accounting info Global variables User space System space Threads Kernel Threads Kernel Per thread items Program counter Registers Stack & stack pointer State Per thread items Program counter Registers Stack & stack pointer State Per thread items Program counter Registers Stack & stack pointer State Operating Systems and 55Distributed Systems Operating Systems and 56Distributed Systems
Threads & stacks Multithreaded Web server Thread 1"s stack Thread 1 Thread 2 Thread 3 User space Dispatcher thread Worker thread while(true) { getnextrequest(&buf); handoffwork(&buf); } Thread 2"s stack Kernel => Each thread has its own stack! Thread 3"s stack Network connection Kernel Web page cache while(true) { waitforwork(&buf); lookforpageincache(&buf,&page); if(pagenotincache(&page)) { readpagefromdisk(&buf,&page); } returnpage(&page); } Operating Systems and 57Distributed Systems Operating Systems and 58Distributed Systems Threads and processes The design space Most modern OS s (Mach, Chorus, NT, modern UNIX) therefore support two entities: the process, which defines the address space and general process attributes (such as open files, etc.) the thread, which defines a sequential execution stream within a process A thread is bound to a single process / address space address spaces, however, can have multiple threads executing within them sharing data between threads is cheap: all see the same address space creating threads is cheap too! Threads become the unit of scheduling processes / address spaces are just containers in which threads execute Operating Systems and 59Distributed Systems Key address space thread MS/DOS Java one thread/process one process many threads/process one process Operating Systems and 60Distributed Systems one thread/process many processes many threads/process many processes older UNIXes Mach, NT, Chorus, Linux,
(old) address space 0xFFFFFFFF address space 0x00000000 stack (dynamic allocated mem) heap (dynamic allocated mem) static data (data segment) code (text segment) Operating Systems and 61Distributed Systems SP PC (new) address space with threads 0xFFFFFFFF address space 0x00000000 thread 1 stack thread 2 stack thread 3 stack heap (dynamic allocated mem) static data (data segment) code (text segment) Operating Systems and 62Distributed Systems SP (T1) SP (T2) SP (T3) PC (T2) PC (T1) PC (T3) /thread separation: advantages Concurrency (multithreading) is useful for: handling concurrent events (e.g., web servers and clients) building parallel programs (e.g., matrix multiply, ray tracing) improving program structure (the Java argument) Multithreading is useful even on a uniprocessor even though only one thread can run at a time Supporting multithreading that is, separating the concept of a process (address space, files, etc.) from that of a minimal thread of control (execution state), is a big win creating concurrency does not require creating new processes faster / better / cheaper Operating Systems and 63Distributed Systems Where do threads come from? Natural answer: the kernel is responsible for creating/managing threads for example, the kernel call to create a new thread would allocate an execution stack within the process address space create and initialize a Thread Control Block stack pointer, program counter, register values stick it on the ready queue we call these kernel threads Operating Systems and 64Distributed Systems
Where do threads come from? (2) Threads can also be managed at the user level (that is, entirely from within the process) a library linked into the program manages the threads because threads share the same address space, the thread manager doesn t need to manipulate address spaces (which only the kernel can do) threads differ (roughly) only in hardware contexts (PC, SP, registers), which can be manipulated by user-level code the thread package multiplexes user-level threads on top of kernel thread(s), which it treats as virtual processors we call these user-level threads Run-time system Implementing threads Thread table Kernel table User-level threads + No need for kernel support - May be slower than kernel threads - Harder to do non-blocking I/O Thread Kernel table Kernel-level threads + More flexible scheduling + Non-blocking I/O - Not portable Thread table Operating Systems and 65Distributed Systems Operating Systems and 66Distributed Systems Kernel Threads Supported by the Kernel Examples Windows XP/2000 Solaris Linux Tru64 UNIX Mac OS X User Threads Thread management done by user-level threads library Three primary thread libraries: POSIX Pthreads Win32 threads Java threads
Kernel threads OS now manages threads and processes all thread operations implemented in the kernel OS schedules all of the threads in a system one thread in a process blocks ( I/O): the OS knows about it, and can run other threads from that process possible to overlap I/O and computation inside a process Kernel threads are cheaper than processes less state to allocate and initialize Still pretty expensive for fine-grained use (e.g., orders of magnitude more expensive than a procedure call) thread operations are all system calls context switch argument checks must maintain kernel state for each thread User-level threads To make threads cheap and fast, they need to be implemented at the user level managed entirely by user-level library, e.g., libpthreads.a The thread package multiplexes user-level threads on top of kernel thread(s), which it treats as virtual processors User-level threads are small and fast each thread is represented simply by PC & registers, stack, small thread control block (TCB) creating a thread, switching between threads, and synchronizing threads are done via procedure calls no kernel involvement is necessary! user-level thread operations can be 10-100x faster than kernel threads as a result Operating Systems and 69Distributed Systems Operating Systems and 70Distributed Systems Performance example On a 700MHz Pentium running Linux 2.2.16: es fork/exit: 251 µs Kernel threads pthread_create()/pthread_join(): 94 µs (2.5x faster) User-level threads pthread_create()/pthread_join: 4.5 µs (another 20x faster) Performance example (2) On a 700MHz Pentium running Linux 2.2.16: On a DEC SRC Firefly running Ultrix, 1989 es fork/exit: 251 µs / 11,300 µs Kernel threads pthread_create()/pthread_join(): 94 µs / 948 µs (12x faster) User-level threads pthread_create()/pthread_join: 4.5 µs / 34 µs (another 28x faster) Operating Systems and 71Distributed Systems Operating Systems and 72Distributed Systems
The design space Kernel thread implementation address space MS/DOS one thread/process one process one thread/process many processes older UNIXes address space Mach, NT, Chorus, Linux, os kernel thread Java many threads/process one process many threads/process many processes Mach, NT, Chorus, Linux, thread CPU (thread create, destroy, signal, wait, etc.) Operating Systems and 73Distributed Systems Operating Systems and 74Distributed Systems User-level threads: conceptually user-level thread library User-level threads: implemented user-level thread library address space? os kernel Mach, NT, Chorus, Linux, (thread create, destroy, signal, wait, etc.) address space os kernel Mach, NT, Chorus, Linux, (thread create, destroy, signal, wait, etc.) kernel threads thread CPU thread CPU (kernel thread create, destroy, signal, wait, etc.) Operating Systems and 75Distributed Systems Operating Systems and 76Distributed Systems
User-level threads: implemented Multithreading Models user-level thread library address space Mach, NT, Chorus, Linux, (thread create, destroy, signal, wait, etc.) Many-to-One One-to-One thread os kernel CPU kernel threads Many-to-Many (kernel thread create, destroy, signal, wait, etc.) Operating Systems and 77Distributed Systems Many-to-One Many-to-One Model Many user-level threads mapped to single kernel thread Examples: Solaris Green Threads GNU Portable Threads
One-to-One One-to-one Model Each user-level thread maps to kernel thread Examples Windows NT/XP/2000 Linux Solaris 9 and later Many-to-Many Model Many-to-Many Model Allows many user level threads to be mapped to many kernel threads Allows the operating system to create a sufficient number of kernel threads Solaris prior to version 9 Windows NT/2000 with the ThreadFiber package
Two-level Model Two-level Model Similar to M:M, except that it allows a user thread to be bound to kernel thread Examples IRIX HP-UX Tru64 UNIX Solaris 8 and earlier User-level thread implementation The kernel believes the user-level process is just a normal process running code But, this code includes the thread support library + its associated thread scheduler The thread scheduler determines when a thread runs it uses queues to keep track of what threads are doing: run, ready, wait just like the OS and processes but, implemented at user-level as a library How to keep a user-level thread from hogging the CPU? Strategy 1: force everyone to cooperate a thread willingly gives up the CPU by calling yield() yield() calls into the scheduler, which context switches to another ready thread what happens if a thread never calls yield()? Strategy 2: use preemption scheduler requests that a timer interrupt be delivered by the OS periodically usually delivered as a UNIX signal (man signal) signals are just like software interrupts, but delivered to user-level by the OS instead of delivered to OS by hardware at each timer interrupt, scheduler gains control and context switches as appropriate Operating Systems and 87Distributed Systems Operating Systems and 88Distributed Systems
Thread context switch Very simple for user-level threads: save context of currently running thread push machine state onto thread stack restore context of the next thread pop machine state from next thread s stack return as the new thread execution resumes at PC of next thread This is all done by assembly language it works at the level of the procedure calling convention thus, it cannot be implemented using procedure calls e.g., a thread might be preempted (and then resumed) in the middle of a procedure call What if a thread tries to do I/O? The kernel thread powering it is lost for the duration of the (synchronous) I/O operation! Could have one kernel thread powering each user-level thread no real difference from kernel threads common case operations (e.g., synchronization) would be quick Could have a limited-size pool of kernel threads powering all the user-level threads in the address space the kernel will be scheduling these threads, obliviously to what s going on at user-level Operating Systems and 89Distributed Systems Operating Systems and 90Distributed Systems What if the kernel preempts a thread holding a lock? Other threads will be unable to enter the critical section and will block (stall) tradeoff, as with everything else Solving this requires coordination between the kernel and the user-level thread manager Possible solution: scheduler activations each process can request one or more kernel threads process is given responsibility for mapping user-level threads onto kernel threads kernel promises to notify user-level before it suspends or destroys a kernel thread Summary Operating Systems and 91Distributed Systems