Learning Outcomes System Calls Interface and Implementation A high-level understanding of System Call interface Mostly from the user s perspective From textbook (section.6) Understanding of how the application-kernel boundary is crossed with system calls in general Including an appreciation of the relationship between a case study (OS/6 system call handling) and the general case. Exposure architectural details of the MIPS R3 Detailed understanding of the of exception handling mechanism From Hardware Guide on class web site Understanding of the existence of compiler function calling conventions Including details of the MIPS C compiler calling convention 2 The Structure of a Computer System System Calls Interface Application System Libraries User Mode Kernel Mode Device OS Device Interaction via System Calls Memory 3 4 System Calls Can be viewed as special function calls Provides for a controlled entry into the kernel While in kernel, they perform a privileged operation Returns to original caller with the result The system call interface represents the abstract machine provided by the operating system. The System Call Interface: A Brief Overview From the user s perspective Process Management File I/O Directories management Some other selected Calls There are many more On Linux, see man syscalls for a list 5 6
Some System Calls For Process Management Some System Calls For File Management 7 8 Some System Calls For Directory Management Some System Calls For Miscellaneous Tasks 9 A stripped down shell: System Calls System Calls while (TRUE) { /* repeat forever */ type_prompt( ); /* display prompt */ read_command (command, parameters) /* input from terminal */ if (fork()!= ) { /* fork off child process */ /* Parent code */ waitpid( -, &status, ); /* wait for child to exit */ } else { /* Child code */ execve (command, parameters, ); /* execute command */ } } Some Win32 API calls 2 2
System Call Implementation Crossing user-kernel boundary A Simple Model of CPU Computation The fetch-execute cycle Load memory contents from address in program counter () The instruction Execute the instruction Increment Repeat CPU Registers : x3 3 4 A Simple Model of CPU Computation Stack Pointer Register Condition codes Positive result Zero result Negative result General Purpose Registers Holds operands of most instructions Enables programmers (compiler) to minimise memory references. CPU Registers : x3 SP: xcbf3 R Rn Privileged-mode Operation To protect operating system execution, two or more CPU modes of operation exist Privileged mode (system-, kernelmode) All instructions and registers are available User-mode Uses safe subset of the instruction set Only affects the state of the application itself They cannot be used to uncontrollably interfere with OS Only safe registers are accessible CPU Registers Interrupt Mask Exception Type MMU regs Others : x3 SP: xcbf3 R Rn 5 6 Example Unsafe Instruction cli instruction on x86 architecture Disables interrupts Example exploit cli /* disable interrupts */ while (true) /* loop forever */; 7 Privileged-mode Operation The accessibility of addresses within an address space changes depending on operating mode To protect kernel code and data Note: The exact memory ranges are usually configurable, and vary between CPU architectures and/or operating systems. xffffffff x8 x Memory Address Space Accessible only to Kernel-mode Accessible to User- and Kernel-mode 8 3
System Call Questions we ll answer User Mode Kernel Mode System call mechanism securely transfers from user execution to kernel execution and back. Application System Call Handler 9 There is only one register set How is register use managed What does an application expect a system call to look like How is the transition to kernel mode triggered Where is the OS entry point (system call handler) How does the OS know what to do 2 System Call Mechanism Overview System call transitions triggered by special processor instructions User to Kernel System call instruction Kernel to User Return from privileged mode instruction System Call Mechanism Overview Processor mode Switched from user-mode to kernel-mode Switched back when returning to user mode SP User-level SP is saved and a kernel SP is initialised User-level SP restored when returning to user-mode User-level is saved and set to kernel entry point User-level restored when returning to user-level Kernel entry via the designated entry point must be strictly enforced 2 22 System Call Mechanism Overview Registers Set at user-level to indicate system call type and its arguments A convention between applications and the kernel Some registers are preserved at user-level or kernellevel in order to restart user-level execution Depends on language calling convention etc. Result of system call placed in registers when returning to user-level Another convention Why do we need system calls Why not simply jump into the kernel via a function call Function calls do not Change from user to kernel mode and eventually back again Restrict possible entry points to secure locations To prevent entering after any security checks 23 24 4
Steps in Making a System Call The MIPS R2/R3 Before looking at system call mechanics in some detail, we need a basic understanding of the MIPS R3 There are steps in making the system call read (fd, buffer, nbytes) 25 26 MIPS R3 Load/store architecture No instructions that operate on memory except load and store Simple load/stores to/from memory from/to registers Store word: sw r4, (r5) Store contents of r4 in memory using address contained in register r5 Load word: lw r3,(r7) Load contents of memory into r3 using address contained in r7 Delay of one instruction after load before data available in destination register» Must always an instruction between a load from memory and the subsequent use of the register. lw, sw, lb, sb, lh, sh,. MIPS R3 Arithmetic and logical operations are register to register operations E.g., add r3, r2, r No arithmetic operations on memory Example add r3, r2, r r3 = r2 + r Some other instructions add, sub, and, or, xor, sll, srl move r2, r r2 = r 27 28 MIPS R3 All instructions are encoded in 32-bit Some instructions have immediate operands Immediate values are constants encoded in the instruction itself Only 6-bit value Examples Add Immediate: addi r2, r, 248 r2 = r + 248 Load Immediate : li r2, 234 r2 = 234 Example code Simple code example: a = a + lw r4,32(r29) li r5, add r4, r4, r5 sw r4,32(r29) // r29 = stack pointer Offset(Address) 29 3 5
MIPS Registers User-mode accessible registers 32 general purpose registers r hardwired to zero r3 the link register for jumpand-link (JAL) instruction HI/LO 2 * 32-bits for multiply and divide Not directly visible Modified implicitly by jump and branch instructions Branching and Jumping Branching and jumping have a branch delay slot li r2, sw r,(r3) The instruction li r2, 2 following a branch li r2, 3 or jump is always : sw r2, (r3) executed prior to destination of jump j f 3 32 MIPS R3 RISC architecture 5 stage pipeline Instruction partially through pipeline prior to jmp having an effect Jump and Link Instruction JAL is used to implement function calls r3 = +8 Return Address register (RA) is used to return from function call x x4 x8 : x2a jal f nop lw r4,(r6) sw r2, (r3) 33 x38 x3a jr r3 nop 34 Compiler Register Conventions Compiler Register Conventions Given 32 registers, which registers are used for Local variables Argument passing Function call results Stack Pointer 35 36 6
int fact(int n) { int r = ; int i; for (i = ; i < n+; i++) { r = r * i; } return r; } Simple factorial : 88b blez a,3 <fact+x3> 4: 2484 addiu a,a, 8: 243 li v, c: 242 li v, : 438 mult v,v 4: 2463 addiu v,v, 8: 2 mflo v c: nop 2: 464fffc bne v,a,4 <fact+x4> 24: 438 mult v,v 28: 3e8 jr ra 2c: nop 3: 3e8 jr ra 34: 242 li v, Function Stack Frames Each function call allocates a new stack frame for local variables, the return address, previous frame pointer etc. Frame pointer: start of current stack frame Stack pointer: end of current stack frame Example: assume f() calls f2(), which calls f3(). Frame Pointer Stack Pointer Stack f() stack frame 37 38 Function Stack Frames Each function call allocates a new stack frame for local variables, the return address, previous frame pointer etc. Frame pointer: start of current stack frame Stack pointer: end of current stack frame Example: assume f() calls f2(), which calls f3(). Frame Pointer Stack Pointer Stack f() stack frame f2() stack frame Function Stack Frames Each function call allocates a new stack frame for local variables, the return address, previous frame pointer etc. Frame pointer: start of current stack frame Stack pointer: end of current stack frame Example: assume f() calls f2(), which calls f3(). Frame Pointer Stack Pointer Stack f() stack frame f2() stack frame f3() stack frame 39 4 Stack Frame Example Code MIPS calling convention for gcc Args -4 have space reserved for them main () { int i; } i = sixargs(,2,3,4,5,6); int sixargs(int a, int b, int c, int d, int e, int f) { return a + b + c + d + e + f; } 4 42 7
4c <main>: 4c: 27bdffd8 addiu sp,sp,-4 42: afbf24 sw ra,36(sp) 424: afbe2 sw s8,32(sp) 428: 3af2 move s8,sp 42c: 2425 li v,5 43: afa2 sw v,6(sp) 434: 2426 li v,6 438: afa24 sw v,2(sp) 43c: 244 li a, 44: 2452 li a,2 444: 2463 li a2,3 448: c2c jal 4b <sixargs> 44c: 2474 li a3,4 45: afc28 sw v,24(s8) 454: 3ce82 move sp,s8 458: 8fbf24 lw ra,36(sp) 45c: 8fbe2 lw s8,32(sp) 46: 3e8 jr ra 464: 27bd28 addiu sp,sp,4... 4b <sixargs>: 4b: 27bdfff8 addiu sp,sp,-8 4b4: afbe sw s8,(sp) 4b8: 3af2 move s8,sp 4bc: afc48 sw a,8(s8) 4c: afc5c sw a,2(s8) 4c4: afc6 sw a2,6(s8) 4c8: afc74 sw a3,2(s8) 4cc: 8fc38 lw v,8(s8) 4d: 8fc2c lw v,2(s8) 4d4: nop 4d8: 622 addu v,v,v 4dc: 8fc3 lw v,6(s8) 4e: nop 4e4: 432 addu v,v,v 4e8: 8fc34 lw v,2(s8) 4ec: nop 4f: 432 addu v,v,v 4f4: 8fc38 lw v,24(s8) 4f8: nop 43 44 4fc: 432 addu v,v,v 4: 8fc3c lw v,28(s8) 44: nop 48: 432 addu v,v,v 4c: 3ce82 move sp,s8 4: 8fbe lw s8,(sp) 44: 3e8 jr ra 48: 27bd8 addiu sp,sp,8 Coprocessor The processor control registers are located in CP Exception/Interrupt management registers Translation management registers CP is manipulated using mtc (move to) and mfc (move from) instructions mtc/mfc are only accessible in kernel mode. CP CP (floating point) : x3 HI/LO R Rn 45 46 CP Registers c_status Exception Management c_cause of the recent exception c_status Current status of the CPU c_epc Address of the instruction that caused the exception c_badvaddr Address accessed that caused the exception Miscellaneous c_prid Processor Identifier Memory Management c_index c_random c_entryhi c_entrylo c_context More about these later in course For practical purposes, you can ignore most bits Green background is the focus 47 48 8
c_status c_cause IM Individual interrupt mask bits 6 external 2 software KU = kernel = user mode IE = all interrupts masked = interrupts enable Mask determined via IM bits c, p, o = current, previous, old 49 IP Interrupts pending 8 bits indicating current state of interrupt lines CE Coprocessor error Attempt to access disabled Copro. BD If set, the instruction that caused the exception was in a branch delay slot ExcCode The code number of the exception taken 5 Exception Codes Exception Codes 5 52 c_epc Exception Vectors The Exception Program Counter Points to address of where to restart execution after handling the exception or interrupt Example Assume sw r3,(r4) causes a restartable fault exception Aside: We are ignore BD-bit in c_cause which is also used in reality on rare occasions. nop sw r3 (r4) nop C_epc C_cause C_status CP (floating point) : x3 HI/LO R Rn 53 54 9
Simple Exception Walk-through Application User Mode Kernel Mode Interrupt Handler 55 Hardware exception handling x2345678 Let s now walk through an exception Assume an interrupt occurred as the previous instruction completed Note: We are in user mode with interrupts enabled 56 Hardware exception handling x2345678 Instruction address at which to restart after the interrupt is transferred to x2345678 Hardware exception handling x2345678 Kernel Mode is set, and previous mode shifted along Interrupts disabled and previous x2345678 state shifted along 57 58 Hardware exception handling x2345678 Code for the exception placed in. Note Interrupt code = x2345678 59 Hardware exception handling x88 Address of general exception vector placed in x2345678 6
Hardware exception handling x88 CPU is now running in kernel mode at x88, with interrupts disabled All information required to Find out what caused the exception Restart after exception handling is in coprocessor registers Badvaddr x2345678 6 Returning from an exception For now, lets ignore how the exception is actually handled how user-level registers are preserved Let s simply look at how we return from the exception 62 Returning from an exception x8234 This code to return is lw r27, saved_epc nop jr r27 rfe x2345678 Load the contents of which is usually moved earlier to somewhere in memory by the exception handler 63 Returning from an exception x2345678 This code to return is lw r27, saved_epc nop jr r27 rfe x2345678 Store the back in the 64 Returning from an exception x2345678 In the branch delay x2345678 slot, execute a restore from This code to return is exception instruction lw r27, saved_epc nop jr r27 rfe Returning from an exception x2345678 We are now back in the same state we were in when the exception happened x2345678 65 66
MIPS System Calls System calls are invoked via a syscall instruction. The syscall instruction causes an exception and transfers control to the general exception handler A convention (an agreement between the kernel and applications) is required as to how user-level software indicates Which system call is required Where its arguments are Where the result should go 67 OS/6 Systems Calls OS/6 uses the following conventions Arguments are passed and returned via the normal C function calling convention Additionally Reg v contains the system call number On return, reg a3 contains : if success, v contains successful result not : if failure, v has the errno.» v stored in errno» - returned in v 68 Convention for kernel entry ra fp sp gp k k s7 s t9 t a3 a2 a a v v AT zero Args in Preserved Preserved for C calling convention Preserved SysCall No. Success Result ra fp sp gp k k s7 s t9 t a3 a2 a a v v AT zero Convention for kernel exit 69 Seriously lowlevel code follows This code is not for the faint hearted move a,s3 addiu a,sp,6 jal 468c <read> li a2,24 move s,v blez s,494 <docat+x9 7 User-Level System Call Walk Through Calling read() int read(int filehandle, void *buffer, size_t size) Three arguments, one return value Code fragment calling the read function 424: 2622 move a,s3 428: 27a5 addiu a,sp,6 42c: ca3 jal 468c <read> 43: 2464 li a2,24 434: 482 move s,v 438: a6 blez s,494 <docat+x94> Args are loaded, return value is tested Inside the read() syscall function part 468c <read>: 468c: 89 j 464 < syscall> 469: 2425 li v,5 Appropriate registers are preserved Arguments (a-a3), return address (ra), etc. The syscall number (5) is loaded into v Jump (not jump and link) to the common syscall routine 7 72 2
The read() syscall function part 2 Generate a syscall exception 464 < syscall>: 464: c syscall 4644: e5 beqz a3,465c < syscall+xc> 4648: nop 464c: 3c lui at,x 465: ac22 sw v,(at) 4654: 243ffff li v,- 4658: 242ffff li v,- 465c: 3e8 jr ra 466: nop The read() syscall function part 2 Test success, if yes, branch to return from function 464 < syscall>: 464: c syscall 4644: e5 beqz a3,465c < syscall+xc> 4648: nop 464c: 3c lui at,x 465: ac22 sw v,(at) 4654: 243ffff li v,- 4658: 242ffff li v,- 465c: 3e8 jr ra 466: nop 73 74 The read() syscall function part 2 464 < syscall>: If failure, store code 464: c syscall in errno 4644: e5 beqz a3,465c < syscall+xc> 4648: nop 464c: 3c lui at,x 465: ac22 sw v,(at) 4654: 243ffff li v,- 4658: 242ffff li v,- 465c: 3e8 jr ra 466: nop The read() syscall function part 2 464 < syscall>: Set read() result to 464: c syscall - 4644: e5 beqz a3,465c < syscall+xc> 4648: nop 464c: 3c lui at,x 465: ac22 sw v,(at) 4654: 243ffff li v,- 4658: 242ffff li v,- 465c: 3e8 jr ra 466: nop 75 76 The read() syscall function part 2 464 < syscall>: 464: c syscall Return to location after where read() was called 4644: e5 beqz a3,465c < syscall+xc> 4648: nop 464c: 3c lui at,x 465: ac22 sw v,(at) 4654: 243ffff li v,- 4658: 242ffff li v,- 465c: 3e8 jr ra 466: nop Summary From the caller s perspective, the read() system call behaves like a normal function call It preserves the calling convention of the language However, the actual function implements its own convention by agreement with the kernel Our OS/6 example assumes the kernel preserves appropriate registers(s-s8, sp, gp, ra). Most languages have similar libraries that interface with the operating system. 77 78 3
System Calls - Kernel Side OS/6 Exception Handling Things left to do Change to kernel stack Preserve registers by saving to memory (on the kernel stack) Leave saved registers somewhere accessible to Read arguments Store return values Do the read() Restore registers Switch back to user stack Return to application 79 Note: The following code is from the uniprocessor variant of OS6 (v.x). Simpler, but broadly similar. 8 exception: move k, sp /* Save previous stack pointer in k */ mfc k, c_status /* Get status register */ andi k, k, CST_Kup /* Check the we-were-in-user-mode bit */ beq k, $, f /* If clear, from kernel, already have stack */ nop /* delay slot */ /* Coming from user mode - load kernel stack into sp */ la k, curkstack /* get address of "curkstack" */ lw sp, (k) /* get its value */ nop /* delay slot for the load */ Note k, k registers : available for mfc k, c_cause /* Now, load the exception cause. */ j common_exception /* kernel Skip to use common code */ nop /* delay slot */ exception: move k, sp /* Save previous stack pointer in k */ mfc k, c_status /* Get status register */ andi k, k, CST_Kup /* Check the we-were-in-user-mode bit */ beq k, $, f /* If clear, from kernel, already have stack */ nop /* delay slot */ : /* Coming from user mode - load kernel stack into sp */ la k, curkstack /* get address of "curkstack" */ lw sp, (k) /* get its value */ nop /* delay slot for the load */ mfc k, c_cause /* Now, load the exception cause. */ j common_exception /* Skip to common code */ nop /* delay slot */ 8 82 common_exception: /* * At this point: * Interrupts are off. (The processor did this for us.) * k contains the exception cause value. * k contains the old stack pointer. * sp points into the kernel stack. * All other registers are untouched. */ /* * Allocate stack space for 37 words to hold the trap frame, * plus four more words for a minimal argument block. */ addi sp, sp, -64 /* The order here must match mips/include/trapframe.h. */ sw ra, 6(sp) /* dummy for gdb */ sw s8, 56(sp) /* save s8 */ sw sp, 52(sp) /* dummy for gdb */ sw gp, 48(sp) /* save gp */ sw k, 44(sp) /* dummy for gdb */ sw k, 4(sp) /* dummy for gdb */ These six stores are a hack to avoid sw k, 52(sp) /* real saved sp */ confusing GDB nop /* delay slot for store */ You can ignore the details of why and mfc k, c_epc /* Copr. reg 3 == for exception */ sw k, 6(sp) /* real saved */ how 83 84 4
/* The order here must match mips/include/trapframe.h. */ sw ra, 6(sp) /* dummy for gdb */ sw s8, 56(sp) /* save s8 */ sw sp, 52(sp) /* dummy for gdb */ sw gp, 48(sp) /* save gp */ sw k, 44(sp) /* dummy for gdb */ sw k, 4(sp) /* dummy for gdb */ sw k, 52(sp) /* real saved sp */ nop /* delay slot for store */ The real work starts here mfc k, c_epc /* Copr. reg 3 == for exception */ sw k, 6(sp) /* real saved */ 85 sw t9, 36(sp) sw t8, 32(sp) sw s7, 28(sp) sw s6, 24(sp) sw s5, 2(sp) sw s4, 6(sp) sw s3, 2(sp) sw s2, 8(sp) sw s, 4(sp) sw s, (sp) sw t7, 96(sp) sw t6, 92(sp) sw t5, 88(sp) sw t4, 84(sp) sw t3, 8(sp) sw t2, 76(sp) sw t, 72(sp) sw t, 68(sp) sw a3, 64(sp) sw a2, 6(sp) sw a, 56(sp) sw a, 52(sp) sw v, 48(sp) sw v, 44(sp) sw AT, 4(sp) sw ra, 36(sp) Save all the registers on the kernel stack 86 /* * Save special registers. */ mfhi t mflo t sw t, 32(sp) sw t, 28(sp) /* * Save remaining exception context information. */ sw k, 24(sp) /* k was loaded with cause earlier */ mfc t, c_status /* Copr. reg == status */ sw t, 2(sp) mfc t2, c_vaddr /* Copr. reg 8 == faulting vaddr */ sw t2, 6(sp) /* * Pretend to save $ for gdb's benefit. */ sw $, 2(sp) We can now use the other registers (t, t) that we have preserved on the stack 87 /* * Prepare to call mips_trap(struct trapframe *) */ addiu a, sp, 6 /* set argument */ jal mips_trap /* call it */ nop /* delay slot */ Create a pointer to the base of the saved registers and state in the first argument register 88 struct trapframe { }; u_int32_t tf_vaddr; /* vaddr register */ u_int32_t tf_status; /* status register */ u_int32_t tf_cause; /* cause register */ u_int32_t tf_lo; u_int32_t tf_hi; u_int32_t tf_ra;/* Saved register 3 */ u_int32_t tf_at;/* Saved register (AT) */ u_int32_t tf_v;/* Saved register 2 (v) */ u_int32_t tf_v;/* etc. */ u_int32_t tf_a; u_int32_t tf_a; u_int32_t tf_a2; u_int32_t tf_a3; u_int32_t tf_t; u_int32_t tf_t7; u_int32_t tf_s; u_int32_t tf_s7; u_int32_t tf_t8; u_int32_t tf_t9; u_int32_t tf_k;/* dummy (see exception.s comments) */ u_int32_t tf_k;/* dummy */ u_int32_t tf_gp; u_int32_t tf_sp; u_int32_t tf_s8; u_int32_t tf_epc; */ By creating a pointer to here of type struct trapframe *, we can access the user s saved registers as normal variables within C /* coprocessor epc register Kernel Stack epc s8 sp gp k k t9 t8 at ra hi lo cause status vaddr 89 Now we arrive in the C kernel /* * General trap (exception) handling function for mips. * This is called by the assembly-language exception handler once * the trapframe has been set up. */ void mips_trap(struct trapframe *tf) { u_int32_t code, isutlb, iskern; int savespl; /* The trap frame is supposed to be 37 registers long. */ assert(sizeof(struct trapframe)==(37*4)); /* Save the value of curspl, which belongs to the old context. */ savespl = curspl; /* Right now, interrupts should be off. */ curspl = SPL_HIGH; 9 5
What happens next The kernel deals with whatever caused the exception Syscall Interrupt Page fault It potentially modifies the trapframe, etc E.g., Store return code in v, zero in a3 mips_trap eventually returns exception_return: /* 6(sp) no need to restore tf_vaddr */ lw t, 2(sp) /* load status register value into t */ nop /* load delay slot */ mtc t, c_status /* store it back to coprocessor */ /* 24(sp) no need to restore tf_cause */ /* restore special registers */ lw t, 28(sp) lw t, 32(sp) mtlo t mthi t /* load the general registers */ lw ra, 36(sp) lw AT, 4(sp) lw v, 44(sp) lw v, 48(sp) lw a, 52(sp) lw a, 56(sp) lw a2, 6(sp) lw a3, 64(sp) 9 92 lw t, 68(sp) lw t, 72(sp) lw t2, 76(sp) lw t3, 8(sp) lw t4, 84(sp) lw t5, 88(sp) lw t6, 92(sp) lw t7, 96(sp) lw s, (sp) lw s, 4(sp) lw s2, 8(sp) lw s3, 2(sp) lw s4, 6(sp) lw s5, 2(sp) lw s6, 24(sp) lw s7, 28(sp) lw t8, 32(sp) lw t9, 36(sp) /* 4(sp) "saved" k was dummy garbage anyway */ /* 44(sp) "saved" k was dummy garbage anyway */ lw gp, 48(sp) /* restore gp */ /* 52(sp) stack pointer - below */ lw s8, 56(sp) /* restore s8 */ lw k, 6(sp) /* fetch exception return into k */ lw sp, 52(sp) /* fetch saved sp (must be last) */ /* done */ jr k /* jump back */ rfe /* in delay slot */.end common_exception Note again that only k, k have been trashed 93 94 6