Laboratory work in TDDI04 Pintos Assignments 2, 3

Transcription

1 Laboratory work in TDDI04 Pintos Assignments 2, 3 Viacheslav Izosimov viaiz@ida.liu.se Instead of Motivation The deadline for Lab 1 is extended to 1st of February! KERNEL PANIC!!! YOUR PINTOS IMPLEMENATION IS NOT YET READY!

2 Lab 2 System calls Single user program Memory issues Console Lab 2 :: Systems Calls At High Level System calls: communication between the user program and the kernel functions, called from the user program and performed by the kernel computers often use interrupts to accomplish that switch from user code to system code

3 Lab 2:: Systems Calls At High Level user_program.c bool flag; Calling flag = create( file.txt, 1000); corresponding function in the wrapper Calling corresponding exception in Pintos kernel the kernel System call wrapper pintos/src/lib/user/syscall.c bool create (const char *file, unsigned initial_size) { return syscall2 (SYS_CREATE, file, initial_size); User exception handler // Your code here! Return result back to the user program pintos/src/userprog/syscall.c Return result back to the wrapper Lab 2::What to Implement? You will need to implement: create - creates a file. open - opens a file. close - closes a file. read - reads from a file or the console (the keyboard). write - writes to a file or the console (the monitor). halt - halts the processor. exit - Terminates a program and deallocates resources occupied by the program, for example, closes all files opened by the program.

4 Lab 2::Useful Files You have to have a look at: pintos/src/lib/user/syscall[.h.c] the wrapper userprog/syscall[.h.c] Your implementation of system calls!!! threads/interrupt.[h c] important structures!!! lib/syscall-nr.h system call constants filesys/filesys.[h c] Pintos file system implementation examples/lab2test.c the official test program for this lab filesys/file.[h c] useful functions for operations with files. Things which you don t find in filesys, you find here userprog/process.c Look up what is here and tell me Lab 2::How Do I Start? (1) STEP 1 Understand what the user program is Look into examples directory and a couple of the user programs, especially halt.c Look into Makefile in this directory to understand how the user programs are compiled Compile the user programs by gmake STEP 2 Prepare Pintos disk Copy one or two user programs to the disk

5 Lab 2::How Do I Start? (2) STEP 2 (Continuation) Dealing with Pintos disk In userprog/build : pintos-mkdisk fs.dsk 2 This will create a file fs.dsk with a 2MB simulated disk in the directory. Format: pintos --qemu -- -f -q. Copy user programs: pintos --qemu -p programname -- -q with new name pintos --qemu -p programname -a newname -- -q Lab 2::How Do I Start? (3) STEP 2 (Continuation) Dealing with Pintos disk Example: pintos --qemu -p../../examples/halt -a severe_student_program -- -q To copy a file from the disk: pintos --qemu -g programname -- -q or pintos --qemu -g programname -a newname -- -q * -p = put, -g = get To run: pintos --qemu -- run programname Example: pintos --qemu -- run severe_student_program

6 Lab 2::How Do I Start? (4) STEP 3 into userprog/process.c, find setup_stack() *esp = PHYS_BASE; change to *esp = PHYS_BASE - 12; STEP 4 Make sure that you understand the problems which may arise if you access (in the kernel) data stored in the user memory using the pointers provided by the user program as system call arguments. (First, make sure that you understand what this sentence means ) Lab 2::Shutdown Example halt.c #include <syscall.h> int main (void){ halt (); userprog/syscall.c #include "userprog/syscall.h #include <stdio.h> #include <syscall-nr.h> get a name of the system callthread_exit (); SYS_HALT from the stack use power_off() #include "threads/interrupt.h #include "threads/thread.h static void syscall_handler (struct intr_frame *); voidsyscall_init (void) { intr_register_int (0x30, 3, INTR_ON, syscall_handler, "syscall"); static void syscall_handler (struct intr_frame *f UNUSED) printf ("system call!\n");! f->esp stack pointer

7 Lab 2::Shutdown Example static void syscall_handler (struct intr_frame *f) { // printf ("system call!\n"); int syscall_name; syscall_name = *(int *)(f->esp); switch(syscall_name) { case SYS_HALT: printf("halt system call\n"); power_off(); break; default: printf("unknown syscall %d\n", syscall_name); 13 Lab 2::Create Example bool create (const char *file, unsigned initial_size) Example: create( file.txt, 1000); How to get them? Answer: f->esp Hint: note that, in order to get a string, you will need get a void pointer from esp and then get a char pointer to which points that void pointer. The char pointer will point to the first element of the string How to return a value? Answer: f->eax

8 Lab 2::Create Example void SysCall_Create(struct intr_frame *f) { char *file_name; int file_size; printf("create system call\n"); file_name = (char *)*(int *)(f->esp + 4); file_size = *(int *)(f->esp + 8); if (filesys_create(file_name, file_size)) f->eax = true; else f->eax = false; case SYS_CREATE: SysCall_Create(f); break; 15 All the pointers on the variables, which you get from the user program, must be validated!

9 Problem 1: If the pointer above PHYS_BASE, It points to Kernel memory! UNSAFE! 17 Memory Issues in Pintos Kernel VM Physical Memory Kernel process PHYS_BASE User process Page directory If no entry? a)kill user b) Handle page fault

10 Lab 2::Memory Issues in Pintos Kernel VM Physical Memory Kernel process PHYS_BASE Check if the pointer is below User process Page directory Check if the pointer is in the page directory If no entry? a)kill user b) Handle page fault STRUCT THREAD 20

11 Lab 2::Shutdown Security (Safety?) static void syscall_handler (struct intr_frame *f) { int syscall_name; You have to implement validate_user_ptr(f->esp); validate_user_ptr(void *) syscall_name = *(int *)(f->esp); function to check if switch(syscall_name) { the pointer is case SYS_HALT: in the user space and printf("halt system call\n"); in the page directory! power_off(); break; default: printf("unknown syscall %d\n", syscall_name); 21 Lab 2::Create Security void SysCall_Create(struct intr_frame *f) { char *file_name; int file_size; printf("create system call\n"); validate_user_ptr(f->esp + 4) A string can span across several pages in memory! validate_user_ptr_str((void *)*(int *)(f->esp + 4)) file_name = (char *)*(int *)(f->esp + 4); validate_user_ptr(f->esp + 8) file_size = *(int *)(f->esp + 8); if (filesys_create(file_name, file_size)) f->eax = true; else f->eax = false; All pointers of the string have to be valid. You should provide an efficient solution to validate as less string pointers as possible! 22

12 Lab 2::Other System Calls To implement the system calls, which operate with files, look at filesys/filesys.[h c] and filesys/file.[h c] (except the console) Everything is already done there for you! Just call the functions! But don t forget validation of system call parameters! Lab 2::Open and Close int open (const char *file) void close (int fd) Sink when you manage file IDs but not too deep! I suggest using an array of *file elements and return indexes of this array as file IDs 0 and 1 IDs must be always reserved for the console!

13 Lab 2::Read System Call int read (int fd, void *buffer, unsigned size) Reads size bytes from the file open as fd into buffer. Returns the number of bytes actually read (0 at end of file), or -1 if the file could not be read (due to a condition other than end of file). Fd 0 reads from the keyboard using input_getc() (defined in devices/input.h). Lab 2::Write System Call int write (int fd, const void *buffer, unsigned size) Writes size bytes from buffer to the open file fd Returns the number of bytes actually written or -1 if the file could not be written The expected behavior is to write as many bytes as possible up to end-of-file and return the actual number written or -1 if no bytes could be written at all When fd=1 then the system call should write to the console with just one call to putbuf() (check lib/kernel/stdio.h and lib/kernel/console.c)

14 Lab 2::Console The reading from the console should be done in a convenient way: You should implement echo The typing should be permitted until Enter is pressed (ASCII code 13) Do not use the graphical qemu terminal for keyboard input (it behaves strangely), instead use the terminal from which you start pintos Only the requested number of characters passed in a read system call should be stored (for example, 10), the rest should be ignored Do not forget to add NULL character at the end of the string that you have read. Lab 2::Exit System Call void exit (int status) Terminates the current user program, returning status to the kernel. Conventionally, a status of 0 indicates success and nonzero values indicate errors. Remember to free all the resources will be not needed anymore. This system call will be improved in the following labs. Important: Free resource in process_exit() function located in process.c CLOSE ALL OPEN FILES

15 Lab 3:: General Description Lab 3: Execution, termination and synchronization of user programs Handling program arguments Execution of several user programs Termination of a user program Synchronization of shared data structures Wait system call

16 Lab 3::Main Goals Part A Provide synchronization for multiple programs Provide synchronization between programs (the most important part of the lab) Part B Loading program arguments (probably, was the most difficult part of the lab) File synchronization: Not yet addressed. It is a part of Lab 4 Lab 3::Exec, Exit and Wait (1) pid_t exec (const char *cmd_line) Runs the executable whose name is given in cmd_line, passing any given arguments, and returns the new process s program id (pid) Must return pid -1, if the program cannot load or run for any reason (!)

17 Lab 3::Exec, Exit and Wait (2) void exit (int status) Terminates the current user program, returning the exit code status to the kernel. status of 0 indicates success and nonzero values indicate errors Remember to free all the resources that will be not needed anymore. Lab 3::Exec, Exit and Wait (3) int wait (pid t pid) Provides synchronization between user programs. "Parent" process waits until its pid-"child" process dies (if the child is still running) and receives the "child" exit code. If the child has been finished, wait() should return child's exit value without waiting. Seems to be difficult? Not really

18 pid = process ID tid = thread ID Lab 3::Exec Add your implementation of exec() functionalities into process_execute() and process_start() in process.c Your Exec() system call in syscall.c { f-eax = process_execute process_execute() { tid = thread_create generate pid from tid; wait until start_process(); return pid or -1 start_process() { loading DONE! initialization DONE! putting program arguments into stack is a PART B; signal to process_execute pid = -1, if the program cannot load or run for any reason. Use an array or a list to keep track of pid:s. pid might equal tid, because we have only one thread per process. Limit the number of user programs (t.ex. 64 or 128). Lab 3::Exit (1) The most suitable place for Exit() functionalities is in your implementation of systems calls in syscall.c Your Exit() system call in syscall.c { get exit code from user; save the exit code if needed; thread_exit thread_exit() { process_exit process_exit() { clean up program s resources; Exit() must return -1 to the parent program if something is wrong, for example, if the child has caused a memory violation. You should take care of it! Clean up program s resources before the exit! printf("%s: exit(%d)\n", thread-name, thread-exit-value) before any exit. (This is needed for testing purposes.)

19 Lab 3::Exit (2) Lab 3::Wait Once you get pid, just call process_wait() (located in process.c) from Wait() system call: Steps to accomplish wait(): 1.Wait until the exit code of child pid is available 2.Get the exit code and remove it from the system 3.Return the exit code (or -1 if something is wrong)

20 Lab 3::Situations with Wait (1) "Parent" exits without calling wait() while the "child" is still running "Child" exits before the "parent" and: "parent" calls wait() afterwards, or "parent" will exit without calling wait(). "Parent" calls wait() before the "child" exits. +All the situations above under the condition that the child does not exit normally. Lab 3::Situations with Wait (2) "Parent" exits without calling wait() while the "child" is still running Child exit(0) Parent exec(child) Do not store the exit code! exit(0)

21 Lab 3::Situations with Wait (3) "Child" exits before the "parent" and: "parent" calls wait() afterwards Child exit(0) keep the exit value Destroy the exit value! Parent exec(child) wait(child) exit(0) wait() returns child s exit value without waiting Lab 3::Situations with Wait (4) "Child" exits before the "parent" and: "parent" will exit without calling wait(). Child exit(0) keep the exit value Destroy the exit value! Parent exec(child) exit(0) You should keep child s exit value until the parent exits (since the child doesn t know if the parent calls wait() later on)

22 Lab 3::Situations with Wait (5) "Parent" calls wait() before the "child" exits. Child Destroy the exit value! exit(0) Parent exec(child) wait for the child wait(child) exit(0) the parent waits for its child Lab 3::Situations with Wait (6) +All the situations above under the condition that the child does not exit normally. Do it as a homework!

23 Lab 3::To Hit The Target! 1. Parts of the functions accessing shared resources must be thread safe, e.g. employ synchronization techniques such as locks and semaphores. 2. Particularly, access to global objects and data must be synchronized. 3. Only one thread can have access to the console at a time. Other threads must wait until completion of reading/writing. 45 To debug: /* grandfather.c */ #include <syscall.h> #include <stdio.h> int main (void){ int pid1, pid2, pid3; pid1 = exec("parent1"); wait(pid1); pid2 = exec("parent2"); wait(pid2); pid3 = exec("parent3"); wait(pid3); exit(0); Lab 3::Test (2) /* parentx.c */ #include <syscall.h> #include <stdio.h> int main (void){ int i; int pid[10]; for(i = 0; i < 10; i++) { pid[i] = exec("childx"); for(i = 0; i < 10; i++) { wait(pid[i]); exit(0); 46

24 Lab 3::Test (3) /* childx.c */ #include <syscall.h> #include <stdio.h> int main (void){ int i; for(i = 0; i < 20000; i++) { int a = (i * i) + (i * i); int b = i; i = a; a = b; i = b; write(1, PASS Lab X ON Time.\n,20); exit(0); Create 1 grandfather, 3 parents and 3 children in examples directory: grandfather.c parent1.c parent2.c parent3.c child1.c child2.c child3.c Replace X in all the places in the code for parents and children with 1, 2, and 3, respectively. 47 Lab 3::Test (4) Modify Makefile in examples, such that all the programs are compiled: PROGS = parent1 parent2 parent3 child1 \ child2 child3 grandfather parent1_src = parent1.c child1_src = child1.c parent2_src = parent2.c child2_src = child2.c parent3_src = parent3.c child3_src = child3.c grandfather_src = grandfather.c Compile the programs with gmake. 48

25 Lab 3::Test (5) Copy them all on the Pintos disk in userprog (don t forget to create the disk and format it). Start them as pintos --qemu -- run grandfather As the output programs should print: Firstly, all 10 PASS Lab 1 ON Time. Secondly, all 10 PASS Lab 2 ON Time. Thirdly, all 10 PASS Lab 3 ON Time. Then, try to comment some wait() in grandfather and parents to see how it behaves. The order should be destroyed (Don t forget to recompile the programs and copy on the disk again. 49 You may need to format the disk before.) Lab 3::Part B (PB) Task: Load program arguments. GOOD NEWS: You will receive the source code once you are done with Part B BAD NEWS: The code will contain a couple of nasty bugs 50

26 Lab 3::PB::Preparatory Steps (1) Lab 2::STEP 3 into userprog/process.c, find setup_stack() *esp = PHYS_BASE; change to *esp = PHYS_BASE - 12; So that you have *esp = PHYS_BASE; CHANGE IT BACK! 51 Lab 3::PB::Preparatory Steps (2) Before continuing, explain why you have "KERNEL PANIC" after you have removed "-12". What is wrong and why did the program work before? 52

27 Lab 3::PB::Preparatory Steps (3) The user program with arguments should be called with '...' from the Pintos command line: pintos --qemu -- run nasty_program arg1 arg2 arg3 When the user program with arguments is called from exec(), you have to call it like this: exec("nasty_program arg1 arg2 arg3") The implementation of all these things have to be done in start_process() 53 pid = process ID tid = thread ID Lab 3::PB::Exec Add your implementation of exec() functionalities into process_execute() and process_start() in process.c Your Exec() system call in syscall.c { f-eax = process_execute process_execute() { tid = thread_create generate pid from tid; wait until start_process(); return pid or -1 start_process() { loading DONE! initialization DONE! putting program arguments into stack is a PART B; signal to process_execute pid = -1, if the program cannot load or run for any reason. Use an array or a list to keep track of pid:s. pid might equal tid, because we have only one thread per process. Limit the number of user programs (t.ex. 64 or 128).

28 Lab 3::PB::Implementation (1) STEP 1. Parse the string: Use strtok_r(), prototyped in lib/string.h Read comments in lib/string.c or man page (run man strtok_r) Limit the number of arguments (for simplicity) STEP 2. Set up the stack: Necessary details about setting up the stack for this task you can find in Program Startup 55 Details section of Pintos documentation. Lab 3::PB::Implementation (2) nasty_program arg1 arg2 arg3 After parsing: nasty_program, arg1, arg2, arg3 Place the words at the top of the stack Align to 4-byte-words, add 0 s Reference the words through the pointers (pointers should point to the addresses of the words in the stack) Put the pointers to the stack (followed with NULL pointer) 56

29 Lab 3::PB::Implementation (2) Put the address of the first pointer to the stack Put the number of words to the stack (the number of arguments + 1) Put faked return address to the stack (e.g. NULL). This is needed in order to meet x86 conventions for program arguments, even though this return address will not be used. So, what we get if we assume that PHYS_BASE is 0xc Address 0xbffffffb 0xbffffff6 0xbffffff1 0xbfffffe3 0xbfffffe0 0xbfffffdc 0xbfffffd8 0xbfffffd4 0xbfffffd0 0xbfffffcc 0xbfffffc8 0xbfffffc4 0xbfffffc0 Name argv[3][...] argv[2][...] argv[1][...] argv[0][...] word-align argv[4] argv[3] argv[2] argv[1] argv[0] argv argc return address Data arg3\0 arg2\0 arg1\0 nasty_program\ > xbffffffc 0xbffffff8 0xbffffff4 0xbfffffe6 0xbfffffd0 4 0 Type char[5] char[5] char[5] char[14] uint24_t char * char * char * char * char * char ** int 58 void (*) ()

30 Lab 3::Test (1) Now you are ready for a complete check! Run gmake check from userprog/build The following tests should pass: 1) Argument passing when executing: args-none, args-single, args-multiple, args-many, args-dbl-space 2) Different exec-tests: exec-once, exec-arg, exec-multiple, exec-missing, exec-bad-ptr 3) Wait-tests: wait-simple, wait-twice, wait-killed, wait-bad-pid 59 Lab 3::Test (2) Many of the other checks in gmake check belong to the second lab. If any of them fails, then it means that something is wrong with your implementation of Lab 2: sc-bad-sp, sc-bad-arg, sc-boundary, sc-boundary-2, halt, exit, create-normal, create-empty, create-null, create-bad-ptr, create-long, create-exists, create-bound, open-normal, open-missing, open-boundary, open-empty, open-null, open-bad-ptr, open-twice, close-normal, close-stdin, close-stdout, close-bad-fd, read-bad-ptr, read-boundary, read-zero, read-stdout, read-bad-fd, write-normal, write-bad-ptr, write-boundary, write-zero, write-stdin, write-bad-fd 60

31 Lab 2 & Lab 3 Lab 2 13th of February Lab 3 5th of March Discussion about Lab 4 + Repetition of Lab 3 next lesson 61