Laboratory work in TDDB68 Pintos Assignments 2, 3. Viacheslav Izosimov

Transcription

1 Laboratory work in TDDB68 Pintos Assignments 2, 3 Viacheslav Izosimov viaiz@ida.liu.se

2 Timeline Lab 2: System calls Description Pintos stack Pintos memory Pintos file system Halt, Exit Create, Open/Close Read/Write Outline (1)

3 Outline (2) Lab 3: Execution, termination and synchronization of user programs Description Loading executable, starting a user program Wait, queuing mechanism Exit, pass the exit code! Argument passing Description of the 3rd lab will be available by 30th of September ( on viaiz@ida.liu.se if you want to obtain a draft version earlier, only for those who have already passed lab 2).

4 Timeline Pass assignments on time!!! Lab 00, 0 passed Lab 1 20 th of September Lab 2 15 th of October Lab 3 15 th of November Lab 4 10 th of December

5 Lab 2 :: General Description Lab 2: System calls System calls Single user program Memory issues Console

6 Lab 2 :: Systems Calls At High Level System calls: communication between the user program and the kernel functions, called from the user program and performed by the kernel computers often use interrupts to accomplish that switch from user code to system code

7 Lab 2:: Systems Calls At High Level Calling corresponding function in the wrapper user_program.c Calling } corresponding exception in Pintos kernel the kernel bool flag; flag = create( file.txt, 1000); System call wrapper pintos/src/lib/user/syscall.c bool create (const char *file, unsigned initial_size) { return syscall2 (SYS_CREATE, file, initial_size); User exception handler // Your code here! Return result back to the user program pintos/src/userprog/syscall.c Return result back to the wrapper

8 Lab 2::Preparatory Questions (1) 1. What is the idea behind the system calls? Why, for example, the code of the system calls cannot be available simply as a library to user processes? 2. Pintos uses one interrupt (0x30) for all system calls, so there is only one interrupt handler syscall_handler to be called for different system calls. How it is possible to distinguish in syscall_handler which system call it is? Where are the arguments of a system call stored (if there are any) and how can you access them? 3. Where are the user-mode stack and the kernel-mode stack of a process located? How can you address the user-mode stack in the kernel code, particularly, in an interrupt handler? What is the reason of having two stacks instead of one?

9 Lab 2::Preparatory Questions (2) 4. When a user program executes a system call like Open(), an address to a string containing the file name is provided as an argument. In which memory is this string stored, and how can we access it in the kernel code? 5. In some system calls the user passes pointers as arguments, which is supposed to point to some data in the user-space (for instance, a pointer to a string with a file name in Create()). Specify the situations when accessing the data via that pointer can lead to problems. What can be done about it? 6. Assume that in the kernel code you have got a pointer to a data in the user space which potentially spans across few pages. Reconsider the previous question again. Does your solution still works? Improve it, if not. Is the solution efficient?

10 Lab 2::Preparatory Questions (3) 7. Why a user (program) cannot just call functions in filesys directly instead of calling system calls? 8. All (or almost all) operating systems require that user programs first open a file before using (reading or writing from) it and close when they are done. Explain the motivation behind such a requirement. In other words, why cannot we have only read and write system calls in an operating system without open and close? Answer on the preparatory questions before you begin with the lab assignment!!!

11 Lab 2::What to Implement? You will need to implement: create - creates a file. open - opens a file. close - closes a file. read - reads from a file or the console (the keyboard). write - writes to a file or the console (the monitor). halt - halts the processor. exit - Terminates a program and deallocates resources occupied by the program, for example, closes all files opened by the program.

12 Lab 2::Useful Files You have to have a look at: pintos/src/lib/user/syscall[.h.c] the wrapper userprog/syscall[.h.c] Your implementation of system calls!!! threads/interrupt.[h c] important structures!!! lib/syscall-nr.h system call constants filesys/filesys.[h c] Pintos file system implementation examples/lab2test.c the official test program for this lab filesys/file.[h c] useful functions for operations with files. Things which you don t find in filesys, you find here userprog/process.c Look up what is here and tell me

13 Lab 2::How Do I Start? (1) STEP 1 Understand what the user program is Look into examples directory and a couple of the user programs, especially halt.c Look into Makefile in this directory to understand how the user programs are compiled Compile the user programs by gmake STEP 2 Prepare Pintos disk Copy one or two user programs to the disk

14 Lab 2::How Do I Start? (2) STEP 2 (Continuation) Dealing with Pintos disk In userprog/build : mkdisk fs.dsk 2 This will create a file fs.dsk with a 2MB simulated disk in the directory. Format: pintos --qemu -- -f -q. Copy user programs: pintos --qemu -p programname -- -q with new name pintos --qemu -p programname -a newname -- -q

15 Lab 2::How Do I Start? (3) STEP 2 (Continuation) Dealing with Pintos disk Example: pintos --qemu -p../../examples/halt -a severe_student_program -- -q To copy a file from the disk: pintos --qemu -g programname -- -q or pintos --qemu -g programname -a newname -- -q * -p = put, -g = get To run: pintos --qemu -- run programname Example: pintos --qemu -- run severe_student_program

16 Lab 2::How Do I Start? (4) STEP 3 into userprog/process.c, find setup_stack() *esp = PHYS_BASE; change to *esp = PHYS_BASE - 12; STEP 4 Make sure that you understand the problems which may arise if you access (in the kernel) data stored in the user memory using the pointers provided by the user program as system call arguments. (First, make sure that you understand what this sentence means )

17 Lab 2::Shutdown Example halt.c #include <syscall.h> int main (void){ halt (); } userprog/syscall.c #include "userprog/syscall.h #include <stdio.h> #include <syscall-nr.h> #include "threads/interrupt.h #include "threads/thread.h static void syscall_handler (struct intr_frame *); voidsyscall_init (void) { intr_register_int (0x30, 3, INTR_ON, syscall_handler, "syscall"); } static void syscall_handler (struct intr_frame *f UNUSED) printf ("system call!\n"); thread_exit (); }

18 system call! 18

19 Lab 2::Shutdown Example halt.c #include <syscall.h> int main (void){ halt (); } userprog/syscall.c #include "userprog/syscall.h #include <stdio.h> #include <syscall-nr.h> get a name of the system callthread_exit (); SYS_HALT from the stack} use power_off() #include "threads/interrupt.h #include "threads/thread.h static void syscall_handler (struct intr_frame *); voidsyscall_init (void) { intr_register_int (0x30, 3, INTR_ON, syscall_handler, "syscall"); } static void syscall_handler (struct intr_frame *f UNUSED) printf ("system call!\n");! f->esp stack pointer

20 Lab 2::Useful Files for Shutdown You have to have a look at: pintos/src/lib/user/syscall[.h.c] the wrapper userprog/syscall[.h.c] Your implementation of system calls!!! threads/interrupt.[h c] important structures!!! lib/syscall-nr.h system call constants filesys/filesys.[h c] Pintos file system implementation examples/lab2test.c the official test program for this lab filesys/file.[h c] useful functions for operations with files. Things which you don t find in filesys, you find here userprog/process.c Look up what is here and tell me

21 Pintos Stack (1) 21

22 Why 4? Pintos Stack (2) f->esp f->esp + 4 f->esp + 8 f->esp

23 Lab 2::Create Example bool create (const char *file, unsigned initial_size) Example: create( file.txt, 1000); How to get them? Answer: f->esp Hint: note that, in order to get a string, you will need get a void pointer from esp and then get a char pointer to which points that void pointer. The char pointer will point to the first element of the string How to return a value? Answer: f->eax

24 f->esp 24

25 Lab 2::Create Example To implement the system calls, which operate with files, look at filesys/filesys.[h c] and filesys/file.[h c]. Everything is already done there for you! Just call the functions So simple? Not yet Memory issues!!!

26 All the pointers on the variables, which you get from the user program, must be validated!

27 Problem 1: If the pointer above PHYS_BASE, It points to Kernel memory! UNSAFE! 27

28 Memory Issues in Pintos Kernel VM Physical Memory Kernel process PHYS_BASE User process Page directory If no entry? a)kill user b) Handle page fault

29 Lab 2::Memory Issues in Pintos Kernel VM Physical Memory Kernel process PHYS_BASE Check if the pointer is below User process Page directory Check if the pointer is in the page directory If no entry? a)kill user b) Handle page fault

30 STRUCT THREAD 30

31 Lab 2::Other System Calls (1) To implement the system calls, which operate with files, look at filesys/filesys.[h c] and filesys/file.[h c]. Everything is already done there for you! Just call the functions!

32 Lab 2::Other System Calls (2) int open (const char *file) void close (int fd) Sink when you manage file IDs! I suggest using an array of *file elements and return indexes of this array as file IDs 0 and 1 IDs must be always reserved for the console! NO SYNCHRONIZATION YET!!!

33 Lab 2::Other System Calls (3) int read (int fd, void *buffer, unsigned size) Reads size bytes from the file open as fd into buffer. Returns the number of bytes actually read (0 at end of file), or -1 if the file could not be read (due to a condition other than end of file). Fd 0 reads from the keyboard using input_getc() (defined in devices/input.h). NO SYNCHRONIZATION YET!!!

34 Lab 2::Other System Calls (4) int write (int fd, const void *buffer, unsigned size) Writes size bytes from buffer to the open file fd. Returns the number of bytes actually written or -1 if the file could not be written. Writing past end-of-file would normally extend the file, but file growth is not implemented by the basic file system. The expected behavior is to write as many bytes as possible up to end-of-file and return the actual number written or -1 if no bytes could be written at all. When fd=1 then the system call should write to the console. Your code which writes to the console should write all of buffer in one call to putbuf() (check lib/kernel/stdio.h and lib/kernel/console.c), at least as long as size is not bigger than a few hundred bytes. (It is reasonable to break up larger buffers.) Otherwise, lines of text output by different processes may end up interleaved on the console, confusing the user. NO SYNCHRONIZATION YET!!!

35 Lab 2::Other System Calls (5) void exit (int status) Terminates the current user program, returning status to the kernel. Conventionally, a status of 0 indicates success and nonzero values indicate errors. Remember to free all the resources will be not needed anymore. This system call will be improved in the following labs. CLOSE ALL OPEN FILES CLOSE ALL OP

36 Lab 2::Other System Calls (6) Where did I take this piece of code from?

37

38 Lab 3:: General Description Lab 3: Execution, termination and synchronization of user programs Handling program arguments Execution of several user programs Termination of a user program Synchronization of shared data structures Wait system call

39 Lab 3 Description of the 3rd lab will be available by 30th of September ( on viaiz@ida.liu.se if you want to obtain a draft version earlier, only for those who have already passed lab 2).

40 Lab 3::Preparatory Questions (1) 1. What does wait() is doing? How to call mechanism of using wait/exit? Do we need them in case of one user program? What is the child and what is the parent process? 2. What is the default value, which is returned by exit. Do you know any typical error exit codes that are returned by the user programs in Linux/Windows? 3. Name which data structures in your implementation will be shared? Which synchronization mechanisms do you need to use for them? 4. What are the possible situations with wait() regarding completion of parent and child processes?

41 Lab 3::Preparatory Questions (2) 5. How can you avoid using busy waiting in wait()? 6. What is the page allocation? How is it implemented in Pintos? Why do we need that? What are the advantages vs. continuous memory allocation? 7. When the program starts, what and how should be initialized? E.g. registers, program counter, Look into current Pintos implementation. 8. When you exit, how would you close all open files associated to the current thread? Think about extension of your implementation from Lab 2. Any synchronization problems? 9. How does Pintos handle program arguments?

42 Lab 3::Main Goals Provide synchronization for multiple programs Provide synchronization between programs Add loading program arguments (Part extra) File synchronization: Not yet addressed. It is a part of Lab 4

43 Lab 3::What is already done? Loading several programs into the memory Initialization of user program That was really hard to do in Nachos many students failed. So, you are lucky! However! You must understand how it is done in Pintos before you do anything! Look into pintos/src/userprog/process.c.

44 Lab 3::Exec, Exit and Wait (1) pid_t exec (const char *cmd_line) int wait (pid t pid) void exit (int status) Currently: For exec(), you may want to look at start_process() in process.c It should help. int process_wait (tid_t child_tid UNUSED) { return -1; } Certain functionalities are already in exit() from Lab 2

45 Lab 3::Exec, Exit and Wait (2) pid_t exec (const char *cmd_line) Runs the executable whose name is given in cmd line, passing any given arguments, and returns the new process s program id (pid). Must return pid -1, which otherwise should not be a valid pid, if the program cannot load or run for any reason.

46 Lab 3::Exec, Exit and Wait (3) void exit (int status) Terminates the current user program, returning status to the kernel. If the process s parent waits for it, this is the status that will be returned. a status of 0 indicates success and nonzero values indicate errors.

47 Lab 3::Exec, Exit and Wait (4) int wait (pid t pid) provides synchronization between user programs waits for process pid to die and returns the status it passed to exit. Sounds very simple? Well

48 Lab 3::Exec, Exit and Wait (5) Consider all possible situations with Wait that your program must handle before starting your implementation: "Parent" exits before the "child". "Child" exits before the "parent": "parent" calls Wait; "parent" will exit without calling Wait. "Parent" calls Wait before the "child" exits. 48

49 Lab 3 Commemorative Issues Synchronization is one of the most essential aspects in these assignments. Requirements regarding synchronization are as follows: Parts of the functions accessing shared resources must be thread safe (especially in the system calls), e.g. employ synchronization techniques such as locks and semaphores. Particularly, access to global objects and data must be synchronized. Only one threat can have access to the Console at a time. Other threads must wait 49 until completion of reading/writing.

50 Lab 3::Possible Test To de defined! 50

51 Lab 3::Mandatory extra Part (1) Task: Load program arguments. Lab 2::STEP 3 into userprog/process.c, find setup_stack() *esp = PHYS_BASE; change to *esp = PHYS_BASE - 12; CHANGE IT BACK! 51

52 Lab 3::Mandatory extra Part (2) A user program may be called with arguments in the command line. Implement arguments passing, so the arguments of a user program can be accessible within it. Although you can parse the string from the command line any way you like, we can recommend to take a look at the function strtok_r(), prototyped in lib/string.h and implemented with thorough comments in lib/string.c. You can find more about it by looking at the man page (run man strtok_r at the prompt). Necessary details about setting up the stack for this task you can find in Program Startup Details section of Pintos documentation. 52

53 Lab 3::Mandatory extra Part (3) From Pintos Manual: Consider how to handle arguments for the following example command: /bin/ls -l foo bar. First, break the command into words: /bin/ls, - l, foo, bar. Place the words at the top of the stack. Order doesn't matter, because they will be referenced through pointers. In general very similar to what you are doing when you need to get system call parameters from the stack 53

54 Address Name Data Type 0xbffffffc argv[3][...] bar\0 char[4] 0xbffffff8 argv[2][...] foo\0 char[4] 0xbffffff5 argv[1][...] -l\0 char[3] 0xbfffffed argv[0][...] /bin/ls\0 char[8] 0xbfffffec word-align 0 uint8_t 0xbfffffe8 argv[4] 0 char * 0xbfffffe4 argv[3] 0xbffffffc char * 0xbfffffe0 argv[2] 0xbffffff8 char * 0xbfffffdc argv[1] 0xbffffff5 char * 0xbfffffd8 argv[0] 0xbfffffed char * 0xbfffffd4 argv 0xbfffffd8 char ** 0xbfffffd0 argc 4 int 0xbfffffcc return address 0 54 void (*) ()

55 Lab 2 & Lab 3 Lab 2 15th of October Lab 3 15th of November Discussion about Lab 4 + Repetition of Lab 3 next lesson 55