CS 25200: Systems Programming Lecture 14: Files, Fork, and Pipes Dr. Jef Turkstra 2018 Dr. Jeffrey A. Turkstra 1
Lecture 14 File table and descriptors Fork and exec Fd manipulation Pipes 2018 Dr. Jeffrey A. Turkstra 2
Shell Final Command Table ls -al aab aaa grep me In:dft Out:flee Err:dft Lexer Parser shell.l shell.y wildcards env vars executor ls -al a* grep me > flee <ls> <-al> <a*> <PIPE> <grep> <me> <GREAT> <flee> Command Table ls -al a* grep me In:dft Out:flee Err:dft 2018 Dr. Jeffrey A. Turkstra 3
File descriptors Open fles, pipes, network sockets are referred to by an integer value called the file descriptor or fd This value is an index into a fle descriptor table maintained in the kernel Cannot be directly manipulated by a process 2018 Dr. Jeffrey A. Turkstra 4
fd File descriptors can be viewed in a number of ways $ lsof $ ls /proc/pid/fd Most processes will have three default fds: 0 stdin, e stdout, and 2 stderr Dictated by POSIX 2018 Dr. Jeffrey A. Turkstra 5
File descriptor table fd flags file pointer 0 1 FD_CLOEXEC... n Open fle object (fle_t) access mode status flags offset 52 reference count 1 i-node pointer O_WRONLY O_APPEND O_SYNC fcntl() can be used to manipulate fd fags and status fags :-) 2018 Dr. Jeffrey A. Turkstra 6
Open fle object Holds most of a fle s state Pointer to an inode (really a vnode) Access mode (O_RDONLY, O_RDWR, O_WRONLY) Status fags (O_ASYNC, O_APPEND, O_NONBLOCK, etc) Ofset where the next read or write operation will commence Reference count similar to inodes 2018 Dr. Jeffrey A. Turkstra 7
open() system call int open(const char *pathname, int flags[, mode_t mode]); Flags includes: Access mode (O_RDONLY, O_WRONLY, O_RDRWR) required File creation fags (O_CLOEXEC, O_CREAT, O_TRUNC, etc) optional File status fags (O_APPEND, O_SYNC, O_NONBLOCK, etc) Mode is your usual fle creation mode 2018 Dr. Jeffrey A. Turkstra 8
close() system call int close(int fd); Decrements the reference count for the appropriate open fle object Object is reclaimed if reference count == 0 Returns -e on error and sets errno Failing to close() fds results in a fle descriptor leak Arguably worse than a memory leak 2018 Dr. Jeffrey A. Turkstra 9
errno When system call wrappers return -e, they usually set a global variable errno. #include <errno.h> 2018 Dr. Jeffrey A. Turkstra 10
fork() system call pid_t fork(void); Wrapper for the clone system call Don t worry about this too much The only way to create a new process in *nix Creates an identical copy of the currently running process Copy-on-write optimization avoids the overhead of duplicating memory 2018 Dr. Jeffrey A. Turkstra 11
fork() it New process is a child of the parent process Open fle descriptor table is copied Open fle objects are shared Reference count is increased by one 2018 Dr. Jeffrey A. Turkstra 12
After fork() Parent fdtable fd flags file pointer 0 1 FD_CLOEXEC... n Child fdtable fd flags file pointer 0 1 FD_CLOEXEC... n Open fle object (fle_t) access mode status flags offset 52 reference count 2 i-node pointer O_WRONLY O_APPEND O_SYNC :-) 2018 Dr. Jeffrey A. Turkstra 13
Shared fle_t We can use our shared fle objects to establish a communication channel between the parent and child Or even among multiple children Note: Solaris doesn t share the fle position This can cause headaches between platforms 2018 Dr. Jeffrey A. Turkstra 14
Executing something else What if what we want to execute is not part of our program? What if it is somewhere else? execve()! 2018 Dr. Jeffrey A. Turkstra 15
execvp() system call int execvp(const char *file, char *const argv[]); Replaces the current process image with a new process image Really wraps execve() execvp() even searches $PATH for the executable, if no path provided Remember, argv must end with a NULL! Successful execve() s never return 2018 Dr. Jeffrey A. Turkstra 16
void main() { // Create a new process int ret = fork(); if (ret == 0) { // Child process: execute ls al const char *argv[3]; argv[0] = ls ; argv[1] = -al ; argv[2] = NULL; execvp(argv[0], argv); // There was an error perror( execvp ); exit(1); } else if (ret < 0) { // There was an error in fork perror( fork ); exit(2); } else { // This is the parent process // ret is the pid of the child // Wait until the child exits waitpid(ret, NULL,0); } // end if exit(0); // No error }// end main 2018 Dr. Jeffrey A. Turkstra 17
Our shell Command::execute() { int ret; for (int i = 0; i < _numberofsimplecommands; i++) { ret = fork(); if (ret == 0) { //child execvp(scom[i]->_args[0], scom[i]->_args); perror( execvp ); exit(1); } else if (ret < 0) { perror( fork ); return; } // Parent shell continue } // for if (!background) { // wait for last process waitpid(ret, NULL); } }// execute 2018 Dr. Jeffrey A. Turkstra 18
dup2() system call int dup2(int oldfd, int newfd); Creates a copy of the fle descriptor using the provided newfd newfd will be silently closed if it is already open! And it s atomic! 2018 Dr. Jeffrey A. Turkstra 19
File descriptor table fd flags file pointer 0 1 FD_CLOEXEC 2 FD_CLOEXEC n Open fle object (fle_t) access mode status flags offset 52 reference count 2 i-node pointer O_WRONLY O_APPEND O_SYNC dup2(2, e); :-) 2018 Dr. Jeffrey A. Turkstra 20
Redirecting stdout int main(int argc,char**argv) { // Create a new file int fd = open( myoutput.txt, O_CREAT O_WRONLY O_TRUNC, 0664); if (fd < 0) { perror( open ); exit(1); } // Redirect stdout to file dup2(fd,1); close(fd); // fd no longer needed. } // Now printf that prints to stdout, will write to // myoutput.txt printf( Hello world\n ); 2018 Dr. Jeffrey A. Turkstra 21
dup() system call int dup(int oldfd); Creates a copy of the fle descriptor using the next available fd Handy if you want to save an fd for some reason Hmmmm 2018 Dr. Jeffrey A. Turkstra 22
pipe() system call int pipe(int pipefd[2], int flags); Creates a unidirectional data channel Two fle descriptors pipefd[0]: read end pipefd[e]: write end There is kernel bufering Flags are optional O_NONBLOCK, O_CLOEXEC, etc Solaris has bidrectional pipes 2018 Dr. Jeffrey A. Turkstra 23
Pipe dream File descriptor table fd flags file pointer... 3 4 n Open fle object (fle_t) access mode status flags offset 0 reference count 1 i-node pointer O_RDONLY int fds[2]; pipe(fds); fdpipe[0] == 3 fdpipe[e] == 4 Open fle object (fle_t) access mode status flags offset 0 reference count 1 i-node pointer O_WRONLY 2018 Dr. Jeffrey A. Turkstra 24 :-)
lsgrep int main(int argc,char**argv) { if (argc < 3) { fprintf(stderr, "usage: lsgrep arg1 arg2\n"); exit(1); } // Strategy: parent does the redirection before fork() save stdin/stdout int tempin = dup(0); int tempout = dup(1); // create pipe int fdpipe[2]; pipe(fdpipe); //redirect stdout for "ls dup2(fdpipe[1],1); close(fdpipe[1]); // fork for "ls int ret= fork(); if (ret==0) { // close file descriptors as soon as are not needed close(fdpipe[0]); char *args[3]; args[0]="ls"; args[1]= -al"; args[2]=null; execvp(args[0], args); // error in execvp perror("execvp"); exit(1); } 2018 Dr. Jeffrey A. Turkstra 25
//redirection for "grep // redirect stdin dup2(fdpipe[0], 0); close(fdpipe[0]); //create outfile int fd=open(argv[2], O_WRONLY O_CREAT O_TRUNC, 0600); if (fd < 0){ perror("open"); exit(1); } //redirect stdout dup2(fd,1); close(fd); // fork for grep ret= fork(); if(ret==0) { char * args[3]; args[0]= grep"; args[1]=argv[1]; args[2]=null; execvp(args[0], args); // error in execvp perror("execvp"); _exit(1); } 2018 Dr. Jeffrey A. Turkstra 26
// Restore stdin/stdout dup2(tempin,0); dup2(tempout,1); close(tempin); close(tempout); // Parent waits for grep process waitpid(ret,null,0); printf( All done!!\n ); } // main 2018 Dr. Jeffrey A. Turkstra 27
Questions? 2018 Dr. Jeffrey A. Turkstra 28