The Life and Death of A Process

Transcription

1 COMP 111: Operating Systems (Fall 2013) The Life and Death of A Process Noah Mendelsohn Tufts University noah@cs.tufts.edu Web: Based on a presentation by Professor Alva Couch Copyright 2013 Noah Mendelsohn

2 Today How processes are created, managed and terminated Sharing the computer (redux) From file.c to a.out to running image Library routines and shared libraries 2

3 Review: Processes & the Kernel 3

4 Operating systems do two things for us: They make the computer easier to use The facilitate sharing of the computer by multiple programs and users 4

5 actually, Unix & Linux have one more goal: To facilitate running the same program (and OS!) on different types of computer 5

6 The protected OS Kernel Multiple Programs Running at once The operating system is a special, privileged program, with its own code and data. We call the protected, shared part of the OS the kernel. MAIN MEMORY Angry Birds Play Video Browser OPERATING SYSTEM KERNEL CPU

7 We need help from the hardware to protect the kernel! The hardware has memory mapping features that the OS can use to: Hide the kernel from other programs Hide programs from each other Convince each program it s got its own private memory starting at address zero MAIN MEMORY Angry Birds Play Video Browser OPERATING SYSTEM KERNEL CPU

8 Privileged instructions only the OS can use The hardware has special instructions that only the kernel can use to: * Initiate I/O * Set clocks and timers * Control memory mapping The Kernel runs in privileged or kernel or supervisor Angry state. Birds Play Video Browser MAIN MEMORY OPERATING SYSTEM KERNEL CPU Ordinary programs run in user mode. If a user program tries a privileged operation, the hardware will tell the kernel!

9 A process is an instance of a running program Angry Birds Play Video Browser OPERATING SYSTEM KERNEL MEMORY Keyboard, mouse,display CPU Disk Printer

10 A process is an instance of a running program How does this process get started? How does the OS know what code to run? Angry Birds Play Video Browser OPERATING SYSTEM KERNEL MEMORY Keyboard, mouse,display CPU Disk Printer

11 Today How processes are created, managed and terminated Sharing the computer (redux) From file.c to a.out to running image Library routines and shared libraries 11

12 Cloning a Process with fork 12

13 Process creation in Unix/Linux Each process starts life as a clone of its parent Use the fork() system call to create a clone When it s born, each process inherits from its parent Open files (related processes share a file pointer) Environment variables Many other things: e.g. current directory An exact copy of all memory segments from the parent The same code running at the same place, I.E. dropping through the fork! Each copy can tell whether it is parent or child Parent gets process ID as return value from fork Child gets zero 13

14 Example of fork() system call int main(int argc, char *argv[]) { pid_t child_pid; /* child s process id or zero */ fprintf(stderr,"parent: Parent has started\n"); child_pid = fork(); } if (child_pid) { fprintf(stderr, "PARENT: my pid is %d and my parent is %d\n", getpid(), getppid()); wait(child_pid); fprintf(stderr,"parent: my child with pid=%d has died\n", child_pid); } else { fprintf(stderr,"child: my pid is %d and my parent is %d\n", getpid(), getppid()); } Using stderr instead of stdout because it s unbuffered output is never delayed 14

15 Example of fork() system call Print startup message and fork into two processes a parent and a child int main(int argc, char *argv[]) { pid_t child_pid; /* child s process id or zero */ fprintf(stderr,"parent: Parent has started\n"); child_pid = fork(); } if (child_pid) { fprintf(stderr, "PARENT: my pid is %d and my parent is %d\n", getpid(), getppid()); wait(child_pid); fprintf(stderr,"parent: my child with pid=%d has died\n", child_pid); } else { fprintf(stderr,"child: my pid is %d and my parent is %d\n", getpid(), getppid()); } Using stderr instead of stdout because it s unbuffered output is never delayed 15

16 Example of fork() system call int main(int argc, char *argv[]) The parent gets the child s process id..the child { gets zero. pid_t child_pid; /* child s process id or zero */ fprintf(stderr,"parent: Parent has started\n"); child_pid = fork(); } if (child_pid) { fprintf(stderr, "PARENT: my pid is %d and my parent is %d\n", getpid(), getppid()); wait(child_pid); fprintf(stderr,"parent: my child with pid=%d has died\n", child_pid); } else { fprintf(stderr,"child: my pid is %d and my parent is %d\n", getpid(), getppid()); } Using stderr instead of stdout because it s unbuffered output is never delayed 16

17 Example of fork() system call In the parent int main(int argc, char *argv[]) { Print a message pid_t child_pid; /* child s Wait process for the id child or to zero complete */ its work Announce that the child has been reaped fprintf(stderr,"parent: Parent Exit has (drop started\n"); through) child_pid = fork(); } if (child_pid) { fprintf(stderr, "PARENT: my pid is %d and my parent is %d\n", getpid(), getppid()); wait(child_pid); fprintf(stderr,"parent: my child with pid=%d has died\n", child_pid); } else { fprintf(stderr,"child: my pid is %d and my parent is %d\n", getpid(), getppid()); } Using stderr instead of stdout because it s unbuffered output is never delayed 17

18 Example of fork() system call int main(int argc, char *argv[]) { In the child pid_t child_pid; /* child s process id or zero */ Print a message fprintf(stderr,"parent: Parent has started\n"); Exit child_pid = fork(); } if (child_pid) { fprintf(stderr, "PARENT: my pid is %d and my parent is %d\n", getpid(), getppid()); wait(child_pid); fprintf(stderr,"parent: my child with pid=%d has died\n", child_pid); } else { fprintf(stderr,"child: my pid is %d and my parent is %d\n", getpid(), getppid()); } Using stderr instead of stdout because it s unbuffered output is never delayed 18

19 Example of fork() system call int main(int argc, char *argv[]) Remember: { pid_t child_pid; /* child s On a process multi-core id machine, or zero the */ parent and the child may really be running at the same time! fprintf(stderr,"parent: Parent has started\n"); child_pid = fork(); } if (child_pid) { fprintf(stderr, "PARENT: my pid is %d and my parent is %d\n", getpid(), getppid()); wait(child_pid); fprintf(stderr,"parent: my child with pid=%d has died\n", child_pid); } else { fprintf(stderr,"child: my pid is %d and my parent is %d\n", getpid(), getppid()); } Using stderr instead of stdout because it s unbuffered output is never delayed 19

20 Example of fork() system call OUTPUT $ fork1 PARENT: Parent has started int main(int argc, char *argv[]) PARENT: my pid is and my parent { is pid_t child_pid; /* child s process CHILD: my id pid or zero is */ and my parent is fprintf(stderr,"parent: Parent has started\n"); child_pid = fork(); PARENT: my child with pid=26930 has died $ } if (child_pid) { fprintf(stderr, "PARENT: my pid is %d and my parent is %d\n", getpid(), getppid()); wait(child_pid); fprintf(stderr,"parent: my child with pid=%d has died\n", child_pid); } else { fprintf(stderr,"child: my pid is %d and my parent is %d\n", getpid(), getppid()); } Using stderr instead of stdout because it s unbuffered output is never delayed 20

21 Example of fork() system call int main(int argc, char *argv[]) { Question? What s the parent s parent? pid_t child_pid; /* child s process id or zero */ fprintf(stderr,"parent: Parent has started\n"); child_pid = fork(); } if (child_pid) { fprintf(stderr, "PARENT: my pid is %d and my parent is %d\n", getpid(), getppid()); wait(child_pid); fprintf(stderr,"parent: my child with pid=%d has died\n", child_pid); } else { fprintf(stderr,"child: my pid is %d and my parent is %d\n", getpid(), getppid()); } Using stderr instead of stdout because it s unbuffered output is never delayed 21

22 $ ps PID TTY TIME CMD pts/22 00:00:00 tcsh pts/22 00:00:00 ps int main(int argc, char *argv[]) $ { $ pid_t child_pid; /* child s $ fork1 process id or zero */ PARENT: Parent has started fprintf(stderr,"parent: PARENT: Parent has my pid started\n"); is and my parent is CHILD: my pid is and my parent is child_pid = fork(); PARENT: my child with pid=26930 has died $ Most commands have the shell as a parent! Example of fork() system call } if (child_pid) { fprintf(stderr, "PARENT: my pid is %d and my parent is %d\n", getpid(), getppid()); wait(child_pid); fprintf(stderr,"parent: my child with pid=%d has died\n", child_pid); } else { fprintf(stderr,"child: my pid is %d and my parent is %d\n", getpid(), getppid()); } Using stderr instead of stdout because it s unbuffered output is never delayed 22

23 Some things to note about fork Code for parent and child is the same we haven t learned to run another program yet Parent and child run in parallel Child gets a copy of variables changes are not seen by the parent There is a tree of processes rooted at a special system init process, which is always pid=1 every process except init has a parent! Parent must wait for the child to die or it becomes a zombie 23

24 Zombies, process IDs and the process table Every process has an ID (you ve seen that) Inside the kernel, there is a data structure known as a process descriptor for each process that hasn t been reaped by a wait call from the parent On 32 bit Linux systems, there are typically process IDs if they get used up, the system can t make new processes Process IDs and descriptors can t be reused until the parent has waited Therefore: be sure to wait for the death of every process you create! What happens if the parent dies without waiting? Children get inherited by init, which will reap them The big problem is if you keep running and don t reap your children! 24

25 Killing a process Send the process a kill signal to ask/tell it to die How? From another process: kill(victim_pid, SIGKILL) system call From the console: kill -9 victim_pid SIGKILL (integer value -9) is a magic signal that the victim cannot intercept: it will kill the process immediately Other signals are used for many purposes...covered next week The parent still must wait/reap or the result is a zombie the shell will reap processes of commands it launches 25

26 Brief Interlude What s a Signal 26

27 Review: How can your process call the kernel? Your program can use system calls to ask the kernel Angry Birds for service (e.g. read, kill, etc.) Play Video Browser Filesystem, Graphics System, Window system, TCP/IP Networking, etc., etc. OPERATING SYSTEM KERNEL MEMORY Keyboard, mouse,display CPU Disk Printer

28 Signals: How the kernel can call your program! Before signal can be caught, child must issue to identify the handler function: The kernel can cause a preset signal handler in your program to run this alerts your program to some news from the kernel Parent signal(signame, handler_function) If no handler, OS supplies default behavior Child OPERATING SYSTEM KERNEL MEMORY CPU

29 Signal handling example /* sigalarm.c */ #include <stdio.h> #include <signal.h> typedef enum { false, true } bool; timeisup is the signal handler int sleeping = false; void timeisup(int sig) { fprintf(stderr, "Quit bothering me, I was sleeping!\n", sig); sleeping = false; } main() { signal(sigalrm,timeisup); alarm(5); /* Please wake me in 5 seconds */ sleeping = true; while (sleeping) { printf("zzz...\n"); sleep(1); } Tell the OS to call the timeisup function when the alarm signal arrives. } fprintf(stderr, "Dang, you woke me up!\n"); 29

30 Signal handling example /* sigalarm.c */ #include <stdio.h> #include <signal.h> typedef enum { false, true } bool; int sleeping = false; void timeisup(int sig) { fprintf(stderr, "Quit bothering me, I was sleeping!\n", sig); sleeping = false; } main() { signal(sigalrm,timeisup); alarm(5); /* Please wake me in 5 seconds */ sleeping = true; while (sleeping) { printf("zzz...\n"); sleep(1); } Tell the OS to send SIGALRM in 5 seconds } fprintf(stderr, "Dang, you woke me up!\n"); 30

31 Signal handling example /* sigalarm.c */ #include <stdio.h> #include <signal.h> typedef enum { false, true } bool; QUESTION: does sleeping ever become false? int sleeping = false; void timeisup(int sig) { fprintf(stderr, "Quit bothering me, I was sleeping!\n", sig); sleeping = false; } main() { signal(sigalrm,timeisup); alarm(5); /* Please wake me in 5 seconds */ sleeping = true; while (sleeping) { printf("zzz...\n"); sleep(1); } Loop printing zzz } fprintf(stderr, "Dang, you woke me up!\n"); 31

32 Signal handling example /* sigalarm.c */ #include <stdio.h> YES!! #include When <signal.h> the signal arrives! typedef enum { false, true } bool; QUESTION: does sleeping ever become false? int sleeping = false; void timeisup(int sig) { fprintf(stderr, "Quit bothering me, I was sleeping!\n", sig); sleeping = false; } main() { signal(sigalrm,timeisup); alarm(5); /* Please wake me in 5 seconds */ sleeping = true; while (sleeping) { printf("zzz...\n"); sleep(1); } } fprintf(stderr, "Dang, you woke me up!\n"); 32

33 Signal handling example /* sigalarm.c */ #include <stdio.h> #include <signal.h> typedef enum { false, true } bool; volatile sigatomic_t sleeping = false; main() { signal(sigalrm,timeisup); alarm(5); /* Please wake me in 5 seconds */ } sleeping = true; while (sleeping) { printf("zzz...\n"); sleep(1); } fprintf(stderr, "Dang, you woke me up!\n"); Advanced topic On most machines, this will work fine if you declare sleeping as an int, however There are two issues in principle: 1) The compiler working on main() needs to know that sleeping could get updated void timeisup(int sig) { fprintf(stderr, "Quit bothering me, I was sleeping!\n", by code it s sig); not seeing (the handler volatile warns it) sleeping = false; } 2) For some data types, updating a value takes multiple instructions, and the alarm could ring while the data is in an inconsistent state. Very unlikely for an int, but sigatomic_t is guaranteed to be updated atomically Glad you asked? Lesson: asynchronous programming is tricky, and OS s do it all the time! 33

34 One process can ask kernel to signal another Parent Child OPERATING SYSTEM KERNEL MEMORY CPU

35 One process can ask kernel to signal another Before signal can be caught, child must issue: kill(sig_xxxx, child_pid) Oddly, kill is used not just for kill signals, but for all signals! Parent signal(sig_xxxx, handler_function) If no handler, OS supplies default behavior Child handler_function()called OPERATING SYSTEM KERNEL MEMORY CPU

36 One process can ask kernel to signal another Before signal can be caught, child must issue: kill(sig_stp, child_pid) Oddly, kill is used not just for kill signals, but for all signals! Parent signal(signame, handler_function) If no handler, OS supplies default behavior Child OPERATING SYSTEM KERNEL By the way, the shell has a kill command you can use to send signals to any of your processes (or other people s MEMORY processes if you have permission). Use man 1 kill for more info on the shell command, and man 2 kill & man 2 signal for the system calls. CPU

37 Background and suspended processes From most shells: emacs myfile.txt shell (parent process) stays busy while Emacs runs emacs myfile.txt & & says: run in background: let shell run while Emacs runs If you start a program in the foreground and want to do something else emacs myfile.txt shell (parent process) stays busy while Emacs runs CTRL-Z: suspend Emacs and let shell run Choices after CTRL-Z: fg puts job back in foreground; bg resumes it in background To find out about running background jobs Run the jobs command Each job is named: %1, %2, etc. You can do things like kill -9 %1 or fg %2 Most of this is implemented with signals (e.g. CTRL-Z sends SIGSTP which by default pauses the process) so you can do this from a program as well as from the shell Summary from Prof. Couch: typing./a.out in the shell is an explicit wait. typing./a.out & in the shell is a background execution. 37

38 Running a new Program with fork and exec 38

39 Using exec to launch a new program Fork creates parallel copies of the same program Exec replaces the code for a process with a brand new program and calls its main function Common idiom: to run a new program fork() /* to create a child process */ exec() /* have the child replace itself with the program to be run */ [ optional: continue to do work in the parent while the child runs ] wait(): /* in the parent for the new program to complete */ Ever wondered where your return values from exit() go? The are available to the parent via: pid = wait(int *child_exit_status) So, the parent can find out if the child returned success or an error 39

40 Examples of fork/exec Example: running a "cat" command in the foreground: Example: running a "cat" command in the foreground with explicit wait: Example: running a "cat" command in the background with implicit wait: Example: running a user-typed command in the foreground without arguments: Example: running a user-typed command in the background without arguments: These examples are from Prof. Couch s lecture 40

41 Some things to watch with exec Read the man page to find out the arguments it takes there are several flavors As with fork, the new program retains: Open files, environment variables, current working directory, owner, etc., etc. All data and variables from the caller are replaced if you have buffered I/O that is be lost Consider the following: main() { printf("this won't get seen at all "); execl("/bin/cat", "cat", "/dev/null", 0); } // prints nothing at all, because // the execl erases the unwritten line buffer! 41

42 Some things to watch with exec Read the man page to find out the arguments it takes there are several flavors As with fork, the new program retains: Open files, environment variables, current working directory, owner, etc., etc. All data and variables from the caller are replaced if you have buffered I/O that will be lost Consider the following: By default, stderr does not buffer it s output as soon as you print it goes out. Try it! } main() { printf("this won't get seen at all "); fprintf(stderr, "this will get seen, because stderr flushes buffers on each write "); execl("/bin/cat", "cat", "/dev/null", 0); /* prints this will get seen, because stderr flushes buffers on each write */ /* the execl erases the unwritten stdout buffer, but the stderr output is already done */ 42

43 Today How processes are created, managed and terminated Sharing the computer (redux) From file.c to a.out to running image Library routines and shared libraries 43

44 Running, Runnable & Waiting Processes 44

45 Sharing the CPU Multiple Programs Running at once CPU is shared can only do one thing at a time* Angry Birds Play Video Browser OPERATING SYSTEM MAIN MEMORY CPU *Modern multi-core CPUs can schedule one process/core at a time

46 Process scheduling (For now, assume we have a simple one core CPU) If processes are ready to run, the OS picks one and runs it The chosen process is in the running state Processes have priority and high priority processes are run more often The others are marked as ready (or runnable) (I.e. they d like to run but need to wait their turn) Some processes are healthy but waiting for something Reasons: sleep(), waiting for I/O, wait(), select(), page fault These processes are in the blocked state and they aren t scheduled until that changes Multicore CPUs: exactly the same, but we can have one running process on each core! Designing process schedulers is an art. The strategy that gives good interactive response on a shared server may not be what you need for a massive database system! 46

47 The five state process model Run queue (processes in line for CPU) Dispatch Admit release New Ready Running Exit Timeout Blocked See: Stallings 7 th Edition Page

48 The five state process model In some OS s, a blocked process can die without running any cleanup. Run queue (processes in line for CPU) Dispatch Admit release New Ready Running Exit Timeout Blocked See: Stallings 7 th Edition Page

49 Sharing Memory 49

50 Sharing Memory All programs share memory Multiple Programs Running at once Angry Birds Play Video Browser OPERATING SYSTEM MAIN MEMORY CPU

51 Memory shortage What if we need more memory than we have? Angry Birds Play Video Browser C compiler Browser Emacs OPERATING SYSTEM MAIN MEMORY CPU

52 Swapping The OS can swap some process memory to disk Angry Birds Play Video Browser C compiler Browser Emacs OPERATING SYSTEM MAIN MEMORY CPU Disk

53 Swapping The OS can swap some process memory to disk Angry Birds Browser Browser Emacs OPERATING SYSTEM MAIN MEMORY CPU C compiler Play Video Disk

54 The five seven state process model Run queue (processes in line for CPU) Dispatch Admit Release New Ready Running Exit Timeout Ready / Suspend Blocked / Suspend Blocked See: Stallings 7 th Edition Page

55 The five seven state process model Run queue (processes in line for CPU) Processes that have been swapped to disk aren t scheduled Dispatch to run even if they re otherwise ready Admit Release New Ready Running Exit Timeout Ready / Suspend Blocked / Suspend Blocked See: Stallings 7 th Edition Page

56 Summary of paging and swapping 30 years ago, systems swapped whole processes Today, page-size chunks of memory are moved and mapped individually A process is often partially resident The scheduler blocks when a particular page that s needed is on disk On our systems, pagesize = 4096 bytes (try command: getconf PAGESIZE ) Special hardware is needed to make this work Real memory pages can be mapped to arbitrary locations in one or more virtual memories The hardware faults (tells the kernel) if a needed page is reference but not mapped Pages can be mapped read-only : fault if write is attempted The system tends to run well if the working sets of pages that programs reference a lot fit together in memory Historical reference: Denning, P.J. (1968), The working set model for program behavior. Communications of the ACM, 5/1968, Volume 11, pp

57 Copy on Write A Classic OS Optimization 57

58 Each process has its own virtual memory CPU MAIN MEMORY OPERATING SYSTEM Angry Birds Play Video Browser argv, environ Stack Call Stack) Heap (malloc d) Static uninitialized Data) Static initialized Data) Text code) 0 Angry Birds Virtual Memory 0xFFF FFFF

59 Each process has its own virtual memory CPU MAIN MEMORY OPERATING SYSTEM Angry Birds Consider what happens when we fork we ll need two complete copies of the whole process memory! Play Video Browser argv, environ Stack Call Stack) Heap (malloc d) Static uninitialized Data) Static initialized Data) Text code) 0 Angry Birds Virtual Memory 0xFFF FFFF

60 Fork needs to copy the virtual memory CPU MAIN MEMORY OPERATING SYSTEM Angry Birds Angry Birds argv, environ Stack Call Stack) Heap (malloc d) Static uninitialized Data) argv, environ Stack Call Stack) Heap (malloc d) Static uninitialized Data) 0xFFF FFFF Play Video Browser Static initialized Data) Text code) Static initialized Data) Text code) 0

61 Each process has its own virtual memory CPU MAIN MEMORY OPERATING SYSTEM Angry Birds argv, environ Stack Call Stack) Can we find a way to make the copy cheap? Angry Birds Heap (malloc d) Yes! Static uninitialized Data) argv, environ Stack Call Stack) Heap (malloc d) Static uninitialized Data) 0xFFF FFFF Play Video Browser Static initialized Data) Text code) Static initialized Data) Text code) 0

62 Each process has its own virtual memory CPU MAIN MEMORY OPERATING SYSTEM Map all the pages in both argv, environ argv, environ VMs to the same actual Angry Birds Angry Birds Stack Call Stack) pages in memory Heap (malloc d) Static uninitialized Data) Stack Call Stack) Heap (malloc d) Static uninitialized Data) 0xFFF FFFF Play Video Browser Static initialized Data) Text code) Static initialized Data) Text code) 0

63 Each process has its own virtual memory CPU MAIN MEMORY OPERATING SYSTEM Map all the pages in both argv, environ argv, environ VMs to the same actual Angry Birds Stack Call Stack) pages in memory and mark them all read only Angry Birds Heap (malloc d) Static uninitialized Data) Stack Call Stack) Heap (malloc d) Static uninitialized Data) 0xFFF FFFF Play Video Browser Static initialized Data) Text code) Static initialized Data) Text code) 0

64 Each process has its own virtual memory CPU MAIN MEMORY OPERATING SYSTEM Map all the pages in both Play Video argv, environ argv, environ VMs to the same actual Angry Birds Browser Stack Call Stack) pages in memory and mark them all read only Angry Birds Heap (malloc d) Static If either process changes (Angry uninitialized Birds Data) data, the OS will get the Static initialized write fault and Data) copy just the updated page! Text code) Stack Call Stack) Heap (malloc d) Static uninitialized Data) Static initialized Data) Text code) 0xFFF FFFF 0

65 Shared executables By the way: the same mapping tricks allow us to share a single copy of each executable (e.g. Emacs, gcc, firefox) This works even if they are launched using unrelated exec calls The system typically loads at most one copy of a given executable any later exec calls just map it! We can also share copies of libraries by using something called shared libraries we ll study those later today 65

66 Summary of copy on write A classic optimization used in many systems Hardware must support read-only mapping at the page level The time to clone a process or other data is small and often independent of the size Some work is necessary to set up the new maps: depends on the hardware Read-only data is never copied (e.g. the program code!) Writeable data is still not copied until it s actually updated 66

67 Sharing executable programs A similar trick is used to share the code for programs We ve seen how that works with fork(), but Even if lots of copies of a program are loaded independently using exec(), there s typically only one copy in memory or swapped to disk This is a huge savings. Think of how many times our Halligan homework server runs: bash, tcsh, gcc, make, emacs, vim There s typically just one loaded copy of each, no matter how many users Launching a new copy is very quick 67

68 Today How processes are created, managed and terminated Sharing the computer (redux) From file.c to a.out to running image Library routines and shared libraries 68

69 How do we get from source to executable program? 69

70 From source code to executable #include <stdio.h> int main(int argc, char *argv[]) { printf( The sum is %d\n sum(1,2)) } two_plus_one.c int sum(int a, int b) { return a+b; } arith.c gcc c arith.c gcc c two_plus_one.c Relocateable object code for sum() Relocateable object code for sum() arith.o two_plus_one.o 70

71 From source code to executable #include <stdio.h> int main(int argc, char *argv[]) { printf( The sum is %d\n sum(1,2)) } two_plus_one.c gcc c two_plus_one.c Relocatable.o files int sum(int a, int b) { return a+b; } Contain machine code References within the file are resolved References to external files not resolved arith.c Some address fields may need adjusting depending on final location in executable gcc c arith.c program Relocateable object code for sum() Relocateable object code for main() arith.o two_plus_one.o 71

72 Linking.o files to create executable gcc actually runs a program named ld to create the executable. Relocateable object code for sum() two_plus_one.o Relocateable object code for sum() arith.o gcc o two_plus_one two_plus_one.o arith.o Executable Program two_plus_one 72

73 Linking.o files to create executable Relocateable object code for sum() two_plus_one.o The executable contains all the code, with references resolved. It is ready to be invoked using the exec_() family of system calls. Relocateable object code for sum() arith.o gcc o two_plus one two_plus_one.o arith.o Executable Program two_plus_one 73

74 Linking.o files to create executable Relocateable object code for sum() two_plus_one.o The default name for an executable is a.out so programmers sometimes informally refer to any executable as an a.out. Relocateable object code for sum() arith.o gcc o two_plus_one two_plus_one.o arith.o Executable Program two_plus_one 74

75 Today How processes are created, managed and terminated Sharing the computer (redux) From file.c to a.out to running image Library routines and shared libraries 75

76 Routines like printf live in libraries. Ooops! Where does printf come from? Relocateable object code for sum() two_plus_one.o Relocateable object code for sum() arith.o gcc o two_plus one two_plus_one.o arith.o Executable Program two_plus_one 76

77 Ooops! Where does printf come from? Relocateable object code for sum() two_plus_one.o Routines like printf live in libraries. These are created with the ar command, which packages up several.o files together into a.a archive or library. You can list the.a along with your separate.o files and ld will pull from it any.o files it needs. Relocateable object code for sum() arith.o gcc o two_plus one two_plus_one.o arith.o Executable Program two_plus_one 77

78 Ooops! Where does printf come from? Relocateable object code for sum() two_plus_one.o Routines like printf live in libraries. These are created with the ar command, which packages up several.o files together into a.a archive or library. You can list the.a along with your separate.o files and ld will pull from it any.o files it needs. Executable Program two_plus_one Relocateable object code for sum() printf used to live in the system library named gcc o two_plus one two_plus_one.o arith.o libc.a, which the compiler links automatically into the executable (so you don t have to list it). arith.o 78

79 Why shared libraries? Problem: if printf is linked from the libc.a, then we get a separate copy in each program that uses printf Idea: what if we could have one copy and use memory mapping to put it into every executable that needs it? Challenges: We can t link it when ld builds the rest of the executable: we can just note we need it The same copy is likely to be mapped at different addresses in different programs 79

80 Why shared libraries? Problem: if printf is linked from the libc.a, then we get a separate copy in each program it s built that uses in to printf the system Idea: what if we could have one copy and use memory mapping to put it into every executable that needs it? Challenges: We ll use printf as an example even though Compile the source with fpic to make a position-independent.o file. We can t link it when ld builds the rest of the executable: we can just note we need it The same copy is likely to be mapped at different addresses in different programs Solution: compiler, linker and OS work together to support shared libraries gcc fpic printf.c generates position-independent code that can load at any address gcc shared o libc.so printf.o xxx.o obj3.o creates shared library gcc o two_plus_one two_plus_one.o arith.o libc.so 80

81 Why shared libraries? Problem: if printf is linked from the libc.a, then we get a separate copy in each program Link that uses printf.o and any other files with Idea: what if we could have one copy and use memory mapping to put it into every executable (.so) that file. needs it? Challenges: the shared option to create a shared library We can t link it when ld builds the rest of the executable: we can just note we need it The same copy is likely to be mapped at different addresses in different programs Solution: compiler, linker and OS work together to support shared libraries gcc fpic printf.c generates position-independent code that can load at any address gcc shared o libc.so printf.o xxx.o obj3.o creates shared library gcc o two_plus_one two_plus_one.o arith.o libc.so 81

82 Why shared libraries? Problem: if printf is linked from the libc.a, then we get a separate copy in each program that uses printf Idea: what if we could have one copy and use memory mapping to put it into every executable that needs it? Challenges: The linker recognizes.so files instead of including the code, it leaves a little stub that tells the OS to find and map the shared copy of the.so file when exec loads the program. (Actually, libc.so is so widely used that it s automatically linked, so you don t need to list it as you would your own.so libraries). We can t link it when ld builds the rest of the executable: we can just note we need it The same copy is likely to be mapped at different addresses in different programs Solution: compiler, linker and OS work together to support shared libraries gcc fpic printf.c generates position-independent code that can load at any address gcc shared o libc.so printf.o xxx.o obj3.o creates shared library gcc o two_plus_one two_plus_one.o arith.o libc.so 82

83 Memory mapping allows sharing of.so libraries CPU MAIN MEMORY OPERATING SYSTEM Play Video Angry Birds Angry Birds Browser argv, environ Stack (Browser Call Stack) Heap (malloc d) libc.so Static uninitialized (Browser Data) Static initialized (Browser Data) Text (Browser code) libc.so (with printf code) shows up at different locations in the two programs argv, environ Stack Call Stack) Heap (malloc d) libc.so??? Static uninitialized Data) Static initialized Data) Text code)

84 Memory mapping allows sharing of.so libraries CPU MAIN MEMORY OPERATING SYSTEM Play Video Angry Birds Angry Birds Browser argv, environ Stack Call Stack) Heap (malloc d) libc.so Static uninitialized (Browser Data) Static initialized (Browser Data) Text (Browser code) Only one copy lives in memory everyone shares it! libc.so argv, environ Stack Call Stack) Heap (malloc d) libc.so??? Static uninitialized Data) Static initialized Data) Text code)

85 Memory mapping allows sharing of.so libraries CPU MAIN MEMORY OPERATING SYSTEM Play Video Angry Birds Angry Birds Browser argv, environ Stack Call Stack) Heap (malloc d) libc.so Static uninitialized (Browser Data) Static initialized (Browser Data) Text (Browser code) Memory mapping hardware can do this Code must be positionindependent! libc.so argv, environ Stack Call Stack) Heap (malloc d) libc.so??? Static uninitialized Data) Static initialized Data) Text code)

86 Wrapup 86

87 Summary of today s topics Processes are cloned using fork To run a new program: fork then exec Kernel-to-program communication: A process calls the kernel using a system call (trap) The kernel calls a process using a signal Processes form a tree, and dead processes must be reaped The OS scheduler chooses high priority processes to run Process memory can be swapped or paged to disk when memory is tight Memory mapping & copy-on-write are used: To make fork quick To save memory by sharing executables and shared libraries across processes 87

88 Thank you! 88