Part Workbook 9. Managing Processes

Part Workbook 9. Managing Processes

Table of Contents 1. An Introduction to Processes... 5 Discussion... 5 Processes are How Things Get Done... 5 What is a Process?... 5 Viewing Processes with the ps Command... 9 Process Selection... 10 Output Selection... 10 Oddities of the ps Command... 10 Monitoring Processes with the top Command... 11 Monitoring Processes with the gnome-system-monitor Application... 12 Locating processes with the pgrep Command.... 14 Examples... 15 Example 1. Viewing All Processes with the "User Oriented" Format... 15 Example 2. Viewing A User's Processes with the "Long" Format... 15 Example 3. Viewing A Particular Command with the "job oriented" Format... 16 Example 4. Viewing Processes with a Custom Format... 16 Online Exercises... 17 Specification... 17 Deliverables... 18 Questions... 18 2. Process States... 21 Discussion... 21 A Process's Life Cycle... 21 How Processes are Started... 21 The Lineage of Processes (and the pstree Command)... 23 How a Process Dies... 24 The 5 Process States... 24 Viewing Process States... 26 Examples... 26 Example 1. Identifying Process States... 26 Online Exercises... 27 Specification... 27 Deliverables... 28 Questions... 28 3. Process Scheduling: nice and renice... 31 Discussion... 31 Process Scheduling Nomenclature... 31 Process Scheduling, in Essence... 31 Process Priorities... 32 Process Niceness... 32 Changing a Process's Niceness... 32 Using nice to Start a low Priority Command... 33 Using renice to Alter a Running Process... 33 Using top to Renice a Process... 34 Making Processes Greedier... 34 Examples... 35 Example 1. Viewing Priorities... 35 Example 2. Changing Priorities with renice... 35 Online Exercises... 36 Specification... 36 Deliverables... 36 2

Managing Processes Cleaning Up... 36 Questions... 36 4. Sending Signals... 40 Discussion... 40 Signals... 40 Why Are Signals Sent?... 41 Sending Signals: the kill Command... 41 Receiving Signals... 42 Using Signals to Terminate Processes... 42 Alternatives to the kill Command... 43 The pkill Command... 43 The killall Command... 44 The System Monitor... 44 The top Command... 45 Examples... 45 Example 1. Using Signals to Terminate Processes... 45 Example 2. Using Signals to Kill Processes... 46 Example 3. Make it Stop!... 46 Online Exercises... 46 Specification... 46 Deliverables... 47 Questions... 47 5. Job Control... 50 Discussion... 50 Running Commands in the Foreground... 50 Running Commands in the Background as Jobs... 50 Managing Multiple Jobs... 51 Listing Current Jobs with jobs... 51 Killing Jobs... 52 Summary... 53 Examples... 53 Example 1. Deciding to Background a Command Running in the Foreground... 53 Example 2. Using CTRLC to Kill a Background Job... 53 Online Exercises... 54 Specification... 54 Deliverables... 54 Clean Up... 54 Questions... 54 6. Scheduling Delayed Tasks: at... 58 Discussion... 58 Daemons... 58 The atd Daemon... 58 Submitting Jobs with at... 59 Delaying Tasks with batch... 59 Summary of at Commands... 60 Examples... 60 Example 1. Submitting an at Job as a File... 60 Example 2. Examining the at Spool Syntax... 60 Online Exercises... 62 Online Exercise 1. Submitting a job for Delayed Execution... 62 Specification... 62 Deliverables... 62 Questions... 62 7. Scheduling Periodic Tasks: cron... 65 3

Managing Processes Discussion... 65 Performing Periodic Tasks... 65 The cron Service... 65 crontab Syntax... 65 Using the crontab Command... 66 Editing crontab Files in Place... 67 Where does the output go?... 68 Environment Variables and cron... 68 Examples... 69 Example 1. Monitoring a Web Site... 69 Example 2. Monitoring Large Files... 69 Example 3. Running scripts from cron... 70 Example 4. Printing fanmail... 70 Online Exercises... 71 Online Exercise 1. Monitoring Who is on the System.... 71 Specification... 71 Deliverables... 71 Questions... 71 4

Chapter 1. An Introduction to Processes Key Concepts A process is an instance of a running executable, identified by a process id (pid). Because Linux implements virtual memory, every process possesses its own distinct memory context. A process has a uid and a collection of gid as credentials. A process has a filesystem context, including a cwd, a umask, a root directory, and a collection of open files. A process has a scheduling context, including a niceness value. A process has a collection of environment variables. The ps command can be used to examine all currently running processes. The top command can be used to monitor all running processes. Discussion Processes are How Things Get Done Almost anything that happens in a Linux system, happens as a process. If you are viewing this text in a web browser, that browser is running as a process. If you are typing at a bash shell's command line, that shell is running as a process. If you are using the chmod command to change a file's permissions, the chmod command operates as a separate process. Processes are how things get done, and the primary responsibility of the Linux kernel is to provide a place for processes to do their stuff without stepping on each other's toes. Processes are an instance of an executing program. In other operating systems, programs are often large, elaborate, graphical applications that take a noticeably long time to start up. In the Linux (and Unix) world, these types of programs exist as well, but so do a whole class of programs which usually have no counterpart in other operating systems. These programs are designed to be quick to start, specialized in function, and play well with others. On a Linux system, processes running these programs are constantly popping into and out of existence. For example, consider the user maxwell performing the following command line. [maxwell@station maxwell]$ ps aux grep httpd > daemons.$(date +%d%b%y) In the split second that the command line took to execute, no less four than processes (ps, grep, bash, and date) were started, did their thing, and exited. What is a Process? By this point, you could well be tired of hearing the answer: a process in an instance of a running program. Here, however, we provide a more detailed list of the components that constitute a process. Execution Context Every process exists (at least to some extent) within the physical memory of the machine. 5

An Introduction to Processes Because Linux (and Unix) is designed to be a multiuser environment, the memory allocated to a process is protected, and no other process can access it. In its memory, a process loads a copy of its executable instructions, and stores any other dynamic information it is managing. A process also carries parameters associated with how often it gets the opportunity to access the CPU, such as its execution state and its niceness value (more on these soon). I/O Context Every process interacts to some extent with the filesystem in order to read or write information that exists before or will exist after the lifespan of the process. Elements of a process's input/output context include the following. Open File Descriptors Almost every process is reading informatio from or writing informatio to external sources, usually both. In Linux, open file descriptors act as sources or sinks of informatio Processes read informatio from or write informatio to 6

An Introduction to Processes 7 file descriptors which may be connected to regular files, device nodes, network sockets, or even each other as pipes (allowing interproces communic Memory Mapped Files Memory mapped files are files whose contents have been mapped directly into the process's memory. Rather than reading or writing to a file descriptor, the process just accesses the appropriate memory

An Introduction to Processes 8 address. Memory maps are most often used to load a process's executable code, but may also be used for other types of nonsequential access to data. Filesystem Context We have encountere several pieces of informatio related to the filesystem that processes maintain, such as the process's current working directory (for translating relative file references)

and the process's umask (for setting permission on newly created files). 1 An Introduction to Processes Environment Variables Heritage Information Credentials Every process maintains its own list of name-value pairs, referred to as environment variables, or collectively as the process's environment. Processes generally inherit their environment on startup, and may refer to it for information such as the user's preferred language or favorite editor. Every process is identified by a PID, or process id, which it is assigned when it is created. In a later Lesson, we will discover that every process has a clearly defined parent and possibly well defined children. A process's own identity, the identity of its children, and to some extent the identity of its siblings are maintained by the process. Every process runs under the context of a given user (or, more exactly, a given user id), and under the context of a collection of group id's (generally, all of the groups that the user belongs to). These credentials limit what resources a process can access, such as which files it can open or with which other processes it is allowed to communicate. Resource Statistics and Limits Viewing Processes with the ps Command Every process also records statistics to track the extent to which system resources have been utilized, such as its memory size, its number of open files, its amount of CPU time, and others. The amount of many of these resources that a process is allowed to use can also be limited, a concept called resource limits. We have already encountered the ps command many times. Now, we will attempt to familiarize ourselves with a broader selection of the many command line switches associated with it. A quick ps --help will display a summary of over 50 different switches for customizing the ps command's behavior. To complicate matters, different versions of Unix have developed their own versions of the ps command, which do not use the same command line switch conventions. The Linux version of the ps command 9

An Introduction to Processes tries to be as accommodating as possible to people from different Unix backgrounds, and often there are multiple switches for any give option, some of which start with a conventional leading hyphen ( - ), and some of which do not. Process Selection By default, the ps command lists all processes started from a user's terminal. While reasonable when users connected to Unix boxes using serial line terminals, this behavior seems a bit minimalist when every terminal window within an X graphical environment is treated as a separate terminal. The following command line switches can be used to expand (or reduce) the processes which the ps command lists. Table 1.1. Common ps Command Line Switches for Process Selection Switch -A, -e, ax Which Processes are Listed All processes. -C command All instances of command -U, --user, --User user -t, --tty terminal -p, p, --pid N Output Selection All processes belonging to user All processes started from terminal Process with pid N As implied by the initial paragraphs of this Lesson, there are many parameters associated with processes, too many to display in a standard terminal width of 80 columns. The following table lists common command line switches used to select what aspects of a process are listed. Table 1.2. Common ps Command Switches for Output Selection Switch Output Format -f "full" listing -l, l -j, j -o, o, --format str long format jobs format user defined format, using fields specified by str (Available fields for str can be listed with ps L, or by consulting the ps(1) man page.) Additionally, the following switches can be used to modify how the selected information is displayed. Table 1.3. Common ps Command Switches for Output Formatting Switch Output Format -H Show process hierarchy f, --forest Show process hierarchy including ASCII decorations h Do not print header lines -w "wide" output (include longer command names) Oddities of the ps Command The ps command, probably more so than any other command in Linux, has oddities associated with its command line switches. In practice, users tend to experiment until they find combinations that work for them, and then stick to them. For example, the author prefers ps aux for a general purpose listing of all 10

An Introduction to Processes processes, while many people prefer ps -ef. The above tables should provide a reasonable "working set" for the novice. The command line switches tend to fall into two categories, those with the traditional leading hyphen ("Unix98" style options), and those without ("BSD" style options). Often, a given functionality will be represented by one of each. When grouping multiple single letter switches, only switches of the same style can be grouped. For example, ps axf is the same as ps a x f, not ps a x -f. Monitoring Processes with the top Command The ps command displays statistics for specified processes at the instant that the command is run, providing a snapshot of an instance in time. In contrast, the top command is useful for monitoring the general state of affairs of processes on the machine. The top command is intended to be run from within a terminal. It will replace the command line with a table of currently running processes, which updates every few seconds. The following demonstrates a user's screen after running the top command. top - 08:18:11 up 2 days, 46 min, 1 user, load average: 0.00, 0.01, 0.00 Tasks: 111 total, 1 running, 110 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.3%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 1021740k total, 910724k used, 111016k free, 133604k buffers Swap: 2064376k total, 0k used, 2064376k free, 471164k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1652 root 20 0 122m 17m 5328 S 0.0 1.8 0:08.68 Xorg 1782 gdm 20 0 364m 14m 10m S 0.0 1.5 0:12.93 gdm-simple-gree 1765 gdm 20 0 329m 12m 9560 S 0.0 1.2 2:04.38 gnome-settings- 1781 gdm 20 0 276m 11m 9072 S 0.0 1.2 0:00.05 plymouth-log-vi 1780 gdm 20 0 302m 10m 8260 S 0.0 1.1 0:06.56 gnome-power-man 1778 gdm 20 0 275m 9576 7152 S 0.0 0.9 0:01.68 metacity 1740 gdm 20 0 257m 7584 5960 S 0.0 0.7 0:00.16 gnome-session 1779 gdm 20 0 236m 7096 5580 S 0.0 0.7 0:00.01 polkit-gnome-au 1747 gdm 20 0 129m 5172 2084 S 0.0 0.5 0:01.34 gconfd-2 1261 root 20 0 97408 5064 4176 S 0.0 0.5 0:00.11 NetworkManager 1385 haldaemo 20 0 26056 4784 4024 S 0.0 0.5 0:01.95 hald 1565 root 20 0 256m 4532 3308 S 0.0 0.4 0:00.61 abrtd 1764 gdm 20 0 116m 4260 3428 S 0.0 0.4 0:02.40 at-spi-registry 1794 gdm 9-11 402m 4020 3004 S 0.0 0.4 0:00.08 pulseaudio 1785 root 20 0 49924 3952 2772 S 0.0 0.4 0:00.15 polkitd 8054 root 20 0 97636 3648 2844 S 0.0 0.4 0:00.09 sshd 1183 root 20 0 242m 3464 948 S 0.0 0.3 0:00.07 rsyslogd While the command is running, the keyboard is "live". In other words, the top command will respond to single key presses without waiting for a return key. The following table lists some of the more commonly used keys. Table 1.4. Commonly used top Commands Key Press q h or? s space M P F or O Command quit help set the delay between updates (in seconds) update display Sort processes by Memory Size Sort processes by CPU (Processor) Activity Select a sort field 11

An Introduction to Processes Key Press Command < > Move sort field: '<' next col left; '>' next col right u k r Reduce display to processes owned by a specific user Kill a process (send a process a signal) Renice a process The last two command, which either kill or renice a process, use concepts that we will cover in more detail in a later Lesson. Although most often run without command line configuration, top does support the following command line switches. Table 1.5. Command Line Switches for the top Command Switch Effect -d secs Delay secs seconds between refreshes (Default = 5 seconds). -q Refresh as often as possible. -n N Run for N iterations, then exit. -b Run in "batch mode", writing simply as if to a dumb terminal. Monitoring Processes with the gnome-system-monitor Application If running an X server, the GNOME desktop environment provides an application similar in function to top, with the benefits (and drawbacks) of a graphical application. The application can be started from the command line as gnome-system-monitor, or by selecting the System : Administration : System Monitor menu item. Figure 1.1. The GNOME System Monitor 12

An Introduction to Processes Like the top command, the System Monitor displays a list of processes running on the local machine, refreshing the list every few seconds. In its default configuration, the System Monitor provides a much simpler interface: it lists only the processes owned by the user who started the application, and reduces the number of columns to just the process's command, owner, Process ID, and simple measures of the process's Memory and CPU utilization. Processes may be sorted by any one of these fields by simply clicking on the column's title. When right-clicking on a process, a pop-up menu allows the user to perform many of the actions that top allowed, such as renicing or killing a process, though again with a simpler (and not as flexible) interface. Figure 1.2. Right Clicking on a Process in the GNOME System Monitor The System Monitor may be configured by opening the Edit : Preferences menu selection. Within the Preferences dialog, the user may set the update interval (in seconds), and configure many more fields to be displayed. Figure 1.3. Configuring Display Fields in the GNOME System Monitor 13

An Introduction to Processes Locating processes with the pgrep Command. Often, users are trying to locate information about processes identified by the command they are running, or the user who is running them. One technique is to list all processes, and use the grep command to reduce the information. In the following, maxwell first looks for all instances of the sshd daemon, and then for all processes owned by the user maxwell. [maxwell@station maxwell]$ ps aux grep sshd root 1474 0.0 0.1 63944 1072? Ss Jul18 0:00 /usr/sbin/sshd maxwell 14716 0.0 0.0 103240 824 pts/0 S+ 08:51 0:00 grep sshd [maxwell@station maxwell]$ ps aux grep maxwell root 14684 0.0 0.4 163332 4600 pts/0 S 08:51 0:00 su - maxwell maxwell 14691 0.0 0.1 108324 1792 pts/0 S 08:51 0:00 -bash maxwell 14721 13.1 5.8 589760 60220 pts/0 Sl 08:52 0:02 /usr/lib64/firefox-3.6/firefox maxwell 14754 0.2 0.4 132912 5056? S 08:52 0:00 /usr/libexec/gconfd-2 maxwell 14772 0.0 0.1 108052 1048 pts/0 R+ 08:53 0:00 ps aux maxwell 14773 0.0 0.0 103236 820 pts/0 S+ 08:53 0:00 grep maxwell While maxwell can find the information he needs, there are some unpleasant issues. 1. The approach is not exacting. Notice that, in the second search, a su process showed up, not because it was owned by maxwell, but because the word maxwell was one of its arguments. 2. Similarly, the grep command itself usually shows up in the output. 3. The compound command can be awkward to type. In order to address these issues, the pgrep command was created. Named pgrep for obvious reasons, the command allows users to quickly list processes by command name, user, terminal, or group. pgrep [SWITCHES] [PATTERN] Its optional argument, if supplied, is interpreted as an extended regular expression pattern to be matched against command names. The following command line switches may also be used to qualify the search. Table 1.6. Common Command Line Switches for Specifying pgrep Process Selection Criteria Switch Effect -n Select only the newest (most recently started) matching process. -u USER Select processes owned by the user USER. -t TERM Select processes controlled by terminal TERM. In addition, the following command line switches can be use to qualify the output formatting of the command. Table 1.7. Formatting Common Command Line Switches for Specifying pgrep Output Switch Effect -d delimiter Use delimiter to delimit each process ID (by default, a newline is used). -l List process name as well as process ID. For a complete list of switches, consult the pgrep(1) man page. 14

An Introduction to Processes As a quick example, maxwell will repeat his two previous process listings, using the pgrep command. [maxwell@station maxwell]$ pgrep -l sshd 1474 sshd [maxwell@station maxwell]$ pgrep -lu maxwell 14691 bash 14721 firefox 14754 gconfd-2 Examples Viewing All Processes with the "User Oriented" Format In the following transcript, maxwell uses the ps -e u command to list all processes (-e) with the "user oriented" format (u). [maxwell@station maxwell]$ ps -e u USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 1380 76? S Oct12 0:04 init [ root 2 0.0 0.0 0 0? SW Oct12 0:00 [keventd] root 3 0.0 0.0 0 0? SW Oct12 0:00 [kapmd]... root 174 0.0 0.0 0 0? SW Oct12 0:00 [kjournald] root 250 0.0 0.0 1356 4 tty2 S Oct12 0:00 /sbin/mingetty tt root 496 0.0 0.1 2104 448? S Oct12 0:00 /sbin/dhclient -1 root 566 0.0 0.0 1448 160? S Oct12 0:00 syslogd -m 0 root 570 0.0 0.0 1376 164? S Oct12 0:00 klogd -x rpc 588 0.0 0.0 1548 104? S Oct12 0:00 portmap... maxwell 4202 0.0 1.3 57948 3400? S Oct12 0:02 nautilus --no-def maxwell 4204 0.0 0.1 16392 436? S Oct12 0:00 magicdev --sm-cli maxwell 4207 0.0 0.5 16784 1500? S Oct12 0:00 eggcups --sm-clie maxwell 4210 0.0 0.3 11596 988? S Oct12 0:00 pam-panel-icon -- maxwell 4212 0.1 0.8 24464 2152? S Oct12 0:41 /usr/bin/python / root 4213 0.0 0.0 1416 136? S Oct12 0:00 /sbin/pam_timesta maxwell 4220 0.0 0.3 17024 1012? S Oct12 0:00 /usr/libexec/noti maxwell 4293 2.4 1.4 18356 3760? S Oct12 15:28 gnome-system-moni apache 5048 0.0 0.6 18424 1776? S Oct12 0:00 /usr/sbin/httpd... maxwell 13166 0.7 0.5 4304 1392 pts/5 S 07:35 0:00 -bash maxwell 13200 0.0 0.2 2696 744 pts/5 R 07:35 0:00 ps -e u The "user oriented" view displays the user who is running the process, the process id, and a rough estimate of the amount of CPU and memory the process is consuming, as well as the state of the process. (Process states will be discussed in the next Lesson). Viewing A User's Processes with the "Long" Format In the following transcript, maxwell uses the ps -U maxwell l command to list his own processes (-U maxwell) with the "long " format (l). [maxwell@station maxwell]$ ps -U maxwell l F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND 4 515 4132 1062 15 0 18632 4 schedu S? 0:02 /usr/bin/gnom 1 515 4175 4132 15 0 3068 72 schedu S? 0:00 /usr/bin/ssh- 0 515 4180 1 15 0 11384 776 schedu S? 0:00 /usr/libexec/ 0 515 4182 1 15 0 6364 4 schedu S? 0:00 /usr/libexec/ 0 515 4184 1 15 0 17336 4 schedu S? 0:00 gnome-setting 0 515 4193 1 15 0 3728 620 schedu S? 0:00 xscreensaver 0 515 4196 1 15 0 12816 1884 schedu S? 0:08 /usr/bin/meta 0 515 4200 1 15 0 21160 3340 schedu S? 0:05 gnome-panel - 0 515 4202 1 15 0 57948 3192 schedu S? 0:02 nautilus --no 15

An Introduction to Processes 0 515 4204 1 15 0 16392 424 schedu S? 0:00 magicdev --sm 0 515 4207 1 15 0 16784 1348 schedu S? 0:00 eggcups --sm- 0 515 4210 1 15 0 11596 908 schedu S? 0:00 pam-panel-ico 0 515 4212 1 15 0 24464 2152 schedu S? 0:43 /usr/bin/pyth 0 0 4213 4210 15 0 1416 136 schedu S? 0:00 /sbin/pam_tim 0 515 4220 1 15 0 17024 756 schedu S? 0:00 /usr/libexec/ 0 515 4293 1 15 0 18356 3760 schedu S? 15:43 gnome-system- 4 515 13166 13163 15 0 4304 1388 wait4 S pts/5 0:00 -bash 0 515 13201 4193 25 10 6676 2592 schedu SN? 0:00 pulsar -root 0 515 13265 13166 20 0 3140 1188 - R pts/5 0:00 ps -U maxwell The long format focuses on scheduling parameters, such as the priority and niceness of the process, which will be discussed in a later Lesson. Viewing A Particular Command with the "job oriented" Format In the following transcript, maxwell uses the ps -C bash j command to list all instances of the bash command (-C bash) with the "job oriented " format (j). [maxwell@station maxwell]$ ps -C bash j PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND 1184 2311 2311 2311 pts/4 2311 S 2291 0:01 bash 1184 2565 2565 2565 pts/0 2565 S 2291 0:04 bash 1184 2757 2757 2757 pts/2 2757 S 2291 0:00 bash 1184 3024 3024 3024 pts/3 3052 S 2291 0:00 bash 1184 3348 3348 3348 pts/6 3348 S 2291 0:00 bash 1184 6033 6033 6033 pts/5 13414 S 2291 0:00 bash 1184 6534 6534 6534 pts/8 6534 S 2291 0:00 bash 13163 13166 13166 6033 pts/5 13414 S 515 0:00 -bash The job oriented format focuses on parent processes, process groups, session groups, and controlling terminals. Many of these concepts will be discussed in later workbooks. Curious that the parent process of many of these shells seems to be process ID 1184, maxwell goes on to examine that individual process in detail. [maxwell@station maxwell]$ ps u 1184 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND einstein 1184 0.2 3.3 26900 8660? S Oct12 2:51 /usr/bin/gnome-te Apparently, all of the bash shells were started from within a gnome-terminal. Viewing Processes with a Custom Format Curious to see what aspects of a process can be displayed with the ps command, maxwell uses ps L to list all possible headers. [maxwell@station maxwell]$ ps L %cpu %CPU %mem %MEM alarm ALARM args COMMAND blocked BLOCKED bsdstart START bsdtime TIME c C... vsize VSZ vsz VSZ wchan WCHAN 16

An Introduction to Processes Curious about the majflt field, maxwell wants to see how many major memory faults processes have had, and uses the -o command line switch to list the number of faults and the command. [maxwell@station maxwell]$ ps -e -o majflt,cmd MAJFLT CMD 41 init [5] 0 [migration/0] 0 [ksoftirqd/0] 0 [watchdog/0]... 0 auditd 14 python /sbin/audispd 0 syslogd -m 0 0 klogd -x 0 mcstransd 1 portmap 40 /usr/bin/python -E /usr/sbin/setroubleshootd 0 rpc.statd To better organize the output, he sorts (reverse) numerically, and only lists the top 5 processes. [maxwell@station ~]$ ps -e -o majflt,cmd sort -rn head 42 /usr/bin/python /usr/sbin/yum-updatesd 41 init [5] 40 /usr/bin/python -E /usr/sbin/setroubleshootd 24 /usr/sbin/gdm-binary -nodaemon 14 python /sbin/audispd 12 nautilus --no-default-window --sm-client-id default3 8 /usr/bin/python -E /usr/bin/sealert -s 5 /usr/lib/firefox-1.5.0.9/firefox-bin 5 cupsd 5 automount Online Exercises Lab Exercise Specification Objective: View Process Information Estimated Time: 20 mins. In order to have a fixed set of processes to examine, you are to take a snapshot of all current processes on your machine. Use the following sequence of commands to first capture the headers for the columns of the ps aux command into a file called snapshot. Next rerun the ps aux command, stripping the header and sorting the remainder of the output by the size of each process's virtual memory. The sorted list of processes will then be appended to the previously captured header in the file snapshot. It's easier than it sounds. [student@station student]$ ps aux head -1 > snapshot [student@station student]$ ps aux --noheader sort -rn -k4 >> snapshot [student@station student]$ head -5 snapshot USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND einstein 3057 0.2 7.4 97088 18932? S Oct12 2:01 /usr/bin/galeonroot 1063 14.3 6.2 36528 15936? S Oct12 163:48 /usr/x11r6/bin/x einstein 1184 0.2 3.8 27160 9868? S Oct12 3:00 /usr/bin/gnome-t einstein 1164 0.0 3.0 27856 7792? S Oct12 0:11 /usr/bin/python Use your snapshot file to answer the following questions. 17

An Introduction to Processes 1. In the file ~/biggest.pid, store the process ID of the process with the largest size of virtual memory (the VSZ column). 2. Experiment with the cut command, until you can extract the initial single letter column of the STAT Field. It should be filled almost exclusively R 's and S 's, implying that processes are in the Running or Sleeping state. Save this one extracted column (minus the header!) into a file called ~/ pstates.txt. 3. Use the grep command, perhaps with the wc command, to determine how many instances of the program /sbin/mingetty are running. Store the answer as a single number in the file ~/ nmingetty.txt. 4. Use the grep command, perhaps with the wc command, to determine how many processes are running as the user root. Store the answer as a single number in the file ~/nroot.txt. 5. Start the top command in a terminal, and leave it running while you grade your exercise. Do not edit or remove your snapshot file until you have finished grading your exercise. Deliverables 1. 1. The file ~/biggest.pid, which contains the process ID of the process with the largest virtual memory. Questions 2. The file ~/pstates.txt, which contains the extracted column of process states, minus the header line. 3. The file ~/nmingetty.txt, which contains the number of instances of the /sbin/ mingetty program were running on your machine. 4. The file ~/nroot.txt, which contains the number of processes running as the user root on the machine. 5. A running instance of the top command. 1. Which of the following commands can be used to view processes running on a Red Hat Enterprise Linux Machine? a. ps b. top c. gnome-system-monitor d. A and B e. All of the Above 2. Which of the following command lines would list all processes for the user maxwell? a. ps -a maxwell b. ps -k maxwell 18

An Introduction to Processes c. ps -U maxwell d. ps -l maxwell e. None of the above 3. When running the top command, which key is used to sort processes by CPU activity? a. C b. A c. P d. U e. None of the above 4. When running the top command, which key is used to sort processes by Memory size? a. M b. S c. V d. T e. None of the above 5. Which of the following command lines would display a listing for every process on the machine? a. ps -e l b. ps ax f c. ps aux d. ps -A j e. All of the above 6. When using the GNOME System Monitor, how does one add new fields to the display? a. By right clicking in any column title, and choosing "Add New Column". b. By choosing the Edit:Preferences menu selection. c. By pressing the C key. d. By clicking the "More Info" button. e. None of the above 7. When using the GNOME System Monitor, how does one change the order in which processes are sorted? a. By pressing the S key. 19

An Introduction to Processes b. By clicking in the appropriate column title. c. By choosing the Edit:Preferences menu selection. d. By clicking the "Sort By" button. e. None of the above 8. Which of the following commands lists fields available for custom formatting of the ps command's output? a. ps l b. ps --list c. ps V d. ps --columns e. None of the above 9. Which of the following would list all instances of the httpd command? a. ps --cmd httpd b. ps p httpd c. ps -C httpd d. ps -l httpd e. None of the above 10. Which of the following command line switches to the ps command is used to specify custom formatting? a. -o b. -c c. --custom d. -f e. None of the above 20

Chapter 2. Process States Key Concepts In Linux, the first process, /sbin/init, is started by the kernel on bootup. All other processes are the result of a parent process duplicating itself, or forking. A process begins executing a new command through a process called execing. Often, new commands are run by a process (often a shell) first forking, and then execing. This mechanism is referred to as the fork and exec mechanism. Processes can always be found in one of five well defined states: runnable, voluntarily sleeping, involuntarily sleeping, stopped, or zombie. Process ancestry can be viewed with the pstree command. When a process dies, it is the responsibility of the process's parent to collect it's return code and resource usage information. When a parent dies before it's children, the orphaned children are inherited by the first process (usually /sbin/init). Discussion A Process's Life Cycle How Processes are Started In Linux (and Unix), unlike many other operating systems, process creation and command execution are two separate concepts. Though usually a new process is created so that it can run a specified command (such as the bash shell creating a process to run the chmod command), processes can be created without running a new command, and new commands can be executed without creating a new process. Creating a New Process (Forking) New processes are created through a technique called forking. When a process forks, it creates a duplicate of itself. Immediately after a fork, the newly created process (the child) is an almost exact duplicate of the original process (the parent). The child inherits an identical copy of the original process's memory, any open files of the parent, and identical copies of any parameters of the parent, such as the current working directory or umask. About the only difference between the parent and the child is the child's heritage information (the child has a different process ID and a different parent process ID, for starters), and (for the programmers in the audience) the return value of the fork() system call. 21

Process States As a quick aside for any programmers in the audience, a fork is usually implemented using a structure similar to the following. int rc, child_pid; rc = fork(); if (rc == -1) { perror("bad fork"); } else if (rc == 0) { do_child(); } else { child_pid = rc; do_parent(child_pid); } When a process wants to create a new process, it calls the fork() system call (with no arguments). Though only one process enters the fork() call, two processes return from in. For the newly created process (the child), the return value is 0. For the original process (the parent), the return value is the process ID of the child. By branching on this value, the child may now go off to do whatever it was started to do (which often involves exec()ing, see next), and the parent can go on to do its own thing. Executing a New Command (Exec-ing) New commands are run through a technique called execing (short for executing). When execing a new command, the current process wipes and releases most of its resources, and loads a new set of instructions from the command specified in the filesystem. Execution starts with the entry point of the new program. After execing, the new command is still the same process. It has the same process ID, and many of the same parameters (such as its resource utilization, umask, current working directory, and others). It merely forgets its former command, and adopts the new one. Again for any programmers, execs are performed through one of several variants of the execve() system call, such as the execl() library call. rc = execl("chmod", "chmod 755 /etc/passwd"); perror("bad exec"); The process enters the the execl(...) call, specifying the new command to run. If all goes well, the execl(...) call never returns. Instead, execution picks up at the entry point (i.e., main()) of the new program. If for some reason execl(...) does return, it must be an 22

Process States error (such as not being able to locate the command's executable in the filesystem). Combining the Two: Fork and Exec Some programs may fork without execing. Examples include networking daemons, who fork a new child to handle a specific client connection, while the parent goes back to listen for new clients. Other programs might exec without forking. Examples include the login command, which becomes the user's login shell after successfully confirming a user's password. Most often, and for shell's in particular, however, forking and execing go hand and hand. When running a command, the bash shell first forks a new bash shell. The child then execs the appropriate command, while the parent waits for the child to die, and then issues another prompt. The Lineage of Processes (and the pstree Command) Upon booting the system, one of the responsibilities of the Linux kernel is to start the first process (usually /sbin/init). All other processes are started because an already existing process forked. 1 Because every process except the first is created by forking, there exists a well defined lineage of parent child relationships among the processes. The first process started by the kernel starts off the family tree, which can be examined with the pstree command. [maxwell@station maxwell]$ pstree init-+-apmd -atd -automount -battstat-applet... -evolution-execu -evolution-mail -fetchmail -galeon-bin -gconfd-1-2*[gconfd-2] -gdm-binary-+-gdm-binary-+-x `-gnome-session---ssh-agent `-gdm-binary---gnome-session---ssh-agent -2*[gnome-panel] -2*[gnome-settings-] -gnome-system-mo -gnome-terminal-+-3*[bash] -bash---man---sh---sh---less -bash---mutt -bash---su---bash---pstree `-gnome-pty-helpe -gpm -gvim -httpd---11*[httpd] -kapmd -keventd -khubd 1 Linux also includes kernel threads, which in special situations may fork and exec to become user processes, confusing matters. This behavior is not usually found in traditional Unix, however, and is very much the exception instead of the rule. 23

Process States... How a Process Dies Orphans Zombies When a process dies, it either dies normally by electing to exit, or abnormally as the result of receiving a signal. We here discuss a normally exiting process, postponing a discussion of signals until a later Lesson. We have mentioned previously that processes leave behind a status code (also called return value) when they die, in the form of an integer. (Recall the bash shell, which uses the $? variable to store the return value of the previously run command.) When a process exits, all of its resources are freed, except the return code (and some resource utilization accounting information). It is the responsibility of the process's parent to collect this information, and free up the last remaining resources of the dead child. For example, when the bash shell forks and execs the chmod command, it is the parent bash shell's responsibility to collect the return value from the exited chmod command. If it is a parent's responsibility to clean up after their children, what happens if the parent dies before the child does? The child becomes an orphan. One of the special responsibilities of the first process started by the kernel is to "adopt" any orphans. (Notice that in the output of the pstree command, the first process has a disproportionately large number of children. Most of these were adopted as the orphans of other processes). In between the time when a process exits, freeing most of its resources, and the time when its parent collects its return value, freeing the rest of its resources, the child process is in a special state referred to as a Zombie. Every process passes through a transient zombie state. Usually, users need to be looking at just the right time (with the ps command, for example) to witness a zombie. They show up in the list of processes, but take up no memory, no CPU time, or any other system resources. They are just the shadow of a former process, waiting for their parent to come and finish them off. Negligent Parents and Long Lived Zombies Occasionally, parent processes can be negligent. They start child processes, but then never go back to clean up after them. When this happens (usually because of a programmer's error), the child can exit, enter the zombie state, and stay there. This is usually the case when users witness zombie processes using, for example, the ps command. Getting rid of zombies is perhaps the most misunderstood basic Linux (and Unix) concept. Many people will say that there is no way to get rid of them, except by rebooting the machine. Using the clues discussed in this section, can you figure out how to get rid of long lived zombies? You get rid of zombies by getting rid of the negligent parent. When the parent dies (or is killed), the now orphaned zombie gets adopted by the first process, which is almost always /sbin/init. /sbin/init is a very diligent parent, who always cleans up after its children (including adopted orphans). The 5 Process States The previous section discussed how processes are started, and how they die. While processes are alive they are always in one of five process states, which effect how and when they are allowed to have access to the CPU. The following lists each of the five states, along with the conventional letter that is used by the ps, top, and other commands to identify a process's current state. 24

Process States Runnable (R) Voluntary (Interruptible) Sleep (S) Involuntary (Non-interruptible) Sleep (D) Processes in the Runnable state are processes that, if given the opportunity to access the CPU, would take it. More formally, this is know as the Running state, but because only one process may be executing on the CPU at any given time, only one of these processes will actually be "running" at any given instance. Because runnable processes are switched in and out of the CPU so quickly, however, the Linux system gives the appearance that all of the processes are running simultaneously. As the name implies, a process which is in a voluntary sleep elected to be there. Usually, this is a process that has nothing to do until something interesting happens. A classic example is a networking daemon, such as the httpd process that implements a web server. In between requests from a client (web browser), the server has nothing to do, and elects to go to sleep. Another example would be the top command, which lists processes every five seconds. While it is waiting for five seconds to pass, it drops itself into a voluntary sleep. When something that the process in interested in happens (such as a web client makes a request, or a five second timer expires), the sleeping process is kicked back into the Runnable state. Occasionally, two processes try to access the same system resource at the same time. For example, one process attempts to read from a block on a disk while that block is being written to because of another process. In these situations, the kernel forces the process into an involuntary sleep. The process did not elect to sleep, it would prefer to be runnable so it can get things done. When the resource is freed, the kernel will put the process back into the runnable state. Although processes are constantly dropping into and out of involuntary sleeps, they usually do not stay there long. As a result, users do not usually witness processes in an involuntary sleep except on busy systems. Stopped (Suspended) Processes (T) Occasionally, users decide to suspend processes. Suspended processes will not perform any actions until they are restarted by the user. In the bash shell, the CTRL+Z key sequence can be used to suspend a process. In programming, debuggers often suspend 25

Process States the programs the are debugging when certain events happen (such as breakpoints occur). Zombie Processes (Z) Viewing Process States As mentioned above, every dieing process goes through a transient zombie state. Occasionally, however, some get stuck there. Zombie processes have finished executing, and have freed all of their memory and almost all of their resources. Because they are not consuming any resources, they are little more than an annoyance that can show up in process listings. When viewing the output of commands such as ps and top, process states are usually listed under the heading STAT. The process is identified by one of the following letters. Runnable - R Sleeping - S Stopped - T Uninterruptible sleep - D Zombie - Z Examples Identifying Process States [maxwell@station maxwell]$ ps -alx F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND 100 0 1 0 15 0 1344 436 schedu S? 0:06 init 100 500 4248 775 15 0 4136 1412 wait4 S tty3 0:00 -bash 100 500 4292 776 15 0 4144 1420 schedu S tty4 0:00 -bash 004 0 1829 1774 17 0 1472 528 down D pts/3 0:00 updatedb 004 0 1827 1774 16 0 1464 520 - R pts/3 0:00 updatedb 000 500 4333 4292 15 0 7612 2616 do_sig T tty4 0:00 vim proj1_s 000 500 4334 4248 15 0 3612 1052 schedu S tty3 2:57 top 004 501 5486 1220 16 0 0 0 do_exi Z? 0:00 [netstat 000 501 5793 2600 15 0 7708 2816 wait4 S pts/0 0:00 vim c 000 501 5798 5793 16 0 3804 944 wait4 S pts/0 0:00 /bin/bash - 040 501 5799 5798 17 0 3808 1000 wait4 S pts/0 0:00 /bin/bash - 000 501 5800 5799 17 0 3148 1240 - R pts/0 0:00 ps -alx 000 501 5801 5799 17 0 3144 420 pipe_w S pts/0 0:00 tail A (voluntarily) sleeping process. The init process is waiting for something "interesting" to happen, such a newly created orphan to inherit. A (involuntarily) sleeping, or "blocked" process. The updatedb command is competing for some resource on the system, probably with the other instance of the updatedb process just below it. A stopped process. This vim editor has probably been manually suspended with the CTRL+Z key sequence. A zombie process, probably left by a negligent galeon web browser. A runnable process, in this case the running ps command. 26

Process States Online Exercises Lab Exercise Specification Objective: Explore different process states Estimated Time: 10 mins. 1. In order to explore process states, we must create processes which are competing for the same resources. Use a simple text editor to create the following script, store it as ~/clobber_it.sh, and make it executable. [maxwell@station maxwell]$ cat clobber_it.sh #!/bin/bash for i in $(seq 1000); do echo "hello world" > poor_overworked_file done 2. While this script will write to the poor, overworked file 1000 times, it will do it sequentially, and so will never be competing for access to the file. Because we need multiple processes competing over the same resource, create the following script as well. Name it ~/clobber_it_lots.sh, and make it executable. [maxwell@station maxwell]$ cat clobber_it_lots.sh #!/bin/bash for i in $(seq 1000); do./clobber_it.sh & done That should do the trick. 3. In one terminal, run the command top -q. This should run the top command, updating continuously. While top is running, use the U command to limit the display to your own processes (i.e., type in your own username). 4. When we say go, in a separate terminal (either another terminal window in an X graphical environment, or a separate virtual console), run the script really_clobber_it.sh. This will start about 1000 processes on your machine, which will obviously stress the system, but it should be able to handle it. 2 The system may seem sluggish, but with patience, it should still be responsive. If things get unbearable, the script can be canceled with a CTRL+C. We haven't said go yet. 5. While the script is running, observe the terminal with the top command. You should see many processes starting and stopping, as well as many processes in the D (involuntary sleep) state. If you are lucky, you might even catch a zombie. We haven't said go yet. 6. Also while the script is running, in yet another terminal, run the ps aux command, and redirect the output to the file ~/lotsa_processes.txt. Glance at the contents of the script, and make sure that at lest five processes in the D state have been recorded. We still haven't said go yet. 7. Go. 8. After you have created your ~/lotsa_processes.txt file, and feel that you have gotten the point, you can cancel the clobber_it_lots.sh script with a CTRL+C, and quit the top command. 27

Process States Deliverables 1. Questions 1. The file ~/lotsa_processes.txt, which contains the output of the ps aux command, with at least 5 instances of a process in the D (uninterruptible sleep) state. 1. If a process were expecting keyboard input before continuing, which state would it be in? a. voluntary sleep b. involuntary sleep c. runnable d. stopped e. zombie 2. The Apache Web Server uses multiple httpd processes, so it can serve multiple concurrent requests. If hundreds of were to request information stored in the same file concurrently, which state would most of the httpd processes be found in? a. stopped b. voluntary sleep c. zombie d. runnable e. involuntary sleep 3. A physics simulation is performing intensive numerical calculations. What state would the process probably be found in? a. voluntary sleep b. runnable c. involuntary sleep d. stopped e. zombie 4. You have suspended the vi editor with the CTRL+Z key combination. Which state would the process be found in? a. runnable b. voluntary sleep c. involuntary sleep d. stopped 28

Process States e. zombie 5. You are running a complicated application (such as evolution), that uses many subprocesses to perform its various tasks. Occasionally, some of the subprocesses seem to be finished (taking no memory or CPU time), but still show up in the process list. What state are these subprocesses in? a. zombie b. stopped c. runnable d. involuntary sleep e. voluntary sleep 6. While observing a list of processes, your friend is concerned about a few zombie processes which show up. What advice should you give her? a. Your friend should reboot the machine when its convenient, because the zombie processes will slowly begin consuming more and more resources. b. Your friend should not be too concerned. They are not consuming resources, and will go away when their parent process dies. Besides, they're fun to talk about. c. Your friend should reboot the machine immediately, before the zombie processes begin infecting other processes, turning them into zombies as well. What a ghoulish nightmare that would be. d. Your friend should log out and log back on again to get rid of the zombies. e. None of the above suggestions apply. 7. What does Linux (and Unix) call the act of creating a new process? a. spawning b. forking c. launching d. execing e. None of the above 8. What does Linux (and Unix) call the act of executing a new command? a. spawning b. forking c. launching d. execing e. None of the above 9. Which letter does ps and top use to represent the involuntary sleep state? 29

Process States a. I b. T c. D d. Z e. None of the above 10. Which letter does ps and top use to represent the stopped (suspended) state? a. I b. T c. D d. Z e. None of the above 30

Chapter 3. Process Scheduling: nice and renice Key Concepts A primary task of the Linux kernel is scheduling processes. Every process has a niceness value that influences its scheduling. The nice and renice commands can change a process's scheduling priority. Discussion Process Scheduling Nomenclature One of the fundamental tasks of the Linux kernel is ensure that processes share system resources effectively. One of the most fundamental resources which has to be shared is the CPU. How the kernel decides which process gets to execute on the CPU at which time is known as scheduling. Every process has two values which influence its scheduling. The first is a dynamic value which is constantly being changed by the kernel. The second is a fixed value, which is only occasionally (if ever) explicitly changed by a user. In the open source community, the nomenclature used to describe these two values has been inconsistent (at best), which leads to confusion. As much as possible, this text will try to be consistent with the ps and top command, and refer to the first (dynamic) value as the process's priority, and the second (fixed) value as the process's niceness. Process Scheduling, in Essence Recently, much attention has been focused on methods used by the Linux kernel to implement scheduling, and the technique has varied from kernel release to kernel release. While the following discussion is not correct at a detailed level, it nevertheless conveys the essence of how the Linux kernel schedules processes. In order to more easily illustrate scheduling, maxwell will start four versions of the cat command, running in the background. (Processes can be run in the background by appending an ampersand ( & ), as will be discussed in a later Lesson). The cat commands read from /dev/zero (a pseudo device that acts as an infinite source of binary zeros), and write to /dev/null (a pseudo device which throws away everything that is written to it). [maxwell@station maxwell]$ cat /dev/zero > /dev/null & [1] 6698 [maxwell@station maxwell]$ cat /dev/zero > /dev/null & [2] 6699 [maxwell@station maxwell]$ cat /dev/zero > /dev/null & [3] 6700 [maxwell@station maxwell]$ cat /dev/zero > /dev/null & [4] 6701 How long will these cat commands run? Forever. The user maxwell next monitors the processes on his machine using the top command. top - 07:41:10 up 51 min, 2 users, load average: 3.55, 1.39, 0.51 Tasks: 123 total, 8 running, 115 sleeping, 0 stopped, 0 zombie 31