Sperimentazioni I LINUX commands tutorial - Part II A. Garfagnini, M. Mazzocco Università degli studi di Padova 24 Ottobre 2012 Streams and I/O Redirection Pipelines Create, monitor and kill processes
The BASH I/O Streams All the UNIX shells use three standard I/O streams: stdout, the standard output stream which displays output from commands. stderr, the standard error stream which displays error output from streams. stdin, the standard input stream which provides input to commands. Stream File descriptor stdin 0 stdout 1 stderr 2 Input streams provide input to programs, usually from terminals. Output streams print text characters, usually to terminals. Redirecting output There are two ways to redirect output: n> redirects output from file descriptor n to a file. You must have write permission to the file. If the file does not exist, it is created. If the file exists, the existing content is lost without any warning. n >> redirects output from file descriptor n to a file. You must have write permission to the file. If the file does not exist, it is created. If the file exists, the output is appended to the existing file.
Redirecting output streams - examples No redirection $ ls a* z* /bin/ls: z*: No such file or directory a2 a3 a4 a5 a.out* a.ps stderr redirection $ ls a* z* 2>errori.txt a2 a3 a4 a5 a.out* a.ps $ cat errori.txt /bin/ls: z*: No such file or directory stdout redirection $ ls a* z* >stdout.txt /bin/ls: z*: No such file or directory $ cat stdout.txt a2 a3 a4 a5 a.out* a.ps both output streams redirected $ ls a* z* >stdout.txt 2>>errori.txt $ cat errori.txt /bin/ls: z*: No such file or directory /bin/ls: z*: No such file or directory Redirecting both streams to the same file Sometimes both output streams have to be redirected to one file The procedure is often used for automated processes or in background jobs. Three possibilities: 1. command 1> output.log 2> output.log 2. command > output.log 2>&1 3. command &> output.log To append output to a file, replace > with >> Q: How to ignore output streams? A: redirect the appropriate stream to /dev/null Ex: Ignoring error output stream $ ls a* z* 2>/dev/null a2 a3 a4 a5 a.out* a.ps
Input redirection stdin stream can be redirected from a file, using the < operator stdin/stdout redirect example $ ls -1 > list.out Redirect stdout to list.out $ cat list.out a2 a3 $ sort -r < list.out a3 a2 All entries in ascending order Process input from file with sort giving a reverse ordering The cat command The cat command, short for catenate, allows to display the contents of a file on stdout: $ cat list.out... a3 The cat command takes input from stdin if you do not specify a file name; it keeps reading from stdin until the end-of-file. (ctrl-d is used to signal end-of-file). Creating a file with cat $ cat >list2.out Ctrl-d is pressed... a3 $
Input redirection with here-document The here-document is another form of input redirection It uses the << along with a word, such as END for a marker to indicate the end of input. [agarfa]$ cat«end>ex-here.sh > cat «-EOF > apple > EOF > END [agarfa]$ cat ex-here.sh cat «-EOF apple pear EOF [agarfa]$. ex-here.sh apple pear source the script, i.e. run in the current shell context It is often used in script files where there is no other way to indicate which lines should be treated as input. Input redirection summary > file Direct standard output to file < file Take standard input from file >> file Direct standard output to file; append to file if it exists << label Here-document n> file Direct file descriptor n to file n< file Take file descriptor n from file n >> file Direct file descriptor n to file; append to file if it exists n>& Duplicate standard output to file descriptor n n<& Duplicate standard input from file descriptor n n>&m File descriptor n becomes a copy of the output file descriptor n<&m File descriptor n becomes copy of the input file descriptor &>file Directs standard output and standard error to file
Pipelines Text filtering is the process of taking an input stream of text and performing some conversion before sending it to an output stream Filtering is done constructing a pipeline of commands where the output from one command is piped or redirected as input to the next: command1 command2 [ command3...] [agarfa]$ ls *a *z sort /bin/ls: z*: No such file or directory a2 [agarfa]$ ls *a *z 2>&1 sort a2 /bin/ls: z*: No such file or directory N.B. pipelines only pipe stdout to stdin! if stderr has been redirected to stdout, both streams will be piped. One advantage of pipes on LINUX and UNIX is that there is no intermediate file involved: the stdout of the first command is sent directly to the second command. head or tail... and wc While cat allows to concatenate files and print them on the standard output head output the first part of files (10 lines by default) head -n K head -c B prints the first K lines of a file; prints the first B bytes of a file; tail output the last part of files (10 lines by default) tail -n K tail -c B prints the last K lines of a file; prints the last B bytes of a file; wc prints newline, word and byte counts for each file wc -c wc -l wc -w print the byte counts; print the newline counts; print the word counts;
Finding text in files The grep command print lines matching a pattern grep [OPTIONS] PATTERN [FILE...] grep searches the named input FILEs, or standard input if no files are named, for lines containing a match to the given PATTERN. grep -i grep -n input file. ignore case distinctions in both the PATTERN and the input files. prefix each line of output with the 1-based line number within its [agarfa]$ grep iostream *.cxx primo.cxx:#include <iostream> secondo.cxx:#include <iostream> [agarfa]$ grep -n iostream *.cxx primo.cxx:1:#include <iostream> secondo.cxx:1:#include <iostream> [agarfa]$ ls -l grep cxx -rw-r-r- 1 agarfa fisica 100 Oct 21 12:27 primo.cxx -rw-r-r- 1 agarfa fisica 100 Oct 19 22:54 secondo.cxx Foregroud and background jobs When you run a command in a terminal window, you are running it in the foreground. The Bash shell has a suspend key, Ctrl-z. Pressing this key combination, you get the terminal prompt again. The clock is still on your destop but has stopped running. It can be restarted with the fg command fg brings the job right back to the foreground, but you no longer have a shell prompt. The bg command continues running your job in backgorund and giving back the terminal prompt [agarfa]$ xclock -d -update 1 [2]+ Stopped xclock -d -update 1 [agarfa]$ fg %2 xclock -d -update 1 [1]+ Stopped xclock -d -update 1 [agarfa]$ bg %1 [1]+ xclock -d -update 1 &
Using & To start a new job in background, add a & character at the end of the command The message shows a job number and a process id (PID). With the jobs command is possible to find out what jobs are running The -l option prints PIDs and the plus sign (+) beside the job number indicates that is the current job [agarfa]$ xclock -d -update 2 & [1] 29696 [agarfa]$ jobs -l [1]+ 29696 Running xclock -d -update 1 & ps: inspecting process status information The ps command accepts zero or more PIDs as argument and displays the associated process status Using ps with no options will list all processes that have our terminal as their controlling terminal [agarfa]$ jobs -l [1] 29696 [agarfa]$ ps 29696 29696 pts/6 S 0:00 xclock -d -update 1 [agarfa]$ ps PID TTY TIME CMD 27082 pts/6 00:00:00 bash 29696 pts/6 00:00:00 xclock 30170 pts/6 00:00:00 ps
More process status information Several options give control of how much information is displayed with the process status: -f : full -j : jobs -l : long --forest : display the commands in a tree hierarcy [agarfa]$ ps -l F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD 0 S 8266 422 8479 0 80 0-1519 - pts/1 00:00:00 bash 0 R 8266 822 422 0 80 0-607 - pts/1 00:00:00 ps [agarfa]$ ps -j PID PGID SID TTY TIME CMD 2388 2388 27082 pts/6 00:00:00 ps 27082 27082 27082 pts/6 00:00:00 bash [agarfa]$ ps --forest PID TTY TIME CMD 422 pts/1 00:00:00 bash 764 pts/1 00:00:00 \_ ps Displaying other processes The -x option displays processes without a controlling terminal The -e option displays dinformation for every process Many options possible, get a brief summary with ps --help [agarfa]$ ps afx # Use BSD syntax 2? S< 0:00 [kthreadd]... 1? Ss 0:01 /sbin/init... 5852? Ss 0:00 /usr/sbin/gdm 5855? S 0:00 \_ /usr/sbin/gdm 8057 tty7 SLs+ 24:26 \_ /usr/bin/x :0 -br -audit 0 -auth /var/lib/gdm/:0.xauth -nolisten tc 8087? Ss 0:00 \_ /bin/sh /etc/xdg/xfce4/xinitrc -- /etc/x11/xinit/xserverrc 8175? Ss 0:00 \_ /usr/bin/seahorse-agent --execute startxfce4 8191? S 0:50 \_ xscreensaver -no-splash 8196? Sl 0:35 \_ /usr/bin/xfce4-session 8253? S 0:02 \_ /usr/lib/xfce4/[...] socket_i 8479? Ssl 0:27 \_ gnome-terminal 8485? S 0:00 \_ gnome-pty-helper... 26523 pts/4 Ss 0:00 \_ bash
top The top command displays a continuosly updated process list, along with useful summary information the -p option allows to control a single process [agarfa]$ top -p 29696 top - 18:58:50 up 10:32, 7 users, load average: 0.18, 0.28, 0.27 Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie Cpu(s): 9.9%us, 1.8%sy, 0.2%ni, 87.2%id, 0.6%wa, 0.1%hi, 0.3%si, 0. Mem: 2074216k total, 2018076k used, 56140k free, 54776k buffers Swap: 3903784k total, 23580k used, 3880204k free, 1501260k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 29696 alberto 20 0 7692 3400 2808 S 0 0.2 0:03.36 xclock kill To Ctrl-c sequence terminates a running program. The sequence sends a SIGINT or interrupt signal to the process. The kill command sends a signal to a job or process. Type man kill to get a table of all available signals. The SIGTSTP and SIGCONT signals stop and resume a background job [agarfa]$ xcalc & [2] 6080 [agarfa]$ ps 6215 6215 pts/6 S 0:00 xcalc [agarfa]$ kill -s SIGCONT 6215 [agarfa]$ ps 6215 6215 pts/6 S 0:00 xcalc [agarfa$ kill -s SIGTSTP 6215 [2]+ Stopped xcalc [agarfa]$ ps 6215 6215 pts/6 T 0:00 xcalc [agarfa]$ kill -s SIGINT 6215 [agarfa]$ ps 6215 [agarfa]$