Assignment clarifications How many errors to print? at most 1 per token. Interpretation of white space in { } treat as a valid extension, involving white space characters. Assignment FAQs have been updated.
Shells and Shell Programming
Shells A shell is a command line interpreter that is the interface between the user and the OS. A program launcher of sorts. The shell: analyzes each command determines what actions are to be performed performs the actions Example: wc l file1 > file2
Which shell? sh Bourne shell Most common, other shells are a superset Good for programming csh or tcsh default for command line on CDF C-like syntax Best for interactive use. Not good for programming. bash default on Linux (Bourne again shell) Based on sh, with some csh features. korn written by David Korn Based on sh Some claim best for programming. Commercial product.
Shell Diagram
Shell Startup When a shell is invoked, it does the following: 1. Read a special startup file (usually in home directory) 2. display prompt and wait for command 3. Ctrl-D on its own line terminates shell, otherwise, goto step 2.
Shell startup files used to set shell options, set up environment variables, alias sh executes.profile if it s there. ksh executes.profile if in interactive mode. Executes $ENV (usually $HOME/.kshrc) csh executes.cshrc if it exists. If a login shell, executes.login bash executes.bashrc, if a login shell, executes.bash_profile instead
Executables vs. Built-in Commands Most commands you run are other compiled programs. Found in /bin Example: ls shell locates ls binary in /bin directory and launches it Some are not compiled programs, but built into the shell: cd, echo
Common shell facilities Input-output redirection prog < infile > outfile ls >& outfile ls > outfile 2>&1 Pipelining commands # csh stdout and stderr # sh stdout and stderr send the output from one command to the input of the next. ls -l wc ps aux grep reid sort
Job Control A job is a program whose execution has been initiated by the user. At any moment, a job can be running or suspended. Foreground job: a program which has control of the terminal Background job: runs concurrently with the parent shell and does not take control of the keyboard. Start a job in the background by appending & Commands: ^Z, jobs, fg, bg, kill
File Name Expansion ls *.c rm file[1-6].? cd ~/bin ls ~reid ls *.[^oa] - ^ in csh,! in sh * stands in for 0 or more characters? stands in for exactly one character [1-6] stands in for one of 1, 2, 3, 4, 5, 6 [^oa] stands in for any char except o or a ~/ stands in for your home directory ~reid stands in for reid s home directory
Sequences Enter series of simple commands or pipelines separated by semicolons (;) Commands will be executed in sequence Each process the shell executes has return value shell checks for it conditional execution with &&, && executes next command if return value is 0 (success) executes next command if return value is nonzero (failure) example: g++ myprog.cpp echo compile failed
Scripts A bunch commands stored inside a text file Must add execute permission with chmod provides all the programming essentials: variables, logic, control flow, loops powerful way to automate tasks First line of script should always be #! Pathname Pathname will be used to interpret script If #! Not there, sh (Bourne shell) is used Should provide explicit return value by using exit
Subshells A copy of the current shell (child) Created when: grouped commands with ; are executed. Parent waits until child (subshell) finishes Script is executed. Parent sleeps until script is done A background job is executed. Parent runs concurrently with sub-shell In 1st two cases, parent doesn t wait if jobs are in background
Subshell Cont d Environment space copied from parent Local variables are clean:
Shell Programming (Bourne shell) file starts with #!/bin/sh absolute path to the shell program not the same on every machine. can also write programs interactively by starting a new shell at the command line. Tip: this is a good way to test your shell programs
Example In a file: #! /bin/sh echo Hello World! At the command line: penguin% sh bash$ echo Hello World! Hello World! bash$ exit penguin% eddie% sh $ echo Hello World! Hello World! $ exit eddie%
Commands You can run any program in a shell by calling it as you would on the command line. When you run a program like grep or ls in a shell program, a new process is created. There are also some built-in commands where no new process is created. echo set read exit test shift wait man sh to see all builtins.
Variables Local shell variables vs. environment variables Environment variables are copied to subshell, local shell variables are not. Use export to make local shell variables into environment variables: sh-2.05a$ pet1=fuzzoo sh-2.05a$ pet2=moomoo sh-2.05a$ echo $pet1 $pet2 fuzzoo moomoo sh-2.05a$ export pet1 sh-2.05a$ sh sh-2.05a$ echo $pet1 $pet2 fuzzoo sh-2.05a$ exit sh-2.05a$ echo $pet1 $pet2 fuzzoo moomoo
Predefined Environment Variables $HOME full path of your home directory $PATH list of directories to search for files $MAIL full path of mailbox $USER your user name $SHELL full path of your login shell The initial values of the environment variables are set by looking at the.profile file (Bourne shell) in your home directory.
Other Built-in Variables $$ process id of the current shell $! process id of last background command $0 name of shell script $1...$9 commandline arguments $* list of all commandline args $? return value of the previously executed command $# number of commandline params.
Variable Access $name replaced by value of name ${name} replaced by value of name. Useful if ${name} is followed by alphanumerical characters: verb=play Doesn t work: echo I like $verbing Works: echo I like ${verb}ing Output: I like playing
Command line arguments positional parameters: variables that are assigned according to position in a string Command line arguments are placed in positional parameters: giant #!/bin/sh echo arg1: $1 echo arg2: $2 echo name: $0 echo all: $* $ giant fee fie fo fum arg1: fee arg2: fie name: giant all: fee fie fo fum
Another Example sh-2.05a$ cat > shelldemo.sh #! /bin/sh echo $HOME echo $PATH echo commandline=$* echo 1st arg=$1 echo $? gcc hello.cpp echo $? gcc noexist.cpp echo $? sh-2.05a$ chmod ugo+x shelldemo.sh sh-2.05a$ shelldemo.sh arg1 arg2 arg3 /u/kenxu /u/kenxu/bin/linux:/u/kenxu/bin:/local/bin/x11:/usr/x11r6/bin:/local/bin:/usr/ucb:/bin:/usr/bin:.:/u/kenxu commandline=arg1 arg2 arg3 1st arg=arg1 0 0 gcc: noexist.cpp: No such file or directory gcc: no input files 1
Local Variables local variables spaces matter name=value assignment variables can have a single value or list of values. Single value: bindir= /usr/bin List of values (separated by spaces): searchdirs= ~/tests $HOME/test2.
Example: ($ is the default sh prompt) $ bindir= /usr/bin $ searchdirs= ~/tests $HOME/test2. $ echo $searchdirs ~/tests /u/reid/test2. $ echo $bindir /usr/bin
Quoting Single and double quotes are on the same key. Back quote is often on the same key as ~. Double quotes inhibit wildcard replacement only. Single quotes inhibit wildcard replacement, variable substitution and command substitution. Back quotes cause command substitution. Practice and pay attention.
Quoting example $ echo Today is date Today is date $ echo Today is `date` Today is Thu Sep 19 12:28:55 EST 2002 $ echo Today is `date` Today is Thu Sep 19 12:28:55 EST 2002 $ echo Today is `date` Today is `date` double quotes single quote ` - back quote
Another Quoting Example What do the following statements produce if the current directory contains the following non-executable files? a b c $ echo * $ echo ls * $ echo `ls *` $ echo ls * $ echo ls * $ echo `*` double quotes single quote ` - back quote
More on Quoting Command substitution causes another process to be created. Which is better? What is the difference? src=`ls *.c` or src=*.c
Here Documents Easy way to provide input to commands without using auxiliary files: sh-2.05a$ cat << ENDOFTEXT > type some text and > have cat repleat > it! > ENDOFTEXT type some text and have cat repleat it! sh-2.05a$
Arithmetic Bourne Shell doesn t support arithmetic natively. Use expr command (arithmetic & string) sh-2.05a$ x=1 sh-2.05a$ echo $x 1 sh-2.05a$ x=`expr $x + 2` sh-2.05a$ echo $x 3 sh-2.05a$ echo `expr \( 2 + 4 \) \* 5` 30 sh-2.05a$ echo `expr length "cat"` 3 sh-2.05a$ echo `expr substr "donkey" 4 3` key sh-2.05a$ echo `expr index "donkey" "key"` 4 sh-2.05a$ echo `expr match "smalltalk" '.*lk'` 9 sh-2.05a$ echo `expr match "transputer" '.*lk'` 0 sh-2.05a$ echo `expr \( 4 \> 5 \)` 0 sh-2.05a$ echo `expr \( 4 \> 5 \) \ \( 6 \< 7 \)` 1
Test test is an utility that gives 0 exit status if boolean condition is true and non-zero status otherwise Some versions of Bourne shell has it built in syntax: test expression or [ expression ] sh-2.05a$ if test 2 -eq 3 > then > echo equal > else > echo not equal >fi not equal sh-2.05a$ if [ 2 -eq 2 ]; then echo equal; else echo not equal; fi equal sh-2.05a$
Test Common conditions that test supports: test arguments -d filename Exists as a directory -f filename Exists as a regular file. -r filename Exists as a readable file -w filename Exists as a writable file. -x filename Exists as an executable file. -z string True if empty string str1 = str2 True if str1 equals str2 str1!= str2 True if str1 not equal to str2 int1 -eq int2 True if int1 equals int2 -ne, -gt, -lt, -le, >, <, -a, -o And, or.
Control Statements The general syntax for if is: if command1 then command2 command3...etc elif command4 command5...etc else command6 fi Looks at exit status to make decision
The null command Does nothing Consider an unload program: ls a cpio o > /dev/rmt0 if [ $? eq 0 ] then rm * else echo problem occurred with cpio fi How to convert this to a backup program?
Case Take a string and match it to a pattern: sh-2.05a$ choice=2 sh-2.05a$ case $choice in > "1") > echo choice was 1 > ;; > "2" "3") > echo choice was 2 or 3 > ;; > *) > echo unknown choice > ;; > esac choice was 2 or 3
Loops The for loop takes a list of arguments and executes the loop one time for each word in the list for color in red green blue pink do echo The sky is $color done
While The while loop tests on a boolean condition sh-2.05a$ count=1 sh-2.05a$ while [ $count -lt 10 ] > do > echo $count > count=`expr $count + 1` > done 1 2 3 4 5 6 7 8 9 sh-2.05a$
set and shift set assigns positional parameters to its arguments. $ set `date` $ echo The date today is $2 $3, $6 The date today is Aug 27, 2001 shift change the meaning of the positional parameters giant2 #!/bin/sh while test $1 do echo $1 shift done $ giant2 fee fie fo fum fee fie fo fum
read read one line from standard input and assigns successive words to the specified variables. Leftover words are assigned to the last variable. name #!/bin/sh echo Enter your name: read fname lname echo First: $fname echo Last: $lname $ name Enter your name: Alexander Graham Bell First: Alexander Last: Graham Bell
Debugging set vx (v: echo commands as they are read, x: echo as commands as they are executed) Set to turn off tracing
Example: $ cat testscript2 #!/bin/sh set -vx a=`ls -l` b=5 ls -l awk '{print $1;}' $ testscript2 a=`ls -l` ls -l ++ ls -l + a=total 1 -rw------- 1 kenxu instrs 0 May 25 10:26 myfile1 -rw------- 1 kenxu instrs 0 May 25 10:26 myfile2 -rwx--x--x 1 kenxu instrs 58 May 25 10:19 testscript2 b=5 + b=5