CS 25200: Systems Programming. Lecture 10: Shell Scripting in Bash

Transcription

1 CS 25200: Systems Programming Lecture 10: Shell Scripting in Bash Dr. Jef Turkstra 2018 Dr. Jeffrey A. Turkstra 1

2 Lecture 10 Getting started with Bash Data types Reading and writing Control loops Decision making Indexed arrays Basic regular expressions 2018 Dr. Jeffrey A. Turkstra 2

3 Shells There are many diferent shells available for *NIX-based computers. Some of them even run under that other OS. A shell is basically a command interpreter. It provides an interface between the user and the computer (operating system). Shells may be graphical (explorer.exe, for instance) or text-based - often times called a CLI or command line interface 2018 Dr. Jeffrey A. Turkstra 3

4 Shells cont When we write a shell script, the first line of the file tells the operating system which shell to use. Some common *NIX shells include: #! /bin/sh Bourne shell #! /bin/csh C-Shell #! /bin/ksh KornShell (more powerful) #! /bin/bash Bourne-Again SHell 2018 Dr. Jeffrey A. Turkstra 4

5 Bash "Bash is the GNU Project's shell. Bash is the Bourne Again SHell. Bash is an sh-compatible shell that incorporates useful features from the Korn shell (ksh) and C shell (csh). It is intended to conform to the IEEE POSIX P1003.2/ISO Shell and Tools standard. It ofers functional improvements over sh for both programming and interactive use. In addition, most sh scripts can be run by Bash without modification." Dr. Jeffrey A. Turkstra 5

6 Diving in - variables Declaration and definition: NAME="whatever" with no spaces between or after the "=" Accessed as $NAME,, or better, ${NAME} Examples: MagicVariable1=7 MagicVariable2="Hello!" MagicVariable3= MagicVariable4=${MagicVariable1} 2018 Dr. Jeffrey A. Turkstra 6

7 Special Bash "variables" Bash has some built-in variables that allow us to get at some important information $# Number of command line arguments $* Command line arguments $0 The name of the shell script $$ Current process ID number $? Return value from last executed command $1 to $N Command line parameters (as separated by whitespace) 2018 Dr. Jeffrey A. Turkstra 7

8 echo command echo <options> Arguments Data is not formatted Writes its arguments, separated by blanks (spaces) and ending with a newline, to standard output. Whitespace can be protected with double or single quotes as needed. Example: $ echo Hello, World! Hello, World! $ 2018 Dr. Jeffrey A. Turkstra 8

9 Common echo options See man page for more detail. -n do not add a newline at the end -e turn on the special meaning of the backslash \ character -E turn of backslash (default) 2018 Dr. Jeffrey A. Turkstra 9

10 Examples echo -n "Hello" echo ", World!" echo "-n hmm" echo ":-)\n:-(" echo -e ":-)\n:-(" Results in the following output: Hello, World! -n hmm :-)\n:-( :-) :-( 2018 Dr. Jeffrey A. Turkstra 10

11 printf command - formatted output Almost like in C: printf "format" list of variables Format string is like that found in C The list of variables should be separated by spaces. Example: A=3.45 S="Big Deal" printf "A: %5.2f, S: %s\n" $A "$S" 2018 Dr. Jeffrey A. Turkstra 11

12 More Bash variables Untyped variables can be used to hold string and integer values Output: #! /bin/bash Output: A="Big Deal" Big Deal echo $A 3 A=3 2.3 echo $A A=2.3 echo $A exit Dr. Jeffrey A. Turkstra 12

13 Typed variables Integers typeset -i variable_name Integers Cannot be used to store strings 2018 Dr. Jeffrey A. Turkstra 13

14 Example #! /bin/bash typeset -i A A=3 echo "A: $A" A="Oh Well" echo "A: $A" A=2.4 echo "A: $A" exit 0 Results in the following output: A: 3 X1: line 5: Oh Well: syntax error in expression (error token is Well ) A: 3 X1: line 7: 2.4: syntax error: invalid arithmetic operator (error token is.4 ) A: Dr. Jeffrey A. Turkstra 14

15 Floating point variables Bash does not support foating point values or math. :-( 2018 Dr. Jeffrey A. Turkstra 15

16 Constants We can create read only variables too typeset -r Name="Ffej" typeset -r K=8 Or how about a read only integer typeset -ri K= Dr. Jeffrey A. Turkstra 16

17 Example #! /bin/bash typeset -r Name="Ffej Artskrut" echo $Name Name="Big Deal" echo $Name exit 0 Results in the following: $ C Ffej Artskrut./C: line 4: Name: readonly variable Ffej Artskrut 2018 Dr. Jeffrey A. Turkstra 17

18 for loops Two diferent syntaxes in Bash for variable in list do commands done where list is a set of strings separated by whitespace for (( initial_expression; loop_condition; loop_expression )) do commands done do must be on the 2 nd line in both cases! 2018 Dr. Jeffrey A. Turkstra 18

19 Examples for I in do echo -n ${I} done for (( I = 1; I < 6; I++ )) do echo -n ${I} done Both result in the same output: Dr. Jeffrey A. Turkstra 19

20 Examples cont #! /bin/bash ls *.c > temp:$$ touch temp:$$ for File in $(cat temp:$$) do lp -dsomeprinter ${File} done rm -f temp:$$ exit Dr. Jeffrey A. Turkstra 20

21 while loop Note that a command is considered "true"" as long as its exit status is zero.. This is at times backwards from what you might be used to. while <command exit status is true (0)> do <whatever commands you need to do the job> done 2018 Dr. Jeffrey A. Turkstra 21

22 Example #! /bin/bash typeset -i I=0 while (( I < 10 )) do echo -n " ${I}" (( I = I + 1 )) done echo exit 0 Outputs: Dr. Jeffrey A. Turkstra 22

23 read command read can be used to obtain user input. Eg, echo -n "Feed me data: " read DataVar echo "You fed me ${DataVar}!" It may also be used to read from other streams such as a fle. Everyone always forgets that the name of the file goes at the end of the while loop 2018 Dr. Jeffrey A. Turkstra 23

24 reading from a fle #! /bin/bash cat InFile echo while read A B C do echo "A: ${A} " \ "B: ${B} " \ "C: ${C} " done < InFile exit Dr. Jeffrey A. Turkstra 24

25 Output This is the output generated from the cat command, which simply displays a file's contents to stdout: This is Neat! CS 252 is Great! Ffej is the best Big Deal 1234 This is the output generated by the echo statement inside of the while loop: A: This B: is C: Neat! A: CS B: 252 C: is Great! A: Ffej B: is C: the best. A: 1 B: 2 C: A: Big B: Deal C: A: 1234 B: C: 2018 Dr. Jeffrey A. Turkstra 25

26 Bash math Bash only has integer math. Use let or (( )) to isolate mathematical statements Basic math operators include: addition (+),( subtraction (-),( multiplication (*),( division (/), and modulus (%)( 2018 Dr. Jeffrey A. Turkstra 26

27 Integer math example #! /bin/bash typeset -i a=11 typeset -i b=3 typeset -i x (( x = a + b )) echo "(( x = $a + $b )) x = $x" (( x = a - b )) echo "(( x = $a - $b )) x = $x" (( x = a * b )) echo "(( x = $a * $b )) x = $x" (( x = a / b )) echo "(( x = $a / $b )) x = $x" (( x = a % b )) echo "(( x = $a % $b )) x = $x" exit Dr. Jeffrey A. Turkstra 27

28 Output (( x = )) x = 14 (( x = 11-3 )) x = 8 (( x = 11 * 3 )) x = 33 (( x = 11 / 3 )) x = 3 (( x = 11 % 3 )) x = Dr. Jeffrey A. Turkstra 28

29 Branching if command true then commands else commands fi Note: the command is "true" if it returns 0 - neat! 2018 Dr. Jeffrey A. Turkstra 29

30 Nested if statements As one might expect, it is possible to nest branches in ksh if (( A < B )) then echo "A < B" else if (( A == B )) then echo "A = B" else echo "A > B" fi fi 2018 Dr. Jeffrey A. Turkstra 30

31 elif statement Equivalent to "else if" in C if (( A < B )) then echo "A < B" elif (( A == B )) then echo "A = B" else echo "A > B" fi 2018 Dr. Jeffrey A. Turkstra 31

32 Testing Tests of any sort should be surrounded by double brackets with spaces: [[ space whatever space ]] Example if [[ -r MyFile ]] then echo MyFile is readable! fi Tests include logical comparisons, string comparisons, and file permission tests 2018 Dr. Jeffrey A. Turkstra 32

33 File testing -a file exists -d is a directory -f is an ordinary file -r is readable -s has non-zero length -w is writable -x is executable and lots more! reverse the test 2018 Dr. Jeffrey A. Turkstra 33

34 Arithmetic comparison Comparisons between numbers should always be surrounded by double parentheses: (( whatever )) For example: if (( 7 <= 5 )) then echo "The world is at an end!" fi Arithmetic comparisons include: == equal >= greater than or equal > greater than <= less than or equal < less than!= not equal 2018 Dr. Jeffrey A. Turkstra 34

35 String tests Equality, if [[ string1 = string2orpattern ]] if [[ string1 == string2orpattern ]] if [[ string1!= string2orpattern ]] Lexicographical ordering, if [[ string1 < string2 ]] if [[ string1 > string2 ]] Emptiness, if [[ -n string1 ]] # string is not NULL if [[ -z string1 ]] # string is NULL 2018 Dr. Jeffrey A. Turkstra 35

36 Example #! /bin/bash # if (( $#!= 1 )); then echo "Usage: $0 <filename>" exit 1 fi File="$1" if [[! -f "${File}" ]]; then echo "File: ${File} is not an ordinary file" else echo "File: ${File} is an ordinary file" fi exit Dr. Jeffrey A. Turkstra 36

37 Output $ File_Check Usage: File_Check <filename> $ File_Check x File x is not an ordinary file $ File_Check File_Check File File_Check is an ordinary file 2018 Dr. Jeffrey A. Turkstra 37

38 Arrays in Bash Unlike C, bash supports sparse arrays ArrVar[5]=8 ArrVar[15]=12 ArrVar[19]=7 If the indices aren't consecutive, how do we know the array's size? How do we know the indices for all values? 2018 Dr. Jeffrey A. Turkstra 38

39 Special operators # and! You can obtain a list (a string of whitespace-separated values) of every element in an array: echo ${ArrVar[*]} echo ${ArrVar[@]} The size of an array can be found by using the # operator: ${#ArrVar[*]} or ${#ArrVar[@]} The array subscripts can be found by using the! operator: ${!ArrVar[*]} or ${!ArrVar[@]} 2018 Dr. Jeffrey A. Turkstra 39

40 Indexed array example #! /bin/bash A[5]=34 A[1]=3 A[2]=56 A[100]=89 echo "Size of array: ${#A[*]}" echo "Array indices: ${!A[*]}" for I in ${!A[*]} do echo "A[${I}]=${A[I]}" done exit Dr. Jeffrey A. Turkstra 40

41 Indexed array output Size of array: 4 Array indices: A[1]=3 A[2]=56 A[5]=34 A[100]= Dr. Jeffrey A. Turkstra 41

42 Read array example #! /bin/bash echo "Data_File:" cat Data_File # Remember, cat just dumps the file's echo # contents to stdout echo "Formatted output:" while read -a Data # Split on whitespace do # (spaces and tabs) for (( I = 0; I < ${#Data[*]}; I++ )) do printf "%6.2f" ${Data[I]} done echo done < Data_File exit Dr. Jeffrey A. Turkstra 42

43 Output Data_File: Formatted output Dr. Jeffrey A. Turkstra 43

44 More on loops Similar to C, bash has continue n Used to stop the execution of the innermost n loops and then continue with the next loop. The default is n = 1. break n Used to end the execution of the innermost n loops. Default is n = Dr. Jeffrey A. Turkstra 44

45 Examples #! /bin/bash for (( I = 0; I <= 4; I++ )); do if (( I == 1 )); then continue fi echo -n " ${I}" done echo exit 0 Results in the following output: Dr. Jeffrey A. Turkstra 45

46 Examples cont #! /bin/bash I=0 while (( I <= 4 )); do if (( I == 1 )); then break fi echo -n " ${I}" (( I++ )) done echo exit 0 Results in the following: Dr. Jeffrey A. Turkstra 46

47 What about arguments? What if we want to loop through the command line arguments? Even if we know how many there are, we still can't use a loop construct #! /bin/bash for (( I = 0; I < $#; I++ )); do echo $what??? done 2018 Dr. Jeffrey A. Turkstra 47

48 shift n Used to left shift the parameters on the command line n places Default is n = 1 $0 is never changed Often used when an unknown number of parameters are passed, or for looping through a large number of parameters Dr. Jeffrey A. Turkstra 48

49 Example #! /bin/bash echo '$0 -- ' $0 echo '$# -- ' $# X=0 while (( $#!= 0 )); do (( X = X + 1 )) echo "\"\$${X}\" was $1" shift done exit 0 Sample run $ parameters q "1 2 3" xyz $0 -- parameters $# -- 3 "$1" was q "$2" was "$3" was xyz 2018 Dr. Jeffrey A. Turkstra 49

50 Trouble with quotes There are various kinds of quotes, and each one can mean something diferent in ksh. ' The single forward quote character " The double quote character ` The back quote character \ The backslash character (sometimes used to escape quotes) 2018 Dr. Jeffrey A. Turkstra 50

51 The single forward quote ' Must appear in pairs Protects all characters between the pair of quotes Ignores all special characters Protects whitespace 2018 Dr. Jeffrey A. Turkstra 51

52 Single quote examples Path='/b/cs252' echo The path for cs252 is $Path Displays: The path for cs252 is /b/cs252 echo 'The path for cs252 is $Path' Displays: The path for cs252 is $Path echo 'The book costs $2.00' Displays: The book cost $ Dr. Jeffrey A. Turkstra 52

53 Wildcard * exception #! /bin/bash ls echo * echo '*' yuk='*' echo $yuk echo '$yuk' exit 0 Produces the following output: Chap-1 TESTS TV memo x y Chap-1 TESTS TV memo x y * Chap-1 TESTS TV memo x y $yuk 2018 Dr. Jeffrey A. Turkstra 53

54 The double quote " Must come in pairs Protects whitespace Does not ignore Dollar signs - $ Back quotes - ` Backslashes - \ 2018 Dr. Jeffrey A. Turkstra 54

55 Double quote example Path="/b/bee264" echo "The path for ee264 is $Path" Yields: The path for ee264 is /b/ee264 echo "The book cost \$2.00" Yields: The book cost $ Dr. Jeffrey A. Turkstra 55

56 Wildcard * exception #! /bin/bash ls echo * echo "*" yuk="*" echo $yuk echo "$yuk" exit 0 Produces the following output: Chap-1 TESTS TV memo x y Chap-1 TESTS TV memo x y * Chap-1 TESTS TV memo x y * 2018 Dr. Jeffrey A. Turkstra 56

57 The back quote ` Used to issue a UNIX command and obtain its output... #! /bin/bash echo "Current directory is `pwd`" DIR=`pwd` echo "Directory is ${DIR}" exit 0 Outputs: Current directory is /a/turkstra/252 Directory is /a/turkstra/ Dr. Jeffrey A. Turkstra 57

58 $(command) "Better" way to issue a UNIX command and obtain its output Makes your code easier to read, easier to debug #! /bin/bash echo "Current working directory is $(pwd)" DIR=$(pwd) echo "Directory is ${DIR}" exit 0 Results in Current directory is /a/turkstra/252 Directory is /a/turkstra/ Dr. Jeffrey A. Turkstra 58

59 The backslash \ Used to remove any special meaning that a symbol may have. \$1.00 "escapes" the $ character, treating it as a literal $ instead of $1 (the first argument passed to the script) Used to add special meaning to symbols. \n for newline, for example. If it is the last symbol on a line, it will act as a continuation indicator Dr. Jeffrey A. Turkstra 59

60 Backslash example #! /bin/bash echo "This item costs \$2.00" echo -n "This is line 1, " echo "this is the rest of the line" echo "This is" \ "\"$(whoami)\"" # \" used to print " exit 0 Outputs This item costs $2.00 This is line 1, this is the rest of the line This is "turkstra" 2018 Dr. Jeffrey A. Turkstra 60

61 Whitespace protection #! /bin/bash DATA=$(cat $0) echo ${DATA} echo echo "${DATA}" exit 0 Displays: #! /bin/ksh DATA=$(cat $0) print ${DATA} print print "${DATA}" exit 0 #! /bin/ksh DATA=$(cat $0) print ${DATA} print print "${DATA}" exit Dr. Jeffrey A. Turkstra 61

62 grep command Used to search files for lines of information. Many, many fags - see the man page. grep -flags regular_expression filename Useful fags -x Exact match of line -i Ignore upper/lower case -c Only count the number of lines which match -n Add relative line numbers -b Add block numbers -v Output all lines which do not match 2018 Dr. Jeffrey A. Turkstra 62

63 Simple regular expressions Regular expressions express patterns. They are used to find and/or extract pieces of information from a string.. Matches any character ^ Start of line $ End of line \ Escape character [list] Matches any character in the list [^list] Matches any character not in the list * Match zero or more occurrences of the previous regular expression \{min,max\} Matches at least min and at most max occurrences of the previous regular expression 2018 Dr. Jeffrey A. Turkstra 63

64 Examples grep "^string$" file_name collects all lines which contain only string grep " " file_name collects all lines which have any three characters surrounded by spaces grep " [0-9]\{1,3\} " file_name collects all lines containing a sequence of one to three digits surrounded by spaces grep "^x*[abc]" file_name collects all lines which start with zero or more x's followed by a single a, b, or c 2018 Dr. Jeffrey A. Turkstra 64

65 More examples Let's pretend we have a file named data abd asdf And this script #! /bin/bash # begins with 1 or 2 grep "^[0-9]\{1,2\} " data # digits followed by exit 0 # a space We should get this output abd 2018 Dr. Jeffrey A. Turkstra 65

66 -v option, inverting the match Let's pretend we have a file named data abd asdf And this script #! /bin/bash grep -v "^[0-9]\{1,2\} " data exit 0 We should get this output asdf 2018 Dr. Jeffrey A. Turkstra 66

67 -c option, counting the matches Let's pretend we have a file named data abd asdf And this script #! /bin/bash grep -c "^[0-9]\{1,2\} " data exit 0 We should get this output Dr. Jeffrey A. Turkstra 67

68 -n option, adding line numbers Let's pretend we have a file named data abd asdf And this script #! /bin/bash grep -n "^[0-9]\{1,2\} " data exit 0 We should get this output 2: :3 abd 2018 Dr. Jeffrey A. Turkstra 68

69 Using grep inside a script #! /bin/bash if (( $#!= 1 )); then echo "Usage: $0 <user_id>" exit 1 fi USER="$1" if grep "${USER}" Id_File > /dev/null then echo "Bad way: ${USER} in file" else echo "Bad way: ${USER} not in file" fi if grep "^${USER}$" Id_File > /dev/null then echo "Good way: ${USER} in file" else echo "Good way: ${USER} not in file" fi exit Dr. Jeffrey A. Turkstra 69

70 Output $ cat Id_File $ Check sam Usage: Check <user_id> maryann $ Check jeff john Bad way: jeff in file jeff Good way: jeff in file jeffrey $ Check son bill Bad way: son in file william Good way: son not in file peterson 2018 Dr. Jeffrey A. Turkstra 70

71 Questions? 2018 Dr. Jeffrey A. Turkstra 71