Vi & Shell Scripting

Transcription

1 Vi & Shell Scripting Comp-206 : Introduction to Week 3 Joseph Vybihal Computer Science McGill University

2 Announcements Sina Meraji's office hours Trottier 3rd floor open area Tuesday 1:30 2:30 PM Thursday 1:00 2:00 PM

3 Part 1 The Vi Editor

4 WhyVi? Most installed command-line editor Next comes Emacs We don't need to learn about windowed text editors... they are all the same Most installed on limitted resourced real-time systems Concern is low memory usage

5 Vi Modes Edit Mode You can input and edit your text file Command Mode Issue editor commands like File/Open, File/Save, Search and Replace,... Escape Mode Takes you out of the other two modes Gives you immediate commands: delete line, next page,...

6 Vi Modes Command Mode Press the : key Takes you from Escape to Command mode Edit Mode Many keys get you into this mode: i, a, o, O Takes you from Escape to Edit mode Escape Mode Press the ESC key Takes you from Edit/Command to Escape mode

7 A sample Vi session...

8

9

10 Part 2 Introduction to Scripts

11 Shell A shell is an application that allows you to relay commands to the Operating System. These commands are executed by the Operating System and the output is sent back to the shell. Most often, shells are used to execute commands. Most shells also include a simple programming language, often known as a scripting language. The first popular shell was the Bourne shell (sh). It remains at the core of most UNIX OS.

12 The Architecture The Hardware The OS The Shell The Command-line The Script A window is a special case of this YOU 12

13 Script (Batch) Scripts are collections of commands, grouped in a file and sequentially execute. Scripts are not compiled programs, they are interpreted. Scripts run from top to bottom, with little control flow (does not have the complexity of a piece of software). System administrators use script to automate complex tasks. Be sure to CHMOD it to execute!

14 Example script clear who finger bob ie outlook quoteoftheday Tasks at login. 14

15 Bash BASH is a scripting language that implements some programming language control flows, like: goto if for while But it is a script, it is interpreted by the OS, not compiled and executed by the CPU. 15

16 Uses of script files Backup scripts archiving important files and saving them in a safe place. Startup scripts application which require a complicated environment to run. programs you often run when your session begins Scheduled scripts regularly rotating and archiving logs. Maintenance scripts a script that creates a user or changes a password on multiple system, all in one command. Programmer scripts compiling, version control and backups in one step

17 Default Bash Files.cshrc.kshrc.login.bash_profile csh login script ksh login script sh login script bash login script.plan.forward extra finger info forwarding.logout sh and csh logout 17

18 Changing ENV at Login When you login or start a shell, specific scripts are executed to configure your environment. The specific script depends on which shell you are using. With Bash, ~/.bash_profile is executed on login. With csh (and it s derivative, like tcsh), the /.cshrc file is executed.

19 A Session Login prompt Launch shell Execute login script rlogin Display prompt until logout or exit Execute logout script Login prompt Sessions are stacked and independent. 19

20 Environment Changes While customising your account, you might want to set some shell variables, such as the PATH, CLASSPATH and PROMPT. set prompt = Best Student $home> set history = 100 setenv prompt Best Student $home> 20

21 Other startup things... You can customize your command prompt by changing an environment variable. Check your shell s documentation for information on how to do this (often set prompt=something) You can use the alias command to set up shortcuts. alias ll= /usr/bin/ls -l You can set your default editor. EDITOR=vi Some applications might require you to set up an environment variable. PVM_ROOT=/usr/local

22 Path and Classpath The PATH is a set of directory a shell searches for executables. In, it s a colon ( : ) separated list. In Windows, it s a semi-colon ( ; ) separated list. You can use the which command to figure out what file path is needed. The CLASSPATH is the set of directories the JVM searches when loading classes.

23 Other defaults HOME SHELL TERM USER PWD path to home directory path to your shell type of terminal I/O your user name your current directory 23

24 Part 3 Regular Expressions and commands that use it

25 Regular Expressions Several commands and editors allow you to search on text patterns. These text patterns are known as regular expressions (regex).

26 grep, sed and awk grep [options] string files search for occurrences of the string. sed [options] files stream editor for editing files. awk [options] files scan for patterns in a file and process the results. (script execution)

27 Grep grep is used to search for the occurrence of a regular expression in files. Regular expressions, are best specified in apostrophes (or single quotes) when used with grep. Some common options include: -i : ignore case -c : report only a count of the number of lines containing matches -v : invert the search, displaying only lines that do not match -n : display the line number along with the line on which a match was found -l : list filenames, but not lines, in which matches were found

28 Examples of grep Consider the following text file : Alex Marc Micheal Ting Juan Jeremy Jessica Yannick Nicolas Jean-Sebastien Nadeem

29 Examples of grep (cont.) Grep for a specific string... [jvybihal][~/cs206] grep Je demo.txt Jeremy Jessica Jean-Sebastien [jvybihal][~/cs206] grep -n Je demo.txt 6:Jeremy 7:Jessica 10:Jean-Sebastien [jvybihal][~/cs206] grep -c Je demo.txt 3

30 Examples of grep (cont.) Grep for vowels... [jvybihal][~/cs206] grep -i ^[aeiouy] demo.txt Alex Yannick [jvybihal][~/cs206] grep -i [aeiouy]$ demo.txt Jeremy Jessica [jvybihal][~/cs206] grep -i [aeiouy]\{2,\} demo.txt Micheal Juan Yannick Jean-Sebastien Nadeem

31 Examples of grep (cont.) Grep for specific characters... [jvybihal][~/cs206] grep -i ^.e demo.txt Jeremy Jessica Jean-Sebastien [jvybihal][~/cs206] grep -i ^.e\ a.$ demo.txt Micheal Juan Jeremy Jessica Nicolas Jean-Sebastien

32 Regex Syntax Take a look at the Regex Syntax quick sheet. Literal characters are combination to represent special characters. Character classes are combination to represent groups of characters. Repetition indicate how often a character should be appear to be a match. Anchors determine where the matching string must be found.

33 33

34 When to use grep Grep is a useful tool to find specific strings. Outlining all the errors in a log file. Finding a specific string in a collection of source files. It becomes an even more powerful tool when combined with other utilities. [jvybihal][~/cs206] ps -e grep java pts/1 00:18:30 java 21395? 00:00:08 java pts/4 00:03:51 java

35 Part 4 Shell Scripting

36 Shell Scripting A shell program (or script) containing a series of shell commands. The first line of the script should start with #! (sha-bang) which indicates to the kernel that the script is directly executable. You immediately follow this with the name of the shell, or program (spaces are allowed), to execute, using the full path name. Different languages can be used to script (sh, bash, perl, python, ruby, etc). To set up a Bourne shell script the first line must be: #!/bin/sh You also need to specify that the script is executable by setting the proper permissions on the file. % chmod +x shell_script

37 Variables There are three kinds of variables in a shell script: Environment Variable : these variables are used to customize the operating system and the shell to your needs. User-created : these variables are created by the script itself. Positional Parameters : these variables store the parameter used to start the script.

38 Positional Variables $# : number of arguments on the command line $- : options supplied to the shell $? : exit value of the last command executed $$ : process number of the current process $! : process number of the last command done in background $n : argument on the command line, where n is from 1 through 9, reading left to right $0 : the name of the current shell or program $* : all arguments on the command line ( $1 $2... $9 ) $@ : all arguments on the command line, each separately quoted ( $1 $2... $9 )

39 Your first Script The following script will print out the positional variables: #!/bin/sh echo "$#:" echo $-: echo $?: echo $$: echo $!: echo $3: echo $0: echo $*: echo $# $$? $$ $! $3 $0 $*

40 Changing an environment variable You can output an environment variable using echo. Before something goes wrong, you might want to backup the old value. TEMP_VAR_NAME=$VAR_TO_CHANGE You can then set the new value. VAR_TO_CHANGE=SomethingElse You can use any existing variables. TEMP_VAR_NAME=$VAR_TO_CHANGE:SomethingElse If something goes wrong, you can revert to the old value. VAR_TO_CHANGE=$TEMP_VAR_NAME This works in bash. In tcsh, you need to use setenv.

41 Shell Scripts A shell script runs from top to bottom. If statements and loop can be used to alter the control flow. You can also create functions. The # character is usually used to denote a comment. The #! (called a sha-bang) at the start of the script indicates which program should execute/interpret the script. Unlike other programming languages, scripts are sometime sensitive to extra spaces.

42 Simple Script The following script gathers information about the system and stores it in a file specified at the command line. #!/bin/sh uname -a > $1 date >> $1 who >> $1 The output was as follows : [jvybihal][~/cs206]./info.sh output.txt [jvybihal][~/cs206] cat output.txt Linux rogue gentoo-r4 #1 SMP... Thu Aug 10 10:57:38 EDT 2006 jvybihal pts/0 Aug 10 08:04 (dz2.cs.mcgill.ca)

43 Reading from STDIN The read command allows you to read a string from STDIN. That string is then stored in the specified variable. #!/bin/sh echo "What is your name?" read name echo "Your name is $name."

44 Arithmetic Operations The shell was never designed for numerical work. To do mathematical (integer) operations, you can to use the expr command. The following example script adds two numbers passed at the command line and outputs the answer to STDOUT. #!/bin/sh sum= expr $1 + $2 echo $sum

45 Question Write a script that will tell you if your friend is currently logged into the system. 45

46 SH.login Example e 100 ile 46

47 CSH.cshrc Example 47

48 Part 5 Shell Programming

49 test Before discussing control statements (if, for, etc), we need to check out the test command. The test command is used to evaluate an expression, or in our case, a condition. Although shells do contain operators to test a condition, they are not as versatile and universal as test. The test command can evaluate condition at the file, string or integer level.

50 File Tests -r : true if it exists and is readable -w : true if it exists and is writable -x : true if it exists and is executable -f : true if it exists and is a regular file -d : true if it exists and is a directory -h or -L : true if it exists and is a symbolic link and many more...

51 String Tests -z string : true if the string length is zero -n string : true if the string length is non-zero string1 = string2 : true if string1 is identical to string2 string1!= string2 : true if string1 is non identical to string2 string : true if string is not NULL

52 Integer Tests n1 -eq n2 : true if integers n1 and n2 are equal n1 -ne n2 : true if integers n1 and n2 are not equal n1 -gt n2 : true if integer n1 is greater than integer n2 n1 -ge n2 : true if integer n1 is greater than or equal to integer n2 n1 -lt n2 : true if integer n1 is less than integer n2 n1 -le n2 : true if integer n1 is less than or equal to integer n2

53 If statements If statements have a syntax similar to Java, but without brackets. if _condition_ then _code_ elif _condition_ then _code_ else _code_ fi

54 Example of If statement The following example program can be used to add or subtract two numbers. #!/bin/sh if test $1 = add then result= expr $2 + $3 elif test $1 = sub then result= expr $2 - $3 else result=0 fi echo "The result is $result \n"

55 Case statements A case statement is similar to a Java switch statement. case _condition_ in _condition1_) _action1_;; _condition2_) _action2_;; _condition3_ _condition4_) _action3_;; *) _else_action_;; esac

56 Examples of Case The following example program is a remodeling of the if example, but with a case statement. #!/bin/sh case $1 in add addition) result= expr $2 + $3 ;; sub substraction) result= expr $2 - $3 ;; *) result=0;; esac echo "The result is $result \n"

57 For loops The for loop is similar to a Java iterator. It allows you to iterate (loop) over a list strings. for _var_ in _list_ do _action_ done

58 Example of a For loop The following script executes the file command for each file in the specified path. #!/usr/bin/sh for i in ls $1 do file $i; done

59 While Statement The last control statement we will need is the while statement. Again, it is very similar to its Java equivalent. while _condition action_ [continue] [break] end

60 Using parameters The following script will pad a file with zeros. #!/usr/bin/sh i= wc -c < $1 ; while test $i -lt $2 do echo -n "0" >> $1; i= wc -c < $1 ; done

61 Capturing Complex Output Some commands, such as date, have output that require an extra bit of parsing to use. Sun Aug 13 11:42:38 EDT 2006 You can use the set command to capture and parse the output. set date The output will be stored in $n ($1, $2, $3, etc). Note that using set will erase any data you might already have in $n.

62 Example of set The following script executes the date command and outputs the parsed result. #!/usr/bin/sh set date echo "Time: $4 $5" echo "Day: $1" echo "Date: $3 $2 $6" The output would be as follows: Time: 12:45:54 EDT Day: Sun Date: 13 Aug 2006

63 Examples

64 While 64

65 Case 65

66 Wrap Feature 66

67 Functions 67

68 CSH Example 68

69 At Home

70 Try These Regular Expressions Practice writing regular expressions You can use grep to help you do his 70