Introduction to Perl

Transcription

1 Introduction to Perl Scott Hazelhurst August 2013

2 Introduction and Motivation Introduction and Motivation Practical Extraction and Report Language General language Intended for systems programming, scripting Popular with systems programmers & Web developers Relatively new language (ca. 1990) Big language support for concurrency and OO. Portable.

3 Introduction and Motivation Perl powerful and flexible language: Many devotees Many criticisms of the language Easy to write difficult-to-understand code. Focus on writing readable code (for yourself, others)

4 Introduction and Motivation Objectives Describe basic features of Perl Write simple Perl programs Use basic matching facilities Lots of resources: Books perldoc, info perl Assume: knowledge of programming, C-like language

5 Perl s Data Structures Perl s Data Structures I Perl is an imperative language. State is represented by a set of variables. Program is an ordered sequence of commands. Computation is accomplished by the execution of these commands in the specified order Will explore Perl s OO features later.

6 Perl s Data Structures Flexible (perhaps too flexible) language. Scalars: numbers, strings Arrays, lists Hash tables References Variables have a special prefatory character to indicate to which genre of variable (%, )

7 Perl s Data Structures Feature Warning Variables automatically declared Variables given default values Implicit type coercion common Meaning dependent on context Good idea to use the use strict; use warnings pragmas

8 Perl s Data Structures Scalar types Scalar types All scalar variables have a $ as prefix: e.g. $x, $year$, $etc. $c = 10; $f = 9/5* $c +32; $cname = Introduction to Perl ; print " Course $cname : size $c\n";

9 Perl s Data Structures String operations Interpolation Inside a double-quoted string, Perl does variable interpolation, and interprets escape characters print "Course $cname size \t $f\n;";

10 Perl s Data Structures String operations Interpolation Inside a double-quoted string, Perl does variable interpolation, and interprets escape characters print "Course $cname size \t $f\n;"; Not done inside single-quoted strings. print Course $cname size \t $f\n ;

11 Perl s Data Structures String operations Other string operations Concatenation:. $fruits = apples and pears ; $base = pies ; $full = $fruits. make. $base ; $full.= :. fruits ; Length of string: length. Substrings: substr Repeat operator: x Powerful & flexible string matching and processing regular expressions. Type conversion to and from integers as necessary!!?!!

12 Perl s Data Structures Artithmetic operations Arithmetic Operations The basic Perl arithmetic operations are: +, -, *, /, %, **. Default: floating point arithmetic There are many arithmetic procedures: int, sqrt,... Short-cuts: $x = 3; $x += $y +1; $x ++; --$y; Perl has many C-like features. eg.

13 Perl s Data Structures Truth and Logical operations Truth and Logical operations 1. Any string is true except for the empty string and 0 ; 2. Any number is true except for 0 3. Any reference is true 4. Any undefined value is false

14 Perl s Data Structures Truth and Logical operations Logical operators High binding Low binding && and or! not Short-circuit evaluation! Logical expressions returns the last value evaluated. e.g. 2 or 3 and 5 2 and (0 or 5) $a = 2

15 Perl s Data Structures Truth and Logical operations Relational operators Comparison Numeric String Equal, Not equal ==,!= eq, ne Less than (equal) <, <= lt, le Greater than (equal) >, >= gt, ge Comparison <=> cmp Smart matching ~~ ~~

16 Perl s Data Structures Truth and Logical operations File operators Example Name Example Name -e $a File Exists -T $a File is a text file -r $a Readable file -w $a Writable file -d $a File is a directory -f $a Regular file

17 Standard input and output Standard input and output Default: file handle STDIN associated with keyboard (input), STDOUT with console output. Input is line oriented.

18 Standard input and output Output print command by default sends output to STDOUT. print "Hello"; same as print STDOUT "Hello";

19 Standard input and output Input A reference to <STDIN> waits for input from console. data typed in is returned print "Enter the temp in C: "; $c = <STDIN>; print "Temperature in F is ". (9/5*$c+32);

20 Control Structures Making decisions Control structures Conditional if/else/elsif: if ($x < $y) {$x=1} else {$y=1}; unless: unless ($ok) { die "Error in file I/O"; } no explicit switch/case implicit through use of short-circuit (-e "f.dat") or ($numf++) ternary conditional operator cond? ex1 : ex2

21 Control Structures Making decisions Example unless (-e " data. dat ") { die " File data. dat not found " };

22 Control Structures Making decisions Example $c = $a cmp $b; if ($c ==0) { print " The strings are the same.\ n } elsif ($c <0) { print "$a comes before $b\n"; } else { print "$b comes before $a\n"; }

23 Control Structures Loops Loops while: while (cond} {... } for statement: C-style: for (init; condition; change) {... } foreach

24 Control Structures Loops Example while loop Read in a list of numbers terminated by 0. Compute the sum. $sum =0; $num =<STDIN >; while ( $num!= 0) { $sum = $sum + $num ; $num =<STDIN >; } print " Sum is $sum \n";

25 Control Structures Loops use strict ; my my $i; $j; for ( $i =1; $i <11; $i ++) { for ($j =0; $j <$i; $j ++) { print "*"; } print "\n"; }

26 Control Structures Loops use strict ; my $i; for ( $i =1; $i <11; $i ++) { print "*" x $i; print "\n"; }

27 Control Structures Other flow of control... Other flow of control... next, last continue and break scalar range operator:.. don t use unless you re an expert. goto don t use

28 Simple process control Simple process control Can execute Unix commands or executables using the backtick operator $today = date "+%C%y%m%d" ; print " Today is $today \n";

29 Simple process control Other system calls system Similar to backtick but returns return code $x = system ("ls"); print " Returned <$x >\n"; exec Similar to system, but does not wait for completion.

30 File operations File operations To read data from a file or write it to a file, use a file handle. Need to associate file handle with external file. File can be virtual could be a pipe.

31 File operations Files in general Files in General File handle is the Perl construct used to manipulate files Open it associate with external file or pipe Use it Close it

32 File operations Files in general open(dataf, "figs.dat") read

33 File operations Files in general open(dataf, "figs.dat") open(dataf, "<figs.dat") read read

34 File operations Files in general open(dataf, "figs.dat") open(dataf, "<figs.dat") open(dataf, ">figs.dat") read read write

35 File operations Files in general open(dataf, "figs.dat") open(dataf, "<figs.dat") open(dataf, ">figs.dat") open(dataf, ">>figs.dat") read read write append

36 File operations Files in general open(dataf, "figs.dat") read open(dataf, "<figs.dat") read open(dataf, ">figs.dat") write open(dataf, ">>figs.dat") append open(dataf, " output-pipe-cmd"); set up output filter

37 File operations Files in general open(dataf, "figs.dat") open(dataf, "<figs.dat") open(dataf, ">figs.dat") open(dataf, ">>figs.dat") open(dataf, " output-pipe-cmd"); up output filter read read write append set open(dataf, "input-pipe-cmd "); set up input filter

38 File operations Files in general open(dataf, "figs.dat") open(dataf, "<figs.dat") open(dataf, ">figs.dat") open(dataf, ">>figs.dat") open(dataf, " output-pipe-cmd"); up output filter read read write append set open(dataf, "input-pipe-cmd "); set up input filter

39 File operations Files in general open(dataf, "figs.dat") open(dataf, "<figs.dat") open(dataf, ">figs.dat") open(dataf, ">>figs.dat") open(dataf, " output-pipe-cmd"); up output filter read read write append set open(dataf, "input-pipe-cmd "); set up input filter To print to a file that is open for writing, use the related file handle in the print statement.

40 File operations Input from files Input from files Suppose that INP is a file handle. A reference to <INP> does the following: Returns the next line of input; Consumes the input in the case of a file, advances the file pointer to the next line of the file. NB: end-of-line marker read in.

41 File operations Input from files for ($i =0; $i <3; $i ++) { $x=<inp >; print "** $x ##" } Assuming that file contains apple, banana, cherry.

42 File operations Input from files for ($i =0; $i <3; $i ++) { $x=<inp >; print "** $x ##" } Assuming that file contains apple, banana, cherry. **apple ##**banana ##**cherry ##

43 File operations Input from files To add up the numbers in a file with three numbers. open (INP," nums. txt "); $x1 = <INP >; $x2 = <INP >; $x3 = <INP >; print " Answer is ". $x1 + $x2 + $x3 ; or open (INP," nums. txt "); print " Answer is ". ( < INP > + <INP > + <INP >);

44 File operations Input from files What s the difference between print " Answer is ". ( < INP > + <INP > + <INP >); and print " Answer is. ( < INP > + <INP > + <INP >) ";

45 File operations Input from files Add up odd numbers in file print " Enter the file name : "; $namef = < STDIN >; open (DATAF, $namef ); while ($x = <DATAF >) { if ($x % 2) { $sum = $sum + $x; } } close ( DATAF ); print " The sum is $sum \n";

46 File operations Input from files A common Perl idiom is open (DATAF, $namef ) or die " Can t open

47 File operations Implicit operands Implicit Operands Perl uses implicit operands extensively. We are told that this is a feature. The main culprit: $_ or $ARG Set (among other places) by reference to a file handle. Many operators use $ARG if not given explicit operator explicitly. The following prints out the contents of a file: while (<DATAF>) {print};

48 File operations Implicit operands Exercise Write a Perl program that reads integers from a file called nums.dat and counts how many numbers are less than zero between zero and 10 greater than or equal to 11 If there is no such file, your program should print an error message and halt.

49 Arrays/lists Arrays/lists Array variables are prefixed = (" Sun ", " Mon ", " Tue ", " Wed ", " Thu ", " F Arrays indexed from 0

50 Arrays/lists Arrays/lists Array variables are prefixed = (" Sun ", " Mon ", " Tue ", " Wed ", " Thu ", " F Arrays indexed from 0 Individual element are scalars: so $days[0]

51 Arrays/lists Common to use = (" Sun ", " Mon ", " Tue ", " Wed ", " foreach $d ) { print " Day $d\n"; }

52 Arrays/lists Quote words Short hand for arrays of = qw( Sun Mon Tue Wed Thu Fri Sat

53 program s Predefined variable array Contains the program s arguments values passed by the caller. If the program is run as follows:./example.pl apple pear contains those values $ARGV[0] is apple; $ARGV[3] is 2;

54 Arrays/lists Array semantics Expressions are evaluated in scalar or list context. Similar or identical expressions can evaluate to different things in different contexts.

55 Arrays/lists Array semantics Expressions are evaluated in scalar or list context. Similar or identical expressions can evaluate to different things in different contexts. Context determined by operation: LHS of assignment determines context Operators or functions may determine context

56 Arrays/lists Array context yields array $weekdays set to Mon.. is a one element array

57 Arrays/lists Array context yields array $weekdays set to Mon.. is a one element array Scalar context $numdays sets $numdays to 7

58 Arrays/lists Array context yields array $weekdays set to Mon.. is a one element array Scalar context $numdays sets $numdays to 7 $y = ("Sun","Mon","Tue","Wed","Thu","Fri","Sat") sets $y to Sat

59 Arrays/lists Array context yields array $weekdays set to Mon.. is a one element array Scalar context $numdays sets $numdays to 7 $y = ("Sun","Mon","Tue","Wed","Thu","Fri","Sat") sets $y to Sat $weekdays = $days[1..5] sets $weekdays to 5 $#days is 6 the highest index

60 Arrays/lists Useful functions push : adds something to the right of the array pop : removes the rightmost element of the array. shift/unshift sort: returns a sorted list by default string order ascending, but can be changed.

61 Arrays/lists Arrays are inherently 1D Need references to handle multi-dimensional arrays

62 Arrays/lists Asides Asides on strings = split ";", " 1;2;3 "; foreach $s (@x) { print "$s\n"; } chop, chomp. see regexes later

63 Hash tables Hash tables Unordered set of scalars which allows fast information retrieval: Elements accessed/indexed by a string value associated with it Variables prefixed by a %, e.g. %currency Individual elements are scalar: $currency["south Africa"]

64 Hash tables $currency [" South Africa "]="ZAR "; $currency [" Britain "] = " GBP ";

65 Hash tables Example %day2num = ("Sun",0, "Mon",1, "Tue",2, "Wed",3, "Thu",4, "Fri",5, "Sat",6) Better is %day2num= ("Sun"=>0, "Mon"=>1, "Tue"=>2, "Wed"=>3, "Thu"=>4, "Fri"=>5, "Sat"=>6) Referred to as $day2num{"wed"}

66 Hash tables Useful hash operations keys returns list of keys in hash table. Order non-deterministic.

67 Hash tables Useful hash operations keys returns list of keys in hash table. Order non-deterministic. values returns list values stored in the hash table. Order of no significance.

68 Hash tables Useful hash operations keys returns list of keys in hash table. Order non-deterministic. values returns list values stored in the hash table. Order of no significance. sort keys %table lists the keys in lexicographic order

69 Hash tables Example # read in from file while ($name = <INP>) { $reg = <INP>; $carreg{$name} = $reg; } # print out in order of name foreach $n (sort keys %carreg) { print "Car reg of $n is $carreg{$n} \n"; }

70 Hash tables Can also tell sort how to sort: sort {$a <=> $b} list sorts the list in numeric order. sort {$table{$a} cmp $table{$b}} keys %tabl sorts the keys so that corresponding hash table elements are in order

71 Hash tables #print out in order of car reg foreach $n (sort {$carreg{$a} cmp $carreg{$b}} keys %carreg) { print "$carreg{$n} owned by $n\n"; }

72 Procedures Procedures Perl has procedures but its parameter passing mechanism is poor. Suppose we have a procedure plus that adds up two numbers. Called: $x = &plus($a, $b); where $a and $b are the parameters. Call by value semantics

73 Procedures Declaring a subroutine To declare a subroutine: sub NAME BLOCK e.g. sub printhead { print "Name Age Number Balance\n"; }... &printhead();

74 Procedures Parameters Procedures have one formal The values of the actual parameter are given to the formal parameter a list local to the procedure. sub plus { sub plus { $x = $_[0]; my ($x,$y) $y = $_[1]; return $x+$y; return $x + $y; } }

75 Procedures A common idiom uses shift sub plus { $x = $y = return $x + $y; }

76 Procedures Example A procedure to add up a list: sub addlist {$sum = 0; foreach $num (@_) {$sum += $num}; return $sum;} sub proclist { my ($f1,$f2,@nums) foreach $n (@nums) {$sum = $sum+$n; } return ($f1*$f2*$sum); } $x = &addlist(1,2,3,5,6,9); $y = &proclist(3,4,1,1,0,2);

77 Procedures The following does not work (s sub dotprod { (@a,@b) = = (4,5,6); &dotprod(@x,@y);

78 Procedures Formal parameter is an unstructured list (possible danger in passing multiple lists as parameters) Forward declarations: sub addlist; Telling Perl that addlist is a procedure which will be declared elsewhere. Can have anonymous procedures that get assigned to variables. Weakness in parameter system can be overcome by using references. Default: all variables are global wherever defined. This is even recognised by the Perl community to be a Bad Thing Declare variables local inside procedures. Best way: prefix first use of variable in a procedure with with my. Lexical scoping.

79 References References References are mechanisms for a variable to refer to something, e.g. (1) another variable; (2) a piece of data; (3); a function Symbolic references Can turn a string into a variable $x = 123; $y = "x"; $$y = 5; print $x;

80 References Hard references Can use the \ operator to dereference a variable, and -> to = (10,20,30,40,50); $x = \@a; for $e ) { print "$e\n"; } print $x->[0];

81 References Call by reference sub count { my ( $fname, $rcount ) chomp $fname ; unless (-T $fname ) { return ; } open (FINP, $fname ); while (<FINP >) { $$rcount ++ } close ( FINP ); }

82 = ls -1 $ARGV [0] ; $n = 0; for $f ) { & count ($f,\ $n ); } print " Total number of lines is $n\ n";

83 References Passing multiple arrays sub dotprod { ($a,$b)=@_;... = = (4,5,6); &dotprod(\@x,\@y); Note that only scalars are passed.

84 References Refs to anonymous arrays and hashes Square brackets creates an array returns a reference $a = [ 10, 20, 30, 40]; = [ 10, 20, 30, 40]; Curly braces creates a hash returns a reference $day = {"sun"=>0, "mon"=>1, "tues"=>...}

85 Modules Modules A module is a collection of code, data structures that can be used as a library Often see it in OO programming Magic word is use use IO; To access something is in a module use ::. So, module::thing Modules can be nested.

86 Modules Example use IO :: Compress :: Bzip2 ; IO :: Compress :: Bzip2 :: bzip2 ("do.pl", "do.pl.bz

87 Modules Example use IO :: Compress :: Bzip2 ; IO :: Compress :: Bzip2 :: bzip2 Can also import specific things ("do.pl", "do.pl.bz use IO :: Compress :: Bzip2 qw( bzip2 $Bzip2Error ); unless ( bzip2 ("do.plx ", "do.pl.bz2 ")) { warn " Compression failed returning : $Bzip2 }

88 Objects Objects Data structures which contain data, know functions that can apply to them; anonymous accessed through references typically organised in classes

89 Objects use Bio :: Seq ; $seqio = Bio :: SeqIO -> new ( - format => embl, -file $seqobj = $seqio - > next_ seq (); $seqstr = $seqobj -> seq (); $seqstr = $seqobj -> subseq features = $seqobj - > get_ SeqFeatures (); foreach my $feat features ) { print " Feature ",$feat -> primary_tag, " starts ",$feat -> start, " ends ", $feat ->end," strand ",$feat -> strand,"\n"; }

90 Regular expressions Regular expressions, matching, and more Perl s regular expression support powerful concise way of describing a set of strings. Typical: a command uses a regular expression to process some argument. Example: split uses a regex to split a string $line = "Gauteng;Johannesburg GP = split("[ ;]", $line); ($prov, $capital,$reg, $pop) = split("[ ;] ", $line);

91 Regular expressions Specifying Regular Expressions Most characters stand for themselves: F Fred 6312

92 Regular expressions Specifying Regular Expressions Most characters stand for themselves: F Fred 6312 \ ( ) [ { ^ $ * +?. are metacharacters (have special meaning) escape with a backslash for the chars: Fred stands for the string (Fred)

93 Regular expressions Specifying Regular Expressions Most characters stand for themselves: F Fred 6312 \ ( ) [ { ^ $ * +?. are metacharacters (have special meaning) escape with a backslash for the chars: Fred stands for the string (Fred) To group things together, use parentheses.

94 Regular expressions Specifying Regular Expressions Most characters stand for themselves: F Fred 6312 \ ( ) [ { ^ $ * +?. are metacharacters (have special meaning) escape with a backslash for the chars: Fred stands for the string (Fred) To group things together, use parentheses. To specify alternatives, use (green red) apples stands for green apples or red apples

95 Regular expressions Specifying Regular Expressions Most characters stand for themselves: F Fred 6312 \ ( ) [ { ^ $ * +?. are metacharacters (have special meaning) escape with a backslash for the chars: Fred stands for the string (Fred) To group things together, use parentheses. To specify alternatives, use (green red) apples stands for green apples or red apples

96 Regular expressions Specifying Regular Expressions A list of characters in square brackets matches any of the characters. [YyNn] matches any of an upper or lower case y or n. [A-Za-z0-9] is all the alphanumeric characters

97 Regular expressions \n new line; \t tab; \s a whitespace;

98 Regular expressions \n new line; \t tab; \s a whitespace; \d digit; \D non-digit;

99 Regular expressions \n new line; \t tab; \s a whitespace; \d digit; \D non-digit; \w a word charater, \W a non-word character

100 Regular expressions \n new line; \t tab; \s a whitespace; \d digit; \D non-digit; \w a word charater, \W a non-word character. anything but a \n

101 Regular expressions \n new line; \t tab; \s a whitespace; \d digit; \D non-digit; \w a word charater, \W a non-word character. anything but a \n m{3} stands for mmm (map){2,3} stands for mapmap or mapmapmap m* stands for 0 or more m s

102 Regular expressions \n new line; \t tab; \s a whitespace; \d digit; \D non-digit; \w a word charater, \W a non-word character. anything but a \n m{3} stands for mmm (map){2,3} stands for mapmap or mapmapmap m* stands for 0 or more m s m+ stands for 1 or more m s

103 Regular expressions \n new line; \t tab; \s a whitespace; \d digit; \D non-digit; \w a word charater, \W a non-word character. anything but a \n m{3} stands for mmm (map){2,3} stands for mapmap or mapmapmap m* stands for 0 or more m s m+ stands for 1 or more m s Many others. Rules long and complex.

104 Regular expressions Processing with regular expressions Various commands that allow finding a pattern matching a regular expression in a string; extracting out the regular expression; substituting; or other modification e.g. matching, substitution, translation, substitution.

105 Regular expressions Splitting up input split(pat,string) Takes the input string and splits the string wherever the pattern pat occurs.

106 Regular expressions Splitting up input split(pat,string) Takes the input string and splits the string wherever the pattern pat occurs. returns a list of strings as a result can choose whether the split pattern is part of the returned string or not

107 Regular expressions Splitting up input split(pat,string) Takes the input string and splits the string wherever the pattern pat occurs. returns a list of strings as a result can choose whether the split pattern is part of the returned string or not $x = split /\* ::/, $c processes the string $c, splits it wherever a * or :: appear, and returns the split list. So, if $c="dat: 23 * Mon: 11; LgA -632 ::: LgB -217* a", $x=?

108 Regular expressions Matching m/regexpr/ returns true if the regular expression matches $_.

109 Regular expressions Matching m/regexpr/ returns true if the regular expression matches $_. The m is optional in most cases (unless slash in string) m-/apple-

110 Regular expressions Matching m/regexpr/ returns true if the regular expression matches $_. The m is optional in most cases (unless slash in string) m-/apple- To match a particular string use the binding operator: =~ $inp =~ m/(pascal) (\WC\W) (C\+\+)/;

111 Regular expressions Matching m/regexpr/ returns true if the regular expression matches $_. The m is optional in most cases (unless slash in string) m-/apple- To match a particular string use the binding operator: =~ $inp =~ m/(pascal) (\WC\W) (C\+\+)/; A number of modifiers available: i (case insensitive) g (global) Count number of matches in a string: while ($inp =~ m/(pascal) (\WC\W) (C\+\+)/g) { $wrd++ }

112 Regular expressions Extracting patterns * Can refer to and extract out subexpressions matched $1, $2,... To find a repeat m/\b(\w+)\s+\1/ To extract out while ($line = <INP>) { $line =~ m/(\w+)\s+(\w+)/; $name = $1; $reg = $2; $carreg{$name}=$reg; } Look for repeats:

113 Regular expressions Substitution s/regexpra/regexprb/ substitutes regular expression A by regular expression B. Default string processed is $. s/apple/orange/; Replaces the first occurrence of Apple in $ with Orange Similar modifiers to m: i, g,... Use the binder operator to choose the string chosen: $name = "Alan Turing"; $name =~ s/(\w)\w*/\1\./; $c = ($name =~ s/(\w)\w*/\1\./); $c = ($name =~ s/(\w)\w*/\1\./g);

114 Regular expressions Translation Use the tr or y operator. Operates on strings, rather than regular expressions. tr/range1/range2/ replaces any character in range1 and replaces it with the corresponding character in range2. $c =~ tr/a-z/a-z/ change every character in $c to uppercase. $cnt = $c =~ tr/a-z/a-z/ sets $cnt to the number of lower case letters in $c.

115 And more And more maths functions more sophisticated I/O lots of built in variables: e.g. \$NR modules, OO IPC, networking process control