Introduction to Functional Programming and Scripting (Exam Revision) John Woodward

Transcription

1 Introduction to Functional Programming and Scripting (Exam Revision) John Woodward

2 Functional Programming Immutable, Stateless (good news?) Function are first class objects (higher order functions). Programs in functional languages are generally shorter, easier to understand, design, debug, and maintain, than imperative counterpart. Modern functional languages are strongly typed. guarantees no type errors at runtime (trapped by the compiler). Type Inference Typically recursion is used in functional programming languages, iteration is used in imperative languages.

3 State & Referential Transparency Java object store state information (instance variables). E.g. in a constructor Person( John, 45, UK, Lecturer ). This stores Name, Age, Nationality, Job Print Function_A(x) Print Function_B(x) Print Function_A(x) (side effects).

4 Data Type and Type Signatures In maths a function f maps between two sets A and B type signature A->B (domain and codomain, input and output) type signatures. (Boolean, real, float, natural, integers, ) B,R, F, N, Z, (a pair of Booleans) BXB or B^2 Type inference can be used to infer facts about types. e.g. composition - theorems for free

5 Higher Order Functions (HOF) In most programming languages we pass around integers, Booleans, strings, as argument to function and return types For example Boolean isprime(integer n) Takes an integer and returns true/false depending on if it is prime or not. With functional languages we can pass around functions. Type signatures

6 Naming a String String name = John Print(name);//will print John Or we can just print the name directly Print( John ) If we only use John once we could use a string literal (a one-off use). If we use John multiple times define a variable name and use that.

7 Anonymous Functions/lambda (PYTHON) def f (x): return x**2 print f(8) Or we can do g = lambda x: x**2 print g(8) Or just do it directly print (lambda x: x+2) (4) y = (lambda x: x*2) (4) print y

8 Anonymous Functions (SCALA) println(((a: Int) => a + 1)(4)) An increment function with the argument 4 println(((a: Int, b: Int) => a + b)(4, 6)) The addition function with the arguments 4 and 6.

9 Lambda filter, map, reduce (PYTHON) nums= [2, 18, 9, 22, 17, 24, 8, 12, 27] print filter(lambda x: x % 3 == 0, nums) #[18, 9, 24, 12, 27] print map(lambda x: x * , nums) #[14, 46, 28, 54, 44, 58, 26, 34, 64] print reduce(lambda x, y: x + y, nums) #139 (more detail in a few slides)

10 Lambda filter, map, reduce (SCALA) nums = [2, 18, 9, 22, 17, 24, 8, 12, 27] println(nums.filter((x: Int) => x % 3 == 0)) //[18, 9, 24, 12, 27] println(nums.map((x: Int) => x*2+10)) //[14, 46, 28, 54, 44, 58, 26, 34, 64] println(nums.reduce((a:int, b:int)=>a+b)) #139 (more detail in a few slides)

11 Simple example: Higher order Function (Python) Takes a function f and value x and evaluates f(x), returning the result. def apply(f, x): return f(x)

12 Simple example: Higher order Function (Scala) //Takes a function f and value x and //evaluates f(x), returning the //result. def apply(f:int=>int, x:int):int = { f(x) }

13 f(x) and f In maths f and f(x) are different. f is a function! f(x) is the value of f at value x Never say the function f(x) Say - The function f that takes a variable x (i.e. f takes var x) Or the function f at the points x (i.e. f given x has the value??)

14 Example: A linear function (PYTHON) #return a function #note return result, not result(x) def linear(a, b): def result(x): return a*x + b return result mylinearfunction = linear(4,3) #make a function 4*x+3 print mylinearfunction(8) print apply(mylinearfunction, 8)

15 Example: A linear function (SCALA) //return a function def linear(a: Double, b: Double): Double => Double = { x:double => a*x + b }mylinearfunction = linear(4,3) //make a function?*?+? println(linear(2,1)(9))

16 Summation (PYTHON) i=10 i=1 In maths f(x) = f(1)+f(2)+ +f(10) takes 3 arguments, and upper and lower bound and a function to sum over. i=10 e.g. i=1 2x = =2+4+6=12 def sum(f, a, b): total = 0 for i in range(a, b+1): total += f(i) return total

17 Summation (SCALA) i=10 i=1 In maths f(x) = f(1)+f(2)+ +f(10) takes 3 arguments, and upper and lower bound and a function to sum over. i=10 e.g. i=1 2x = =2+4+6=12 def sum(f: Int => Int, a: Int, b: Int): Int = { if (a > b) 0 else f(a) + sum(f, a + 1, b) }

18 Some simple functions def timesone(x): return 1*x def timestwo(x): return 2*x def timesthree(x): return 3*x def apply(f, x): return f(x)

19 Returning a Function (PYTHON) def compositionfunction(f, g): def result(x): return f(g(x)) return result myfunction = compositionfunction(timesthree, timestwo) print myfunction(4) print apply(compositionfunction(timesthree, timestwo), 4) print (lambda x, y :compositionfunction(x,y)) (timesthree, timestwo) (4)

20 Returning a Function (SCALA) def compositionfunction(f: Int => Int, g: Int => Int): Int => Int = { x: Int => f(g(x)) }

21 Maximum of two function (PYTHON) We can find the max of two functions (PICTURE) def maximumvalue(f1, f2, x): return max(f1(x), f2(x)) #The inputs are??? #The output is??? #what is the data-type signature #Can we return a function? #What is the signature of the returned function

22 Returning the Max Function (PYTHON) def maxfunction(f1, f2): def maxfun(x): return max(f1(x), f2(x)) return maxfun biggerfun = maxfunction(fun1, fun2) print biggerfun(2)

23 Max Function (SCALA) def maxfunction(f: Int => Int, g: Int => Int): Int => Int = { x: Int => { if (f(x) > g(x)) f(x) else g(x) } }

24 Partial Application Motivating example - diagram Addition (+) takes two arguments (arg1 + arg2) What if we only supply one argument? We cannot compute e.g. (1+) 1+WHAT But (1+) is a function of one argument (1+ could be called what???)

25 Inc as partial add (PYTHON) #add (2 args), inc(x)=add(1,x) #inc is a special case of add from functools import partial def add(a,b): return a+b inc = partial(add, 1) print inc(4)

26 Inc as partial add (SCALA) def add(a:int, b:int):int = { a+b } val inc = add(1, _: Int) def incfun: Int=>Int = { x: Int => add(x,1) } print(incfun(4))

27 double as partial mul (PYTHON) #mul(2 args), double(x)=mul(2,x) #double is a special case of mul from functools import partial def mul(a,b): return a*b double = partial(mul, 2) print double(10)

28 referential transparency Haskell is a pure functional language referential transparency - the evaluation of an expression does not depend on context. The value of an expression can be evaluated in any order (all sequences that terminate return the same value) (1+2)+(3+4) we could reduce this in any order. In the 2nd world war, Richard Feynman was in charge of making calculation for the atomic bomb project. These calculations were done by humans working in parallel. e.g. calculate exponential

29 Map, Filter, Reduce Let us now look at 3 higher order functions which are used a lot in functional programming. First we will look LISTs Then we will look at Map, Filter and Reduce.

30 Map - Scala Map applies a function to each element in a list var nums: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9) println(nums.map(x => x * x * x)) Will print what???

31 answer List(3, 4, 5, 6, 7, 8, 9, 10, 11)

32 Filter (Exercise) Scala Filter pass only elements of the list which pass a test var nums: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9) println(nums.filter(x => x % 2 == 1)) println(nums.filter(x => x % 3 == 2)) println(nums.filter(x => (x % 2 == 0) (x % 3 == 2)))

33 Filter (answers) Scala println(nums.filter(x => x % 2 == 1)) List(1, 3, 5, 7, 9) println(nums.filter(x => x % 3 == 2)) List(2, 5, 8) println(nums.filter(x => (x % 2 == 0) (x % 3 == 2))) List(2, 4, 5, 6, 8)

34 How does reduce work? //what does the following print def add(x: Int, y: Int): Int = { println(x + " " + y) x + y } nums = List(1, 2, 3, 4) println(nums.reduce(add)) //=1*2*3*4

35 Answer - add //try with mul (*), sub(-) def mul(x: Int, y: Int): Int = { println(x + " " + y) x * y }

36 Answer - mul var nums: List[Int] = List(1, 2, 3, 4) println(nums.reduce(mul))

37 Answer sub var nums: List[Int] = List(1, 2, 3, 4) println(nums.reduce(sub))

38 fold Fold does the same as reduce except we can pass in a starting value. var nums: List[Int] = List(1, 2, 3, 4) println(nums.reduce(add)) println(nums.fold(1)(add)) println(nums.reduce(mul)) println(nums.fold(1)(mul)) println(nums.fold(0)(mul))

39 Fold - answers

40 combinations println( nums.map(x=>x+2).filter(x => x % 2==0)) println( nums.map(x=>x+3).fold(0)(add)) Answers List(4, 6) 22

41 SCRIPTING LANGUAGES

42 Scripting Languages

43 Basics proper programmers write commands E.g. mkdir john1 rather than clicking on icons If you write a (set of) command more that once you can put them in a file and you do not need to write them again. Two of more, use a for (Dijkstra) Interpreted not compiled (like java) Automate the execution of tasks Glue or high level languages.

44 Scripting language 1 Scripting languages are designed to automate frequently used tasks that usually involve calling or passing commands to external programs. Those that are interpretive are often called scripting languages. In the following slides (py means python and sh means bash)

45 Scripting language 2 Recently, many applications have built-in traditional scripting languages, such as Perl or Visual Basic, but there are quite a few "native" scripting languages still in use. Many scripting languages are compiled to bytecode and then this (usually) platform independent bytecode is run through a virtual machine (compare to Java).

46 Examples of Scripting Langauges Bash (unix/linux) Cygwin (windows freely downloadable) Python (adopted by BIG DATA communities) perl "script" often used for small programs (up to a few thousand lines of code) text-processing languages sed and AWK Nothing special about scripting languages

47 What you see

48 Working with Files cp <filename> <new filename> copy - Make a copy of a file cp -R <directory> <new directory> Make a copy of a directory mv <filename> <new filename> move - Move or rename a file rm <filename> remove - Delete a file 48

49 Some Bash/cygwin Commands 2 echo "hi" # prints hi to screen echo "hi" > file.txt #writes hi to file.txt (overwrite) echo "bye" >> file.txt #writes bye to file.txt (appends) read usermessage #asks user for message head -2 file.txt #prints first two lines of file.txt tail -2 file.txt #prints last two lines of file.txt

50 Spaces and $ (values of) (sh) todo jrw@tambala ~ $ x= 8 -bash: 8: command not found jrw@tambala ~ $ x=8 jrw@tambala ~ $ echo $x 8 jrw@tambala ~ $ echo x x jrw@tambala ~ $

51 Input/Output (py) userinput = raw_input( Type in a any input') print You typed "+userinput Gives output Type in a any input: this is my input You typed: this is my input #cf. java

52 Exec command (py, sh) Many scripting languages treat the programs as text strings. Therefore you can construct a string and execute it. mycmd = "print 'hi'" exec mycmd

53 Eval (py) userequation= raw_input('type in a simple maths equation in one variable x e.g. x+1') print "you typed "+userequation x = raw_input('type in an integer value for x') print "x is "+ x x = int(x) y = eval(userequation) print y

54 Improved (py) userequation= raw_input('type in a simple maths equation in one variable x e.g. x+1') print "you typed "+userequation x = raw_input('type in a positive integer value for x: ') print "x is "+ str(x) if (x.isdigit()):#chech is it a positive integer x = int(x) y = eval(userequation) print y else: print "you did not enter a number"

55 Example output type in a simple maths equation in one variable x e.g. x+1: x**2+1 you typed x**2+1 type in a positive integer value for x: 3 x is 3 10

56 Question How easy is that to do in java? Could you do it? Can the program be improved? E.g. how do we know the user types in a mathematical expression??? What is dangerous about the exec command?

57 More Scripting A program (java, python) is just a text file. We can manipulate text files easily (with python, sh) Therefore we can manipulate programs themselves. This is my research areas ask me for details

58 Summary Scripts are interpreted. Good for repeated tasks. We covered some basic commands. You can use them to glue other programs together

59 Regular Expressions (in Python)

60 Python or Egrep We will use Python. In some scripting languages you can call the command grep or egrep egrep pattern file.txt E.g. egrep ^A file.txt Will print all the line of file.txt which start with (^) the letter A (capital A)

61 Regular expression (abbreviated regex or regexp) a search pattern, mainly for use in pattern matching with strings, i.e. "find and replace"- like operations. Each character in a regular expression is either understood to be a metacharacter with its special meaning, or a regular character with its literal meaning. We ask the question does a given string match a certain pattern?

62 List of Meta characters. a single character + one or more of the preceding element? zero or one of the preceding element * zero or more of the preceding element ^ Matches the starting position within the string $ Matches the ending position of the string or the position just before a string-ending newline []Matches a single character that is contained within the brackets [^...] exceptions. [^b]at matches all strings matched by.at except "bat". or () grouping {m,n} Matches the preceding element at least m and not more than n times

63 -v option egrep -v pattern file.txt -v is a extra option It inverts the matching, and matches everything NOT matched by the pattern.

64 e.g. Username /^[a-z0-9_-]{3,16}$/ Starts and ends with 3-16 numbers, letters, underscores or hyphens Any lowercase letter (a-z), number (0-9), an underscore, or a hyphen. At least 3 to 16 characters. Matches E.g. my-us3r_n4m3 but not th1s1swayt00_l0ngt0beausername

65 e.g. Password /^[a-z0-9_-]{6,18}$/ Starts and ends with 6-18 letters, numbers, underscores, hyphens. Matches e.g. myp4ssw0rd but not mypa$$w0rd

66 e.g. Hex Value /^#?([a-f0-9]{6} [a-f0-9]{3})$/ Starts with a +/- (optional) followed by one or more Matches e.g. #a3c113 but not #4d82h4

67 e.g. String that matches: String that doesn't match: (TLD is too long)

68 Match n characters egrep.exe "^...$" data1.txt Will match any line with exactly 3 characters ^ starts with. And contains (i.e. 3 characters) $ ends with Or just egrep.exe "^.{3}$" data1.txt What about egrep.exe "(..){2}" data1.txt?