Essentials for Scientific Computing: Introduction to Python Day 11 & 12

Transcription

1 Essentials for Scientific Computing: Introduction to Python Day 11 & 12 Ershaad Ahamed TUE-CMS, JNCASR May Conditionals 1.1 if Construct In python blocks of code are created by indentation. That means code within a block are separated from others by an equal amount of spaces/tabs on the left side. The standard ifelifelse construct. myint = 234 if myint % 2 == 0: print "Multiple of 2" elif myint % 3 == 0: print "Multiple of 3" elif myint % 4 == 0: print "Multiple of 4" else: print "Not multiple of 2, 3 or 4" Multiple of 2 If the condition for the first if evaluates to True, then the statements within that block are executed. Otherwise the condition for each of the elif statements is evaluated until one of them evaluates to True. If none of them are True then the else block is executed. The elif and else blocks are optional. 2 Looping 2.1 The for Loop In python the for loop actually loops over values in a sequence. A common way to loop over a sequence of integers is to use the range() function which returns a list of integers. 1

2 seq = range(10) print seq [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] for i in range(5): print "Current value of i is ", i print "twice that is", i*2 Current value of i is 0 twice that is 0 Current value of i is 1 twice that is 2 Current value of i is 2 twice that is 4 Current value of i is 3 twice that is 6 Current value of i is 4 twice that is 8 We can also loop over a list of tuples and use sequence unpacking. mylist = [("name","ershaad"), ("surname","basheer"), ("dept.", "TUE-CMS")] for i in mylist: print i ( name, ershaad ) ( surname, basheer ) ( dept., TUE-CMS ) for field, value in mylist: print field, " has value ", value name has value ershaad surname has value basheer dept. has value TUE-CMS Since strings are a sequence type, we can loop over them too. mystring = "JNCASR" for alph in mystring: print alph J N C A S R 2

3 2.2 while loop The while loop continues looping as long as a condition is true. In the while loop below, the loop continues as long as count is less than 5. count = 0 while count < 5: print count count = count break and continue The break statement will terminate the loop within which it is enclosed and the continue statement will cause the loop to skip to the next iteration. while True: inp = raw_input("enter word :") if inp == "end" or inp == "End": break elif inp == "skip" or inp == "Skip": continue else: print inp Enter word :World World Enter word :end end Enter word :word word Enter word :skip Enter word :End The loop above reads input from the terminal and writes the string that it read. It continues until the user enters the work end or End. If the user enters the word skip or Skip, the program does not print the string and instead proceeds to read new input. raw input() is a python built-in function that reads input from the keyboard and returns it as a string. The python keyword True is a built-in boolean type which always evaluates to a true value. In each iteration of the loop, input from the user is read and compared with the string End or end. If the input matches, the break keyword causes python to exit from the loop which ends the program. If the input string does not match, then it proceeds to check if it matches the string skip or Skip. If it matches, the continue 3

4 statement is executed which causes python to skip all the remaining statements within the while loop and immediately begin with the next iteration of the loop. If the input does not match either of the conditions, the else block is executed which contains the statement to print the input string Membership Testing The python in keyword can be used to check if a value is a member of a sequence type. The following examples can make this clear. mylist = ["orange", "mango", 23, 33] if "orange" in mylist: print "YES" YES if "apple" in mylist: print "YES" mystring = "statehouse" if "teh" in mystring: print "YES" YES if "the" in mystring: print "YES" 3 Python Scripts In the same way as shell scripts, python code can be placed in a file and executed. Python program files are usually named with a.py extension. In order to make python files directly executable from the shell, we add executable permissions to the file with chmod u+x filename.py and set the first line of the script to be 4 Functions A function allows us to group code into a callable subroutine. Below is a function that replaces all occurrences of character fchar to tchar in string srcstr and returns the resulting string. def subst(fchar, tchar, srcstr): result = "" for char in srcstr: if char == fchar: 4

5 result += tchar else: result += char return result mystring = "helloworld" mystring = subst( o, x, mystring) print mystring A function is defined using the def keyword follwed by the function name, and parameters in parenthesis. Unline C and FORTRAN, only parameter names are necessary and not types. Within the function, the variable name result is bound to an empty string. The for loop iterates over each character in srcstr while assigning it to the variable char. If char is not the character fchar that needs to be replaced, then char is appended to the end of result, otherwise, the character tchar is appended to result 5 Modules Below is an example of a function that returns the quadratic roots, given the coefficients. We assume that the roots are real. import math def quadroot(a, b, c): discr = (b * b) - (4 * a * c) discr_sqrt = math.sqrt(discr) root1 = (-b + discr_sqrt) / (2 * a) root2 = (-b - discr_sqrt) / (2 * a) return root1, root2 r1, r2 = quadroot(1, 3, -10) print r1, r2 In the example script above quadroot.py, we need the square root function which is not a built-in function in python (like C). We need to use a library to get that function. Libraries in python are called modules and using a module in our python program is done my simply importing the particular module. The line import math imports the module math. Once this line is executed, the functions defined in the math modules are available for use. Unlike C, the members (functions, variables, classes, etc.) of the math module are not imported into the same namespace as our main program, rather they are imported into the math namespace. Thus, in order to access any member of the math module, we need to prefix it with math.. For example the sqrt() function is called as math.sqrt(). This example also illustrates another feature of python. Python functions can return tuples. This means that multiple values can be returned and assigned to an equal number of variables. In this case, two values are returned. 5

6 After making the script file above executable with chmod, we can run it on the shell with the command line../quadroot The output will be If we run the program with the parameters to quadroot() to be 1, 3 and 10. We get the following output since the math.sqrt() function will not work for negative numbers. Traceback (most recent call last): File "./quadroot.py", line 12, in <module> r1, r2 = quadroot(7, 3, 10) File "./quadroot.py", line 7, in quadroot discr_sqrt = math.sqrt(discr) ValueError: math domain error We need to change our program so that it works for complex numbers too. We could do this my modifying our program to check for the value of the discriminant and calculate the complex solution accordingly. Python as we mentioned earlier has extensive standard library. It turns out that a complex version of the math module exists and is called cmath. All we need to do is to import cmath instead of math. But, now since the functions we need (including sqrt()) will be imported into the cmath namespace instead of the math namespace, we would have to change every occurence of math. in our program to cmath.. We can avoid this. Python supports renaming the namespace that the members of a module are imported into, the default being the same name as the module. Thus our new complex capable program is. import cmath as math def quadroot(a, b, c): discr = (b * b) - (4 * a * c) discr_sqrt = math.sqrt(discr) root1 = (-b + discr_sqrt) / (2 * a) root2 = (-b - discr_sqrt) / (2 * a) return root1, root2 r1, r2 = quadroot(1, 3, 10) print r1, r2 Only the import line has been changed. Now the cmath module which has complex versions of the same functions as the math module is imported. The members are imported into the math namespace by using the as keyword in the import line. And the output is. ( j) ( j) 6

7 6 help(), dir() and type() Within the python interpreter, we can get documentation about a built-in function or imported function using the help() built-in function. For example. help(len) Displays short documentation on the len() built-in function. Also import math help(math.sqrt) Will display short documentation on the sqrt() function defined in the math module. The dir() built-in function prints the members of a module or class. For example, in order to know what are the functions, classes and variables defined in the math module we can use. import math dir(math) [ doc, name, package, acos, acosh, asin, asinh, atan, atan2, atanh, ceil, copysign, cos, cosh, degrees, e, exp, fabs, factorial, floor, fmod, frexp, fsum, hypot, isinf, isnan, ldexp, log, log10, log1p, modf, pi, pow, radians, sin, sinh, sqrt, tan, tanh, trunc ] From the output above, we know that some of the names in the dir() output are variables and others are functions. In order to know the type of a variable, the type() built-in function can be used. type(math.asin) <type builtin_function_or_method > type(math.pi) <type float > 7 Methods The datatypes that we described earlier are in fact objects. Although that might not make sense at this moment, it means that a variable of a particular datatype has some functions that operate on the value that it contains. These functions, called methods are bound to that variable. In order to know what are the methods available for particular variable we can use the dir() built-in function which will list them. The string datatype has several methods that operate on the string value. a = "hello, world" b = "another string" dir(a) [ add, class, contains, delattr, doc, eq, 7

8 format, ge, getattribute, getitem, getnewargs, getslice, gt, hash, init, le, len, lt, mod, mul, ne, new, reduce, reduce_ex, repr, rmod, rmul, setattr, sizeof, str, subclasshook, _formatter_field_name_split, _formatter_parser, capitalize, center, count, decode, encode, endswith, expandtabs, find, format, index, isalnum, isalpha, isdigit, islower, isspace, istitle, isupper, join, ljust, lower, lstrip, partition, replace, rfind, rindex, rjust, rpartition, rsplit, rstrip, split, splitlines, startswith, strip, swapcase, title, translate, upper, zfill ] a.upper() HELLO, WORLD a.title() Hello, World a.replace("l","x") hexxo, worxd b.replace("the","yy") anoyyr string In a similar way the list datatype also supports several functions. mylist = ["orange", "apple", 23, (5,6)] mylist.append("endval") print mylist [ orange, apple, 23, (5, 6), endval ] mylist.remove("apple") print mylist [ orange, 23, (5, 6), endval ] mylist.insert(1, "mango") print mylist [ orange, mango, 23, (5, 6), endval ] print mylist.index(23) 2 print mylist.index((5,6)) 3 8 List comprehensions List comprehensions provide a concise way to write statements that produce a list. A list comprehension consists of an expression with a for statement within brackets. More for or if statments may follow the initial for. example, in order to generate a list of the first 10 powers of two using a for loop. powertwo = [] for i in range(10): 8

9 powertwo.append(2**i) print powertwo The same result can be obtained by using a list comprehension which is far more compact. powertwo = [2**i for i in range(10)] print powertwo Output will be [1, 2, 4, 8, 16, 32, 64, 128, 256, 512] If we wanted to wanted get powers of two for odd exponents between 0 and 10. powertwo = [2**i for i in range(10) if i%2!= 0] print powertwo The Output being [2, 8, 32, 128, 512] 9 File Input/Output The open() built-in function is used to open a file for reading or writing. If the open() function succeeds, it return a file object which has several methods and behaves in certain ways when used in constructs. The examples will illustrate this. We use our earlier example text file comfilens.txt. C C C C B N B N B N A program to read each line and print lines that have C in the first column would look like. import sys ifile = open(sys.argv[1], r ) for line in ifile: 9

10 cols = line.split() if "C" in cols[0]: print line, ifile.close() In the example above, we first import the module sys which contains the list argv. The elements of sys.argv are the command line parameters, with sys.argv[0] being the command itself. The first parameter to the open() function is the name of the file to open, and the second parameter is the mode which is usually "r" for reading, "w" for writing and "rw" for both. The additional mode "b" has an effect on non- UNIX platforms like Windows. open() returns a file object which is bound to the name ifile. The file object supports iteration just like Lists and other sequence types. This means the object can be used in the for loop, where in each iteration, a line from the file is read, and the resulting string is assigned to the variable name line. The split() method of the string object returns a list in which each element is a word split by whitespace. Like other methods, split() can take parameters (see help(str.split)). mystring = "This is a sentence with some words" wrdlist = mystring.split() print wrdlist [ This, is, a, sentence, with, some, words ] The first element of the list cols which is cols[0] contains the first word of the input line. If the string C matches or is a substring of the string in col[0] then the line is printed. The trailing comma after the variable name in the print statement, suppresses the printing of a newline character. We do this because a newline character is already present at the end of the string value of variable line which will be printed. The close() method of the file object closes the file. The next example shows how the write() method of the file object is used. The import sys ofile = open(sys.argv[1], w ) num = int(sys.argv[2]) a, b = 0, 1 for i in range(num): ofile.write(str(a) + "\n") a, b = b, a+b ofile.close() Running the script with./fibwrite.py fibfile.txt 10 will write the first 10 fibonacci numbers to the file fibfile.txt. 10

11 10 Examples 10.1 cwords.py: Count Occurrences of a Word This python program does the same thing as the bash shell script we had written earlier with the same name. It counts the number of occurrences of a word in the text read from standard input (stdin). The word to count is specified as a command line argument. import sys count = 0 for line in sys.stdin: for word in line.lower().translate(none,,.()"\ ).split(): if word == sys.argv[1]: count += 1 print count The for loop will be familiar from earlier examples. Looking at the inner for loop, we chain together method calls. For example the line in the program for word in line.lower().translate(none,,.()"\ ).split() first, calling the lower() method on line returns the string in line with all characters converted to lowercase. The translate() method is called on that result (which is still a string). The translate() method of a string takes two arguments, the first is a translation table for mapping one set of characters to another set, and the second argument is the set of characters to delete from the string. Since we want to delete all punctuation marks from the string and do not need to do any translation. We give the null data type None as the first argument, and the set of characters to delete as the second. The split() method returns a list of words which have been split from line separated by whitespace. The above chained method call is equivalent to writing temp = line.lower() temp = temp.translate(none,,.()"\ ) temp = temp.split() for word in temp: Since each step above returns a string and we need to call a method of that string in the next step, chaining can be used to make the calls more compact. The condition for the if statement checks if word is equal to the string supplied as the command line argument, and increments the count variable if it is os.path, glob and re Modules The program below reads each file with the extension.txt from the directory source, removes duplicate lines and writes the output to a file with the same name but with the extension changed to.out in the directory dest. 11

12 import glob import os.path import re srcfiles = glob.glob(os.path.join( source, *.txt )) for filename in srcfiles: ifile = open(filename, r ) lines = ifile.readlines() lines = set(lines) ofilename = re.sub(.txt$,.out, filename) ofilename = re.sub( ^source, dest, filename) ofile = open(ofilename, w ) for line in lines: ofile.write(line) ifile.close() ofile.close() The import lines import the modules glob, os.path and re. The module glob includes functions for expanding shell glob expressions, os.path has functions for manipulation of pathnames and re is the regular expression module. The line srcfiles = glob.glob(os.path.join( source, *.txt )) stores the names of all the files in the directory source that end with the extension.txt into a list named srcfiles. The function os.path.join() joins each argument together using the appropriate pathname separator and returns it. On a Linux system, the command written in the interactive interpreter will give the output below. os.path.join( source, *.txt ) source/*.txt We use the join() function instead of inserting the / character because join() will use the appropriate character for the platform it is running on. Thus the line above will work even if the script is run on a system running Windows. The glob() function of the glob module expands the glob expression that is given as an argument and returns the list of pathnames that it matches. The for loop iterates over each of the filenames in srcfiles and opens the file and assigns the name ifile to the file object. The readlines() method of the file object returns a list in which each element is a line from the file. lines = set(lines) The list of lines is passed as an argument to the set() built-in function which creates a set datatype from the list. Since a set cannot contain duplicates, only unique lines will be present in the set. Note that the variable name lines now references a set rather than a list. The lines below utilise the sub() function of the re regular expression module. The sub() function performs the same function as the sed substitution 12

13 s command. It takes three arguments. The first is a regular expression. The second is the replacement string. And the third argument is the string on which the substitution is to be made. sub() will substitute all matches of the regular expression with the replacement string. ofilename = re.sub(.txt$,.out, filename) ofilename = re.sub( ^source, dest, filename) Here, the.txt extension is replaced with the.out extension and the directory source is replaced with dest in the pathname referenced by variable filename. The line ofile = open(ofilename, w ) opens the filename referenced by variable ofilename and assigns the file object to the variable name ofile. The loop for line in lines: ofile.write(line) iterates over each element in the set lines which contains the unique lines and associates it with the variable name line. line is written out to the file ofile using the write() method. Finally, before the loop iterates over the next file, ifile and ofile are closed with the close() method of the file object. ifile.close() ofile.close() 10.3 SciPy Package The SciPy package is an extensive scientific computing library for python. Among the many subpackages, it includes Discrete Fourier Transform algorithms, sparse and dense linear algebra, integration routines, signal and image processing tools. Documentation for each of the modules in the SciPy package is available using the python help() built-in function. For a list of packages in SciPy use help scipy after loading the SciPy module using import scipy. The SciPy packages is built upon and uses the NumPy library which implements fast multidimensional array types in python. The arrays are faster than native python lists and support array operations. For example the multiplication operator * between a NumPy array and a scalar return an array with each of the elements multiplied by the scalar. The same operation on a list will return a list with twice the length and its elements repeated. A package related to SciPy and NumPy is matplotlib. Matplotlib is a python library that provides 2D plotting capabilities. Matplotlib can produce publication quality images. It provides an interface to the user called pyplot which is in many respects similar to that provided by MATLAB R. The following example illustrates some of the capabilities for scientific computing that the combination of NumPy, SciPy and matplotlib can provide. The example creates a time series signal consisting of three summed sinusoids of different frequencies. The SciPy package fftpack is used to calculate the Discrete Fourier Transform of the signal. Matplotlib is used to plot the time series signal and its frequency domain representation. 13

14 import scipy as sp import matplotlib.pyplot as plt import scipy.fftpack as fft taxis = sp.linspace(0, 2*sp.pi, 16384) sig = sp.cos(100*taxis) + sp.cos(400*taxis) + sp.cos(1000*taxis) plt.subplot( 211 ) plt.plot(taxis, sig) SIG = fft.fft(sig) SIG = fft.fftshift(sig) SIG = abs(sig) faxis = sp.arange(-8192, 8192, 1) plt.subplot( 212 ) plt.plot(faxis, SIG) plt.xlim(-1200, 1200) plt.show() We take a closer look at the lines in the program. The lines import scipy as sp import matplotlib.pyplot as plt import scipy.fftpack as fft each import the modules scipy, matplotlib.pyplot and scipy.fftpack. Instead of using the default namespace to load the modules into, which is the same name as the module, we specify shorter names for convenience. The scipy module is loaded into the namespace sp. Module matplotlib.pyplot is loaded into namespace plt. And scipy.fftpack is loaded into namespace fft. We can now use the namespace names that we have chosen to access members of each of the modules. The line taxis = sp.linspace(0, 2*sp.pi, 16384) uses the NumPy (which is included in the SciPy module) function linspace(). linspace() returns a sequence of numbers between an interval. It takes three arguments. The first is the start of the sequence, the second is the end, and the third argument is the number of entries in the sequence. linspace() returns a NumPy array. The pi member of the scipy module is a constant with the value of π. The line above will generate an array of numbers beginning from 0 and ending with 2π. The result of the statement above can be seen in the interactive interpreter. taxis = sp.linspace(0, 2*sp.pi, 16384) print taxis [ e e e-04, e e e+00] 14

15 Next, we look at the line. sig = sp.cos(100*taxis) + sp.cos(400*taxis) + sp.cos(1000*taxis) In this line we use the cos() function defined in the SciPy module. The math module also provides the cos() function but we cannot use it here. Notice that the argument to the cos() function is an array which is multiplied by a scalar. Thus we are passing an array as the argument to cos(). The cos() function in the math module only supports scalar arguments. The cos() function in the SciPy module accepts array arguments. In this case it applies the cos() function to each element of the array and returns the resulting array. Thus the line above creates an array which is the sum of three discrete sinusoids and associates the name sig with it. The next two lines are matplotlib functions. The pyplot interface design is such that when each pyplot function is called, a change is made to the current figure. That is, pyplot is stateful and remembers the change that each pyplot function make. Once the current figure is in the state that is desired, it can be finally displayed. plt.subplot( 211 ) plt.plot(taxis, sig) The subplot(211) function tells matplotlib to create two plots arranged as 2 rows and 1 column and to select the first among them as the current plot. If we wanted 6 plots arranged as 3 rows and 2 columns and the 4th one as the current plot, we would use subplot(324). All subsequent pyplot functions affect the current plot. In our case the next line creates a line plot consisting of x-axis values in array taxis and the corresponding y-axis values in array sig. The next line uses the fft function defined in the scipy.fftpack module. SIG = fft.fft(sig) fft.fft(sig) returns the Discrete Fourier Transform of the array sig. The result is an array of complex numbers. The array returned by the fft() function is in frequency space. In the example, SIG[0] will contain the zero-frequency term. SIG[1] to SIG[8191] will contain the positive frequency terms and SIG[8192] to SIG[16383] will have negative frequency terms. We would like the frequency domain array to be centred at the zero frequency term with negative terms on the left and positive terms on the right. The fftshift() function returns the centred frequency domain array. SIG = fft.fftshift(sig) The next line SIG = abs(sig) converts each of the complex values in SIG to its absolute value or magnitude. We need to generate the values of the x-axis so that they match the centred time domain array. We do this with the line. faxis = sp.arange(-8192, 8192, 1) 15

16 The arange() function returns an array whose first value is and last value is 8191 with the difference of 1 between consecutive values. This corresponds with the frequencies of the coefficients in the frequency domain array SIG. The next lines are similar to what we have seen earlier. The subplot() function sets the second plot as the current plot. The plot() function generates a line plot with the frequencies contained in faxis as the x-axis and the corresponding coefficients contained in SIG as the y-axis. The function call xlim(-1200, 1200 tells matplotlib to limit the displayed range of the x-axis for subplot 2 to the values between and 1200, which is the frequency range we are interested in. plt.subplot( 212 ) plt.plot(faxis, SIG) plt.xlim(-1200, 1200) Finally the pyplot function plt.show() displays the all the plots on the output device, which is the screen in our case. 16