Python for Beginners. John B. Johnston Bioinforma3cs Engineer

Transcription

1 Python for Beginners John B. Johnston Bioinforma3cs Engineer

2 Python What is it? Python is an interpreted, interac3ve programming language that incorporates external reusable modules. Python is widely used by biologists for data manipula3on, analysis, and visualiza3on.

3 Python Why for biology? More readable than languages such as Perl Rapid applica3on development Many external libraries/modules available Highly scalable Incorporates advanced design features Free and widely available

4 Running Python Programs Python can be run interac3vely inside of the Python shell where commands can be issued. To invoke the interac3ve shell use the command python (no quotes).

5 Running Python Programs Python can be encapsulated in a text file and run outside the interac3ve shell. Save your code to a file, use python./<progname>

6 hello.py

7 Exercise #1 Open the python interac3ve shell Using the print statement, have python print out some message to your screen

8 Variables and Types Not sta3cally typed Numbers integers and floa3ng point (7, 16.23, 9.0, 1254, 6.7E- 03) Strings defined with single or double quotes ( Hello, Hello ) Variable assignment: someint = 77 thefloat = 63.2 mystring = hello QB- 826

9 Python Keywords print print to console while flow control (e.g. loops) for iterate over collec3on of items if, elif, else condi3onal control is tests for object iden3ty not, and, or boolean truth tests class create user defined objects import, from, as external module control break, con3nue interrupt flow control Not a comprehensive list!

10 Common Data Structures Lists a simple collec3on of data objects. Can be mul3- dimensional. mylist = [1,2,3,4,5,6] print mylist[2] 3 print mylist[0] 1 print mylist[-1] 6 Each element in the list can be accessed by an index that begins at 0

11 Common Data Structures Tuples these are similar to lists, but are immutable (lists however, are mutable). mytuple = 100, 2300, 75, bingo print mytuple[0] 100 print mytuple (100, 2300, 75, bingo )

12 Common Data Structures Sets collec3ons of unordered data objects with no duplicate elements. These can be used like sets basket = [ apple, orange, banana ] techcos = [ google, apple, dell ] fruit = set(basket) IT = set(techcos) fruit & IT (intersection) set([ apple ]) fruit IT (union) set(['google', 'apple', 'orange', 'banana', 'dell'])

13 Common Data Structures Dic3onaries a mapped collec3on of unique key- value pairs phonenums = { jack : , jenny : ] phonenums[ jenny ]

14 Common Data Structures We can also create complex structures, for example: Lists of lists Dic3onaries of lists Lists of tuples, etc. We can use built- in operators on these basic structures to create other structure forms, like: Stacks Queues Dequeues, etc.

15 Math in Python - Operators Command Opera,on Example Output + Addi3on Subtrac3on * Mul3plica3on 7*3 21 / Division 19/3 6 % Remainder/Modulo 19%3 1 ** Exponent 2**4 16 *Not a comprehensive list!

16 Exercise #2 Try some simple math in python: * 8 20 / / 3 20/3.0 No3ce anything interes3ng about the last 3 expressions?

17 Number Types & Conversion Python converts numbers in an expression containing mixed types. This can lead to unexpected results, so be careful! Integers (1, 555, 32, 7) Floats (4.56, , 122.0) Complex (2E- 09, 3.4E10) int(x) float(x) complex(x)

18 Built- in Func3ons Func,on Descrip,on sqrt(x) Square root of x where x > 0 round(x,n) min(x1, x2, x3..) max(x1, x2, x3 ) log10(x), log(x) x rounded to n digits Smallest value in the list Largest value in the list Log base 10 and natural log, respec3vely random() Generate a random float between 0-1, exclusive of 1 shuffle(lst) cos(x), sin(x), tan(x), etc. pi, e Randomize items in list in- place Various trigonometric func3ons Mathema3cal constants Usually prefaced with math.func(). E.g. math.sqrt(3) Not a complete list

19 Python Statements A line of instruc3on in Python is called a statement Single line statements: mycolor = blue thesum = A mul3- line statement: X = \ \ Mul3ple statements in a line: a = 1; b=2; sum = a+b

20 Exercise #3 Try execu3ng a statement: total = ((40/5) + 10) * 8 How can you see the value of total? What is the quan3ty of total?

21 Indenta3on in Python Python uses colons (:) and indenta3on to define blocks of code. Code blocks might include func3ons, the body of a loop, or statements to execute if a condi3onal evaluates to True or False. Indent using tabs or whitespace, as much as you like so long as it is consistent in the block Indenta3on also makes the code more readable

22 Indenta3on Example Here is a loop that uses indenta3on to define its code block: for i in range(1,11): print(i) if i == 5: break How many levels of indenta3on do we have?

23 Exercise #4 Try the indenta3on example below Note the change in the shell prompt Use whitespace or tabs to achieve the right level of indenta3on Hit return on blank line to exit block for i in range(1,11): print(i) if i == 5: break

24 Other Kinds of Operators We ve already seen and used mathema3cal operators, but there are others Assignment operators Comparison operators Logical operators Membership operators Bitwise operators

25 Assignment Operators Operator Descrip3on Example = Assign value resolved from right operands to operand on le += Adds right operand to le and assigns to le operand - = Subs right operand from le and assigns to le operand *= Mul3plies right operand with le and assigns result to le operand **= Right operand exponent to le, assign to le operand /= Divide le operand with right, assign to le C = A + B C += 1 (C = C + 1) C - = 1 (C = C 1) C *= 7 (C = C * 7) C **= a (C = C**a) C /= b (C = C/b)

26 Comparison Operators Operator Descrip3on Example == TRUE if operands ARE equal (2 == 4) is false!= TRUE if operands are NOT equal (2!= 4) is true <> TRUE if operands are NOT equal (2 <> 4) is true > TRUE if le operand is greater than right (2 > 4) is false < TRUE if le operand is less than right (2 < 4 ) is true >= TRUE if le operand is greater than or equal to right <= TRUE if le operand is less than or equal to right (2 >= 4) is false (2 <= 4) is true (2 <= 2) is true

27 Logical Operators Operator Descrip3on Example and TRUE if BOTH operands are true (a and b) is true or TRUE if EITHER operand is true (a or b) is true not Reverses the logical state of the operands not(a and b) is false

28 Membership Operators Operator Descrip3on Example in TRUE if BOTH operands are true (a and b) is true not in TRUE if EITHER operand is true (a or b) is true A = 10 B = 20 list = [10, 2, 300, 4, 50] If (a in list): print a is available in the list else: print a is NOT available in the list

29 Python Condi3onals

30 Exercise #5 Lets use that operator example as a condi3onal Vary the value of A, test B A = 10 B = 20 list = [10, 2, 300, 4, 50] If (A in list): print A is available in the list else: print A is NOT available in the list

31 A = 10 B = 20 list = [10, 2, 300, 4, 50] If- Elif- Else If (A in list) and (B in list): print A and B are available in the list elif (A in list): print A is available in the list elif(b in list): print B is available in the list else: print Neither A or B are in the list

32 Python Loops

33 Major Loop Types while loop code block is repeated while a given condi3on is TRUE. Test at top of loop for loop code execu3on is repeated according to an iterator value that is updated in the loop block Nested loops loops may be nested inside one another

34 Loop Controls break terminates loop and pops control out of block con3nue causes loop to skip statements that follow it and return to top of block pass does nothing but provide a syntac3c placeholder

35 For Loop Form Commonly used to iterate over collec3ons (e.g. lists) mylist = [a, b, c, d, e] for item in mylist: print item

36 Exercise #6 Given this list of numbers, construct a for loop to sum those numbers and print the result: numbers = [6,5,3,8,4,2,5,4,11] What is your answer (no chea3ng)?

37 Exercise #7 Try inpung the following code: for val in "string": if val == "i": break print(val) print("the end") What was your result? What happened there?

38 For Loops and Range The range func3on generates lists of numbers that can be used as- is or to access indices. for number in range(6): print number

39 For Loops and Range Using two numbers, range will count up from the first number (inclusive) to the second (exclusive) for number in range(2,6): print number With three, it counts from first to second in a step size given by the third for number in range(2, 14, 4): print number

40 While Loop Example # take input from the user n = int(input("enter n: ")) # initialize sum and counter sum = 0 i = 1 while i <= n: sum = sum + i i = i+1 # update counter # print the sum print("the sum is",sum)

41 Python Strings The manipula3on of strings is vitally important to bioinforma3cians Python does not have a character type, it treats these as strings of length 1 Strings in Python can also be thought of as lists of characters and can be accessed/manipulated in much the same way

42 String Processing Examples Case opera3ons: >>> dna = 'CTGACCACTTTACGAGGTTAGC >>> dna CTGACCACTTTACGAGGTTAGC >>> dna.lower() ctgaccactttacgaggttagc Accessing string elements: >>> dna[0] 'C' >>> dna[1] 'T' >>> dna[2] 'G' >>> dna[-1] C'

43 String Processing Examples Given the GFF file excerpt: chri SGD gene ID=YAL049C;gene=AIM2 chri SGD gene ID=YAL048C;gene=GEM1 chri SGD gene ID=YAL040C; gene=cln3 Suppose each line is stored as a string in the list mygff >>> print mygff[0] chri SGD gene ID=YAL049C;gene=AIM2 >>> print mygff[1] chri SGD gene ID=YAL048C;gene=GEM1

44 String Processing Examples Spling a string: >>> mygff[0].split() ['chri', 'SGD', 'gene', '51855', '51855', '.', '- ', '.', 'ID=YAL049C;gene=AIM2'] String split based on whitespace Non- whitespace elements of string become items in a list Now lets save our split opera3on output in a new variable: >>> recordone = mygff[0].split() >>> recordone ['chri', 'SGD', 'gene', '51855', '51855', '.', '- ', '.', 'ID=YAL049C;gene=AIM2'] >>> recordone[- 1] 'ID=YAL049C;gene=AIM2'

45 String Processing Examples Using this technique, we can parse out the bits we re interested in, say chromosome, posi3on, and gene name: >>> recordone ['chri', 'SGD', 'gene', '51855', '51855', '.', '- ', '.', 'ID=YAL049C;gene=AIM2'] >>> recordone[- 1] 'ID=YAL049C;gene=AIM2' >>> geneone = recordone[- 1].split(';') >>> geneone ['ID=YAL049C', 'gene=aim2'] >>> geneonename = geneone[- 1].split('=') >>> geneonename ['gene', 'AIM2'] >>> NewRecord = recordone[0] + " " + recordone[3] + " " + recordone[4] + " " + geneonename[1] >>> NewRecord 'chri AIM2'

46 Other String Operators string[2:5] return slice of string (first index inclusive, last, exclusive) len(string) return the string length string.isdigit() - returns TRUE if all characters are digits string.isalpha() returns TRUE if all characters are alphabe3c string.replace(old, new) replace all occurrences of substring old with substring new string.find(sub) find the substring sub in the string, return lowest index posi3on if found, otherwise - 1. string.count(sub) count number of occurrences of substring in string Many others

47 String Processing Observa3ons Using these or similar techniques we can see how we can parse targeted informa3on Combined with a loop structure to operate record- by- record, and a data structure for storage, we can create a complete, reduced data set Useful for analysis, visualiza3on, crea3ng databases, etc. There are many string func3ons, and many ways to use them. This is just one example.

48 Exercise #8 Use our collec3on of string func3ons to calculate the GC content in percent, of the DNA sequence: TAGGCTAATGCTAGAATCCATGATTA (or even make up your own)

49 Exercise #8 One Solu3on >>> dna = "TAGGCTAATGCTAGAATCCATGATTA" >>> ((dna.count("g") + dna.count("c")) / float(len(dna))) * >>>

50 Exercise #9 Let s pull together a few things we ve learned about loops, condi3onals, and strings. You are given the following code: protein = vlswadktnvvs mylength = len(protein) Construct a for loop that iterates over protein and prints out every THIRD residue (star3ng with the very first residue), unless that residue is w ; in which case, print out Tryptophan.

51 Exercise #9 One Solu3on >>> protein = "vlswadktnvvs >>> mylength = len(protein) >>> for residue in range(0, mylength, 3):... if protein[residue] == "w":... print "Tryptophan"... else:... print protein[residue]... v Tryptophan k v

52 Python Func3ons A self- contained block of organized, reusable code used to perform a par3cular single func3on. Provide modularity Improve readability Easier to maintain Reduce code redundancy

53 General Func3on Form def myfunction( parameters ): do something here something else.. return [expression] myval = myfunction(someparam)

54 An Applica3on of Func3ons Recall our exercise where we computed GC content? Imagine we wanted to perform that func3onality on many records. What would be the advantages of encapsula3ng that as a discrete func3on?

55 Exercise #10 Create a func3on to calculate GC content, and invoke it, by passing a DNA sequence to the func3on. Feel free to reuse your code from Exercise #8.

56 Exercise #10 One Solu3on >>> def mygc(dna):... GCcount = ((dna.count("g") + dna.count("c")) / float(len(dna))) * return (GCcount)... >>> mygc("taggctaatgctagaatccatgatta") >>>

57 Func3ons and Scope Recall my solu3on from Exercise #10: >>> def mygc(dna):... GCcount = ((dna.count("g") + dna.count("c")) / float(len(dna))) * return (GCcount) What happens if I try to print the variable GCcount outside the func3on block? >>> print GCcount Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'GCcount' is not defined >>>

58 Func3ons and Scope Why did that error happen? Values assigned or declared in a func3on block are only accessible in that block. This is local scope Values assigned or declared in the main program block have global scope and can be accessed by all func3ons once declared

59 Improving our Func3on >>> def mygc(dna):... GCcount = ((dna.count("g") + dna.count("c")) / float(len(dna))) * return (GCcount)... >>> mygc("taggctaatgctagaatccatgatta") >>> What happens if I pass a lower or mixed case sequence to mygc? What about the precision of my return value? How can I make my func3on more flexible, say, to count whatever I want (AT, A, G, T, C, etc.)?

60 Improving our Func3on >>> def mygc(dna, substr):... GCcount = 0... for base in substr:... GCcount += dna.upper().count(base.upper())... GCcount = (GCcount / float(len(dna))) * return str(round(gccount,2))... >>> mygc("taggctaatgctagaatccatgatta", "AT") '65.38' >>> mygc("taggctaatgctagaatccatgatta", "GC") '34.62 >>> mygc("taggctaatgctagaatccatgatta", "gc") '34.62' >>> mygc("taggctaatgctagaatccatgatta", "at") '65.38' >>> mygc("taggctaatgctagaatccatgatta", "t") '30.77 >>> mygc("taggctaatgctagaatccatgatta", "gcat") '100.0'

61 Escaping the Shell All of the code we ve worked with in the interac3ve shell could have been saved in a file and executed as a standalone program. To write code, you need to use a text editor First exit the Python interac3ve shell using either: quit() Cntl- d

62 Nano A simple text editor If you are familiar with a par3cular Linux text editor, feel free to use it For everyone else, let s use nano Start the editor by issuing the command: nano myprog.py This will open a new file myprog.py for edi3ng.

63 Crea3ng a Program The first thing we need to add to our program is the shebang. The shebang tells the OS what programming language we re using: #!/usr/bin/env python The /usr/bin/env invokes the env command which determines the currently ac3ve copy/version of python. We could also simply use: #!/usr/bin/python

64 Exercise #11 For this exercise, we simply want to complete our simple program and run it. For your program: Create a variable DNA and assign a string value of bases to it. Create a simple for loop that iterates through the string and prints each leƒer to the screen. At the very end of the program, print the single statement END! When you re finished, save the program, and run it using python./mprog.py

65 Notes on Exercise #11 To run the program without using the python, set the executable bits on the file: chmod u+x myprog.py Then run the program simply with:./myprog.py

66 Interac3ve User Input You can add user input prompts very easily to the program you just created User input will wait un3l a value is entered mydna = raw_input( Enter a DNA sequence ) This will store the value entered into the variable mydna

67 Exercise #12 Reopen your file from Exercise #11 Replace your hard- coded statement for the value of dna with the interac3ve user input statement Save and rerun your program to test

68 Working with Files You can also use files for input or output for your Python program To access a file, you need an new object, called a file handle myhandle = open( /users/somebody/file.pdb, r ) This specifies the file path/name, and in this case, that it should be opened for reading

69 Working with Files Once the file handle has been created and opened, you can access the elements inside. One way: for item in myhandle: print item Another reads each line one at a 3me: myhandle.readline() Or read it all at once: myhandle.readlines()

70 Working with Files Wri3ng to files is very similar. First open a file handle: myhandle = ( /users/someuser/output.txt, w ) Then do your write: myhandle.write( This is my output file\n ) somedata = myhandle.write(somedata) Close your files when done: myhandle.close()

71 Exercise #13 First, copy this PDB file to your home directory: cp /users/john1545/1bck.pdb. Now write a python program (saved to file) that: Reads in the file (enter filenames via user input) Checks each line to see if it is an ATOM, and, if the atom is an oxygen (O). If so, write the following fields to an output file: ATOM, residue name (e.g., CYS, GLU, etc), and the atom type (O) E.g. ATOM THR O ATOM SER O ATOM SER O ATOM LYS O

72 Exercise #13 One Solu3on #!/usr/bin/env python filein = raw_input("enter the pdb filename: ") fileout = raw_input("enter output filename: ") fhandle = open(filein, "r") fout = open(fileout, "w") for item in fhandle: if ("ATOM" in item): recordsplit = item.split() if (recordsplit[- 1] == "O"): fout.write(recordsplit[0] + " " + recordsplit[3] + " " + recordsplit[- 1] + "\n") print "End!"

73 Exercise #13 One Solu3on ~]$ cat output.txt ATOM MET O ATOM VAL O ATOM ASN O ATOM ASN O ATOM PRO O ATOM THR O ATOM THR O ATOM VAL O ATOM PHE O ATOM PHE O ATOM ASP O ATOM ASP O ATOM ASP O ATOM ILE O ATOM ALA O

74 Wrapping Up Python is an extremely versa3le language that can be used effec3vely for biological data analysis We barely scratched the surface of the language s features Consider looking into Biopython addon modules and tools for Python, specific for bioinforma3cs hƒp://biopython.org/

75 Thank You John B. Johnston Bioinforma3cs Engineer