Managing Your Biological Data with Python

Transcription

1 Chapman & Hall/CRC Mathematical and Computational Biology Series Managing Your Biological Data with Python Ailegra Via Kristian Rother Anna Tramontano CRC Press Taylor & Francis Group Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Group, an informa business A CHAPMAN & HALL BOOK

2 Table of Contents Preface, xix Acknowledgements, xxv PART I Getting Started CHAPTER 1 The Python Shell IN THIS CHAPTER YOU WILL LEARN STORY: CALCULATING THE AG OF ATP HYDROLYSIS Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? How to Run the Example on Your Computer Variables Importing Modules Calculations EXAMPLES TESTING YOURSELF 21 CHAPTER 2 Your First Python Program IN THIS CHAPTER YOU WILL LEARN STORY: HOW TO CALCULATE THE AMINO ACID FREQUENCY IN THE SEQUENCE OF INSULIN Problem Description Example Python Session 26

3 viii Table of Contents 2.3 WHAT DO THE COMMANDS MEAN? How to Execute the Program 27 ' How Does the Program Work? Comments String Variables Loops with for Indentation Printing to the Screen EXAMPLES TESTING YOURSELF 38 PART I Summary PART II Data Management CHAPTER 3 Analyzing a Data Column IN THIS CHAPTER YOU WILL LEARN STORY: DENDRITIC LENGTHS Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? Reading Text Files Writing Text Files Collecting Data in a List Converting Text to Numbers Converting Numbers to Text Writing a Data Column to a Text File Calculations on a List of Numbers EXAMPLES TESTING YOURSELF 57 CHAPTER 4 Parsing Data Records ^ IN THIS CHAPTER YOU WILL LEARN 59

4 Table of Contents ix 4.2 STORY: INTEGRATING MASS SPECTROMETRY DATA INTO METABOLIC PATHWAYS Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? The if/elif/else Statements List Data Structures Concise Ways to Create Lists EXAMPLES TESTING YOURSELF 75 CHAPTER 5 Searching Data IN THIS CHAPTER YOU WILL LEARN STORY: TRANSLATING AN RNA SEQUENCE INTO THE CORRESPONDING PROTEIN SEQUENCE Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? Dictionaries The while Statement Searching with while Loops Searching in a Dictionary Searching in a List EXAMPLES TESTING YOURSELF 90 CHAPTER 6 Filtering Data IN THIS CHAPTER YOU WILL LEARN STORY: WORKING WITH RNA-SEQ OUTPUT DATA Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? Filtering with a Simple for...if Combination 97

5 x Table of Contents Combining Two Data Sets Differences between Two Data Sets Removing from Lists, Dictionaries, and Files Removing Duplicates Preserving and Not Preserving Order Sets EXAMPLES TESTING YOURSELF 108 CHAPTER 7 Managing Tabular Data IN THIS CHAPTER YOU WILL LEARN STORY: DETERMINING PROTEIN CONCENTRATIONS Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? Representing a Two-Dimensional Table Accessing Rows and Single Cells Inserting and Removing Rows Accessing Columns Inserting and Removing Columns EXAMPLES TESTING YOURSELF 127 CHAPTER 8. Sorting Data IN THIS CHAPTER YOU WILL LEARN STORY: SORT A DATA TABLE Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? Python Lists Are Good for Sorting The sorted() Built-in Function Sorting with itemgetter Sorting in Ascending/Descending Order 134

6 Table of Contents xi Sorting Data Structures (Tuples, Dictionaries) Sorting Strings by Their Length EXAMPLES TESTING YOURSELF 141 CHAPTER 9 Pattern Matching and Text Mining IN THIS CHAPTER YOU WILL LEARN STORY: SEARCH A PHOSPHORYLATION MOTIF IN A PROTEIN SEQUENCE Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? Compiling Regular Expressions Pattern Matching Grouping Modifying Strings EXAMPLES TESTING YOURSELF 158 PART II Summary PART III Modular Programming CHAPTER 10 Divide a Program into Functions IN THIS CHAPTER YOU WILL LEARN STORY: WORKING WITH THREE-DIMENSIONAL COORDINATE FILES Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? How to Define and Call Functions Function Arguments The struct Module 179

7 xii Table of Contents 10.4 EXAMPLES TESTING YOURSELF 185 CHAPTER 11 Managing Complexity with Classes IN THIS CHAPTER YOU WILL LEARN STORY: MENDELIAN INHERITANCE Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? Classes Are Used to Create Instances Classes Contain Data in t he Form of Attributes Classes Contain Methods The repr Method Makes Classes and Instances Printable Using Classes Helps to Master Complex Programs EXAMPLES TESTING YOURSELF 201 CHAPTER 12 Debugging IN THIS CHAPTER YOU WILL LEARN STORY: WHEN YOUR PROGRAM DOES NOT WORK Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? Syntax Errors Runtime Errors Handling Exceptions When There Is No Error Message EXAMPLES TESTING YOURSELF 222 &

8 Table of Contents xiii CHAPTER 13 Using External Modules: The Python Interface to R IN THIS CHAPTER YOU WILL LEARN STORY: READING NUMBERS FROM A FILE AND CALCULATING THEIR MEAN VALUE USING R WITH PYTHON Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? Therobjects Object of rpy2 and the r Instance Accessing an R Object from Python Creating Vectors Creating Matrices Converting Python Objects into R Objects How to Deal with Function Arguments That Contain a Dot EXAMPLES TESTING YOURSELF 241 CHAPTER 14 Building Program Pipelines IN THIS CHAPTER YOU WILL LEARN STORY: BUILDING AN NGS PIPELINE Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? How to Use TopHat and Cu fflinks What Is a Pipeline? Exchanging Filenames and Data between Programs Writing a Program Wrapper Lag When Closing Files Using Command-Line Parameters 254

9 xiv Table of Contents Testing Modules: if name == ' main ' Working with Files and Directories EXAMPLES TESTING YOURSELF 260 CHAPTER 15 Writing Good Programs IN THIS CHAPTER YOU WILL LEARN PROBLEM DESCRIPTION: UNCERTAINTY There Is Uncertainty in Writing Programs Example Programming Project SOFTWARE ENGINEERING Dividing a Programming Project into Smaller Tasks Split a Program into Functions and Classes Writing Well-Formatted Code Using a Repository to Control Program Versions How to Release Your Program to Other People The Cycle of Software Development EXAMPLE TESTING YOURSELF 280 PART III Summary PART IV Data Visualization CHAPTER 16. Creating Scientific Diagrams IN THIS CHAPTER YOU WILL LEARN STORY: NUCLEOTIDE FREQUENCIES IN THE RIBOSOME Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? The matplotlib Library 289

10 Table of Contents xv Drawing Vertical Bars Adding Labels to an x-axis and y-axis Adding Tick Marks Adding a Legend Box Adding a Figure Title Setting the Boundaries of the Diagram Exporting an Image File in Low Resolution and High Resolution EXAMPLES TESTING YOURSELF 298 CHAPTER 17 Creating Molecule Images with PyMOL IN THIS CHAPTER YOU WILL LEARN STORY: THE ZINC FINGER What Is PyMOL? Example PyMOL Session SEVEN STEPS TO CREATE A HIGH-RESOLUTION IMAGE Writing PyMOL Script Files Loading and Saving Molecules Selecting Parts of Molecules Choose Representations for Each Selection Setting Colors Setting the Camera Position Exporting a High-Resolution Image EXAMPLES TESTING YOURSELF 321 CHAPTER 18 Manipulating Images IN THIS CHAPTER YOU WILL LEARN STORY: PLOT A PLASMID Problem Description Example Python Session 325

11 xvi Table of Contents 18.3 WHAT DO THE COMMANDS MEAN? Creating an Image Reading and Writing Images Coordinates Drawing Geometrical Shapes Rotating an Image Adding Text Labels Colors Helper Variables EXAMPLES TESTING YOURSELF 336 PART IV Summary PART V Biopython CHAPTER 19 Working with Sequence Data IN THIS CHAPTER YOU WILL LEARN STORY: HOW TO TRANSLATE A DNA CODING SEQUENCE INTO THE CORRESPONDING PROTEIN SEQUENCE AND WRITE IT TO A FASTA FILE Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? The Seq Object Working with Sequences as Strings The MutableSeq Object The SeqRecord Object The SeqIO Module EXAMPLES TESTING YOURSELF 361

12 Table of Contents xvii CHAPTER 20 «Retrieving Data from Web Resources IN THIS CHAPTER YOU WILL LEARN ' STORY: SEARCHING PUBLICATIONS BY KEYWORDS IN PUBMED AND DOWNLOADING AND PARSING THE CORRESPONDING RECORDS Problem Description Python Session WHAT DO THE COMMANDS MEAN? The Entrez Module The Medline Module EXAMPLES TESTING YOURSELF 372 CHAPTER 21 Working with 3D Structure Data IN THIS CHAPTER YOU WILL LEARN STORY: EXTRACTING ATOM NAMES AND THREE- DIMENSIONAL COORDINATES FROM A PDB FILE Problem Description Example Python Session WHAT DO THE COMMANDS MEAN? The Bio.PDB Module The SMCRA Object Hierarchy EXAMPLES TESTING YOURSELF 388 PART V Summary PART VI Cookbook Recipe 1 : The PyCogent Library 395 Recipe 2: Reversing and Randomizing a Sequence 399 Recipe 3: Creating a Random Sequence with Probabilities 403 Recipe 4: Parsing Multiple Sequence Alignments Using Biopython 405

13 xviii Table of Contents Recipe 5: Calculating a Consensus Sequence from a Multiple Sequence Alignment 409 Recipe 6: Calculating the Distance between Phylogenetic Tree Nodes 413 Recipe 7: Codon Frequencies in a Nucleotide Sequence 417 Recipe 8: Parsing RNA 2D Structures in the Vienna Format 421 Recipe 9: Parsing BLAST XML Output 425 Recipe 10: Parsing SBML Files 427 Recipe 11: Running BLAST 431 Recipe 12: Accessing, Downloading, and Reading Web Pages 437 Recipe 13: Parsing HTML Files 441 Recipe 14: Splitting a PDB File into PDB Chain Files 445 Recipe 15: Finding the Two Closest Ca Atoms in a PDB Structure 447 Recipe 16: Extracting the Interface between Two PDB Chains 451 Recipe 17: Building Homology Models Using Modeller 455 Recipe 18: RNA 3D Homology Modeling with ModeRNA 459 Recipe 19: Calculating RNA Base Pairs from a 3D Structure 463 Recipe 20: A Real Case of Structural Superimposition: The Serine Protease Catalytic Triad 467 APPENDIX A: COMMAND OVERVIEW, 471 APPENDIX B: PYTHON RESOURCES, 495 APPENDIX C: RECORD SAMPLES, 499 APPENDIX D: HANDLING DIRECTORIES AND PROGRAMS WITH UNIX, 507 INDEX, 525