XML for Bioinformatics

Transcription

1 XML for Bioinformatics

2 Ethan Cerami XML for Bioinformatics

3 Library of Congress Cataloging-in-Publication Data Cerami, Ethan. XML for bioinformatics / Ethan Cerami. p. cm. Includes bibliographical references and index. ISBN XML (Document markup language) 2. Bioinformatics. I. Title. QA76.76.H94C dc ISBN C 2005 Springer Science+Business Media, Inc. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, Inc., 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed in the United States of America springeronline.com

4 To Lily Cerami. Welcome to the world.

5 Preface Introduction The goal of this book is to introduce XML to a bioinformatics audience. It does so by introducing the fundamentals of XML, Document Type Definitions (DTDs), XML Namespaces, XML Schema, and XML parsing, and illustrating these concepts with specific bioinformatics case studies. The book does not assume any previous knowledge of XML and is geared toward those who want a solid introduction to fundamental XML concepts. The book is divided into nine chapters: Chapter 1: Introduction to XML for Bioinformatics. This chapter provides an introduction to XML and describes the use of XML in biological data exchange. A bird s-eye view of our first case study, the Distributed Annotation System (DAS), is provided and we examine a sample DAS XML document. The chapter concludes with a discussion of the pros and cons of using XML in bioinformatic applications. Chapter 2: Fundamentals of XML and BSML. This chapter introduces the fundamental concepts of XML and the Bioinformatic Sequence Markup Language (BSML). We explore the origins of XML, define basic rules for XML document structure, and introduce XML Namespaces. We also explore several sample BSML documents and visualize these documents in the Rescentris Genomic Workspace TM Viewer. Chapter 3: DTDs for Bioinformatics. This chapter introduces XML Document Type Definitions (DTDs). With DTDs, you can define specific rules for XML document construction and validate XML instance documents against these rules. This chapter builds a DTD for representing protein sequences and does so in incremental stages we therefore start out simply and add layers of complexity as new concepts are introduced. The chapter also includes an overview of XML data formats available from The National Center for Biotechnology Information (NCBI), and provides a complete description of the NCBI TinySeq DTD. Chapter 4: XML Schemas for Bioinformatics. XML Schema represents the successor to XML Document Type Definitions (DTDs). We begin by comparing the two specifications and describe some of the advantages of using XML Schema. To illustrate core concepts, we rebuild the protein sequence DTD from Chapter 3 as an XML Schema, enabling you to compare the two specifications directly. The chapter concludes with a discussion of the Proteomics Standards Initiative Molecular Interaction (PSI-MI) XML format, an XML exchange format used to encode protein protein interactions. Chapter 5: Parsing NCBI XML in Perl. Perl remains the programming language of choice for many in bioinformatics. This chapter therefore explores several options for parsing XML in Perl, vii

6 viii Preface and focuses on two standard interfaces: the Simple API for XML (SAX) and the Document Object Model (DOM). We also explore the NCBI E-Fetch service, and retrieve nucleotide sequence records in XML in real time. This chapter assumes some prior knowledge of Perl. Chapter 6: The Distributed Annotation System. This chapter provides comprehensive coverage of the Distributed Annotation System (DAS), a distributed XML protocol used to exchange genome annotation data. To put DAS in perspective, we begin by exploring the process of genome annotation, and illustrate the DAS protocol from the end-user perspective. We then describe the DAS XML protocol in detail, and examine numerous sample XML documents from the Ensembl and UCSC DAS servers. The chapter includes a reference guide to all DAS commands, and a preview of anticipated features in the next version of DAS. Chapter 7: Parsing DAS Data in SAX. Despite the popularity of Perl, Java is becoming increasingly popular in bioinformatic applications. This chapter therefore describes the mechanics of parsing XML documents using the Java Simple API for XML (SAX). SAX is the de facto event-based XML parsing standard, and is widely implemented by many XML parsers, including several open source XML parsers. Several sample DAS applications are demonstrated, including one sample application which makes use of the open source BioJava library. This chapter assumes some prior knowledge of Java. Chapter 8: Parsing DAS Data in JDOM. This chapter focuses on the fundamentals of the JDOM API, a popular alternative to the SAX API. With JDOM, Java applications can easily navigate through XML document tree structures and extract elements, attributes, and character data. JDAS, an open source DAS client library, created by the author, is explored in detail. This chapter also assumes some prior knowledge of Java. Chapter 9: Web Services for Bioinformatics. Web services represent a new paradigm for building distributed web applications, and are currently being used extensively in bioinformatics. This chapter begins by presenting two broad approaches to building web services: the Representational State Transfer (REST) approach and the SOAP approach. We explore each approach in detail, and provide complete details on the latest SOAP specification. We also explore cabio, a comprehensive web service built by the National Cancer Institute (NCI). Companion Web Site This book includes a companion web site, available at: All the sample XML documents, and example Perl/Java programs described in the book are available for download. Acknowledgments Many people deserve special acknowledgments for making this book happen. First, I want to thank Lincoln Stein of Cold Spring Harbor Laboratory. Lincoln s presentation at the 2002 O Reilly Open Bioinformatics Conference and his subsequent paper in Nature (described in Chapter 1), inspired me to write this book in the first place. Lincoln s vision of creating a bioinformatics nation is a compelling one; I hope this book provides readers with the nuts and bolts information to make Lincoln s dream a reality. Lincoln also provided answers to many of my questions regarding the DAS protocol, and a complete technical review of Chapter 6. His feedback and detailed explanations were invaluable.

7 Preface ix Second, I want to thank everyone at the Computational Biology Center (cbio) at Memorial Sloan-Kettering Cancer Center, where I work. Chris Sander has created a unique and intellectually vibrant center, where I have been able to learn and thrive, and gain hands-on experience in many of the technologies described in this book. Alex Lash provided help in understanding the database resources at NCBI, and pointed me to the NCBI E-Fetch service described in Chapter 5. Anton Enright provided detailed feedback on Chapter 5. Gary Bader provided me with much-needed background information about specific biological databases and detailed background information about the PSI-MI XML format. Gary also provided feedback on Chapter 4. Third, I want to thank Lorrie LeJeune, Simon St. Laurent, Tracey Cranston, and Brian Gilman who helped out with an earlier incarnation of this book, before it found a new home at Springer- Verlag. Fourth, I want to thank several additional individuals who generously agreed to review specific chapters and provided scientific and technical feedback. Peter Covitz, Director of Bioinformatics Core Infrastructure at the National Cancer Institute (NCI) provided feedback on Chapter 9, and answered many of my questions regarding the NCI cabio bioinformatics framework. Jeff Spitzner, Chief Science Officer of Rescentris, Ltd., reviewed Chapter 2, and provided valuable feedback regarding BSML. I also want to thank Paul Farrell, my editor at Springer, for ushering the book to completion and keeping me on schedule. Finally, I want to thank my entire family for supporting me during the whole writing process associated with this book. Thanks to Dad, who hired me at the ripe age of twelve to complete my first bioinformatics programming project (really), and instilled in me a love of scientific ideas and ideals. Thanks to Mom for buying me my first computer (a Commodore Vic 20), and always reminding me to remain balanced. Special thanks to Nelli and Carla for their support and encouragement. Lastly, I want to thank my wife, Amy. All authors thank their wives in the acknowledgments, but Amy has the distinction of supporting me in this fourth book endeavor while also being pregnant. I do not know how she puts up with me, but she has been my rock, my soulmate, my everything. I love you.

8 Contents 1. Introduction to XML for Bioinformatics Introduction to XML XML Defined Origins of XML The XML Family of Specifications Web Services Defined Using XML for Biological Data Exchange Case Study: The Distributed Annotation System XML Formats for Bioinformatics Evaluating XML Usage in Bioinformatics Advantages of XML Disadvantages of XML Useful Resources Articles Web Site and Web Resources Fundamentals of XML and BSML Getting Started with BSML Using Genomic Workspace TM Fundamentals of XML Working with Elements Working with Attributes The XML Prolog Comments Processing Instructions Character Encoding CDATA Sections Creating Well-Formed XML Documents Creating Valid XML Documents Working with XML Parsers Fundamentals of XML Namespaces Why We Need XML Namespaces Declaring and Using XML Namespaces Declaring a Default Namespace 34 xi

9 xii Contents 2.4 Fundamentals of BSML BSML File Formats BSML Document Structure Representing Sequences Representing Sequence Features Retrieving Live BSML Data via XEMBL Useful Resources DTDs for Bioinformatics Introduction to DTDs A Bird s-eye View: Protein DTD Validating XML Documents Document Type Declarations Declaring Elements EMPTY ANY #PCDATA Child Elements Mixed Content Declaring Attributes Attribute Types Attribute Behaviors Working with Entities General Entities Parameter Entities Entity Summary Conditional DTD Sections Case Study: NCBI TinySeq NCBI and XML The TinySeq DTD XML Schemas for Bioinformatics Introduction to XML Schemas XML Schemas for Bioinformatics Essential Concepts: Representing Protein Data The <schema> element Schema Documentation Simple Types vs. Complex Types Global Elements vs. Local Elements Creating Instance Documents Validating Instance Documents Working with Simple Types Built-in Schema Types Working with Facets Working with Complex Types Introduction to Complex Types Declaring Empty Element Types 96

10 Contents xiii Declaring Mixed Element Types Occurrence Constraints Declaring Default Values Compositors: Sequence and Choice Defining Named Complex Types All Together Now! Basic Namespace Issues Case Study: The HUPO PSI Molecular Interaction Format PSI-MI Schema Overview A Sample PSI-MI Instance Document Working with the PSI-MI Controlled Vocabulary Parsing NCBI XML in Perl Introduction to XML Parsing in Perl Tree-Based vs. Event-Based XML Parsers Installing Modules via CPAN The Simple API for XML (SAX) Introduction to SAX SAX and Bioinformatics Applications SAX Introduction to XML::SAX Using NCBI EFetch and XML::SAX The Document Object Model (DOM) DOM Traversal with XML::LibXML Validating XML Documents with XML::LibXML Creating New Documents with XML::LibXML Using NCBI EFetch and XML::LibXML The Distributed Annotation System (DAS) Genome Annotation Introduction to DAS The WormBase DAS Viewer DAS Protocol Overview Getting Started DAS Requests DAS Responses X-DAS-Capabilities Header DAS Command Reference Retrieving Data Sources Retrieving Entry Points Retrieving Sequence Data Retrieving Annotations Working with Reference Maps Traversing the Ensembl Reference Map Working with Evolving Reference Maps The Future of DAS 172

11 xiv Contents 7. Parsing DAS Data with SAX Introduction to SAX A First Example The XMLReader Interface The ContentHandler Interface Extending the DefaultHandler Using InputSource Objects Validating XML Documents Checking for Well-Formedness Validating XML Documents: Overview Activating the SAX Validation Feature The ErrorHandler Interface Validating against XML Schemas Elements, Attributes, and Namespaces Working with Elements and Namespaces Working with Attributes Building Custom Data Structures with SAX Parsing DAS Feature Data Integrating with BioJava Parsing DAS Data with JDOM JDOM Basics JDOM Package Overview Parsing XML Documents with JDOM Parsing DAS Documents with JDOM Introduction to the JDOM Element API Traversing DAS Documents Parsing DAS dsn Documents Creating DAS Documents with JDOM Creating New Documents Creating New Elements A Complete Example Building the JDAS Library Using JDAS The JDAS Source Code Web Services for Bioinformatics Introduction to Web Services Web Services Defined Architectural Options Case Study: Introduction to the NCI cabio Project Background: Connecting to cabio via the Java RMI Interface Introduction to REST-Based Web Services Introduction to REST Connecting to the cabio REST Interface Example Application: Command Line cabio Browser 262

12 Contents xv 9.4 Introduction to SOAP SOAP Overview Constructing SOAP Messages Transporting SOAP via HTTP Introduction to Apache Axis Building a Web Service with Axis Connecting to cabio with Axis 281 Appendix Nucleotide Base Codes Amino Acid Codes 283 Bibliography 285 Index 291