SolexaLIMS: A Laboratory Information Management System for the Solexa Sequencing Platform

Transcription

1 SolexaLIMS: A Laboratory Information Management System for the Solexa Sequencing Platform Brian D. O Connor, 1, Jordan Mendler, 1, Ben Berman, 2, Stanley F. Nelson 1 1 Department of Human Genetics, David Geffen School of Medicine, University of California, Los Angeles, CA, USA 2 USC/Keck School of Medicine, University of Southern California, Los Angeles, CA, USA Brian D. O Connor - boconnor@ucla.edu; Jordan Mendler - jmendler@ucla.edu; Ben Berman -benberman@usc.edu; Stanley F. Nelson - snelson@ucla.edu; Corresponding author Abstract The SolexaLIMS system is a collection of software components and web tools designed to facilitate the processing, analysis, sharing, and long term storage of sequence data produced with the Solexa sequencing instrument. The SolexaLIMS system sits between the end user and the processing pipeline provided by Solexa for processing raw sequence data into final base calls and for performing simple alignments with a reference genome. It consists of three major components: a database component that tracks sample annotations, a web component for entering and editing sample annotations, and an analysis component that allows for basic reporting. The SolexaLIMS system is a free and open source project that welcomes community involvement to further enhance the LIMS components. Software and a user forum are available at 1

2 Background The Solexa 1G genome analyzer provides the ability to perform massively parallel sequencing of 1 billion bases per run. With this extreme amount of sequence data comes challenges to efficiently manage the acquisition, analysis, and long-term storage of these results. Solexa provides software that automates many of the analysis steps involved in generating sequence data from a sample. However, a comprehensive laboratory information management system (LIMS) for controlling this data flow has previously been unavailable. Here we present SolexaLIMS, a web application and application program interface (API) for managing Solexa experiments. LIMS systems are designed to manage samples, instruments, and laboratory work flows in a way that maximizes sample throughput and minimizes experimental bottlenecks. The SolexaLIMS system was designed to meet these needs and provides the ability to record the basic sample information key to describing what is being sequenced on the instrument. This functionality uses a web-based LIMS interface which is linked to an analysis and reporting functions via a database back-end. This makes it possible to enter a sample information once and generate many reports, such as genomic alignments with polymorphism detected, in an automated way. The SolexaLIMS system also includes the ability to backup and validate data from Solexa datasets. Implementation The SolexaLIMS software is divided into 3 major functional categories: the LIMS user website, the Solexa pipeline wrappers, and finally the LIMS reports and analysis tool. These components are integrated and use a common database to track the overall progress of a Solexa run within the LIMS system. Figure 1 shows an overview of the specific steps and components making up the processing of a typical Solexa experiment. SolexaLIMS Web Application The SolexaLIMS web application is the main conduit through which users interact with the LIMS system. All sample annotation entry happens here and the progress a sample takes through various processing steps is documented here as well. The SolexaLIMS web application is built using standard Java web application APIs including the popular Spring framework and the Hibernate data access layer. Using Java allows this web application to be portable and easy to customize for many web developers already familiar with these popular technologies. 2

3 The LIMS website allows for basic access restrictions and requires that each user to create an account and login to use the site. Once logged in, a user can see the list of Solexa datasets that have been, or are currently being, processed by the LIMS system. Uses can add new experiments and this process begins with collecting some basic information about the experiment being done along with details describing the contents of each lane in the flowcell. This process is iterative and the user may come back to edit an experiment s information multiple times as the sample is processed by the Solexa instrument. For example, as additional quality information is determined, the user may specify which sequencing cycles should be used for further analysis. This type of information is typically not available before a sample is processed by the Solexa instrument. Once a user is satisfied with the accuracy of the entries for a given sample in the Solexa LIMS website, they can activate the processing pipeline by selecting process now in the experiment page. This indicates that the sample has been completely processed by the Solexa instrument and is ready for processing by both the Solex-provided pipeline and additional validation, analysis, and reporting tools provided by the LIMS system. Each step is documented and its status is updated in real time to the website so users can track each experiment. Solexa LIMS Database The Solexa LIMS web site stores information about each run in a database along with information about the algorithms, reporting and validation tools applied to the sample data. It represents these as experiment and process tables respectively. Since Solexa using multiple distinct lanes in their flowcell, each lane is represented as a separate lane entry to track information about it. The LIMS database is designed to act as a natural point of integration and all processing, reporting, and validation tools write to the database to track their progress and status. Solexa LIMS Pipeline Information written to the database by the LIMS web application is read using a daemon that monitors for new experiments. This process then spawns the solexa pipeline and various reporting/ validation tools. This pipeline wrapper is cluster aware and integrates with the Sun Grid Engine to distribute the Solexa pipeline across cluster nodes. The pipeline code in the Solexa LIMS system acts as a wrapper for the intensity and base calling algorithms provided by Solexa. 3

4 Solexa LIMS Reporting and Analysis Tools The SolexaLIMS system provides several analysis and reporting tools designed to examine sequencing experiments of several different types. The first is a wrapper for the ELAND alignment algorithm that generates fast, ungapped alignments between a reference genome and the Solexa sequence with less than 3 mismatches. Alternatively the LIMS system includes another alignment wrapper for BLAT which allows for more flexibility in the search criteria. Depending on the experiment performed and the reference genome used the SolexaLIMS system provides two reporting mechanisms. First, for genomic alignments BED and WIG files are produced that represent polymorphisms observed from the Solexa sequences and the sequence coverage per base position respectively. Second, alignments to a RefSeq cdna library can be analyzed to generate a report of RefSeq counts based on the number of Solex sequences aligned within each cdna sequence. All reports are linked to via the SolexaLIMS website. The SolexaLIMS analysis and reporting component also includes tools for validating the various steps in the Solexa pipeline. This ensures that the experimental data integrity at each step in the process. The analysis component also includes compression tools for representing Solexa data in a more compact form. This includes converting intensity information derived from the Solexa pipeline from plain text files to NetCDF binary files. This conversion preserves all information in the text files yet takes only a fraction of the storage space. Similarly, sequence data produced by the Solexa pipeline is also converted from plain text output to a binary version following the 2bit format. This allows for a both sequence and quality information to be stored in separate tracks while reducing overall file size considerably. Results In our SolexaLIMS system number of lanes have been processed through the Solexa pipeline. This includes the Firecrest algorithm for image processing and the Bustard base-calling algorithm. This has resulting in number sequenced bases. Datasets of this size require considerable storage space with the average run taking up numbergb of space for image files. The intensity and base calling process adds another number GB to that size. While each alignment using ELAND takes approximately number GB per lane for alignments to the human genome. The SolexaLIMS system has been successfully used to sequence approximately? bases worth of sequence, many of which representing deep sequencing of exons.? bases of this total sequence were aligned to the human genome (build 18, UCSC) using the ELAND tool from the Solexa pipeline. This process was facilitated by the data entered in the SolexaLIMS system. Here, only lanes from experiments annotated as 4

5 human genomic DNA were aligned to the human genome reference sequence in this way. Of this sequence, approximately? novel SNPs were identified in exon regions. These were identified using the reporting components of SolexaLIMS including tools for outputting BED and WIG file formats for display in the UCSC browser. Figure 2 shows and example region visualized in this way. This section should be expanded... I could just go into the types of experiments that have been done and how many sequences/alignments have been produced. Conclusions The SolexaLIMS system was designed to meet the needs of research groups, laboratories, and core facilities using the next generations sequencing techniques provided by Solexa. While software exists to process raw data from the 1G instrument to finished sequence files, there are currently no options for managing this overall work-flow from start to finish. SolexaLIMS provides this missing capability and wraps not only the raw data processing but also common analysis and archiving of the results. The interface controlling the overall flow and analysis is a simple web-based application familiar to most users. This web interface provides the ability to annotate key pieces of information about the experiment that describe the nature of the sequencing being done. It also enables users to follow the validation, backup, analysis and reporting modules resulting, making it easy to ensure a successful experiment and easy access to results. Since the Solexa instrument, and other next generation sequencing technologies, are just now coming online in core facilities and laboratories, many of the applications and common usage patterns are still to be worked out. A challenge going forward with the SolexaLIMS project is the adequate representation of experimental procedures in the LIMS system. For example, paired-end sequencing equipment and procedures will soon be available for the 1G instrument. This will necessitate a different data processing flow in the processing pipeline wrapper script and front-end changes to the SolexaLIMS web application to capture appropriate sample annotations. The opensource nature of SolexaLIMS means that many changes in responses to new experimental procedures using the Solexa instrument can be modeled in the LIMS system directly by users. This community approach ensures that the SolexaLIMS system meets users needs in the future. Another major challenge for the platform, and other similar devices, is the large amount of data produced with each run. With over 1/2TB of data produced per experiment, the need to develop more efficient storage techniques is a top concern. This is a current focus of the SolexaLIMS project and the development of a NetCDF-based file format for intensity scores and a 2BIT format for sequence data produced is a 5

6 near-term goal. Finally, interoperability with existing systems is another key concern for the effective distribution of sequence results from the Solexa system. Current development efforts are focused on integrating gene expression counting information, raw sequences, and sequence alignments from Solexa experiments into the Celsius pipeline. This introduces the possibility of distributing results via the ubiquitous DAS protocol which would greatly enhance the ability to share Solexa data with other parties. Availability and Requirements The Nelson lab open source project provides community mailing lists where users can discuss and contribute changes to the SolexaLIMS system. This website provides access to the sourcecode, user forums, and documentation to setup and use the SolexaLIMS system. We fully intended to respond to user feedback and integrate suggestions and patches from end users that are active participants in the development process. In order to use Solexa data the image files need to be analyzed to produce sequence. The SolexaLIMS system uses the pipeline provided by Solexa to accomplish this. The pipeline is configurable to run on a Sun grid engine (SGE cluster) using standard Linux tools. Processing time greatly depends on the hardware and network capacity connection computers. With a such large collection of files, moving them efficiently across the network can be a major bottleneck. In our setup we use a temporary processing diskspace and 8 cluster nodes to run the Solexa pipeline. Each cluster node is powered by a 2.? GHz Intel processor verify, fill in and contains 8GB of memory. Processing a complete Solexa dataset requires approximately 22 hours of compute time. Further parallelization of the pipeline is limited by the input and output (I/O) bottlenecks at the temporary diskspace storage level. Transfer times to move a complete Solexa dataset onto this temporary cluster space are approximately for a complete run. Give our real-world performance benchmarking local disk I/O and network transfer times are the major limiting factor affecting the processing of raw images to base calls using the Solexa pipeline. Authors contributions B.O. is the author of the SolexaLIMS web application and pipeline with contributions by J.M. and B.B. Acknowledgments Grants... 6

7 Figures Figure 1 - SolexaLIMS Flow Overview SolexaLIMS is layered on top of the processing pipeline that transforms raw images from the instrument to finished sequence. It manages sample annotations via a web interface and coordinates additional analysis and reporting via a centralized database. Figure 2 - Genomic Alignment Report A genomic alignment produces reports on potential polymorphisms and overall sequence coverage that can easily be viewed in the UCSC browser. The use of standardized output formats and visualization streamlines analysis. 7

8 LIMS Website Solexa Sequencer store annoations Solexa Images LIMS DB Solexa pipeline Solexa Sequence Genomic or cdna genomic alignment cdna alignment alignment report mrna counts Figure 1: An overview of the SolexaLIMS system. 8

9 Figure 2: An example report showing polymorphisms identified by the genomic sequencing of KCNJ12. 9