Algorithms and Tools for Bioinformatics on GPUs. Bertil SCHMIDT

Transcription

1 Algorithms and Tools for Bioinformatics on GPUs Bertil SCHMIDT

2 Contents Motivation Pairwise Sequence Alignment Multiple Sequence Alignment Short Read Error Correction using CUDA Some other CUDA-enabled Bioinformatics tools

3 Data Explosion

4 Typical Bioinformatics Applications Database Search BLAST, Smith-Waterman Multiple Sequence Alignment ClustalW Hidden Markov Models HMMer Motive Finding MEME De-novo Genome Assembly Velvet Phylogenetic Trees RAxML Protein Structure Prediction

5 Local Pairwise Sequence Alignment Align S=ATCTCGTATGATG S=GTCTATCAC G T C T A T C A C A T C T C G T A T G A T G = = else ) ( if ), ( y x y x Sbt α=, β= A T C T C G T A T G A T G G T C T A T C A C + = ), ( ), ( ), ( ), ( max ), ( j i S S Sbt j i H j i H j i H j i H

6 Extraction of Parallelism Ligowski, Rudnicki, Liu, Schmidt: "Accurate scanning of sequence databases with the Smith-Waterman algorithm", GPU Computing Gems, Vol., Chapter Liu, Maskell, Schmidt: "CUDASW++: optimizing Smith-Waterman sequence database searches for CUDA-enabled graphics processing units", BMC Research Notes, :, 9 G T C T A T C A C A T C T C G T A T G A T G l l P P P

7 Multiple Sequence Alignment (MSA) FOS_RAT MMFSGFNADYEASSSRCSSASPAGDSLSYYHSPADSFSSMGSPVNTQDFCADLSVSSANF FOS_MOUSE MMFSGFNADYEASSSRCSSASPAGDSLSYYHSPADSFSSMGSPVNTQDFCADLSVSSANF FOS_CHICK MMYQGFAGEYEAPSSRCSSASPAGDSLTYYPSPADSFSSMGSPVNSQDFCTDLAVSSANF FOSB_MOUSE -MFQAFPGDYDS-GSRCSS-SPSAESQ--YLSSVDSFGSPPTAAASQE-CAGLGEMPGSF FOSB_HUMAN -MFQAFPGDYDS-GSRCSS-SPSAESQ--YLSSVDSFGSPPTAAASQE-CAGLGEMPGSF *:..*.:*::.***** **:.:* * *..***.* :.. :*: *:.*....* Exact DP solution has exponential complexity Heuristic optimization methods used in practice; e.g. progressive alignment method (ClustalW) (a) (b) (c) Distance Matrix Compuation Guided Tree Progressive Alignment along the Guided Tree Profiling the three stages of ClustalW Stage (Distance matrix) requires more than 9% of overall runtime

8 MSA with ClustalW HAHU HA HU HBHU. HB HU HAHO.9 9. HA HO HB HO MY WH P LH Dist Matrix: O(n l ) LG HB HBHO HBHU Dynamic Programming Alignment HBHU HBHO HBHO. 9.. MYWH PLH LGHB NJ-Tree: O(n ) LGHB PLH MYWH HAHU HAHO HAHO HAHU HAHO HAHU Dynamic Programming HBHU HBHO Dynamic Programming Alignment Progressive: O(n l ) Alignment HAHU HAHO HBHU HBHO HAHU HAHO HBHU HBHO

9 Distance Computation G T C T A T C A C A T C T C G T A T G A T G = ), ( ), ( ), ( ), ( max ), ( j i S S sbt j i H g j i H g j i H j i H sbt(x,y) = + if x=y; otherwise Linear Gap Penalty : g = nid(s,s) = d(s,s) = / Disadvantage of nid-impementation with trace-back: Quadratic Memory or Divide-and-Conquer ), min( ), ( ), ( j i j i j i l l S S nid S S d =

10 Our Approach: Distance Computation with DP in linear space G T C T A T C A C A T C T C G T A T G A T G / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / 9/ / / / / / / / / / / / / / / / / / / / / / / / / / / //9/ / / / / / / / / / / / 9/ 9/ /

11 Recurrence Relations for Affine Gap Penalties

12 CUDA Parallelization Inter-task parallelization Each alignment (task) is assigned to exactly one thread dimblock alignments are performed in parallel within a thread block. Load balancing Sequences sorted by lengths all threads within a thread block have similar workload Memory access O(min{l a,l b }) storage for intermediate results per thread Stored in global memory using coalesced memory access pattern Partitioning of DP-matrix into blocks reduces global memory access by using shared memory and registers Substitution matrix stored in shared memory Sequences stored in texture memory

13 Overview: Parallelization Approaches FPGA: Systolization with Verilog HDL FPGA: Load Balancing, Partitioning (FIFO, Multi-Lanes) Sequential ClustalW Algorithm DP-Modification to allow more efficient parallelization GPU: SIMT Parallelization with CUDA GPU: Load Balancing, Optimization of memory accesses (coalesced, shared) Cell/BE: MIMD Parallelization with Cell/BE SDK Cell/BE: Load Balancing, SIMD Vectorization

14 Comparison: Speedup FPGA: Xilinx XCVLX PEs MHz GPU: GeForce GTX SPs.GHz Cell/BE: PlayStation SPEs SIMD vector-length:.ghz Speedup compared to ClustalW..9 FPGA GPU Cell/BE 9 9 () () () () () ()

15 Comparison: Productivity, Power

16 MSA-CUDA: Performance Stage + MSA-CUDA on a GPU (GeForce GTX) Better Performance than ClustalW-MPI on a PC-cluster with Cores for all tested datasets

17 Profile Hidden Markov Models Statistical model of a set of biological similar sequences HMMs can be created from MSAs General transition structure for a global alignments: D D D D Example: align HEIKO H E I K - O I I I I I M M - - M M M (start) M M M M M (end) Viterbi Algorithm: Finds most likely path through the HMM generating the given sequence Dynamic Programming: O(sequence length HMM length) Efficient parallel implementation (similar to SW) Transition structure does not allow for local or multi-hit alignments

18 Improving MSA accuracy We have designed MSAProbs to significantly improve alignment accuracy Currently working on a CUDA version

19 Short Read Error Correction May contain errors! DNA Sequencing machine Read-sequences DNA-sequence

20 Next-Generation Sequencing (NGS) NGS technology has ultra-high throughput short read-length NGS Platform Illumina (HiSeq) SOLiD Reads per run billion. billion Read length - bps - bps Run time (single-end) - days - days Run time (mate-pair) - days - days Example: Human Genome Sequencing (Li et al., ) billion reads, average read-length bp (x coverage) NGS Bioinformatics Challenges Scalability (to deal with huge amounts of reads) Algorithm design (to deal with short reads)

21 Error Correction Illumina reads contain about %-% sequencing errors (substitutions) Prior Error Correction can improve assembly quality as well as Graph size N-values produced by Edena for simulated reads (.M reads of length generated from Saccharomyces C Chr ) Human Genome (Li et al. ): distinct -mers reduced from.b to.b through prior error correction Error Correction usually most time-consuming step in assembly % of total runtime (h on a high-end PC cluster) for human genome Error Correction also increases the amount of mapped reads to the reference genome for Re-sequencing applications

22 Spectral Alignment Problem (SAP) Changing the single error at position in the given read from G to A results in l corresponding matches in the spectrum.

23 Spectral Alignment Problem (SAP) Reads R={r,...,r k } of length L Length l<l Reference genome G l-mer spectrum T(G) read error-free all its l-mers have corresponding exact match in T(G) read has a single mutation error overlapping l-mers have (in most cases) a low number of corresponding exact matches in T(G) Changing error to correct base-pair, all overlapping l-mers have a corresponding match in T(G)

24 Spectral Alignment Problem (SAP) G usually unknown T(G) approximated by T(R,m) T(R,m): all l-mers that occur at least m times (multiplicity) in R Possible complications. Some l-mers in T(G) not necessarily in T(R,m) (false negatives). Some l-mers in T(R,m) not necessarily int(g) (false positives) Quality of approximation can be reduced by using quality-scores associated with each read (i.e. Use only high-quality l-mers to build T(m,R)) Q-score: value between and

25 Bloom Filter Data Structure Membership test most important operation (test if an l-mer is in T(m,R)) Use of a space-efficient Bloom filter for probabilistic hashing stored in CUDA texture memory C A C T G G A A G T C l-tuple s BF[..m] & if s T m,l (R) or FP(s) if s T m,l (R) and not FP(s)

26 SAP Voting Procedure

27 DecGPU error correction algorithm. (Distributed) spectrum construction. filtering out error-free reads. fixing erroneous reads using a voting algorithm. trimming (or discarding entirely) the fixed reads that remain erroneous. optional iterative policy between the filtering and fixing stages for the correction of more than one base error in a single read. On a cluster DecGPU uses a one-toone correspondence between an MPI process and one GPU

28 Performance Evaluation Datasets

29 Runtime Comparison Hardware Configurations DecGPU: single Tesla T DecGPU (CPU version): AMD Opteron quad-core (multithreaded with OpenMP) Euler-SR: Intel Core i (single threaded) All times in seconds Data set Euler-SR DecGPU (CPU) DecGPU (GPU) Overall Speedup Overall Speedup SC EC SC EC SC EC Euler-SR DecGPU(CPU) A.. B C

30 Accuracy and Runtime Comparison Comparison to hshrec Salmela (Bioinformatics, ) extension of the suffix tree approach by Schroeder and Schmidt (Bioinformatics, 9) hshrec and DecGPU both run a dual quad-core Intel Xeon E (both multithreaded)

31 Accuracy Evaluation for Re-sequencing

32 Quality Evaluation for de-novo Assembly

33 Scalability FPP of a Bloom filter (where N B = number of buckets, N E = number of elements, α=hn E / N B ): DecGPU: h =, max N B = GB memory FPP FPP and maximal N E for representative α values ( α e ) h We estimate the maximal number of reads that can be processed by N PE MPI processes as: E( N kmer ) C N R = N PE N E = N PE N E L k + L Example: C=, L=, N PE =: N R =. billion for α =.

34 mcuda-meme: Motif finding ATCCCG gene TTCCGG gene ATGCCG gene ATCCAC gene Complications of identifying common motifs in DNA sequences: We do not know the motif sequence We do not know where it is located relative to the genes start Motifs can differ slightly from one gene to the next How to discern it from random motifs?

35 mcuda-meme: Motif finding CUDA-MEME is integrated in the CompleteMotifs pipeline ( for ChiP-Seq data analysis

36 CUDA-BLASTP Stage Stage Stage Stage database Word hits Ungapped Matching Extension HSPs Gapped Extension HSAs Traceback & Display alignments Filtration approach: Assumes good alignments contain short exact matches Find such matches quickly using data structures such as lookup tables Identified short matches are used as seeds for further detailed analysis

37 CUDA-BLASTP

38 Summary and Current Work DecGPU CUDA-SW++ mcuda-meme CUDA-BLASTP Tools currently under development CRiSPy (Computing Species Richness in S rrna Pyrosequencing Datasets with CUDA) PASHA (denovo genome assembler) MSAProbs (CUDA version) More versions of BLAST