Computational models for bionformatics

Transcription

1 Computational models for bionformatics De-novo assembly and alignment-free measures Michele Schimd Department of Information Engineering July 8th, 2015 Michele Schimd (DEI) DEI July 8th, / 19

2 Before starting at DEI... After graduating in Computer Engineer here at unipd (October 2009) I applied for a Ph.D. at DEI, got admitted, but without grants. In January 2010 I started working at Avanade Italy s.r.l. as Junior Analyst in the R&D team to develop prototype applications for touch devices. Microsoft Surface Table Microsoft Windows 8 touch ipad Michele Schimd (DEI) DEI July 8th, / 19

3 Second attempt Aurora Science In September 2010 I started at the ACG group with a 11 months grant on the Aurora Science Project. Michele Schimd (DEI) DEI July 8th, / 19

4 Second attempt Aurora Science In September 2010 I started at the ACG group with a 11 months grant on the Aurora Science Project. Bioinformatics Collaborating with Paolo Fontana s group (part of Aurora project) at IASMA a I started working on the De-novo assembly of DNA. a Istituto Agrario San Michele all Adige Michele Schimd (DEI) DEI July 8th, / 19

5 Second attempt Aurora Science In September 2010 I started at the ACG group with a 11 months grant on the Aurora Science Project. Bioinformatics Collaborating with Paolo Fontana s group (part of Aurora project) at IASMA a I started working on the De-novo assembly of DNA. a Istituto Agrario San Michele all Adige Gap filling Specifically their focus was on gap filling where gaps between reconstructed fragments (contigs) of a genomic sequence need to be filled using new data (i.e., new reads). Michele Schimd (DEI) DEI July 8th, / 19

6 De-novo assembly problem basics Joint work with Gianfranco Bilardi Statement (an attempt) Given a set R = {r 1, r 2,..., r R } of reads (i.e., strings) find a sequence S that is the most likely sequencing source of reads in R. Michele Schimd (DEI) DEI July 8th, / 19

7 De-novo assembly problem basics Joint work with Gianfranco Bilardi Statement (an attempt) Given a set R = {r 1, r 2,..., r R } of reads (i.e., strings) find a sequence S that is the most likely sequencing source of reads in R. Current approaches (highly distilled) Most approaches involve two steps: 1 overlap reads 2 reconstruct the sequence (resolve the overlaps). Michele Schimd (DEI) DEI July 8th, / 19

8 De-novo assembly problem basics Joint work with Gianfranco Bilardi Statement (an attempt) Given a set R = {r 1, r 2,..., r R } of reads (i.e., strings) find a sequence S that is the most likely sequencing source of reads in R. Current approaches (highly distilled) Most approaches involve two steps: 1 overlap reads 2 reconstruct the sequence (resolve the overlaps). Example: Overlap Layout Consensus (OLC) 1 Find best overlaps (e.g., test all O(R 2 ) pairs) 2 Layout overlap relation on a directed graph 3 Find paths corresponding to consensus sequences Michele Schimd (DEI) DEI July 8th, / 19

9 Computational aspect of the assembly problem Can we improve current approaches? Expose locality/parallelism from current approaches. Unfortunately graph-based algorithms do not directly expose such properties Michele Schimd (DEI) DEI July 8th, / 19

10 Computational aspect of the assembly problem Can we improve current approaches? Expose locality/parallelism from current approaches. Unfortunately graph-based algorithms do not directly expose such properties State of the art assemblers In late 90 first 2000s the community faced challenges posed by Next Generation Sequencers (NGS). Developed algorithms were focused on the characteristics of data: short reads, error rate, massive sequencing. Resulting assemblers are effective but have little focus on computational aspects. a a Meaning not enough to allow aggressive optimizations. Michele Schimd (DEI) DEI July 8th, / 19

11 Developing a computational oriented assembler Good news Use of well known primitives: sorting Development of bounds Michele Schimd (DEI) DEI July 8th, / 19

12 Developing a computational oriented assembler Good news Use of well known primitives: sorting Development of bounds Bad news Real data differs substantially from ideal behavior Lots of heursitics have been used Models may become complicated due to errors and mutations Michele Schimd (DEI) DEI July 8th, / 19

13 A first (draft) sort-based assembly algorithm Overlap by sorting (simple case) Given R = r 1, r 2,..., r R 1 sort R by lexicographic order; 2 scan sorted list and find equivalence classes Michele Schimd (DEI) DEI July 8th, / 19

14 A first (draft) sort-based assembly algorithm Overlap by sorting (simple case) Given R = r 1, r 2,..., r R 1 sort R by lexicographic order; 2 scan sorted list and find equivalence classes Finding the overlap If x and y overlap (without errors),there exist a (maximal) k such that suff k (x) = x x k+1 x x k+2... x x = y 1 y 2... y k = pref k (y) for each x create two copies (i.e., aliases) (x, suff k (x)) (x, pref k (x)) and use as sorting key. Michele Schimd (DEI) DEI July 8th, / 19

15 A more realistic picture Mismatches (i.e., errors and mutations) Create ε aliases forming a pairwise disjoint partition of x. Each matched alias guarantees a correspondence of its size (e.g., m/ε if partitions are even) x m a b c m ε a d c y Refine the overlap (e.g., count mismatches for unmatched partitions) Michele Schimd (DEI) DEI July 8th, / 19

16 A more realistic picture Mismatches (i.e., errors and mutations) Create ε aliases forming a pairwise disjoint partition of x. Each matched alias guarantees a correspondence of its size (e.g., m/ε if partitions are even) x m a b c m ε a d c y Refine the overlap (e.g., count mismatches for unmatched partitions) Michele Schimd (DEI) DEI July 8th, / 19

17 A more realistic picture Mismatches (i.e., errors and mutations) Create ε aliases forming a pairwise disjoint partition of x. Each matched alias guarantees a correspondence of its size (e.g., m/ε if partitions are even) x m a b c m ε a d c y Refine the overlap (e.g., count mismatches for unmatched partitions) Michele Schimd (DEI) DEI July 8th, / 19

18 General Algorithm Algorithm: Sort and overlap 1 Partition each x R in ε substrings 2 For each partition x (h) (h = 1,..., ε) Add the alias (x, x (h) ) to L 3 Sort L using x (h) as sorting keys 4 Identify equivalent classes form sorted L 5 Refine overlap and create the contigs 6 Iterate using produced contigs as new R larger ε (i.e., smaller partition) until no more contigs are created or ε > ε max Michele Schimd (DEI) DEI July 8th, / 19

19 Coarse analysis of the algorithm Sorting Sorting step require O(R log R) comparisons, each comparison requires O(m) operations. Michele Schimd (DEI) DEI July 8th, / 19

20 Coarse analysis of the algorithm Sorting Sorting step require O(R log R) comparisons, each comparison requires O(m) operations. Equivalence class identification The step can be conducted in O(R), but the average size of the classes n e is important... Michele Schimd (DEI) DEI July 8th, / 19

21 Coarse analysis of the algorithm Sorting Sorting step require O(R log R) comparisons, each comparison requires O(m) operations. Equivalence class identification The step can be conducted in O(R), but the average size of the classes n e is important overlap extension and refinement For each of the equivalence class we must identify true overlapping reads. The all-against-all requires O(n 2 e) (hence the smaller n e the better) Michele Schimd (DEI) DEI July 8th, / 19

22 Contig analysis I Definitions Let N be the size of the genome to be reconstructed, R the number of strings, m the length of the strings (suppose m constant) and define the coverage as c = mr/n. Contig extension Starting from one string, a contig is extended if the next read starts at a distance m ( depends from ε). Uncovered probability The probability of no reads starting at a given position is ( ρ 0 1 c ) m Michele Schimd (DEI) DEI July 8th, / 19

23 Contig analysis II A simple model The probability of seeing uncovered positions is P[ uncovered] = ρ 0 At each position n reads are added with probability (1 ρ 0 ) n ρ 0 The average number of reads n r in a contig is E[n r ] = n=0 n(1 ρ 0 ) n ρ 0 = 1 ρ 0 ρ 0 Michele Schimd (DEI) DEI July 8th, / 19

24 Contig analysis III Average number of contig Suppose that we have R d < R distinct reads. The average number of contig n c can be estimate by E[n c ] R d ρ 0 E[n r ] = R d 1 ρ 0 A more accurate result Since R d is not infinite the above expected value E[n r ] should be R d E[n r ] = n(1 ρ 0 ) n ρ 0 n=0 which gives a different results. The error of assuming the simple form is negligible for R d 0. Michele Schimd (DEI) DEI July 8th, / 19

25 Contig analysis IV A (half-surprising) similar work: [Preparata 2013] The work by Franco P. Preparata On Contigs and Coverage published on Journal of Computational Biology solves (almost) the same problem N E[n c ] = E[L c ] + E[L g ] where E[L c ] is the average length of contigs and E[L g ] is the average length of gaps between contigs. We are further developing our theory to match first and expand next the work of Preparata. Michele Schimd (DEI) DEI July 8th, / 19

26 Wrapping up for de-novo assembly Our goal Stochastic analysis of efficacy (number of contigs) and efficiency (size of equivalence classes). Development of an assembler Michele Schimd (DEI) DEI July 8th, / 19

27 Wrapping up for de-novo assembly Our goal Stochastic analysis of efficacy (number of contigs) and efficiency (size of equivalence classes). Development of an assembler Current results A promising sorting-based algorithm Preliminary bounds (e.g., contigs) Michele Schimd (DEI) DEI July 8th, / 19

28 Wrapping up for de-novo assembly Our goal Stochastic analysis of efficacy (number of contigs) and efficiency (size of equivalence classes). Development of an assembler Current results A promising sorting-based algorithm Up next Preliminary bounds (e.g., contigs) Size of equivalent classes n e Complexity analysis T (N, R, m, ε) Associated trade-off Michele Schimd (DEI) DEI July 8th, / 19

29 Alignment-free measures Joint work with Matteo Comin Background When I started working on de-novo assembly we focused on quality scores which are numerical values assigned to each nucleobase produced by modern sequencer. Michele Schimd (DEI) DEI July 8th, / 19

30 Alignment-free measures Joint work with Matteo Comin Background When I started working on de-novo assembly we focused on quality scores which are numerical values assigned to each nucleobase produced by modern sequencer. QVs and assembly The ideas was to use QVs to solve de-novo assembly, this task lead to a stochastic model of sequencing (i.e., my Ph.D. thesis) with applications to alignment-free measures. Michele Schimd (DEI) DEI July 8th, / 19

31 Alignment-free measures Joint work with Matteo Comin Background When I started working on de-novo assembly we focused on quality scores which are numerical values assigned to each nucleobase produced by modern sequencer. QVs and assembly The ideas was to use QVs to solve de-novo assembly, this task lead to a stochastic model of sequencing (i.e., my Ph.D. thesis) with applications to alignment-free measures. Alignment-free measures Alignment-free measures are statistics used to assess the (dis)similarity between sequences without using alignment (i.e., overlap) primitives. Michele Schimd (DEI) DEI July 8th, / 19

32 Where are we now? Our measure D q 2 Use an IID model of errors to compute w ( ) P C (w) = 1 10 q 10 l=1 For each observed k-mer (i.e., a k-long string) w compute X q w = r R Finally compute a statistic measure: D q 2 = P C (w) w r w Σ k X q w Y q w = X q w Y q w Michele Schimd (DEI) DEI July 8th, / 19

33 Results and future extensions So far... These measures give good results on clustering of reads (e.g., identification of reads from different species). Recent preliminary results indicate that phylogeny tree reconstruction can benefit from them Michele Schimd (DEI) DEI July 8th, / 19

34 Results and future extensions So far... These measures give good results on clustering of reads (e.g., identification of reads from different species). Recent preliminary results indicate that phylogeny tree reconstruction can benefit from them... what s next We believe that D q 2 measures can be used to improve compression overlap in our sort-based assembler... ideas? Michele Schimd (DEI) DEI July 8th, / 19

35 Thank you Questions and discussion Further... Coffee break ACG Lab, Room 416 DEI/G, Michele Schimd (DEI) DEI July 8th, / 19