Divide and Conquer. Bioinformatics: Issues and Algorithms. CSE Fall 2007 Lecture 12

Transcription

1 Divide and Conquer Bioinformatics: Issues and Algorithms CSE Fall 007 Lecture 1 Lopresti Fall 007 Lecture 1-1 -

2 Outline MergeSort Finding mid-point in alignment matrix in linear space Linear space sequence alignment Block alignment Four-Russians speedup Constructing LCS in sub-quadratic time Lopresti Fall 007 Lecture 1 - -

3 Divide and Conquer Algorithms Divide problem into sub-problems. Conquer by solving sub-problems recursively. If subproblems are small enough, solve them in brute force fashion. Combine solutions of sub-problems into solution for original problem (tricky part). Lopresti Fall 007 Lecture 1-3 -

4 The Sorting Problem revisted Given an unsorted array of integers: The goal is to put it in sorted order: Lopresti Fall 007 Lecture 1-4 -

5 Lopresti Fall 007 Lecture MergeSort: divide step Step 1 Divide log(n) divisions to split an array of size n into single elements

6 MergeSort: conquer step Step Conquer O(n) O(n) O(n) O(n) We have logn iterations, each taking O(n) time. So... Total time: O(n logn) Lopresti Fall 007 Lecture 1-6 -

7 MergeSort: combine step Step 3 Combine 5 5 Two arrays of size 1 can be easily merged to form a single sorted array of size. Two sorted arrays of size n and m can be merged in O(n + m) time to form a single sorted array of size n + m. Lopresti Fall 007 Lecture 1-7 -

8 Lopresti Fall 007 Lecture MergeSort: combine step Etcetera Combining two arrays of size 4:

9 Merge subroutine Merge(a, b) n1 size of array a; n size of array b a n1+1 ; b n+1 i 1; j 1 for k 1 to n1 + n if a i < b j c k a i i i + 1 else c k b j j j + 1 return c Lopresti Fall 007 Lecture 1-9 -

10 MergeSort example Divide Conquer Lopresti Fall 007 Lecture

11 MergeSort algorithm MergeSort(c) n size of array c if n = 1 return c left list of first n / elements of c right list of last n n / elements of c sortedleft MergeSort(left) sortedright MergeSort(right) sortedlist Merge(sortedLeft, sortedright) return sortedlist Lopresti Fall 007 Lecture

12 MergeSort running time The problem is simplified to baby steps: For i th merging iteration, complexity of problem is O(n). Number of iterations is O(logn). Hence, total running time is O(n logn). Lopresti Fall 007 Lecture 1-1 -

13 Divide and Conquer Approach to LCS Recall that LCS is longest common subsequence problem Path(source, sink) if source and sink are in consecutive columns output longest path from source to sink else middle middle vertex between source and sink Path(source, middle) Path(middle, sink) Only problem left is how to find this middle vertex! Lopresti Fall 007 Lecture

14 Computing alignment path requires quadratic memory Alignment path: Space complexity for computing alignment path for sequences of length n and m is O(nm). We need to keep all backtracking references in memory to reconstruct path (backtracking). n m Lopresti Fall 007 Lecture

15 Computing alignment score with linear memory Alignment score: Space complexity of computing just score itself is O(n). We only need previous column to calculate current column, and we can then throw away that column once we re done using it. nn Lopresti Fall 007 Lecture

16 Computing alignment score: recycling columns Only two columns of scores are saved at any given time: memory for column 1 used to calculate column 3 memory for column used to calculate column 4 Lopresti Fall 007 Lecture

17 Crossing the middle line We want to calculate longest path from (0, 0) to (n, m) passing through (i, m/), where i ranges from 0 to n and represents i th row. Define length(i) as length of longest path from (0, 0) to (n, m) passing through vertex (i, m/). n m/ m (i, m/) Prefix(i) Suffix(i) Vertex where longest path crosses middle column is (mid, m/): length(mid) = optimal length = max 0 i n length(i) Lopresti Fall 007 Lecture

18 Computing Prefix(i) Prefix(i) is length of longest path from (0, 0) to (i, m/). Compute Prefix(i) by using dynamic programming in left half of matrix. store Prefix(i) column 0 m/ m Lopresti Fall 007 Lecture

19 Computing Suffix(i) Suffix(i) is length of longest path from (i, m/) to (n, m). Suffix(i) is length of longest path from (n, m) to (i, m/) with all edges reversed. Compute Suffix(i) by using dynamic programming in right half of the reversed matrix. store Suffix(i) column 0 m/ m Lopresti Fall 007 Lecture

20 Length(i) = Prefix(i) + Suffix(i) Add prefix(i) and suffix(i) to compute length(i): length(i) = prefix(i) + suffix(i) We now have middle vertex of maximum path (i, m/) as maximum of length(i). 0 middle point found i 0 m/ m Lopresti Fall 007 Lecture 1-0 -

21 Finding the middle point 0 m/4 m/ 3m/4 m Lopresti Fall 007 Lecture 1-1 -

22 Finding the middle point 0 m/4 m/ 3m/4 m Lopresti Fall 007 Lecture 1 - -

23 And again... 0 m/8 m/4 3m/8 m/ 5m/8 3m/4 7m/8 m Lopresti Fall 007 Lecture 1-3 -

24 Determining the time complexity: first pass On first pass, algorithm covers entire area: Time = area = nm Computing prefix(i) Computing suffix(i) Lopresti Fall 007 Lecture 1-4 -

25 Determining the time complexity: second pass On second pass, algorithm covers only 1/ of area: Time = area = nm / Lopresti Fall 007 Lecture 1-5 -

26 Determining the time complexity: third pass On third pass, only 1/4 of area is covered: Time = area = nm / 4 Lopresti Fall 007 Lecture 1-6 -

27 Determining the time complexity: overall Geometric reduction at each successive iteration: 1 + ½ + ¼ (½) k 5 th pass: 1/16 first pass: 1 3 rd pass: 1/4 4 th pass: 1/8 nd pass: 1/ Hence, time = O(area) = O(nm). Lopresti Fall 007 Lecture 1-7 -

28 Can we improve time complexity for LCS as well? Dynamic Programming takes O(n ) for global alignment. We can use Four-Russians Speedup to do better. Partitioning sequences into blocks: Partition n x n grid into blocks of size t x t. We are comparing two sequences, each of size n, and each sequence is sectioned off into chunks, each of length t. Sequence u = u 1 u n becomes: u 1 u t u t+1 u t u n-1+1 u n and sequence v = v 1 v n becomes: v 1 v t v t+1 v t v n-1+1 v n Lopresti Fall 007 Lecture 1-8 -

29 Partitioning alignment grid into blocks n n/t t n t n/t partition Lopresti Fall 007 Lecture 1-9 -

30 Block alignment Block alignment of sequences u and v: An entire block in u is aligned with an entire block in v. An entire block is inserted. An entire block is deleted. A block path is a path that traverses every t x t square through its corners. A valid block alignment An invalid block alignment Lopresti Fall 007 Lecture

31 The Block Alignment Problem Block Alignment Problem: Find the longest block path through an edit graph. Input: Two sequences, u and v partitioned into blocks of size t. This is equivalent to an n x n edit graph partitioned into t x t subgrids. Output: The block alignment of u and v with maximum score (longest block path through edit graph). Lopresti Fall 007 Lecture

32 Constructing alignments within blocks To solve: compute alignment score ß i,j for each pair of blocks u (i-1)*t+1 u i*t and v (j-1)*t+1 v j*t. How many blocks are there per sequence? (n/t) blocks of size t How many pairs of blocks for aligning the two sequences? (n/t) x (n/t) For each block pair, solve mini-alignment problem of size t x t. Lopresti Fall 007 Lecture 1-3 -

33 Constructing alignments within blocks n/t Solve mini-alignmnent problems Block pair represented by each small square Lopresti Fall 007 Lecture

34 Block alignment: Dynamic Programming Let s i,j denote the optimal block alignment score between first i blocks of u and first j blocks of v. s i,j = max s i-1,j σ block s i,j-1 σ block s i-1,j-1 β i,j σ block is penalty for inserting or deleting an entire block. β i,j is score of pair of blocks in row i and column j. Lopresti Fall 007 Lecture

35 Block alignment runtime Indices i,j range from 0 to n/t. Hence, running time of algorithm is O([n/t]*[n/t]) = O(n /t ) if we don t count the time to compute each ß i,j. Computing all ß i,j requires solving (n/t)*(n/t) mini-block alignments, each of size (t*t). So computing all ß i,j takes time: O([n/t]*[n/t]*t*t) = O(n ) But this is same as original dynamic programming algorithm. How do we speed this up? Lopresti Fall 007 Lecture

36 Four Russians Technique Instead of creating (n/t)*(n/t) mini-alignments, construct 4 t x 4 t mini-alignments for all pairs of strings of t nucleotides (huge size), and put this in a lookup table. However, size of lookup table is not really that big if t is small. Let t = (logn)/4. Then 4 t x 4 t = n. Each sequence has t nucleotides AAAAAA AAAAAC AAAAAG AAAAAT AAAACA Lookup table Score AAAAAA AAAAAC AAAAAG AAAAAT Size is only n, instead of (n/t)*(n/t) AAAACA Lopresti Fall 007 Lecture

37 New recurrence The new lookup table Score is indexed by a pair of t-nucleotide strings, so: s i,j = max s i-1,j σ block s i,j-1 σ block s i-1,j-1 Score(i th block of v, j th block of u) Lopresti Fall 007 Lecture

38 Four Russians Speedup runtime Since computing the lookup table Score of size n takes O(n{logn} ) time, running time is mainly limited by (n/t)*(n/t) accesses to lookup table. Each access takes O(logn) time. Overall running time: O([n /t ] * logn). Since t = logn, substitute in: O([n /{logn} ] * logn) = O(n / logn) Lopresti Fall 007 Lecture

39 Where are we? We can divide up grid into blocks and run dynamic programming only on corners of these blocks. In order to speed up mini-alignment calculations to under n, we create a lookup table of size n, which consists of all scores for all t-nucleotide pairs. Running time goes from O(n ) to subquadratic, O(n / logn). But, unlike blockpartitioned graph, LCS path does not have to pass through vertices of blocks. vs. Lopresti Fall 007 Lecture

40 Block alignment vs. LCS In block alignment, we only care about corners of blocks. In LCS, we care about all points on edges of blocks, because those are points that path can traverse. Recall, each sequence is of length n, each block is of size t, so each sequence has (n/t) blocks. vs. Lopresti Fall 007 Lecture

41 Block alignment vs. LCS: points of interest Block alignment has (n/t)*(n/t) = (n /t ) points of interest. LCS alignment has O(n /t) points of interest. Lopresti Fall 007 Lecture

42 Traversing blocks for LCS Given alignment scores s i,* in the first row and scores s *,j in the first column of a t x t mini-block, compute alignment scores in last row and column of mini-block. To compute last row and last column score, we use these 4 variables: 1. alignment scores s i,* in first row.. alignment scores s *,j in first column. 3. substring of sequence u in this block (4 t possibilities). 4. substring of sequence v in this block (4 t possibilities). Lopresti Fall 007 Lecture 1-4 -

43 Traversing blocks for LCS If we used this to compute the grid, it would take quadratic, i.e., O(n ), time, but we want to do better. we know these scores we can calculate these scores t x t block Lopresti Fall 007 Lecture

44 Four Russians Speedup Build a lookup table for all possible values of four variables: 1. all possible scores for the first row s i,*. all possible scores for the first column s *,j 3. substring of sequence u in this block (4 t possibilities) 4. substring of sequence v in this block (4 t possibilities) For each quadruple we store the value of the score for the last row and last column. This will be a huge table, but we can eliminate alignments scores that don t make sense. Lopresti Fall 007 Lecture

45 Reducing lookup table size Alignment scores in LCS are monotonically increasing, and adjacent elements can t differ by more than 1. Example: 0,1,,,3,4 is ok; 0,1,,4,5,8, is not because and 4 differ by more than 1 (and so do 5 and 8). Therefore, we only need to store quadruples whose scores are monotonically increasing and differ by at most 1. Lopresti Fall 007 Lecture

46 Efficient encoding of alignment scores Instead of recording numbers that correspond to the index in the sequences u and v, we can use binary to encode the differences between the alignment scores: original encoding binary encoding Lopresti Fall 007 Lecture

47 Reducing lookup table size There are t possible scores (t = size of blocks). There are 4 t possible strings. Hence, lookup table size is ( t * t ) * (4 t * 4 t ) = 6t Let t = (logn)/4, then we have: Table size = 6((logn)/4) = n (6/4) = n (3/) And since time = O([n /t ] * logn), we have an overall time complexity of O( [n / {logn} ] * logn) = O(n / logn). Lopresti Fall 007 Lecture

48 Summary We take advantage of fact that for each block of size t = log(n), we can pre-compute all possible scores and store them in a lookup table of size n (3/). We used Four Russian speedup to go from quadratic running time for LCS to subquadratic running time: O(n / logn). Lopresti Fall 007 Lecture

49 Wrap-up Readings for next time: IBA Sections (sequencing and assembly). Remember: Come to class having done the readings. Check Blackboard regularly for updates. Lopresti Fall 007 Lecture