Dynamic Programming Part I: Examples. Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

Transcription

1 Dynamic Programming Part I: Examples Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

2 Dynamic Programming Recall: the Change Problem Other problems: Manhattan Tourist Problem, LCS Problem Finally: Sequence alignments Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

3 Manhattan Tourist Problem (MTP) Imagine seeking a path (from source to sink) to travel (only eastward and southward) with the most number of attractions (*) in the Manhattan grid ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

4 Manhattan Tourist Problem (MTP) Imagine seeking a path (from source to sink) to travel (only eastward and southward) with the most number of attractions (*) in the Manhattan grid ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

5 Manhattan Tourist Problem: Formulation Goal: Find the longest path in a weighted grid. Input: A weighted grid G with two distinct vertices, one labeled source" and the other labeled sink" Output: A longest path in G from source" to sink" Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

6 MTP: An example Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

7 MTP: Greedy Algorithm Is Not Optimal Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

8 MTP: Simple Recursive Program 1 MT(n,m) 2 if n = 0 or m = 0 3 return MT(n,m) 4 x MT(n-1,m)+ length of the edge from (n 1, m) to (n, m) 5 y MT(n,m-1)+length of the edge from (n, m 1) to (n, m) 6 return max{x,y} Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

9 MTP: Simple Recursive Program 1 MT(n,m) 2 if n = 0 or m = 0 3 return MT(n,m) 4 x MT(n-1,m)+ length of the edge from (n 1, m) to (n, m) 5 y MT(n,m-1)+length of the edge from (n, m 1) to (n, m) 6 return max{x,y} What s wrong with this approach? Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

10 MTP: Dynamic Programming Calculate optimal path score for each vertex in the graph Each vertex s score is the maximum of the prior vertices score plus the weight of the respective edge in between Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

11 MTP: Dynamic Programming Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

12 MTP: Dynamic Programming Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

13 MTP: Dynamic Programming Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

14 MTP: Dynamic Programming Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

15 MTP: Dynamic Programming Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

16 MTP: Recurrence Computing the score for a point (i, j) by the recurrence relation: { si 1,j + weight of the edge between (i 1, j)and (i, j) s i,j = max s i,j 1 + weight of the edge between (i, j 1)and (i, j) The running time is n m for a n by m grid (n = # of rows, m = # of columns) Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

17 Manhattan is not a perfect Grid Represented as a DAG: Directed Acyclic Graph The score at point B is given by: s i,j = max { si 1,j + weight of the edge between(i 1, j)and(i, j) s i,j 1 + weight of the edge between(i, j 1)and(i, j) Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

18 Manhattan Is Not A Perfect Grid Computing the score for point x is given by the recurrence relation: { sy + weight of vertex(y, x)where s x = max y Predecessors(x) Predecessors(x): set of vertices that have edges leading to x. The running time for a graph G(V, E) (V is the set of all vertices and E is the set of all edges) is O(E) since each edge is evaluated once. Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

19 Longest Path in DAG Problem Goal: Find a longest path between two vertices in a weighted DAG Input: A weighted DAG G with source and sink vertices Output: A longest path in G from source to sink Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

20 Longest Path in DAG: Dynamic Programming Suppose vertex v has indegree 3 and predecessors {u 1, u 2, u 3 } Longest path to v from source is: s u1 + weight of edge from u 1 to v s v = max s u2 + weight of edge from u 2 to v s u3 + weight of edge from u 3 to v In general: s v = max u {s u + weight of edge from u to v} Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

21 Traveling in the Grid The only hitch is that one must decide on the order in which visit the vertices By the time the vertex x is analyzed, the values s y for all its predecessors y should be computed - otherwise we are in trouble We need to traverse the vertices in some order Try to find such order for a directed cycle??? Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

22 Topological ordering 2 different topological orderings of the DAG Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

23 Traversing the Manhattan Grid 3 different strategies: a) Column by column b) Row by row c) Along diagonals ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

24 Pseudo-code MTP: Dynamic Programming 1 ManhattanTourist(w, w,w i,j,n,m) 2 for i 1 to n 3 s i,0 s i 1,0 + w i,0 4 for j 1 to m 5 s 0,j s 0,j 1 + w 0,j 6 for i 1 to n 7 for j 1 to m 8 s i 1,j + w i,j s i,j = max s i,j 1 + w i,j s i 1,j 1 + w i,j 9 return s n,m Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

25 Dynamic Programming Part II: Edit Distance & Alignments Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

26 Aligning DNA Sequences Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

27 LCS Alignment without mismatches LCS: Longest Common Subsequence Given two sequences v = v 1 v 2...v m and w = w 1 w 2...w n The LCS of v and w is a sequence of positions in v : 1 < i 1 < i 2 <... < i t < m and a sequence of positions in w : 1 < j 1 < j 2 <... < j t < n such that i t -th letter of v equals to j t -th letter of w and t is maximal Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

28 LCS: Example Every common subsequence is a path in 2-D grid Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

29 LCS Problem as Manhattan Tourist Problem Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

30 Edit Graph for LCS Problem Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

31 Edit Graph for LCS Problem Every path is a common subsequence. Every diagonal edge adds an extra element to common subsequence. LCS Problem: Find a path with maximum number of diagonal edges. Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

32 Computing LCS Let v i = prefix of v of length i: v 1...v i and w j = prefix of w of length j: w 1...w j Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

33 Every Path in the Grid Corresponds to an Alignment Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

34 Aligning Sequences without Insertions and Deletions: Hamming Distance Given two DNA sequences v and w : The Hamming distance: d H (v, w) = 8 is large but the sequences are very similar Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

35 Aligning Sequences with Insertions and Deletions By shifting one sequence over one position: The edit distance: d L (v, w) = 2 Hamming distance neglects insertions and deletions in DNA Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

36 Levenshtein or edit distance Definition The Levenshtein distance or edit distance d L between two sequences X and Y is is the minimum number of edit operations of type Replacement, Insertion, or Deletion, that one needs to transform sequence X into sequence Y : d L (X, Y ) = min{r(x, Y ) + I (X, Y ) + D(X, Y )} Using M for match, an edit transcript is a string over the alphabet I, D, R, M that describes a transformation of X to Y. Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

37 Levenshtein or edit distance X = YESTERDAY Example: Given two strings. Y = EASTERS Here is a minimum edit transcript for the above example: edit transcript= D M I M M M M R D D X= Y E S T E R D A Y Y= E A S T E R S The edit distance d L (X, Y ) of X, Y is 5. Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

38 Edit Distance vs Hamming Distance Hamming distance always compares i th letter of v with i th letter of w Hamming distance: d(v, w) = 8 Computing Hamming distance is a trivial task. Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

39 Edit Distance vs Hamming Distance Hamming distance always compares i th letter of v with i th letter of w Edit distance may compare i th letter of v with j th letter of w Hamming distance: d(v, w) = 8 Computing Hamming distance is a trivial task. Edit distance: d(v, w) = 2 Computing edit distance is a non-trivial task. Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

40 Edit Distance: Example ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

41 Edit Distance: Example What is the edit distance? 5? Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

42 Edit Distance: Example ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

43 Edit Distance: Example Can it be done in 3 steps? Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

44 The Alignment Grid Every alignment path is from source to sink ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

45 Alignment as a path in the Edit Graph ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

46 Alignments in Edit Graph ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

47 Alignments in Edit Graph Every path in the edit graph corresponds to alignment: ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

48 Alignments in Edit Graph ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

49 Alignments in Edit Graph ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

50 Alignment: Dynamic Programming s i 1,j s i,j = max s i 1,j s i,j 1 if v i = w j Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

51 Dynamic Programming Example Initialize 1st row and 1st column to be all zeroes Or, to be more precise, initialize 0 th row and 0 th column to be all zeroes ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

52 Dynamic Programming Example s i 1,j 1 : value from NW +1 if v s i,j = max i = w j s i 1,j : value from N s i,j 1 : value from W ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

53 Alignment: Backtracking Arrows indicate where the score originated from: Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

54 Backtracking Example Find a match in row and column 2 i=2, j=2, 5 is a match (T). j=2, i=4, 5, 7 is a match (T). Since v i = w j, s i,j = s i 1,j s 2,2 = [s 1,1 = 1] + 1 s 2,5 = [s 1,4 = 1] + 1 s 4,2 = [s 3,1 = 1] + 1 s 5,2 = [s 4,1 = 1] + 1 s 7,2 = [s 6,1 = 1] + 1 ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

55 Dynamic Programming Example Continuing with the dynamic programming algorithm gives this result. ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

56 Alignment: Dynamic Programming s i 1,j s i,j = max s i 1,j s i,j 1 if v i = w j Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

57 Alignment: Dynamic Programming s i 1,j s i,j = max s i 1,j s i,j 1 if v i = w j This recurrence corresponds to the Manhattan Tourist problem (three incoming edges into a vertex) with all horizontal and vertical edges weighted by zero. Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

58 LCS Algorithm 1 LCS(v,w) 2 for i 1 to n 3 s i,0 0 4 for j 1 to m 5 s 0,j 0 6 for i 1 to n 7 for j 1 to m return s i 1,j s i,j = max s i 1,j s i,j 1 b i,j = if v i = w j if s i,j = s i 1,j if s i,j = s i,j 1 if s i,j = s i 1,j 1 Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

59 Now What? LCS(v,w) created the alignment grid Now we need a way to read the best alignment of v and w Follow the arrows backwards from sink ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

60 Printing LCS: Backtracking 1 PrintLCS(b,v,i,j) 2 if i = 0 or j = 0 3 return 4 if b i,j = 5 PrintLCS(b,v,i-1,j-1) 6 print v i 7 else 8 if b i,j = 9 PrintLCS(b,v,i-1,j) 10 else 11 PrintLCS(b,v,i,j-1) Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

61 Now What? Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

62 Now What? Alignment: A T C G - T A C - A T - G T T A - T ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

63 LCS Runtime It takes O(nm) time to fill in the n m dynamic programming matrix. Why O(nm)? The pseudocode consists of a nested for" loop inside of another for" loop to set up a n m matrix. Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

64 Dynamic Programming Part II: Sequence Alignment Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

65 Outline Dot plots Global Alignment Scoring Matrices Local Alignment Alignment with Affine Gap Penalties Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

66 Types of sequence alignments Dot plots Number of sequences pairwise: compares two sequences multiple: compares several sequences Portion of aligned sequence global: aligns the sequences over all their length local: finds subsequences with the best similarity scores Algorithms Optimal methods: Needleman-Wunsch, Smith-Waterman Heuristics: FASTA, BLAST Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

67 Dot plot the simplest way to visualize the similarity between two protein/dna sequences is to use a similarity matrix Identification of insertions/deletions Identification of direct repeats or inversions Steps to create a dot plot Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

68 Dot plot the simplest way to visualize the similarity between two protein/dna sequences is to use a similarity matrix Identification of insertions/deletions Identification of direct repeats or inversions Steps to create a dot plot 2D matrix One sequence on the top One sequence on the left For each matrix cell, compare the symbols and draw a point if there is a coincidence Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

69 Dot matrix sequence comparison A dot matrix analysis is primarily a method for comparing two sequences. An (n m) matrix relating two sequences of length n and m respectively is produced by placing a dot at each cell for which the corresponding symbols match. Here is an example for the two sequences: IMISSMISSISSIPPI and MYMISSISAHIPPIE: ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

70 Dot matrix sequence comparison Definition Let S = s 1 s 2... s n and T = t 1... t m be two strings of length n and m respectively. Let M be an n m matrix. Then M is a (simple) dot plot if for i, j, 1 i n, 1 j m : M[i, j] = { 1 for si = t j 0 else. Note: The longest common substring within the two strings S and T is then the longest matrix subdiagonal containing only 1s. However, rather than drawing the letter 1 we draw a dot. Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

71 Dot matrix sequence comparison Some of the properties of a dot plot are the visualization is easy to understand it is easy to find common substrings, they appear as contiguous dots along a diagonal it is easy to find reversed substrings it is easy to discover displacements it is easy to find repeats Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

72 Dot plots Sequence length: n and m O(nm) DNA Protein Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

73 Noise in Dot plot LDL human receptor compared to himself The low density lipoprotein (LDL) receptor is a cell surface protein that plays a central role in the metabolism of cholesterol in humans and animals. Mutations affecting its structure and function give rise to one of the most prevalent human genetic diseases, familial hypercholesterolemia. ioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

74 Reducing the noise To reduce the noise, a window size w and a stringency s are used and a dot is only drawn at point (x, y) if in the next w positions at least s characters are equal. Example: Phage P22 w = 1, s = 1 w = 11, s = 7 w = 23, s = 15 Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

75 Random similarity in Dot plots When comparing DNA, there is a 1 4 probability of random matches When comparing protein sequences there is a 1 20 probability of random matches Hence, if coding DNA regions are analized: translate first, then align! You can always go back to DNA after alignment Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

76 w=1 Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

77 w=3 Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

78 w=3, stringency 2 Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

79 DNA sequence Simple dot plot, w = 1 w = 23, stringency = 16 Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

80 Protein sequence Simple dot plot, w = 1 w = 23, stringency = 6 Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

81 Insertion Deletion & Inversion Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

82 Repeats ABCDEFGEFGHIJKLMNO Tandem duplication Compared to non duplicated Tandem duplication Compared to itself Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

83 Palindromic repeat (intra chain) 5 GGCGG 3 Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77

84 Limitations of dot plots No score to quantify identical or similar strings Runtime is quadratic; more efficient algorithms to identify identical substrings exist (eg. based on suffix trees) Bioinfo I (Institut Pasteur de Montevideo) Dynamic Programming -class4- July 25th, / 77