15.4 Longest common subsequence

Transcription

1 15.4 Longest common subsequence Biological applications often need to compare the DNA of two (or more) different organisms A strand of DNA consists of a string of molecules called bases, where the possible bases are Adenine, Guanine, Cytosine, and Thymine We express a strand of DNA as a string over the alphabet {A,C,G,T} E.g., the DNA of two organisms may be = ACCGGTCGAGTGCGCGGAAGCCGGCCGAA = GTCGTTCGGAATGCCGTTGCTCTGTAAA MAT AADS, Fall Oct By comparing two strands of DNA we determine how similar they are, as some measure of how closely related the two organisms are We can de ne similarity in many different ways E.g., we can say that two DNA strands are similar if one is a substring of the other Neither nor is a substring of the other Alternatively, we could say that two strands are similar if the number of changes needed to turn one into the other is small MAT AADS, Fall Oct

2 Yet another way to measure the similarity of and is by nding a third strand in which the bases in appear in each of and these bases must appear in the same order, but not necessarily consecutively The longer the strand we can nd, the more similar and are In our example, the longest strand is = ACCGGTCGAGTGCGCGGAAGCCGGCCGAA = GTCGTTCGGAATGCCGTTGCTCTGTAAA = GTCGTCGGAAGCCGGCCGAA MAT AADS, Fall Oct Formalize this notion of similarity as the longestcommon-subsequence problem A subsequence is just the given sequence with zero or more elements left out Formally, given a sequence =,,,, another sequence =,,, is a subsequence of if there exists a strictly increasing sequence,,, of indices of such that for all =1,2,,, we have = For example, =,,, is a subsequence of =,,,,,, with corresponding index sequence 2,3,5,7 MAT AADS, Fall Oct

3 A sequence is a common subsequence of and if is a subsequence of both and For example, if =,,,,,, and =,,,,,, the sequence,, is a common subsequence of both and It is not a longest common subsequence (LCS) of and The sequence,,, is also common to both and and has length 4 This sequence is an LCS of and, as is,,, ; and have no common subsequence of length 5 or greater MAT AADS, Fall Oct In longest-common-subsequence problem, we are given =,,, and =,,, and wish to nd a max-length common subsequence of and Step 1: Characterizing a longest common subsequence In a brute-force approach, we would enumerate all subsequences of and check each of them to see whether it is also a subsequence of, keeping track of the longest subsequence we nd Each subsequence of corresponds to a subset of the indices 1,2,, of Because has 2 subsequences, this approach requires exponential time, making it impractical for long sequences MAT AADS, Fall Oct

4 The LCS problem has an optimal-substructure property, however, as the following theorem shows The natural classes of subproblems correspond to pairs of pre xes of the two input sequences Precisely, given a sequence =,,,, we de ne the th pre x of, for =0,1,,, as =,,, For example, if =,,,,,,, then =,,, and is the empty sequence MAT AADS, Fall Oct Theorem 15.1 (Optimal substructure of LCS) Let =,,, and =,,, be sequences, and let =,,, be any LCS of and. 1. If =, then = = and is an LCS of and. 2. If, then implies that is an LCS of and. 3. If, then implies that is an LCS of and. MAT AADS, Fall Oct

5 Proof (1) If, then we could append = to to obtain a common subsequence of and of length +1, contradicting the supposition that is a LCS of and. Thus, we must have = =. Now, the pre x is a length-( 1) common subsequence of and. We wish to show that it is an LCS. Suppose for the purpose of contradiction that there exists a common subsequence of and with length greater than 1. Then, appending = to produces a common subsequence of and whose length is greater than, which is a contradiction. MAT AADS, Fall Oct (2) If, then is a common subsequence of and. If there were a common subsequence with length greater than, then would also be a common subsequence of and, contradicting the assumption that is an LCS of and. (3) The proof is symmetric to (2). Theorem 15.1 tells us that an LCS of two sequences contains within it an LCS of pre xes of the two sequences Thus, the LCS problem has an optimalsubstructure property MAT AADS, Fall Oct

6 Step 2: A recursive solution We examine either one or two subproblems when nding an LCS of and If =, we nd an LCS of and Appending = yields an LCS of and If, then we (1) nd an LCS of and and (2) nd an LCS of and Whichever of these two LCSs is longer is an LCS of and These cases exhaust all possibilities, and we know that one of the optimal subproblem solutions must appear within an LCS of and MAT AADS, Fall Oct To nd an LCS of and, we may need to nd the LCSs of and and of and Each subproblem has the subsubproblem of nding an LCS of and Many other subproblems share subsubproblems As in the matrix-chain multiplication, recursive solution to the LCS problem involves a recurrence for the value of an optimal solution Let us de ne [, ] to be the length of an LCS of the sequences and If either =0or =0, one of the sequences has length 0, and so the LCS has length 0 MAT AADS, Fall Oct

7 The optimal substructure of the LCS problem gives 0 if =0 or =0, = 1, 1 +1 if, >0 and = max (, 1, 1, ) if, >0 and Observe that a condition in the problem restricts which subproblems we may consider When =, we consider nding an LCS of and Otherwise, we instead consider the two subproblems of nding an LCS of and and of and In the previous dynamic-programming algorithms for rod cutting and matrix-chain multiplication we ruled out no subproblems due to conditions in the problem MAT AADS, Fall Oct Step 3: Computing the length of an LCS Since the LCS problem has only distinct subproblems, we can use dynamic programming to compute the solutions bottom up LCS-LENGTH stores the [, ] values in [0..,0.. ], and it computes the entries in row-major order I.e., the procedure lls in the rst row of from left to right, then the second row, and so on The procedure also maintains the table [1..,1.. ] Intuitively, [, ] points to the table entry corresponding to the optimal subproblem solution chosen when computing,, contains the length of an LCS of and MAT AADS, Fall Oct

8 LCS-LENGTH(, ) let [1..,1.. ]and [0..,0.. ] be new tables 4. for 1 to 5., for 0to 7. 0, 0 8. for 1 to 9. for 1to 10. if = 11., 1, [, 13. elseif [ 1, [, 1] 14., [ 1, ] 15. [, 16. else, [, 1] 17., 18. return and Running time: ( ) MAT AADS, Fall Oct The and tables computed by LCS-LENGTH on =,,,,,, and =,,,,, MAT AADS, Fall Oct

9 Step 4: Constructing an LCS The table returned by LCS-LENGTH enables us to quickly construct an LCS of and We simply begin at [, ] and trace through the table by following the arrows Whenever we encounter a in entry,, it implies that = is an element of the LCS that LCS-LENGTH found With this method, we encounter the elements of this LCS in reverse order A recursive procedure prints out an LCS of and in the proper, forward order MAT AADS, Fall Oct The square in row and column contains the value of, and the appropriate arrow for the value of [, ] The entry 4 in [7,6] the lower right-hand corner of the table is the length of an LCS,,, For, >0, entry, depends only on whether = and the values in entries 1,,, 1, and 1, 1, which are computed before, To reconstruct the elements of an LCS, follow the [, ] arrows from the lower right-hand corner Each on the shaded sequence corresponds to an entry (highlighted) for which = is a member of an LCS MAT AADS, Fall Oct

10 Improving the code Each, entry depends on only 3 other table entries: 1,,, 1, and 1, 1 Given the value of,, we can determine in (1) time which of these three values was used to compute,, without inspecting table We can reconstruct an LCS in ( + ) time The auxiliary space requirement for computing an LCS does not asymptotically decrease, since we need space for the table anyway MAT AADS, Fall Oct We can, however, reduce the asymptotic space requirements for LCS-LENGTH, since it needs only two rows of table at a time the row being computed and the previous row This improvement works if we need only the length of an LCS if we need to reconstruct the elements of an LCS, the smaller table does not keep enough information to retrace our steps in ( + ) time MAT AADS, Fall Oct

11 15.5 Optimal binary search trees We are designing a program to translate text Perform lookup operations by building a BST with words as keys and their equivalents as satellite data We can ensure an (lg ) search time per occurrence by using a RBT or any other balanced BST A frequently used word may appear far from the root while a rarely used word appears near the root We want frequent words to be placed nearer the root How do we organize a BST so as to minimize the number of nodes visited in all searches, given that we know how often each word occurs? MAT AADS, Fall Oct What we need is an optimal binary search tree Formally, given a sequence =,,, distinct sorted keys ( < < ), we wish to build a BST from these keys For each key, we have a probability that a search will be for Some searches may be for values not in, so we also have +1 dummy keys,,, representing values not in In particular, represents all values less than, represents all values greater than of MAT AADS, Fall Oct

12 For = 1,2,, 1, the dummy key represents all values between and For each dummy key, we have a probability that a search will correspond to MAT AADS, Fall Oct Each key is an internal node, and each dummy key is a leaf Every search is either successful ( nds a key ) or unsuccessful ( nds a dummy key ), and so we have + =1 Because we have probabilities of searches for each key and each dummy key, we can determine the expected cost of a search in a given BST MAT AADS, Fall Oct

13 Let us assume that the actual cost of a search equals the number of nodes examined, i.e., the depth of the node found by the search in +1 Then the expected cost of a search in, E search cost in = depth +1 + (depth + 1) =1+ depth + depth, where depth denotes a node s depth in tree MAT AADS, Fall Oct Node Depth Probability Contribution Total 2.80 MAT AADS, Fall Oct

14 For a given set of probabilities, we wish to construct a BST whose expected search cost is smallest We call such a tree an optimal binary search tree An optimal BST for the probabilities given has expected cost 2.75 An optimal BST is not necessarily a tree whose overall height is smallest Nor can we necessarily construct an optimal BST by always putting the key with the greatest probability at the root The lowest expected cost of any BST with at the root is 2.85 MAT AADS, Fall Oct MAT AADS, Fall Oct

15 Step 1: The structure of an optimal BST Consider any subtree of a BST It must contain keys in a contiguous range,,, for some In addition, a subtree that contains keys,, must also have as its leaves the dummy keys,, If an optimal BST has a subtree containing keys,,, then this subtree must be optimal as well for the subproblem with keys,, and dummy keys,, MAT AADS, Fall Oct Given keys,,, one of them, say, is the root of an optimal subtree containing these keys The left subtree of the root contains the keys,, (and dummy keys,, ) The right subtree contains the keys,, (and dummy keys,, ) As long as we examine all candidate roots, where, and determine all optimal BSTs containing,, and those containing,,, we are guaranteed to nd an optimal BST MAT AADS, Fall Oct

16 Suppose that in a subtree with keys,,, we select as the root s left subtree contains the keys,, Interpret this sequence as containing no keys Subtrees, however, also contain dummy keys Adopt the convention that a subtree containing keys,, has no actual keys but does contain the single dummy key Symmetrically, if we select as the root, then s right subtree contains no actual keys, but it does contain the dummy key MAT AADS, Fall Oct Step 2: A recursive solution We pick our subproblem domain as nding an optimal BST containing the keys,,, where 1,, and 1 Let us de ne [, ] as the expected cost of searching an optimal BST containing the keys,, Ultimately, we wish to compute [1, ] The easy case occurs when = 1 Then we have just the dummy key The expected search cost is, 1 = MAT AADS, Fall Oct

17 When >, we need to select a root from among,, and make an optimal BST with keys,, as its left subtree and an optimal BST with keys,, as its right subtree What happens to the expected search cost of a subtree when it becomes a subtree of a node? Depth of each node increases by 1 Expected search cost of this subtree increases by the sum of all the probabilities in it For a subtree with keys,,, let us denote this sum of probabilities as, = + MAT AADS, Fall Oct Thus, if is the root of an optimal subtree containing keys,,, we have, = +, 1 +, 1 +( + 1, + + 1, ) Noting that, =, ( + 1, ) we rewrite, =, , + (, ) We choose the root that gives the lowest expected search cost:, = min, =, , + (, ) if = 1 if MAT AADS, Fall Oct

18 The, values give the expected search costs in optimal BSTs To help us keep track of the structure of optimal BSTs, we de ne root,, for, to be the index for which is the root of an optimal BST containing keys,, Although we will see how to compute the values of root,, we leave the construction of an optimal binary search tree from these values as an exercise MAT AADS, Fall Oct Step 3: Computing the expected search cost of an optimal BST We store, values in a table ,0.. The rst index needs to run to +1because to have a subtree containing only the dummy key, we need to compute and store + 1, The second index needs to start from 0 because to have a subtree containing only the dummy key, we need to compute and store 1,0 MAT AADS, Fall Oct

19 We use only the entries, for which 1 We also use a table root,, for recording the root of the subtree containing keys,, This table uses only the entries We also store the, [ ,0.. ] values in a table For the base case, we compute, 1 = For, we compute, =, Thus, we can compute the ( ) values of, in (1) time each MAT AADS, Fall Oct OPTIMAL-BST(,, ) 1. let ,0.., [ ,0.. ], root 1..,1.. be new tables 2. for =1 to +1 3., 1 = 4., 1 = 5. for =1 to 6. for =1 to = , 9., =, for = to 11. =, , + [, ] 12. if < [, ] 13., = 14. root, = 15.return and root MAT AADS, Fall Oct

20 MAT AADS, Fall Oct The OPTIMAL-BST procedure takes ( ) time, just like MATRIX-CHAIN-ORDER Its running time is ( ), since its for loops are nested three deep and each loop index takes on at most values The loop indices in OPTIMAL-BST do not have exactly the same bounds as those in MATRIX- CHAIN-ORDER, but they are within 1in all directions Thus, like MATRIX-CHAIN-ORDER, the OPTIMAL- BST procedure takes ( ) time MAT AADS, Fall Oct

21 16 Greedy Algorithms Optimization algorithms typically go through a sequence of steps, with a set of choices at each For many optimization problems, using dynamic programming to determine the best choices is overkill; simpler, more ef cient algorithms will do A greedy algorithm always makes the choice that looks best at the moment That is, it makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution MAT AADS, Fall Oct An activity-selection problem Suppose we have a set =,,, of proposed activities that wish to use a resource (e.g., a lecture hall), which can serve only one activity at a time Each activity has a start time and a nish time, where < If selected, activity takes place during the halfopen time interval [, ) MAT AADS, Fall Oct

22 Activities and are compatible if the intervals [, ) and [, ) do not overlap I.e., and are compatible if or We wish to select a maximum-size subset of mutually compatible activities We assume that the activities are sorted in monotonically increasing order of nish time: MAT AADS, Fall Oct Consider, e.g., the following set of activities: For this example, the subset,, consists of mutually compatible activities It is not a maximum subset, however, since the subset,,, is larger In fact, it is a largest subset of mutually compatible activities; another largest subset is,,, MAT AADS, Fall Oct

23 The optimal substructure of the activityselection problem Let be the set of activities that start after nishes and that nish before starts We wish to nd a maximum set of mutually compatible activities in Suppose that such a maximum set is, which includes some activity By including in an optimal solution, we are left with two subproblems: nding mutually compatible activities in the set and nding mutually compatible activities in the set MAT AADS, Fall Oct Let = and =, so that contains the activities in that nish before starts and contains the activities in that start after nishes Thus, =, and so the maximum-size set in consists of = + +1activities The usual cut-and-paste argument shows that the optimal solution must also include optimal solutions for and MAT AADS, Fall Oct

24 This suggests that we might solve the activityselection problem by dynamic programming If we denote the size of an optimal solution for the set by [, ], then we would have the recurrence, =, +, +1 Of course, if we did not know that an optimal solution for the set includes activity, we would have to examine all activities in to nd which one to choose, so that 0 if, = max, =, +, +1 if MAT AADS, Fall Oct Making the greedy choice For the activity-selection problem, we need consider only the greedy choice We choose an activity that leaves the resource available for as many other activities as possible Now, of the activities we end up choosing, one of them must be the rst one to nish Choose the activity in with the earliest nish time, since that leaves the resource available for as many of the activities that follow it as possible Activities are sorted in monotonically increasing order by nish time; greedy choice is activity MAT AADS, Fall Oct

25 If we make the greedy choice, we only have to nd activities that start after nishes < and is the earliest nish time of any activity no activity can have a nish time Thus, all activities that are compatible with activity must start after nishes Let = : be the set of activities that start after nishes Optimal substructure: if is in the optimal solution, then an optimal solution to the original problem consists of and all the activities in an optimal solution to the subproblem MAT AADS, Fall Oct Theorem 16.1 Consider any nonempty subproblem, and let be an activity in with the earliest nish time. Then is included in some maximum-size subset of mutually compatible activities of. Proof Let be a max-size subset of mutually compatible activities in, and let be the activity in with the earliest nish time. If =, we are done, since is in a max-size subset of mutually compatible activities of. If, let the set =. The activities in are disjoint because the activities in are disjoint, is the rst activity in to nish, and. Since =, we conclude that is a maximum-size subset of mutually compatible activities of and includes. MAT AADS, Fall Oct

26 We can repeatedly choose the activity that nishes rst, keep only the activities compatible with this activity, and repeat until no activities remain Moreover, because we always choose the activity with the earliest nish time, the nish times of the activities we choose must strictly increase We can consider each activity just once overall, in monotonically increasing order of nish times MAT AADS, Fall Oct A recursive greedy algorithm RECURSIVE-ACTIVITY-SELECTOR(,,, ) while and [ ] < [ ] // nd the rst // activity in to nish if 5. return RECURSIVE-ACTIVITY- SELECTOR(,,, ) 6. else return MAT AADS, Fall Oct

27 MAT AADS, Fall Oct An iterative greedy algorithm GREEDY-ACTIVITY-SELECTOR(, ) for 2to 5. if [ [ ] return MAT AADS, Fall Oct

28 The set returned by the call GREEDY-ACTIVITY-SELECTOR(, ) is precisely the set returned by the call RECURSIVE-ACTIVITY-SELECTOR(,,, ) Both the recursive version and the iterative algorithm schedule a set of activities in ( ) time, assuming that the activities were already sorted initially by their nish times MAT AADS, Fall Oct