1 The Knuth-Morris-Pratt Algorithm

Transcription

1 5-45/65: Design & Analysis of Algorithms September 26, 26 Leture #9: String Mathing last hanged: September 26, 27 There s an entire field dediated to solving problems on strings. The book Algorithms on Strings, Trees, and Sequenes by Dan Gusfield overs this field of researh. Here are some sample problems: Given a tet string and a pattern, find all ourrenes of the pattern in the tet. (Classi tet searh) The above problem where the pattern an have don t ares in it. Given a string, find longest string that ours twie in it. Compute the edit distane between two strings, where various editing operations are defined, along with their osts. Given a set of strings, ompute the heapest tree that onnets them all together (phylogeny tree) Compute the shortest superstring of a set of strings. That is the shortest string that ontains all the given strings as a substring (important as a theoretial model in DNA sequening). Not only do these problems arise in our everyday lives (how do the grep and diff algorithms work?), but they also have many appliations in omputational biology. We re only going to srath the surfae of this field and talk about a ouple of these problems and algorithms for them. The Knuth-Morris-Pratt Algorithm This algorithm an solve the lassi tet searh problem in linear time in the length of the tet string. (It an also be used for a variety of other string searhing problems.) Formally, you have a pattern P of length p, and a tet T of length t, and you want to find all loations i suh that T [i... i + p ] = P. For eample, if you have a pattern P = ana and T = banana, then you would want to output loations {, 3}, sine T [... 4] = T [3... 6] = ana. An easy solution to this problem takes time O(p t). This is fine if p is small, but as p gets large this beomes bad. Can we do better? It seems that if one we have heked whether T [i... i + p ] mathes P or not, we have a lot of information that we an use to determine if T [i +... i + p] gives a math. How do we do this? One high-level solution to this problem is to build a deterministi finite automaton M P, whih depends on the pattern. It onsists of p = P states, whih you should think of as arranged in a line. We feed in the tet T to this automaton. The properties of the DFA M ensure that we re in the j th state if and only if at the urrent loation in the string we ve already mathed a sequene of j haraters from the pattern. Now we ompare the net two haraters. If we get a math we move to the net state in the list (mathing j + haraters). If we get a mismath, we an skip bak to some previous state. Whih one? The start state of the automaton? Nope, that would be wrong. We go to the furthest bak state that we know we an go to based on the information that we have. For eample, if the pattern P = and suppose we onsider the DFA: We will assume that the strings are from some alphabet Σ. In some ases we will also assume that Σ is a onstant, we ll mention when we re doing so.

2 ε [] [] [] We would reah the final state (the one with the double irle) eatly when we have seen the pattern. This would allow us to find the first ourrene of pattern P in the tet T. And in order to find all the ourrenes of P, we ould alter the DFA slightly to get this one: ε [] [] [] We ould output the urrent loation in T every time we hit the final state, and then output all ourrenes of P in the tet T. Another eample: if P = then we d build ε [] [] [] [] [] Clearly, one we have the DFA M P, it takes just O(t) time to feed the tet T to the DFA, and reord the loations in T when we reah the final state. If it takes us f(p ) time to build the DFA, the total run-time of the searh algorithm would be f(p ) + O(t). It is easy to onstrut the DFA in time O(p 3 Σ ), where reall that Σ is the alphabet. However, the algorithm of Knuth, Morris, and Pratt 2 suggests a way to ompute the DFA in only O(p Σ ) time. The DFA is of size O(p Σ ), so this is optimal. In fat, their algorithm goes even further. While it is not presented as suh, it an be viewed as building a related DFA-like automaton of size O(p) (independent of the size of alphabet size Σ) that an be used for the string mathing problem in O(t) time.. A Fast Constrution of a DFA OK, so how did we really onstrut the DFA? For a pattern P = s s s 2... s p, let P k denote the prefi s s... s k. We have one state q k for eah prefi P k. Suppose we are at state q k. Now we want to see where to go when we see the harater after this. (Let P k denote the string s s... s k.) If = s k+, then we go to the state q k orresponding to P k+ = s s... s k s k+. But suppose s k+. Where ould the net math begin? Here s a piture of the situation after a mismath at k + : 2 The historial notes in the original paper are interesting to read. Also, the dramatis personae may be familiar to you: Jim Morris is a professor of omputer siene here at Carnegie Mellon. He was Dean of the Shool of Computer Siene from Both Don Knuth and Vaughan Pratt are professors of omputer siene at Stanford. Don Knuth s books The Art of Computer Programming have been super-influential for the field, and he also developed the TEX typesetting system whih has been used to reate this doument. He won the Turing Award in 974. Vaughan Pratt was one of the authors of the deterministi median finding algorithm (along with Manuel Blum, Bob Floyd, Ron Rivest and Bob Tarjan); he also gave the proof you saw that Primes is in NP. 2

3 P: k T: If a math were to start someplae in the region urrently mathed to P then: that math would start with a prefi of P (sine all mathes start that way), and that prefi would have to math in T up to k. Suh a string is marked as the shaded region in the figure above. We want the longest suh region so that we don t skip over a math in T. This means we need to find the prefi of P that orresponds to the longest suffi of P k. That is, we need to find what is the fewest haraters that we an drop from the beginning of P k to get something that looks like a prefi of P again. To see some eamples of this, let us ompute an array memo[] whih reords: memo[k]= the length of the longest proper suffi of P k whih is also a prefi of P. Note we require a proper suffi, sine we already know that the full suffi of P k doesn t math. Let s do an eample. Suppose s is the string. Here is memo: i memo[i] Let s see how we got these numbers. Consider memo[]: s..s = (s is the first har of string) The math is of length so memo[]=. All the suffies we onsider start at inde sine we require proper suffies. s..s2 = The math is of length 2 so memo[2]=2. s..s3 = (shifted to show no math) No math so memo[3]= s..s4 = 3

4 One math, so memo[4]=. s..s5 = Again a math so memo[5]=2. s..s6 = Again a math so memo[6]=3. s..s7 = The length of the math is 3 so memo[7]=3. memo[8] = 4. memo[9] = 5. Got it? Good. s..s8 = s..s9 =.2 Using the memo array for faster mathing The main idea is that memo gives us a ompat representation of the DFA, eept slightly simplified to have only 2 edge types: math and mismath. Math edges just go from q i to q i+, while memo gives the mismath edges. Here s the pseudoode: har[] T = "whatever tet to searh is"; // Assume strings are -indeed int j = ; // position in pattern for (int i=; i <= T.length; i++) { while (j > && T[i]!= P[j+]) j = memo[j]; // mismath edge if (T[i] == P[j+]) j++; // math edge if (j == n) { // end state System.out.println("math found at "+(i-n+)); j = memo[j]; } } Some notes: 4

5 When j = and we ontinue to mismath, nothing in the body of the for loop will happen, and we will just be sanning T by inrementing i. Why do we eeute j = memo[j] after we find a math? Beause you an think of falling off the end of P as a mismath between \ and T. Here s a piture to help understand the while loop: P: memo[j] a j this is the same as the above situation, but with the new j, so we set j = memo[j] again. P: T: a j i a = P: shift P over to align the strings to eah other; inrement j (and i) j = T: i T: i Why is the algorithm orret? It s simulating a simplified DFA for P. Eah time there is a mismath, P is shifted by the least amount for whih a math ould ontinue at i. So no mathes will be missed. How fast is it? The running time of the algorithm is linear. Eah iteration of the inner while loop dereases j (all the mismath DFA edges point bakward). Eah iteration of the outer loop inreases i and inreases j by at most. Consider the quantity q = 2i j. This inreases for every bit of work the algorithm does: e.g. in the while loop, i is onstant while j dereases = q inreases for eah bit of work the while loop does. in the for loop if both i and j are inremented, q inreases by. (This is why we use 2i instead of i in q.) If i is inremented, and j stays the same or dereases, then q inreases by at least 2. Finally, q is bounded between (j an t get ahead of i) and 2m (m i, j ). j will never get ahead of i sine j is only inremented when i is..3 Computing memo Last step: How do we ompute memo quikly? Beause one it is omputed, the atual mathing takes time O( T ), if we an ompute memo in time O( P ), we will ahieve a total runtime of O( T + P ), whih is linear in the input size. In fat, we an do this. Here s the ode: har[] P = "test"; // the pattern // Assume strings are indeed starting from 5

6 int n = P.length; int[] memo = new int[n+]; /* memo[i] will store the length of the longest prefi of P that mathes the tail of P2...Pi */ int j=; for (int i=2; i <= n; i++) { while (j > && P[i]!= P[j+]) j = memo[j]; // (*) if (P[i] == P[j+]) j++; memo[i] = j; } This should look very familiar! It is essentially the math ode we saw above but with T = P. Why should this make sense? When we are mathing in T, we re always looking to etend the prefi of P that mathes a tail of T. Using P in plae of T does the same thing we just have to reord how long a prefi we mathed (whereas in the math ode, we only ared about full-length mathes). We again use the fat that j will never be ahead of i, so we will have already omputed all the memo[] that we need in the while loop for i. The running time of omputing memo is O(n) by the same argument as for the math. 2 Boyer-Moore Another algorithm for string mathing that is often very good in pratie is Boyer-Moore. We ll go over it at a high level so we an see some of its unique features. The most ommon version of it runs in O(mn) time so it is not as good theoretially as KMP. But it often does muh better than the worst ase running time, and etensions eist to make it run in O(m + n) time. Idea #: We will move the pattern P along tet T from the left-to-right as before, but at eah loation where we put the pattern, we will ompare P and T right-to-left. Idea #2: Boyer-Moore will have two different rules for shifting the pattern by more than harater at a time. These heuristis will guarantee that no math will be missed, but they may not always apply. First Shift Heuristi: Bad Charater Rule. Define: R i () = position of the rightmost ourrene of harater before position i in the pattern P. So if P = abaaab, then R 3 (b) = 2 and R 3 (a) =. Bad Charater Rule: When a mismath ours at pattern position i against k: 6

7 shift by i R i (T [k]) so that the net ourrene of T [k] in the pattern is mathed to position k in T. (This is alled the bad harater rule beause it fires on a mismath, but really it shifts so that the net good harater mathes.) Seond Shift Heuristi: Good Shift Rule. some suffi α of P : When a mismath ours, you will have mathed Apply these ases in order: To handle these ases, we an pre-ompute various features of the pattern, like we did with memo[] in KMP. Disussing how to do that would take longer than we have time for, unfortunately. However, Gusfield s book has a omplete desription, as does CLRS. Complete algorithm: Putting these together, we have the Boyer-Moore algorithm: k = while k < T - P + : Compare P to T[k..k+ P -] from right to left. Stop at mismath or omplete math. if omplete math: report math k += good shift rule else: k += ma { bad harater rule, good shift rule, } 7