CSC152 Algorithm and Complexity. Lecture 7: String Match

Transcription

1 CSC152 Algorithm and Complexity Lecture 7: String Match

2 Outline Brute Force Algorithm Knuth-Morris-Pratt Algorithm Rabin-Karp Algorithm Boyer-Moore algorithm

3 String Matching Aims to Detecting the occurrence of a particular substring (called a pattern) in another string (called the text) The problem is usually presented in the context of character strings and arises often in text processing, and we will assume this context in the lecture. Algorithms Brute Force Solution The Knuth-Morris-Pratt Algorithm Rabin-Karp Algorithm The Boyer-Moore Algorithm

4 Brute Force solution Algorithm: Brute Force Algorithm Input: P and T, the pattern and text strings; m is the length of P. The pattern is assumed to be nonempty. Output: The return value is the index in T where a copy of P begins, or -1 if no match for P is found.

5 Brute Force Pseudo-Code do if (current text letter == current pattern letter) compare next letter of pattern to next letter of text else move pattern down text by one letter while (entire pattern found or end of text)

6

7 Analysis Worst-case complexity is in Θ(mn) Case 1: if the pattern appears at the beginning of the text, m comparisons are done Case 2: if P is not in T at all, n comparisons are done Waste case: for each possible starting place for P in T all but the last character of P matched the corresponding text characters. For example: P: aaaaab (m-1 a s and one b) T: aaaaaaaaa (n a s) Works quite well on average for natural language.

8 The Knuth-Morris-Pratt Algorithm Pattern Matching with Finite Automata (FA), Let be set of characters, from which the characters in P and T may be chosen. e.g. the following is the FA for P = AABC Try: T= CBBAABBAABC A match was found

9 The Knuth-Morris-Pratt Flowchart Character labels are inside the nodes Each node has two arrows out to other nodes: success link, or fail link Next character is read only after a success link A special node, node 0, called get next char which read in next text character. e.g. P = ABABCB Try: T= ACABAABABA

10 Construction of the KMP Flowchart Definition:Fail links Two arrays: one containing the characters of the pattern, one containing the failure links. The success links are implicit in the ordering of the array entries. We define fail[k] will be the index of the node pointed to by the failure link at the kth node in P, for 1 k m. The special node that merely forces the next text character to be read is considered to be the zero-th node; fail[1] = 0. We define fail[k] as the largest r (with r<k) such that p 1,.. p r-1 in P matches p k-r+1...p k-1 in T.That is the (r-1) character prefix of P is identical to the one (r-1) character substring ending at index k-1. Thus the fail links are determined by repetition within P itself. For example, k=7 p p 1,.. 4 P: ABABABCB Suppose x is not C P: ABABABCB T: ABABABx T: ABABABx p p 3,.. 6 The pattern is moved forward so that the longest initial segment that matches part of the text preceding x is lined up with that part of the text. Now x should be tested to see if it is an A to match the third A of the pattern. Thus the failure link for the node containing the C should point to the node containing the third A. r=5

11 Construction of the KMP Flowchart Thus the failure link for the node containing the C should point to the node containing the third A

12 Algorithm: KMP flowchart construction Input: P,a string of characters;m,the length of P. Output: fail, the array of failure links, defined for indexes 1,...,m. The array is passed in and the algorithm fills it. Step: void kmpsetup(char[] P, int m, int[] fail) int k,s 1. fail[1] = 0; 2. for (k = 2;k <= m;k++) 3. s = fail[k-1]; //s = 0 when k = 2 4. while (s >= 1) 5. if (p s == p k-1 ) 6. break; 7. s = fail[s]; 8. fail[k] = s+1; P: A B A B A B C B p 1 p 2 p 3 p 4 p 5 p 6 p 7 p 8

13 Analysis of KMP Flowchart Cibstructuib The body of for loop is ececuted m 1 times, and each time, the body of the while loop is executed at most m times. Since the character comparison in line 5 is executed in each pass through the while loop, the running time of the algorithm is bounded by a multiple of the number of character comparisons. A successful comparison (ps == pk-1) breaks out of the while loop so at most m-1 successful comparisons are done. After every unsuccessful comparison s is decreased, so we can bound the number of unsuccessful comparisons by determining how many times s can decrease S is initially assigned 0, when k = 2 S in increased only by executing line 8 on one pass of the for loop, following by line 3on the subsequent pass; these two statements increase s by 1. This occurs m 2 times, (when k=2, s will not be increased) s is never negative Therefore s cannot be decreased more than m-2 times, so the number of unsuccessful comparisons is at most m 2. So total, (m-1) + (m 2 ) = 2m -3

14 The Knuth-Morris-Pratt Scan Algorithm int kmpscan(char[] P,char[] T,int m,int[] fail) //m is the length of patter P int match, j,k; //j indexes text characters; k indexes the pattern and fail array match= -1; j=1; k=1; while(endtext(t, j)==false) if(k>m) match = j-m;//match found break; if(k==0) j++; k=1; //start pattern over else if(t j ==p k ) j++; k++; else //Follow fail arrow. k=fail[k]; //continue loop. return match; Try: P: ABABABCB T: ACABAABABA endtext(t,j) returns true if j is greater than the index of the last character of T. and Return false otherwise. The jth character in T equals to the kth character in P The return value is the index in T where a copy of P begins, or 1 if no match for P is found

15 Analysis KMP Flowchart Construction require 2m 3 character comparisons in the worst case The scan algorithm requires 2n character comparisons in the worst case, n is the length of the text T. (see the next slid) Overall: Worst case complexity is θ(n+m)

16 Analysis Show that the scan algorithm requires 2n character comparisons in the worst case. At most one character comparison is done each time the body of the loop is executed, in the last of the three if statements. Each time this if statement is executed, either j is incremented (along with k) or k is decremented. So we count how many times j is incremented overall and how many times k is decremented. Since j begins at 1, is incremented by exactly 1 whenever it is incremented, and never decreases, and the loop terminates when j > n (where n is the length of T), j can increase at most n times. Since k is incremented the same number of times as j (note that in the second if statement, if k = 0, k is assigned 1), k is incremented at most n times. Since k starts at 1 and is never negative, k can decrease at most n times. Thus the last if statement, where the character comparison is done, is executed at most 2n times.

17 Boyer-Moore Algorithm Basic idea is simple. We match the pattern P against substrings in the text string T from right to left. We align the pattern with the beginning of the text string. Compare the characters starting from the rightmost character of the pattern. If fail, shift the pattern to the right, by how far?

18 Boyer-Moore Algorithm Suppose we are comparing the last character P[m-1] of the pattern with some character T[k] in the text. If P[m-1] T[k], then the pattern does not occur here Case (1): if the character T[k] does not appear in P at all, we should shift P all the way to align P[0] with T[k+1] and match P[m-1] with T[k+m] again. This saves a lot of character comparisons. Case (2): if the character T[k] appears in P, then we should shift P to align the rightmost occurrence of this character in P with T[k].

19 Examples T[k+m] T[k] Case (1) T[k] T[k] Case (2) Case (1)

20 If the last character P[m-1] of the pattern matches with T[k], then we continue scanning P from right to left and match with T. If we find a complete match, we are done. Otherwise (case (3)), whenever we fail to find a complete match at t j at T, we should always shift P just far enough to pass the t j and try again. P ababbb T abaabbaababbb ababbb ababbb Case(3) ababbb Case (2)

21 Boyer-Moore algorithm To implement, we need to find out for each character c in the alphabet, the amount of jump needed if P[m-1] aligns with the character c in the input text and they don t match. Void computejumps (char[] p, int m, int alpha, int[] charjump) char ch; int k; for (ch=0; ch < alpha; ch++) //alpha = charjump[ch] = m; //m is the length of the pattern P for (k = 1; k <= m; k++) charjump[p k ] = m-k; This takes O(m + alpha) time, where m is the length of the pattern P. Afterwards, matching P with substrings in T is very fast in practice.

22 must must must must must must must must must must must If you wish to understand others you must