Lecture T1: Pattern Matching

Transcription

1 Introduction to Theoreticl CS Lecture T: Pttern Mtchin Two fundmentl questions. Wht cn computer do? Wht cn computer do with limited resources? Generl pproch. Don t tlk out specific mchines or prolems. Consider miniml strct mchines. Consider enerl clsses of prolems. Why Lern Theory In theory... Deeper understndin of wht is computer nd computin. Foundtion of ll modern computers. Pure science. Philosophicl implictions. Simple mchine with N sttes. Finite Stte Automton In prctice... We serch: theory of pttern mtchin. Sequentil circuit: theory of finite stte utomt. Compilers: theory of context free rmmr. Cryptorphy: theory of complexity. Dt compression: theory of informtion. 3 4

2 Finite Stte Automt C Code for FSA Simple mchine with N sttes. Strt in stte. Red n input it. Move to new stte depends on input it nd current stte Stop when lst it red. yes if end in ccept stte(s) no otherwise strt stte 3 ccept stte fs3.c #include <stdio.h> #define STATES 4 #define START_STATE #define ACCEPT_STATE 3 int min(void) { int i, stte = START_STATE; int trnsition[states][] = { {, }, {3, }, {, }, {, } }; use D rry Yes lso clled ccepted or reconized inputs from lnue.! Yes:,,,,...! No: ll others. Trnsition Tle Stte 3 3 } while (scnf("%d", &i)!= EOF) stte = trnsition[stte][i]; if (stte == ACCEPT_STATE) printf("yes.\n"); else printf("no.\n"); return ; Trnsition Tle Stte A Second Exmple An Appliction: Bounce Filter Consider the followin two stte FSA. Bounce filter: remove isolted s nd s in input. Input: Output (one-it dely): w Wht it strins does it ccept? Yes:,,,! All it strins with n odd numer of s. BB G W No:,,,! All it strins with n even numer of s. GG B no ccept stte insted output color of ech stte you visit 7 8

3 An Appliction: Bounce Filter Bounce filter: remove isolted s nd s in input. Input: Output (one-it dely): w Text Serchin Build n FSA tht ccepts ll strins tht contin ct s sustrin. ttct cct Stte interprettions. W: strt BB: t lest two consecutive s. G: sequence of s followed y. GG: t lest two consecutive s. B: sequence of s followed y. ct Strt uildin: c t φ c c ct ct Stte nme represents lrest prefix of "ct" tht input currently mtches. 9 Text Serchin Build n FSA tht ccepts ll strins tht contin ct s sustrin. ttct cct Finish uildin: ct c ct We Serch Appliction We serch enines uild FSA s. Stndrd We serch for: cos 6 pttern mtchin! Finds one mtchin pe on whole Internet. Serch enines hve different methods for specifyin ptterns. Which one is most powerful? Theory of computtion helps us ddress such issues. φ c c c t ct t ct 4 5

4 Unix Pttern Mtchin Tool: erep Crossword Puzzle or Scrle Too Hrd? Generl reulr expressions pttern mtchin. Acts s filter. /usr/dict/words is list of (5,43) words in dictionry. /u/cs6/files/textfiles/wordlist.txt is list of 34,936 words. Sends lines from stdin to stdout tht "mtch" rument strin. Elementry Exmples %erep eth clsslist 3/Smythe/Elizeth/6/esmythe 3/Bethke/Kristen/3/kethke % erep /3/ clsslist 3/Mrin/Anthony/3/mrin 3/Arellno/Belen/3/rellno... 3/Weiss/Jco/3/weiss Find ll lines in file clsslist with sustrin eth List ll people in precept 3. More Exmples % erep hh /usr/dict/words echhed hihhnded Two consecutive h s. withheld withhold % erep u.u.u /usr/dict/words cumulus A dot mtches ny sinle chrcter %erep zeulodon moydick.txt rechristened the monster zeulodon nd in his %erep ct humn.dt cccccctctttt 6 % erep..oo..oo /usr/dict/words loodroot nincompoophood schoolook ut not "cookook" schoolroom 7 Erep Pttern Conventions Still More Exmples Conventions for erep: c ny non-specil chrcter mtches itself. ny sinle chrcter r* zero or more occurrence of r r+ one or more occurrence of r r? zero or one occurrence of r (r) roupin r r loicl OR [eiou] ny vowel [^ eiou] ny non-vowel ^ einnin of line $ end of line Fls for erep: erep -v mtch ll lines except those specified y pttern Unix % erep n(ie ei)ther /usr/dict/words neither % erep ct(tc)*ct humn.dt tctctc % erep ct(tc)*ct student.dt ttcttctctcctttc % erep ^y.(..)*y$ /usr/dict/words yesterdy % erep -v [eiou] /usr/dict/words erep... rhythm syzyy Do spell checkin y specifyin wht you know. Strts nd ends with y, odd numer of chrcters. Find ll words with no vowels nd 6 or more letters. 8 9

5 Fundmentl Questions: Theoreticl Minimum Fundmentl Questions: Theoreticl Minimum Specifyin "pttern" for erep cn e complex. ^[^eiou]*[^eiou]*e[^eiou]*i[^eiou]*o[^eiou]*u[^eiou]*! stemious! fcetious Which spects re essentil? Unix erep reulr expressions re useful. But more complex thn theoreticl minimum. Reulr expressions. Mtch WHOLE strin. (to convert to equivlent erep pttern, surround y ^ nd $) reulr expression: ( )* erep pttern: ^( )*$ c ny non-specil chrcter mtches itself r* zero or more occurrence of r (r) roupin r r loicl OR r r conctente (usully suppress symol). ny sinle chrcter [eiou] ny vowel [^ eiou] ny non-vowel r+ one or more occurrences of r not needed r? zero or one occurrence of r -v mtch ll ptterns except Fundmentl Questions: Wht Kinds of Ptterns Fundmentl Questions: Wht Kinds of Ptterns Wht kinds of ptterns cn e specified y reulr expressions? (ll ut one of followin) Wht kinds of ptterns cn e specified y reulr expressions? (ll ut one of followin) All it strins tht: Exmple Bein with nd end with. Equl numer of s nd s. Hve no consecutive s. Hs nd odd numer of s. Hs s sustrin. All it strins tht: Reulr Expression Bein with nd end with. ( )* Equl numer of s nd s. not possile Hve no consecutive s. ( )*( *) Hs nd odd numer of s. (***)*(**) Hs s sustrin. ( )*( )* 3

6 Forml Lnues An ALPHABET is finite set of symols. Binry lphet = {, } Lower-cse lphet = {,, c, d,..., y, z} Genetic lphet = {, c, t, } A STRING is finite sequence of symols in the lphet. is strin in the inry lphet. tiers is strin in the lower-cse lphet. cctct is strin in the enetic lphet. Forml Lnues Cn cst ny computtion s lnue reconition prolem. Is x = 3,536,48,73 prime numer?! L = {, 3, 5, 7,, 3, 7,... }! Is x in lnue L? FSA. Mchine determines whether strin is in lnue. Reulr expression. Shorthnd method for specifyin lnue. A FORMAL LANGUAGE is n (unordered) set of strins in n lphet. Cn hve infinitely mny strins. Exmples: {,,,,,,...} {,,,,,...} (***)*(**) even # of s exctly one 4 5 Dulity Between FSA s nd RE s Oservtion: for ech FSA we crete, we cn find reulr expression tht mtches the sme strins tht the FSA ccepts. Is this lwys the cse?! Yes. Wht out the OTHER wy round?! Yes. I don t see why? FSA re simple mchines. Limittions of FSA N sttes cn t "rememer" more thn N thins. Some lnues require "rememerin" more thn N thins. No FSA cn reconize the lnue of ll it strins with n equl numer of s nd s. A wrmup exercise: Sty tuned: see Lecture T. If xyz ccepted then so is xyz 6 7

7 Limittions of FSA No FSA cn reconize the lnue of ll it strins with n equl numer of s nd s. Suppose n N-stte FSA cn reconize this lnue. Consider followin input: N+ s N+ s Lookin Ahed Tody. Defined simple strct mchine = FSA. Cple of pttern mtchin. Incple of "countin." Need to consider more powerful mchines. Hmm. Which will we run out of first? FSA must ccept this strin. Some stte x is revisited durin first N+ s since only N sttes. x x Mchine would ccept sme strin without intervenin s. Future lectures. Define n strct mchine. Understnd how it works nd wht it cn do. Find thins it cn t do. Define more powerful mchine. Repet until we run out of prolems or mchines. This strin doesn t hve n equl numer of s nd s. 8 9 Lecture T: Supplementl Notes C Code for FSA fs.c #include <stdio.h> int min(void) { int c, stte = ; while ((c = etchr())!= EOF) { if (stte == && c == ) stte = ; if (stte == && c == ) stte = ; if (stte == && c == ) stte = 3; if (stte == && c == ) stte = ; if (stte == && c == ) stte = ; if (stte == && c == ) stte = ; if (stte == 3 && c == ) stte = ; if (stte == 3 && c == ) stte = ; } } if (stte == 3) printf("yes.\n"); else printf("no.\n"); return ; strihtforwrd to convert FSA s into C prorm or to uild with hrdwre 3

8 A Fourth Exmple A Fourth Exmple FSA to decide if inteer (represented in inry) is divisile y 3? FSA to decide if input (convert inry to deciml) is divisile y 3? is strt nd ccept stte Wht it strins does it ccept? Yes: (3 ), (6 ), (9 ), ( ), (5 ), (8 ), inteers divisile y 3. No: ( ), ( ), (4 ), (5 ), (7 ), inteers not divisile y 3. How does it work? Stte : input so fr is divisile y 3. Stte : input hs reminder upon division y 3. Stte : input hs reminder upon division y 3. Trnsition exmple. Input ( ) ends in stte. If next it is then sty in stte : (4 ). Addin to lst it is sme s multiplyin numer y. Remins divisile y Reulr Expressions Rules for cretin reulr expressions (RE s): or or ε symols () roupin conctention + loicl OR * closure ( or more replictions) use + insted of where nd re reulr expressions. Exmples: ε = empty strin ()* ε,,,,... ( + )*,,,,,... (****)* ε,,,,... 37