CS143 Handout 07 Summer 2011 June 24 th, 2011 Written Set 1: Lexical Analysis

Transcription

1 CS143 Hndout 07 Summer 2011 June 24 th, 2011 Written Set 1: Lexicl Anlysis In this first written ssignment, you'll get the chnce to ply round with the vrious constructions tht come up when doing lexicl nlysis. This prolem set will strt off y reviewing those constructions, then sk you to think criticlly out them. You'll e considering different lgorithms for lexicl nlysis, long with some of the limittions of wht we presented in clss. In prticulr, you'll explore the vrious lgorithms for generting utomton driven scnners nd for resolving conflicts tht might rise in them. This ssignment is due on July 6, so you hve plenty of time to work on it. However, I'd suggest strting erly while the mteril is still fresh in your mind. Feel free to emil us with questions or to drop y office hours. Overll, this ssignment is worth 10% of your totl grde in the course. Ech prolem is weighted eqully. Best of luck! Prolem One: Converting Regulr Expressions Due: July 6 th t 5 p.m. Using the regulr expression to NFA lgorithm we covered in clss, convert the following regulr expressions into NFAs. In wht follows, recll tht * hs highest precedence, then conctention, then. For exmple, c* is interpreted s () ((c*)). Similrly, * is () (*). i. *c ii. ( c) iii. ( ε)(c*)

2 2 Prolem Two: Suset Construction Use the suset construction to convert the following NFAs into DFAs. You cn ssume tht the lphet is just the two letters nd., strt i. strt ii. strt iii. ε, ε

3 3 Prolem Three: Mximl Munch Given the following tokens nd their ssocited regulr expressions, show wht output is produced when this flex scnner is run over the following strings: * %% * printf("1"); ( )* printf("2"); c* printf("3"); i. cc ii. cc iii. cc Prolem Four: The Limits of Conflict Resolution In clss we discussed one prticulr mens for resolving conflicts tht occur when using regulr expressions to scn n input. Our pproch ws to use mximl munch to lwys choose the longest possile mtch t ny point, then to rek ties sed on the priorities of the regulr expressions. However, this is not the only wy tht we could hve resolved conflicts. Recll from lecture our exmple with two types of regulr expressions: n expression for the keyword for nd n expression for identifiers: %% "for" { return T_For; } [A-Z-z_][A-Z-z0-9_]* { return T_Identifier; } We sw how the string fort could e tokenized in nine possile wys, sed on how we chose to pply the regulr expression. In this cse, mximl munch lgorithm dicttes tht we would scn the string s the identifier fort. However, in some cses we my hve set of regulr expressions for which it is possile to tokenize prticulr input string, ut for which the mximl munch lgorithm will not e le to rek the input into tokens. Give n exmple of such set of regulr expressions nd n input string, indicting oth how the input string could indeed y prtitioned ppropritely nd why mximl munch fils to find this prtition. * If you wnt to, you cn ctully use flex to compile nd run this scnner to check your nswer. However, plese e sure tht you understnd why the output is s it is.

4 Prolem Five: Right to Left Scnning The scnning lgorithm we covered in lecture is sed on left to right scn of the input. We convert the series of regulr expressions descriing the lexicl nlysis into mtching utomton, then feed the chrcters from the input file one t time into the utomton from left to right. However, we just s esily could hve considered doing right to left scn of the input, in which we feed the the lst chrcter into the utomton, then the second to lst chrcter, etc. Notice tht this mens tht the mtching utomton we crete must mtch given pttern in reverse, since it will e seeing the chrcters in reverse order. 4 i. Modify the existing lgorithm for converting regulr expressions to NFAs so tht the generted NFA ccepts the reverse of strings tht mtch the regulr expression. ii. Give n exmple of set of regulr expressions nd string so tht the left toright scn of the string produces different set of tokens thn the right to left scn. Assume tht you're using the mximl munch lgorithm for conflict resolution. Prolem Six: Converting Extended Regulr Expressions In Wednesdy's clss, we discussed recursive lgorithm for converting regulr expressions into NFAs. If you'll recll, we ssocited with ech regulr expression n NFA with three properties: 1. The generted NFA hs exctly one terminl stte. 2. The generted NFA hs no trnsitions into its strt stte. 3. The generted NFA hs no trnsitions out of its terminl stte. Additionlly, we encountered three shorthnd nottions tht extended our stndrd regulr expression grmmr: 1. R?, which mtches zero or one copies of R, 2. R+, which mtches one or more copies of R, nd 3. R{n}, which mtches exctly n copies of R. Show how to extend the existing RE to NFA construction to support these new regulr expressions. In prticulr, you should show the utomt you would construct to mtch ech of these expressions.

5 5 Prolem Seven: Slowing Down flex Scnners As mentioned in clss, in order to implement mximl munch scnning, we run mtching utomton over the input, keeping trck of the lst loction of mtch. When the utomton gets stuck, the scnner reports the lst known munch, the restrts the scnner just fter the reported token. Becuse the scnner my hve to ck up over mny chrcters tht it hs lredy scnned, it is possile for the scnner generted from set of regulr expressions to rered prts of the input string mny times, cusing the scnner to run more slowly. As n exmple, consider the following flex script: %% "photo" { printf("photo"); } "photosynthesis" { printf("photosynthesis"); }. \n { printf("x"); } If we run this script on the input photosynthesize, the scnner will note tht mtch exists with the prefix photo. It will then continue reding up until it encounters the z, t which point it will note tht none of the ove three regulr expressions could mtch. The scnner will then mtch photo, outputting the text PHOTO, nd will restrt the scnner t the s just fter photo in the input. It will then output n x for ech of the remining letters, so the overll output will e PHOTOxxxxxxxxxx. In this exmple, the scnner hd to ck up over hlf the chrcters in the input. Consequently, the scnner ended up revisiting ll of the chrcters in the second hlf of the input twice. This is noticely slower thn single pss over the input. However, it's possile to come up with set of regulr expressions so tht on certin inputs, the scnner ends up doing sustntilly worse thn this. In fct, there re regulr expressions tht on certin strings of length n end up spending Θ(n 2 ) time rereding the chrcters in the string. * Your jo is to pick 1. A set of regulr expressions, nd 2. A function f tht ccepts n integer n nd produces string of length n such tht mximl munch scnning lgorithm for those regulr expressions, when run on string f(n), spends Θ(n 2 ) time scnning. Agin, you hve full control of wht regulr expressions re used nd wht strings they're run on. The point of this question is to let you think out the pthologicl cses of mximl munch scnning y seeing how the runtime scles qudrticlly in some cses. If it will mke it esier, you cn hve your function only work for n lrger thn n integer of your choice. * Θ(n 2 ) is similr to O(n 2 ), except tht it implies tight ound insted of n upper ound. If function is O(n 2 ), it could lso e O(n). However, if function is Θ(n 2 ), it grows symptoticlly t the sme rte s n 2.

6 Prolem Eight: Exponentil Suset Constructions In clss, we mentioned how the suset construction for converting NFAs to DFAs cn result in DFA tht is exponentilly lrger thn the originl NFA, since in the worst cse the construction will produce DFA with one stte for ech set of sttes in the originl NFA. In this prolem, you'll see one such pthologicl exmple. i. Come up with n NFA tht ccepts ll strings over the lnguge {,, c} tht end in symol tht hs not yet ppered in the string. For exmple, c is in the lnguge, since only ppers t the end, s is c nd c. However, c is not in the lnguge ecuse the string ends in, which ppers twice in the string. Also, the empty string is not in the lnguge, since it does not end in symol t ll. Your NFA should hve t most five sttes. ii. Use the suset construction to minimize this NFA to DFA nd show your result. How mny sttes does this DFA hve? (This prolem is sed on Prolem from Christos Ppdimitriou's textook Elements of the Theory of Computtion). Collortion on prolem sets Although you cn discuss ides with others, you must sumit your own independent solution nd properly give credit to nyone you worked with. Sumission instructions There re two wys to sumit this ssignment: 1. Sumit physicl copy of your nswers in the filing cinet in the open spce ner the hndout hngout in the Gtes uilding. If you hven't een there efore, it's right inside the entrnce leled Stnford Venture Fund Lortories. 2. Send n emil with n electronic copy of your nswers to the stff list t cs143 sum1011 stff@lists.stnford.edu. 6