Lexical Analysis. Amitabha Sanyal. ( as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay
|
|
- Moses Day
- 6 years ago
- Views:
Transcription
1 Lexicl Anlysis Amith Snyl ( s) Deprtment of Computer Science nd Engineering, Indin Institute of Technology, Bomy Septemer 27
2 College of Engineering, Pune Lexicl Anlysis: 2/6 Recp The input progrm s you see it. min () { int i,sum; sum = ; for (i=; i<=; i++); sum = sum + i; printf("%d\n",sum); } Amith Snyl IIT Bomy
3 College of Engineering, Pune Lexicl Anlysis: 4/6 Recp The sme progrm s the compiler sees it (initilly). min for () { (i=; i<=; int i++); printf("%d\n",sum); } i,sum; sum = sum sum = + ; i; The lnk spce chrcter The return chrcter Amith Snyl IIT Bomy
4 College of Engineering, Pune Lexicl Anlysis: 4/6 Recp The sme progrm s the compiler sees it (initilly). min for () { (i=; i<=; int i++); i,sum; sum = sum sum = + printf("%d\n",sum); } How do you mke the compiler see wht you see? ; i; Amith Snyl IIT Bomy
5 College of Engineering, Pune Lexicl Anlysis: 6/6 Recp - Discovering the structure of the progrm Step :. Brek up this string into words the smllest logicl units. min ; for ( ) { ( i = ; sum = sum = i <= ; sum + int ; printf ( "%d\n", sum ) ; } We get sequence of lexemes or tokens. i; i, sum i ++ ) ; Amith Snyl IIT Bomy
6 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step :. Clen up remove the nd the chrcters. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } Steps. nd. re interleved. Amith Snyl IIT Bomy
7 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step :. Clen up remove the nd the chrcters. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } This is lexicl nlysis or scnning. Amith Snyl IIT Bomy
8 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step 2: Now group the lexemes to form lrger structures. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } Amith Snyl IIT Bomy
9 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step 2: Now group the lexemes to form lrger structures. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } fundef Amith Snyl IIT Bomy
10 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step 2: Now group the lexemes to form lrger structures. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } fundef fnme prms compound-stmt Amith Snyl IIT Bomy
11 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step 2: Now group the lexemes to form lrger structures. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } fundef fnme prms compound-stmt identifier Amith Snyl IIT Bomy
12 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step 2: Now group the lexemes to form lrger structures. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } fundef fnme prms compound-stmt identifier min Amith Snyl IIT Bomy
13 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step 2: Now group the lexemes to form lrger structures. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } fundef fnme prms compound-stmt identifier ( ) min Amith Snyl IIT Bomy
14 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step 2: Now group the lexemes to form lrger structures. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } fundef fnme prms compound-stmt identifier ( ) { vdecl slist } min Amith Snyl IIT Bomy
15 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step 2: Now group the lexemes to form lrger structures. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } fundef fnme prms compound-stmt identifier min ( ) { vdecl slist } type vrlist ; Amith Snyl IIT Bomy
16 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step 2: Now group the lexemes to form lrger structures. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } fundef fnme prms compound-stmt identifier min ( ) { vdecl slist } type vrlist ; int Amith Snyl IIT Bomy
17 College of Engineering, Pune Lexicl Anlysis: /6 Recp - Discovering the structure of the progrm Step 2: Now group the lexemes to form lrger structures. min ( ) { int i, sum ; sum = ; for ( i = ; i <= ; i ++ ) ; sum = sum + i ; printf ( "%d\n", sum ) ; } fundef fnme prms compound-stmt identifier ( ) { vdecl slist } min type vrlist ;... int... Amith Snyl IIT Bomy
18 College of Engineering, Pune Lexicl Anlysis: 2/6 Recp - Discovering the structure of the progrm fundef fnme prms compound-stmt identifier ( ) { vdecl slist } min type vrlist ;... int vrlist, vr vr identifier identifier sum i This is syntx nlysis or prsing. Amith Snyl IIT Bomy
19 College of Engineering, Pune Lexicl Anlysis: 4/6 Lexemes, Tokens nd Ptterns Definition: Lexicl nlysis is the opertion of dividing the input progrm into sequence of lexemes (tokens). Amith Snyl IIT Bomy
20 College of Engineering, Pune Lexicl Anlysis: 4/6 Lexemes, Tokens nd Ptterns Definition: Lexicl nlysis is the opertion of dividing the input progrm into sequence of lexemes (tokens). Distinguish etween Amith Snyl IIT Bomy
21 College of Engineering, Pune Lexicl Anlysis: 4/6 Lexemes, Tokens nd Ptterns Definition: Lexicl nlysis is the opertion of dividing the input progrm into sequence of lexemes (tokens). Distinguish etween lexemes smllest logicl units (words) of progrm. Exmples i, sum, for,, ++, "%d\n", <=. Amith Snyl IIT Bomy
22 College of Engineering, Pune Lexicl Anlysis: 4/6 Lexemes, Tokens nd Ptterns Definition: Lexicl nlysis is the opertion of dividing the input progrm into sequence of lexemes (tokens). Distinguish etween lexemes smllest logicl units (words) of progrm. Exmples i, sum, for,, ++, "%d\n", <=. tokens sets of similr lexemes. Exmples identifier = {i, sum, uffer,... } int constnt = {,,... } ddop = {+, -} Amith Snyl IIT Bomy
23 College of Engineering, Pune Lexicl Anlysis: 6/6 Lexemes, Tokens nd Ptterns Things tht re not counted s lexemes Amith Snyl IIT Bomy
24 College of Engineering, Pune Lexicl Anlysis: 6/6 Lexemes, Tokens nd Ptterns Things tht re not counted s lexemes white spces t, lnks nd newlines Amith Snyl IIT Bomy
25 College of Engineering, Pune Lexicl Anlysis: 6/6 Lexemes, Tokens nd Ptterns Things tht re not counted s lexemes white spces t, lnks nd newlines comments Amith Snyl IIT Bomy
26 College of Engineering, Pune Lexicl Anlysis: 6/6 Lexemes, Tokens nd Ptterns Things tht re not counted s lexemes white spces t, lnks nd newlines comments These too hve to e detected nd ignored. Amith Snyl IIT Bomy
27 College of Engineering, Pune Lexicl Anlysis: /6 Lexemes, Tokens nd Ptterns The lexicl nlyser: detects the next lexeme ctegorises it into the right token psses to the syntx nlyser the token nme for further syntx nlysis the lexeme itself, in some form, for stges eyond syntx nlysis Amith Snyl IIT Bomy
28 College of Engineering, Pune Lexicl Anlysis: 2/6 Recp - Lexemes, Tokens nd Ptterns How does one descrie the lexemes tht mke up the token identifier. Vrints in different lnguges. Amith Snyl IIT Bomy
29 College of Engineering, Pune Lexicl Anlysis: 2/6 Recp - Lexemes, Tokens nd Ptterns How does one descrie the lexemes tht mke up the token identifier. Vrints in different lnguges. String of lphnumeric chrcters. The first chrcter is n lphet. Amith Snyl IIT Bomy
30 College of Engineering, Pune Lexicl Anlysis: 2/6 Recp - Lexemes, Tokens nd Ptterns How does one descrie the lexemes tht mke up the token identifier. Vrints in different lnguges. String of lphnumeric chrcters. The first chrcter is n lphet. string of lphnumeric chrcters in which the the first chrcter is n lphet. It hs length of t most 3. Amith Snyl IIT Bomy
31 College of Engineering, Pune Lexicl Anlysis: 2/6 Recp - Lexemes, Tokens nd Ptterns How does one descrie the lexemes tht mke up the token identifier. Vrints in different lnguges. String of lphnumeric chrcters. The first chrcter is n lphet. string of lphnumeric chrcters in which the the first chrcter is n lphet. It hs length of t most 3. string of lphet or numeric or underline chrcters in which the the first chrcter is n lphet or n underline. It hs length of t most 3. Any chrcters fter the 3st chrcter re ignored. Amith Snyl IIT Bomy
32 College of Engineering, Pune Lexicl Anlysis: 2/6 Recp - Lexemes, Tokens nd Ptterns How does one descrie the lexemes tht mke up the token identifier. Vrints in different lnguges. String of lphnumeric chrcters. The first chrcter is n lphet. string of lphnumeric chrcters in which the the first chrcter is n lphet. It hs length of t most 3. string of lphet or numeric or underline chrcters in which the the first chrcter is n lphet or n underline. It hs length of t most 3. Any chrcters fter the 3st chrcter re ignored. Such descriptions re clled ptterns. The description my e informl or forml. Amith Snyl IIT Bomy
33 College of Engineering, Pune Lexicl Anlysis: 22/6 Recp - Bsic concepts nd issues A pttern is used to specify tokens precisely uild recognizer from such specifictions Amith Snyl IIT Bomy
34 College of Engineering, Pune Lexicl Anlysis: 24/6 Exmple tokens in Jv. Identifier: A Jvletter followed y zero or more Jvletterordigits. A Jvletter includes the chrcters -z, A-Z, _ nd $. 2. Constnts: 2. Integer Constnts Octl, Hex nd Deciml 4 yte nd yte representtion 2.2 Floting point constnts flot - ends with f doule 2.3 Boolen constnts true nd flse 2.4 Chrcter constnts, \u34, \t 2.5 String constnts "", "\"", "A string". 2.6 Null constnt null. 3. Delimiters: (, ), {, },, ], ;,. nd, 4. Opertors: =, >, <... >>>= 5. Keywords: strct, oolen... voltile, while. Amith Snyl IIT Bomy
35 College of Engineering, Pune Lexicl Anlysis: 26/6 Recp - Bsic concepts nd issues Where does lexicl nlyser fit into the rest of the compiler? The front end of most compilers is prser driven. When the prser needs the next token, it invokes the Lexicl Anlyser. Insted of nlysing the entire input string, the lexicl nlyser sees enough of the input string to return single token. The ctions of the lexicl nlyser nd prser re intertwined. prser input progrm lexicl nlyser rest of the compiler Amith Snyl IIT Bomy
36 College of Engineering, Pune Lexicl Anlysis: 2/6 Recp - Token Attriutes Aprt from the token itself, the lexicl nlyser lso psses other informtions regrding the token. These items of informtion re clled token ttriutes EXAMPLE lexeme <token, token ttriute> 3 < const, 3> A <identifier, A> if <if, > = <ssignop, > > <gt, > ; <semicolon, > Amith Snyl IIT Bomy
37 College of Engineering, Pune Lexicl Anlysis: 3/6 Creting Lexicl Anlyzer Two pproches: Amith Snyl IIT Bomy
38 College of Engineering, Pune Lexicl Anlysis: 3/6 Creting Lexicl Anlyzer Two pproches:. Hnd code This is only of historicl interest now. Possily more efficient. Amith Snyl IIT Bomy
39 College of Engineering, Pune Lexicl Anlysis: 3/6 Creting Lexicl Anlyzer Two pproches:. Hnd code This is only of historicl interest now. Possily more efficient. 2. Use genertor To generte the lexicl nlyser from forml description. The genertion process is fster. Less prone to errors. Amith Snyl IIT Bomy
40 College of Engineering, Pune Lexicl Anlysis: 32/6 Automtic Genertion of Lexicl Anlysers Inputs to the lexicl nlyser genertor: A specifiction of the tokens of the source lnguge, consisting of: regulr expression descriing ech token, nd code frgment descriing the ction to e performed, on identifying ech token. The generted lexicl nlyser consists of: A deterministic finite utomton (DFA) constructed from the token specifiction. A code frgment ( driver routine) which cn trverse ny DFA. Code for the ction specifictions. Amith Snyl IIT Bomy
41 College of Engineering, Pune Lexicl Anlysis: 34/6 Automtic Genertion of Lexicl Anlysers specifiction regulr expression ction routines processed copied input progrm DFA ction routines generted lexicl nlyser Driver routine tokens Amith Snyl IIT Bomy
42 College of Engineering, Pune Lexicl Anlysis: 36/6 Exmple of Lexicl Anlyser Genertion Suppose lnguge hs two tokens Pttern Action * { printf( "Token found");} c+ { printf( "Token 2 found");} From the description, construct structure clled deterministic finite utomton (DFA). c 2 3 c Amith Snyl IIT Bomy
43 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: IIT Bomy
44 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: IIT Bomy
45 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: IIT Bomy
46 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: IIT Bomy
47 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: Token found IIT Bomy
48 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: Token found IIT Bomy
49 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: Token found IIT Bomy
50 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: Token found IIT Bomy
51 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: Token found IIT Bomy
52 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: Token found Token found IIT Bomy
53 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: Token found Token found IIT Bomy
54 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: Token found Token found IIT Bomy
55 College of Engineering, Pune Lexicl Anlysis: 3/6 Exmple of Lexicl Anlyser Genertion Now consider the following together: The DFA c The ctions void ction(); { switch(stte) 2:{printf("Token found"); rek;} 3:{printf("Token 2 found"); rek;} } Amith Snyl 3 c 2 The driver routine void nexttoken () {stte = ; c = nextchr(); while (vlid(nextsttestte,c])) {stte = nextsttestte,c]; c = nextchr();} if (!finl(stte)) {error; return;} else {unput(c);ction();return;}} The input nd output Input: dcc Output: Token found Token found Token 2 found IIT Bomy
56 College of Engineering, Pune Lexicl Anlysis: 4/6 Exmple of Lexicl Anlyser Genertion In summry: The DFA, the driver routine nd the ction routines tken together, constitute the lexicl nlyser. ctions re supplied s prt of specifiction. driver routine is common to ll generted lexicl nlyzers The only issue how re the ptterns, specified y regulr expressions, converted to DFA. In two steps: Convert regulr expression into NFA. Convert NFA to DFA. Amith Snyl IIT Bomy
57 College of Engineering, Pune Lexicl Anlysis: 42/6 Exmple of Lexicl Anlyser Genertion Consider lnguge with the following tokens: egin representing the lexeme egin integer Exmples:, -5, 25 identifier Exmples:, A, mx Amith Snyl IIT Bomy
58 College of Engineering, Pune Lexicl Anlysis: 44/6 Converting Regulr Expressions to NFA In two prts; First convert the regulr expression corresponding to ech token into NFA. Invrint: A single finl stte corresponding to ech token. Join the NFAs otined for ll the tokens. Amith Snyl IIT Bomy
59 College of Engineering, Pune Lexicl Anlysis: 46/6 Converting Regulr Expressions to DFA RE for ɛ RE for ε ε Amith Snyl IIT Bomy
60 College of Engineering, Pune Lexicl Anlysis: 4/6 Converting Regulr Expressions to NFA r r2 RE for r r 2 r r r 2 r 2 Amith Snyl IIT Bomy
61 College of Engineering, Pune Lexicl Anlysis: 5/6 Converting Regulr Expressions to NFA r RE for r r 2 r r 2 r r2 ε r 2 ε ε ε Amith Snyl IIT Bomy
62 College of Engineering, Pune Lexicl Anlysis: 52/6 Converting Regulr Expressions to NFA r RE for r ε r r* ε ε ε Amith Snyl IIT Bomy
63 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA Amith Snyl IIT Bomy
64 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA 2 e g 3 4 i 5 n 6 Amith Snyl IIT Bomy
65 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA 2 e g 3 4 i 5 n digit 9 digit digit Amith Snyl IIT Bomy
66 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA 2 e g 3 4 i 5 n digit 9 digit letter letter digit digit Amith Snyl IIT Bomy
67 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit Amith Snyl IIT Bomy
68 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit,,7, Amith Snyl IIT Bomy
69 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit,,7, 2, Amith Snyl IIT Bomy
70 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit,,7, - 2, 6 Amith Snyl IIT Bomy
71 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit,,7, 2, digit Amith Snyl IIT Bomy
72 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit,,7, 2, digit Amith Snyl -{} IIT Bomy
73 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit Amith Snyl,,7, -{} 2, digit e -{e} 3, 2 IIT Bomy
74 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit Amith Snyl,,7, -{} 2, digit 7 digit e -{e} 3, 2 IIT Bomy
75 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit Amith Snyl,,7, -{} 2, digit 7 digit e -{e} 3, 2 letter digit IIT Bomy
76 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit Amith Snyl,,7, -{} 2, digit 7 digit e -{e} 3, 2 letter digit 4, g 3 -{g} IIT Bomy
77 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit Amith Snyl,,7, -{} 2, digit 7 digit e -{e} 3, 2 letter digit 4, g 3 -{g} digit IIT Bomy
78 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit Amith Snyl,,7, -{} 2, digit 7 digit e -{e} 3, 2 letter digit 4, g 3 -{g} -{i} digit i 5, 4 IIT Bomy
79 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit,,7, 2, 6 e -{e} 3, digit 7 4, g 3 -{g} -{i} digit i 5, 4 -{n} n 6, 5 -{} digit letter digit Amith Snyl IIT Bomy
80 College of Engineering, Pune Lexicl Anlysis: 54/6 Converting NFA to DFA ɛ 2 e g 3 4 i 5 n 6 ɛ 7 - digit 9 digit ɛ digit letter letter digit,,7, 2, 6 e -{e} 3, digit 7 4, g 3 -{g} -{i} digit i 5, 4 -{n} n 6, 5 letter digit -{} digit letter digit Amith Snyl IIT Bomy
81 College of Engineering, Pune Lexicl Anlysis: 56/6 LEXICAL ERRORS Primrily of two kinds:. Lexemes whose length exceed the ound specified y the lnguge. In Fortrn, n identifier more thn 7 chrcters long is lexicl error. Most lnguges hve ound on the precision of numeric constnts. A constnt whose length exceeds this ound is lexicl error. 2. Illegl chrcters in the progrm. The chrcters ~, & occuring in Pscl progrm (ut not within string or comment) re lexicl errors. 3. Unterminted strings or comments. Amith Snyl IIT Bomy
82 College of Engineering, Pune Lexicl Anlysis: 5/6 Hndling Lexicl Errors issuing n error messge, the ction tken on detection of n error re:. Issue n pproprite error messge. 2. Error of the first type the entire lexeme is red nd then truncted to the specified length. Error of the second type skip illegl chrcter. pss the chrcter to the prser which hs etter knowledge of the context in which error hs occurred. more possiilities of recovery - replcement insted of deletion. Error of the third type wit till end of file n issue error messge. Amith Snyl IIT Bomy
83 College of Engineering, Pune Lexicl Anlysis: 6/6 MINIMIZING THE NUMBER OF STATES The DFA constructed for ( ɛ)( ) Amith Snyl IIT Bomy
84 College of Engineering, Pune Lexicl Anlysis: 6/6 MINIMIZING THE NUMBER OF STATES The DFA constructed for ( ɛ)( ). There is nother DFA for the sme regulr expression with lesser numer of sttes A B C D Amith Snyl IIT Bomy
85 College of Engineering, Pune Lexicl Anlysis: 6/6 MINIMIZING THE NUMBER OF STATES The DFA constructed for ( ɛ)( ). There is nother DFA for the sme regulr expression with lesser numer of sttes A B C D For typicl lnguge, the numer of sttes of the DFA is in order of hundreds. Amith Snyl IIT Bomy
86 College of Engineering, Pune Lexicl Anlysis: 6/6 MINIMIZING THE NUMBER OF STATES The DFA constructed for ( ɛ)( ). There is nother DFA for the sme regulr expression with lesser numer of sttes A B C D For typicl lnguge, the numer of sttes of the DFA is in order of hundreds. Therefore we should try to minimize the numer of sttes. Amith Snyl IIT Bomy
87 College of Engineering, Pune Lexicl Anlysis: 62/6 MINIMIZING THE NUMBER OF STATES The second DFA hs een otined y merging sttes nd 3 of the first DFA A B C D Amith Snyl IIT Bomy
88 College of Engineering, Pune Lexicl Anlysis: 62/6 MINIMIZING THE NUMBER OF STATES The second DFA hs een otined y merging sttes nd 3 of the first DFA. Under wht conditions cn this merging tke plce? A B C D Amith Snyl IIT Bomy
89 College of Engineering, Pune Lexicl Anlysis: 64/6 MINIMIZING THE NUMBER OF STATES A B C D The string tkes oth sttes nd 3 to finl stte. Amith Snyl IIT Bomy
90 College of Engineering, Pune Lexicl Anlysis: 64/6 MINIMIZING THE NUMBER OF STATES A B C D The string tkes oth sttes nd 3 to finl stte. The string tkes oth sttes nd 3 to non-finl stte. Amith Snyl IIT Bomy
91 College of Engineering, Pune Lexicl Anlysis: 64/6 MINIMIZING THE NUMBER OF STATES A B C D The string tkes oth sttes nd 3 to finl stte. The string tkes oth sttes nd 3 to non-finl stte. The string ɛ tkes oth sttes nd 3 to non-finl stte. Amith Snyl IIT Bomy
92 College of Engineering, Pune Lexicl Anlysis: 64/6 MINIMIZING THE NUMBER OF STATES A B C D The string tkes oth sttes nd 3 to finl stte. The string tkes oth sttes nd 3 to non-finl stte. The string ɛ tkes oth sttes nd 3 to non-finl stte. The string tkes oth sttes nd 3 to finl stte. Amith Snyl IIT Bomy
93 College of Engineering, Pune Lexicl Anlysis: 64/6 MINIMIZING THE NUMBER OF STATES A B C D The string tkes oth sttes nd 3 to finl stte. The string tkes oth sttes nd 3 to non-finl stte. The string ɛ tkes oth sttes nd 3 to non-finl stte. The string tkes oth sttes nd 3 to finl stte. Conclusion: Any string tht tkes stte to finl stte lso tkes 3 to finl stte. Conversely, ny string tht tkes stte to non-finl stte lso tkes 3 to non-finl stte. Amith Snyl IIT Bomy
94 College of Engineering, Pune Lexicl Anlysis: 66/6 MINIMIZING THE NUMBER OF STATES Sttes nd 3 re sid to e indistinguishle. Minimimiztion strtegy: Find indistinguishle sttes. Merge them. A B C D Question: How does one find indistingushle sttes? Amith Snyl IIT Bomy
95 College of Engineering, Pune Lexicl Anlysis: 6/6 MINIMIZING THE NUMBER OF STATES Key ide: Amith Snyl IIT Bomy
96 College of Engineering, Pune Lexicl Anlysis: 6/6 MINIMIZING THE NUMBER OF STATES Key ide: Initilly ssume ll sttes to e indistinguishle. Put them in single set. Amith Snyl IIT Bomy
97 College of Engineering, Pune Lexicl Anlysis: 6/6 MINIMIZING THE NUMBER OF STATES Key ide: The string ɛ distinguishes etween finl sttes nd non-finl sttes. Crete two prtitions. Amith Snyl IIT Bomy
98 College of Engineering, Pune Lexicl Anlysis: 6/6 MINIMIZING THE NUMBER OF STATES Key ide: tkes 4 to red prtition nd retins other lue sttes in lue prtition. Put 4 in seprte prtition. Amith Snyl IIT Bomy
99 College of Engineering, Pune Lexicl Anlysis: 6/6 MINIMIZING THE NUMBER OF STATES Key ide: The string distinguishes 3 from other sttes in the lue prtition. Amith Snyl IIT Bomy
100 College of Engineering, Pune Lexicl Anlysis: 6/6 MINIMIZING THE NUMBER OF STATES Key ide: A B C D No other prtition possile. Merge ll sttes in the sme prtition. Amith Snyl IIT Bomy
101 College of Engineering, Pune Lexicl Anlysis: 7/6 Summry of the Method. Construct n initil prtition π = {S F,F,..., F n,}, where F = F F 2... F n s, nd ech F i is the set of finl sttes for some token i. 2. for ech set G in π do prtition G into susets such tht two sttes s nd t of G re in the sme suset if nd only if for ll input symols, sttes s nd t hve trnsitions onto sttes in the sme set of π; replce G in π new y the set of ll susets formed 3. If π new = π, let π finl := π nd continue with step 4. Otherwise repet step 2 with π := π new. 4. Merge sttes in the sme set of the prtition. 5. Remove ny ded sttes. Amith Snyl IIT Bomy
102 College of Engineering, Pune Lexicl Anlysis: 72/6 EFFICIENT REPRESENTATION OF DFA A nive method to represent DFA uses two dimensionl rry. c c c x 3 3 For typicl lnguge: the numer of DFA sttes is in the order of hundreds (sometimes ), the numer of input symols is greter thn. It is desirle to find spce-efficient representtion of the DFA. Amith Snyl IIT Bomy
103 College of Engineering, Pune Lexicl Anlysis: 74/6 The Four Arrys Scheme Key Oservtion For DFA tht we hve seen erlier, the sttes mrked with # ehve like stte on ll symols except for one symol. # - -{} # - digit 6 7 digit e -{e} letter digit # # # g 2 3 i 4 -{g} -{i} digit -{n} n 5 letter digit Therefore informtion out stte cn lso e used for these sttes. Amith Snyl IIT Bomy
104 College of Engineering, Pune Lexicl Anlysis: 76/6 Four Arrys Representtion of DFA Symols nd their numering z DEFAULT BASE NEXT CHECK Amith Snyl IIT Bomy
105 College of Engineering, Pune Lexicl Anlysis: 7/6 Four Arrys Representtion of DFA Symols nd their numering z DEFAULT BASE NEXT CHECK Amith Snyl IIT Bomy
106 College of Engineering, Pune Lexicl Anlysis: /6 Four Arrys Representtion of DFA Symols nd their numering z DEFAULT BASE NEXT CHECK Amith Snyl IIT Bomy
107 College of Engineering, Pune Lexicl Anlysis: 2/6 Four Arrys Representtion of DFA Symols nd their numering z DEFAULT BASE NEXT CHECK Amith Snyl IIT Bomy
108 College of Engineering, Pune Lexicl Anlysis: 4/6 Four Arrys Representtion of DFA If s is stte nd is the numeric representtion of symol, then. BASEs] gives the se loction for the informtion stored out stte s. 2. NEXT BASEs]+] gives the next stte for s nd symol, only if CHECKBASEs]+] = s. 3. If CHECKBASEs]+] s, then the next stte informtion is ssocited with DEFAULT s]. function nextstte(s,); egin if CHECKBASEs] + ] = s then NEXT BASEs]+] else return(nextstte(default s],)) end Amith Snyl IIT Bomy
109 College of Engineering, Pune Lexicl Anlysis: 6/6 Four Arrys Representtion of DFA All the entries for stte hve een stored in the rry NEXT. The CHECK rry shows tht the entries re vlid for stte. Stte hs trnsition on e(4), which is different from the corresponding trnsition on stte. This differing entry is stored in NEXT 37]. Therefore BASE] is set to 37 4 = 33. By similr resoning BASE] is set to 36. To find nextstte, ], we first refer to NEXT 33 + ], But since CHECK33 + ] is not we hve to refer to DEFAULT ] which is. So the correct next stte is found from NEXT BASE] + ] =. To fill up the four rrys, we hve to use heuristic method. One possiility, which works well in prctice, is to find for given stte, the lowest BASE, so tht the specil entries cn e filled without conflicting with existing entries. Amith Snyl IIT Bomy
Fig.25: the Role of LEX
The Lnguge for Specifying Lexicl Anlyzer We shll now study how to uild lexicl nlyzer from specifiction of tokens in the form of list of regulr expressions The discussion centers round the design of n existing
More informationReducing a DFA to a Minimal DFA
Lexicl Anlysis - Prt 4 Reducing DFA to Miniml DFA Input: DFA IN Assume DFA IN never gets stuck (dd ded stte if necessry) Output: DFA MIN An equivlent DFA with the minimum numer of sttes. Hrry H. Porter,
More informationCS321 Languages and Compiler Design I. Winter 2012 Lecture 5
CS321 Lnguges nd Compiler Design I Winter 2012 Lecture 5 1 FINITE AUTOMATA A non-deterministic finite utomton (NFA) consists of: An input lphet Σ, e.g. Σ =,. A set of sttes S, e.g. S = {1, 3, 5, 7, 11,
More informationCS 432 Fall Mike Lam, Professor a (bc)* Regular Expressions and Finite Automata
CS 432 Fll 2017 Mike Lm, Professor (c)* Regulr Expressions nd Finite Automt Compiltion Current focus "Bck end" Source code Tokens Syntx tree Mchine code chr dt[20]; int min() { flot x = 42.0; return 7;
More informationIn the last lecture, we discussed how valid tokens may be specified by regular expressions.
LECTURE 5 Scnning SYNTAX ANALYSIS We know from our previous lectures tht the process of verifying the syntx of the progrm is performed in two stges: Scnning: Identifying nd verifying tokens in progrm.
More informationDr. D.M. Akbar Hussain
Dr. D.M. Akr Hussin Lexicl Anlysis. Bsic Ide: Red the source code nd generte tokens, it is similr wht humns will do to red in; just tking on the input nd reking it down in pieces. Ech token is sequence
More informationCS412/413. Introduction to Compilers Tim Teitelbaum. Lecture 4: Lexical Analyzers 28 Jan 08
CS412/413 Introduction to Compilers Tim Teitelum Lecture 4: Lexicl Anlyzers 28 Jn 08 Outline DFA stte minimiztion Lexicl nlyzers Automting lexicl nlysis Jlex lexicl nlyzer genertor CS 412/413 Spring 2008
More informationΕΠΛ323 - Θεωρία και Πρακτική Μεταγλωττιστών
ΕΠΛ323 - Θωρία και Πρακτική Μταγλωττιστών Lecture 3 Lexicl Anlysis Elis Athnsopoulos elisthn@cs.ucy.c.cy Recognition of Tokens if expressions nd reltionl opertors if è if then è then else è else relop
More informationCS 430 Spring Mike Lam, Professor. Parsing
CS 430 Spring 2015 Mike Lm, Professor Prsing Syntx Anlysis We cn now formlly descrie lnguge's syntx Using regulr expressions nd BNF grmmrs How does tht help us? Syntx Anlysis We cn now formlly descrie
More informationLexical Analysis: Constructing a Scanner from Regular Expressions
Lexicl Anlysis: Constructing Scnner from Regulr Expressions Gol Show how to construct FA to recognize ny RE This Lecture Convert RE to n nondeterministic finite utomton (NFA) Use Thompson s construction
More informationTopic 2: Lexing and Flexing
Topic 2: Lexing nd Flexing COS 320 Compiling Techniques Princeton University Spring 2016 Lennrt Beringer 1 2 The Compiler Lexicl Anlysis Gol: rek strem of ASCII chrcters (source/input) into sequence of
More informationDefinition of Regular Expression
Definition of Regulr Expression After the definition of the string nd lnguges, we re redy to descrie regulr expressions, the nottion we shll use to define the clss of lnguges known s regulr sets. Recll
More informationLanguages. L((a (b)(c))*) = { ε,a,bc,aa,abc,bca,... } εw = wε = w. εabba = abbaε = abba. (a (b)(c)) *
Pln for Tody nd Beginning Next week Interpreter nd Compiler Structure, or Softwre Architecture Overview of Progrmming Assignments The MeggyJv compiler we will e uilding. Regulr Expressions Finite Stte
More informationCSc 453. Compilers and Systems Software. 4 : Lexical Analysis II. Department of Computer Science University of Arizona
CSc 453 Compilers nd Systems Softwre 4 : Lexicl Anlysis II Deprtment of Computer Science University of Arizon collerg@gmil.com Copyright c 2009 Christin Collerg Implementing Automt NFAs nd DFAs cn e hrd-coded
More informationLexical analysis, scanners. Construction of a scanner
Lexicl nlysis scnners (NB. Pges 4-5 re for those who need to refresh their knowledge of DFAs nd NFAs. These re not presented during the lectures) Construction of scnner Tools: stte utomt nd trnsition digrms.
More informationLexical Analysis and Lexical Analyzer Generators
1 Lexicl Anlysis nd Lexicl Anlyzer Genertors Chpter 3 COP5621 Compiler Construction Copyright Roert vn Engelen, Florid Stte University, 2007-2009 2 The Reson Why Lexicl Anlysis is Seprte Phse Simplifies
More informationΕΠΛ323 - Θεωρία και Πρακτική Μεταγλωττιστών. Lecture 3b Lexical Analysis Elias Athanasopoulos
ΕΠΛ323 - Θωρία και Πρακτική Μταγλωττιστών Lecture 3 Lexicl Anlysis Elis Athnsopoulos elisthn@cs.ucy.c.cy RecogniNon of Tokens if expressions nd relnonl opertors if è if then è then else è else relop è
More informationCompilation
Compiltion 0368-3133 Lecture 2: Lexicl Anlysis Nom Rinetzky 1 2 Lexicl Anlysis Modern Compiler Design: Chpter 2.1 3 Conceptul Structure of Compiler Compiler Source text txt Frontend Semntic Representtion
More informationCS143 Handout 07 Summer 2011 June 24 th, 2011 Written Set 1: Lexical Analysis
CS143 Hndout 07 Summer 2011 June 24 th, 2011 Written Set 1: Lexicl Anlysis In this first written ssignment, you'll get the chnce to ply round with the vrious constructions tht come up when doing lexicl
More informationCompiler Construction D7011E
Compiler Construction D7011E Lecture 3: Lexer genertors Viktor Leijon Slides lrgely y John Nordlnder with mteril generously provided y Mrk P. Jones. 1 Recp: Hndwritten Lexers: Don t require sophisticted
More informationCSCI 3130: Formal Languages and Automata Theory Lecture 12 The Chinese University of Hong Kong, Fall 2011
CSCI 3130: Forml Lnguges nd utomt Theory Lecture 12 The Chinese University of Hong Kong, Fll 2011 ndrej Bogdnov In progrmming lnguges, uilding prse trees is significnt tsk ecuse prse trees tell us the
More informationCMPSC 470: Compiler Construction
CMPSC 47: Compiler Construction Plese complete the following: Midterm (Type A) Nme Instruction: Mke sure you hve ll pges including this cover nd lnk pge t the end. Answer ech question in the spce provided.
More informationImplementing Automata. CSc 453. Compilers and Systems Software. 4 : Lexical Analysis II. Department of Computer Science University of Arizona
Implementing utomt Sc 5 ompilers nd Systems Softwre : Lexicl nlysis II Deprtment of omputer Science University of rizon collerg@gmil.com opyright c 009 hristin ollerg NFs nd DFs cn e hrd-coded using this
More informationPrinciples of Programming Languages
Principles of Progrmming Lnguges h"p://www.di.unipi.it/~ndre/did2c/plp- 14/ Prof. Andre Corrdini Deprtment of Computer Science, Pis Lesson 5! Gener;on of Lexicl Anlyzers Creting Lexicl Anlyzer with Lex
More informationCS 340, Fall 2014 Dec 11 th /13 th Final Exam Note: in all questions, the special symbol ɛ (epsilon) is used to indicate the empty string.
CS 340, Fll 2014 Dec 11 th /13 th Finl Exm Nme: Note: in ll questions, the specil symol ɛ (epsilon) is used to indicte the empty string. Question 1. [5 points] Consider the following regulr expression;
More informationAssignment 4. Due 09/18/17
Assignment 4. ue 09/18/17 1. ). Write regulr expressions tht define the strings recognized by the following finite utomt: b d b b b c c b) Write FA tht recognizes the tokens defined by the following regulr
More informationCS 241 Week 4 Tutorial Solutions
CS 4 Week 4 Tutoril Solutions Writing n Assemler, Prt & Regulr Lnguges Prt Winter 8 Assemling instrutions utomtilly. slt $d, $s, $t. Solution: $d, $s, nd $t ll fit in -it signed integers sine they re 5-it
More informationScanner Termination. Multi Character Lookahead. to its physical end. Most parsers require an end of file token. Lex and Jlex automatically create an
Scnner Termintion A scnner reds input chrcters nd prtitions them into tokens. Wht hppens when the end of the input file is reched? It my be useful to crete n Eof pseudo-chrcter when this occurs. In Jv,
More informationExample: Source Code. Lexical Analysis. The Lexical Structure. Tokens. What do we really care here? A Sample Toy Program:
Lexicl Anlysis Red source progrm nd produce list of tokens ( liner nlysis) source progrm The lexicl structure is specified using regulr expressions Other secondry tsks: (1) get rid of white spces (e.g.,
More informationCSE 401 Midterm Exam 11/5/10 Sample Solution
Question 1. egulr expressions (20 points) In the Ad Progrmming lnguge n integer constnt contins one or more digits, but it my lso contin embedded underscores. Any underscores must be preceded nd followed
More informationCMPT 379 Compilers. Lexical Analysis
CMPT 379 Compilers Anoop Srkr http://www.cs.sfu.c/~noop 9//7 Lexicl Anlysis Also clled scnning, tke input progrm string nd convert into tokens Exmple: T_DOUBLE ( doule ) T_IDENT ( f ) T_OP ( = ) doule
More informationthis grammar generates the following language: Because this symbol will also be used in a later step, it receives the
LR() nlysis Drwcks of LR(). Look-hed symols s eplined efore, concerning LR(), it is possile to consult the net set to determine, in the reduction sttes, for which symols it would e possile to perform reductions.
More informationCOMP 423 lecture 11 Jan. 28, 2008
COMP 423 lecture 11 Jn. 28, 2008 Up to now, we hve looked t how some symols in n lphet occur more frequently thn others nd how we cn sve its y using code such tht the codewords for more frequently occuring
More informationCS 340, Fall 2016 Sep 29th Exam 1 Note: in all questions, the special symbol ɛ (epsilon) is used to indicate the empty string.
CS 340, Fll 2016 Sep 29th Exm 1 Nme: Note: in ll questions, the speil symol ɛ (epsilon) is used to indite the empty string. Question 1. [10 points] Speify regulr expression tht genertes the lnguge over
More informationCompilers Spring 2013 PRACTICE Midterm Exam
Compilers Spring 2013 PRACTICE Midterm Exm This is full length prctice midterm exm. If you wnt to tke it t exm pce, give yourself 7 minutes to tke the entire test. Just like the rel exm, ech question hs
More informationCSCE 531, Spring 2017, Midterm Exam Answer Key
CCE 531, pring 2017, Midterm Exm Answer Key 1. (15 points) Using the method descried in the ook or in clss, convert the following regulr expression into n equivlent (nondeterministic) finite utomton: (
More informationFall Compiler Principles Lecture 1: Lexical Analysis. Roman Manevich Ben-Gurion University
Fll 2014-2015 Compiler Principles Lecture 1: Lexicl Anlysis Romn Mnevich Ben-Gurion University Agend Understnd role of lexicl nlysis in compiler Lexicl nlysis theory Implementing professionl scnner vi
More information10/12/17. Motivating Example. Lexical and Syntax Analysis (2) Recursive-Descent Parsing. Recursive-Descent Parsing. Recursive-Descent Parsing
Motivting Exmple Lexicl nd yntx Anlysis (2) In Text: Chpter 4 Consider the grmmr -> cad A -> b Input string: w = cd How to build prse tree top-down? 2 Initilly crete tree contining single node (the strt
More informationDeterministic. Finite Automata. And Regular Languages. Fall 2018 Costas Busch - RPI 1
Deterministic Finite Automt And Regulr Lnguges Fll 2018 Costs Busch - RPI 1 Deterministic Finite Automton (DFA) Input Tpe String Finite Automton Output Accept or Reject Fll 2018 Costs Busch - RPI 2 Trnsition
More informationShould be done. Do Soon. Structure of a Typical Compiler. Plan for Today. Lab hours and Office hours. Quiz 1 is due tonight, was posted Tuesday night
Should e done L hours nd Office hours Sign up for the miling list t, strting to send importnt info to list http://groups.google.com/group/cs453-spring-2011 Red Ch 1 nd skim Ch 2 through 2.6, red 3.3 nd
More informationDiscussion 1 Recap. COP4600 Discussion 2 OS concepts, System call, and Assignment 1. Questions. Questions. Outline. Outline 10/24/2010
COP4600 Discussion 2 OS concepts, System cll, nd Assignment 1 TA: Hufeng Jin hj0@cise.ufl.edu Discussion 1 Recp Introduction to C C Bsic Types (chr, int, long, flot, doule, ) C Preprocessors (#include,
More informationFall Compiler Principles Lecture 1: Lexical Analysis. Roman Manevich Ben-Gurion University of the Negev
Fll 2016-2017 Compiler Principles Lecture 1: Lexicl Anlysis Romn Mnevich Ben-Gurion University of the Negev Agend Understnd role of lexicl nlysis in compiler Regulr lnguges reminder Lexicl nlysis lgorithms
More informationScanner Termination. Multi Character Lookahead
If d.doublevlue() represents vlid integer, (int) d.doublevlue() will crete the pproprite integer vlue. If string representtion of n integer begins with ~ we cn strip the ~, convert to double nd then negte
More informationLR Parsing, Part 2. Constructing Parse Tables. Need to Automatically Construct LR Parse Tables: Action and GOTO Table
TDDD55 Compilers nd Interpreters TDDB44 Compiler Construction LR Prsing, Prt 2 Constructing Prse Tles Prse tle construction Grmmr conflict hndling Ctegories of LR Grmmrs nd Prsers Peter Fritzson, Christoph
More informationFinite Automata. Lecture 4 Sections Robb T. Koether. Hampden-Sydney College. Wed, Jan 21, 2015
Finite Automt Lecture 4 Sections 3.6-3.7 Ro T. Koether Hmpden-Sydney College Wed, Jn 21, 2015 Ro T. Koether (Hmpden-Sydney College) Finite Automt Wed, Jn 21, 2015 1 / 23 1 Nondeterministic Finite Automt
More informationSome Thoughts on Grad School. Undergraduate Compilers Review and Intro to MJC. Structure of a Typical Compiler. Lexing and Parsing
Undergrdute Compilers Review nd Intro to MJC Announcements Miling list is in full swing Tody Some thoughts on grd school Finish prsing Semntic nlysis Visitor pttern for bstrct syntx trees Some Thoughts
More informationSlides for Data Mining by I. H. Witten and E. Frank
Slides for Dt Mining y I. H. Witten nd E. Frnk Simplicity first Simple lgorithms often work very well! There re mny kinds of simple structure, eg: One ttriute does ll the work All ttriutes contriute eqully
More informationECE 468/573 Midterm 1 September 28, 2012
ECE 468/573 Midterm 1 September 28, 2012 Nme:! Purdue emil:! Plese sign the following: I ffirm tht the nswers given on this test re mine nd mine lone. I did not receive help from ny person or mteril (other
More informationCS 241. Fall 2017 Midterm Review Solutions. October 24, Bits and Bytes 1. 3 MIPS Assembler 6. 4 Regular Languages 7.
CS 241 Fll 2017 Midterm Review Solutions Octoer 24, 2017 Contents 1 Bits nd Bytes 1 2 MIPS Assemly Lnguge Progrmming 2 3 MIPS Assemler 6 4 Regulr Lnguges 7 5 Scnning 9 1 Bits nd Bytes 1. Give two s complement
More informationFrom Dependencies to Evaluation Strategies
From Dependencies to Evlution Strtegies Possile strtegies: 1 let the user define the evlution order 2 utomtic strtegy sed on the dependencies: use locl dependencies to determine which ttriutes to compute
More informationCMSC 331 First Midterm Exam
0 00/ 1 20/ 2 05/ 3 15/ 4 15/ 5 15/ 6 20/ 7 30/ 8 30/ 150/ 331 First Midterm Exm 7 October 2003 CMC 331 First Midterm Exm Nme: mple Answers tudent ID#: You will hve seventy-five (75) minutes to complete
More informationControl-Flow Analysis and Loop Detection
! Control-Flow Anlysis nd Loop Detection!Lst time! PRE!Tody! Control-flow nlysis! Loops! Identifying loops using domintors! Reducibility! Using loop identifiction to identify induction vribles CS553 Lecture
More informationLexical Analysis. Role, Specification & Recognition Tool: LEX Construction: - RE to NFA to DFA to min-state DFA - RE to DFA
Lexicl Anlysis Role, Specifiction & Recognition Tool: LEX Construction: - RE to NFA to DFA to min-stte DFA - RE to DFA Conducting Lexicl Anlysis Techniques for specifying nd implementing lexicl nlyzers
More informationOperator Precedence. Java CUP. E E + T T T * P P P id id id. Does a+b*c mean (a+b)*c or
Opertor Precedence Most progrmming lnguges hve opertor precedence rules tht stte the order in which opertors re pplied (in the sence of explicit prentheses). Thus in C nd Jv nd CSX, +*c mens compute *c,
More informationAgenda & Reading. Class Exercise. COMPSCI 105 SS 2012 Principles of Computer Science. Arrays
COMPSCI 5 SS Principles of Computer Science Arrys & Multidimensionl Arrys Agend & Reding Agend Arrys Creting & Using Primitive & Reference Types Assignments & Equlity Pss y Vlue & Pss y Reference Copying
More informationA Tautology Checker loosely related to Stålmarck s Algorithm by Martin Richards
A Tutology Checker loosely relted to Stålmrck s Algorithm y Mrtin Richrds mr@cl.cm.c.uk http://www.cl.cm.c.uk/users/mr/ University Computer Lortory New Museum Site Pemroke Street Cmridge, CB2 3QG Mrtin
More informationWhat are suffix trees?
Suffix Trees 1 Wht re suffix trees? Allow lgorithm designers to store very lrge mount of informtion out strings while still keeping within liner spce Allow users to serch for new strings in the originl
More informationToday. CS 188: Artificial Intelligence Fall Recap: Search. Example: Pancake Problem. Example: Pancake Problem. General Tree Search.
CS 88: Artificil Intelligence Fll 00 Lecture : A* Serch 9//00 A* Serch rph Serch Tody Heuristic Design Dn Klein UC Berkeley Multiple slides from Sturt Russell or Andrew Moore Recp: Serch Exmple: Pncke
More information12 <= rm <digit> 2 <= rm <no> 2 <= rm <no> <digit> <= rm <no> <= rm <number>
DDD16 Compilers nd Interpreters DDB44 Compiler Construction R Prsing Prt 1 R prsing concept Using prser genertor Prse ree Genertion Wht is R-prsing? eft-to-right scnning R Rigthmost derivtion in reverse
More informationCS 321 Programming Languages and Compilers. Bottom Up Parsing
CS 321 Progrmming nguges nd Compilers Bottom Up Prsing Bottom-up Prsing: Shift-reduce prsing Grmmr H: fi ; fi b Input: ;;b hs prse tree ; ; b 2 Dt for Shift-reduce Prser Input string: sequence of tokens
More informationPointers and Arrays. More Pointer Examples. Pointers CS 217
Pointers nd Arrs CS 21 1 2 Pointers More Pointer Emples Wht is pointer A vrile whose vlue is the ddress of nother vrile p is pointer to vrile v Opertions &: ddress of (reference) *: indirection (dereference)
More informationTheory of Computation CSE 105
$ $ $ Theory of Computtion CSE 105 Regulr Lnguges Study Guide nd Homework I Homework I: Solutions to the following problems should be turned in clss on July 1, 1999. Instructions: Write your nswers clerly
More informationApplied Databases. Sebastian Maneth. Lecture 13 Online Pattern Matching on Strings. University of Edinburgh - February 29th, 2016
Applied Dtses Lecture 13 Online Pttern Mtching on Strings Sestin Mneth University of Edinurgh - Ferury 29th, 2016 2 Outline 1. Nive Method 2. Automton Method 3. Knuth-Morris-Prtt Algorithm 4. Boyer-Moore
More informationLecture 10 Evolutionary Computation: Evolution strategies and genetic programming
Lecture 10 Evolutionry Computtion: Evolution strtegies nd genetic progrmming Evolution strtegies Genetic progrmming Summry Negnevitsky, Person Eduction, 2011 1 Evolution Strtegies Another pproch to simulting
More informationMidterm I Solutions CS164, Spring 2006
Midterm I Solutions CS164, Spring 2006 Februry 23, 2006 Plese red ll instructions (including these) crefully. Write your nme, login, SID, nd circle the section time. There re 8 pges in this exm nd 4 questions,
More informationSystems I. Logic Design I. Topics Digital logic Logic gates Simple combinational logic circuits
Systems I Logic Design I Topics Digitl logic Logic gtes Simple comintionl logic circuits Simple C sttement.. C = + ; Wht pieces of hrdwre do you think you might need? Storge - for vlues,, C Computtion
More informationASTs, Regex, Parsing, and Pretty Printing
ASTs, Regex, Prsing, nd Pretty Printing CS 2112 Fll 2016 1 Algeric Expressions To strt, consider integer rithmetic. Suppose we hve the following 1. The lphet we will use is the digits {0, 1, 2, 3, 4, 5,
More informationSymbol Table management
TDDD Compilers nd interpreters TDDB44 Compiler Construction Symol Tles Symol Tles in the Compiler Symol Tle mngement source progrm Leicl nlysis Syntctic nlysis Semntic nlysis nd Intermedite code gen Code
More informationQuiz2 45mins. Personal Number: Problem 1. (20pts) Here is an Table of Perl Regular Ex
Long Quiz2 45mins Nme: Personl Numer: Prolem. (20pts) Here is n Tle of Perl Regulr Ex Chrcter Description. single chrcter \s whitespce chrcter (spce, t, newline) \S non-whitespce chrcter \d digit (0-9)
More informationCS481: Bioinformatics Algorithms
CS481: Bioinformtics Algorithms Cn Alkn EA509 clkn@cs.ilkent.edu.tr http://www.cs.ilkent.edu.tr/~clkn/teching/cs481/ EXACT STRING MATCHING Fingerprint ide Assume: We cn compute fingerprint f(p) of P in
More informationRecognition of Tokens
42 Recognton o Tokens The queston s how to recognze the tokens? Exmple: ssume the ollowng grmmr rgment to generte specc lnguge: stmt expr expr then stmt expr then stmt else stmt term relop term term term
More informationJava CUP. Java CUP Specifications. User Code Additions. Package and Import Specifications
Jv CUP Jv CUP is prser-genertion tool, similr to Ycc. CUP uilds Jv prser for LALR(1) grmmrs from production rules nd ssocited Jv code frgments. When prticulr production is recognized, its ssocited code
More informationTO REGULAR EXPRESSIONS
Suject :- Computer Science Course Nme :- Theory Of Computtion DA TO REGULAR EXPRESSIONS Report Sumitted y:- Ajy Singh Meen 07000505 jysmeen@cse.iit.c.in BASIC DEINITIONS DA:- A finite stte mchine where
More informationContext-Free Grammars
Context-Free Grmmrs Descriing Lnguges We've seen two models for the regulr lnguges: Finite utomt ccept precisely the strings in the lnguge. Regulr expressions descrie precisely the strings in the lnguge.
More informationPresentation Martin Randers
Presenttion Mrtin Rnders Outline Introduction Algorithms Implementtion nd experiments Memory consumption Summry Introduction Introduction Evolution of species cn e modelled in trees Trees consist of nodes
More informationStack. A list whose end points are pointed by top and bottom
4. Stck Stck A list whose end points re pointed by top nd bottom Insertion nd deletion tke plce t the top (cf: Wht is the difference between Stck nd Arry?) Bottom is constnt, but top grows nd shrinks!
More informationbox Boxes and Arrows 3 true 7.59 'X' An object is drawn as a box that contains its data members, for example:
Boxes nd Arrows There re two kinds of vriles in Jv: those tht store primitive vlues nd those tht store references. Primitive vlues re vlues of type long, int, short, chr, yte, oolen, doule, nd flot. References
More informationRegular Expressions and Automata using Miranda
Regulr Expressions nd Automt using Mirnd Simon Thompson Computing Lortory Univerisity of Kent t Cnterury My 1995 Contents 1 Introduction ::::::::::::::::::::::::::::::::: 1 2 Regulr Expressions :::::::::::::::::::::::::::::
More informationAlgorithm Design (5) Text Search
Algorithm Design (5) Text Serch Tkshi Chikym School of Engineering The University of Tokyo Text Serch Find sustring tht mtches the given key string in text dt of lrge mount Key string: chr x[m] Text Dt:
More informationSolving Problems by Searching. CS 486/686: Introduction to Artificial Intelligence Winter 2016
Solving Prolems y Serching CS 486/686: Introduction to Artificil Intelligence Winter 2016 1 Introduction Serch ws one of the first topics studied in AI - Newell nd Simon (1961) Generl Prolem Solver Centrl
More information2 Computing all Intersections of a Set of Segments Line Segment Intersection
15-451/651: Design & Anlysis of Algorithms Novemer 14, 2016 Lecture #21 Sweep-Line nd Segment Intersection lst chnged: Novemer 8, 2017 1 Preliminries The sweep-line prdigm is very powerful lgorithmic design
More informationInformation Retrieval and Organisation
Informtion Retrievl nd Orgnistion Suffix Trees dpted from http://www.mth.tu.c.il/~himk/seminr02/suffixtrees.ppt Dell Zhng Birkeck, University of London Trie A tree representing set of strings { } eef d
More informationVirtual Machine (Part I)
Hrvrd University CS Fll 2, Shimon Schocken Virtul Mchine (Prt I) Elements of Computing Systems Virtul Mchine I (Ch. 7) Motivtion clss clss Min Min sttic sttic x; x; function function void void min() min()
More informationAnnouncements. CS 188: Artificial Intelligence Fall Recap: Search. Today. Example: Pancake Problem. Example: Pancake Problem
Announcements Project : erch It s live! Due 9/. trt erly nd sk questions. It s longer thn most! Need prtner? Come up fter clss or try Pizz ections: cn go to ny, ut hve priority in your own C 88: Artificil
More informationTries. Yufei Tao KAIST. April 9, Y. Tao, April 9, 2013 Tries
Tries Yufei To KAIST April 9, 2013 Y. To, April 9, 2013 Tries In this lecture, we will discuss the following exct mtching prolem on strings. Prolem Let S e set of strings, ech of which hs unique integer
More informationAgilent Mass Hunter Software
Agilent Mss Hunter Softwre Quick Strt Guide Use this guide to get strted with the Mss Hunter softwre. Wht is Mss Hunter Softwre? Mss Hunter is n integrl prt of Agilent TOF softwre (version A.02.00). Mss
More informationSuffix trees, suffix arrays, BWT
ALGORITHMES POUR LA BIO-INFORMATIQUE ET LA VISUALISATION COURS 3 Rluc Uricru Suffix trees, suffix rrys, BWT Bsed on: Suffix trees nd suffix rrys presenttion y Him Kpln Suffix trees course y Pco Gomez Liner-Time
More informationSample Midterm Solutions COMS W4115 Programming Languages and Translators Monday, October 12, 2009
Deprtment of Computer cience Columbi University mple Midterm olutions COM W4115 Progrmming Lnguges nd Trnsltors Mondy, October 12, 2009 Closed book, no ids. ch question is worth 20 points. Question 5(c)
More informationThe dictionary model allows several consecutive symbols, called phrases
A dptive Huffmn nd rithmetic methods re universl in the sense tht the encoder cn dpt to the sttistics of the source. But, dpttion is computtionlly expensive, prticulrly when k-th order Mrkov pproximtion
More informationMid-term exam. Scores. Fall term 2012 KAIST EE209 Programming Structures for EE. Thursday Oct 25, Student's name: Student ID:
Fll term 2012 KAIST EE209 Progrmming Structures for EE Mid-term exm Thursdy Oct 25, 2012 Student's nme: Student ID: The exm is closed book nd notes. Red the questions crefully nd focus your nswers on wht
More informationCIS 1068 Program Design and Abstraction Spring2015 Midterm Exam 1. Name SOLUTION
CIS 1068 Progrm Design nd Astrction Spring2015 Midterm Exm 1 Nme SOLUTION Pge Points Score 2 15 3 8 4 18 5 10 6 7 7 7 8 14 9 11 10 10 Totl 100 1 P ge 1. Progrm Trces (41 points, 50 minutes) Answer the
More information6.3 Volumes. Just as area is always positive, so is volume and our attitudes towards finding it.
6.3 Volumes Just s re is lwys positive, so is volume nd our ttitudes towrds finding it. Let s review how to find the volume of regulr geometric prism, tht is, 3-dimensionl oject with two regulr fces seprted
More informationcisc1110 fall 2010 lecture VI.2 call by value function parameters another call by value example:
cisc1110 fll 2010 lecture VI.2 cll y vlue function prmeters more on functions more on cll y vlue nd cll y reference pssing strings to functions returning strings from functions vrile scope glol vriles
More informationIf you are at the university, either physically or via the VPN, you can download the chapters of this book as PDFs.
Lecture 5 Wlks, Trils, Pths nd Connectedness Reding: Some of the mteril in this lecture comes from Section 1.2 of Dieter Jungnickel (2008), Grphs, Networks nd Algorithms, 3rd edition, which is ville online
More informationRegular Expression Matching with Multi-Strings and Intervals. Philip Bille Mikkel Thorup
Regulr Expression Mtching with Multi-Strings nd Intervls Philip Bille Mikkel Thorup Outline Definition Applictions Previous work Two new problems: Multi-strings nd chrcter clss intervls Algorithms Thompson
More informationPPS: User Manual. Krishnendu Chatterjee, Martin Chmelik, Raghav Gupta, and Ayush Kanodia
PPS: User Mnul Krishnendu Chtterjee, Mrtin Chmelik, Rghv Gupt, nd Ayush Knodi IST Austri (Institute of Science nd Technology Austri), Klosterneuurg, Austri In this section we descrie the tool fetures,
More informationCS201 Discussion 10 DRAWTREE + TRIES
CS201 Discussion 10 DRAWTREE + TRIES DrwTree First instinct: recursion As very generic structure, we could tckle this problem s follows: drw(): Find the root drw(root) drw(root): Write the line for the
More informationUNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS INFORMATICS 1 COMPUTATION & LOGIC INSTRUCTIONS TO CANDIDATES
UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS INFORMATICS COMPUTATION & LOGIC Sturdy st April 7 : to : INSTRUCTIONS TO CANDIDATES This is tke-home exercise. It will not
More informationContext-Free Grammars
Context-Free Grmmrs Descriing Lnguges We've seen two models for the regulr lnguges: Finite utomt ccept precisely the strings in the lnguge. Regulr expressions descrie precisely the strings in the lnguge.
More informationBefore We Begin. Introduction to Spatial Domain Filtering. Introduction to Digital Image Processing. Overview (1): Administrative Details (1):
Overview (): Before We Begin Administrtive detils Review some questions to consider Winter 2006 Imge Enhncement in the Sptil Domin: Bsics of Sptil Filtering, Smoothing Sptil Filters, Order Sttistics Filters
More information