CS 432 Fall Mike Lam, Professor a (bc)* Regular Expressions and Finite Automata

Transcription

1 CS 432 Fll 2017 Mike Lm, Professor (c)* Regulr Expressions nd Finite Automt

2 Compiltion Current focus "Bck end" Source code Tokens Syntx tree Mchine code chr dt[20]; int min() { flot x = 42.0; return 7; } 7f 45 4c Lexing Prsing Code Genertion & Optimiztion "Front end"

3 Lexicl Anlysis Lexemes or tokens: the smllest uilding locks of lnguge's syntx Lexing or scnning: the process of seprting chrcter strem into tokens totl = sum(vls) / n chr *str = "hi"; totl identifier = equls_op sum identifier ( left_pren vls identifier ) right_pren / divide_op n identifier chr keyword * str_op str identifier = equls_op "hi" str_literl ; semicolon

4 Discussion question Wht is lnguge?

5 Lnguge A lnguge is " (potentilly infinite) set of strings over finite lphet"

6 Discussion question How do we descrie lnguges? xyy xy xyyzzz xyz xyzz xyyzz xyyz xyzzz (etc.) xy xyy xyz xyyz xyzz xyyzz xyzzz xyyzzz (etc.) xy xyy xyz xyyz xyzz xyyzz xyzzz xyyzzz (etc.)

7 Lnguge description Wys to descrie lnguges Ad-hoc prose A single x followed y one or two y s followed y ny numer of z s Forml regulr expressions (current focus) x(y yy)z* Forml grmmrs (in two weeks) A x B C B y y y C z C ε

8 Lnguges Chomsky Hierrchy of Lnguges Recursively enumerle Context-sensitive Context-free Regulr Most useful for compilers Alphet: Σ = { set of ll chrcters } Lnguge: L = { set of sequences of chrcters from Σ }

9 Regulr expressions Regulr expressions descrie regulr lnguges Cn lso e thought of s generlized serch ptterns Three sic recursive opertions: Alterntion: Conctention: ("Kleene") Closure: * Extended constructs: Chrcter sets: [0-9] == Grouping: ( )c == c c Positive closure: + == *

10 Discussion question How would you implement regulr expressions? Given regulr expression nd string, how would you tell whether the string elongs to the lnguge descried y the regulr expression?

11 Lexicl Anlysis Implemented using stte mchines (finite stte utomt) Set of sttes with single strt stte Trnsitions etween sttes on inputs (w/ implicit ded sttes) Some sttes re finl or ccepting Deterministic vs. non-deterministic Non-deterministic: multiple possile sttes for given sentence One edge from ech stte per chrcter (deterministic) Multiple edges from ech stte per chrcter (non-deterministic) Empty or ε-trnsitions (non-deterministic) Regex:

12 Deterministic finite utomt Forml definition S: set of sttes Σ: lphet (set of chrcters) δ: trnsition function: (S, Σ) S s 0: strt stte S A : ccepting/finl sttes Acceptnce lgorithm s := s 0 for ech input c: s := δ(s,c) return s S A s1 S = { s1, s2 } Σ = { } δ = { (s1, s2), (s2, Ø) } s 0: = s1 S A = { s2 } s2 Alterntive δ representtion: s1 s2 s2 Ø

13 Non-deterministic finite utomt Forml definition DFA w/ multiple pths nd ε-trnsitions δ: (S, (Σ {ε})) -> [S] ε-closure: ll sttes rechle from s vi ε-trnsitions Formlly: {s} { t S (s,ε t) δ } (extended to sets y union) Acceptnce lgorithm T := ε-closure(s 0 ) for ech input c: N := {} for ech s in T: N := N ε-closure(δ(s,c)) T := N return T S A > 0

14 Summry DFAs S: set of sttes Σ: lphet (set of chrcters) δ: trnsition function: (S, Σ) S s0: strt stte SA : ccepting/finl sttes ccept(): s := s 0 for ech input c: s := δ(s,c) return s S A δ my contin ε-trnsitions δ my contin trnsitions to multiple different sttes on the sme symol ccept(): T := ε-closure(s 0 ) for ech input c: N := {} for ech s in T: N := N ε-closure(δ(s,c)) T := N NFAs return T S A > 0

15 Lexicl Anlysis Exmples:

16 Lexicl Anlysis Exmples: * * * (c c*)

17 Lexicl Anlysis Exmples: * * * c (c c*) c c

18 Equivlence A regulr expression nd finite utomton re equivlent if they recognize the sme lnguge Sme pplies etween different REs nd etween different FAs Regulr expressions, NFAs, nd DFAs ll descrie the sme set of lnguges "Regulr lnguges" from Chomsky hierrchy Next week, we will lern how to convert etween them

19 Appliction PA2: Use Jv regulr expressions to tokenize Decf files Process the input one line t time Generlly: one regex per token type Ech regex egins with ^ (only mtch from eginning) Prioritize regexes nd try ech of them in turn When you find mtch, extrct the mtching text Repet until no mtch is found or input is consumed Less efficient thn n uto-generted lexer However, it is simpler to understnd (Our pproch to PA3 will e similr)

20 Activity Construct stte mchines for the following regulr expressions: x*yz* 1(1 0)* 1(10)* ( c)( c) (dd*.d*) (d*.dd*) ε-trnsitions my mke this one slightly esier