2. λ is a regular expression and denotes the set {λ} 4. If r and s are regular expressions denoting the languages R and S, respectively

Transcription

1 Regular expressions: a regular expression is built up out of simpler regular expressions using a set of defining rules. Regular expressions allows us to define tokens of programming languages such as identifiers. Regular expressions can be defined inductively by the following set of defining rules: Let A be an alphabet: 1. is a regular expression and denotes the empty set 2. λ is a regular expression and denotes the set {λ} 3. for all a A, a is regular expression and dentes the set {a} 4. If r and s are regular expressions denoting the languages R and S, respectively, then (a) r s is a regular expression denotes the set R S (b) (rs) is a regular expression denotes the set RS (c) (r ) is a regular expression denotes the set R Operators precedence: as a convention we will consider the following precedence of the regular expressions operators: () * concatenation By this convention a regular expression like (a) ((b) (c)) can be written in the simple form a b c. The language denoted by a regular expression r will be expressed as L(r) 1

2 Example: for the alphabet A={a, b} 1. a b is a regular expression denotes the set {a} {b} 2. a is a regular expression denotes the set {a, aa, aaa, } 3. a b is a regular expression denotes the set {a, aa, aaa, } {b} 4. a b is a regular expression denotes the set {ab, aab, aaab, } Regular definitions For convenience, we will give names to regular expressions. For an alphabet A the regular definitions is a sequence of definitions of the form d 1 r 1 d 2 r 2 d n r n where each d i is a distinct name, and each r i is a regular expression over the symbols in A {d 1, d 2,, d i 1 } Example The set of identifiers in Pascal can be expressed by the following regular definition: letter A B Z a b z id letter (letter ) 2

3 Shorthands We will use the following shorthands frequently in regular expressions 1. + which means one or more instance. For example if r is a regular expression corresponding to the language {a} then r + is the regular expression representing the language {a, aa, }. Note that r + =rr. 2.? which means zero or one instance. For example if r is a regular expression corresponding to the language {a} then r? is a regular expression represents the language {a} {λ} 3. [] which used as a notation for character classes. For example if r=a b c is a regular expression then we abbreviate it to r=[abc] and if r=a b z we abbreviate it to r=[a-z] Example The following regular definition represents unsigned numbers in C s + fraction (.s)? exponent (E(+ -)? s)? num s fraction exponent 3

4 Recognition of Tokens: We illustrate the process of tokens recognition by the following example Example: consider the following grammar statement if expression then statement if expression then statement else statement λ expression term relop term term term id number In the above grammar statement, expression, and term are nonterminal and if, then, else, relop, id, and number are terminal symbols stand for the names of the regular expressions given by the following regular definitions: if if then then else else relop < <= = <> > >= id letter (letter ) number + (. + )? ( E( + )? + )? where letter and are defined as: letter [A Za z] [0 9] 4

5 In addition, we assume lexemes are separated by white space, consisting of blanks, tabs, and newlines. This is given by the following regular definition: delimiter blank tab newline whitespace delimiter + We define the attribute values for the relational operators by the symbolic constants LT, LE, EQ, NE, GT, GE. The following table shows the relationship between the tokens, regular expressions and attribute values for the above grammar: regular expression token whitespace - - if if - then then - else else - attribute-value id id pointer to table entry number number pointer to table entry < relop LT <= relop LE = relop EQ < relop NE relop GT = relop GE 5

6 The following is the transition diagram for relational operators which was given by the regular definition relop < <= = < = start < = return(relop, LE) return(relop, NE) = * return(relop, LT) return(relop, EQ) = return(relop, GE) * return(relop, GT) 6

7 The following is the transition diagram for identifiers which was given by the regular definition id letter (letter ) letter or start letter * return(gettoken(), install id()) The following is the transition diagram for numbers which was given by the regular definition num + (. + )? ( E( + -)? + )? start. E + or - return(gettoken(), install num()) 7