COMPILERS BASIC COMPILER FUNCTIONS A compiler accepts a program written in a high level language as input and produces its machine language equivalent as output. For the purpose of compiler construction, a high level programming language is described in terms of a grammar. This grammar specifies the formal description of the syntax or legal statements in the language. Example: Assignment statement in Pascal is defined as: < variable > : = < Expression > The compiler has to match statement written by the programmer to the structure defined by the grammars and generates appropriate object code for each statement. The compilation process is so complex that it is not reasonable to implement it in one single step. It is partitioned into a series of sub-process called phases. A phase is a logically cohesive operation that takes as input one representation of the source program and produces an output of another representation. The basic phases are - Lexical Analysis, Syntax Analysis, and Code Generation. Lexical Analysis: It is the first phase. It is also called scanner. It separates characters of the source language into groups that logically belong together. These groups are called tokens. The usual tokens are: Keyword: Identifiers: Operator symbols: Punctuation symbols: such as DO or IF, such as x or num, such as <, =, or, +, and such as parentheses or commas. The output of the lexical analysis is a stream of tokens, which is passed to the next phase; the syntax analyzer or parser. Syntax Analyzer: It groups tokens together into syntactic structure. For example, the three tokens representing A + B might be grouped into a syntactic structure called as expression. Expressions might further be combined to form statements. Often the syntactic structures can be regarded as a tree whose leaves are the tokens. The interior nodes of the tree represent strings of token that logically belong together. Fig. 1 shows the syntax tree for READ statement in PASCAL (read)
Compilers 185 (id - list) READ ( id ) {value} Fig. 1 Syntax Tree Code Generator: It produces the object code by deciding on the memory locations for data, selecting code to access each datum and selecting the registers in which each computation is to be done. Designing a code generator that produces truly efficient object program is one of the most difficult parts of compiler design. In the following sections we discuss the basic elements of a simple compilation process, illustrating this application to the example program in fig. 2. PROGRAM STATS VAR SUM, SUMSQ, I, VALUE, MEAN, VARIANCE : INTEGER BEGIN SUM : = 0 ; SUMSQ : = 0 ; FOR I : = 1 to 100 Do BEGIN READ (VALUE) ; SUM : = SUM + VALUE ; SUMSQ : = SUMSQ + VALUE * VALUE END; MEAN : = SUM DIV 100; VARIANCE : = SUMSQ DIV 100 - MEAN * MEAN ; WRITE (MEAN, VARIANCE) END Fig. 2 Pascal Program GRAMMARS A grammar for a programming language is a formal description of the syntax of programs and individual statements written in the language. The grammar does not describe the semantics or memory of the various statements. To differentiate between syntax and semantics consider the following example: VAR X, Y : REAL VAR I, J, K : INTEGER I : INTEGER X : = I + Y ; I : = J + K ; Fig.3
186 System Software These two programs statement have identical syntax. Each is an assignment statement; the value to be assigned is given by an expression that consists of two variable names separated by the operator '+'. The semantics of the two statements are quite different. The first statement specifies that the variables in the expressions are to be added using integer arithmetic operations. The second statement specifies a floating-point addition, with the integer operand 2 being connected to floating point before adding. The difference between the statements would be recognized during code generation. Grammar can be written using a number of different notations. Backus-Naur Form (BNF) is one of the methods available. It is simple and widely used. It provides capabilities that are different for most purposes. A BNF grammar consists of a set of rules, each of which defines the syntax of some construct in the programming language. A grammar has four components. They are: 1. A set of tokens, known as terminal symbols non-enclosed in bracket. Example: READ, WRITE 2. A set of terminals. The character strings enclosed between the angle brackets (<, >) are called terminal symbols. These are the names of the constructs defined in the grammar. 3. A set of productions where each production consists of a non-terminal called the left side of the production, as "is defined to be" (:: = ), and a sequence of token and/or non-terminal, called the right side of the product. Example: < reads > : : = READ <id - list >. 4. A designation of one of the non-terminals as the start symbol. This rule offers two possibilities separated by the symbol, for the syntax of an < id - list > may consist simply of a token id (the notation id denotes an identifier that is recognized by the scanner). The second syntax. Example: ALPHA ALPHA, BETA ALPHA is an < id - list > that consist of another < id - list > ALPHA, followed by a comma, followed by an id BETA. Tree: It is also called parse tree or syntax tree. It is convenient to display the analysis of a source statement in terms of a grammar as a tree. Example: READ (VALUE) GRAMMAR: (read) : : = READ ( < id -list>) Example: Assignment statement: SUM : = 0 ; SUM : = + VALUE ; SUM : = - VALUE ;
Compilers 187 Grammar: < assign > : : = id : = < exp > < exp > : : = < term > < exp > - < term > < term > : : = < factor > < term > * < factor > < term > DIV < factor > < factor > : : = id int ( < exp > ) Assign consists of an id followed by the token : =, followed by an expression <exp > Fig. 4(a). Show the syntax tree. Expressions are sequence of <terms> connected by the operations + and - Fig. 4(b). Show the syntax tree. Term is a sequence of < factor > S connected by * and DIV Fig. 4(c). A factor may consists of an identifies id or an int ( which is also recognized by the scan) or an < exp > enclosed in parenthesis. Fig. 4(d). < assign > < exp > id : = <exp > < term > + < exp > {variance } Fig. 4 (a) Fig. 4 (b) < term > factor < factor > Dir < term > id X < factor > int Fig.4 (c) Fig. 4 Parse Trees Id (< exp > ) Fig. 4 (d) For the statement Variance : = SUMSQ Div 100 - MEAN * MEAN ; The list of simplified Pascal grammar is shown in fig.5. 1. < prog > : : = PROGRAM < program > VAR <dec - list > BEGIN < stmt > - list > END. 2. < prog - name >: : = id 3. < dec - list > : : = < dec > < dec - list > ; < dec > 4. < dec > : : = < id - list > : < type > 5. < type > : : = integer 6. < id - list > : : = id < id - list >, id 7. <stmt - list > : : = < stmt > <stmt - list > ; < stmt > 8. < stmt > : : = < assign > <read > < write > < for >
188 System Software 9. < assign > : : = id : = < exp > 10. < exp > : : = < term > < exp > + < term > < exp > - < term > 11. < term > : : = < factor > < term > <factor> <term> DIV <factor> 12. < factor > : : = id ; int (< exp >) 13. < READ > : : = READ ( < id - list >) 14. < write > : : = WRITE ( < id - list >) 15. < for > : : = FOR < idex - exp > Do < body > 16. < index - exp> : : = id : = < exp > To ( exp > 17. < body > : : = < start > BEGIN < start - list > END Fig. 5 Simplified Pascal Grammar ( < prog >) PROGRAM < prog - name > VAR dec - list BEGIN <Stmt - list > END Id < dec > {STATS} < stmt - list > ; < stmt > < id - list > : < type > INTEGER < write > (id - list), id {VARIANCE} < stmt - list > ; <stmt > WRITE ( <id - list > ) (id - list ) ; id < stmt - list > ; <stmt > < assign > (id - list ). id (MEAN) {VARIANCE} id (id - list ), id id : = <MEAN> <VALUE > < stmt - list > ; <stmt > {VARIANCE} < exp > < assign > (id - list ), id {I} <stmt > ; < start > id : = <exp> <term> <id -list >, id {mean} <exp> {SMSQ} < stmt > < assign > <term> <term> <term> * <factor> id {SUM} < assign > id : = <exp> <factor> id {SUMSQ} <term> Div <factor> [MEAN] term >term> Div <factor > id id : < exp > factor {MEAN} factor int < term > id {100} <factor> int int {SUM} {100} < factor > { 0} id Next {SUMSQ} int {0} Page
Compilers 189 < for > FOR <index - exp > Do < body > Id : = <exp> To <exp> BEGIN <stm - list> END {I} < term > <term> <factor> <factor> <stmt - list > ; < stmt > int int {I} {100} <symt - list> ; <stmt> <assign > < stmy > <assign > id : = <emp> < read > (SUMSQ id : = <exp> {SUM} READ ( < id - list > ) < exp > + < term > < exp > + < term > id {VALUE? <term > < factor > < term > <term> * <factor> < factor >. id <factor > <factor > id { value} {value} id id id {SUM} {SUMSQ} {value} Fig. 6 Parse tree for the Program 1 Parse tree for Pascal program in fig.1 is shown in fig. 6 1 (a) Draw parse trees, according to the grammar in fig. 5 for the following <id-list> S: (a) ALPHA < id - list > id { ALPHA } (b) ALPHA, BETA, GAMMA < id - list > id < id - list >, {GAMMA} id < id - list >, {BETA} id [ ALPHA ]
190 System Software 2 (a) Draw Parse tree, according to the grammar in fig. 5 for the following < exp > S : (a) ALPHA + BETA < exp > < term > (b) ALPHA - BETA + GAMMA < exp < term > < factor > + < factor > id id {ALPHA} {BETA} < exp > - term < term > < term > * factor < factor > < factor > id {GAMMA} id id {ALPHA} {BETA} (c) ALPHA DIV (BETA + GAMMA) = DELTA < exp > < exp > - < term > < term > < factor > < term > Div < factor > {DELTA} < factor > id {ALPHA} ( < exp > ) < exp > + < term > < term > factor id {BETA} id {GAMMA}
Compilers 191 3. Suppose the rules of the grammar for < exp > and < term > is as follows: < exp > :: = < term > < exp > * < term> < exp> Div < term > < term > :: = <factor> < term > + < factor > < term > - < factor > Draw the parse trees for the following: (a) A1 + B1 (b) A1 - B1 * G1 (c) A1 + DIV (B1 + G1) - D1 (a) A1 + B1 < exp > term < term > + < factor > factor id id {A1} {B1} (b) A1 - B1 * G1 < exp > teerm teerm - < factor > factor term * factor id factor id {A1} id {B1} {G1} (c) A1 DIV (B1 + A1) - D1 < exp > < exp > DIV < term > < term > < term > - < factor > < factor > < factor > id {D1} id {A1} ( < exp > ) < term > < term > + < factor > < factor > id
192 System Software LEXICAL ANALYSIS id {B1} {G1} Lexical Analysis involves scanning the program to be compiled. Scanners are designed to recognize keywords, operations, identifiers, integer, floating point numbers, character strings and other items that are written as part of the source program. Items are recognized directly as single tokens. These tokens could be defined as a part of the grammar. Example: <ident> : : = <letter> <ident> <letter> <ident> <digit> <letter> : : = A B C... Z <digit> : : = 0 1 2... 9 In a such a case the scanner world recognize as tokens the single characters A, B,... Z,, 0, 1,... 9. The parser could interpret a sequence of such characters as the language construct < ident >. Scanners can perform this function more efficiently. There can be significant saving in compilation time since large part of the source program consists of multiple-character identifiers. It is also possible to restrict the length of identifiers in a scanner than in a passing notion. The scanner generally recognizes both single and multiple character tokens directly. The scanner output consists of sequence of tokens. This token can be considered to have a fixed length code. The fig. 7 gives a list of integer code for each token for the program in fig. 5 in such a type of coding scheme, the PROGRAM is represented by the integer value 1, VAR has the integer value 2 and so on. Token Program VAR BEGIN END END INTEGER FOR Code 1 2 3 4 5 6 7 Token READ WRITE To Do ; :, Token : = + - K DIV ( ) Token : = + - K DIV ( ) Code 15 16 17 18 17 20 21 Token Id Int Code 22 23 Fig. 7 Token Coding Scheme For a keyword or an operator the token loading scheme gives sufficient information. In the case of an identifier, it is also necessary to supply particular identifier name that was scanned. It is true for the integer, floating point values, character-string constant etc. A token specifier can be associated with the type of code for such tokens. This specifier gives the identifier name, integer value, etc., that was found by the scanner. Some scanners enter the identifiers directly into a symbol table. The token specifier for the identifiers may be a pointer to the symbol table entry for that identifier. The functions of a scanner are:
Compilers 193 The entire program is not scanned at one time. Scanner is a operator as a procedure that is called by the processor when it needs another token. Scanner is responsible for reading the lines of the source program and possible for printing the source listing. The scanner, except for printing as the output listing, ignores comments. Scanner must look into the language characteristics. Example: FOTRAN : Columns 1-5 Statement number : Column 6 Continuation of line : Column 7. 22 Program statement PASCAL : Blanks function as delimiters for tokens : Statement can be continued freely : End of statement is indicated by ; (semi column) Scanners should look into the rules for the formation of tokens. Example: 'READ': Should not be considered as keyword as it is within quotes. i.e., all string within quotes should not be considered as token. Blanks are significant within the quoted string. Blanks has important factor to play in different language Example 1: FORTRAN Statement: Do 10 I = 1, 100 ; Do is a key word, I is identifier, 10 is the statement number Statement: Do 10 I = 1 ;It is an identifier Do 10 I = 1 Note: Blanks are ignored in FORTRAN statement and hence it is a assignment statement. In this case the scanner must look ahead to see if there is a comma (,) before it can decide in the proper identification of the characters Do. Example 2: In FORTRAN keywords may also be used as an identifier. Words such as IF, THEN, and ELSE might represent either keywords or variable names. IF (THEN.EQ ELSE) THEN IF = THEN ELSE THEN = IF ENDIF Modeling Scanners as Finite Automata Finite automatic provides an easy way to visualize the operation of a scanner. Mathematically, a finite automation consists of a finite set of states and a set of transition from one state to another. Finite automatic is graphically represented. It is shown in fig, State is represented by circle. Arrow indicates the transition from one state to another.
194 System Software Each arrow is labeled with a character or set of characters that can be specified for transition to occur. The starting state has an arrow entering it that is not connected to anything else. 1 State Final State Transition Fig. 8 Example: Finite automata to recognize tokens is gives in fig. 9. The corresponding algorithm is given in fig. 10 0-9 A - Z B A - Z 1 2 3 Fig. 9 Get first Input-character If Input-character in [ 'A'.. ' Z' ] then while Input - character in [ 'A'.. 'Z', ' 0'.. ' 9' ] do get next input - character End {while} end {if first is [ 'A'.. ' Z' ] } else return (token-error) Fig. 10 SYNTACTIC ANALYSIS During syntactic analysis, the source programs are recognized as language constructs described by the grammar being used. Parse tree uses the above process for translation of statements, Parsing techniques are divided into two general classes: -- Bottom up and -- Top down. Top down methods with the rule of the grammar that specifies the goal of the analysis ( i.e., the root of the tree), and attempt to construct the tree so that the terminal nodes match the statement being analyzed. Bottom up methods with the terminal nodes of the tree and attempt to combine these into successively high - level nodes until the root is reached. OPERATOR PRECEDENCE PARSING The bottom up parsing technique considered is called the operator precedence method. This method is loaded on examining pairs of consecutive operators in the source program and making decisions about which operation should be performed first.
Compilers 195 Example: A + B * C - D (1) The usual procedure of operation multiplication and division has higher precedence over addition and subtraction. Now considering equation (1) the two operators (+ and *), we find that + has lower precedence than *. This is written as + * [+ has lower precedence *] Similarly ( * and - ), we find that * - [* has greater precedence -]. The operation precedence method uses such observations to guide the parsing process. A + B * C - D (2) VAR BEGIN END END INTEGER FOR REAS WRITE TO DO : :, : = + - * DIV ) ( Id Int PROGRAM VAR BEGIN END INTEGER FOR READ WRITE TO DO ; :, : = + - * DIV ) ( id Int < Fig 11 Precedence Matrix for the Grammar for fig 5 Equation (2) implies that the sub expression B * C is to be computed before either of the other operations in the expression is performed. In times of the parse tree this means that the * operation appears at a lower level than does either + or -. Thus a bottom up parses should recognize B * C by interpreting it in terms of the grammar,
196 System Software before considering the surrounding terms. The first step in constructing an operatorprecedence parser is to determine the precedence relations between the operators of the grammar. Operator is taken to mean any terminal symbol (i.e., any token). We also have precedence relations involving tokens such as BEGIN, READ, id and (. For the grammar in fig. 5, the precedence relations is given in the fig. 11. Example: PROGRAM VAR ; These two tokens have equal precedence Begin FOR ; BEGIN has lower precedence over FOR. There are some values which do not follows precedence relations for comparisons. Example: ; END and END ; i.e., when ; is followed by END, the ' ; ' has higher precedence and when END is followed by ; the END has higher precedence. In all the statements where precedence relation does not exist in the table, two tokens cannot appear together in any legal statement. If such combination occurs during parsing it should be recognized as error. Let us consider some operator precedence for the grammar in fig. 5. Example: Pascal Statement: BEGIN READ (VALUE); These Pascal statements scanned from left to right, one token at a time. For each pair of operators, the precedence relation between them is determined. Fig. 12(a) shows the parser that has identified the portion of the statement delimited by the precedence relations and to be interpreted in terms of the grammar. (a)... BEGIN READ ( id ) (b)... BEGIN READ ( < N1 > ) ; (c)... BEGIN < N2 > ; (d)... < N2 > READ ( <N1 > ) id (VALUE) Fig. 12 According to the grammar id may be considered as < factor >. (r ule 12), <program > (rule 9) or a < id -list > (rule 6). In operator precedence phase, it is not necessary to indicate which non-terminal symbol is being recognized. It is interpreted as non-terminal < N1 >. Hence the new version is shown in fig. 12(b).
Compilers 197 An operator-precedence parser generally uses a stack to save token that have been scanned but not yet parsed, so it can reexamine them in this way. Precedence relations hold only between terminal symbols, so < N1 > is not involved in this process and a relationship is determined between (and). READ (<N1>) corresponds to rule 13 of the grammar. This rule is the only one that could be applied in recognizing this portion of the program. The sequence is simply interpreted as a sequence of some interpretation < N2 >. Fig. 12(c) shows this interpretation. The parser tree is given in fig. 12(d). Note: Example: (1) The parse tree in fig. 1 and fig. 12 (d) are same except for the name of the non-terminal symbols involved. (2) The name of the non-terminals is arbitrarily chosen. VARIANCE ; = SUMSQ DIV 100 - MEAN * MEAN (i).. id 1 : = id 2 Div... <N1> (ii)... id 1 : = <N1> Div int - <id 2> {SUMSQ} (iii)... id 1 : = <N1> Div <N2> - <N1> <N2> <id 2> int {SUMSQ} {100} (iv).... id 1 : = <N3> - id 3 * <N3> <N1> DIV <N2> id2 int {SUMSQ} {100} v).... id 1 : = <N3> - <N4> * id 4 ; <N4> (vi)... id 1 : = <N3> - <N4> * <N5> id 4 {MEAN} id 3 {MEAN} <N5> (vii)... id 1 : = <N3> - <N6> <N6> <N4> * <N5> id 3 id 4 {MEAN} {MEAN}
198 System Software (viii)... id : = <N7> <N7> <N3> - <N6> (ix)... <N8> <N8> <N7> <N3> <N6> id 1 : = <N1> <N2> <N4> <N5> {VARIANCE} DIV * id 2 int - id 3 id 4 {SUMSQ} {100} {MEAN} {MEAN} SHIFT REDUCE PARSING The operation procedure parsing was developed to shift reduce parsing. This method makes use of a stack to store tokens that have not yet been recognized in terms of the grammar. The actions of the parser are controlled by entries in a table, which is somewhat similar to the precedence matrix. The two main actions of shift reducing parsing are Shift: Push the current token into the stack. Reduce: Recognize symbols on top of the stack according to a rule of a grammar Example: BEGIN READ ( id )... Steps Token Stream 1.... BEGIN READ ( id )... Stack Shift 2.... BEGIN READ ( id ) Shift BEGIN 3.... BEGIN READ ( id )... Shift READ BEGIN
Compilers 199 4... BEGIN READ ( id )... Shift 5.... BEGIN READ ( id )... ( READ BEGIN Explanation Shift 6.... BEGIN READ ( id )... id ( READ BEGIN. < id-list > ( READ BEGIN 1. The parser shift (pushing the current token onto the stack) when it encounters BEGIN 2 to 4. The shift pushes the next three tokens onto the stack. 5. The reduce action is invoked. The reduce converts the token on the top of the stack to a non-terminal symbol from the grammar. 6. The shift pushes onto the stack, to be reduced later as part of the READ statement. Note: Shift roughly corresponds to the action taken by an operator precedence parses when it encounters the relation and. Reduce roughly corresponds to the action taken when an operator precedence parser encounters the relation. RECURSIVE DESCENT PARSING Shift Recursive-Descent is a top-down parsing technique. A recursive-descent parser is made up of a precedence for each non-terminal symbol in the grammar. When a precedence is called it attempts to find a sub-string of the input, ning with the current token, that can be interpreted as the non-terminal with which the procedure is associated. During this process it may call other procedures, or call itself recursively to search for other non-terminals. If the procedure finds the non-terminal that is its goal, it returns an indication of success to its caller. It also advances the current-token pointer past the sub-string it has just recognized. If the precedence is unable to find a sub-string that can be interpreted as to the desired non-terminal, it returns an indication of failure. Example: < read > : : = READ ( < id - list > ) The procedure for < read > in a recursive descent parser first examiner the next two input, looking for READ and (. If these are found, the procedures for < read > then call the procedure for < id - list >. If that procedure succeeds, the < read > procedure examines the next input token, looking for). If all these tests are successful, the < read > procedure returns an indication of success. Otherwise the procedure returns a failure. There are problems to write a complete set of procedures for the grammar of fig. 15.
200 System Software Example: The procedure for < id - list >, corresponding to rule 6 would be unable to decide between its alternatives since id and < id-list > can with id. <id-list > : : = id < id-list >, id If the procedure somehow decided to try the second alternative <id-list>, it would immediately call itself recursively to find an <id-list>. This causes unending chain. Topdown parsers cannot be directly used with a grammar that contains this kind of immediate left recursion. Similarly the problem occurs for rules 3, 7, 10 and 11. Hence the fig. 13 shows the rules 3, 6, 7, 10 and 11 modification. 3 < dec - list > : : = < dec > { ; <dec > } 6 < id - list > : : = id {; id } 7 < stmt - list > : : = < stmt > { ; < stmt > } 10 < exp > : : = < term > { + < term. -- < term > } 11 < term > : : = < factor > { + < factor > Div < factor >.} Fig. 13 Fig. 14 illustrates a recursive-descent parse of the READ statement: READ (VALUE); The modified grammar is considered in the procedure for the non-terminal <read > and < id-list >. It is assumed that TOKEN contains the type of the next input token. PROCEDURE READ BEGIN ROUND : = FALSE If TOKEN + 8 { read } THEN BEGIN advance to next token IF TOKEN + 20 { ( } THEN BEGIN advance to next token IF IDLIST returns success THEN IF token = 21 { ) } THEN BEGIN FOUND : = TRUE advance to next token END { if ) } END { if READ } IF FOUND = TRUE THEN return success else failure end (READ) Fig. 14 Procedure IDLIST FOUND = FALSE
Compilers 201 if TOKEN = 22 {id} then FOUND : = TRUE advance to Next token while (TOKEN = 14 {,}) and (FOUND = TRUE) do advance to next token if TOKEN = 22 {id} then advance to next token else FOUND = FALSE End {while} End {if id} if FOUND : = TRUE then return success else return failure end {IDLIST} Fig. 15 The fig. 15 IDLIST procedure shows an error message if (, ) is not followed by a id. It indicates the failure in the return statement. If the sequence of tokens such as " id, id " could be a legal construct according to the grammar, this recursive-descent technique would not work properly. Fig. 16 shows a graphic representation of the recursive parsing process for the statement being analyzed. (i) (ii) (iii) In this part, the READ procedure has been invoked and has examined the tokens READ and ' ( " from the input stream (indicated by the dashed lines). In this part, the READ has called IDLIST (indicated by the solid line), which has examined the token id. In this part, the IDLIST has returned to READ indicating success; READ has then examined the input token. Note that the sequence of procedure calls and token examinations has completely defined the structures of the READ statement. The parser tree was constructed ning at the root, hence the term top-down parsing. (i) (II) (iii) READ READ READ READ READ IDLIST READ IDLIST ( ( ( id id { Value } { Value Fig. 16 Fig. 17 illustrates a recursive discard parse of the assignment statement.
202 System Software Variance: = SUNSQ DIVISION - MEAN * MEAN The fig. 17 shows the procedures for the non-terminal symbols that are involved in parsing this statement. Procedure ASSIGN FOUND = FALSE if TOKEN = 22 {id} then advance to Next token if TOKEN = 15 {: =} then advance to next token if EXP returns success then FOUND : = TRUE end {if : =} if FOUND : = TRUE then return success else return failure end {ASSIGN} Procedure EXP FOUND = FALSE If TERM returns success then FOUND: = TRUE while ((TOKEN = 16 {+ } ) or (TOKEN = 17 { - } ) ) and (FOUND = TRUE) do advance to next token if TERM returns success then FOUND = FALSE end {while} end {if TERM} if FOUND : = TRUE then return success else return failure end {EXP} Procedure TERM FOUND : = FALSE If FACTOR returns success then
Compilers 203 FOUND : = TRUE while ((TOKEN = 18 { * }) or (TOKEN = 19 {DIV }) and (FOUND = TRUE) do advance to next token if TERM returns failure then FOUND : = FALSE end {while} end {if FACTOR} if FOUND : = TRUE then return success else return failure end {TERM} Procedure FACTOR FOUND : = FALSE if (TOKEN = 22 { id } ) or (TOKEN = 23 {int } ) then FOUND : = TRUE advance to next token end { if id or int } else if TOKEN = 20 { ( } then advance to next token if EXP returns success then if TOKEN = 21 { ) } then (FOUND = TRUE) advance to next token end { if ) } end {if ( } if FOUND : = TRUE then return success else return failure end {FACTOR} Fig. 17 Recursive-Descent Parse of an Assignment Statement A step-by-step representation of the procedure calls and token examination is shown in fig. 1
204 System Software (i) (ii) (iii) ASSIGN ASSIGN ASSIGN id 1 : = id 1 : = EXP id 1 : = { VARIANCE } { VARIANCE } {VARIANCE} (iv) (v) ASSIGN (vi) EXP ASSIGN TERM id 1 : = EXP id 1 : = EXP id 1 : = {VARIANCE} {VARIANCE} {VARIANCE} EXP TERM TERM TERM - TERM FACTOR FACTOR FACTOR FACTOR FACTOR DIV DIV id 2 id 2 int id 2 int {SUMSQ} {SUMSQ} {100} {SUMSQ} {100} (vii) ASSIGN id 1 : = {VARIANCE} TERM EXP - TERM FACTOR DIV FACTOR FACTOR id 2 int id 3 {SUMSQ} {100} {MEANS} (viii) ASSIGN id 1 : = (VARIANCE} TERM EXP - TERM FACTOR FACTOR FACTOR FACTOR * DIV DIV id 2 int id 3 id 4 {SUMSQ} {100} {MEANS} {MEANS} Fig. 18 Step by step Representation for Variance : = SUMSQ Div 100 - MEAN * Mean
Compilers 205 GENERATION OF OBJECT CODE After the analysis of system, the object code is to be generated. The code generation technique used in a set of routine, one for each rule or alternative rule in the grammar. The routines that are related to the meaning of he compounding construct in the language is called the semantic routines. When the parser recognizes a portion of the source program according to some rule of the grammar, the corresponding semantic routines are executed. These semantic routines generate object code directly and hence they are referred as code generation routines. The code generation routines that is discussed are designed for the use with the grammar in fig..5. This grammar is used for code generations to emphasize the point that code generation techniques need not be associated with any particular parsing method. The parsing technique discussed in 1.3 does not follow the constructs specified by this grammar. The operator precedence method ignores certain non-terminal and the recursive-descent method must use slightly modified grammar. The code generation is for the SIC/XE machine. The technique use two data structure: (1) A List (2) A Stack List Count: A variable List count is used to keep a count of the number of items currently in the list. The token specifiers are denoted by ST (token) Example: id ST (id) ; name of the identifier int ST (int) ; value of the integer, # 100 The code generation routines create segments of object code for the compiled program. A symbolic representation is given to these codes using SIC assembler language. LC (Location Counter): It is a counter which is updated to reflect the next variable address in the compiled program (exactly as it is in an assembler). Application Process to READ Statement: (read) + JSUB XREAD WORD 1 < id - list > WORD VALUE READ ( ) {VALUE} Fig. 19(a) Parse Tree for Read Using the rule of the grammar the parser recognizes at each step the left most sub-string of the input that can be interpreted. In an operator precedence parse, the recognition occurs when a sub-string of the input is reduced to some non-terminal <N i>. In a recursive-descent parse, the recognition occurs when a procedure returns to its caller, indicating success. Thus the parser first recognizes the id VALUE as an < id - list >, and then recognizes the complete statement as a < read >.
206 System Software The symbolic representation of the object code to be generated for the READ statement is as shown in fig. 19(b). This code consists of a call to a statement XREAD, which world be a part of a standard library associated with the compiler. The subroutine any program that wants to perform a READ operation can call XREAD. XREAD is linked together with the generated object program by a linking loader or a linkage editor. The technique is commonly used for the compilation of statements that perform voluntarily complex functions. The use of a subroutine avoids the repetitive generation of large amounts of in-line code, which makes the object program smaller. The parameter list for XREAD is defined immediately after the JSUB that calls it. The first word is the number of variable that will be assigned values by the READ. The following word gives the addresses of three variables. Fig. 19(c) shows the routines that might be used to accomplish the code generation. 1. < id - list > : : = id add ST (id) to list add 1 to List_count 2. < id - list > : : = < id - list >, id add ST (id) to list add 1 to LC List_Current 3. < read > : : = READ (< id - list >) generate [ + JSUB XREAD ] record external reference to XREAD generate [WORD List - count] for each item on list of do remove ST (ITEM) from list generate [WORD ST (ITEM)] end List _count : = 0 Fig. 19 (c) Routine for READ Code Generation The first two statements (1) and (2) correspond to alternative structure for < id - list >, that is < id - list > : : = id < id - list >, id. In each case the token specifies ST (id) for a new identifier being called to the < id - list > is inserted into the list used by the code-generation routine, and list-count is updated to reflect the insertion. After the entire < id-list > has been parsed, the list contains the token specifiers for all the identifiers that are part of the < id- list >. When the < read > statement is recognized, the token specifiers are removed from the list and used to generate the object code for the READ. Code-generation Process for the Assignment Statement Example: VARIANCE: = SUMSQ DIV 100 - MEAN * MEAN The parser tree for this statement is shown in fig. 20. Most of the work of parsing involves the analysis of the < exp > on the right had side of the " : = " statement.:
Compilers 207 < assign > < exp > < exp > < exp > (term) < term > < term > < term > < factor > < factor > < factor > < factor > id : = id DIV int _ id * id {VARIANCE} { SUMSQ } {100} {MEAN} {MEAN} Fig. 20 The parser first recognizes the id SUMSQ as a < factor > and < term > ; then it recognizes the int 100 as a < factor >; then it recognizes SUNSQ DIV 100 as a < term >, and so forth. The order in which the parts of the statements are recognized is the same as the order in which the calculations are to be performed. A code-generation routine is called for each portion of the statement is recognized. Example; For a rule < term > 1 : : = < term > 2 * < factor > a code is to be generated. The subscripts are used to distinguish between the two occurrences of < term >. The code-generation routines perform all arithmetic operations using register A. Hence the multiple < term > 2 * < factor > after multiplication is available in register A. Before multiplication one of the operand < term > 2 must be located in A-register. The results after multiplication will be left in register A. So we need to keep track of the result left in register A by each segment of code that is generated. This is accomplished by extending the token-specifier idea to non-terminal nodes of the parse tree. The node specifier ST (< term1>) would be set to ra, indicating that the result of the completion is in register A. the variable REGA is used to indicate the highest level node of the parse tree when value is left in register A by the code generated so far. Clearly there can be only one such node at any point in the code-generation process. If the value corresponding to a node is not in register A, the specifier for the node is similar to a token specifier: either a pointer to a symbol table entry for the variable that contains the value or an integer constant. Fig. 21 shows the code-generation routine considering the A-register of the machine. 1. < assign > : : = id := < exp > GETA (< exp >) generate [ STA ST (id)] REGA : = null
208 System Software 2. <exp> :: =< term > ST < exp > : = ST (< term >) if ST < exp > = ra then REGA : = < exp > 3. < exp > 1 : : = < exp > 2 + < term > if SR (< exp > 2 ) = ra then generate [ADD ST (< term >)] else if ST (< term >) = ra then generate [ADD ST (< exp > 2 )] else GETA (< EXP > 2 ) generate [ADD ST(< term >)] end ST (< exp > 1 ) : = ra REGA : = < exp > 1 4. < exp > 1 : : = < exp > 2 - < term > if ST (< exp > 2 ) = ra then generate [SUB ST (< term >)] else GETA (< EXP > 2 ) generate [ SUB ST (< term >)] end SR (< exp > 1 ) : = ra REGA : = < exp > 1 5. < term > : : = < factor > ST (< term >) : = ST (< factor >) if ST (<term >) = ra then REGA : = < term > 6. < term > 1 : : = < term > 2 * < factor > if ST (< term > 2 ) = ra then generate [ MUL ST (< factor >)] else if S (< factor >) = ra then generate [ MUL ST (< term > 2 )] else GETA (< term > 2 ) generate [ MUL SrT(< factor >)] end ST (< term > 1 ) : = ra REGA : = < term > 1 7. < term > : : = < term > 2 DIV < factor > if SR (< term > 2 ) = ra then generate [DIV ST(< factor >)]
Compilers 209 else GETA (< term > 2 ) generate [ DIV ST (< factor >)] end SR (< term > 1 ) : = ra REGA : = < term > 1 < factor > : : = id ST (< factor >) : = ST (id) 9. < factor > : : = int ST (< factor >) : = ST (int) 10. < factor > : : = < exp > ST (< factor >) : = ST (< exp >) if ST (< factor >) = ra then REGA : = < factor > Fig. 21 Code Generation Routines If the node specifies for either operand is ra, the corresponding value is already in register A, the routine simply generates a MUL instruction. The node specifier for the other operand gives the operand address for this MUL. Otherwise, the procedure GETA is called. The GETA procedure is shown in fig. 22. Procedure - GETA (NODE) if REGA = null then generate [LDA ST (NODE) ] else if ST (NODE) π ra then creates a new looking variable Temp i generate [STA Temp i ] record forward reference to Temp i ST (REGA) : = Temp i Generate [LDA ST (NODE)] end (if ra) ST(NODE) : = ra REGA : = NODE end {GETA } Fig. 22 The procedure GETA generates a LDA instruction to load the values associated to <term> 2 into register A. Before loading the value into A-register, it confirms whether A is null. If it is not null it generates STA instruction to save the contents of register-a into Temp-variable. There can be number of Temp variable like Temp 1, Temp 2... etc. The temporary variables used during a completion will be assigned storage location at the end of the object program. The node specifies for the node associated with the value
210 System Software previously in register A, indicated by REGA is reset to indicate the temporary variable used. After the necessary instructions are generated, the code-generation routine sets ST (< term > 1 ) and REGA to indicate that the value corresponding to < terms > 1 is now in register A. This completes the code-generation action for the * operation. The code-generation routine for ' + ' operation is the same as the ' * ' operation. The routine ' DIV ' and ' - ' are similar except that for these operations it is necessary for the first operand to be in register A. The code generation for < assign > consists of bringing the value to be assigned into register A (using GETA) and then generating a STA instruction. The remaining rules in fig. 21 do not require the generation of any instruction since no computation and data movement is involved. The object code generated for the assignment statement is shown in fig. 22. LDA SUMSQ DIV * 100 STA TMP 1 LDA MEAN MUL MEAN STA TMP 2 LDA TMP 1 SUB TMP 2 STA VARIABLE Fig. 22 For the grammar < prog > the code-generation routine is shown in fig. 23. When <prog> is recognized, storage locations are assigned to any temporary (Temp) variables that have been used. Any references to these variables are then fixed in the object code using the same process performed for forward references by a one-pass assembler. The compiler also generates any modification records required to describe external references to library subroutine. < prog > : : = PROGRAM < prog-name > VAR < dec list > BEGIN < stmp -- list > END. generate [LDL RETADR] generate [RSUB] for each Temp variable used do generate [ Temp RESW 1] insert [ J EXADDR ] {jump to first executable instruction} in bytes 3-5 of object program. fix up forward reference to Temp variables generate modification records for external references generates [END]. The < prog-name > generates header information in the object program that is similar to that created from the START and EXTREF as assembler directives. It also
Compilers 211 generates instructions to save the return address and jump to the first executable instruction in the compiled program. Fig. 24 shows the code generation routine for the grammar < prog-name >. < Program > : : = id generate [START 0] generate [EXTREF XREAD, XWRITE] generate [STL RETADR] add 3 to LC {leave room for jump to first executable instruction} generate [RETADR RESW 1] Fig. 24 Similar to the previous code-generation routine fig. 25 shows the codegeneration for < dec - list >, < dec >, < write >, < for >, < index - exp > and body. < dec - list > : : = { alternatives } save LC as EXADDR {tentative address of first executable instruction} < dec > : : = > id - list > : < type > for each item on list do remove ST (NAME) from list enter LC symbol table as address for NAME generate [ST (NAME) RESW 1] end LIST COUNT : = 0 < write > : : = WRITE ( < id - list > ) generate [ + JSUB XWRITE] record external reference to XWRITE generate [WORD LISTCOUNT] for each item on list do remove ST (ITEM) from list generate [WORD ST (ITEM)] end LIST COUNT : = 0 < for > : : = FOR < id ex -- exp > Do < body > POP JUMPADDR from stack {address of jump out of loop} POP ST (INDEX) from stack {index variable} POP LOOPADDR from stack {ning address of loop} generate [LDA ST (INDEX)] generate [ADD #1]
212 System Software generate insert [ J LOOPADDR] [ JGT LC ] at location JUMPADDR < index - exp > : : = id : = < exp > TO < exp > 2 GETA (< exp >;) Push LC onto stack {ning addressing loop} Push ST (id) onto stack {index variable} Generate [STA ST (id)] Generate [ COMP ST (< exp > 2 )] Push LC onto stack {address of jump out of loop} and 3 to LC [ leave room for jump instruction] REGA : = null Fig. 25 There are no code-generation for the statements < type > : : = INTEGER < stmt - list > : : = {either alternative} < stmt > : : = {any alternative} < body > : : = {either alternative} For the Pascal program in fig. 1 the complete code-generation process is shown in fig. 26. 1 STATS START 0 {Program Header} EXTREF XREAD, XREAD, XWRITE STL TETADR {Save return address} J {EXADDR} 2 RETADDR RESW 1 3 SUM RESW 1 SUMSQ RESW 1 I RESW 1 VALUE RESW 1 MEAN RESW 1 VARIANCE RESW 1 5 {EXADDR} LDA # 0 {SUM = 0} STA SUM 6 LDA # 0 {SUMSQ : = 0} STA SUMSQ 7 LDA # 1 {FOR I : = 1 TO 100} {L1}STA I COMP # 100 JGT {L2} 9 + JSUB X READ {READ (VALUE) } WORD 1 WORD VALUE 10 LDA SUM {SUM : = SUM + VALUE} ADD VALUE STA SUM 11 LDA VALUE {SUMSQ : = SUMSQ * VALUE * VALUE}
Compilers 213 MUL VALUE ADD SUMSQ STA SUMSQ LDA I {END OF FOR LOOP} ADD # 1 J {L1} 13 {L2} LDA SUM {MEAN : = SUM DIVISION} DIV # 100 STA MEAN 14 LDA SUM {VARIABLE : = SUMSQ DIV DIV # 100 100 - MEAN * MEAN} STA TEMP1 LDA MEAN MUL MEAN STA TEMP2 LDA TEMP1 SUB TEMP2 STA VARIANCE 15 +JSUB XWRITE {WRITE (MEAN, VARIANCE) } WORD 2 WORD MEAN WORD VARIABLE LDL RETADR RSUB TEMP 1` RESW 1 {WORKING VARIABLE USED} TEMP 2 RESW 1 END Fig. 25 Object Code Generated for Pascal Program 8.1 MACHINE DEPENDENT COMPILER FEATURES At an elementary level, all the code generation is machine dependent. This is because, we must know the instruction set of a computer to generate code for it. There are many more complex issues involved. They are: Allocation of register Rearrangement of machine instruction to improve efficiency of execution Considering an intermediate form of the program being compiled normally does such types of code optimization. In this intermediate form, the syntax and semantics of the source statements have been completely analyzed, but the actual translation into machine code has not yet been performed. It is easier to analyze and manipulate this intermediate code than to perform the operations on either the source program or the machine code. The intermediate form made in a compiler, is not strictly dependent on the machine for which the compiler is designed. 8.1.1 INTERMEDIATE FORM OF THE PROGRAM The intermediate form that is discussed here represents the executable instruction of the program with a sequence of quadruples. Each quadruples of the form
214 System Software Where Example 1: Operation, OP1, OP2, result. Operation - is some function to be performed by the object code OP 1 & OP2 - are the operands for the operation and Result - designation when the resulting value is to be placed. SUM : = SUM + VALUE could be represented as +, SUM, Value, i, i 1 : = i 1,, SUM The entry i 1, designates an intermediate result (SUM + VALUE); the second quadruple assigns the value of this intermediate result to SUM. Assignment is treated as a separate operation ( : =). Example 2 : VARIANCE : = SUMSQ, DIV 100 -- MEAN * MEAN DIV, SUMSQ, #100, i 1 *, MEAN, MEAN, i 2 -, i 1, i 2, i 3 : : = i 3, VARIABLE Note: Quadruples appears in the order in which the corresponding object code instructions are to be executed. This greatly simplifies the task of analyzing the code for purposes of optimization. It is also easy to translate into machine instructions. For the source program in Pascal shown in fig. 1. The corresponding quadruples are shown in fig. 27. The READ and WRITE statements are represented with a CALL operation, followed by PARM quadruples that specify the parameters of the READ or WRITE. The JGT operation in quadruples 4 in fig. 27 compares the values of its two operands and jumps to quadruple 15 if the first operand is greater than the second. The J operation in quadruples 14 jumps unconditionally to quadruple 4. Line Operation OP 1 OP 2 Result Pascal Statement 1. : = # 0 SUM SUM : = 0 2. : = # 0 SUMSQ SUMSQ : = 0 3. : = # 1 I FOR I : = 1 to 100 4. JGT I #100 (15) 5. CALL XREAD READ (VALUE) 6. PARAM VALUE 7. + SUM VALUE i 1 SUM : = SUM + VALUE ; = i 1 SUM 9. * VALUE VALUE i 2 SUMSQ : = SUMSQ + VALUE 10. + SUMSQ i 2 i 3 * VALUE 11. : = i 3 SUMSQ
Compilers 215 12. + I #1 i 4 End of FOR loop 13. : = i 4 I 14. J (4) 15. DIV SUM #100 i 5 MEAN : = SUM DIV 100 16. : = i 5 MEAN 17. DIV SUMSQ #100 i 6 VARIANCE : = 1 * MEAN MEAN i 7 SUMSQ DIV 100 19. - i 6 i 7 i 8 - MEAN * MEAN 20. : = i 8 VARIANCE 21. CALL XWRITE WRITE (MEAN, VALIANCE 22. PARAM MEAN 23. PARAM VARIANCE Fig..27 Intermediate Code for the Pascal Program 8.1.2 MACHINE - DEPENDENT CODE OPTIMIZATION There are several different possibilities for performing machine-dependent code optimization. -- Assignment and use of registers: Here we concentrate the use of registers as instruction operand. The bottleneck in all computers to perform with high speed is the access of data from memory. If machine instructions use registers as operands the speed of operation is much faster. Therefore, we would prefer to keep in registers all variables and intermediate result that will be used later in the program. There are rarely as many registers available as we would like to use. The problem then becomes which register value to replace when it is necessary to assign a register for some other purpose. On reasonable approach is to scan the program for the next point at which each register value would be used. The value that will not be needed for the longest time is the one that should be replaced. If the register that is being reassigned contains the value of some variable already stored in memory, the value can simply be discarded. Otherwise, this value must be saved using a temporary variable. This is one of the functions performed by the GETA procedure. In using register assignment, a compiler must also consider control flow of the program. If they are jump operations in the program, the register content may not have the value that is intended. The contents may be changed. Usually the existence of jump instructions creates difficulty in keeping track of registers contents. One way to deal with the problem is to divide the problem into basic blocks. A basic block is a sequence of quadruples with one entry point, which is at the ning of the block, one exit point, which is at the end of the block, and no jumps within the blocks. Since procedure calls can have unpredictable effects as register contents, a CALL operation is usually considered to a new basic block. The assignment and use of registers within a basic block can follow as described previously. When control passes from one block to another, all values currently held in registers are saved in temporary variables. For the problem is fig..27, the quadruples can be divided into five blocks. They are:
216 System Software Block -- A Quadruples 1-3 Block -- B Quadruples 4 Block -- C Quadruples 5-14 Block -- D Quadruples 15-20 A : 1-3 B : 4 C : 5-14 D : 15-20 Block -- E Quadruples 21-23 Fig. 28 E : 21-23 Fig. 28 shows the basic blocks of the flow group for the quadruples in fig. 27. An arrow from one block to another indicates that control can pass directly from one quadruple to another. This kind of representation is called a flow group. -- Rearranging quadruples before machine code generation: Example : 1) DIV SUMSQ # 100 i 1 2) * MEAN MEAN i 2 3) - i 1 i 2 i 3 4) : = i 3 VARIANCE LDA SUMSQ LDA T 1 DIV # 100 SUB T 2 STA T 1 STA VARIANCE LDA MEAN MUL MEAN STA T 2 Fig. 29 Fig. 29 shows a typical generation of machine code from the quadruples using only a single register. Note that the value of the intermediate result, is calculated first and stored in temporary variable T 1. Then the value of i 2 is calculated subtracting i 2 from i i. Even though i 2 value is in the register, it is not possible to perform the subtraction operation. It is necessary to store the value of i 2 in another temporary variable T 2 and then load the value of i 1 from T 1 into register A before performing the subtraction. The optimizing compiler could rearrange the quadruples so that the second operand of the subtraction is computed first. This results in reducing two memory accesses. Fig. 29 shows the rearrangements. * MEAN MEAN i 2 DIV SUMSQ # 100 i 1
Compilers 217 - i 1 i 2 i 3 := i 3 VARIANCE LDA MEAN MUL MEAN STA T 1 LDA SUMSQ DIV # 100 SUB T 1 STA VARIANCE Fig. 29 Rearrangement of Quadruples for Code Optimization -- Characteristics and Instructions of Target Machine: These may be special loop - control instructions or addressing modes that can be used to create more efficient object code. On some computers there are high-level machine instructions that can perform complicated functions such as calling procedure and manipulating data structures in a single operation. Some computers have multiple functional blocks. The source code must be rearranged to use all the blocks or most of the blocks concurrently. This is possible if the result of one block does not depend on the result of the other. There are some systems where the data flow can be arranged between blocks without storing the intermediate data in any register. An optimizing compiler for such a machine could rearrange object code instructions to take advantage of these properties. Machine Independent Compiler Features Machine independent compilers describe the method for handling structured variables such as arrays. Problems involved in compiling a block-structured language indicate some possible solution. 3.1 STRUCTURED VARIABLES Structured variables discussed here are arrays, records, strings and sets. The primarily consideration is the allocation of storage for such variable and then the generation of code to reference then. Arrays: In Pascal array declaration - (i) Single dimension array: A: ARRAY [ 1.. 10] OF INTEGER If each integer variable occupies one word of memory, then we require 10 words of memory to store this array. In general an array declaration is ARRAY [ l.. u ] OF INTEGER Memory word allocated = ( u - l + 1) words. (ii) Two dimension array : B : ARRAY [ 0.. 3, 1.. 3 ] OF INTEGER In this type of declaration total word memory required is 0 to 3 = 4 ; 1-3 = 3 ; 4 x 3 = 12 word memory locations.