Jim Lambers ENERGY 211 / CME 211 Autumn Quarter 2008-09 Programming Project 4 This project is due at 11:59pm on Friday, October 31. 1 Introduction In this project, you will do the following: 1. Implement functions that recognize expressions from text, where the expressions include numbers, variables, arithmetic operators, function calls, and array references. This recognition process, called parsing or syntax analysis, will include checking for syntax errors. 2. Implement a type, called SymExpr, that represents these expressions in such a way that one can perform operations on them. In this project, the only such operation will be conversion of an expression to a string for display. 2 Files Before beginning this project, you will need to download the source and header files from the Assignments page on the course web site: symtab.h contains preprocessor definitions that are used by several other files. You do not need to edit this file. parse expr.h contains the declaration of the functions that are is used to recognize complete expressions from text. You do not need to edit this file. parse expr.cpp contains the implementation of the parsing functions, which you will provide. symexpr.h contains the declaration of the SymExpr type, which is used to store expressions and support operations on them. You do not need to edit this file. symexpr.cpp contains the implementation of functions that operate on SymExpr objects, which you will provide. main.cpp, the main application file in which the code in the other source files is to be tested. You do not need to edit this file, except to comment out usages of functions that have not yet been implemented. 1
Makefile, for compiling your code on UNIX/Linux/Mac OS X. You do not need to edit this file. input4.txt and output4.txt, sample input on which you can run your program, and the output your program should generate, provided that you have not yet modified main.cpp. In addition, you will need the files scan expr.h and scan expr.cpp from Project 2. You will not need to make any changes to these files, unless you have bugs to fix from Project 2. We will assist you in fixing these bugs, if necessary. 3 Expression Language This section describes the language of expressions that your program will recognize. 3.1 Syntax In Project 2, you implemented a scanner, or lexical analyzer, that recognizes the tokens of our expression language. In this project, you must write a program that will group the language s words, or tokens, into sentences, which are individual expressions. To prescribe how expressions are built up from tokens, we use a context-free grammar. Such a grammar consists of grammar symbols, which fall into two categories: terminal symbols, which are simply tokens nonterminal symbols, which are defined recursively in terms of other grammar symbols. They correspond to either a complete sentence or components of a sentence, such as a phrase or clause. A grammar also includes rules, called productions, which describes how each nonterminal symbol is defined in terms of other symbols. One nonterminal symbol is designated the start symbol, which corresponds to a complete sentence. In our grammar, Stmt is the start symbol. The following table lists the productions of the grammar. 2
Grammar Symbol Production Description Stmt Expr = Expr Assignment statement (left side must be identifier or array reference) Stmt Expr Expression statement Expr Expr + T erm Addition of term Expr Expr - T erm Subtraction of term Expr T erm Single-term expression T erm T erm * F actor Multiplication by factor T erm T erm / F actor Division by factor T erm F actor Single-factor term F actor - F actor Negated factor F actor Base F actor Exponentiation F actor Base Factor consisting of base expression Base num Floating-point literal Base id Variable Base id ArrayRef s Array reference Base func ( Arglist ) Function call with arguments Base ( Expr ) Parenthesized expression Arglist Expr, Arglist Comma-separated arguments Arglist Expr Last argument in list ArrayRef s ArrayRef s [ Expr ] List of array subscripts ArrayRef s [ Expr ] Single array reference 3.2 Recursive-Descent Parsing To parse the text of an expression, we will use an approach called recursive-descent parsing, which, while more limited than other approaches in terms of the grammars it can implement, is easier to implement. The basic idea is to associate each nonterminal symbol with a function that recognizes that nonterminal, and implements each of the productions associated with the nonterminal. The appropriate production is selected by examining the tokens as they are read. For example, the function for Base can select the correct production based on whether the next token from the input is a num, id, func or (. If it is none of these, the input contains a syntax error. If it is an id, there are two productions that can apply, but the correct one can be determined by examining the next token. Typically, the function for a given nonterminal calls some of the functions for other nonterminals. For example, the function for Factor begins by calling the one for Base, and if the next token is, it will also call the function for Factor. The nonterminals Expr, Term and ArrayRefs are unusual in that their productions include left recursion, because each nonterminal has a production that begins with itself. Therefore, it can t simply call itself recursively, because the recursion has no 3
way of ending. The easiest way to work around this is to use a loop. For example, by viewing an Expr as a sequence of one or more terms that are added or subtracted, the associated function can call the function for Term, and then check if the next token is a + or -. If it is, then Term is called again, and if not, then the function for Expr should exit. 4 The Parser The parsing functions, assisted by nexttoken from Project 2, construct a SymExpr object (see the next section) that represents, in a hierarchical data structure, an expression given as input. 4.1 Functions GetFunctionCode( const std::string& s ); This function returns an integer code corresponding to the name of the function stored in the string s. The table below lists the function names and corresponding codes, which are defined in symtab.h. Function name "exp" "sin" "cos" "sqrt" "log" "diff" "clear" Symbol table code SYMTAB EXP SYMTAB SIN SYMTAB COS SYMTAB SQRT SYMTAB LOG SYMTAB DIFF SYMTAB CLEAR SymExpr Parse( std::istream& in ); This is the main function for the parser. It reads text from the input stream in and recognizes a single Stmt from the text. It initiates the parsing process by calling nexttoken to read the first token, and then calling Stmt (see below) to continue parsing. After Stmt finishes parsing the statement and returns, Parse must check whether the next token is TOKEN EOS, since each input string must contain only one statement and nothing else. If this is not the case, then an exception must be thrown with error message "end-of-string expected". int Stmt( std::istream& in, int token, std::string& tok str, SymExpr& S ); This function recognizes a single Stmt from text read from the input stream in. The second argument token is the code of the last token that was read. The third argument tok str is an input and output argument. Initially, it contains the text of the last token that was read before this function was called. When the function exits, it contains the text of the token 4
that has been most recently read, which is the first token that follows the Stmt. The numeric code for this token is to be returned. The last argument, S, is the SymExpr object that stores the expression corresponding to the Stmt that has been parsed. A Stmt consists of an Expr, followed by an optional = and another Expr. If there is an =, then the Expr on the left-hand side must be a single id, or an array reference, or an exception must be thrown, with the error message "invalid lhs". int Expr( std::istream& in, int token, std::string& tok str, SymExpr& E ); This function recognizes a single Expr from text read from the input stream in. The second argument token is the code of the last token that was read. The third argument tok str is an input and output argument. Initially, it contains the text of the last token that was read before this function was called. When the function exits, it contains the text of the token that has been most recently read, which is the first token that follows the Expr. The numeric code for this token is to be returned. The last argument, E, is the SymExpr object that stores the expression corresponding to the Expr that has been parsed. An Expr consists of a sequence of one or more T erms, connected by addition or subtraction operators. int Term( std::istream& in, int token, std::string& tok str, SymExpr& T ); This function recognizes a single T erm from text read from the input stream in. The second argument token is the code of the last token that was read. The third argument tok str is an input and output argument. Initially, it contains the text of the last token that was read before this function was called. When the function exits, it contains the text of the token that has been most recently read, which is the first token that follows the T erm. The numeric code for this token is to be returned. The last argument, T, is the SymExpr object that stores the expression corresponding to the T erm that has been parsed. A T erm consists of a sequence of one or more F actors, connected by multiplication or division operators. int Factor( std::istream& in, int token, std::string& tok str, SymExpr& F ); This function recognizes a single F actor from text read from the input stream in. The second argument token is the code of the last token that was read. The third argument tok str is an input and output argument. Initially, it contains the text of the last token that was read before this function was called. When the function exits, it contains the text of the token that has been most recently read, which is the first token that follows the F actor. The numeric code for this token is to be returned. The last argument, F, is the SymExpr object that stores the expression corresponding to the F actor that has been parsed. A F actor is either the negation of another F actor, or a Base raised to an exponent that is another F actor. int Base( std::istream& in, int token, std::string& tok str, SymExpr& B ); This function recognizes a single Base from text read from the input stream in. The second argument token is the code of the last token that was read. The third argument tok str is 5
an input and output argument. Initially, it contains the text of the last token that was read before this function was called. When the function exits, it contains the text of the token that has been most recently read, which is the first token that follows the Base. The numeric code for this token is to be returned. The last argument, B, is the SymExpr object that stores the expression corresponding to the Base that has been parsed. A Base is either a number, a variable, an array reference consisting of a variable and indices enclosed in square brackets, a function call consisting of a function name followed by a commaseparated argument list enclosed in parentheses, or a parenthesized Expr. If it is none of these, an exception must be thrown with the error message "base expression expected". If an Expr that is used as an array index is not followed by a ], than an exception must be thrown with the error message "] expected". If an argument in a function call is not followed by a comma or a ), then an exception must be thrown with the error message ", or ) expected". If the function name is not followed by a (, then an exception must be thrown with the error message "( expected". Finally, if the Base is a parenthesized Expr, then if the Expr is not followed by a ), then an exception must be thrown with the error message ") expected". 4.2 Exceptions The table below summarizes all of the errors that must be reported by functions in the parser. Message Situation "end-of-string expected" Stmt not followed by null terminator "invalid lhs" Expr on left side of = is not id or array reference "] expected" Expr used as array index not followed by ] ", or ) expected" Expr in Arglist not followed by, or ) "( expected" No ( after func ") expected" No ) after parenthesized Expr "base expression expected" Invalid token at beginning of Base 5 The SymExpr type This type is used to represent expressions in such a way that they can have operations performed on them, using recursion. A SymExpr object can represent a number, a variable, an array reference, an application of a function to another SymExpr object, or a sum, product, difference, quotient, exponentiation, or negation of other SymExpr objects. Because of the hierarchical structure of these expressions, we can easily implement operations on expressions by implementing them for each basic type of expression, and letting recursion handle the rest. 6
5.1 Structure Members The following members store the properties of an SymExpr object. int m type; This is the only member that is always needed. It indicates the type of the expression, using the values given in the description of CreateSymExpr (see below). int m symbol; If the SymExpr object represents a function call, this variable stores the numeric code of the function, using the values listed in symtab.h. double m num; If the SymExpr object represents a number, its value is stored in this variable. bool m ispar; This variable is set to true if the expression, when converted to a string by tostring(), is to be enclosed in parentheses. std::string m var; If the SymExpr object represents a variable, its name is stored in this member variable. std::vector<symexpr> m operands; If the SymExpr object represents an expression that has operands, they are stored in this vector. The order of the elements in the vector corresponds to the order in which the operands appear in the expression, from left to right. 5.2 Functions CreateSymExpr( int type = ETYPE NONE, int symbol = -1 ); This function initializes a SymExpr object with the given type. The allowed types, defined in sym expr.h, are as follows: 7
Type ETYPE NONE ETYPE NUM ETYPE VAR ETYPE ADD ETYPE SUB ETYPE MUL ETYPE DIV ETYPE EXP ETYPE NEG ETYPE FUNC ETYPE ASN ETYPE ARRAY Description Empty expression Number Variable Sum of expressions Difference of expressions Product of expressions Quotient of expressions Exponentiation of expressions Negation of expression Function applied to expression Assignment of expression to another Array reference The second argument, symbol, is only needed if the type is ETYPE FUNC. It indicates which function is to be applied, using its code as defined in symtab.h. CreateSymExpr( double num ); This function initializes a SymExpr object that is equal to the number num by setting its type accordingly and storing num inside the object. CreateSymExpr( const std::string& var ); This function initializes a SymExpr object that is equal to a variable named var by setting its type accordingly and storing var inside the object. SymExpr Add( const SymExpr& e1, const SymExpr& e2 ); This operator returns a new SymExpr object that represents the sum of e1 (the left operand) and e2 (the right operand). SymExpr Subtract( const SymExpr& e1, const SymExpr& e2 ); This operator returns a new SymExpr object that represents the difference of e1 (the left operand) and e2 (the right operand). SymExpr Multiply( const SymExpr& e1, const SymExpr& e2 ); This operator returns a new SymExpr object that represents the product of e1 (the left operand) and e2 (the right operand). SymExpr Divide( const SymExpr& e1, const SymExpr& e2 ); This operator returns a new SymExpr object that represents the quotient of e1 (the left operand) and e2 (the right operand). 8
SymExpr Power( const SymExpr& e1, const SymExpr& e2 ); This operator returns a new SymExpr object that represents the exponentiation of e1 (the base) and e2 (the exponent). SymExpr Negate( const SymExpr& e ); This operator returns a new SymExpr object that represents the negation of its operand, e. SymExpr ApplyFunc( int func, std::vector<symexpr>& arglist ); This function returns a new SymExpr object that represents a Base expression that is a function call the arguments stored in Arglist. The numeric code of the function is given by func. The arguments must be stored in the m operands vector of the new SymExpr object. SymExpr Assign( const SymExpr& e1, const SymExpr& e2 ); This function returns a new SymExpr object that represents an assignment Stmt, in which the left-hand side is represented by e1, and the right-hand side is represented by e2. Both expressions must be stored in the m operands vector of the new SymExpr object. SymExpr ArrayRef( const SymExpr& e, std::vector<symexpr>& arglist ); This function returns a new SymExpr object that represents a Base expression that is an array reference, where e is the array and arglist contains the indices. The variable represented by e and the indices stored in arglist must be stored in the m operands vector of the new SymExpr object. std::string tostring( const SymExpr& e ) const; This function returns a std::string describing the expression stored in e. If e is a variable, tostring( e ) simply returns the variable name. If e is a number, tostring( e ) simply returns a string containing the number s value. Most other expressions are handled by simply calling tostring() recursively on the operands of e and combining the resulting strings with the appropriate operators such as + or *, with the following exceptions: If e is a function call, and the function is not diff, then the string to be returned consists of the name of the function, followed by its arguments, separated by commas, and enclosed in parentheses. If e is a function call and the function is diff, then the string to be returned consists of the first argument, the expression to be differentiated, enclosed in square brackets, and preceded by a string describing the differentiation operator that is applied. If the differentiation is with respect to only a single variable, the differentiation operator is described by the string d/d followed by the name of that variable. If differentiating with respect to n variables where n > 1, the differentiation operator is described by the string 9
d n/ followed by each variable in the argument list, in reverse order, preceded by a d. For example, the string returned for the expression diff(sin(x),x) is d/dx[sin(x)], and the string returned for the expression diff(x 2+y 2,x,y) is d 2/dydx[x 2+y 2]. If e is an array reference, the string to be returned consists of the variable name, which is the first operand of e, followed by each of the other operands, which are the indices, with each operand enclosed in square brackets. If e is marked as being parenthesized, then the resulting string must be enclosed in parentheses before it is returned. void Parenthesize( const SymExpr& e ); This function, which is already implemented, is used to indicate that the expression in the current object, when converted to a string by tostring(), is to be enclosed in parentheses. This function must be called when creating the SymExpr object from an Expr that is enclosed in parentheses in the original text. It must also be called for various reasons when combining expressions; Section 6 discusses these scenarios. bool isempty( const SymExpr& e ); This function, which is already implemented, returns true if e does not contain an expression; that is, its type is ETYPE NONE. bool IsZero( const SymExpr& e ); This function, which is already implemented, returns true if e represents the number zero. bool IsOne( const SymExpr& e ); This function, which is already implemented, returns true if e represents the number one. bool IsNum( const SymExpr& e ); This function, which is already implemented, returns true if e represents a number. bool IsVar( const SymExpr& e ); This function, which is already implemented, returns true if e represents a Variable. bool IsArray( const SymExpr& e ); This function, which is already implemented, returns true if e represents an array reference. std::ostream& operator<<( std::ostream& out, const SymExpr& e ); This operator uses tostring() to write an expression to the output stream out. It does not need to be implemented. 10
6 Parenthesizing The following simple rules will help ensure that expressions are properly parenthesized, without introducing unnnecessary parentheses. If an expression is required by the grammar to be enclosed in parentheses, then the SymExpr object representing that expression should be parenthesized by calling the Parenthesize() member function. When building a SymExpr object from other SymExpr objects that serve as operands, parenthesize the operands using the Parenthesize() member function if their m type corresponds to an operator of lower precedence than the operator that corresponds to the value of m type in the newly built SymExpr object. For example, in Multiply, a SymExpr object of type ETYPE MUL is being built from two operands. If either of those operands is of type ETYPE ADD or ETYPE SUB, then they should be parenthesized. If the denominator of a newly formed quotient is of type ETYPE MUL or ETYPE DIV, then it should be parenthesized using the Parenthesize() member function. Similarly, if the right operand of a subtraction is of type ETYPE ADD or ETYPE SUB, then it should be parenthesized. This is due to the left associativity, and non-commutativity, of subtraction and division. If the base of a newly formed exponentiation is of type ETYPE EXP, then it should be parenthesized. This is due to the right associativity, and non-commutativity, of exponentiation. The types of SymExpr objects, listed in order of precedence, from highest to lowest, are 1. ETYPE FUNC, ETYPE NUM, ETYPE VAR, ETYPE ARRAY 2. ETYPE ExP 3. ETYPE NEG 4. ETYPE MUL, ETYPE DIV 5. ETYPE ADD, ETYPE SUB 6. ETYPE ASN Note that the higher an operator s precedence, the lower its level in the hierarchy of expressions described by the grammar. Based on these rules, numbers, variables, function calls, and array references never need to be parenthesized, because they have the highest precedence. 11
7 The Driver The main function is implemented for you, and serves as a driver. This means that it s only intended to aid in testing your parsing code; it does not actually do anything with the expressions that are parsed, except to display them. It also catches exceptions, so that parsing can continue. It is worth noting how it processes input. It uses getline to read a line from std::cin into a std::string, and then associates a std::istringstream with this string. This stream is then passed to Parse, so your code is only processing a single line of text. When running your program, you will test your code by entering expressions, one per line. The syntax requires that each line consists of only a single statement and nothing else. Your Parse function should create an empty SymExpr object before calling Stmt, so that if there is no input (i.e., the std::istringstream object is associated with an empty string, or a string containing only white space), Parse will return a SymExpr object of ETYPE NONE, and then main will exit. Therefore, you can end your program simply by hitting enter twice in a row after your last statement. 8 Suggested Approach The code from the Lecture 15 example eval expr contains code that is very similar to that which you will need to write for parsing, so that should be used as a starting point. Then, it is recommended that you build your project in stages: 1. When writing the parsing functions, begin with simpler, lower-level expressions like numbers and variables, and then proceeding to higher-level expressions like products and sums. You don t have to implement every production before you can test anything. For example, you can begin by only implementing the productions Stmt Expr, Expr T erm, T erm F actor, and then implementing Base. This corresponds to simply having Stmt call Expr, Expr call Term, Term call Factor, and Factor call Base. You can fill in the other productions for Stmt, Expr, T erm, and F actor later. 2. Implement the tostring() function as soon as possible, at least for the simplest expressions, so that SymExpr objects can be output at any time, to aid with debugging. 9 Testing Your Code In the file main.cpp, you can write any code you like to test your member functions. However, make sure you test them thoroughly, because when we grade your project, we will replace your main.cpp with one of our own and expect your code to work! Be sure to test both valid and invalid input, because we will too. 12
10 Submission To submit your project, you must do the following: 1. Transfer your source files to your account on elaine.stanford.edu. This can be done using SecureFX on Windows, or Fetch on Mac OS X. 2. Place all of your source files and the Makefile in the same directory. 3. Use the make command to build the executable project2, so that you can ensure that it runs on elaine. 4. Log in to elaine using an SSH client and execute the following command from the directory containing your code and Makefile: /usr/class/energy211/submit 4 This command runs a script that will build your executable, even if you didn t earlier, to make sure that your code compiles. If it does, then it will submit your source files for grading. 11 Grading Your grade will be determined using the following criteria: Correctness (70%) Your code must be able to parse valid expressions and display them as prescribed. Errors must be reported in the situations described in Section 4.2. When an error is reported, the remainder of the current expression must be skipped, and the next expression is read and parsed. Efficiency (10%) Your code must implement the operations with reasonable efficiency. Points may be deducted for unnecessary operations, particularly comparisons or data movements. Style (10%) Sound coding practices should be observed, or points may be deducted. Avoid the use of global variables, and avoid declaring variables outside of the scope in which they are used. Write modular code, using functions to perform individual tasks. Documentation (10%) Your code must be documented. In each function that you implement, add comments to explain what you are doing so that your code can be readily understood by a reader. Points may be deducted for code that is difficult to decipher. 13