Jim Lambers ENERGY 211 / CME 211 Autumn Quarter Programming Project 2

Transcription

1 Jim Lambers ENERGY 211 / CME 211 Autumn Quarter Programming Project 2 This project is due at 11:59pm on Friday, October Introduction In this project, you will implement functions in order to recognizes identifiers (i.e., variable or function names), floating-point numbers, arithmetic operators and various punctuation in text. This code will serve as the foundation for an application that will eventually be able to recognize and work with mathematical expressions. 2 Files Before beginning this project, you will need to download the source and header files from the Assignments page on the course web site: scan expr.h contains the declarations of the functions that are used to recognize individual components of a mathematical expression from text. You do not need to edit this file. scan expr.cpp will contain the implementation of the functions declared in scan expr.h. This where you will do all of your work. main.cpp, the main application file in which the code in the other source files is to be tested. Makefile, for compiling your code on UNIX/Linux/Mac OS X. You do not need to edit this file. input2.txt, sample input to help you test your program. output2.txt, the output that should be generated by running your program on input2.txt. project2.msp, an MDS project file for this project (Windows Zip file only). 3 Expression Language This section describes the tokens in the language of expressions that your program will recognize. Think of this project as grouping letters (characters) into words (tokens). Project 4 will build on this project, and focus on grouping words into sentences (expressions). In Project 6, these expressions will be given meaning through code that will be able to evaluate or manipulate them. 1

2 3.1 Tokens The words and punctuation, called tokens, of our expression language are described in the table below. Each token is defined using a regular expression over the alphabet, which is the set of permitted characters (letters, digits, underscores, arithmetic operators, parentheses, square brackets, comma, and the equal sign). A regular expression is one of the following: A character from the alphabet ab, where a and b are regular expressions, representing the concatenation of a and b a b, where a and b are regular expressions, representing the selection of a or b a, where a is a regular expression, representing zero or more occurrences of a a?, where a is a regular expression, representing zero or one occurrences of a (that is, a is optional) a+, where a is a regular expression, representing one or more occurrences of a Parentheses can also be used to indicate how regular expressions are built up from simpler expressions. For example, (a b) is interpreted as zero or more occurrences of either a or b, so any sequence of a s and b s, including the empty string, can be described using this regular expression. In the table below, we use the following regular expressions: digit is the regular expression ( ) letter is the regular expression (A B Z a b z) Each token is associated with an integer constant, or code, used to identify it for the purposes of syntax analysis, or parsing. The constants are defined in scan expr.h using #define preprocessor directives. 2

3 Code Regular Expression Description TOKEN ADD + Addition operator TOKEN SUB - Subtraction/negation operator TOKEN MUL * Multiplication operator TOKEN DIV / Division operator TOKEN EXP Exponentiation operator TOKEN LPAR ( Left parenthesis TOKEN RPAR ) Right parenthesis TOKEN EOS ASCII code 1 End-of-file character TOKEN NUM (digit + (.digit )?) (.digit+) num (floating-point number): Zero or more digits, followed by optional decimal point and zero or more digits (must contain at least one digit) TOKEN ID (letter )(letter digit )* id (identifier): Letter or underscore, followed by zero or more letters, digits or underscores TOKEN FUNC (letter )(letter digit )* func (function name): same definition as id TOKEN ASN = Assignment operator TOKEN LSQ [ Left square bracket TOKEN RSQ ] Right square bracket TOKEN COMMA, Comma 3.2 State Transition Diagrams To write a program that matches incoming text against a set of regular expressions like the ones in the preceding table, we can design a state transition diagram. A state transition diagram consists of a set of states, which are numbered, and transitions between states, which are associated with characters. One state, numbered 0, is the initial state, which corresponds to the beginning of the process of reading a single token. Once a character is read, we make a transition to another state, where the choice of state depends on the character. This process of reading characters and making transitions continues until a final state is reached, which signifies that a complete token has been recognized. Then, we return to the initial state to begin recognizing the next token. A state transition diagram can be implemented using a switch statement, where each case corresponds to a state. In the code for each state, we examine the next character and determine the correct transition. For example, suppose that we are at the initial state, state 0. If the first character is a letter or an underscore, we make a transition to another state, which we will call state 1. At state 1, we read another character, and if it is a letter, digit or an underscore, we remain in 3

4 state 1. If it is any other character, we make a transition to a final state, at which we place that unwanted character back in the input stream (since it is the first character in the next token), and recognize a id. In the example code from Lecture 6, each return statement can be viewed as a final state; these states are not assigned numbers in order to reduce the amount of code. Also, in this code, and in this project, white space, meaning spaces, tabs and newlines, are to be ignored. If such a character is read while in state 0, the code must remain in state The Symbol Table Note that in the description of the tokens, ids and funcs have the same regular expression, so there is no way to distinguish them from the text alone. To get around this, we maintain a symbol table, which is a map that associates symbols in a program with their attributes. Symbol tables are used by compilers to store information about all of the names that occur in a program, such as the types of variables. They can also be used to easily distinguish keywords such as if from variable names. In this project, we will use a symbol table, which is of type const char *[], an array of const C-style strings. This array, called Keywords and defined in scan expr.cpp, contains the names of all of the functions. When a name is recognized from the text, we look it up in the symbol table. If it is found, then we know that its token code is TOKEN FUNC. Otherwise, it is TOKEN ID. 4 Functions to Implement You are to write the bodies of the following functions: int nexttoken( std::istream& in, std::string& tok str ); This function is the main function of this class. It reads characters from the input stream in, which could be cin, a istringstream, or any other input stream. It then recognizes a token from those characters, and returns the corresponding numeric code. A string containing the token is returned through the reference parameter tok str. This parameter only needs to be assigned a value if the token is a num, id or func. If the characters do not form a valid token, an exception must be thrown (see below). int LookupWord( const string& tok str ); This function returns the value TOKEN FUNC if tok str is equal to any of the strings in the array Keywords that is defined in scan expr.cpp, and TOKEN ID otherwise. char nextchar( std::istream& in ); This function reads and returns the next character from the input stream in, or the standard EOF (end-of-file) character, with ASCII code 1, if no more input is available. 4

5 void retract( std::istream& in, char c ); This function places the character c back onto the input stream in, so that it will be the next character that is read. int main(); This function repeatedly calls nexttoken to read tokens from standard input until it returns the value TOKEN EOS, signifiying the end of the input (EOS means end-of-string ). For each token, the appropriate name of the token must be displayed, as listed in the token names array defined in main.cpp, and, in the case of num, id or func, the text of the token itself must also be displayed. Token names and text must be displayed in two columns, where the first column, containing the token name, must have a width of 25. You do not display the end-of-string token. Whenever an error occurs in nexttoken, an exception will be thrown. This exception must be caught in main. An appropriate error message must be displayed (see the next section), and then reading of tokens must continue. There is no need to call cin.clear() or perform extra reading operations to consume erroneous input, since we are only reading one character at a time anyway, as opposed to using >> to perform formatted input. 5 Exceptions The table below lists situations in which an error must be reported. To report an error, you must throw an exception. The type of the exception that must be thrown is std::runtime error. Its constructor takes a single argument, which is of type const char *, a C-style string. Message invalid token invalid number Situation First character of token does not belong to any valid token num consists of only a decimal point without digits For example, at the point in the code where the first error is detected, use the statement throw std::runtime error( "invalid token" ); In main, this exception is caught, and the error message that was passed to the constructor std::runtime error is printed. Then, reading of tokens must continue. 6 Testing Your Code In this project, unlike Project 1, we will not substitute our own main.cpp for yours. Instead, we will use all of your source files, and provide our own input. Therefore, be sure to test your code on a variety of tokens (both valid and invalid), because we will too. 5

6 7 Submission To submit your project, you must do the following: 1. Transfer your source files to your account on elaine.stanford.edu. This can be done using SecureFX on Windows, or Fetch on Mac OS X. 2. Place all of your source files and the Makefile in the same directory. 3. Use the make command to build the executable project2, so that you can ensure that it runs on elaine. 4. From this directory, execute the following command: /usr/class/energy211/submit 2 This command runs a script that will build your executable, even if you didn t earlier, to make sure that your code compiles. If it does, then it will submit your source files for grading. 8 Grading Your grade will be determined using the following criteria: Correctness (70%) Your code must be able correctly identify all valid tokens in the input, and report errors according to the specifications in Section 5 if any attempt to recognize a token fails. Reading of tokens must continue after an error. Efficiency (10%) Your code must implement the operations with reasonable efficiency. Points may be deducted for unnecessary comparisons or data movements. Style (10%) Sound coding practices should be observed, or points may be deducted. Avoid the use of global variables, and avoid declaring variables outside of the scope in which they are used. Write modular code, using functions to perform individual tasks. Documentation (10%) Your code must be documented. In each function that you implement, add comments to explain what you are doing so that your code can be readily understood by a reader. Points may be deducted for code that is difficult to decipher. 6