KU Compilerbau - Programming Assignment

Transcription

1 KU Compilerbau - Programming Assignment Dr. Birgit Hofer, Univ.-Prof. Dr. Franz Wotawa Institute for Software Technology, Graz University of Technology March 11, 2014 Introduction During this semester you will build a simple Calculator compiler in Java using ANTLR 3. Calculator knows basic arithmetic as well as if and for statements and function calls, please see Listing 1 for an example program written in this language. Your compiler should be able to translate a program written in the Calculator grammar to Java bytecode. Figure 1(b) illustrates the different phases of a compiler. The phases which will be part of your exercise are highlighted in Grey. (a) Language processing system (b) Compiler phases Figure 1: Compiler 1

2 Listing 1: Example Calculator program 1 int main ( ) { 2 int n, r e t v a l ; 3 n = 5 ; 4 r e t v a l = print ( n th f i b o n a c c i number : ) ; 5 return print ( f i b o ( n ) ) ; 6 } 7 int f i b o ( int n ) { 8 int a, b, tmp, i, r e t v a l ; 9 n = n 2 ; a = 1 ; b = 1 ; 10 for ( i = 0 ; i < n ; i = i + 1) { 11 tmp = a + b ; 12 a = b ; 13 b = tmp ; 14 } 15 return b ; 16 } Schedule The following table provides an overview of your tasks and the individual delivery deadlines. Task Points Deadline Question Time 1 Lexical and Syntax Analysis 17.5 April, 2 nd March, 31 st 2 Type Checking 17.5 May, 7 th May, 5 th 3 Intermediate Code 17.5 May, 28 th May, 26 th 4 Code Generation 17.5 June, 18 th June, 16 th General rules You have to deliver tasks 1-4 via SVN. Deliveries are mandatory, as further tasks build upon your previous solutions. If you fail to solve one of the tasks, you may request a sample solution to build the further tasks upon. Please note that there are multiple valid approaches for most tasks and the approach chosen for the sample solution may differ significantly from your previously chosen approach. You must document the percentage of participation of each team member for each task in the README file. For tasks 1, 2, and 3 you will receive a notification about your score and comments regarding your solution via . In case you have any questions regarding the points or comments, make use of the office hour (Wednesday, 1 pm) as well as the appointed question time. For general questions, please use the newsgroup (tu-graz.lv.cb) and for personal questions you may reach us via the mailing list (cb@ist.tugraz.at). There is a mandatory interview after completion of task 4, where you will receive your final score. Group Registration Groups of up to four students are allowed. Due to the effort we recommend a size of four students. In order to get access to a SVN repository you have to register your group using the Web-Interface. The link to the Web-Interface will be posted in the newsgroup. 2

3 File/folder hierarchy Your SVN repository must have a folder for each task. The folders must be named task {number}, where {number} is replaced by the actual task number. Each folder must contain the following structure: (Please take care that the required documents are in the correct folders!) task {number}/ build.xml readme.txt lib/ src/ test/ readme.txt junit-4.10.jar antlr-3.5-complete.jar jasmin.jar **/*.java Calculator.g The file readme.txt must contain the table with the percentage of participation all changes made (e.g. bug correction) with respect to the previous tasks in clear and short sentences known limitations / bugs implemented additional tasks build.xml The file build.xml must define an ant task compile which compiles the grammar-file to Java files and compiles all Java files from src. You can use the BUILD file from the framework. It defines an additional ant task run-junit which runs all JUnit tests and creates an xml file in the output folder. Make use of this task for testing purposes! Please note that you have to adopt your build file if you use for example an additional grammar (.g) file. Framework You can download a simple framework from the course web site. This framework contains all required libraries (ANTLR, JUnit, Jasmin), skeletons of the required classes, JUnit test files, example input and output files, an example README file and an example BUILD file. Upload these files to your repository, modify and extend them. 3

4 1 Lexical and Syntax Analysis Tasks 1. Write a lexical and syntactical analyzer for the Calculator language described below in Java with ANTLR version 3.5. You have to rewrite the provided grammar into LL(1) using the methods described in the lecture. Modify the provided grammar file Calculator.g in the framework to contain your solution. Do not modify the header with the options of the grammar file, as they are set to only compile LL(1) grammars (backtrack = false;k = 1;). Implement the following methods of the class LexicalAndSyntaxAnalyzer.java in the package at.tugraz.ist.cc in the directory src: public int lexer(string file path, boolean debug) This method returns 0 if the lexical analysis was successful, otherwise it returns the number of errors found. Write a summary of errors you have found to the standard output. This is how the output could look like: Number of lexical errors: 3 #1: line 6:2 no viable alternative at character ï 1 2 #2: line 6:3 no viable alternative at character ï 1 2 #3: line 7:1 no viable alternative at character # If the flag debug is set write the input program abstracted to lexemes and keywords to the standard output. You can find an example input and output in the framework. Note: This output should help you when you are debugging your compiler. (5 points) public int checksyntax(string file path, boolean debug) This method returns 0 if the syntax analysis was successful, otherwise it returns the number of errors found. If there exist lexical errors, no syntactical analysis has to be performed. Instead the number of lexical errors should be returned. Write a summary of the syntactical errors to the standard output. This is how the output could look like: Number of parser errors: 2 #1: line 2:0 mismatched input ; expecting return #2: line 3:0 missing } at int If the flag debug is set write the input program abstracted to function definitions to the standard output. It should contain a line for every definition of a function/procedure (name + signature) an output for the end of every function/procedure line numbers You can find example input and output in the framework. (9 points) 2. Create your own example programs - at least one error-free program, one program which leads to lexical errors and one program which leads to syntactical errors - and add them to the test-folder. Extend the JUnit test to test those programs.(3 points) 3. Create the README file as described above. (0.5 points) 4. Adapt the existing BUILD file or create your own BUILD file. (no points, but mandatory) Hints It makes sense to build an abstract syntax tree, since you need it for the next task. Be careful to properly handle left and right associativity and precedence: All used binary operators are left-associative: a + b + c is equal to (a + b) + c. Operators abide to the following precedence: NOT / unary SIGN > MULOP > SIGN > RELOP > AND > OR > ASSIGNOP Whereby NOT and unary SIGN will be resolved first and ASSIGNOP last. 4

5 Bonus Task You are allowed to extend the given grammar, e.g., you can add in-line comments. If you extend the language, please make sure that your grammar stays downward compatible, and document your changes in the README file. Make suggestions for bonus tasks in the newsgroup. We will answer to your posts if these extensions make sense and how many bonus points you can get for implementing them. The Grammar Operators ASSIGNOP = OR AND && RELOP < <= > >= ==! = SIGN + - MULOP * / % NOT! fragment OPERATORS < > = + / % & Numerical literals INT 0 DIGIT DIGIT0* fragment DIGIT [1-9] fragment DIGIT0 [0-9] Boolean literals BOOLEAN true false Identifiers ID LETTER ( LETTER DIGIT0 )* fragment LETTER [a-za-z] Punctuation PUNCT., ; :! White spaces WS \n \t \r String literals LITERAL ( [ 0-9a-zA-Z ] \\ OPERATORS PUNCT WS )* Terminals are displayed in green, lexemes in uppercase and parser rules in lower case letters. White spaces between tokens are optional, with one exception: keywords must be surrounded by white spaces, newlines or the beginning of the program. 5

6 Syntax program functions type int boolean String functions function declaration functions ɛ function declaration function head function body function head type ID arguments arguments ( parameter list ) ( ) parameter list type ID parameter list, type ID function body { declarations optional stmt return stmt } declarations declarations type idlist; ɛ idlist ID idlist, ID optional stmt stmt list ɛ stmt list statement stmt list statement return stmt return expression ; statement compound stmt if(expression) statement else statement for(assignment; expression; assignment) statement assignment ; compound stmt { optional stmt } assignment ID ASSIGNOP expression expression expression OR factor expression AND factor expression RELOP factor expression SIGN factor expression MULOP factor factor factor ID INT BOOLEAN LITERAL NOT factor SIGN factor function call ( expression ) function call ID ( extend assign expr list ) ID ( ) extend assign expr list expression extend assign expr list, expression 6

7 2 Type Checking Tasks 1. Implement the method public int checktypes(string file path, boolean debug) within the class TypeChecker.java in the package at.tugraz.ist.cc in the directory src. This method returns 0 if the type checking analysis was successful. Otherwise, it returns the number of errors found. If there exist lexical or syntactical errors, no type checking has to be performed. Instead, the number of lexical or syntactical errors should be returned. Write a summary of the type errors you have found to the standard output. The output should contain: Use of undeclared identifiers. This includes access of undefined variables as well as calls to undefined functions. Use of incorrect operand types. Double declarations of variables and functions. This is how the output could look like: Number of type checking errors: 2 #1: Usage of undefined identifier x (line 3) #2: Call to undefined function test (line 26) You do not need to implement statement-level error recovery. Thus, the remainder of the statement can be skipped after discovery of a type error within it. If the flag debug is set write all type coercions to the standard output. Your output should contain: Definitions of variables, including type and scope The type of any operator All type coercions Keep your output readable, e.g., in the form of tables like these: Variable definitions: name type scope line a int int main () 9 b int int main () 10 a int int wrongreturntype (int a) 1 b boolean int wrongreturntype (int a) 2 c int int wrongreturntype (int a) 3 Operators: operator left right result line operator return int int int 11 return b / int boolean int 4 2 / b - int int int int int int 4 b - 4 = int int int 4 a = 1 return int boolean int 5 return b 7

8 Type Coercions: from to line operator boolean int 4 2 / b boolean int 5 return b Note, that this example does not prescribe the exact form of your output. (12 points) 2. Enable support of predefined print functions. See section: Print. (3 points) 3. Create your own example programs - at least one type-correct program and one typeincorrect program - and add them to the test-folder. Extend the JUnit test to test those programs.(2 points) 4. Adapt the README file. (0.5 points) 5. Adapt your BUILD file if necessary. (no points, but mandatory) Variables and Scopes Note that there is only one scope type: the scope of each function. A global scope as well as nested scoping is not supported by the grammar. It is allowed to declare a variable with the same name in each local scope. But it is an error to declare a variable more than once in any given scope. It is also not allowed to define a variable of the same name as a function parameter of the current function. Function declarations Multiple declarations of the same function name, expecting a different number of parameters or varying parameter types should be supported. Whereas multiple declarations using the exact same combination of these factors should be reported as an error. Note that the return type does not need to be taken into consideration in this case. Implicit function declarations do not need to be supported. For this programming assignment, you can assume the following: If a function call to an undefined function occurs, this function will not be declared at a later point in the program and can be treated as an error. Type Checking Type checking involves the task of determining the validity of each combination of operator and operands occuring in the source program. Depending on the types of the operator and the supplied operands, either the expression is deemed valid, a typechecking error is issued, or a type coercion occurs, ensuring the validity of the expression. During a coercion, the type of an operand is automatically converted to the type expected by the operator. Typechecking reference by operator tokens: RELOP: Expected type: Coerced type: Rejected type: Result type: int boolean String boolean 8

9 SIGN, MULOP: Expected type: Coerced type: Rejected type: Result type: int boolean String int OR, AND, NOT: Expected type: Coerced type: Rejected type: Result type: boolean int String int ASSIGNOP (and return statements): Valid assignments: Coerced assignments: Rejected assignments: Result type: int = int; boolean = boolean; String = String; int = boolean; boolean = int; int = String; boolean = String; String = int; String = boolean; type of left operand (or return type of function) Assignments and return statements use the same set of coercion rules. Additionally, left associativity and operator precedence needs to be ensured as well: Print Every used binary operator is left-associative: a + b + c is equal to (a + b) + c. Thus, left hand operators within a chain of operators of the same precedence shall be computed first. Operators abide to the following precedence: NOT / unary SIGN > MULOP > SIGN > RELOP > AND > OR > ASSIGNOP Whereby NOT and unary SIGN will be resolved first and ASSIGNOP last. We want to test your compiler after Task 4 Code Generation as to whether it computes the correct results. To that end, we need a built-in mechanism for output: the print function. Since you are translating Calculator programs to Java byte code you have to use the I/O functions available in byte code. Thus, in Task 4 you will have to map the function calls to print to the corresponding byte code statements. At present, you only have to warrant that print is a predefined function and that a function call to print will not lead to an error. The exact signatures you will need to add are Hints int print (String) int print (int) int print (boolean) Create data structures to hold the type information for the identifiers. You will have (at least) int, boolean, String, and functions with argument- and return types. Implement the code to fill the data structures, and to perform type checking. The use of additional grammar files (e.g. TreeWalker.g) is recommended, but not mandatory. 9

10 Suggested Bonus Tasks Implementation of nested scopes (2 points) Implementation of forward declarations (2 points) Implementation of statement-level error recovery (2 points) Extend the type system to include more types. (points depend on your exact plans) If you intend to alter your grammar in favour of a bonus task, be sure to ascertain that the basic Calculator language is still matched. Don t forget to clearly document the implemented bonus tasks in the README file and add JUnit test cases, demonstrating the implemented features. 10

11 3 Intermediate Code Tasks 1. Build an Intermediate Code Generator which is able to build the intermediate code (Three- Address Code) for any program written in our grammar. Create the class IntermediateCode.java in the package at.tugraz.ist.cc with the method public int createintermediatecode(string file path, OutputStream stream, boolean debug). This method returns 0 if the intermediate code generation was successful. Otherwise, no intermediate code processing has to be done, and it returns the number of (lexical, syntactical or type) errors found. In addition, it writes the ASCII representation of the intermediate code to the given output stream. Use comments to clarify which part of the Calculator code corresponds to the intermediate code. If the debug-flag is not set, no comments should be printed to the stream. You can find an example input and output in the framework. (17 points) 2. Adapt the README file. (0.5 points) 3. Adapt your BUILD file if necessary. (no points, but mandatory) Three-Address-Code The three address code should support the following commands: Assignments of the form x := y op z, where x is a temporary variable and op is one of int+, int-, int*, int/, int%, which take two integers and return the corresponding result. int&&, int, which take two integers and return 0 or 1. int<, int<=, int>=, int>, int==, int!= which take two integers and return 0 or 1. Assignments of the form x := op y, where x is a temporary variable and op is one of intun- int! and y is a variable. op is one of intconst,stringconst, and y is an integer constant or a constant string, respectively. Copy statements: x := y. Jumps The label label L. The unconditional jump goto L, where L is a label Conditional jump ifint x goto L, where x is an int and L a label. Function Calls The statement param x, which says that x is a parameter to the next function call. Parameters should be listed from left to right. The statement call f rettype typelist, where f is the name of a function, rettype is the return type of the function and typelist is a list of all parameter types starting from left to right. The statement function f n rettype typelist, where f is a function, and n is the number of local variables including parameters and temporary variables. The rettype and typelist are the same as in the function call. The statement return x, which returns from the function, storing the value in x. The statement local x, where x is a local variable, a temporary variable, or a parameter. Start by listing the parameters from left to right, as seen in the example. 11

12 The statement getresult x, which stores the return value of the last function call in x. The statement comment x, which is ignored but should be used for debugging purposes. This list might be incomplete. Your are allowed to add additional commands. Post your additions to the newsgroup so that others can benefit from them. Don t forget to document your extensions in the README file. You need not to optimize your code concerning time or space. IF Statement Example (comments are for easier understanding only): To implement if statements, you will need to utilize jumps and customly added labels to jump to. One label will need to be added at the beginning of the else-part and one at the end of the whole statement. If the condition of the statement computes to true, skip the first jump, process the then-part and jump to the end-label. Otherwise jump to the beginning of the else-part and just continue there. 1 i f ( a == b ) 2 c = 1 0 ; 3 else 4 c = 2 0 ; comment --- cond --- comment Operator == [line: 5, col: 7, context: "( a ###==### b ) "] _t1 := a int== b comment inverting if condition _t2 := intnot _t1 comment conditional jump to else part ifint _t2 goto _L1 comment --- then comment ---END then --- comment jump to end of if goto _L2 comment label for else part label _L1 comment --- else comment --- END else --- comment label for end of if label _L2 comment #### -- IF END -- #### FOR Statement Example (comments are for easier understanding only): To implement for statements, you will, again, need to utilize jumps and customly added labels to jump to. One label will need to be added at the beginning of the condition evaluation part and one at the end of the whole statement. The initialization part should be put at the beginning, followed by the condition evaluation. If the condition of the statement computes to false, jump to the end of the statement. Otherwise, skip the jump, process the body- and update-parts and finally jump back to the condition evaluation. 1 for ( a = 0 ; a < b ; a = a + 1) 2 b = a 2 4 ; 12

13 comment -- begin init code FOR comment -- end init code FOR --- comment label for FOR loop label _L1 comment -- begin eval of condition FOR --- _t2 := a int< b comment -- end eval of condition FOR --- comment inverting condition _t3 := intnot _t2 comment jump to end of for if condition broken ifint _t3 goto _L2 comment -- begin body of FOR comment -- end body of FOR --- comment -- begin update of FOR comment -- end update of FOR --- comment jump to begin of loop goto _L1 comment label for end of for label _L2 comment #### END FOR #### Boolean values All boolean types are converted to integers in the Three-Address-Code. Truth values are represented by integers: 0 is false, anything else is true. A truth value generated by a comparison, &&, or may only be 0 or 1. You can assume, that this conversion will not lead to conflicting function definitions. Hints Create an inheritance hierarchy, with the class Statement on top. Other Statements inherit from this class. Your intermediate code program is a vector of Statements, each Statement can print itself. Later, each Statement will know how to translate itself to e.g. assembler code or Java bytecode. Use an attributed grammar on the abstract syntax tree to generate the code. A function node, for example, should get the intermediate code from its children (declarations, statements), plus the number of temp variables, so that it can construct the function name number statement. The intermediate code for the function node consists of this statement and the intermediate code for the body. For the comments, we display the Token context. You may get this by retaining the Tokenindex for each element and looking it (and its surrounding tokens) up in the CommonTokenStream from the Lexer. Bonus Task Implementation of short-circuit evaluation for Boolean expressions (2 points) Don t forget to clearly document the implemented bonus task in the README file. 13

14 4 Code Generation Tasks 1. Implement the translation of the intermediate code to Java bytecode. Implement the method public int createcode(string file path) of the class CodeGeneration.java in the package at.tugraz.ist.cc in the directory src. This method creates the output file <file>.j, where <file>.calc is the name of the input file. No code generation needs to be processed of either lexical, syntactical or type errors have been found. If this is the case, return number of reported errors. The method returns 0 if the code generation was successful. It must be possible to translate the generated output files to byte code by using the provided Jasmin binary. The files, generated by Jasmin based on your output, must be executable by the Java Virtual machine. In addition, calls to the print functions must work as intended. (17 points) 2. Adapt the README file. (0.5 points) 3. Adapt your BUILD file if necessary. (no points, but mandatory) Jasmin Jasmin is an assembler-language which produces Java bytecode. A small tutorial (description of how Jasmin operates and the statements you will need) and additional helpful links are available on the course web page. Convert a Jasmin *.j file into *.class Java bytecode: java -jar <path to jasmin.jar> <source.j> Execute the created source.class file: java <source> Example of a conversion Input: simple.calc 1 int main ( ) { 2 S t r i n g a ; 3 a = t e s t ; 4 return t e s t ( a ) ; 5 } 6 7 int t e s t ( S t r i n g a ) { 8 return print ( a ) ; 9 } Result: simple.j 1. s o u r c e simple. c a l c 2. class public s t a t i c simple 3. super java / lang / Object 4 5. method public s t a t i c main ( [ Ljava / lang / S t r i n g ; )V 6. l i m i t s t a c k l i m i t l o c a l s 4 8 l d c t e s t 9 a s t o r e 1 10 aload 1 11 a s t o r e 0 12 aload 0 13 i n v o k e s t a t i c simple. t e s t ( Ljava / lang / S t r i n g ; ) I 14 i s t o r e 2 14

15 15 i l o a d 2 16 return 17. end method method public s t a t i c t e s t ( Ljava / lang / S t r i n g ; ) I 20. l i m i t s t a c k l i m i t l o c a l s 2 22 aload 0 23 g e t s t a t i c java / lang /System/ out Ljava / i o / PrintStream ; 24 swap 25 i n v o k e v i r t u a l java / i o / PrintStream / p r i n t l n ( Ljava / lang / S t r i n g ; )V 26 bipush 0 27 i s t o r e 1 28 i l o a d 1 29 i r e t u r n 30. end method Main As we need to test your resulting bytecode assemblies, for this task each of our.calc testfiles will contain a main function with the signature int main(). You need to convert the signature of this function to the Java standard main method signature: public static void main(string[] args) within your assembly. Print Calls to the predefined print functions will need to be redirected as well. A call to int print(int) should invoke the method: void java.io.printstream.println(int) A call to int print(string) should invoke the method: void java.io.printstream.println(string) Calls to int print(boolean) will have been converted to int print(int) within the intermediate code. Both calls will need to be invoked using the invokevirtual Jasmin-statement after pushing a reference to java/lang/system/out and the desired parameter onto the stack. As the Java implementation of println(var) does not provide a return value you will need to provide a workaround in your assembly which supports each call to this function with a default return value of 0. Hints Comments within your assembly relating the assembly statements to the intermediate code can be useful for debugging. It will not be sufficient to generate Jasmin statements solely based on the current intermediate code statement. Information like the currently processed function, the result type of the current function and more will need to be memorized. Jasmin does not directly provide relation operators like > or ==. Instead you will have to utilize conditional jump statements like if icmpgt < label name > and if icmpeq < label name > in combination with custom labels to jump to and commands to put the desired outcome onto the operand stack. 15

16 Bonus Tasks Implementation of String read() which reads a String from the standard input. (1 points) Implementation of int read() which reads an integer from the standard input. (2 points) Implementation of int printf(string, int) which prints an integer using a format string to the standard output. (2 points) Support int-return of main function (1 points) Don t forget to clearly document the implemented bonus tasks in the README file. 16