COMP 412, Lab 1: A Front End for ILOC Due date: Submit to: Friday, September 7, 2018 at 11:59 PM comp412code@rice.edu Please report suspected typographical errors to the class Piazza site. We will issue corrections as needed. This document is version 2. See the change log on the last page for details. Table of Contents 1. Introduction 1 2. Overview of the Problem 2 3. Code Specifications 3 4. Code Submission Requirements 6 5. Code Grading Rubric 7 6. Honor Code Policy 7 7. The ILOC Subset 8 8. Intermediate Representations 9 1. Introduction In this programming assignment, you will create a front end that is, a scanner and a parser for the intermediate representation, ILOC, that will be used as input in the next two assignments in COMP 412: a local register allocator and an instruction scheduler. Your program will take, as input, a program written in ILOC. It will determine whether or not the input is a syntactically correct ILOC program. The output that your front end produces will depend on the specific options specified on the command line that invoked it. In a general sense, the front end should check the input code for validity. If the input program contains errors, your front end should find as many errors as possible and report each of them to the user. If the input program does not contain errors, your front end should report that the input file was a valid ILOC program and it should build an internal representation for the input program that is suitable for use in the subsequent assignments. You will reuse the code that you write for this assignment in both the local register allocator and the instruction scheduler. Therefore, you should take care to make it correct, robust, and efficient. You do not want to spend time in the later assignments fixing work that you should have done in the first assignment. The work product of this assignment will be an archive, in the form of Unix tar file. The tar file will contain (1) the source code for your front end, (2) a makefile that performs any necessary compilation and linking, and (3) a README file that describes the contents of the tar file and how to use them. Grading is largely automated; thus, if these files do not conform to the specifications, the grading tools may not properly evaluate your submission.
Your front end will be tested and evaluated on Rice s CLEAR systems. Anything that you develop for this assignment or submit for this assignment should work on CLEAR. You are responsible for testing your code, makefile, and directions on CLEAR. The teaching assistants are not expected to fix incompatibilities in your submission. 2. Overview of the Problem You will build a front end that is, a scanner and a parser that will take, as input, a single basic-block 1 written in the ILOC subset described in 7. The scanner will break the input stream into a series of individual words. ILOC has a particularly simple syntax, and a small set of word types. Lecture 2 in the course discusses, specifically, how to scan ILOC. The scanner produced, as output, a stream of classified words represented as tokens: a pair <category,lexeme> where category is the kind of word and lexeme is the specific word. The scanner should be incremental; that is, the parser will call it on demand and it will produce the next word in the input stream. In the next two labs, you will test your code s performance (that is, the running time of your code) at scale. It is critical that your scanner and parser efficiently handle input programs with at least more than 100,000 lines of input. A batch scanner that is, one that scans the entire program before returning its first token is likely to break or run slowly on reasonably large files. The parser will examine the stream of tokens produced by the scanner, break them into operations, and check the syntax of each operation. Lecture 3 in the course discusses how to parse ILOC. The simple structure of ILOC allows the parser to perform detailed syntax checking based largely on the opcode for the operation. As the parser checks each operation, it will build a representation for that operation, for later use in subsequent compiler passes specifically, in your local register allocator and your instruction scheduler. Section 8 describes the representation that we recommend, based on our experience. You must decide the details of the representation used in your front end. Remember that your register allocator and instruction scheduler will re-use the code and data structures from this assignment, unless you choose to re-implement them. 3. Code Specifications The COMP 412 teaching assistants will use a script to grade Lab 1 code submissions. You must adhere to the following interface specifications to ensure that your front end works correctly with the script that the teaching assistants will use to grade it. Name: The executable version of your front end must be named 412fe. Behavior: Your front end must work in four distinct modes, as controlled by flags and parameters on the command line. Your front-end must check the correctness of the command-line arguments. It should respond to an incorrect command-line option by 1 A basic block is a maximal-length sequence of straight-line (i.e., branch-free) code. An ILOC basic block consists of a sequence of operations drawn from the set of operations shown in the table on page 8. COMP 412 2
printing a reasonable error message and then printing the list of valid options specified for the -h command-line flag. Output from 412fe must be printed to the standard output stream (stdout). All error messages must be printed to the standard error stream (stderr) Command-Line Syntax: Your implementation of 412fe must support the following command-line options: 412fe -h 412fe -s <file name> 412fe -p <file name> 412fe -r <file name> When a h flag is detected, 412fe must produce a list of valid command-line arguments that includes a description of all command-line arguments required for Lab 1 as well as any additional command-line arguments supported by your 412fe implementation. 412fe is not required to process command-line arguments that appear after the h flag. When a -s flag is detected, 412fe should read the file specified by <file name> and print, to the standard output stream, a list of the tokens that the scanner found. For each token, it should print the line number, the token s type (or syntactic category), and its spelling (or lexeme). When a -p flag is detected, 412fe should read the file specified by <file name>, scan it and parse it, build the intermediate representation, and report either success or report all of the errors that it finds in the input file. When a -r flag is detected, 412fe should read the file specified by <file name>, scan it, parse it, build the intermediate representation, and print out the information in the intermediate representation (in an appropriately human readable format). These four command-line flags are intended to be exclusive; that is, your front end only needs to handle one of these flags in any given invocation. It the user invokes your front end with multiple command-line flags, it should politely report the error and implement the highest priority flag. Priority is, from highest to lowest, -h, -r, -p, and -s. If no command-line flag is specified, the front end will behave as if it had received the flag -p. (That is, -p is the default behavior.) The reference implementation supports these same four flags. Input file: The input file specified in the command line will contain a single basic block of ILOC code, written in the subset described in 7 of this document. Your front end should scan and parse each line, checking them against the ILOC syntax specification. If your program cannot read the input file, or if it discovers that the code in the file is not valid ILOC, it should produce an informative error message that includes the line number in the input file where the error occurs and a brief description of the problem. COMP 412 3
The message should be sufficiently descriptive to allow a reasonable programmer to identify and fix the error. Your front end should find as many errors as possible in the input file; that is, it should continue scanning and parsing after it finds an error. Once it finds an error in a given line, it may skip to the end of that line to avoid multiple error messages for a single line. Scanning the input: COMP 412 is a course about the implementation of programming languages. Thus, you must write your own code to scan the input. You may not use formatted I/O, regular expression libraries, or other pattern-matching software, unless you write them yourself. Your scanner should process the input one character at a time. It may read one or more lines into a buffer and then process the buffer one character at a time. (See the discussion of double buffering in 2.5.3 of the textbook.) Character-by-character algorithms lend some clarity to the scanner. They can be significantly more efficient 2 than calls to generalized pattern-matching libraries or formatted I/O routines. Makefile & Shell Script: Lab 1 submissions written in languages that require a compilation step, such as C, C++, or Java, must include a makefile. 3 If your submission is written in a language that does not require compilation, such as Python, your submission does not need to include a makefile. The makefile must produce an executable named 412fe. If your submission is written in a language in which it is not possible to create a standalone executable and rename it to 412fe, such as Python or Java, then you must submit an executable shell script named 412fe that correctly accepts the required command-line arguments and invokes the program. For example, a front end written in Python and named lab1.py could include an executable shell script named 412fe that contained the following instructions: #!/bin/bash python lab1.py $@ Similarly, a project written Java with a jar file named lab1.jar or a class named lab1.class that contains the main function could provide, respectively, one of the following two executable shell scripts named 412alloc: #!/bin/bash java -jar lab1.jar $@ #!/bin/bash java lab1 $@ 2 Again, remember that your scanner and parser will be used in the next two assignments, where your code will be evaluated, in part, on the speed with which it processes successively larger files. Those programs are expected to handle large files correctly, gracefully, and efficiently. The timing tests for those labs include a file with 128,000 lines of ILOC. 3 If you prefer to use another build manager that is available on CLEAR to create your executable, you may invoke that build manager from your makefile. The makefile, however, is the interface that the grading scripts will expect and will test. COMP 412 4
Warning: There are significant differences between language implementations and tool implementations across different platforms, including Linux, Windows, and Mac OS X. Your code will be tested and graded on the CLEAR systems (log in to ssh.clear.rice.edu). In recent years, students have encountered a number of bugs that they found baffling when they ported code from their laptops to CLEAR. Examples include differences between language implementations, differences in handling end-of-line characters, and incompatibility in archive files and shell scripts. It is essential that you test your code on CLEAR early in the process, so that you can avoid the last-minute panic of discovering that your code does not work. Equally important, you should write a makefile early in the process and discover how to create and learn how to use tar files. Several students in prior years have missed the submission deadline because they could not construct a working makefile or build a correct tar file. To ensure that such a script is executable on a Linux system such as CLEAR, for either Python or Java, you must set its file permissions appropriately. In the directory where your script resides, execute the command: chmod a+x 412fe To avoid problems related to the translation of carriage return and line feed between Windows systems and Linux, we recommend that you write your shell script and makefile on CLEAR rather than on a Windows system. CLEAR: Your code and makefile/shell script must compile, run, execute, and produce correct output on Rice s CLEAR systems. CLEAR (Curricular Linux Environment @ Rice) provides a fairly standard Linux environment. Your netid should allow you to log into the CLEAR systems and use them. You can ssh to ssh.clear.rice.edu, which will connect you to one of the systems. This interface attempts to spread the load across the servers. The login notice will show the current load by server. (You can login to a specific server by specifying its name in your ssh command.) Programming Language: You may use any programming language available on CLEAR, except for Perl. Your goal should be to use a language that is available on CLEAR, in which you are comfortable programming, for which you have decent debugging tools, and with which you can reuse code in the second and third assignments. This freedom is not a call for language virtuosity; it is a bad idea to decide to learn a new language for your COMP 412 labs. If you do, and it does not work out well, you will be forced to reimplement your front end during the limited time allowed to build the local register allocator. If you decide to develop code on your laptop, as most students do, be sure to check that the version of the software on your laptop is the same as the version that is supported on CLEAR. Minor differences between implementations can cause last minute panics when a program that runs fine on your laptop produces incorrect results on CLEAR. The best practice is to regularly test your code in the environment where it will be graded, on CLEAR. README File: Your tar file must contain a README file that contains directions for building and invoking your program. (Note that Linux has a case-sensitive file system, COMP 412 5
so README must be in all capital letters.) Include a description of all command-line options that your code supports, including any that you added beyond the four required flags. To work with the grading scripts, the first two lines of your README file must be in the following format: //NAME: <your name> //NETID: <your NetID> Notice that that there are no spaces after the slashes, and that the keywords NAME and NETID are in all capital letters. Your name should be capitalized normally. Your netid should be all lowercase letters and numbers. 4. Code Submission Requirements Due Date: Your lab is due at 11:59 PM on Friday, September 7, 2018. Individual extensions to this deadline will not be granted. Early-Submission Bonus: Code received before the due date will be awarded a bonus of three points per day up to a maximum of six points. For example, to receive six points, you must submit your code by 11:59 PM on Wednesday, September 5, 2018. Late Penalty: Submissions received after the due date will lose three points per day. Late labs will be automatically accepted until 11:59 PM on Friday, September 14, 2018. Permission from the instructors is required to submit Lab 1 after this deadline. Contact both instructors by e-mail to request permission. Late Penalty Waivers: To cover potential illness, travel, and other conflicts, we will waive up to six days of late penalties per semester when computing your final COMP 412 grade. Submission Details: Create a tar file that contains your submission, including (1) the source code for your front end, (2) your makefile and/or shell script, (3) your README file, and (4) any other files that are needed for the TAs to build and test your code. If you not created a tar file previously, you should learn how to create one before the code submission deadline. You do not want to be reading the man page ten minutes before the due date. Note that a tar file is different than a zip archive. You must submit a tar file. If your front end does not work, include in your tar file a file named STATUS that contains a brief description of the current state of the passes in your front end (complete / incomplete / not implemented, working / broken, etc.). You may include ILOC files that your lab handles correctly and ILOC files that it handles incorrectly. If you include such files, describe them in both the README and the STATUS files. Name the tar file with your Rice NetID (e.g., jed12.tar for a student with the Rice NetID jed12). Email the tar file as an attachment to comp412code@rice.edu before 11:59 PM on the date. Use Lab 1 as the subject line of your e-mail submission. Note: comp412code@rice.edu should only be used for submitting code since it is only monitored during periods when code is due. Questions should be posted to the course discussion site on Piazza. COMP 412 6
5. Code Grading Rubric The grade for this assignment accounts for 14% of the points awarded in COMP 412. Grading will be based on a 100-point scale. Those points are allocated as follows. 20 points for adherence to the Lab 1 code specifications and the submission requirements. You will lose points if the grading script cannot unpack your tar file and run the front end without manual intervention by the graders. Possible problems include the README file, the tar file, the makefile and/or shell script, and problems in the actual code. (Again, test on CLEAR.) 80 points for correct handling of the input, including recognizing and reporting success, identifying errors and reporting them, and the quality of the error messages that your front end produces. 6. Honor Code Policy Your submitted source code for the front end and your README file must consist of code and/or text that you wrote, not edited or copied versions of code and/or text written by others or in collaboration with others. You may not look at COMP 412 labs written by other people in past semesters. You may not invoke the reference implementation, or any other front end that you did not write, from your submitted code. You are welcome to collaborate with current COMP 412 students when preparing your makefile and/or shell script. You can submit a makefile and /or shell script produced in such collaborations. However, as indicated in the previous paragraph, all other code and text that you submit must be your own work, not the result of a collaborative effort. You are welcome to discuss this assignment with the COMP 412 staff and with students currently taking COMP 412. You are encouraged to use the libraries of test blocks produced by students in previous semesters. However, you may not make your COMP 412 labs available to students (other than the COMP 412 teaching assistants) in any form during or after this semester. In particular, you may not place your code anywhere on the Internet where it would be viewable by others. Design and Implementation Timeline: To produce a reasonably good front end, you need to start the implementation during the first week. By August 29, your scanner should be reading and tokenizing the input stream. At this point, you should have a driver that call the scanner repeatedly, until input is exhausted, and prints out the token stream. See the -s command-line option. By September 5, your parser should be functional and producing correct results on the test programs supplied for lab 1. It should be building the internal representation correctly. It should implement both the -p and -r command-line options. Use your final two days to clean up the output from each of the command-line options, to fix any lastminute bugs revealed by your extensive testing, and to create the README and tar files. COMP 412 7
7. The ILOC Subset ILOC is the assembly language for a simple abstract machine. An ILOC program consists of an ordered sequence of operations, each drawn from the following set: Syntax Meaning Latency load r1 => r2 r2 ß MEM(r1) 3 loadi x => r2 r2 ß x 1 store r1 => r2 MEM(r2) ß r1 3 add r1, r2 => r3 r3 ß r1 + r2 1 sub r1, r2 => r3 r3 ß r1 - r2 1 mult r1, r2 => r3 r3 ß r1 * r2 1 lshift r1, r2 => r3 r3 ß r1 << r2 1 rshift r1, r2 => r3 r3 ß r1 >> r2 1 output x prints MEM(x) to stdout 1 nop idle for one cycle 1 Each operation consists of a mnemonic operation code, or opcode, a series of operands, and one or more separators. The opcodes are load, loadi, store, add, sub, mult, lshift, rshift, output, and nop. The operands are either register names or constants. Register names have an initial lowercase r followed immediately by a non-negative integer. Leading zeros in the register name are not significant; thus, r017 and r17 refer to the same register. Arguments that do not begin with r, which appear as x in the table above, are non-negative integer constants in the range 0 to 2 31 1. ILOC assumes a register-toregister memory model (see page 250 in EaC2e) with one class of registers. ILOC operations also contain commas and assignment arrows, which are composed of an equal sign followed by a greater than symbol, as shown (=>). Each ILOC operation in an input block must begin on a new line. 4 Whitespace is defined to be any combination of blanks and tabs. ILOC opcodes must be followed by whitespace. Whitespace preceding and following all other symbols is optional. Whitespace is not allowed within operation names, register names, or the assignment arrow. A double slash (//) indicates that the rest of the line is a comment and can be discarded. Empty lines may also be discarded. The syntax of ILOC is described in further detail in Section 3 of the ILOC simulator document, and, at a higher level, in Appendix A of EaC2e. (See the grammar for Operation that starts on page 726.) Note that your front end is not required to support either labels on operations or the square bracket notation used to group multiple ILOC instructions. 4 Carriage returns (CR, \r, 0x0D) and line feeds (LF, \n, 0x0A) may appear as valid characters in end-of-line sequences. COMP 412 8
8. Intermediate Representations Your front end must create an intermediate representation for the ILOC code that it parses. The local register allocator and the instruction scheduler will manipulate the code in that form. The reference implementation for all three assignments use a simple data structure to represent ILOC code. It consists of a record for each operation. For example, the record for an operation mult r1, r2 => r3 contains the following information: Opcode Operand 1 Operand 2 Operand 3 SR VR PR NU SR VR PR NU SR VR PR NU mult r1 r2 r3 The fields filled with a dash ( ) have values that are not yet known. They will be filled in by the local register allocator or the instruction scheduler. You may also find it useful to store the source-code line number with each operation. The largest operations, such as add and mult, have an opcode and three operands. For each operand, you will need space to store a non-negative integer either a register number (the source register or SR) or a constant value. In subsequent labs. you will also need to store, for each reference, a virtual-register name (VR), a physical-register name (PR), and a next-use location (NU). Each of those fields should be represented as an integer. To represent an entire block, the reference implementations chain together a series of these records. For example, the ILOC sequence: loadi 12 => r3 load r3 => r4 mult r3, r4 => r4 your front end might build a data structure that looks like: The records might be individually allocated as structures. They might be block allocated in large bunches (100; 1,000; or 10,000 records per bunch). They might be stored as columns of an array with an additional field for the next record. COMP 412 9
Additions and Corrections: 1. Fixed footnote 1 to refer to the table on page 8. It was, originally, an incomplete sentence. 2. If you are unfamiliar with handling command-line arguments, an instructor post on Piazza gives the following suggested resources: In Python: https://www.pythonforbeginners.com/system/python-sys-argv https://pythonprogramminglanguage.com/command-line-arguments/ In Java: https://docs.oracle.com/javase/tutorial/essential/environment/cmdlineargs.html In C++ or C: https://www.geeksforgeeks.org/command-line-arguments-in-c-cpp/ COMP 412 10