Bringing Undergraduate Residents Greater Evaluative Reasoning

Transcription

1 Brgg Undergraduate Residents Greater Evaluative Reasong Team Members: Jordan Lee (jal2283) Manager Jacquele Kong (jek2179) Language Guru Adrian Traviezo (aft2115) Systems Architect Ashley Nguyen (akn2121) Tester COMS W4115: Programmg Languages and Translators Fall 2017

2 Table of Contents Table of Contents 1 1. Introduction 5 2. Language Tutorial Environment Setup Usg the Compiler Writg Programs Hello World! Variable Declarations Functions If/Else Loops Recursion 8 3. Language Reference Manual Lexical elements Identifiers Comments Keywords, Symbols, and Operators 9 Keywords 9 Symbols and Operators Constants Integer constants Boolean constants Strgs Data Types Primitive Types Non-primitive Types Expression and Statement Syntax Operators Relational Operators Equality Operators Logical Operators Assignment Operator Arithmetic Operators Operator Precedence Delimiters Parentheses 11 1

3 Whitespace Semicolons Declaration and Initialization of Variables and Functions Variable Declaration Variable Initialization Functions Built- Functions Control Flow Expressions if...else statements while loops return statements Program Structure and Scope Rules Program Structure Scope Globals Locals Grammar Project Plan Planng Process Specification Process Development Process Testg Process Programmg Style Guide Project Timele Roles & Responsibilities Software Development Environment Project Log Architectural Design Major Components Interfaces Between Components Division of Labor Test Plan Source Code & their Target Language Programs tests/test-hello.bun tests/test-assign1.bun tests/test-fib.bun Test Suites Used Significance of Chosen Test Cases Test Automation Division of Labor 29 2

4 7. Lessons Learned Jordan Jacquele Adrian Ashley Advice for Future Teams Appendix with Code Listgs scanner.mll parser.mly ast.ml semant.ml codegen.ml burger.ml stdlib.c demo.bun (sample code) Makefile burgr.sh (compilg + runtime script) burgr-c.sh (compilg script) testall.sh (automated testg script) Test Cases 59 fail-assign1.bun 59 fail-dup1.bun 59 fail-expr1.bun 59 fail-expr2.bun 60 fail-func1.bun 60 fail-func2.bun 60 fail-func3.bun 60 fail-func4.bun 60 fail-func6.bun 60 fail-func7.bun 61 fail-func8.bun 61 fail-if.bun 61 fail-var1.bun 61 fail-while1.bun 61 test-assign1.bun 61 test-assign2.bun 62 test-comm1.bun 62 test-fib.bun 62 test-func1.bun 63 test-func2.bun 63 3

5 test-func3.bun 63 test-func4.bun 64 test-func5.bun 64 test-global1.bun 65 test-hello.bun 65 test-if2.bun 65 test-if3.bun 66 test-if4.bun 66 test-if5.bun 66 test-it1.bun 67 test-it2.bun 67 test-math1.bun 67 test-math2.bun 67 test-ops1.bun 68 test-ops2.bun 68 test-ops3.bun 68 test-prt.bun 69 test-prtln.bun 69 test-scope1.bun 69 test-var1.bun 69 test-var2.bun 70 test-while1.bun 70 test-while2.bun 70 4

6 1. Introduction BURGer is a general purpose programmg language designed for convenient, tuitive use. By usg a program structure similar to Python while adherg to conventions familiar to users of Java and C, programmers usg BURGer can write strongly-typed code with the flexibility a orded by top-level code. It is tended to be lightweight and easily readable for users who are familiar with the syntax and types available popular programmg languages, by supportg common data types (ts, booleans, strgs), control flow, and user-defed function behavior. Like any good hamburger, BURGer is an amalgamation of di erent flavors and ideas from discrete sources with the tent to please a diverse user base. It tegrates, for example, variable declarations spired by C with the Strg type available other languages, without requirg a ma function. BURGer aims to bridge similar gaps between a variety of languages with a lightweight but powerful set of operators and features. 2. Language Tutorial 2.1 Environment Setup Before writg a program BURGer, you ll need to download the files associated with the language. They can be accessed on Github at the followg lk: BURGer was developed with LLVM and OCaml , so order to have your programs run correctly, please make sure those versions are stalled on your mache. 2.2 Usg the Compiler Once you have the BURGer project files as well as OCaml and LLVM stalled, you can navigate to the BURGer/src directory. Programs written the BURGer language have a file extension of.bun. You can run the followg commands: To build the compiler: make To compile a.bun file:./burgr c.sh <name of.bun file> 5

7 To compile and run a BURGer program:./burgr.sh <name of.bun file> Example of commands used to run a program usr/burger/src/hamburgers.bun: make./burgr.sh hamburgers 2.3 Writg Programs Writg programs BURGer is very similar to writg programs Java, C, and Python. Like Java and C, BURGer is strongly typed. Like Python, the programmer does not need to declare a ma function and uses the def keyword to declare functions Hello World! hello-word.bun: prt( Hello World!\n ); strg s = hello world ; prtln(s); Output: Hello World! hello world This program demonstrates the use of the built- prt and prtln functions, as well as variable itialization. In the first le of the code, the prt function recognizes the \n the strg literal Hello World!\n as a newle character. In the second, the variable of type strg, s, is assigned the strg literal hello world. The prtln function is passed the identifier s, prts hello world, and then appends a new le to it Variable Declaration and Initialization test-assign.bun: t x; x = 2; bool y = true; t z; bool zero; prtln(x); prtln(y); prtln(z); prtln(zero); 6

8 Output: This program demonstrates a mix of approaches for variable declarations BURGer. As shown the first two les of test-assign.bun, a variable can be declared on one le and then assigned a value on another le. Or, a value can be declared and stantiated to a value the same le. Types of bool and t can be declared but never itialized by the programmer, and their value will return Functions test-func1.bun: def t add(t a, t b) { prtln("add numbers!"); return a + b; prtln(add(1, 3)); Output: add numbers! 4 This program demonstrates function declaration BURGer. A user-defed function is declared with the def keyword, followed by the return type of the function, its identifier (name), and a comma-separated list of arguments. The function is called at the end of test-func1.bun order to execute it If/Else test-if.bun: if(true){ t x; prtln("fff"); if(1 < 0){ prtln("yes"); else{ prtln("no"); 7

9 Output: fff NO This is a typical use of control flow BURGer. An if statement tests a boolean and executes the statement enclosed its brackets if the boolean returns true. It can be followed by an else clause, which executes if the boolean is false, but it need not be Loops test-while.bun: t i; i = 0; while(i<3){ prtln("yum"); prtln("burgers"); i = i+1; Output: yum burgers yum burgers yum burgers This is a typical use of a loop BURGer. As long as the boolean dicated parentheses evaluates to true at the end of the block with curly braces, that block will execute aga Recursion test-fib.bun: def t fib(t x){ if (x < 2) return 1; return fib(x 1) + fib(x 2); prtln(fib(3)); Output: 3 This program demonstrates BURGer s support for recursion and callg functions with other functions by runng the code correspondg to the algorithm for the Fibonacci sequence. 8

10 3. Language Reference Manual 3.1 Lexical elements Identifiers Identifiers are used for namg data types. Identifiers consist of any combation of alphanumeric characters. The first character of an identifier must be a letter Comments Comments are denoted like so: /* text wow look at me multiple les credible */ Keywords, Symbols, and Operators Keywords if else while def return t bool strg true false null Symbols and Operators + / < == && % >= % >.!= = *, <= ( )! ; { 9

11 3.1.4 Constants Integer constants An teger constant consists of one or more digits. Integers are optionally signed and default to positive if they are unsigned Boolean constants Boolean constants can be true or false, which respectively correspond to logical true and false values Strgs A strg is a sequence of characters enclosed sgle or double quotes. BURGer strgs are mutable and iterable. 3.2 Data Types Primitive Types There are three primitive data types: t, bool, and null. Numbers are denoted as an t type, which stores up to 4 bytes. bool holds a Boolean value of either true or false. null is a type used for unitialized variables Non-primitive Types Strgs, as discussed Section 1.5, are a non-primitive type supported BURGer. 3.3 Expression and Statement Syntax Operators Relational Operators BURGer has the followg relational operators: < less than > greater than <= less than or equal to >= greater than or equal to Equality Operators BURGer has the followg equality operators: == equals!= not equals 10

12 Logical Operators BURGer has the followg logical operators: & and or! not Assignment Operator The = bary operator sets the left operand equal to the right operand Arithmetic Operators BURGer supports the standard arithmetic operations: + addition - subtraction * multiplication / division % modulo Operator Precedence The order of precedence for classes of operators is as follows, from the highest to the lowest: arithmetic, logical, assignment, equality/relational. Equality and relational operators have the same level of precedence. For arithmetic operations, multiplication and division take precedence over addition and subtraction. Expressions contaed with parentheses always take precedence. Otherwise, operators of equal precedence levels will take precedence from left to right Delimiters Parentheses BURGer uses parentheses to determe the operation precedence and for enclosg function calls Whitespace BURGer uses whitespace to determe separate tokens. However, the amount of whitespace has no other bearg on the language Semicolons BURGer uses semicolons to termate statements. 11

13 3.3.3 Declaration and Initialization of Variables and Functions Variable Declaration Variables are declared with the desired type of the variable and the variable name. For example: t x; bool y; strg z; Variable Initialization t x; x = 42; Figure a t x = 42; Figure b Variables can be itialized a separate assignment statement after beg declared, like Figure a. However, they can also be declared and itialized the same statement, as shown Figure b Functions A function is declared with the def keyword, a function name, and a list of parameters between parentheses. The return type of the function and of its parameters are not specified on declaration, but a function that does return a value is assigned that return type. A function may be declared with curly braces specifyg what the function does, like so: def t adder (x, y) { return x + y; ; Built- Functions BURGer uses the prt() function to prt a strg literal, or an identifier that has been assigned a strg literal, to the console. The prtln() has an identical functionality, but appends a new le character to it. These functions reside stdlib.c, and are called from with codegen.ml usg L.declare_function. The stdlib.c file is then lked to the BURGer environment durg compilation Control Flow Expressions if...else statements BURGer uses the if else statements a similar way to other languages. It s possible to nest these statements as well. 12

14 if( conditional expression ) { statement; else { statement; while loops BURGer performs while loops with the same syntax as Java. while ( conditional expression ) { statement; return statements BURGer uses the return statement to return values from a function. These values can be of any data type, cludg null. 3.4 Program Structure and Scope Rules Program Structure A BURGer program must be contaed a sgle source file with a.bun file extension Scope BURGer is a statically scoped language. An object is not visible to any operations or declarations that have come before it - for example, a variable x 1 may not form part of the declaration of x 2 unless x 2 was previously formally declared. The scope of a global variable is the entire file, and the scope of a local variable is the block of code pertag to the function which is was itialized. Functions can only be defed a global context; a function can be called, but not defed, with another function Globals Global variables may be declared dependently of any code block - as, they do not need to exist with any function or as part of another declaration. By convention, they should be declared at the top of the file. These variables are visible from any pot of the file Locals Local variables are visible only from with the function where they are defed. If a helper function must access a variable declared the outer function from which it is called, the variable must be passed to the helper function as a parameter. 3.5 Grammar program item_list EOF item_list /* nothg */ item_list item item stmt fdecl 13

15 typ INT BOOL STRING NULL /*** Statements ***/ stmt expr SEMI vdecl SEMI RETURN expr SEMI RETURN SEMI LBRACE stmt_list RBRACE IF LPAREN expr RPAREN stmt %prec NOELSE IF LPAREN expr RPAREN stmt ELSE stmt WHILE LPAREN expr RPAREN stmt stmt_list stmt stmt_list stmt /*** Expressions ***/ expr MINUS expr NOT expr ID ASSIGN expr expr PLUS expr expr MINUS expr expr TIMES expr expr DIVIDE expr expr MOD expr expr EQ expr expr NEQ expr expr LT expr expr LEQ expr expr GT expr expr GEQ expr expr AND expr expr OR expr INTLIT TRUE FALSE ID STRINGLIT LPAREN expr RPAREN ID LPAREN actuals_opt RPAREN /*** Variable Declarations ***/ 14

16 vdecl typ ID typ ID ASSIGN expr /*** Function Declarations ***/ fdecl DEF typ ID LPAREN formals_opt RPAREN LBRACE stmt_list RBRACE formals_opt /* nothg */ formal_list formal_list typ ID formal_list COMMA typ ID actuals_opt /* nothg */ actuals_list actuals_list expr actuals_list COMMA expr 15

17 4. Project Plan 4.1 Planng Process Our team met once a week after class on Mondays to work on the project and subsequently met with Freddy, our TA, on Tuesdays at the begng of his o ce hours. Sometimes we scheduled extra meetgs over the weekend if there was more work that we wanted to get done before our next meetg with Freddy. We would use our time on Mondays to set goals that we wanted to complete by our next meetg, work on tasks accordg to our project timele, delegate tasks that we each needed to work on dividually, and come up with questions that we could ask Freddy when we met with him the next day. As soon as we solidified the members of our team, we created a Google Drive folder where we could store notes from all of our meetgs, questions for Freddy, and all of our required documentation (the project proposal, language reference manual, etc.). 4.2 Specification Process At first, we considered dog a poetry generation language or a language that would support server-based multiplayer games. However, after discussg these ideas with some of the TAs, we realized that our ideas were credibly ambitious and would probably not be able to be completed with the timeframe of this class. We decided that we would try to build a text-based adventure game language but underestimated how long it would take to implement as many features as we had described our Language Reference Manual. In the end, we scaled back to a general purpose programmg language that combed Java-like syntax with Python-like top-level code. 4.3 Development Process When we began code development, we would usually meet up and code together, especially sce we were just startg to become familiar with OCaml. As we became more familiar with the language, we would delegate tasks to be done dividually and come back together to discuss any issues that cropped up. Near the end of the process, we started pair programmg and realized that this method of workg helped crease our productivity significantly. After we received feedback on our Language Reference Manual, we revised our grammar significantly. Then we worked on completg the front end of the compiler, maly our scanner, parser, and AST. Once we successfully compiled Hello World, we began workg on implementg function declarations, and then fleshed out other features such as operators, more robust prtg functions, and control flow. 16

18 4.4 Testg Process In the begng of our testg process, we were focused on makg sure our grammar was correct and unambiguous. We were able to check this usg ocamlyacc to check for ambiguities and parsg particular strgs usg menhir. For example when we passed the followg tokens, we would receive the tree below which would be compared to the tree we created by hand. ID LPAREN STRINGLIT RPAREN SEMI ACCEPT [program: [item_list: [item_list: [item: [stmt: [expr: [arith_expr: [arith_term: [atom: ID LPAREN [actuals_opt: [actuals_list: [expr: [arith_expr: [arith_term: [atom: STRINGLIT]]]] ] ] RPAREN ] ] ] ] SEMI ] ] ] Our test cases were written BURGer and focused on both positive and negative testg. We had a separate folder that contaed all of the test cases that could be executed by callg make and runng./testall.sh. Sce we had a standard library written C, the testall script was modified to clude lkg the outside resources. Each of the positive test cases had 17

19 a correspondg.out file which would be compared to the output of the.bun file usg the prt function. The negative test cases had a correspondg.err file that would come from checks created semant. 4.5 Programmg Style Guide We all used the Atom text editor, which handily supported packages like language-ocaml and ocaml-dent. Those helped us more easily read syntax and use e ective dentation. Tab size: 2 Commentg convention: All comments should be separated by a space from the openg and closg comment symbols. For example, /* comment */ or (* comment *) Multi-le comments should be written as follows: /* begng of comment * more comment * end of comment */ or (* begng of comment * more comment * end of comment *) Parser rule convention: rule_name: pattern1 { action1 pattern2 { action2 The first pattern for each parser rule appears on the le below the rule name, tabbed and with two spaces before it. All patterns after that have a tab before the, and a space before the pattern. Actions should align vertically with each other, meang that for each parser rule, all actions will be left-aligned one space to the right of the lengthiest pattern that rule. When a rule accepts an empty strg, the matchg pattern should be denoted with a commented-out nothg, as follows: rule_name: /* nothg */ { [] pattern1 { action1 pattern2 { action2 AST type convention: type some_type = Name1 of type1 Name2 of type2 For AST types with multiple patterns, each name will beg with a capital letter. AST types with only one possible pattern can be written on one le, if desired. 18

20 OCaml codg convention: Align Ocaml declarations and assignments as follows: let example = (* example code *) If a declaration is bound usg the keyword, it should align with the let keyword used to declare the function/variable. The body of functions should be dented with one tab. However, if the body of the assignment consists of only one le, it may occur on the same le as the declaration. In some cases, the may also occur on the same le as the body, for stance a pattern-matchg clause. Use comments as necessary to expla specific portions of the code, and when raisg exception messages, be as specific as possible. 4.6 Project Timele This was our projected timele at the begng of the project. Dates September September 26 October 16 November 7 November 21 November 28 December 5 December 12 December December 20 Goals Assemble team members and develop language idea Submit Proposal Submit Language Reference Manual Fish Hello World Falize language features (scanner, parser, AST) and add more to codegen, especially function declarations Fish semantic checkg and itial tests Fish codegen and C standard library Fish test suite Present Fal Presentation Submit Fal Report 19

21 4.7 Roles & Responsibilities Team Member Jordan Lee Jacquele Kong Adrian Traviezo Ashley Nguyen Role Manager Language Guru Systems Architect Tester The above table shows the assigned roles that we determed at the begng of the project. Because of these assignments, we each did become more specialized our respective areas. However, as we contued to develop our language, these roles became more fluid and our responsibilities were not necessarily restricted to our roles. This was especially evident after we started pair programmg, which not only upped our productivity but also broadened our knowledge of di erent areas of our language. 4.8 Software Development Environment We used LLVM version and OCaml version All of us had Mac OS X (di erent versions, however - Yosemite, Sierra, and High Sierra). We stored our project files a git repository on GitHub and used Google Drive to store documentation (Proposal, LRM, meetg notes, etc.). Our text editor of choice was Atom, which supported some handy OCaml packages that assisted with syntax highlightg. This was especially helpful as we gradually learned to program e ectively OCaml. 4.9 Project Log Dates September September 26 October 15 October 16 October 27 November 10 November 25 Task Assembled team members and develop language idea Submitted Proposal Initial commit created git repository Submitted Language Reference Manual Committed scanner and parser Successfully compiled Hello World Falized language features (scanner, parser, AST) and flesh out codegen 20

22 November 28 December 5 December 12 December December 20 Fish semantic checkg Fish codegen and C standard library Fish test suite Present Fal Presentation Submit Fal Report 21

23 5. Architectural Design 5.1 Major Components Here is a block diagram showg the major components of our compiler. 22

24 5.2 Interfaces Between Components 1..bun program 1.1. This is the source code for the program that is to be compiled the BURGer language. 2. Scanner 2.1. This component, defed scanner.ml, handles the conversion of the put.bun file to a form that can be processed by the rest of the compilation process. The scanner generates a stream of tokens out of the text the.bun file that will be parsed by the parser. 3. Parser 3.1. This component, defed parser.mly, processes the stream of tokens passed to it by the Scanner and creates an AST (Abstract Syntax Tree) as specified by the grammar rules written it and the types ast.ml. 4. Semantic Checker 4.1. The semantic checker, located semant.ml, opens the AST generated by the parser and ensures that the.bun file that is beg processed contas a valid, semantically correct, program. It does so by performg type checkg and raisg errors when there are valid parts of code that, for example, declare the same variable twice. After the AST is semantically checked, it is passed on to the code generation step. 5. Code Generation 5.1. Once the.bun file is checked and it is verified that it contas a semantically correct program, the generated syntax tree is processed by codegen.ml. This file contas the code that converts types like ts and booleans to native LLVM primitives and builds LLVM blocks out of statements. LLVM Intermediate Representation is then generated to the burger.native file. 6. GCC 6.1. This is the component that lks the aforementioned files together. The GCC Compiler system is used to produce an output bary file and lks the stdlib.c (Standard Library) file so that the built- functions can be defed with the generated code. 7. BURGer Executable 7.1. Once a.bun file successfully travels through the previous steps, a bary file is created by GCC and is ready to run. See section 2.2 for how to run this file. 5.3 Division of Labor See Section 6.5: Division of Labor. 23

25 6. Test Plan 6.1 Source Code & their Target Language Programs tests/test-hello.bun prt("hello World!"); tests/test-hello.ll ; ModuleID = 'BURGer' source_filename = = private unnamed_addr constant [4 x i8] = private unnamed_addr constant [3 x i8] = private unnamed_addr constant [13 x i8] c"hello World!\00" declare declare declare defe { entry: %prt = call i32 getelementptr bounds ([13 x i8], [13 x i32 0, i32 0)) ret i tests/test-assign1.bun t x; bool y; x = 2; y = true; prt("x = "); prtln(x); prt("y = "); prtln(y); tests/test-assign2.ll 24

26 ; ModuleID = 'BURGer' source_filename = = global i32 = global i1 = private unnamed_addr constant [4 x i8] = private unnamed_addr constant [3 x i8] = private unnamed_addr constant [5 x i8] c"x = = private unnamed_addr constant [5 x i8] c"y = \00" declare declare declare defe { entry: store i32 2, store i1 true, %prt = call i32 getelementptr bounds ([5 x i8], [5 x i32 0, i32 0)) %x = load i32, %prt1 = call i32 getelementptr bounds ([4 x i8], [4 x i32 0, i32 0), i32 %x) %prt2 = call i32 getelementptr bounds ([5 x i8], [5 x i32 0, i32 0)) %y = load i1, %prt3 = call i32 getelementptr bounds ([4 x i8], [4 x i32 0, i32 0), i1 %y) ret i tests/test-fib.bun def t fib(t x) { if (x < 2) return 1; return fib(x 1) + fib(x 2); prtln(fib(0)); prtln(fib(1)); prtln(fib(2)); prtln(fib(3)); prtln(fib(4)); prtln(fib(5)); 25

27 prtln("fibonacci test over!"); tests/test-fib.ll ; ModuleID = 'BURGer' source_filename = = private unnamed_addr constant [4 x i8] = private unnamed_addr constant [3 x i8] = private unnamed_addr constant [21 x i8] c"fibonacci test = private unnamed_addr constant [4 x i8] = private unnamed_addr constant [3 x i8] c"%d\00" declare declare declare defe { entry: %fib_result = call 0) %prt = call i32 getelementptr bounds ([4 x i8], [4 x i32 0, i32 0), i32 %fib_result) %fib_result1 = call 1) %prt2 = call i32 getelementptr bounds ([4 x i8], [4 x i32 0, i32 0), i32 %fib_result1) %fib_result3 = call 2) %prt4 = call i32 getelementptr bounds ([4 x i8], [4 x i32 0, i32 0), i32 %fib_result3) %fib_result5 = call 3) %prt6 = call i32 getelementptr bounds ([4 x i8], [4 x i32 0, i32 0), i32 %fib_result5) %fib_result7 = call 4) %prt8 = call i32 getelementptr bounds ([4 x i8], [4 x i32 0, i32 0), i32 %fib_result7) %fib_result9 = call 5) %prt10 = call i32 getelementptr bounds ([4 x i8], [4 x i32 0, i32 0), i32 %fib_result9) %prtln = call i32 getelementptr bounds ([21 x i8], [21 x i32 0, i32 0)) ret i32 0 defe %x) { entry: %x1 = alloca i32 store i32 %x, i32* %x1 26

28 %x2 = load i32, i32* %x1 %tmp = icmp slt i32 %x2, 2 br i1 %tmp, label %then, label %else merge: %x3 = load i32, i32* %x1 %tmp4 = sub i32 %x3, 1 %fib_result = call %tmp4) %x5 = load i32, i32* %x1 %tmp6 = sub i32 %x5, 2 %fib_result7 = call %tmp6) %tmp8 = add i32 %fib_result, %fib_result7 ret i32 %tmp8 then: ret i32 1 else: br label %merge ; preds = %else ; preds = %entry ; preds = %entry 6.2 Test Suites Used See Appendix, Section 8.12: Test Cases for the full list of test suites. 6.3 Significance of Chosen Test Cases We tested for the syntax and the various features of our language and made sure that each new addition to our language would be compatible with the rest. Assignment Operators Assignment operators were used to test and make sure our primitive data types ( t, strg, bool, and null ) were strongly typed as well as checkg that variable itialization was possible. Control Flow Control flow statements were tested to make sure our if, if/else, and while statements worked. There were also negative test cases that would return an error if boolean values were not passed to the function. Bary Operators Bary operators were tested by comparg two values and makg sure the correct response was returned by the file. Additionally, the arithmetic operators were tested by performg the calculations and makg sure they matched with the expected output. 27

29 Unary Operators Unary operators were tested by checkg to see they followed their tended purposes by negatg with (!) for logical expressions or (-) for numbers. Built- functions Built- functions such as prt() and prtln() were also tested to make sure that prt() would simply output a value, whereas prtln() would output a value appended with a new le. They were also checked semant.ml so no one function could be defed usg those names. Comments Comments were tested by addg them throughout the programs usg the convention /* Testg Comments */ or (* Testg Comments *) to make sure it would not a ect the code written Top Level Code Top level code was also checked by makg sure programs were able to run without a ma function and that a ma function could not be declared the file. Function Calls Function calls were tested to make sure the correct values were passed and returned. The negative testg ensured that no null values would be passed to the function. Static Scopg Static scopg was tested by assigng variables globally and checkg to make sure the correct output was given after it was changed a function. 6.4 Test Automation We used a script called testall.sh to automatically run all of our tests and create user-friendly output that would be easy to read. This was modified from the script used to test cases for MicroC. The output when./testall.sh runs is as follows: dyn :src jordanlee$./testall.sh n test assign1... n test assign2... n test comm1... n test fib... n test func

30 n test func2... n test func3... n test func4... n test func5... n test global1... n test hello... n test if2... n test if3... n test if4... n test if5... n test it1... n test it2... n test math1... n test math2... n test ops1... n test ops2... n test ops3... n test prt... n test prtln... n test scope1... n test var1... n test var2... n test while1... n test while

31 n fail assign1... n fail dup1... n fail expr1... n fail expr2... n fail func1... n fail func2... n fail func3... n fail func4... n fail func6... n fail func7... n fail func8... n fail if... n fail var1... n fail while Division of Labor While BURGer s development team had clear roles that corresponded to our dividual responsibilities, the bulk of the code was written and revised by every member of the group either simultaneously or with our own time. A few contributions that reflect the roles we set for ourselves, however, are listed below. Jacquele ensured that the grammar as defed our parser was clear and consistent. Adrian tegrated the stdlib.c file, which cludes the built- functions that comprise the standard library. Ashley, the assigned Tester, created most of the tests and set up the automated testg script. Jordan wrote the sample code used for our demo and contributed to the testg script as well, and managed the location and times of our group meetgs. All members helped makg the testg more rigorous and wrote some tests. 30

32 7. Lessons Learned 7.1 Jordan Functional programmg required me to learn a di erent, eye-openg way to solve problems. As someone so used to codg imperative languages, it was hard for me to adjust to codg a functional programmg language like OCaml at first. However, I feel like I ve come away with a better understandg of how to program more concisely. I also learned the importance of takg advantage of valuable resources such as the OCaml documentation and llvm.moe. Discoverg and learng from such resources earlier on could have helped me fulfillg my managerial role and settg goals for our team. By lookg at previous fal reports, I would have had a much better idea of what the scope of our project was and how to plan the work ahead of us. 7.2 Jacquele I had never thought too much about compilers before takg this class, and I have really enjoyed learng about the compiler pipele and about functional programmg this course. Functional programmg is a very di erent paradigm than I m used to, and my CS Theory class didn t cover the Lambda Calculus, so this was defitely an eye-openg experience for me. I have learned a lot about solvg problems functionally vs. imperatively. Additionally, the process of designg a language and implementg a compiler for it helped me learn a balance between preparg adequately to create a project and actually buildg the project. For stance, it s worth spendg some time on settg up a workable development environment, but it might not be a good use of time to deliberate for too long on nitpicky design choices that will probably end up beg altered anyway. I also realized the importance of havg a meangful test suite. 7.3 Adrian Workg on BURGer was perhaps the most educational group project I have experienced at Columbia. There were many smaller steps learng to build the compiler that ended up teachg me more about the bigger picture. Learng how to program OCaml made me a stronger programmer all around, sce I learned to thk about problems from a functional perspective. Understandg how to read assembly code became an valuable asset debuggg. And, defitely, hands-on experience with thgs such as disambiguatg grammars was extremely helpful when learng the material for this class, as well as for learng important topics about computer science general. However, the bigger picture I mentioned was probably even more valuable: by takg on a project so big and out of my comfort zone, I learned more about how to situate myself as part of a team and as part of a multi-step (and multi-doma) process. Because of BURGer, I gaed experience with navigatg the development of a complex product, and with learng to 31

33 consider all the parts of that complex product both as discrete entities but also with respect to their importance to each other. I am therefore much more confident approachg not just compiler design, but projects that would require a similar holistic approach and group dynamics. 7.4 Ashley Throughout this project and workg on our language I learned a lot more about functional programmg as well as workg a collaborative environment. Usg OCaml and LLVM made the fixg issues a lot di erent sce there is not as much documentation about it and I had to understand the entire process to debug the errors I was encounterg. I also learned a lot about the importance of contuous tegration testg sce changg one thg could lk to problems other files that were never touched. It is defitely somethg I will be more aware of the future as a way to fix errors as the arise makg it easier to correct rather than havg it possibly create even more problems as it contues. Usg pair programmg for the first time was also a very useful approach, havg someone there to bounce ideas o of and help made thgs a lot more e cient than everyone workg dividually. From this I have a much better understandg about how to work with a group structure while learng a new language which I will be able to use the future. 7.5 Advice for Future Teams Perhaps the most obvious advice we could give here would be to start early. Even though our team had a good early start with defg our language ( terms of layg out our grammar, rules, and how we wanted to structure functions and statements), we could have done a little more research to how all the components of our compiler would tegrate with each other earlier on. Do not expect to learn everythg it will take to write your language from lectures; attendg class and studyg the slides and textbook will give you a good framework, but if you wait to read up on code generation until after the midterm, chances are you ll have some catchg up to do later on. Make use of resources available to you, cludg MicroC and past projects (but approach everythg with healthy criticism!), and try to really understand what s happeng the code holistically. Another piece of advice we would like to give to future teams is to take time to read as much documentation about OCaml and LLVM as possible. This is paramount not only to learn about how to, say, use builders LLVM or recursion OCaml, but also to see whether there are functions that already exist either that can cut the time you spend writg your own implementations. Makg sure to be well-read on all the di erent steps of implementg a compiler is also extremely important do not try to only be an expert on one aspect of the pipele, as beg knowledgeable about the entire process (from scanng to parsg to code generation to, possibly, a standard library) can help when makg important design decisions. 32

34 8. Appendix with Code Listgs 8.1 scanner.mll (* * Authors: * Jordan Lee * Jacquele Kong *) { open Parser let escape = '\\' ['\\' ''' '"' 'n' 'r' 't'] let ascii = ([' ' '!' '#' '[' ']' '~']) let strg = ('"') ( (ascii escape)* as s) ('"') rule token = parse [' ' '\t' '\r' '\n'] { token lexbuf "/*" { comment lexbuf '(' { LPAREN ')' { RPAREN ';' { SEMI strg { STRINGLIT(s) '{' { LBRACE '' { RBRACE ',' { COMMA '+' { PLUS ' ' { MINUS '*' { TIMES '/' { DIVIDE '%' { MOD '=' { ASSIGN "<" { LT ">" { GT "==" { EQ "!=" { NEQ "<=" { LEQ ">=" { GEQ "&&" { AND " " { OR "!" { NOT "if" { IF "else" { ELSE "while" { WHILE "true" { TRUE "false" { FALSE 33

35 "t" { INT "strg" { STRING "bool" { BOOL "null" { NULL "return" { RETURN "def" { DEF ['0' '9']+ as lxm { INTLIT(t_of_strg lxm) ['a' 'z' 'A' 'Z']['a' 'z' 'A' 'Z' '0' '9' '_']* as lxm { ID(lxm) eof { EOF and comment = parse "*/" { token lexbuf _ { comment lexbuf 8.2 parser.mly /* Parser for BURGer Programmg Language * PLT Fall 2017 * Authors: * Jacquele Kong * Jordan Lee * Adrian Traviezo * Ashley Nguyen */ %{ open Ast % %token <strg> ID %token <strg> STRINGLIT %token <t> INTLIT %token LPAREN RPAREN SEMI LBRACE RBRACE COMMA %token PLUS MINUS TIMES DIVIDE ASSIGN MOD %token LT GT LEQ GEQ EQ NEQ %token AND OR NOT %token IF ELSE NOELSE %token TRUE FALSE %token INT STRING BOOL NULL %token WHILE DEF RETURN %token EOF %nonassoc NOELSE %nonassoc ELSE %right ASSIGN %left OR %left AND 34

36 %left EQ NEQ %left LT GT LEQ GEQ %left PLUS MINUS %left TIMES DIVIDE MOD %right NOT %start program %type <Ast.program> program %% /*** Top Level ***/ program: item_list EOF { List.rev $1 item_list: /*nothg */ { [] item_list item { ($2 :: $1) item: stmt { Stmt($1) fdecl { Function($1) typ: INT { Int BOOL { Bool STRING { Strg NULL { Null /*** Statements ***/ stmt: expr SEMI { Expr($1) vdecl SEMI { $1 RETURN expr SEMI { Return($2) RETURN SEMI { Return(NoExpr) LBRACE stmt_list RBRACE { Block(List.rev $2) IF LPAREN expr RPAREN stmt %prec NOELSE { If($3, $5, Block([])) IF LPAREN expr RPAREN stmt ELSE stmt { If($3, $5, $7) WHILE LPAREN expr RPAREN stmt { While($3, $5) stmt_list: stmt { [$1] stmt_list stmt { ($2 :: $1) /*** Expressions ***/ expr: 35

37 MINUS expr { Unop(Neg, $2) NOT expr { Unop(Not, $2) ID ASSIGN expr { Assign($1, $3) expr PLUS expr { Bop($1, Add, $3) expr MINUS expr { Bop($1, Sub, $3) expr TIMES expr { Bop($1, Mult, $3) expr DIVIDE expr { Bop($1, Div, $3) expr MOD expr { Bop($1, Mod, $3) expr EQ expr { Bop($1, Equal, $3) expr NEQ expr { Bop($1, Neq, $3) expr LT expr { Bop($1, Less, $3) expr LEQ expr { Bop($1, Leq, $3) expr GT expr { Bop($1, Greater, $3) expr GEQ expr { Bop($1, Geq, $3) expr AND expr { Bop($1, And, $3) expr OR expr { Bop($1, Or, $3) INTLIT { IntLit($1) TRUE { BoolLit(true) FALSE { BoolLit(false) ID { Id($1) STRINGLIT { StrgLit($1) LPAREN expr RPAREN { $2 ID LPAREN actuals_opt RPAREN { Call($1, $3) /*** Variable Declarations ***/ vdecl: typ ID { VDecl($1, $2) typ ID ASSIGN expr { VAssign(($1, $2), $4) /*** Function Declarations ***/ fdecl: DEF typ ID LPAREN formals_opt RPAREN LBRACE stmt_list RBRACE { { typ = $2; fname = $3; formals = $5; body = List.rev $8; formals_opt: /* nothg */ { [] formal_list { List.rev $1 formal_list: typ ID { [($1,$2)] formal_list COMMA typ ID { ($3,$4) :: $1 actuals_opt: 36

38 /* nothg */ { [] actuals_list { List.rev $1 actuals_list: expr { [$1] actuals_list COMMA expr { $3 :: $1 8.3 ast.ml (* Abstract Syntax Tree * BURGer Programmg Language * PLT Fall 2017 * Authors: * Jacquele Kong * Jordan Lee * Adrian Traviezo * Ashley Nguyen *) (*** Syntax Types ***) type op = Add Sub Mult Div Equal Neq Less Leq Greater Geq And Or Mod type uop = Neg Not type expr = Id of strg Call of strg * expr list IntLit of t BoolLit of bool StrgLit of strg Assign of strg * expr Bop of expr * op * expr Unop of uop * expr NoExpr type typ = Int Bool Strg Null Array of typ * expr type bd = typ * strg type stmt = Block of stmt list Expr of expr VDecl of bd Return of expr 37

39 If of expr * stmt * stmt While of expr * stmt For of expr * expr * expr * stmt VAssign of bd * expr type func_decl = { typ : typ; fname : strg; formals : bd list; body : stmt list; type item = Stmt of stmt Function of func_decl type program = item list (*** Functions for Prtg ***) let strg_of_typ = function Int > "t" Bool > "bool" Null > "null" Strg > "strg" let strg_of_op = function Add > "+" Sub > " " Mult > "*" Div > "/" Mod > "%" Equal > "==" Neq > "!=" Less > "<" Leq > "<=" Greater > ">" Geq > ">=" And > "&&" Or > " " let strg_of_uop = function Neg > " " Not > "!" let rec strg_of_expr = function IntLit(l) > strg_of_t l BoolLit(true) > "true" BoolLit(false) > "false" 38

40 StrgLit(s) > s Id(s) > s Bop(e1, o, e2) > strg_of_expr e1 ^ " " ^ strg_of_op o ^ " " ^ strg_of_expr e2 Unop(o, e) > strg_of_uop o ^ strg_of_expr e Assign(v, e) > v ^ " = " ^ strg_of_expr e Call(f, el) > f ^ "(" ^ Strg.concat ", " (List.map strg_of_expr el) ^ ")" NoExpr > "" let rec strg_of_stmt = function Block(stmts) > "{\n" ^ Strg.concat "" (List.map strg_of_stmt stmts) ^ "\n" Expr(expr) > strg_of_expr expr ^ ";\n"; Return(expr) > "return " ^ strg_of_expr expr ^ ";\n"; If(e, s, Block([])) > "if (" ^ strg_of_expr e ^ ")\n" ^ strg_of_stmt s If(e, s1, s2) > "if (" ^ strg_of_expr e ^ ")\n" ^ strg_of_stmt s1 ^ "else\n" ^ strg_of_stmt s2 While(e, s) > "while (" ^ strg_of_expr e ^ ") " ^ strg_of_stmt s 8.4 semant.ml (* Semantic Checker for the BURGer Programmg Language PLT Fall 2017 Authors: Adrian Traviezo Jacquele Kong Ashley Nguyen Jordan Lee *) open Ast module StrgMap = Map.Make(Strg) (* Semantic checkg of a program. Returns void if successful, throws an exception if somethg is wrong. Check each global variable, then check each function *) let check_program program = (* Raise an exception if the given list has a duplicate *) let report_duplicate exceptf list = 39

41 let rec helper = function n1 :: n2 :: _ when n1 = n2 > raise (Failure (exceptf n1)) _ :: t > helper t [] > () helper (List.sort compare list) (* figure out which items are statements and make a list of statements *) let stmt_list = let stmts_as_items = List.filter (fun x > match x with Ast.Stmt(x) > true _ > false) program List.map (fun x > match x with Ast.Stmt(x) > x _ > failwith "stmt castg didn't work") stmts_as_items (* after you figure out which items are statements, you need to go through the statements and figure out which ones conta the variable declarations and variable decl+assignment statements *) let globals = let global_list = List.filter (fun x > match x with Ast.VDecl(x) > true Ast.VAssign(x, _) > true _ > false) stmt_list List.map (fun x > match x with Ast.VDecl(x) > x Ast.VAssign(x, _) > x _ > failwith "not turned to global") global_list let functions = let functions_as_items = List.filter (fun x > match x with Ast.Function(x) > true _ > false) program let all_functions_as_items = functions_as_items List.map (fun x > match x with Ast.Function(x) > x _ > failwith "function castg didn't work") all_functions_as_items (* let function_locals = let get_locals_from_fbody fdecl = let get_vdecl locals_list stmt = match stmt with 40

42 Ast.VDecl(typ, strg) > (typ, strg) :: locals_list _ > locals_list List.fold_left get_vdecl [] fdecl.ast.body List.fold_left get_locals_from_fbody (List.hd functions) (List.tl functions) *) let symbols = List.fold_left (fun var_map (vartype, varname) > StrgMap.add varname vartype var_map) StrgMap.empty (globals) let type_of_identifier s = try StrgMap.fd s symbols with Not_found > raise (Failure ("undeclared identifier " ^ s)) (* Raise an exception of the given rvalue type cannot be assigned to the given lvalue type *) let check_assign lvaluet rvaluet err = if lvaluet == rvaluet then lvaluet else raise err (* Raise an exception if a given bdg is to a void type *) let check_not_void exceptf = function (Null, n) > raise (Failure (exceptf n)) _ > () let built decls = StrgMap.add "prtln" { typ = Null; fname = "prtln"; formals = []; body = [] (StrgMap.sgleton "prt" { typ = Null; fname = "prt"; formals = []; body = [] ) let function_decls = List.fold_left (fun m fd > StrgMap.add fd.fname fd m) built decls functions let function_decl s = try StrgMap.fd s function_decls with Not_found > raise (Failure ("unrecognized function " ^ s)) let check_function func = 41

43 report_duplicate (fun n > "duplicate formal " ^ n ^ " " ^ func.fname) (List.map snd func.formals); (); if List.mem "prt" (List.map (fun fd > fd.fname) functions) then raise (Failure ("function prt may not be defed")) else if List.mem "prtln" (List.map (fun fd > fd.fname) functions) then raise (Failure ("function prtln may not be defed")) else (); if List.mem "prtf" (List.map (fun fd > fd.fname) functions) then raise (Failure ("function prtf may not be defed")) else (); report_duplicate (fun n > "duplicate function " ^ n) (List.map (fun fd > fd.fname) functions); (); if List.mem "ma" (List.map (fun fd > fd.fname) functions) then raise (Failure ("function ma may not be defed")) else List.iter (check_not_void (fun n > "illegal null formal " ^ n ^ " " ^ func.fname)) func.formals; ^ (* List.iter (check_not_void (fun n > "illegal void local " ^ n " " ^ func.fname)) func.locals; *) (* report_duplicate (fun n > "duplicate local " ^ n ^ " " ^ func.fname) (List.map snd func.locals); *) let rec expr = function IntLit _ > Int BoolLit _ > Bool StrgLit _ > Strg Id s > type_of_identifier s Assign(var, e) as ex > let lt = type_of_identifier var and rt = expr e check_assign lt rt (Failure ("illegal assignment " ^ strg_of_typ lt ^ " = " ^ strg_of_typ rt ^ " " ^ strg_of_expr ex)) Bop(e1, op, e2) as e > let t1 = expr e1 and t2 = expr e2 (match op with Add Sub Mult Div Mod when t1 = Int && t2 = Int > Int 42

44 Bool Equal Neq when t1 = t2 > Bool Less Leq Greater Geq when t1 = Int && t2 = Int > And Or when t1 = Bool && t2 = Bool > Bool _ > raise (Failure ("illegal bary operator " ^ strg_of_typ t1 ^ " " ^ strg_of_op op ^ " " ^ strg_of_typ t2 ^ " " ^ strg_of_expr e)) ) Call(fname, actuals) as call > let fd = function_decl fname if (fname = "prt" fname = "prtln") then let _ = List.iter (fun e > ignore(expr e)) actuals Null else (if List.length actuals!= List.length fd.formals then raise (Failure ("expectg " ^ strg_of_t (List.length fd.formals) ^ " arguments " ^ strg_of_expr call)) else List.iter2 (fun (ft, _) e > let et = expr e ignore (check_assign ft et (Failure ("illegal actual argument: found " ^ strg_of_typ et ^ " ; expected " ^ strg_of_typ ft ^ " " ^ strg_of_expr e)))) fd.formals actuals; fd.typ) Unop(op, e) as ex > let t = expr e (match op with Neg when t = Int > Int Not when t = Bool > Bool _ > raise (Failure ("illegal unary operator " ^ strg_of_uop op ^ strg_of_typ t ^ " " ^ strg_of_expr ex))) NoExpr > Null let check_bool_expr e = if expr e!= Bool then raise (Failure ("expected Boolean expression " ^ strg_of_expr e)) else () let rec check_stmt s = match s with Block sl > let rec check_block = function [Return _ as x] > check_stmt x Return _ :: _ > raise (Failure "nothg may follow a return") Block sl :: ss > check_block ss) x :: ss > check_stmt x ; check_block ss 43

45 [] > () check_block sl VDecl _ > () VAssign ((typ, strg), e) > ignore (expr (Assign(strg, e))) Expr e > ignore (expr e) If(p, b1, b2) > check_bool_expr p; check_stmt b1; check_stmt b2 While(p, s) > check_bool_expr p; check_stmt s (* Check for assignments and duplicate vdecls *) List.iter check_function functions; List.iter check_stmt stmt_list; (* List.iter stmt stmt_list; *) report_duplicate (fun n > "Duplicate assignment for " ^ n) (List.map snd globals); 8.5 codegen.ml (* * Code generation for BURGer Programmg Language * Authors: * Jordan Lee * Jacquele Kong * Adrian Traviezo * Ashley Nguyen *) module L = Llvm module A = Ast module StrgMap = Map.Make(Strg) let translate (program) = let context = L.global_context () let the_module = L.create_module context "BURGer" and i8_t = L.i8_type context and str_t = L.poter_type (L.i8_type context) and null_t = L.void_type context and i1_t = L.i1_type context and i32_t = L.i32_type context (* types of variables BURGer*) let ltype_of_typ = function A.Strg > str_t 44