Lexical and Syntax Analysis Abstract Syntax
What is Parsing? Parser String of characters Data structure Easy for humans to write Easy for programs to process A parser also checks that the input string is well-formed, and if not, rejects it.
Example 1 Parser Charlton, 49 Lineker, 48 Beckham, 17 49 Charlton 17 Beckham 48 Lineker CSV (Comma Separated Value) Array of pairs
Concrete and Abstract Syntax The concrete syntax is a set of rules that describe valid inputs to the parser. The abstract syntax is a set of rules that describe valid outputs from the parser. The data structure produced by a parser is commonly termed the abstract syntax tree.
Concrete and Abstract Syntax Parser String of characters Data structure Conforms to the Concrete Syntax of the language Conforms to the Abstract Syntax of the language
Concrete syntax The concrete syntax is usually defined by regular expressions and context-free grammars. Example: name = [a-za-z]+ goals = [0-9]+ squad = ( name, goals )*
Abstract syntax The abstract syntax is usually specified as a data type in the programming language being used, in our case C. Example: struct player { char* name; int goals; }; struct squad { struct player* players; int size; }; An abstract syntax tree is a value of this type.
This lecture How: to define abstract syntax to construct abstract syntax trees in the programming language C. Also revisits some important C programming techniques.
POINTERS Pointer: a variable that holds the address of a core storage location. [The Free Dictionary]
Pointers Declare a variable x of type int and initialise it to the value 10. x: int x = 10; 10 Declare a variable p of type int* (read: int pointer). int* p; Make p point to x (or assign the address of x to p). p = &x; p: p: x: 10
Pointers Print the value pointed to by p (here, the value of x). printf("%i\n", *p ); Assign 20 to the location pointed to by p. *p = 20; The value NULL means points to nothing. p = NULL; p: p: p: x: 10 x: 20
DYNAMIC ALLOCATION Dynamic Allocation: the allocation of memory storage for use in a computer program. [The Free Dictionary]
Array allocation Declare a variable p of type int*. int* p; p: Allocate memory for an array of 4 int values and let point p to it. p = malloc(4 * sizeof(int)); p:
Array indexing Assign 10 to the location pointed to by p. *p = 10; Assign 20 to the first element of the array pointed to by p. p[0] = 20; Copy the first element of the array to the third element. p[2] = p[0]; p: p: p: 10 20 20 20
Array deallocation When finished with an array allocated by malloc, call free to release the space, otherwise your program may run out of memory. free(p); p: Space released, so it can be reused by future calls to malloc.
STRINGS String: a series of consecutive characters. [The Free Dictionary]
Strings Declare a variable s, initialised to point to the string hi. char* s = hi ; s: h i \0 Let s point to the next character. s = s + 1; s: h i \0 And let s point to the previous character again. s = s - 1; s: h i \0
Exercise 1 What is printed by the following program? int f(char* s) { int i = 0; while (s[i]!= '\0') i++; return i; } void main() { char* x = Hello ; printf( %i\n, f(x)); }
USER-DEFINED TYPES Type: the general character or structure held in common by a number of things. [The Free Dictionary]
Type definitions A typedef declaration allows a new name to be given to a type. typedef int Integer; typedef char* String; Existing type A new name Example use: String s; /* Declare a string s */ Integer i; /* and an integer i */ i = 0; s = hello ;
Enumerations An enum declaration introduces a new type whose values are members of a given set. enum colour {RED, GREEN, BLUE}; New type Possible values Example use: enum colour c; c = RED; if (c == RED) printf( Red\n ); Give it a shorter name: typedef enum colour Colour;
Structures An struct declaration introduces a new type that is a conjunction of one or more existing types. New type struct rectangle { float width; float height; }; Example use: struct rectangle r; r.width = 10; r.height = 20; A width and a height A circle: struct circle { float radius; };
Unions An union declaration introduces a new type that is a disjunction of one or more existing types. New type union shape { struct circle circ; struct rectangle rect; }; A circle or a rectangle Example use: struct shape s; s.circ.radius = 10;
Tagged unions Often a tag is used to denote the active member of a union. Another definition of shape: struct shape { enum { CIRCLE, RECTANGLE } tag; union { struct circle circ; struct rectangle rect; }; }; typedef struct shape Shape;
Tagged unions Example: s is a circle and t is a rectangle, and both are of type Shape. Shape s, t; s.tag = CIRCLE; s.circ.radius = 10; t.tag = RECTANGLE; t.rect.width = 5; t.rect.height = 15;
Tagged unions It is often convenient to define a constructor function for each member of the tagged union. Shape mkcircle(float r) { Shape s; s.tag = CIRCLE; s.circ.radius = r; return s; } Shape mkrectangle(float w, float h) { Shape s; s.tag = RECTANGLE; s.rect.width = w; s.rec.height = h; return s; }
Tagged unions Example revisited: s is a circle and t is a rectangle, and both are of type Shape. Shape s, t; s = mkcircle(10); t = mkrectangle(5, 15);
Tagged unions Example: compute the area of any given shape s. float area(shape s) { if (s.tag == CIRCLE) { float r = s.circ.radius; return (3.14 * r * r); } if (s.tag == RECTANGLE) { return (s.rect.width * s.rect.height); } }
Exercise 2 Define a function to compute the perimeter of any given shape s. float perim(shape s) {... }
Recursive structures A value of type struct t may contain a value of type struct t*. struct list { int head; struct list* tail; }; typedef struct list List; Suppose p is a value of type List* (*p).head p->head (*(*p).tail).head p->tail->head
Recursive structures Example: inserting an item onto the front of a linked list. List* insert(list* xs, int x) { List* ys = malloc(sizeof(list)); ys->head = x; ys->tail = xs; return ys; }
Exercise 3 Define a function to compute the length of a given list xs. int length(list* xs) {... }
CASE STUDY A simplifier for arithmetic expressions.
Concrete syntax Here is a concrete syntax for arithmetic expressions. v = [a-z]+ n = [0-9]+ e v n e + e e * e ( e ) Example expression: x * y + (foo * 10)
Simplification Consider the algebraic law: x. x * 1 = x This law can be used to simplify expressions by using it as a rewrite rule from left to right. Example simplification: x * (y * 1) x * y
Problem 1. Define an abstract syntax, in C, for arithmetic expressions. 2. Define constructor functions so that we can build abstract syntax trees representing expressions. 3. Implement the simplification rule as a C function over abstract syntax trees.
1. Abstract syntax typedef enum { ADD, MUL } Op; struct expr { enum { VAR, NUM, APP } tag; union { }; }; char* var; int num; struct { struct expr* e1; Op op; struct expr* e2; } app; typedef struct expr Expr; A variable or a number or a left expr and an op and a right expr
2. Constructor functions Expr* mkvar(char* v) { Expr* e = malloc(sizeof(expr)); e->tag = VAR; e->var = v; return e; } Expr* mknum(int n) { Expr* e = malloc(sizeof(expr)); e->tag = NUM; e->num = n; return e; } Expr* mkapp(expr* e1, Op op, Expr* e2) { Expr* e = malloc(sizeof(expr)); e->tag = APP; e->app.op = op; e->app.e1 = e1; e->app.e2 = e2; return e; }
2. Abstract syntax trees An abstract syntax tree represents the expression that x + (y * 2) can be constructed by the following C expression mkapp( mkvar("x"), ADD, mkapp( mkvar("y"), MUL, mknum(2)))
3. Simplification x * 1 x is implemented by void simplify(expr* e) { if (e->tag == APP && e->app.op == MUL && e->app.e2->tag == NUM && e->app.e2->num == 1) { *e = *(e->app.e1); } if (e->tag == APP) { simplify(e->app.e1); simplify(e->app.e2); } }
Homework exercises Implement a pretty printer that prints an abstract syntax tree in a concrete form. void print(expr* e) {... } Extend the simplifier to exploit the following algebraic law. x. x * 0 = 0
Motivation for Parsing In SYAC, we are interested in how to implement the following kind of function Expr* parse(char* string) {... } It takes a string conforming to the concrete syntax and returns an abstract syntax tree.