Compiler Construction I

Transcription

1 TECHNISCHE UNIVERSITÄT MÜNCHEN FAKULTÄT FÜR INFORMATIK Compiler Construction I Dr. Michael Petter, Dr. Axel Simon SoSe / 59

2 Organizing Master or Bachelor in the 6th Semester with 5 ECTS Prerequisites Informatik 1 & 2 Theoretische Informatik Technische Informatik Grundlegende Algorithmen Delve deeper with Virtual Machines Programmoptimization Programming Languages Praktikum Compilerbau Hauptseminars Materials: TTT-based lecture recordings the slides Related literature list online Tools for visualization of virtual machines Tools for generating components of Compilers 2 / 59

3 Organizing Zeiten: Lecture: Mo. 14:15-15:45 Tutorial: Tuesday morning and afternoon Exam Exam managed via TUM-online Successful tutorial exercises earns 0.3 bonus 3 / 59

4 Preliminary content Basics in regular expressions and automata Specification with regular expressions and implementation with automata Reduced context free grammars and pushdown automata Bottom-Up Syntaxanalysis Attribute systems Typechecking Codegeneration for stack machines Register assignment Basic Optimization 4 / 59

5 Topic: Semantic Analysis 5 / 59

6 Semantic Analysis Scanner and parser accept programs with correct syntax. not all programs that are syntacticallly correct make sense 6 / 59

7 Semantic Analysis Scanner and parser accept programs with correct syntax. not all programs that are syntacticallly correct make sense the compiler may be able to recognize some of these these programs are rejected and reported as erroneous the language definition defines what erroneous means 6 / 59

8 Semantic Analysis Scanner and parser accept programs with correct syntax. not all programs that are syntacticallly correct make sense the compiler may be able to recognize some of these these programs are rejected and reported as erroneous the language definition defines what erroneous means semantic analyses are necessary that, for instance: check that identifiers are known and where they are defined check the type-correct use of variables 6 / 59

9 Semantic Analysis Scanner and parser accept programs with correct syntax. not all programs that are syntacticallly correct make sense the compiler may be able to recognize some of these these programs are rejected and reported as erroneous the language definition defines what erroneous means semantic analyses are necessary that, for instance: check that identifiers are known and where they are defined check the type-correct use of variables semantic analyses are also useful to find possibilities to optimize the program warn about possibly incorrect programs 6 / 59

10 Semantic Analysis Scanner and parser accept programs with correct syntax. not all programs that are syntacticallly correct make sense the compiler may be able to recognize some of these these programs are rejected and reported as erroneous the language definition defines what erroneous means semantic analyses are necessary that, for instance: check that identifiers are known and where they are defined check the type-correct use of variables semantic analyses are also useful to find possibilities to optimize the program warn about possibly incorrect programs a semantic analysis annotates the syntax tree with attributes 6 / 59

11 Semantic Analysis Chapter 1: Attribute Grammars 7 / 59

12 Attribute Grammars many computations of the semantic analysis as well as the code generation operate on the syntax tree what is computed at a given node only depends on the type of that node (which is usually a non-terminal) we call this a local computation: only accesses already computed information from neighbouring nodes computes new information for the current node and other neighbouring nodes 8 / 59

13 Attribute Grammars many computations of the semantic analysis as well as the code generation operate on the syntax tree what is computed at a given node only depends on the type of that node (which is usually a non-terminal) we call this a local computation: only accesses already computed information from neighbouring nodes computes new information for the current node and other neighbouring nodes Definition attribute grammar An attribute grammar is a CFG extended by an set of attributes for each non-terminal and terminal local attribute equations 8 / 59

14 Attribute Grammars many computations of the semantic analysis as well as the code generation operate on the syntax tree what is computed at a given node only depends on the type of that node (which is usually a non-terminal) we call this a local computation: only accesses already computed information from neighbouring nodes computes new information for the current node and other neighbouring nodes Definition attribute grammar An attribute grammar is a CFG extended by an set of attributes for each non-terminal and terminal local attribute equations in order to be able to evaluate the attribute equations, all attributes mentioned in that equation have to be evaluated already the nodes of the syntax tree need to be visited in a certain sequence 8 / 59

15 Example: Computation of the empty[r] Property Consider the syntax tree of the regular expression (a b)*a(a b):. *. 2 a 0 1 a b a b / 59

16 Example: Computation of the empty[r] Property Consider the syntax tree of the regular expression (a b)*a(a b):. *. f 2 f f 0 a f f a b a b 9 / 59

17 Example: Computation of the empty[r] Property Consider the syntax tree of the regular expression (a b)*a(a b):. f * f f 0 1 a f 2 b a. f 3 a f f 4 b 9 / 59

18 Example: Computation of the empty[r] Property Consider the syntax tree of the regular expression (a b)*a(a b):. t f *. f f 2 f f 0 1 a f f f a b a b / 59

19 Example: Computation of the empty[r] Property Consider the syntax tree of the regular expression (a b)*a(a b): f f. t f *. f f 2 f f f a b a a f b equations for empty[r] are computed from bottom to top (aka bottom-up) 9 / 59

20 Implementation Strategy attach an attribute empty to every node of the syntax tree compute the attributes in a depth-first traversal: at a leaf, we can compute the value of empty without considering other nodes the attribute of an inner node only depends on the attribute of its children the empty attribute is a synthetic attribute it may be computed by a pre- or post-order traversal 10 / 59

21 Implementation Strategy attach an attribute empty to every node of the syntax tree compute the attributes in a depth-first traversal: at a leaf, we can compute the value of empty without considering other nodes the attribute of an inner node only depends on the attribute of its children the empty attribute is a synthetic attribute it may be computed by a pre- or post-order traversal in general: Definition An attribute is called synthetic if its value is always propagated upwards in the tree (in the direction leaf root) inherited if its value is always propagated downwards in the tree (in the direction root leaf) 10 / 59

22 Attribute Equations for empty In order to compute an attribute locally, we need to specify attribute equations for each node. These equations depend on the type of the node: for leafs: r i x we define empty[r] = (x ɛ). otherwise: empty[r 1 r 2 ] = empty[r 1 ] empty[r 2 ] empty[r 1 r 2 ] = empty[r 1 ] empty[r 2 ] empty[r1 ] = t empty[r 1?] = t 11 / 59

23 Specification of General Attribute Systems The empty attribute is synthetic, hence, the equations computing it can be given using structural induction. 12 / 59

24 Specification of General Attribute Systems The empty attribute is synthetic, hence, the equations computing it can be given using structural induction. In general, attribute equations combine information for children and parents. need a more flexible way to specify attribute equations that allows mentioning of parents and children use consecutive indices to refer to neighbouring attributes empty[0] : the attribute of the current node empty[i] : the attribute of the i-th child (i > 0)... in the example: x : empty[0] := (x ɛ) : empty[0] := empty[1] empty[2] : empty[0] := empty[1] empty[2] : empty[0] := t? : empty[0] := t 12 / 59

25 Observations the local attribute equations need to be evaluated using a global algorithm that knows about the dependencies of the equations in order to construct this algorithm, we need 1 a sequence in which the nodes of the tree are visited 2 a sequence within each node in which the equations are evaluated this evaluation strategy has to be compatible with the dependencies between attributes 13 / 59

26 Observations the local attribute equations need to be evaluated using a global algorithm that knows about the dependencies of the equations in order to construct this algorithm, we need 1 a sequence in which the nodes of the tree are visited 2 a sequence within each node in which the equations are evaluated this evaluation strategy has to be compatible with the dependencies between attributes We illustrate dependencies between attributes using directed graph edges: empty empty empty arrow points in the direction of information flow 13 / 59

27 Observations in order to infer an evaluation strategy, it is not enough to consider the local attribute dependencies at each node the evaluation strategy must also depend on the global dependencies, that is, on the the information flow between nodes the global dependencies thus change with each new abstract syntax tree in the example, the information flows always from the children to the parent node a post-order depth-first traversal is possible in general, variable dependencies can be much more complicated 14 / 59

28 Simultaneous Computation of Multiple Attributes Compute empty, first, next of regular expression: x : empty[0] := (x ɛ) first[0] := {x x ɛ} // (no equation for next ) root: : empty[0] := empty[1] first[0] := first[1] next[0] := next[1] := next[0] f e root n f e x n f e n 15 / 59

29 Regular Expressions: Rules for Alternative : empty[0] := first[0] := next[1] := next[2] := f e n f e n f e n 16 / 59

30 Regular Expressions: Rules for Alternative : empty[0] := empty[1] empty[2] first[0] := first[1] first[2] next[1] := next[0] next[2] := next[0] f e n f e n f e n 16 / 59

31 Regular Expressions: Rules for Concatenation : empty[0] := first[0] := next[1] := next[2] := f e. n f e n f e n 17 / 59

32 Regular Expressions: Rules for Concatenation : empty[0] := empty[1] empty[2] first[0] := first[1] (empty[1]? first[2] : ) next[1] := first[2] (empty[2]? next[0]: ) next[2] := next[0] f e. n f e n f e n 17 / 59

33 Regular Expressions: Kleene-Star and? : empty[0] := first[0] := next[1] :=? : empty[0] := first[0] := next[1] := f e * n f e? n f e n f e n 18 / 59

34 Regular Expressions: Kleene-Star and? : empty[0] := t first[0] := first[1] next[1] := first[1] next[0]? : empty[0] := t first[0] := first[1] next[1] := next[0] f e * n f e? n f e n f e n 18 / 59

35 Challenges for General Attribute Systems an evaluation strategy can only exist if for any abstract syntax tree, the dependencies between attributes are acyclic checking that no cyclic attribute dependencies can arise is DEXPTIME-complete [Jazayeri, Odgen, Rounds, 1975] Idea: Compute a set of dependency graphs for each symbol s T N. Initialize G(s) = for each s N and set S(s) = {G s } for each s T where G s is the dependency graph of s. For each rule s ::= s 1... s n of the non-terminal s N mit RHS s 1... s p extend G(s) with graphs obtained by embedding the dependency graphs G(s 1 ),... G(s n ) into the child attributes of the dependency graph of that rule. 19 / 59

36 Computing Dependencies Example: Given the grammar S ::= a b with these dependencies: h S j h i j k h i a j k h i b j k Start with G(S) =, G(a) = {k[0] j[0]}, and G(b) = {i[0] h[0]}. 20 / 59

37 Computing Dependencies Example: Given the grammar S ::= a b with these dependencies: h S j h i j k h i a j k h i b j k Start with G(S) =, G(a) = {k[0] j[0]}, and G(b) = {i[0] h[0]}. For rule S ::= a, embed G(a) into the child attributes of rule S ::= a, yielding G (S) = {h[1] h[0], h[1] k[1], j[1] i[1], j[1] j[0], k[1] j[1]} 20 / 59

38 Computing Dependencies (cont d) Result so far: G (S) = {h[1] h[0], h[1] k[1], j[1] i[1], j[1] j[0], k[1] j[1]} Given rule S ::= b, embed G(b) into the child attributes of rule S ::= a, yielding G (S) = G (S) {h[1] h[0], h[1] k[1], j[1] i[1], j[1] j[0], i[1] h[1]} { G (S) = h h i S b j j k, h h i S a j j k } 21 / 59

39 Computing Dependencies (cont d) Result so far: G (S) = {h[1] h[0], h[1] k[1], j[1] i[1], j[1] j[0], k[1] j[1]} Given rule S ::= b, embed G(b) into the child attributes of rule S ::= a, yielding G (S) = G (S) {h[1] h[0], h[1] k[1], j[1] i[1], j[1] j[0], i[1] h[1]} { G (S) = h h i S b j j k, h h i S a j j k } None of the graphs in G contain a cycle every derivable abstract syntax tree can be evaluated. 21 / 59

40 Dependencies for Recursive Rules Problem: our approach fails for grammar S ::= T a b, T ::= S with G(T) = {h[0] = j[1], j[0] = h[1], i[1] = k[0], k[1] = i[0]} Consider inserting G(T) into the initial G(S): h S j project h i j k 22 / 59

41 Dependencies for Recursive Rules Problem: our approach fails for grammar S ::= T a b, T ::= S with G(T) = {h[0] = j[1], j[0] = h[1], i[1] = k[0], k[1] = i[0]} Consider inserting G(T) into the initial G(S): h S j project h i j k projection ensures finiteness of graphs 22 / 59

42 Dependencies for Recursive Rules Problem: our approach fails for grammar S ::= T a b, T ::= S with G(T) = {h[0] = j[1], j[0] = h[1], i[1] = k[0], k[1] = i[0]} Consider inserting G(T) into the initial G(S): h S j project h i j k projection ensures finiteness of graphs maximum number of graphs for S T N and n attributes is 22 / 59

43 Dependencies for Recursive Rules Problem: our approach fails for grammar S ::= T a b, T ::= S with G(T) = {h[0] = j[1], j[0] = h[1], i[1] = k[0], k[1] = i[0]} Consider inserting G(T) into the initial G(S): h S j project h i j k projection ensures finiteness of graphs maximum number of graphs for S T N and n attributes is there are 2 (n 2) = n(n 1) possible directed edges the dependency graph of S 22 / 59

44 Dependencies for Recursive Rules Problem: our approach fails for grammar S ::= T a b, T ::= S with G(T) = {h[0] = j[1], j[0] = h[1], i[1] = k[0], k[1] = i[0]} Consider inserting G(T) into the initial G(S): h S j project h i j k projection ensures finiteness of graphs maximum number of graphs for S T N and n attributes is there are 2 (n 2) = n(n 1) possible directed edges the dependency graph of S since G(S) is a set, it contains at most 2 n(n 1) graphs 22 / 59

45 Strongly Acyclic Attribute Dependencies Problem: with larger grammars, this algorithm is too expensive Goal: find a sufficient condition for an attribute system to be acyclic. 23 / 59

46 Strongly Acyclic Attribute Dependencies Problem: with larger grammars, this algorithm is too expensive Goal: find a sufficient condition for an attribute system to be acyclic. Idea: Compute a single dependency graph for each symbol s N. Initialise G(s) with the local dependency graph of s N T. For each rule s ::= s 1 s n of s: embed the graph G(s i ) at the i-th position by project the edges of G(s i) onto a[0]... z[0] add these edges to G(s) as edges over a[i]... z[i] if the new G(s) contains a cycle, report may have cycle re-evaluate each rule until none of the graphs change anymore 23 / 59

47 Strongly Acyclic Attribute Dependencies Problem: with larger grammars, this algorithm is too expensive Goal: find a sufficient condition for an attribute system to be acyclic. Idea: Compute a single dependency graph for each symbol s N. Initialise G(s) with the local dependency graph of s N T. For each rule s ::= s 1 s n of s: embed the graph G(s i ) at the i-th position by project the edges of G(s i) onto a[0]... z[0] add these edges to G(s) as edges over a[i]... z[i] if the new G(s) contains a cycle, report may have cycle re-evaluate each rule until none of the graphs change anymore In the example, G(S) is computed as follows: h S j h S j = h S j h i j k h i j k h i j k 23 / 59

48 From Dependencies to Evaluation Strategies Possible strategies: 24 / 59

49 From Dependencies to Evaluation Strategies Possible strategies: 1 let the user define the evaluation order 24 / 59

50 From Dependencies to Evaluation Strategies Possible strategies: 1 let the user define the evaluation order 2 compute a strategy based on the dependencies: compute a linear order from the partial order defined by G(s i) if the set of dependence graphs is used, compute a different linearization depending on the children evaluate the attributes in the sequence indicated by the linear order Example: regular expression attribute grammar: in each G(s k ), we can add the following edges: e[i] n[j] e[i] f [j] f [i] n[j] any linearization now allows the following strategy: traverse AST trice, each visit computing one of e, f, g f e n 24 / 59

51 From Dependencies to Evaluation Strategies Possible strategies: 1 let the user define the evaluation order 2 compute a strategy based on the dependencies: compute a linear order from the partial order defined by G(s i) if the set of dependence graphs is used, compute a different linearization depending on the children evaluate the attributes in the sequence indicated by the linear order Example: regular expression attribute grammar: in each G(s k ), we can add the following edges: e[i] n[j] e[i] f [j] f [i] n[j] any linearization now allows the following strategy: traverse AST trice, each visit computing one of e, f, g 3 consider a fixed strategy and only allow an attribute system that can be evaluated using this strategy Question: What are good linearizations? f e n 24 / 59

52 Linear Order from Dependency Partial Order Possible automatic strategies: 25 / 59

53 Linear Order from Dependency Partial Order Possible automatic strategies: 1 demand-driven evaluation start with the evaluation of any required attribute if the equation for this attribute relies on as-of-yet unevaluated attributes, compute these recursively visits the nodes of the syntax tree on demand (following a dependency on the parent requires a pointer to the parent) 25 / 59

54 Linear Order from Dependency Partial Order Possible automatic strategies: 1 demand-driven evaluation start with the evaluation of any required attribute if the equation for this attribute relies on as-of-yet unevaluated attributes, compute these recursively visits the nodes of the syntax tree on demand (following a dependency on the parent requires a pointer to the parent) 2 evaluation in passes minimise the number of visits to each node organise the evaluation of the tree in passes for each pass, pre-compute a strategy to visit the nodes together with a local strategy for evaluation within each node type 25 / 59

55 Linear Order from Dependency Partial Order Possible automatic strategies: 1 demand-driven evaluation start with the evaluation of any required attribute if the equation for this attribute relies on as-of-yet unevaluated attributes, compute these recursively visits the nodes of the syntax tree on demand (following a dependency on the parent requires a pointer to the parent) 2 evaluation in passes minimise the number of visits to each node organise the evaluation of the tree in passes for each pass, pre-compute a strategy to visit the nodes together with a local strategy for evaluation within each node type consider example for demand-driven evaluation 25 / 59

56 Example for Demand-Driven Evaluation Compute next at the leaves of a(a b) in the expressionn ((a b) a(a b)): : next[1] := next[0] next[2] := next[0] : next[1] := first[2] (empty[2]? next[0]: ) next[2] := next[0]. *. a 2 a b a b 26 / 59

57 Example for Demand-Driven Evaluation Compute next at the leaves of a(a b) in the expressionn ((a b) a(a b)): : next[1] := next[0] next[2] := next[0] : next[1] := first[2] (empty[2]? next[0]: ) next[2] := next[0]. n *. n a 2 n a b a n b 4 n 26 / 59

58 Example for Demand-Driven Evaluation Compute next at the leaves of a(a b) in the expressionn ((a b) a(a b)): : next[1] := next[0] next[2] := next[0] : next[1] := first[2] (empty[2]? next[0]: ) next[2] := next[0]. n *. n n e f a 2 n a 0 b 1 e f a n e f b 3 4 n 26 / 59

59 Example for Demand-Driven Evaluation Compute next at the leaves of a(a b) in the expressionn ((a b) a(a b)): : next[1] := next[0] next[2] := next[0] : next[1] := first[2] (empty[2]? next[0]: ) next[2] := next[0]. n *. n n e f a 2 n a b a 0 e 1 3 f n e f b 4 n 26 / 59

60 Demand-Driven Evaluation Observations only required attributes are evaluated the evaluation sequence depends in general on the actual syntax tree the algorithm must track which attributes it has already evaluated the algorithm may visit nodes more often than necessary each node must contain a pointer to its parent the algorithm is not local 27 / 59

61 Demand-Driven Evaluation Observations only required attributes are evaluated the evaluation sequence depends in general on the actual syntax tree the algorithm must track which attributes it has already evaluated the algorithm may visit nodes more often than necessary each node must contain a pointer to its parent the algorithm is not local approach only beneficial in principle: evaluation strategy is dynamic: difficult to debug computation of all attributes is often cheaper usually all attributes in all nodes are required 27 / 59

62 Demand-Driven Evaluation Observations only required attributes are evaluated the evaluation sequence depends in general on the actual syntax tree the algorithm must track which attributes it has already evaluated the algorithm may visit nodes more often than necessary each node must contain a pointer to its parent the algorithm is not local approach only beneficial in principle: evaluation strategy is dynamic: difficult to debug computation of all attributes is often cheaper usually all attributes in all nodes are required perform evaluation in passes 27 / 59

63 Evaluation in Passes Idea: traverse the syntax tree several times; each time, evaluate those equations a[i a ] = f (b[i b ],... z[i z ]) whose attributes b[i b ],... z[i z ] are already evaluated 28 / 59

64 Evaluation in Passes Idea: traverse the syntax tree several times; each time, evaluate those equations a[i a ] = f (b[i b ],... z[i z ]) whose attributes b[i b ],... z[i z ] are already evaluated For a strongly acyclic attribute system: the local dependencies in G(s i ) at s i define a sequence in which children can be visited so that at least one attribute can be evaluated after the visit of s i in each pass through the tree at least one more attribute is evaluated requires at most n passes for evaluating n attributes since a traversal strategy exists for evaluating one attribute, it might be possible to find a strategy to evaluate more attributes optimisation problem?! the ability to group attributes depends on the design of the equation system 28 / 59

65 Evaluation in Passes Idea: traverse the syntax tree several times; each time, evaluate those equations a[i a ] = f (b[i b ],... z[i z ]) whose attributes b[i b ],... z[i z ] are already evaluated For a strongly acyclic attribute system: the local dependencies in G(s i ) at s i define a sequence in which children can be visited so that at least one attribute can be evaluated after the visit of s i in each pass through the tree at least one more attribute is evaluated requires at most n passes for evaluating n attributes since a traversal strategy exists for evaluating one attribute, it might be possible to find a strategy to evaluate more attributes optimisation problem?! the ability to group attributes depends on the design of the equation system... in the example: empty and first can be computed together next must be computed in a separate pass 28 / 59

66 Let user chose which attributes should be evaluated together. 28 / 59 Evaluation in Passes Idea: traverse the syntax tree several times; each time, evaluate those equations a[i a ] = f (b[i b ],... z[i z ]) whose attributes b[i b ],... z[i z ] are already evaluated For a strongly acyclic attribute system: the local dependencies in G(s i ) at s i define a sequence in which children can be visited so that at least one attribute can be evaluated after the visit of s i in each pass through the tree at least one more attribute is evaluated requires at most n passes for evaluating n attributes since a traversal strategy exists for evaluating one attribute, it might be possible to find a strategy to evaluate more attributes optimisation problem?! the ability to group attributes depends on the design of the equation system... in the example: empty and first can be computed together next must be computed in a separate pass

67 Implementing Local Evaluation Consider example: numbering the leafs of a syntax tree. *. a 2 a b a b 29 / 59

68 Implementing Numbering of Leafs Idea: use helper attributes pre and post in pre we pass the value of the last leaf down (inherited attribute) in post we pass the value of the last leaf up (synthetic attribute) root: pre[0] := 0 pre[1] := pre[0] post[0] := post[1] node: pre[1] := pre[0] pre[2] := post[1] post[0] := post[2] leaf: post[0] := pre[0] / 59

69 The Local Attribute Dependencies pre post pre post pre post pre post the attribute system is apparently strongly acyclic 31 / 59

70 The Local Attribute Dependencies pre post pre post pre post pre post the attribute system is apparently strongly acyclic each node computes the inherited attributes before descending into a child node (corresponding to a pre-order traversal) the synthetic attributes after returning from a child node (corresponding to post-order traversal) 31 / 59

71 The Local Attribute Dependencies pre post pre post pre post pre post the attribute system is apparently strongly acyclic each node computes the inherited attributes before descending into a child node (corresponding to a pre-order traversal) the synthetic attributes after returning from a child node (corresponding to post-order traversal) if all attributes can be computed in a single depth-first traversal that proceeds from left- to right (with pre- and post-order evaluation) then we call this attribute system L-attributed. 31 / 59

72 L-attributed Definition An attribute system is L-attributed, if for all productions s ::= s 1... s n every inherited attribute of s j where 1 j n only depends on 1 the attributes of s 1, s 2,... s j 1 and 2 the inherited attributes of s. 32 / 59

73 L-attributed Definition An attribute system is L-attributed, if for all productions s ::= s 1... s n every inherited attribute of s j where 1 j n only depends on 1 the attributes of s 1, s 2,... s j 1 and 2 the inherited attributes of s. Origin: the attributes of an L-attributed grammar can be evaluated during parsing important if no syntax tree is required or if error messages should be emitted while parsing example: pocket calculator 32 / 59

74 L-attributed Definition An attribute system is L-attributed, if for all productions s ::= s 1... s n every inherited attribute of s j where 1 j n only depends on 1 the attributes of s 1, s 2,... s j 1 and 2 the inherited attributes of s. Origin: the attributes of an L-attributed grammar can be evaluated during parsing important if no syntax tree is required or if error messages should be emitted while parsing example: pocket calculator L-attributed grammars have a fixed evaluation strategy: a single depth-first traversal in general: partition all attributes into A = A 1... A n such that for all attributes in A i the attribute system is L-attributed perform a depth-first traversal for each attribute set A i craft attribute system in a way that they can be partitioned into few L-attributed sets 32 / 59

75 Practical Applications symbol tables, type checking/inference, and simple code generation can all be specified using L-attributed grammars 33 / 59

76 Practical Applications symbol tables, type checking/inference, and simple code generation can all be specified using L-attributed grammars most applications annotate syntax trees with additional information 33 / 59

77 Practical Applications symbol tables, type checking/inference, and simple code generation can all be specified using L-attributed grammars most applications annotate syntax trees with additional information the nodes in a syntax tree often have different types that depends on the non-terminal that the node represents 33 / 59

78 Practical Applications symbol tables, type checking/inference, and simple code generation can all be specified using L-attributed grammars most applications annotate syntax trees with additional information the nodes in a syntax tree often have different types that depends on the non-terminal that the node represents the different types of non-terminals are characterised by the set of attributes with which they are decorated 33 / 59

79 Practical Applications symbol tables, type checking/inference, and simple code generation can all be specified using L-attributed grammars most applications annotate syntax trees with additional information the nodes in a syntax tree often have different types that depends on the non-terminal that the node represents the different types of non-terminals are characterised by the set of attributes with which they are decorated example: a statement may have two attributes containing valid identifiers: one ingoing (inherited) set and one outgoing (synthesised) set; in contrast, an expression only has an ingoing set 33 / 59

80 Implementation of Attribute Systems In object-oriented languages, use a visitor pattern: class with a method for every non-terminal in the grammar public abstract class Regex { public abstract void accept(visitor v); } by overwriting one of the following methods, we implement an attribute-specific evaluation public interface Visitor { public void visit(dot re) { re.children(this); } public void visit(bar re) { re.children(this); }... public void visit(token tok) {} } we pre-define a depth-first traversal of the syntax tree public class OrEx extends Regex { RegEx l,r; public void accept(visitor v) { v.visit(this); } public void children(visitor v) { l.accept(v); r.accept(v); }} 34 / 59

81 Semantic Analysis Chapter 2: Symbol Tables 35 / 59

82 Symbol Tables Consider the following Java code: void foo() { int A; void bar() { double A; A = 0.5; write(a); } A = 2; bar(); write(a); } within the body of bar the definition of A is shadowed by the local definition each declaration of a variable v requires the compiler to set aside some memory for v; in order to perform an access to v, we need to know to which declaration the access is bound we consider only static binding, where the definition of a name v is in scope at all program points within the block however, the binding is not visible within local declarations of v are in scope 36 / 59

83 Scope of Identifiers void foo() { int A; void bar() { double A; A = 0.5; write(a); } A = 2; bar(); write(a); scope of int A } 37 / 59

84 Scope of Identifiers void foo() { int A; void bar() { double A; A = 0.5; write(a); scope of double A } A = 2; bar(); write(a); } 37 / 59

85 Scope of Identifiers void foo() { int A; void bar() { double A; A = 0.5; write(a); scope of double A } A = 2; bar(); write(a); } administration of identifiers can be quite complicated / 59

86 Visibility Rules in Object-Oriented Languages 1 public class Foo { 2 int x = 17; 3 protected int y = 5; 4 private int z = 42; 5 public int b() { return 1; } 6 } 7 class Bar extends Foo { 8 protected double y = 0.5; 9 public int b(int a) 10 { return a+x; } 11 } Modifier Class Package Subclass World public protected no modifier private Observations: 38 / 59

87 Visibility Rules in Object-Oriented Languages 1 public class Foo { 2 int x = 17; 3 protected int y = 5; 4 private int z = 42; 5 public int b() { return 1; } 6 } 7 class Bar extends Foo { 8 protected double y = 0.5; 9 public int b(int a) 10 { return a+x; } 11 } Modifier Class Package Subclass World public protected no modifier private Observations: private member z is only visible in methods of class Foo protected member y is visible in the same package and in sub-class Bar, but here it is shadowed by double y Bar does not compile if it is not in the same package as Foo methods b with the same name are different if their arguments differ static overloading 38 / 59

88 Dynamic Resolution of Functions public class Foo { protcted int foo() { return 1; } } class Bar extends Foo { protected int foo() { return 2; } public int test(boolean b) { Foo x = (b)? new Foo() : new Bar(); return x.foo(); } } Observations: 39 / 59

89 Dynamic Resolution of Functions public class Foo { protcted int foo() { return 1; } } class Bar extends Foo { protected int foo() { return 2; } public int test(boolean b) { Foo x = (b)? new Foo() : new Bar(); return x.foo(); } } Observations: the type of x is Foo or Bar, depending on the value of b x.foo() either calls foo in line 2 or in line 5 39 / 59

90 Dynamic Resolution of Functions public class Foo { protcted int foo() { return 1; } } class Bar extends Foo { protected int foo() { return 2; } public int test(boolean b) { Foo x = (b)? new Foo() : new Bar(); return x.foo(); } } Observations: the type of x is Foo or Bar, depending on the value of b x.foo() either calls foo in line 2 or in line 5 this decision is made at run-time and has nothing to do with name resolution 39 / 59

91 Resolving Identifiers Observation: each identifier in the AST must be translated into a memory access 40 / 59

92 Resolving Identifiers Observation: each identifier in the AST must be translated into a memory access Problem: for each identifier, find out what memory needs to be accessed by providing rapid access to its declaration Idea: 1 rapid access: replace every identifier by a unique name, namely an integer integers as keys: comparisons of integers is faster replacing various identifiers with number saves memory 40 / 59

93 Resolving Identifiers Observation: each identifier in the AST must be translated into a memory access Problem: for each identifier, find out what memory needs to be accessed by providing rapid access to its declaration Idea: 1 rapid access: replace every identifier by a unique name, namely an integer integers as keys: comparisons of integers is faster replacing various identifiers with number saves memory 2 link each usage of a variable to the declaration of that variable track data structures to distinguish declared variables and visible variables for languages without explicit declarations, create declarations when a variable is first encountered 40 / 59

94 (1) Replace each Occurrence with a Number Rather than handling strings, we replace each string with a unique number. Idea for Algorithm: Input: a sequence of strings Output: 1 sequence of numbers 2 table that allows to retrieve the string that corresponds to a number Apply this algorithm on each identifier in the scanner. 41 / 59

95 Example for Applying this Algorithm Input: Peter Piper picked a peck of pickled peppers If Peter Piper picked a peck of pickled peppers wheres the peck of pickled peppers Peter Piper picked Output: 42 / 59

96 Example for Applying this Algorithm Input: Peter Piper picked a peck of pickled peppers If Peter Piper picked a peck of pickled peppers wheres the peck of pickled peppers Peter Piper picked Output: and 0 Peter 1 Piper 2 picked 3 a 4 peck 5 of 6 pickled 7 peppers 8 If 9 wheres 10 the 42 / 59

97 Implementing the Algorithm: Specification Idea: implement a partial map: S : String int use a counter variable int count = 0; to track the number of different identifiers found so far We thus define a function int getindex(string w): int getindex(string w) { if (S (w) undefined) { S = S {w count}; return count++; else return S (w); } 43 / 59

98 Data Structures for Partial Maps possible data structures: list of pairs (w, i) String int : O(1) O(n) too expensive 44 / 59

99 Data Structures for Partial Maps possible data structures: list of pairs (w, i) String int : O(1) O(n) too expensive balanced trees : O(log(n)) O(log(n)) too expensive 44 / 59

100 Data Structures for Partial Maps possible data structures: list of pairs (w, i) String int : O(1) O(n) too expensive balanced trees : O(log(n)) O(log(n)) hash tables : O(1) O(1) too expensive on average 44 / 59

101 Data Structures for Partial Maps possible data structures: list of pairs (w, i) String int : O(1) O(n) too expensive balanced trees : O(log(n)) O(log(n)) hash tables : O(1) O(1) too expensive on average caveat: we will see that the handling of scoping requires additional operations that are hard to implement with hash tables 44 / 59

102 An Implementation using Hash Tables allocated an array M of sufficient size m choose a hash function H : String [0, m 1] with the following properties: H(w) is cheap to compute H distributes the occurring words equally over [0, m 1] Possible choices ( x = x 0,... x r 1 ): H 0 ( x) = H 1 ( x) = H 2 ( x) = (x 0 + x r 1 ) % m ( r 1 i=0 x i p i ) % m (x 0 + p (x 1 + p (... + p x r 1 ))) % m f"ur eine Primzahl p (z.b. 31) We store the pair (w, i) in a linked list located at M[H(w)] 45 / 59

103 Computing a Hash Table for the Example With m = 7 and H 0 we obtain: 0 If 8 the pickled of 5 6 peck 4 wheres 9 pickled peppers Piper 1 Peter 0 a 3 6 In order to find the index for the word w, we need to compare w with all words x for which H(w) = H(x) 46 / 59

104 Resolving Identifiers: (2) Symbol Tables Check for the correct usage of variables: Traverse the syntax tree in a suitable sequence, such that each definition is visited before its use the currently visible definition is the last one visited for each identifier, we manage a stack of scopes if we visit a declaration of an identifier, we push it onto the stack upon leaving the scope, we remove it from the stack if we visit a usage of an identifier, we pick the top-most declaration from its stack if the stack of the identifier is empty, we have found an error 47 / 59

105 Example: A Table of Stacks { } int a, b; // V, W b = 5; if (b>3) { int a, c; // X, Y a = 3; c = a + 1; b = c; } else { int c; c = a + 1; b = c; } b = a + b; // Z 0 a 1 b 2 c 0 a 1 b 2 c 0 a 1 b 2 c 0 a 1 b 2 c 48 / 59

106 Example: A Table of Stacks { } int a, b; // V, W b = 5; if (b>3) { int a, c; // X, Y a = 3; c = a + 1; b = c; } else { int c; c = a + 1; b = c; } b = a + b; // Z 0 a 1 b 2 c 0 a 1 b 2 c 0 a 1 b 2 c 0 a 1 b 2 c V W X, V W Y V W Z V W 48 / 59

107 Resolving: Rewriting the Syntax Tree 0 d int d declaration node b basic block a assignment b b b d b a b 1 int if = > d 0 int b d 2 int b { int a, b; // V, W b = 5; if (b>3) { int a, c; // X, Y a = 3; c = a + 1; b = c; } else { int c; // Z c = a + 1; b = c; } b = a + b; } b a = d int b b a = / 59

108 Resolving: Rewriting the Syntax Tree 0 d int d declaration node b basic block a assignment b b b d b a b 1 int if = > d 0 int b d 2 int b { int a, b; // V, W b = 5; if (b>3) { int a, c; // X, Y a = 3; c = a + 1; b = c; } else { int c; // Z c = a + 1; b = c; } b = a + b; } b a = d int b b a = / 59

109 Alternative Resolution of Visibility resolving identifiers can be done using an L-attributed grammar equation system for basic block must add and remove identifiers 50 / 59

110 Alternative Resolution of Visibility resolving identifiers can be done using an L-attributed grammar equation system for basic block must add and remove identifiers when using a list to store the symbol table, storing a marker indicating the old head of the list is sufficient a b in front of if-statement 50 / 59

111 Alternative Resolution of Visibility resolving identifiers can be done using an L-attributed grammar equation system for basic block must add and remove identifiers when using a list to store the symbol table, storing a marker indicating the old head of the list is sufficient a b in front of if-statement a c a b then-branch 50 / 59

112 Alternative Resolution of Visibility resolving identifiers can be done using an L-attributed grammar equation system for basic block must add and remove identifiers when using a list to store the symbol table, storing a marker indicating the old head of the list is sufficient a b in front of if-statement a c a b then-branch c a b else-branch 50 / 59

113 Alternative Resolution of Visibility resolving identifiers can be done using an L-attributed grammar equation system for basic block must add and remove identifiers when using a list to store the symbol table, storing a marker indicating the old head of the list is sufficient a b a c a b c a b in front of if-statement then-branch else-branch instead of lists of symbols, it is possible to use a list of hash tables more efficient in large, shallow programs 50 / 59

114 Alternative Resolution of Visibility resolving identifiers can be done using an L-attributed grammar equation system for basic block must add and remove identifiers when using a list to store the symbol table, storing a marker indicating the old head of the list is sufficient a b a c a b c a b in front of if-statement then-branch else-branch instead of lists of symbols, it is possible to use a list of hash tables more efficient in large, shallow programs a more elegant solution is to use a persistent tree in which an update returns a new tree but leaves all old references to the tree unchanged a persistent tree t can be passed down into a basic block where new elements may be added; after examining the basic block, the analysis proceeds with the unchanged t 50 / 59

115 Forward Declarations Most programming language admit the definition of recursive data types and/or recursive functions. a recursive definition needs to mention a name that is currently being defined or that will be defined later on old-fashion programming languages require that these cycles are broken by a forward declaration 51 / 59

116 Forward Declarations Most programming language admit the definition of recursive data types and/or recursive functions. a recursive definition needs to mention a name that is currently being defined or that will be defined later on old-fashion programming languages require that these cycles are broken by a forward declaration Consider the declaration of an alternating linked list in C: struct list1; struct list0 { int info; struct list1* next; } struct list1 { double info; struct list0* next; } 51 / 59

117 Forward Declarations Most programming language admit the definition of recursive data types and/or recursive functions. a recursive definition needs to mention a name that is currently being defined or that will be defined later on old-fashion programming languages require that these cycles are broken by a forward declaration Consider the declaration of an alternating linked list in C: struct list1; struct list0 { int info; struct list1* next; } struct list1 { double info; struct list0* next; } the first declaration struct list1; is a forward declaration. 51 / 59

118 Forward Declarations Most programming language admit the definition of recursive data types and/or recursive functions. a recursive definition needs to mention a name that is currently being defined or that will be defined later on old-fashion programming languages require that these cycles are broken by a forward declaration Consider the declaration of an alternating linked list in C: struct list1; struct list0 { int info; struct list1* next; } struct list1 { double info; struct list0* next; } the first declaration struct list1; is a forward declaration. Alternative: automatically add a forward declaration into the symbol table and check that all these entries have been declared by the time the symbol goes out of scope 51 / 59

119 Declarations of Function Names An analogous mechanism is need for (recursive) functions: in case a recursive function merely calls itself, it is sufficient to add the name of a function to the symbol table before visiting its body; example: int fac(int i) { return i*fac(i-1); } 52 / 59

120 Declarations of Function Names An analogous mechanism is need for (recursive) functions: in case a recursive function merely calls itself, it is sufficient to add the name of a function to the symbol table before visiting its body; example: int fac(int i) { return i*fac(i-1); } for mutually recursive functions all function names at that level have to be entered (or declared as forward declaration). Example ML and C: fun odd 0 = false odd 1 = true odd x = even (x-1) and even 0 = true even 1 = false even x = odd (x-1) int even(int x); int odd(int x) { return (x==0? 0 : (x==1? 1 : even(x-1))); } int even(int x) { return (x==0? 1 : (x==1? 0 : odd(x-1))); } 52 / 59

121 Overloading of Names The problem of using names before their declarations are visited is also common in object-oriented languages: for object-oriented languages with inheritance, the base class must be visited before the derived class in order to determine if declarations in the derived class are correct in addition, the signature of methods needs to be considered () qualify a function symbol with its parameters may also require type checking 53 / 59

122 Overloading of Names The problem of using names before their declarations are visited is also common in object-oriented languages: for object-oriented languages with inheritance, the base class must be visited before the derived class in order to determine if declarations in the derived class are correct in addition, the signature of methods needs to be considered () qualify a function symbol with its parameters may also require type checking Once the names are resolved, other semantic analyses can be applied such as type checking or type inference. 53 / 59

123 Multiple Classes of Identifiers Some programming languages distinguish between several classes of identifiers: C: variable names and type names Java: classes, methods and fields Haskell: type names, constructors, variables, infix variables and -constructors 54 / 59

124 Multiple Classes of Identifiers Some programming languages distinguish between several classes of identifiers: C: variable names and type names Java: classes, methods and fields Haskell: type names, constructors, variables, infix variables and -constructors In some cases a declaration may change the class of an identifier; for example, a typedef in C: the scanner generates a different token, based on the class into which an identifier falls the parser informs the scanner as soon as it sees a declaration that changes the class of an identifier the parser generates a syntax tree that depends on the semantic interpretation of the input so far 54 / 59

125 Multiple Classes of Identifiers Some programming languages distinguish between several classes of identifiers: C: variable names and type names Java: classes, methods and fields Haskell: type names, constructors, variables, infix variables and -constructors In some cases a declaration may change the class of an identifier; for example, a typedef in C: the scanner generates a different token, based on the class into which an identifier falls the parser informs the scanner as soon as it sees a declaration that changes the class of an identifier the parser generates a syntax tree that depends on the semantic interpretation of the input so far the interaction between scanner and parser is problematic! 54 / 59

126 Fixity-Declarations in Haskell Haskell allows for arbitrary binary operators over (?!^& =+-_*/) +. In Standard Library of Haskell: infixr 8 ^ infixl 7 *,/ infixl 6 +,- infix 4 ==,/= The grammar is generic: Exp 0 ::= Exp 0 LOp 0 Exp 1 Exp 1 ROp 0 Exp 0 Exp 1 Op 0 Exp 1 Exp 1. Exp 9 ::= Exp 9 LOp 9 Exp Exp ROp 9 Exp 9 Exp Op 9 Exp Exp Exp ::= ident num ( Exp 0 ) 55 / 59

127 Fixity-Declarations in Haskell Haskell allows for arbitrary binary operators over (?!^& =+-_*/) +. In Standard Library of Haskell: infixr 8 ^ infixl 7 *,/ infixl 6 +,- infix 4 ==,/= The grammar is generic: Exp 0 ::= Exp 0 LOp 0 Exp 1 Exp 1 ROp 0 Exp 0 Exp 1 Op 0 Exp 1 Exp 1. Exp 9 ::= Exp 9 LOp 9 Exp Exp ROp 9 Exp 9 Exp Op 9 Exp Exp Exp ::= ident num ( Exp 0 ) parser enters an infix declaration into a table scanner checks table and produces: operator - turns into token LOp 6. operator * turns into token LOp 7. operator == turns into token Op 4. etc. parser recognizes 3-4*5-6 as (3-(4*5))-6 55 / 59

128 Fixity-Declarations in Haskell: Observations Troublesome changes: the scanner has a state which the parser determines grammar no longer context-free, needs global data structure 56 / 59

129 Fixity-Declarations in Haskell: Observations Troublesome changes: the scanner has a state which the parser determines grammar no longer context-free, needs global data structure a code fragment may have several semantics syntactic correctness may depend on imported modules 56 / 59

130 Fixity-Declarations in Haskell: Observations Troublesome changes: the scanner has a state which the parser determines grammar no longer context-free, needs global data structure a code fragment may have several semantics syntactic correctness may depend on imported modules error messages difficult to understand 56 / 59

131 Fixity-Declarations in Haskell: Observations Troublesome changes: the scanner has a state which the parser determines grammar no longer context-free, needs global data structure a code fragment may have several semantics syntactic correctness may depend on imported modules error messages difficult to understand The GHC Haskell Compiler parses all operators as LOp 0 and transforms the AST afterwards. 56 / 59

132 Type Synonyms and Variables in C The C grammar distinguishes typedef-name and identifier. Consider the following declarations: typedef struct { int x,y } point_t; point_t origin; Relevant C grammar: declaration (declaration-specifier) + declarator ; declaration-specifier static volatile typedef void char char typedef-name declarator identifier 57 / 59

133 Type Synonyms and Variables in C The C grammar distinguishes typedef-name and identifier. Consider the following declarations: typedef struct { int x,y } point_t; point_t origin; Relevant C grammar: declaration (declaration-specifier) + declarator ; declaration-specifier static volatile typedef void char char typedef-name declarator identifier Problem: parser adds point_t to the table of types when the declaration is reduced 57 / 59

134 Type Synonyms and Variables in C The C grammar distinguishes typedef-name and identifier. Consider the following declarations: typedef struct { int x,y } point_t; point_t origin; Relevant C grammar: declaration (declaration-specifier) + declarator ; declaration-specifier static volatile typedef void char char typedef-name declarator identifier Problem: parser adds point_t to the table of types when the declaration is reduced parser state has at least one look-ahead token 57 / 59