Repeating patterns of boilerplate code (like parent/child navigation, Visitor pattern implementation, factory methods for parser tree construction)

Transcription

1 Gymnast overview Study of org.eclipse.jdt.core.dom.astnode and subclasses reveals Repeating patterns of boilerplate code (like parent/child navigation, Visitor pattern implementation, factory methods for parser tree construction) Code is strongly tied to generated Java parser (roughly one ASTNode class for each Java grammar rule) Question: Could we write a language grammar and generate both ASTNode class hierarchy with ASTVisitor implementation Parser for language that builds trees with the ASTNodes Answer: Gymnast tool was developed to do this Gymnast was used to develop Emfatic ( Gymnast also bootstraps itself (Note: Gymnast = Generate Your Mega Nice AST ) 1 Tech Exchange Panel: Language Toolkit s 2005 by IBM; made available under the EPL v1.0

2 Gymnast philosophy Gymnast grammatical rules are intentionally simple No action syntax tree building actions are inserted for you let the parser build a tree and then do analysis and processing on the tree (can use the generated ASTVisitor for this) Grammatical patterns are somewhat limited so we can sensibly map to class relationships for code generation Identifier names in grammar map to class and method names rule names map to class names rule references can be labeled to control getx() method names Generated languages share common classes (ASTNode, IParser) Common tooling can be written by targeting the shared classes syntactic error marking and parse tree view - for free basic outline view and editor with basic syntax coloring - pretty easy 2 Tech Exchange Panel: Language Toolkit s 2005 by IBM; made available under the EPL v1.0

3 More info The Gymnast tool functionality is embedded in Emfatic To try Gymnast, get the Emfatic alphaworks release ( and try some of the things described in these slides Get the sample language plugin org.example.gymnast.game which shows: Simple language definition (game.ast, GameLexer.g) Editor with syntax coloring and text hover Outline view Usage of generated ASTNodeVisitor class References See SableCC ( for another take on the idea of generating ASTNode class frameworks and parsers from a single grammar. 3 Tech Exchange Panel: Language Toolkit s 2005 by IBM; made available under the EPL v1.0

4 Gymnast information flow 4 Tech Exchange Panel: Language Toolkit s 2005 by IBM; made available under the EPL v1.0

5 Gymnast example grammar 5 Tech Exchange Panel: Language Toolkit s 2005 by IBM; made available under the EPL v1.0

6 Gymnast example grammar 6 Tech Exchange Panel: Language Toolkit s 2005 by IBM; made available under the EPL v1.0

7 LDT views (showing simple Java-like language) 7 Tech Exchange Panel: Language Toolkit s 2005 by IBM; made available under the EPL v1.0

8 Token Rules Examples: token tok1 : "foo" ; // String literals token tok2 : "public" "private" "protected" ; token tok3 : PLUS MINUS; // Lexer token references Notes: Combining token rules would be nice (but not supported yet) e.g. token modifiers : tok2 abstract" ; 8 Tech Exchange Panel: Language Toolkit s 2005 by IBM; made available under the EPL v1.0

9 Sequence Rules Examples: sequence seq1 : LPAREN rule RPAREN ; // simple sequence sequence seq2 : plus1=plus plus2=plus; // using labels sequence seq3 : A (B C)? ; // optional parts Notes: Labels are used in method names (i.e. getplus1(), getplus2()) Optional parts can be nested Use of optional (?) operator means getb() and getc() can return null 9 Tech Exchange Panel: Language Toolkit s 2005 by IBM; made available under the EPL v1.0

10 List Rules Examples: list list1 : T* ; // simple list list list2 : t1=t (COMMA tn=t)*; // list with separator Notes: Can use * (0..n) or + (1..n) I d like to change the mapping so that: X (SEP X)* actually means: (X (SEP X)*)? or somehow get that effect Code generation needs improvement Normal children and separators are not distinguished there should be typesafe getx(int index), getsep(int index) methods 10 Tech Exchange Panel: Language Toolkit s 2005 by IBM; made available under the EPL v1.0

11 Abstract (alternation) Rules Examples: abstract type : class iface ; abstract type : (modifiers) class iface ; Notes: Produces inheritance pattern class and iface extend abstract class type Second form matches parenthesized sequence and passes result to the matched alternative. (This turns out to be a useful pattern with Antlr) 11 Tech Exchange Panel: Language Toolkit s 2005 by IBM; made available under the EPL v1.0