Grammar Composition & Extension

Size: px

Start display at page:

Download "Grammar Composition & Extension"

Posy Carson
6 years ago
Views:

1 Grammar Composition & Extension Symposium on Language Composition and Modularity Vadim Zaytsev, SWAT, CWI 2012

2 Introduction to grammarware

3 What is a grammar? Structural description in software systems Description of structures used in software systems P. Klint, R. Lämmel, C. Verhoef, Toward an Engineering Discipline for Grammarware. TOSEM 2005.

4 Grammar use case Abstract use case syntax definition exchange format interaction protocol data model domain model metamodel Concrete use case parsing serialisation renovation refactoring static analysis reengineering P. Klint, R. Lämmel, C. Verhoef, Toward an Engineering Discipline for Grammarware. TOSEM 2005.

slicer API XMLware Modelware Language workbench

5 Grammarware Parser Compiler Interpreter Pretty-printer Scanner Browser Static checker Structural editor IDE DSL framework Preprocessor Postprocessor Model checker Refactorer Code slicer API XMLware Modelware Language workbench Reverse engineering tool Benchmark Recommender Renovation tool

6 Grammars everywhere

7 Grammarware

8 Grammarware

9 Grammarware

10 Grammarware

11 Grammarware

12 Grammarware

13 Grammar decomposition

Preprocessing syntax comments directives Base syntax Syntax highlighting

14 CWI Quo usque tandem? Lexical syntax: character level (tokenisation) block level (indentation) Preprocessing syntax comments directives Base syntax Syntax highlighting Processing order Filtering / disambiguation Error handling Tree construction AST format

15 Grammar composition

16 Behind the Screen / Unknown Chaplin Fair (re)use, picture is the courtesy of chaplin.bfi.org.uk.

17 Grammar composition syntax A = B+; syntax B = b ; syntax D = C*; syntax C = c ; syntax A = B+; syntax D = C*; syntax B = b ; syntax C = c ;

18 Grammar composition syntax A = B C; syntax B = b ; syntax D = C B; syntax C = c ; syntax A = B C; syntax D = C B; syntax B = b ; syntax C = c ;

19 Grammar composition syntax A = B C; syntax B = b ; syntax C = c ; syntax A = B C; syntax B = b ; syntax C = c ; { bc }

20 Grammar composition syntax A = B C; syntax B = b ; syntax C = c ; syntax A = C B; syntax B = c ; syntax C = b ; { bc, cb }

21 Grammar composition syntax A = B CA; syntax B = b ; syntax C = c ; syntax A = B CB; syntax B = b ; syntax C = c +; n { c b }, n=0,1,2,

22 Grammar composition syntax A = B BA; syntax B = b ; syntax A = AB B; syntax B = b ; n { b }, n=1,2,

23 Grammar composition syntax A = B C; syntax C = D+; syntax = ε; syntax C = E; syntax E = B+; syntax = ε; XML

24 Grammar composition syntax A = B C { }; syntax A = B C { };?

25 Grammar composition syntax A = B C; syntax B = b ; syntax C = c ; syntax C = D E; syntax C c ;?

26 Adjacent topics: pgt gc ngt no

27 Programmable Grammar Transformations

28 Programmable G xformation Ad hoc megamodel shown at IPA Spring Days

29 Programmable G xformation Operator known semantics, well-defined algorithm rename, fold, factor, inject, remove, Ad hoc megamodel shown at IPA Spring Days

30 Programmable G xformation Arguments what exactly to rename/factor/inject/? Ad hoc megamodel shown at IPA Spring Days

31 Programmable G xformation Input grammar determines applicability Ad hoc megamodel shown at IPA Spring Days

32 Programmable G xformation Ad hoc megamodel shown at IPA Spring Days

33 XBGF Operator Suite Semantic-preserving operators fold, unfold, extract, inline, massage, factor, deyaccify, (Some) semantic-preserving operators permute, abstractize, concretize, designate, anonymize Language-increasing operators add, appear, widen, upgrade, unite Language-decreasing operators remove, disappear, narrow, downgrade Revising operators redefine, inject, project, replace, V. Zaytsev, BGF Transformation Operator Suite v.1.0, online, 2010.

34 References R. Lämmel. Grammar Adaptation. FME, LNCS 2021: R. Lämmel, G. Wachsmuth. Transformation of SDF Syntax Definitions in the ASF+SDF Meta- Environment. LDTA, ENTCS T. R. Dean, J. R. Cordy, A. J. Malton, K. A. Schneider. Grammar Programming in TXL. SCAM R. Lämmel, Transformations Everywhere. SCP 52(1 3):1 8, P. Klint, R. Lämmel, C. Verhoef. Toward an Engineering Discipline for Grammarware. ACM TOSEM 14(3): , V. Zaytsev, BGF Transformation Operator Suite v.1.0, online, V. Zaytsev, Recovery, Convergence and Documentation of Languages. PhD, R. Lämmel, V. Zaytsev, Recovering Grammar Relationships for the Java Language Specification, SQJ 19(2):

35 Guided Grammar Convergence

38 The most trivial case Equal grammars Algebraically equivalent grammars Nothing to do here V. Zaytsev, Guided Grammar Convergence, CoRR abs/ , draft for 2013.

39 Structural resolution Nonterminal vs. value A vs. string Sequence permutations A B B A vs. B B A A Lists of symbols A* vs. A+ Separator lists irrelevant V. Zaytsev, Guided Grammar Convergence, CoRR abs/ , draft for 2013.

40 Nominal resolution Production rule in the master grammar Production signature p 1 =p(, program, +(function)) {function, +} p 2 =p(, function, seq([str, +(str), expr])) {expr, 1, str, 1+} p 3 =p(, expr, str) {str, 1} p 4 =p(, expr, int) {int, 1} p 5 =p(, expr, apply) {apply, 1} p 6 =p(, expr, binary) {binary, 1} p 7 =p(, expr, cond) {cond, 1} p 8 =p(, apply, seq([str, +(expr)])) {expr, +, str, 1} p 9 =p(, binary, seq([expr, operator, expr])) {expr, 11, operator, 1} p 10 =p(, cond, seq([expr, expr, expr])) {expr, 111} Table 1. Production rules of the master grammar for FL, with their production signatures. V. Zaytsev, Guided Grammar Convergence, CoRR abs/ , draft for 2013.

41 Definitions Nonterminal footprint Production signature Prodsig-equivalence Weak prodsig-equivalence Nominal resolution V. Zaytsev, Guided Grammar Convergence, CoRR abs/ , draft for 2013.

42 Nominal resolution example Production rule q 1 =p, Fragment, Expr q 2 =p, Program, + (Function) Production signature {Expr, 1} {Function, +} q 3 =p(, Function, seq ([str, + (str), Expr])) {str, 1+, Expr, 1} q 4 =p(, Expr,int) {int, 1} q 5 =p(, Expr,str) {str, 1} q 6 =p(, Expr, Expr 1 ) {Expr 1, 1} q 7 =p(, Expr, Expr 2 ) {Expr 2, 1} q 8 =p(, Expr, Expr 3 ) {Expr 3, 1} q 9 =p(, Expr 1, seq ([Ops, Expr, Expr])) {Ops, 1, Expr, 11} q 10 =p(, Expr 2, seq ([Expr, Expr, Expr])) {Expr, 111} q 11 =p(, Expr 3, seq ([str, + (Expr)])) {str, 1, Expr, +} Prerequisite Match p i q j roots {program, Fragment } p 1 q 1 {ω, Fragment } roots {program, Program} p 1 q 2 {function, Function} p 2 q 3 {str, str, expr, Expr} {str, str} p 3 q 5 p 4 q 4 {int, int} {expr, Expr, str, str} p 5 q 8 {apply, Expr 3 } {expr, Expr, str, str} p 8 q 11 {expr, Expr} p 6 q 6 {binary, Expr 1 } {expr, Expr} p 9 q 9 {operator, Ops} {expr, Expr} p 7 q 7 {cond, Expr 2 } {expr, Expr} p 10 q 10 Table 2. On the left: production rules of the servant grammar for FL, derived from the XML schema, with their production signatures. On the right: the process of derivation of the nominal resolution relation p i q j. Note how two hypotheses must be formed and one of them rejected, because this servant grammar has two roots and both need to be checked for prodsig-equivalence with the root of the master grammar. Other than that, all production rules are matched with strong equivalence. V. Zaytsev, Guided Grammar Convergence, CoRR abs/ , draft for 2013.

43 Nominal resolution example Production rule r 1 =p, Program, + (Function) Production signature {Function, +} r 2 =p(, Function, seq([name, +(Name), {CR, +, Expr, 1, Expr, +(CR)])) Name, 1+} r 3 =p(, Expr, Expr 1 ) {Expr 1, 1} r 4 =p(, Expr, Expr 2 ) {Expr 2, 1} r 5 =p(, Expr, Expr 3 ) {Expr 3, 1} r 6 =p(, Expr, Name) {Name, 1} r 7 =p(, Expr, Int) {Int, 1} r 8 =p(, Expr 1, seq ([Expr, Ops, Expr])) {Ops, 1, Expr, 11} r 9 =p(, Expr 2, seq ([Name, + (Expr)])) {Expr, +, Name, 1} r 10 =p(, Expr 3, seq ([Expr, Expr, Expr])) {Expr, 111} Prerequisite Match p i r j roots {program, Program} p 1 r 1 {function, Function} p 2 r 2 {ω, CR, str, Name, expr, Expr} {str, Name} p 3 r 6 p 4 r 7 {int, Int} {expr, Expr, str, Name} p 5 r 4 {apply, Expr 2 } {expr, Expr, p 8 r 9 str, Name} {expr, Expr} {expr, Expr} p 7 r 5 p 10 r 10 {cond, Expr 3 } {expr, Expr} p 6 r 3 {binary, Expr 1 } {expr, Expr} p 9 r 8 {operator, Ops} Table 3. On the left: production rules of the servant grammar for FL, derived from a corresponding SDF syntax definition, with their production signatures. On the right: the process of derivation of the nominal resolution relation p i r j. Note how a special lexical nonterminal for CR nonterminal remains unmatched due to weak equivalence of production rules that contain it. V. Zaytsev, Guided Grammar Convergence, CoRR abs/ , draft for 2013.

44 Abstract Normal Form (1) lack of labels for production rules (2) lack of named subexpressions (3) lack of terminal symbols (4) maximal outward factoring of inner choices (5) lack of horizontal production rules (6) lack of separator lists (7) lack of trivially defined nonterminals (with α, ε or φ) (8) no mixing of chain and non-chain production rules (9) the nonterminal call graph is connected, and its top nonterminals are the starting symbols of the grammar V. Zaytsev, Guided Grammar Convergence, CoRR abs/ , draft for 2013.

45 Grammar design mutation Deyaccification B = C B C vs. B = C+ Layers vs. priorities X = Y; Y = X; vs X = ; Associativity A O A vs. A (O A)*

46 Unresolved Aggregation Master grammar exp: Meaningful chain rules Master grammar exp: STR exp + exp op exp Ecore ApplyExp: Function Exp + he master grammar, function application is defi Ecore BinaryExp: PlusExp BinaryExp: MinusExp BinaryExp: EqualExp PlusExp: Exp Exp MinusExp: Exp Exp EqualExp: Exp Exp

47 Megamodel

49 Grammar transformation composition

50 Adaptation through tolerance V. Zaytsev, Negotiated Grammar Transformation. Draft for XM 2012.

51 Adaptation through tolerance V. Zaytsev, Negotiated Grammar Transformation. Draft for XM 2012.

52 Adaptation through tolerance V. Zaytsev, Negotiated Grammar Transformation. Draft for XM 2012.

53 Adaptation through tolerance V. Zaytsev, Negotiated Grammar Transformation. Draft for XM 2012.

54 Adaptation through adjustment V. Zaytsev, Negotiated Grammar Transformation. Draft for XM 2012.

55 Adaptation through adjustment V. Zaytsev, Negotiated Grammar Transformation. Draft for XM 2012.

56 Adaptation through adjustment V. Zaytsev, Negotiated Grammar Transformation. Draft for XM 2012.

57 Adaptation through adjustment V. Zaytsev, Negotiated Grammar Transformation. Draft for XM 2012.

58 Grammar transformations Suite of well-defined well-studied operators Partial evaluation of transformation operators Classic grammar transformation: inapplicable? error! halt! vacuous? error! halt! transform! next!

59 Negotiated transformations Negotiated grammar transformation: applicable & non-vacuous? transform! next! vacuous? suggest to do nothing! not applicable? suggest better arguments! keep negotiating until applicability or surrender V. Zaytsev, Negotiated Grammar Transformation. Draft for XM 2012.

60 Negotiated transformations Variability limits as a part of transformation command Interactive transformation (ask the user) Display a warning and proceed with minimal adjustment Proceed with one, save other options for fallback Halt and recomment V. Zaytsev, Negotiated Grammar Transformation. Draft for XM 2012.

61 CWI To summarise Grammars define structure Too much stuff in the grammar Decomposition Grammarware works on grammars & languages Composition Adjacent topics?

62 Questions?

of Grammarware by Grammar Transformation

Maintenance and Evolution of Grammarware by Grammar Transformation IPA Spring Days on Model-Driven Software Engineering Vadim Zaytsev, SWAT, CWI 2012 Grammarware Language: Java import types.*; import