Construction of Application Generators Using Eli Uwe Kastens, University of Paderborn, FRG Abstract Application generators are a powerful means for reuse of software design. They produce special purpose programs from domain specic descriptions. An application generator can be understood as a translator for a domain specic description language. The Eli system is an integrated toolset for language implementation. It automates a wide range of language implementation tasks, and it provides numerous reusable solutions of common subtasks. The system is designed to be also applicable by non-specialists in compiler construction. In this paper we show how the Eli system supports the construction of application generators. 1 Introduction Application generators are software systems that generate programs for dierent instances of a problem in a specialized application domain. Typical examples are programs that produce reports of data extracted from a data base. Such a program can be systematically constructed from information describing the data base and the desired layout. Many dierent variants of such programs may be needed to produce dierent reports. An application generator for this problem domain generates such a program from a specication that describes which data have to be retrieved and which layout has to be produced. In Sect. 2 we describe instance of a report generator for a bibliographic data base that is used as a running example in this paper. The term application generator was initially coined in [3] for the problem domain of data base tools. Cleaveland [1] generalizes the principle of application generators, points out their relationship to compilers, and describes tools to construct them. In [7] application generators are discussed as one of several principles for software reuse. The input for an application generator can be considered as a specication within a restricted problem domain. It is formulated in a dedicated specication language. An application generator is a translator for that language. Hence techniques for language implementation have to be applied for the construction for application generators. That correspondence is shown in Sect. 3 by matching the problem decomposition for our running example to a widely accepted one for conventional compilers. Eli [2,8] is an integrated toolset for language implementation. Eli incorporates state-ofthe-art know-how in the eld of compiler construction. Its generating tools and its library of reusable components cover a wide range of language implementation tasks. Users need not be compiler specialists to design and implement their dedicated specication language with Eli. This is most important for application generators, since they are usually constructed by people who are specialists in an application domain rather than in language implementation. Eli 1
has been successfully applied in many industrial and academic projects, and in many application domains. The methodological aspects that make Eli suitable for the construction of application generators are demonstrated in Sect. 4 using extracts of the specication for our running example. The complete executable specication is available on request. 2 Application Generators An application generator produces variants of programs that solve instances of problems in a certain restricted domain. Each problem instance is described in terms of a dedicated specication language, the input language for the application generator. Let us assume for example that we want to produce information extracted from a data base containing publication references. Such a data base report may look like Papers of Kastens since 1993: 1993 Executable Specifications for Language Implementation 1993 Design of a VLIW Architecture Constructed from Standard... 1994 Construction of application generators using Eli 1994 Generating interpreters from compiler specifications 1994 Modularity and Reusabiltiy in Attribute Grammars Kastens published 5 papers. It may be useful to get such reports for dierent authors and time spans. Hence, we need a program to produce the report which takes the author and the year as input. Furthermore we may want to have several such programs that produce reports containing dierent information presented in a certain layouts. According to the application generator principle we describe the desired report in a specication language designed exactly for that problem class. The description for the above example may be string name = int since = int cnt = 0; InString "Which author?"; InInt "Since which year?"; "\n\npapers of ", name, " since ", since, ":\n\n"; [ SELECT name <= Author && since <= Year; cnt = cnt + 1; Year, "\t", Title, "\n"; ] "\n", name, " published ", cnt, " papers.\n\n\n"; It describes which records are to be selected from the data base, the elds to be extracted, and the layout of the desired report. The generated program consists of operations that access the data base and operations that produce the formatted output. Those operations are selected and parameterized according to the description and are systematically composed according to technical requirements for data base use and printing facilities. The application generator translates such a description into a C program, for example. The same principle can be used in other problem domains: An application generator for simulation of mechanical systems may take descriptions of system structures and of feedback 2
Structuring Translation Encoding Lexical analysis Syntactic analysis Semantic analysis Transformation Code generation Assembly Scanning Conversion Parsing Tree construction Name analysis Type analysis Data mapping Action mapping Execution-order determination Register allocation Instruction selection Instruction encoding Internal address resolution External address resolution Figure 1: Compilation Subproblems equations for their components as input. An instance may be the description of a car suspension. The generated simulator program can be run to validate that car suspension under dierent constraints. 3 Problem Decomposition Implementation of application generators can be considered as a language implementation task: Specications of problem instances written in a domain specic language are translated into application programs. Well established knowledge and eective tools for translator construction can be applied to solve that task. The domain specic aspects of the solution then concentrate on the design of the specication language and on the components of which application programs are composed of. Many years of research and experience in the eld of compiler construction led to a generally accepted model for decomposition of compilation tasks, as shown in Figure 1 taken from [2]. This decomposition model is applicable for the implementation of domain specic languages as well as for programming languages: in case of application generator construction both the structuring and the translation task have to be solved. The encoding phase, typical for machine language compilers, is not needed here; the high-level target program is produced by the transformation phase. For our example language the necessity of the structuring task and of the transformation is obvious. Name analysis and type analysis also apply to this language: It has constructs for named entities. They occur in dening and applied contexts. Occurrences of the same names have to be identied according to rather simple scope rules. There are also predened names for elds of the data base records. Named entities and expressions have one of the types int, bool, or string. The types determine the meaning of overloaded operators, how values are output. The transformation task maps directly to the target program. Hence, the encoding tasks of the decomposition model for programming language compilers (Figure 1) are not needed her. This situation is typical for application generators. 3
4 Eli Specications In this section we demonstrate how Eli supports application generator construction. We emphasize the consequences of the problem decomposition model and the solution methods typically applied. Eli comprises a large number of components each dedicated for the solution of a subtask in the decomposition model discussed in the previous section. There are several generating tools including generators for scanners, parsers, grammar mapping, attribute evaluators, operator identication, denition tables, and output transformers. A library of C modules provides implementations of basic tasks that are common in language implementation, e. g. input reading, storing and encoding identiers and literals, error message output, consistent renaming of identiers. Their implementations are generally applicable, validated and ecient according to the state-of-the-art in compiler construction. A library of specication modules contains complete generic solutions of particular subproblem instances which occur often in language implementation, e. g. support for name analysis, type analysis, association of properties to program objects, and transformation into target text. Such specication modules comprise input fragments for generators and applications of basic library functions. They can be easily reused and composed for the specication of particular application tasks. In order to create an application generator an Eli user supplies a set of specications. Most of them are written in languages dedicated for the addressed subtasks. That allows the specications to be less redundant and to concentrate on the particular problems to be solved; typical examples are given below. Eli deduces which of its tools are to be executed for processing the specications. The components of Eli are smoothly integrated on the base of the task decomposition model. Hence, solutions of related subtasks t together, usually without any consideration of the user. Eli also automates the technical work to congure, to compose, and to compile the translator program. 4.1 Structuring For most applications only the concrete input grammar and the notations of non-literal tokens need to be specied. Eli's generators produce a scanner and a tree constructing parser from them. The remaining tasks (reading input, token conversion, and tree construction) are solved by standard solutions added without further specication. The concrete grammar for our example is written in a conventional context-free grammar notation. We omit its presentation in this paper. The non-literal tokens are specied by Ident: NumberLit: StringLit: C_IDENTIFIER C_INTEGER C_STRING_LIT Here precoined token descriptions are related to names of terminal symbols in the concrete grammar. These descriptions also contain information which library modules should be used to store and encode token information. It is obvious that the parsing subtask can be solved by a parser generator given the contextfree grammar. The existence of the tree construction task and its solution are deduced automatically by Eli: The description of the translation task (see below) states which tree nodes are needed. An Eli tool determines how they are to be generated by the parser, without further user interaction needed. 4
4.2 Name Analysis The solution of the name analysis task demonstrates a method that is typical for Eli's reusable specication modules: Computational roles provided by a module are related to certain contexts of the particular grammar using a specication language based on attribute grammars. In our report language names for typed variables can be dened and used. Denition and use of a name has to be identied. A module of Eli's name analysis library is chosen by $/Name/CScope.gnrc:inst It provides a set of computations for name analysis in the style of C-like scope rules. Their implementation [5] is hidden from the user. Its computational roles IdDefScope, IdUseEnv, and RootScope are associated to symbols of our report language grammar that represent dening and applied occurrences of names and the grammar root respectively: SYMBOL Report INHERITS RootScope END; SYMBOL DefIdent INHERITS IdDefScope END; SYMBOL UseIdent INHERITS IdUseEnv, ChkIdUse END; As a result of these computations attributes DefIdent.Key and UseIdent.Key refer to denition table entries, such that properties of objects can be associated and used. Other library modules are used in a similar way to check for multiple denitions, and to introduce predened names for the record elds of our report language. 4.3 Type Analysis There are two major aspects of type analysis for our example language: denition and use of typed objects, and overloading resolution for operators in typed expressions. The former task is solved in a similar way as shown for name analysis: A module provides computational roles for type names, typed denitions, and uses of typed name. They are associated to corresponding grammar symbols. Operator overloading relates one source operator to several target operators that have different signatures. Hence, the properties of source and target operators and their relationship are to be described. Using that specication module a source operator <=, is described by SrcOpr ('<=', CmpOpr, BinOpr, LeOp) It states that the operator has the precedence of a CmpOpr as introduced in the expression syntax, that those operators are generalized to BinOpr nodes in the tree grammar, and introduces a name LeOp referred to in the target operator description. A production, the BinOpr to CmpOpr relation, and an attribute computation for the operator symbol are generated from this description. In our language this source operator is overloaded by two target operators described by TgtOpr (LeOp, ile, (inttype,inttype):booltype, TgtStr={"<="}) TgtOpr (LeOp, sle, (stringtype,stringtype):booltype, TgtStr={"SubStr"}) It states for each target operator a name, ile, sle, its signature, and a string used for its translation into output text. From such a description specications are derived for Eli's operator identication tool. It generates a set of functions used in attribute computations which are also supplied by the specication module. Further operators are described in the same form. This form of operator description allows to keep all information about operators together at one place although several Eli tools are aected by the description. The specication of type analysis for binary operators in expression then reduces to the following specication: 5
SYMBOL BinOpr INHERITS BinTgtOpr, ChkOpr END; RULE: Expr ::= Expr BinOpr Expr COMPUTE BinOpr.LType = Expr[2].Type; BinOpr.RType = Expr[3].Type; Expr[1].Type = BinOpr.ResType; END; The operand types are passed by attributes (LType, RType) to the BinOpr symbol. The computations for overloading resolution are obtained from the module and inherited to BinOpr. They use the above specied operator properties, and yield the result type of the expression. The above is written in the specication language LIDO for the attribute evaluator generator LIGA [4]. LIDO specications basically associate computations to grammar symbols or to grammar rules. The computations are then executed together with those from other specications during a tree walk of the generated evaluator. LIDO also contains object-oriented constructs that allow to associate sets of computations to role symbols, and inherit them to other symbols, as shown above. The design of Eli's reusable library modules is based on this technique [6]. 4.4 Transformation The specication of the transformation task is typical for application generators (and for sourceto-source translations): Two descriptions are given: One species a set of text patterns which are used to compose the target text. Eli's PTG tool generates functions from it which compose and output the text. The other describes which text patterns are used for the translation of particular tree nodes. It is specied in terms of calls of these functions and their dependencies, and is written in LIDO. The specication is further simplied by using library modules which support the composition of C programs from text fragments. Here only the translation of the output statements of our report description language is shown. Each output statement is translated into a C printf statement as described by the following pattern: PrintStmt: "printf (\"" $1 "\", " $2 ");\n" consists of three string literals and two variables ($1, $2) where the format string and the expressions to be printed are inserted. They are supplied as arguments of the calls of the generated function PTGPrintStmt. The pattern is applied at the subtree root of each output statement in the description: CHAIN CFormat, CPrintArgs: PTGNode; RULE: Statement ::= OutputItems ';' COMPUTE CHAINSTART HEAD.CFormat = PTGNULL; CHAINSTART HEAD.CPrintArgs = PTGNULL; Statement.CCode = PTGPrintStmt (TAIL.CFormat, TAIL.CPrintArgs); END; The two arguments are both obtained from the sequence of OutputItems in its subtree. Each OutputItem contributes one component to each argument. They are composed in left-toright order specied using LIDO's CHAIN-construct: The CHAINs CFormat and CPrintArgs are started in this tree context. Their results obtained by TAIL.CFormat and TAIL.CPrintArgs ll the PrintStmt pattern to constitute the translation of the statement. 6
5 Conclusion In this paper we have shown that the development of application generators can be considered as a language implementation problem. Eli is designed such that it can be used without specialized knowledge in language implementation techniques. Hence, constructors of application generators need not be specialists in both areas, their application domain and language implementation. Using data base report generators as an example for application generator we demonstrated Eli's central paradigms: a straight-forward problem decomposition model supported by cooperating tools, high level specication languages dedicated for specic tasks, and large libraries that provide reusable and combinable solutions for common tasks. These facilities allow for eective construction of application generators from descriptions on adequate levels of abstraction. Eli has proven to be a useful toolset in many real life projects in dierent domains, e. g mechanical systems simulation, data base tools, user interface tools, software conguration, signal processing, graphical music scores, and of course compiler construction. Eli is developed in cooperation of A. M. Sloane, James Cook University, Australia, W. M. Waite, University of Colorado, Boulder, the author of this paper, and their groups. For further reference to Eli see http://www.uni-paderborn.de/project-hp/eli.html. Acknowledgements This work was partially supported by the government of Nordrhein-Westfalen through the SofTec-Cooperation, and by the US Army Research Oce under grant DAAL03-92-G-0158. 6 References 1. Cleaveland, J. C., \Building Application Generators", IEEE Software 5 (July 1988), 25{33. 2. Gray, R. W., Heuring, V. P., Levi, S. P., Sloane, A. M. & Waite, W. M., \Eli: A Complete, Flexible Compiler Construction System", Communications of the ACM 35 (February 1992), 121{131. 3. Horowitz, E., Kemper, A. & Narasimhan, B., \A Survey of Application Generators", IEEE Software 2 (Jan 1985), 40{54. 4. Kastens, U., \LIDO - Short Reference", University of Paderborn, FRG, Documentation of the LIGA System, 1992. 5. Kastens, U. & Waite, W. M., \An Abstract Data Type for Name Analysis", Acta Informatica 28 (1991), 539{558. 6. Kastens, U. & Waite, W. M., \Modularity and Reusability in Attribute Grammars", Acta Informatica 31 (1994), 601{627. 7. Krueger, C. W., \Software Reuse", ACM Computing Surveys 24 (June 1992), 131{183. 8. Waite, W. M., Heuring, V. P. & Kastens, U., \Conguration Control in Compiler Construction", in International Workshop on Software Version and Conguration Control '88, Teubner, Stuttgart, 1988. 7