Profile Language Compiler Chao Gong, Paaras Kumar May 7, 2001

Size: px

Start display at page:

Download "Profile Language Compiler Chao Gong, Paaras Kumar May 7, 2001"

Stephanie Barber
6 years ago
Views:

1 Profile Language Compiler Chao Gong, Paaras Kumar May 7, 2001 Abstract Our final project for CS227b is to implement a profile language compiler using PRECCX. The profile, which is a key component in the Data Centers project, is described with the profile language. This paper presents the design and implementation of the profile language compiler. 1. Introduction The Data Centers project is a joint research work held in Brandeis University, Brown University, and University of California, Berkeley. The goal of this project is to support profile-driven data management of large-scale, autonomous data sources (e.g., the web). Profile-driven data management is a framework for determining and implementing data management policies upon the analysis of user data requirements specified in profiles. The fundamental enabler to this approach is the profile. The profile is a formal specification of user/application-level data requirements. Our final project is to implement a profile language compiler using PRECCX (PREttier Compiler Compiler). The profile compiler translates a profile written in ASCII file format into a Java program. Compiling and executing the Java program will construct a series of Java objects representing contents of the profile. These Java objects are resources of information about profile in the profile processing. 2. Profile Language A user's profile describes what data objects are considered interesting and how useful these data objects are relative to each other. The profile language is defined in the parameterized grammar. A parameterized grammar (also called a modular grammar) is a partially specified grammar that includes certain nonterminals (the parameters) that are never reduced. That is, a grammar's parameters appear in the bodies but never in the heads of production rules. Such grammars are partial language specifications; instantiation of parameterized grammars with the missing production rules "completes" the language definition. The variation in instantiations of parameterized grammars means that there are many concrete profile language grammars. These grammars are same in the most part except the definitions of certain nonterminals (the parameters). It's tedious and inefficient to implement a new complier for each concrete profile language grammar. The efficient way is to implement a basic compiler for the parameterized grammar, which is the common part in all profile language grammars, and to implement a specific compiler for every profile language grammar that compiles the part of definitions of the parameters. When we need a compiler for a specific profile language, we combine the basic compiler and the specific compiler for that profile language to get the compiler we need. 1

2 3. PRECCX PRECCX is a compiler compiler. It converts PRECCX-style context-grammar definition scripts (with a.y extension) into ANSI C code scripts (with a.c extension). The output C code is a topdown (LL) parser that can parse grammar defined in the PRECCX-style context-grammar definition scripts. The output code compiles under the GNU Software Foundations ANSI C compiler, gcc. PRECCX is coded in ANSI C generated by PRECCX itself from its own definition script, and is fully portable. Compared with the Unix yacc utility, PRECCX extends facilities in many facets. We list these new facilities that are relevant to our final project below: (1). Modular output Parts of a script can be preccx'ed separately, compiled separately, and then linked together later, which makes maintenance and version control easy. That is the reason why we chose PRECCX as the tool to build the profile compiler. We can write separate PRECCX definition scripts for the basic compiler and the specific compilers. These scripts are preccx'ed and compiled separately, thus we get separate object codes of these compilers. We link together the object code of basic compiler and the object code of a specific compiler to get the compiler for a specific profile language. (2). Infinite lookahead PRECCX is "infinite lookahead", so it will investigate each branch of the grammar to the maximum depth. But it is often the case that grammars do not have explicit termination markers (such as an ENDIF or ENDBLOCK) and then it may be that an initial segment of the token stream will satisfy the grammar specification as well as the full stream will. To preclude this possibility, the potentially longest matches must come first in PRECCX definition scripts, to force the longest matches to be sought first. So in PRECCX one must b where in yacc one would have written a:b bc ; In the PRECCX definition scripts, we re-ordered production rules with the same head to make sure that the longest matches to be sought first. (3). Built-in lexer PRECCX includes a trivial default built-in lexer that passes characters through to PRECCX as tokens. We didn't use other lexical analyses such as lex but build-in lexer in our final project. 2

3 (4). Parameterized grammar definitions That allows context-dependent grammars. With that utility, we can recognize keywords(e.g. PROFILE, UPTO) in the profile grammar use one production rule. In the PRECCX definition script profile.y, that String(s) = ) *(char*)s==0 < *(char*)s > String( ((char*)s) + 1 ) 4. Running Profile Compiler 4.1 Files Our project consists of following files: profile.y: preccx definition script for the basic compiler. profile.h: header file included in profile.y. Contains C macros and C functions checking the attributes of a character. action.h: header file included in profile.y. Contains C functions to write output Java codes of profile compiler into files. stack.h: header file included in profile.y. Contains C functions for operations on stacks used in the parsing process. DExpURL.y: preccx definition script for parameter nonterminal <DomainSetExp>. <DomainSetExp> is defined as URL. OExpURL.y: preccx definition script for parameter nonterminal <ObjectExp>. <ObjecttExp> is defined as the age of a webpage. DExpURL.y and OExpURL.y are preccx definition scripts for the specific compiler that compiles URL_based profiles. traveler: an example Search-Engine-Based profile. investor: an example URL-Based profile. script: commands to run PRECCX and get profile compiler. 4.2 Construct profile compiler We execute PRECCX to convert preccx definition scripts into ANSI C code scripts, then compile and link C code scripts, the obtained executable file is the profile compiler. Below we use the process of constructing the URL_based profile compiler as an example to illustrate the process of constructing the profile compiler. 1. Install PRECCX (read PRECCX user manual for details) Make sure the library and header files are installed in appropriate directories. They will be used in the compile and link process. 2. Execute PRECCX to convert preccx definition scripts into ANSI C code scripts. preccx -p64 profile.y profile.c preccx DExpURL.y DExpURL.c preccx OExpURL.y OExpURL.c 3

4 The flag -p64 specifies the size in Kb of the internal program (tables) built by PRECCX during the scan of a specification script. It correlates with the maximum number of symbols in a single production rule. Default size 20Kb isn't enough for scanning preccx definition script profile.y. 3. Compile C code scripts to get objective codes. gcc -c profile.c -I../include gcc -c DExpURL.c -I../include gcc -c OExpURL.c -I../include PRECCX header file ccx.h is in directory specified by flag -I. If you installed PRECCX header file in a different directory, the flag -I has different form. 4. Link object codes together to get executable file. gcc -o profile profile.o OExpURL.o DExpURL.o -L../lib -lcc1 Library libcc1.a is in the directory specified by flag -I. If you installed PRECCX library in a different directory, the flag -I has different form. Executable file profile is the URL_based profile compiler. 4.3 Run profile compiler We also use the URL_based profile compiler as an example. 1. Execute profile compiler with profile as input. ASCII file investor is a URL-Based profile. Execute command profile < investor We get the output file Actions.java. Actions.java consists of Java statements that call constructors of Java classes to construct Java objects representing contents of the profile. The Java classes are defined in other Java files written by Eddie and David. 2. Compile these Java file and execute Actions class to get Java objects. javac *.java java Actions 5. Implementation In this section, we describe our work in detail. This section is organized as following: In 5.1 we describe the modified profile grammar that maintains the same semantics. In 5.2 we present production rules written in PRECCX. In 5.3 we present actions in PRECCX definition script. 5.1 Modification of profile language grammar Preccx parsers are top-down and left to right. That means we can NOT write production rules in left recursion! We can write rules in right recursion representing same semantics. For example, NEVER write "foo = foo bar bar"! instead, write "foo = bar foo bar". Moreover, END is a keyword in PRECCX, we can't write END in production rules. 4

5 So we made a little modification to the profile language grammar but didn t change the semantics. In the grammar, the production rules of Conds are: Conds :: Sconds Conds AND SCond Conds OR SCond NOT '(' Conds ')' We changed left recursion to right recursion, the modified production rules of Conds are: Conds :: SCond SCond AND Conds SCond OR Conds NOT '(' Conds ')' The production rule of Profile is: Profile :: PROFILE IDENT DomainClause UtilityClause END We replace END with FINISH, the modified production rule of Profile is: Profile :: PROFILE IDENT DomainClause UtilityClause FINISH 5.2 Production rules in PRECCX definition script Because PRECCX is "infinite lookahead", we wrote production rules with the same head into PRECCX definition scripts in the order that longer match is in front of short match. So the longest matches will be sought first. For example, we wrote the production rules of Conds in the order as Conds = Scond ws AND ws SCond ws OR ws NOT ws <'('> ws Conds ws <')'> 5.3 Actions in PRECCX definition script The profile compiler translates a profile written in ASCII file format into a Java program. The Java program constructs a series of Java objects representing contents of the profile. We use 15 classes to represent the parameterized grammar for profile language. For example, we use the PLProfile class to represent the Profile. An object of PLProfile class represents a Profile, it contains 3 data members: ident, domainclause, utilityclause. Ident is a string object representing the identifier of the profile; domainclause is an object of PLDomainClause class representing DomainClause in the Profile, utilityclause is an object of PLUtilityClause class representing UtilityClause in the Profile. We use interfaces to represent parameter nonterminals and use classes implementing the interfaces to represent parameter instantiations. For example, interface PLDomainSetExpParam represents parameter nonterminal <DomainSetExp>, class PLDomainSetExpParamURL implements interface PLDomainSetExpParam and represents URL instantiation of <DomainSetExp>. 5

6 Whenever a rule is reduced, the compiler will execute the action associated with the rule. The action is C codes that write Java codes to output file. The Java codes are statement to call constructors of appropriate Java classes to construct Java objects. Representing a profile need multiple same kind of objects. For example, we use PLSCond class to represent SCond in the profile. It's often the case that a profile contains many SConds, so in Java codes there are many objects of PLSCond class. How to name these objects of the same class? We use an array to store objects of one class, so we can identify an object through the array name and index. Because a legal profile contains just one Profile, one DomainClause, and one UtilityClause, we use one object of PLProfile class, one object of PLDomainClause class, and one object of PLUtilityClause class to represent them. In output Java codes, we should define an array with a specific size before referencing it. Only after finishing parsing, the compiler knows the number of every kind of objects to represent a profile. During parsing process, the compiler writes Java statements that call class constructors into a temporary file "Tmp_Actions.java". After finishing parsing, the compiler writes Java statements that declare object arrays with definite sizes and other necessary codes into the file "Actions.java", then copy the content of temporary file "Tmp_Actions.java" to the end of "Actions.java" and delete the temporary file. Now the file "Actions.java" is the output file. To construct an object in the parsing process, the compiler have to remember its data members that have been parsed. For example, when the production rule UtExp :: UPTO(Integer, UtExp, UtExp) is reduced, the compiler will construct a UtExp object by call appropriate constructor of PLUtExp class with parameters of an integer and two UtExp objects. So the compiler has to remember these parameters. We use stack data structure to remember data objects. The stack is implemented through the chain. We use 4 stacks in the profile compiler, one stack for Java objects, one for identifiers, one for integers, one for relational operators. The reason for that we employ 4 stacks is that the structure of element of each stack is different from each other. The element of the stack for Java objects consists of an integer and a pointer to the element under itself. The integer is the index of object in the array storing that kind of objects. Because the compiler can infer the type of object popped from the stack using the production rule being parsed, we don't record object type in the stack. The element of the stack for identifiers consists of a pointer to a character array with IDENT_LEN size and a pointer to the element under itself. The character array contains an identifier. The structures of elements of the stacks for integers and relational operators are similar to that of the stack for identifiers. The only difference is that the character pointer points to different size character array. 6

7 6. Future Work 1. If we can find and use a Java-base compiler compiler that supports modular output, we can implement a profile compiler that translates profile directly into Java objects. The performance will be better. 2. Error checking of our compiler is simple. When parsing a profile with syntax errors, the profile compiler can output indication of meeting error, but can't output indication of where and what the error is. References [1]. Mitch Cherniack, Michael Franklin, Stan Zdonik, Profile_driven data management [2]. P.T. Breuer, J.P. Bowen, PRECCX user manual 7

COMPILER CONSTRUCTION LAB 2 THE SYMBOL TABLE. Tutorial 2 LABS. PHASES OF A COMPILER Source Program. Lab 2 Symbol table

COMPILER CONSTRUCTION LAB 2 THE SYMBOL TABLE. Tutorial 2 LABS. PHASES OF A COMPILER Source Program. Lab 2 Symbol table COMPILER CONSTRUCTION Lab 2 Symbol table LABS Lab 3 LR parsing and abstract syntax tree construction using ''bison' Lab 4 Semantic analysis (type checking) PHASES OF A COMPILER Source Program Lab 2 Symtab