MetaML and multi-stage programming with explicit annotations

Transcription

1 Orego Health & Sciece Uiversity OHSU Digital Commos CSETech Jauary 1999 MetaML ad multi-stage programmig with explicit aotatios Walid Taha Tim Sheard Follow this ad additioal works at: Recommeded Citatio Taha, Walid ad Sheard, Tim, "MetaML ad multi-stage programmig with explicit aotatios" (1999). CSETech This Article is brought to you for free ad ope access by OHSU Digital Commos. It has bee accepted for iclusio i CSETech by a authorized admiistrator of OHSU Digital Commos. For more iformatio, please cotact champieu@ohsu.edu.

2 MetaML ad Multi-Stage Programmig with Explicit Aotatios? Walid Taha ad Tim Sheard Orego Graduate Istitute Abstract. We itroduce MetaML, a practically-motivated, staticallytyped multi-stage programmig laguage. MetaML is a \real" laguage. We have built a implemetatio ad used it to solve multi-stage problems. MetaML allows the programmer to costruct, combie, ad execute code fragmets i a type-safe maer. Code fragmets ca cotai free variables, but they obey the static-scopig priciple. MetaML performs typecheckig for all stages oce ad for all before the executio of the rst stage. Certai aomalies with our rst MetaML implemetatio has led us to formalize a illustrative subset of the MetaML implemetatio. We preset both a big-step sematics ad type system for this subset, ad prove the type system's soudess with respect to a big-step sematics. From a software egieerig poit of view, this meas that geerators writte i the MetaML subset ever geerate usafe programs. A type system ad sematics for full MetaML is still ogoig work. We argue that multi-stage laguages are useful as programmig laguages i their ow right, that they supply a soud basis for high-level program geeratio techology, ad that they should support features that make it possible for programmers to write staged computatios without sigicatly chagig their ormal programmig style. To illustrate this we provide a simple three stage example elaboratig a umber of practical issues. The desig of MetaML was based o two mai priciples that we idetied as fudametal for high-level program geeratio, amely, crossstage persistece ad cross-stage safety. We preset these priciples, explai the techical problems they give rise to, ad how we address with these problems i our implemetatio. Keywords: Fuctioal Programmig, -Calculus, High-level Program Geeratio, Type-Safety, Type-Systems, Programmig Laguage Sematics, Multilevel Laguages, Multi-stage Laguages.? The research reported i this paper was supported by the USAF Air Materiel Commad, cotract # F C-0069, ad NSF Grat IRI A earlier versio of this paper appeared i The Proceedigs of the ACM SIGPLAN Symposium o Partial Evaluatio ad Sematics Based Program Maipulatio. pp Amsterdam, The Netherlads, Jue 12-13, 1997.

3 1 Itroductio High-level program geerators ca icrease the eciecy, productivity, reliability, ad quality of software systems [26, 22, 23]. Despite these umerous examples of program geerators, almost all these systems deal with the costructio of program fragmets usig ad hoc techiques. Our thesis is that a well-desiged statically-typed multi-stage programmig laguage supplies a soud basis for high-level program geeratio techology. Our goal is to desig a laguage that allows the user to costruct, combie, ad evaluate programs at a higher level of abstractio tha the classic \programsas-strigs" level. Usig such a laguage would make the formal vericatio of geerated-program properties easier. 1.1 Multi-Stage Programs ad Laguages The cocept of a stage arises aturally i a wide variety of situatios. For a compiled laguage, the executio of a program ivolves two distict stages: compiletime, ad ru-time. Three distict stages appear i the cotext of program geeratio: geeratio, compilatio, ad executio. For example, the Yacc parser geerator rst reads a grammar ad geerates C code secod, this program is compiled third, the user rus the compiled code. A multi-stage program is oe that ivolves the geeratio, compilatio, ad executio of code, all iside the same process. Multi-stage laguages express multi-stage programs. Multi-stage programmig is importat because it addresses the eed for geeral purpose solutios which do ot pay ru-time iterpretive overheads. This is the purpose of program stagig ad it ca be highly eective as demostrated i may studies [2, 17, 16, 8, 12, 25, 37, 49]. Recetly, multi-stage laguages have also bee proposed as itermediate represetatios for partial evaluatio [13, 9, 10], ad as formal foudatios for ru-time code geeratio [7]. However, there has geerally bee little support for writig multistage programs directly i high level programmig laguages such as SML or Haskell. 1.2 MetaML MetaML is a SML-like laguage with special costructs for multi-stage programmig. MetaML is tightly itegrated i that programs ca be costructed, combied, compiled, ad executed all uder a sigle paradigm. Programs are represeted as abstract sytax trees i a maer that avoids goig through strig represetatios. This makes verifyig sematic properties of multi-stage programs possible. The key features of MetaML are as follows: { Stagig otatios: Four distict costructs which we believe are a good basis for geeral-purpose multi-stage programmig. { Static type-checkig: A multi-stage program is type-checked oce ad for all before it begis executig, esurig the safety of all computatios.

4 { Cross-stage persistece: A variable boud i a particular stage, will be available i futures stages. { Cross-stage safety: A iput rst available i a particular stage caot be used at a earlier stage. { Static scopig of variables i code fragmets. 2 Relatioship to LISP MetaML has three aotatios, Brackets, Escape, ad Ru, that are aalogous to LISP's back-quote, comma, ad eval costructs. This aalogy is useful if the reader is familiar with LISP. Brackets are similar to back-quote. Escape is similar to comma. Ru is similar to eval i the emptyeviromet. However, the aalogy is ot perfect. LISP does ot esure that variables (atoms) occurrig i a back-quoted expressios are boud accordig to the rules of static scopig. For example `(plus 3 5) does ot bid plus i the scope where the term occurs. We view this as a importat features of MetaML. We view MetaML's sematics as a cocise formalizatio of the sematics of LISP's three costructs, but with static scopig. This is similar i spirit to Bria Smith's sematically motivated LISP [45, 46] Fially, whereas LISP is dyamically typed, MetaML is statically typed. The aotatios ca also be viewed as a providig a simple but staticallytyped macro-expasio system. This will become clear as we itroduce ad demostrate the use of these costructs. But it is also importat to ote that the aotatios do't allow the deitio of ew laguage costructs or bidig mechaisms, as is sometimes expected from macro-expasio systems. Fially,we should poit out that back-quote ad comma are macros i LISP. This leads to two problems. First, they have o-trivial formal sematics (about two pages of LISP code). Secod, because of the way they expad at parse-time, they ca lead to expoetial overhead for represetig multi-level programs [10]. MetaML avoids both problems by a direct treatmet of Bracket ad Escape as laguage costructs. 3 Relatioship to Liguistic Reectio \Liguistic reectio is deed as the ability of a program to geerate ew program fragmets ad to itegrate these ito its ow executio" [47]. MetaML is a descedet of CRML [40, 41, 15], which i tur was greatly iueced by TRPL [38, 39]. All three of these laguages support liguistic reectio. Both CRML ad TRPL were two stage laguages that allowed users to provide compile-time fuctios (much like macros) which directed the compiler to perform compiletime reductios. Both emphasized the use of computatios over represetatios of a program's datatype deitios. By geeratig fuctios from datatype deitios, it was possible to create specic istaces of geeric fuctios like equality fuctios, pretty priters, ad parsers [39]. This provided a abstractio mechaism ot available i traditioal laguages. MetaML improves upo these

5 laguages by addig hygieic variables, geeralizig the umber of stages, ad emphasizig the soudess of its type system. 4 Relatioship to Partial Evaluatio Today, the most sophisticated automatic stagig techiques are foud i partial evaluatio systems [20]. Partial evaluatio optimizes a program usig a priori stagig iformatio about some of that program's iputs. The goal is to idetify ad perform as may computatios as possible i a program before ru-time. Oie partial evaluatio ivolves two distict steps, bidig-time aalysis (BTA) ad specializatio. BTA determies which computatios ca be performed i a earlier stage give the ames of iputs available before ru-time (static iputs). I essece, BTA performs automatic stagig of the iput program. After BTA, the actual values of the iputs are made available to the specializer. Followig the aotatios, the specializer either performs a computatio, or produces text for iclusio i the output (residual) program. The relatioship betwee partial-evaluatio ad multi-stage programmig is that the itermediate data structure betwee the two steps is a two-stage aotated program [1, 34], ad that the specializatio phase is the rst stage i the executio of the two-stage aotated program produced by BTA. Recetly, Gluck ad Jrgese proposed multi-level BTA ad showed that it is a eciet alterative to multiple specializatio [9, 10]. Their uderlyig aotated laguage is closely related to MetaML, but without static-typig. 5 Why Explicit Aotatios? While BTA performs stagig automatically, there are a umber of reasos why the maual stagig of programs is both iterestig ad desirable: Pragmatic. The subtlety of the sematics of aotated programs warrats studyig them i relative isolatio, ad without the added complexity of other partial evaluatio issues such as BTA. As a Pedagogical Tool. It has bee observed that it is sometimes hard for users to uderstad the workigs of partial evaluatio systems [18]. New users ofte lack a good metal model of how partial evaluatio systems work. Furthermore, ew users are ofte ucertai: What is the output of a bidig-time aalysis? What are the aotatios? How are they expressed? What do they really mea? The aswers to these questios are crucial to the eective use of partial evaluatio. Although BTA is a ivolved process, requirig special expertise, the aotatios it produces are relatively simple ad easy to uderstad. Our observatio is that programmers ca uderstad the aotated output of BTA, without actually kowig how BTA works. Havig a programmig laguage with explicit stagig aotatios would help users of partial evaluatio uderstad more of the issues ivolved i staged computatio, ad, hopefully,

6 reduce the steep learig curve curretly associated with learig to use a partial evaluator eectively [20]. For Cotrollig Evaluatio Order. Wheever performace is a issue, cotrol of evaluatio order is importat. BTA optimizes the evaluatio order give the time of arrival of iputs, but sometimes it is just easier to say what is wated, rather tha to force a BTA to discover it [19]. Automatic aalyses like BTA are ecessarily icomplete, ad ca oly approximate the kowledge of the programmer. By usig explicit aotatios the programmer ca exploit his full kowledge of the program domai. I a laguage with automatic stagig, havig explicit aotatios ca oer the programmer a well desiged back-door for dealig with istaces whe the aalysis reaches its limits. For Cotrollig Termiatio Behavior. Aotatios ca alter termiatio behavior i two ways: 1) Specializatio of a aotated program ca fail to termiate, ad 2) the geerated program itself might have termiatio behavior dierig from that of the origial program [20]. While this is a area of active ivestigatio i partial evaluatio, programmig with explicit aotatio gives the user complete cotrol over (ad resposibility for) termiatio behaviour i a staged system. 6 MetaML's Stagig Aotatios MetaML has four stagig aotatios: Brackets < >, Escape ~, Ru ru, ad Lift lift.aexpressio <e> builds a piece of code which is a represetatio of e. A expressio ~e splices the code obtaied by evaluatig e ito the body of a surroudig Bracketed expressio. A expressio ~e is oly legal withi lexically eclosig Brackets. A expressio ru e evaluates e to obtai a piece of code, ad the evaluates that code. The expressio lift e evaluates e to a value v, ad the costructs a piece of code represetig v. The term e must have groud type. A groud type is a type ot cotaiig a fuctio type. To illustrate, cosider the script of a small MetaML sessio below 1 : - val triple = (3+4, <3+4>, lift 3+4) val triple = (7,<3 %+ 4>,<7>) : (it * <it> * <it>) - fu f (x,y,z) = < 8 - ~y > val f = f : ('b * <it> * 'a) -> <it> - val code = f triple val code = <8 %- (3 %+ 4)> : <it> - ru code val it = 1 : it 1 The reader should treate the percetage sigs as white space util they are explaied i the ext sectio.

7 The rst declaratio dees a variable triple. The additio i the rst compoet of the triple is evaluated. The evaluatio of the additio i the secod compoet is deferred by the Brackets. The additio i the third compoet is evaluated ad the the result is Lifted ito a piece of code. Brackets i types such as <it> are read \Code of it", ad distiguish values such as <3+4> from values such as 7. The secod declaratio illustrates that code ca be abstracted over, ad that it ca be spliced ito a larger piece of code. The third declaratio applies the fuctio f to triple performig the actual splicig. Ad the last declaratio evaluates this deferred piece of code. To give a brief feel for how MetaML is used to costruct larger pieces of code at ru-time cosider: - fu mult x = if =0 the <1> else < ~x * ~(mult x (-1)) > val mult = f : <it> -> it -> <it> - val cube = <f y => ~(mult <y> 3)> val cube = <f a => a * (a * (a * 1))> : <it -> it> - fu expoet = <f y => ~(mult <y> )> val expoet = f : it -> <it -> it> The fuctio mult, give a iteger piece of code x ad a iteger, produces a piece of code that is a -way product of x. This ca be used to costruct the code of a fuctio that performs the cube operatio, or geeralized to a geerator for producig a expoetiatio fuctio from a give expoet. Note how the loopig overhead has bee removed from the geerated code. 6.1 Roles Havig Ru i the laguage implies itroducig a kid of reectio [45, 3], ad allows a delayed computatio to be activated. Both Lift ad Brackets create pieces of code. The essetial dierece is that Lift evaluates its argumet, ad Bracket does ot. Fuctio values caot be lifted ito code usig Lift, as we caot derive a high-level itesioal represetatio for them i geeral. Fuctios ca be delayed usig Brackets. 6.2 Sytactic Precedece Issues The Escape operator has the highest precedece eve higher tha fuctio applicatio. This allows us to write: <f ~x y>rather tha <f (~x) y>. The Lift ad Ru operators have the lowest precedece. The scope of these operators exteds to the right as far as possible. This makes it possible to write <f ~(lift g y) z> rather tha <f ~(lift (g y)) z>.

8 7 The Desig of MetaML MetaML was desiged as a statically-typed programmig laguage, ad ot as a iteral represetatio for a multi-stage system. Our primary goals for MetaML were: rst, it should be suitable for writig multi-staged programs, secod it should be as exible as possible, ad third it should esure that oly \reasoable thigs" ca be doe usig the aotatios. Therefore, our desig choices were dieret from those of other multi-stage systems. To dee the sematics of MetaML, a sytactic otio of level is eeded. The level of a expressio is the umber of surroudig Brackets, mius the umber of surroudig Escapes. It is possible to use a variable at a level dieret tha the level of the lambda abstractio which bids it. I this sectios, we discuss two priciples for determiig which uses are acceptable, ad which are ot. 7.1 Cross-Stage Persistece Cross-stage persistece is oe of the distiguishig feature of MetaML. To our kowledge, it has ot bee proposed or icorporated ito ay multi-stage programmig laguage. I essece, cross-stage persistece allows the programmer to use a variable boud at the curret level i ay expressio to be executed i a future stage. We believe this to be a desirable ad atural property ia multi-stage laguage. The type system will have to esure that these variables are available before this expressio is evaluated. Whe the level of the use of a variable is greater tha the level at which it was boud, we say that variable is cross-stage persistet. To the user, cross-stage persistece meas the ability to stage expressios that use variables deed at a previous stage. Bracketed expressios with free variables, like -abstractios with free variables, must resolve their free variable occurreces i the static eviromet where the expressio occurs. Oe ca thik of a piece of code as cotaiig a eviromet which bids its free variables. For example the program fragmet val a = 1+4 i <72+a> computes the code fragmet <72 %+ %a>. The percetage sig (%) idicates that the free variable a ad the operator + are boud i the code's local eviromet. The % is prited by the display mechaism. The variable a has bee boud durig the rst stage to the costat 5. I fact, i MetaML %a is ot a variable, but rather, a ew costat. The ame \a" is oly a hit to the user about where this costat origiated. Whe %a is evaluated i a later stage, it will retur 5 idepedetly of the bidig for the variable a i the ew cotext sice it is boud i the value's local eviromet. Arbitrary values (icludig fuctios) ca be delayed usig this hygieic bidig mechaism. Formally specifyig this behavior i a big-step sematics turs out to be o-trivial. I a iterpreter for a multistage laguage, this behaviour maifests itself as complex variable-bidig rules, the use of closures, or capture-free substitutios. Our implemetatio sematics addresses cross-stage persistece i a ovel way (Sectio 13.1).

9 Cross-Platform Portability For high-level program geeratio, cross-stage persistece comes at a price. I particular, beig able to iject ay value ito the code type meas that some parts of this code fragmetmay ot be pritable. So, if the rst stage is performed o oe computer, ad the secod o aother, we must \port" the local eviromets from the rst machie to the secod. Sice arbitrary objects, such as fuctios ad closures, ca be boud i these local eviromets this ca become a portability problem. Curretly, MetaML assumes that the computig eviromet does ot chage betwee stages. This is part of what we mea by havig a itegrated system. Thus, MetaML curretly lacks cross-platform portability. The loss of this property is the price paid for cross-stage persistece. Cross-platform portability is usually ot a issue for ru-time code geeratio systems, ad hece, cross-stage persistece might i fact be more appropriate for such systems. O the other had, the problem of cross-platform portabilityis similar to that of liftig fuctioal values i partial evaluatio, ad type-directed partial evaluatio may provide a solutio to this problem [4, 42]. 7.2 Cross-Stage Safety Not every staged form of a typable expressio should be typable i MetaML. Whe a variable is used at a level less tha the level of the lambda abstractio i which it is boud, we say the use violates cross-stage safety. Cross-stage safety prevets us from stagig programs i ureasoable ways,as is the case i the expressio f a => <f b => ~(a+b)> Operatioally, the aotatios require computig a+b i the rst stage, while the value of b will be available oly i the secod stage! Therefore, MetaML's type system was desiged to esure that \well-typed programs wo't go wrog", where goig wrog ow icludes the violatio of the cross-stage safety coditio, as well as the stadard otios of goig wrog [27] i statically-typed laguages. I our experiece with MetaML, havig a type system to scree out programs cotaiig this kid of error is a sigicat aid i had-stagig programs. 8 Had-Stagig: A Short Example Usig MetaML, the programer ca stage programs by isertig the proper aotatios at the right places i the program. The programmer uses these aotatios to modify the default (strict) evaluatio order of the program. I our experiece, startig with the type of the fuctio to be had-staged makes the umber of dieret ways i which it ca be aotated quite tractable. This leads us to believe that the locatio of the aotatios i a staged versio of a program is sigicatly costraied by its type. For example, cosider the fuctio member deed as follows 2 : 2 Fuctio \=" hastype (it * it) -> bool which forces v ad l to have types it ad it list, respectively.

10 (* member : it -> it list -> bool *) fu member v l = if (ull l) the false else if v=(hd l) the true else member v (tl l) The fuctio member has type it -> it list -> bool. A good strategy for had aotatig a program is to rst determie the target type of the desired aotated program. Suppose the list parameter l is available i the rst stage, ad the elemet searched for will be available later. Oe target type for the had-staged fuctio is <it> -> it list -> <bool>. Now we ca begi aotatig, startig with the whole expressio, ad workig iwards util all sub-expressios are covered. At each step, we cosider what aotatios will \x" the type of the expressio so that the whole fuctio has a type closer to the target type. The followig fuctio realizes this type: (* member : <it> -> it list -> <bool> *) fu member v l = if (ull l) the <false> else <if ~v=~(lift hd l) the true else ~(member v (tl l))> But ot all aotatios are explicitly dictated by the type. The aotatio ~(lift hd l) has the same type as (ad replaces) hd l i order to esure that hd is performed durig the rst stage. Otherwise, all selectios of the head elemet of the list would have bee delayed util the code costructed was Ru i a later stage. The Brackets aroud the braches of the outermost if-expressio esure that the retur value of member will be a code type (< >). The rst brach (false) eeds o further aotatios, ad makes the retur value precisely a <bool>. Movig iwards i the else brach, the coditio of the ier if-expressio (i particular ~v) forces the type of the v parameter to have type <it> as plaed. Just like the rst brach of the outer if-statemet, the whole of the ier if-statemet must retur bool. So, the rst brach (true) is e. But because the recursive call to member has type <bool>, it must be Escaped. Isertig this Escape also implies that the recursio will be performed i the rst stage, which is exactly the desired behavior. Thus, the result of the staged member fuctio is a recursively-costructed piece of code with type bool. Evaluatig <f x => ~(member <x> [1,2,3])> yields: <f d1 => if d1 %= 1 the true

11 else if d1 %= 2 the true else if d1 %= 3 the true else false> 9 Isomorphisms of Code Types: Back ad Forth While stagig programs, we foud a iterestig pair of fuctios to be useful: (* back: <'A> -> <'B> -> <('A -> 'B)> *) fu back f = <f x => ~(f <x>)> (* forth: <('A -> 'B)> -> (<'A> -> <'B>) *) fu forth f x = <~f ~x> We used a similar costructio to stage the member fuctio of type <it> -> it list -> <bool>, withi the term <f x => ~(member <x> [1,2,3])> which has type <it -> bool>. I our experiece aotatig a fuctio to have type <A> -> <B>requires less aotatios tha aotatig it to have type <A -> B> ad is ofte easier to thik about. Because we are more used to reasoig about fuctios, this leads us to avoid creatig fuctios of the latter kid except whe we eed to ispect the code. This ca also be see i programs with more tha two stages. Cosider the fuctio: (* back2 : (<'A> -> <<'B>> -> <<'C>>) -> <'A -> <'B -> 'C>> *) fu back2 f = <f x => <f y => ~~(f <x> <<y>>)>> This allows us to write a program which takes a <a> ad a <<b>> as argumets ad which produces a <<c>>, ad stage it ito a three-stage fuctio. Our experiece is that such fuctios have cosiderably fewer aotatios, ad are easier to thik about. This is illustrated i the followig sectio. Fially, we should metio that there is aother reaso for our iterest i back ad forth: they are similar to two-level -expasio [5]. I MetaML, however, back ad forth are ot oly meta-level cocepts or optimizatios, but rather, rst class fuctios i the laguage, ad the user ca apply them directly to values of the appropriate type. 10 A Multi-Stage Example Whe iformatio arrives i multiple phases it is possible to take advatage of this fact to get better performace. Cosider a geeric fuctio for computig the ier product of two vectors. I the rst stage the arrival of the size of the vectors oers a opportuity to specialize the ier product fuctio o that size,

12 removig the overhead of loopig over the body of the computatio times. The arrival of the rst vector aords a secod opportuity for specializatio. If the ier product of that vector is to be take may times with other vectors it ca be specialized by removig the overhead of lookig up the elemets of the rst vector each time. This is exactly the case whe computig the multiplicatio of 2 matrixes. For each row i the rst matrix, the dot product of that row will be take with each colum of the secod. This example has appeared i several other works [9, 24] ad we give our versio below. We give three versios of the ier product fuctio. Oe (iprod) with o stagig aotatios, the secod (iprod2) with two levels of aotatios, ad the third (iprod3) with two levels of aotatios but costructed with the back2 fuctio. I MetaML we quote relatioal operators ivolvig less-tha < ad greater-tha > because of the possible cofusio with Brackets. (* iprod : it -> Vector -> Vector -> it *) fu iprod v w = if '>' 0 the ((th v ) * (th w )) + (iprod (-1) v w) else 0 (* iprod2 : it -> <Vector -> <Vector -> it>> *) fu iprod2 = <f v => <f w => ~~(if '>' 0 the << (~(lift th v ) * (th w )) + (~(~(iprod2 (-1)) v) w) >> else <<0>>) >> (* p3 : it -> <Vector> -> <<Vector>> -> <<it>> *) fu p3 v w = if '>' 0 the << (~(lift th ~v ) * (th ~~w )) + ~~(p3 (-1) v w) >> else <<0>> fu iprod3 = back2 (p3 ) Notice that the staged versios are remarkably similar to the ustaged versio, ad that the versio writte with back2 has fewer aotatios. The type iferece mechaism was a great help i placig the aotatios correctly. A importat feature of MetaML is the visualizatio help that the system aords. By testig iprod2 o some iputs we ca see what the results are immediately. val f1 = iprod3 3 f1 : <Vector -> <Vector -> it>> = <f d1 =>

13 <f d5 => (~(lift %th d1 3) * (%th d5 3)) + (~(lift %th d1 2) * (%th d5 2)) + (~(lift %th d1 1) * (%th d5 1)) + 0 >> Whe this piece of code is Ru it will retur a fuctio, which whe applied to a vector builds aother piece of code. This buildig process icludes lookig up each elemet i the rst vector ad splicig i the actual value usig the Lift operator. Usig Lift is especially valuable if we wish to ispect the result of the ext phase. To do that we evaluate the code by Ruig it, ad apply the result to a vector. val f2 = (ru f1) [1,0,4] f2: <Vector -> it> = <f d1 => (4 * (%th d1 3)) + (0 * (%th d1 2)) + (1 * (%th d1 1)) + 0 > Note how the actual values of the rst array appear i the code, ad how the access fuctio th appears as a costat expressio applied to the secod vector d1. While this code is good, it does ot take full advatage of all the iformatio kow i the secod stage. I particular, ote that we geerate code for the third stage which may cotai multiplicatio by 0 or 1. These multiplicatios ca be optimized. To dothiswe write a secod stage fuctio add which give a idex ito a vector i, aactualvalue from the rst vector x, ad a piece of code which ames the secod vector y, costructs a piece of code which adds the result of the x ad y multiplicatio to the code valued fourth argumet e. Whe x is 0 or 1 special cases are possible. (* add : it -> it -> <Vector> -> <it> *) fu add i x y e = if x=0 the e else if x=1 the <(th ~y ~(lift i)) + ~e> else <(~(lift x) * (th ~y ~(lift i))) + ~e> This specialized fuctio is ow used to build the secod stage computatio: (* p3 : it -> <Vector> -> <<Vector>> -> <<it>> *) fu p3 v w = if = 1 the << ~(add (th ~v ) ~w <0>) >> else << ~(add (th ~v ) ~w < ~~(p3 (-1) v w) >) >> fu iprod3 = back2 (p3 )

14 Now let us observe the result of the rst stage computatio. val f3 = iprod3 3 f3: <Vector -> <Vector -> it>> = <f d1 => <f d5 => ~(%add 3 (%th d1 3) <d5> < ~(%add 2 (%th d1 2) <d5> < ~(%add 1 (%th d1 1) <d5> <0>)>)>) >> This code is liear i the size of the vector if we had actually i-lied the calls to add it would be expoetial. This is aother reaso why havig cross-stage persistet costats (such as add) i code is idispesable. Now let us observe the result of the secod stage computatio: val f4 = (ru f3) [1,0,4] f4: <Vector -> it> = <f d1 => (4 * (%th d1 3)) + (%th d1 1) + 0> Note that ow oly the multiplicatios that cotribute to the aswer are evidet i the third stage program. If the vector is sparse the this sort of optimizatio ca have dramatic eects. 11 Formal Sematics ad Developmet of MetaML The study of the formal sematics of MetaML is still ogoig research. I this sectio, we will preset thetype system of MetaML [48], ad outlie a proof of its soudess usig a simplied adaptatio of the proofs appearig i [29]. The reader is ecourage to cosult these sources for more detailed treatmet of how these results where achieved Big-step Sematics The sytax of the core subset of MetaML is as follows: e := i j x j ee j x:e j <e> j ~e j ru e Values. Values are a subset of terms, which deote the results of computatios. Because of the relative ature of Brackets ad Escapes, it is importat to use a family of sets for values, idexed by the level of the term, rather tha just oe set. Values are deed iductively as follows: v 0 2 V 0 := i j x j x:e j <v 1 > v 1 2 V 1 := i j x j v 1 v 1 j x:v 1 j <v 2 > j ru v 1 v +2 2 V +2 := i j x j v +2 v +2 j x:v +2 j <v +3 > j ~v +1 j ru v +2

15 The set of values has three otable poits. First, values ca be Bracketed expressios. This meas that computatios ca retur pieces of code represetig other programs. Secod, values ca cotai applicatios (iside Brackets) such as (y:y) (x:x) 2 V 1. Third, there are o level 1 Escapes i values. The deitio of substitutio is stadard ad is deoted by: e[x := v] for the substitutio of v for x i e. The core subset of MetaML ca be assiged a big-step sematics as follows [29]: e 1,! x:e e2,! v1 e[x := v 1 ],! v 2 i,! i x +1,! x 0 e 1 e 2,! v e 1,! v 1 e 2,! v 2 e 1 e 2 +1,! v 1 v 2 x:e 0,! x:e e +1,! v x:e +1,! x:v e,! 0 <v 0 > v 0 0,! v ru e 0,! v e +1,! v ru e +1,! ru v e,! 0 <v> ~e,! 1 v e +1,! v ~e +2,! ~v e +1,! v <e>,! <v> 11.2 Type System The judgemet ` e : r is read \uder the type eviromet, at level ad sytactically surrouded byr occurreces of Ru, the term e has type." The type assigmet maps variables to a triple. This triple cosists of the type, the level, ad the umber of surroudig Ru-occurreces at the poit where this variable was boud (See Abs rule). Goig Wrog There are three mai kids of errors related to stagig aotatios that ca occur at ru-time: (1) A variable is used at a level less tha the level of the lambda abstractio i which it is boud, or (2) Ru or Escape are passed values havig a o-code type, or (3) Ru alters the level of its argumet, ad ca therefore lead to a type (1) error. The rst kid of error is checked by thevar rules. Let us assume that our program cotais o Ru aotatios, the r is always zero. Havig a rule for 0 allows cross-stage persistece: Variables available i the curret stage ( 0 ) ca be used i all future stages (). The secod kid of error is checked by the Ru ad Esc +1 rules. Detectig the third kid of error is more dicult problem, ad is accomplished by keepig track of surroudig Rus ad comparig it to surroudig (ucacelled) Brackets. I essece, assumig the

16 Domais ad Relatios Levels r := 0 j +1 Types := it j! j <> Type Eviromets := [ ] j x 7! ( r) where (x 7! ( r) )y if x = y the ( r) else y It : Br : Abs : Ru : Iferece Rules ` i : it r Var : +1 ` e : r ǹ <e> : <> r x 7! ( 1 r) ` e : 2 r ` x:e : 1! 2 r ` e : <> r+1 ` ru e : r Esc +1: App : x=( r0 ) r + r 0 ` x : r ` e : <> r +1 ` ~e : r ` e 1 : 1! r ` e 2 : 1 r ` e 1 e 2 : r Fig. 1. Type System type is correct, we oly allow Ru, where it removes explicitly maifest Brackets. This is icorporated ito the variable rule usig the coditio 0 + r + r 0 which esures that every occurrece of a variable has strictly more surroudig Brackets tha Rus. Without this coditio we would wrogly allow the program <f x => ~(ru <x>)> which reduces to the term <f x => ~x> which is either a value or ca be reduced ay further. I geeral, this meas that we have to be careful with ope pieces of code. Specically, wehave tomake sure that if Ru is applied to a ope piece of code, the level of the free variables used i this piece of code will ot drop below the level at which they are boud. For the stadard part of the laguage, code is a ormal type costructor that eeds o special treatmet, ad the level is ever chaged by the other laguage costructs Type Preservatio As is commo i type preservatio proofs, oe must prove a Substitutio lemma. I additio, because our sematics is also expected to respect the otio of level, we also prove so called Promotio ad Demotio lemmas: Lemma 1 (Level Properties). The type system has the followig three importat properties:

17 { Promotio: 1 2 ǹ { Flex: x 7! ( 0 r 0 +1) ǹ { Demotio: v 2 V +1 ad e : r implies 1 +(c+d c) 2 +1 ` +c+d ` e : r + c 0 e : r implies x 7! ( 0 r 0 ) 1 v : r +1implies ` v : r where +(c d) x =( r + d) (+c) wheever x=( r). ǹ e : r Proof. All three properties are proved by straight forward iductio over the rst typig derivatio. The proof of Demotio uses Flex i the case of Abstractio, ad takes advatage of the deitio of values i the case of Escape to show that Escape at level 1 is ot relevat. 0 Lemma 2 (Substitutio). Let r 0 r. The, 0 ` e 0 : 0 r 0 ad x 7! 0 ( 0 r 0 ) ǹ e : r implies 0 ` e[x := e 0 ]: r Proof. By straight forward iductio over the height of the secod typig derivatio. The (o-trivial) Variable case uses promotio ad takes advatage of the coditio that r 0 r. ad we ca ow prove our mai theorem: Theorem 1 (Type Preservatio). If +(1 0) v 2 V ad +(1 0) ǹ v : r. ǹ e : r ad e,! v the Proof. By straight forward iductio over the height of the evaluatio derivatio. Applicatio at level 0 uses substitutio, ad Ru at level 0 uses demotio Cross-Stage Persistece Moolithic Variables Cross-stage persistece ca be relaxed by allowig variablestobeavailable at exactly oe stage. This seems to be the case i all multistage laguages kow to us to date [34,7, 13, 9, 10, 6]. Ituitively, they use the followig moolithic rule for variables (assume r =0): Var (Moolithic): x= 0 ` x : whe 0 = We allow the more geeral coditio 0, so a expressio like val lift like = f x => <x> is accepted, because iside the Brackets, =1, ad x = 0. This expressio is ot accepted by the moolithic variable rule. Note that while the whole fuctio has type! <> it does ot provide us with the fuctioality of Lift, because the result of applyig lift like to ay value always returs the costat <%x>, ot a literal expressio deotig the value. This distictio ca oly be see at

18 the level of the implemetatio sematics (discussed below) but ot the big-step sematics (discussed above). The type system rejects the expressio f a => <f b => ~(a+b)> because, iside the Escape, =0, ad ( b) = 1,but Limitatios to the Expressivity of Ru The type system preseted above does ot admit the lambda-abstractio of Ru. This was, to a large extet, a desig choice ad a compromise. I particular, if a Ru fuctio is itroduced ito the laguage as a costat, it breaks the safety of the type system. I this sectio, we discuss two expressivity problems that arise from this desig choice, ad how they are addressed i the curret implemetatio Typig Top-Level Bidigs Problem. A MetaML program cosists of a sequece of top-level declaratios bidig variables to terms, followed by a term: program := e j val x = e program If we iterpret a top-level declaratio val a=e 1 e 2 as (a:e 2 )e 1,theweare i the icoveiet situatio were we caot bid a value at top-level that we will evetually wat to Ru, eve if it might otherwise be safe to Ru it. This is because a:ru a is ot typable i the type system preseted i this paper. Thus, this iterpretatio of val a = <1> ru a would be utypable. Observatios. Top-level bidigs have a umber of importat properties which other (-boud) bidigs may ot: First, every top-level bidig is at level 0. Secod, all top-level bidigs are oly withi the scope of other top-level bidigs. I particular o top-level bidig is i the scope of a -boud variable which is boud at a level greater tha 0. This is importat because o top-level bidig will ever be i the scope of a piece of code with free variables that ca have problems iteractig with Ru. Oe of the purposes of the type system was to throw away programs where Ru was applied to code with free variables that ca cause the computatio to get stuck (type (3) errors). Because sytactic restrictios guaratee this at top-level we use a dieret type rule for top-level bidigs, allowig more safe terms to be typable. A Solutio. The curret implemetatio avoids this problem i MetaML by usig the followig rule for top-level bidigs i the iteractive loop:

19 Top: a 7! ( 1 r+ h) 0 0 ` e 2 : r 0 ` e 1 0 ` val a=e 1 e2 : r : 1 r+ h For top-level declaratios, the system prits the type of bidig as it is etered by the user. Note, however, that h is ot prited. I theory, h is existetially quatied i the rule above. I practice, a large umber is used. Ituitively, the large h correspods to the ability to Ru values declared at top-level as may times as we wat. Soudess of Top Rule. A let-expressio let a=e 1 i e 2 is usually iterpreted as havig the same operatioal sematics as (a:e 2 )e 1.Thisiterpretatio is ofte used as a basis for derivig a type rule for let: a 7! ( 1 r) ` e : r (Lam) ` e 1 : 1 r ` a:e 2 : 1! r (App) ` (a:e 2 )e 1 : r (By def.) ` let a=e 1 i e 2 : r + a 7! ( 1 r) ` e : r ` e 1 ` let a=e 1 i e 2 : r : 1 r (Let) The Top rule is based o a equivalet but o-stadard operatioal iterpretatio of the declaratio val a=e 1 e 2, amely, ru (h) ((a:< (h) e 2 >)e 1 ) where h is the umber of repeated occurreces of the costruct that it appears as a superscript of. This iterpretatio is motivated by the fact that if this term is typable, ad e 1,! v1, the all the terms i the relatio ru (h) ((a:< (h) e 2 >)e 1 ),! 0 0 e 2 [v 1 =a] are typable wheever the derivatio exists. Thus, we do't perform the substitutio durig type-checkig, but rather, usig the followig derivatio: ` e 2 a 7! ( 1 r+ h) : r (Promotio Lemma) a 7! ( 1 r+ h) +h ` e 2 : r + h a 7! ( 1 r+ h) (Bra h) ` < (h) e 2 > : < (h) > r+ h (Lam) ` a:< (h) e 2 > : 1! < (h) > r+ h ` ((a:< (h) e 2 >)e 1 ): < (h) > r+ h (Ru h) ` ru (h) ((a:< (h) e 2 >)e 1 ): r ` e 1 : 1 r+ h We arrive at the rule for Top, by collectig the assumptios at the top of the tree of this derivatio, ad settig to 0. (App)

20 Pickig a large h works because the Promotio Lemma tells us that if there is a for which type-checkig the top-level let-bidig is possible, the it will also be possible for all 0 >. I practice, this strategy has worked well Use of Ru iside Fuctios It would be useful if the type system allowed us to express fuctios such asthe followig: val f = f x : it list => ru (ge x) This is a fuctio that takes a list of itegers, geerates a itermediate program based o list, ad the executes the geerated program. The type system for the core laguage does ot admit this term (for ay previously declared variable ge). I our experiece, most such fuctios where quite small, ad we could ofte achieve the same eect as f e by takig advatage of the power of the let-rule described above: val a = ge e val r = ru a This trick is useful, but is ot satisfactory from the poit ofviewofthemodularity of the code, as it forces us to do a kid of \iliig" of f to get aroud the type system. We cojecture that it is possible to relax the type system somewhat usig rules such as the followig: Ru : x i 7! (b i r i +1) 0 x i 7! (b i r i ) 0 ` e : < > r+1 ` ru e : r where b is a base-type (such as it or it list). Ituitively, this typig rule assures us that wheever a basic value (that is, ot ivolvig code) is available at level 0, it will ca be used i a cotext with as may surroudig occurreces of Ru as eeded. This rule would allow us to type the expressio above. However, it is still ad hoc, ad we hope to formulate a more systematic basis for such rules i future work. 13 Implemetatio Sematics The purpose of stagig a program is to give the user cotrol over the order of evaluatio. The big-step sematics preseted above does ot capture all the relevat operatioal details addressed i the implemetatio of MetaML. The three mai exceptios are the eed for 1) distiguishig betwee real ad symbolic bidigs, 2) ru-time geeratio of ames (gesym), ad 3) cross-stage persistet costats. I this sectio we preset a sematics which describes our implemetatio. While we have worked hard to keep our implemetatio both eciet ad faithful to the big-step sematics, the formal proof of their relatio is still ogoig work.

21 Domais ad Relatios Judgmets J := ; ` e,! e j ; ` e +1,! e Rules It 0: ; ` i,! i It +1: ; ` i +1,! i Abs 0: ; x 7! Sym(x0 ) ` e 1,! e1 ; ` x:e,! x0 :e1 Abs +1: ; x 7! Sym(x0 ) ` e1 +1,! e 2 ; ` x:e 1 +1,! x0 :e2 App 0: ; ` e 1,! x:e ; ` e 2,! v 2 x7! Real(v 2) ` e,! v ; ` e 1 e 2,! v App +1: ; ` e 1 +1,! e 3 ; ` e 2 +1,! e 4 ; ` e 1 e 2 +1,! e 3 e 4 Var 0: ;x =Real(v) ; ` x,! v SVar +1: ;x =Sym(x0 ) ; ` x +1,! x0 RVar +1: ;x = Real(v) ; ` x +1,! " v Bracket 0: Escape 1: ; ` e 1 1,! e2 ; ` <e 1>,! <e2> Bracket +1: ; ` e 1,! <e 2> ; ` ~e 1 1,! e2 Escape +2: ; ` e 1 +1,! e 2 ; ` <e 1>,! <e2> ; ` e 1 +1,! e 2 ; ` ~e 1 +2,! ~e 2 Ru 0: Costat 0: ; ` e,! <e 1> ` e1,! v 1 ; ` ru e,! v 1 Ru +1: ; ` v,! v0 ; `" v,! v0 Costat +1: ; ` e 1 +1,! e 2 +1 ; ` ru e 1,! ru e 2 ; ` v +1,! v0 ; `" v +1,! " v0 Fig. 2. Implemetatio Sematics 13.1 Real ad Symbolic Bids, gesym, ad Cross-Stage Costats The implemetatio sematics cosists of rules for reductio ; ` e,! v, essetially applyig the Applicatio ad Ru rules, ad rebuildig ; ` e +1,! e, idexed by alevel + 1,essetially evaluatig Escaped computatios iside Brackets, where the eviromet ; bids a variable to a value. Reductio is stadard for the most part. A subtlety relatig to variable bidig causes a problem that makes eviromets somewhat complicated. I particular, some variables are ot yet boud whe rebuildig is takig place. For example, rebuildig the term <f x => ~(id <x>)> requires reducig the applicatio id <x>. But while reducig this applicatio, the variable x is ot yet boud to a

22 value. I the iteded sematics of MetaML, we really wat this variable to be simply a ame that is ot susceptible to accidetal ame capture at ru-time. To solve this problem, bidigs i eviromets come i two avors: real (Real(v)) ad symbolic (Sym(x)). The extesio of the eviromet with real values occurs oly i the rule App 0. Suchvalues are retured (Var 0) uder reductio, or ijected ito costat " terms (RVar +1) uder rebuildig. I essece, the three tags Real, Sym ad " work together to provide us with the set of coercios eeded to deal with free variables ad to implemet cross-stage persistece. Aother feature of the implemetatiosematics is that it is self-cotaied, i that it does ot use a substitutio operatio. Istead, substitutio is performed by the rebuildig operatio. Thus, altogether rebuildig has three distict roles: 1. To replace all kow variables with a costat expressio (" v) where the v comes from Real(v) bidigs i ; (rule RVar +1). 2. To reame all boud variables. Symbolic Sym(x 0 ) bidigs occur i rules Abs 0 ad Abs +1 where a term is rebuilt, ad ew ames are itroduced to avoid potetial variable capture. These ew ames are projected from the eviromet (rule SVar +1). 3. To execute Escaped expressios to obtai code to \splice" ito the cotext where the Escaped term occurs (rule Escape 1). Ituitively, without the stagig aotatios, rebuildig is simply capture-free substitutio of symbolic variables boud i ;. Rebuildig is used i two rules, Abs 0 where it is used for capture-free substitutio, ad Bracket 0 where it is applied to terms iside Brackets ad it describes how the delayed computatios iside a value are costructed. The type system esures that i rule Abs 0, there are o embedded Escapes at level 1 that will be ecoutered by the rebuildig process, so the use of rebuildig i this rule implemets othig more tha capture-free substitutio. The rules Escape 1, Ru 0, adbracket 0 are at the heart of the dyamic sematics. I the rebuildig rule Escape 1, a Escaped expressio at level 1 idicates a computatio must produce a code-valued result <e 2 >, ad rebuildig returs the term e 2. The role of i the judgemet istokeep track ofthelevel of the expressio beig built. The level of a subexpressio is the umber of ucacelled surroudig Brackets. Oe surroudig Escape cacels oe surroudig Bracket. Hece, is icremeted for a expressio iside Brackets (Bracket), ad decremeted for oe iside a Escape (Escape). Note that there is o rule for Escape at level 0: Escape must appear iside ucacelled Brackets. The reductio rule Bracket 0 describes how a code value is costructed from a Bracketed term <e 1 >. The embedded expressio is stripped from its Brackets, rebuilt at level 1, ad the result of this rebuildig is the rewrapped with Brackets. The reductio rule Ru 0 describes how a code-valued term is executed. The term is reduced to a code-valued term, ad the embedded term is the reduced i the empty eviromet to produce the aswer. The empty eviromet is

23 suciet because all free variables i the origial code-valued term have bee replaced by costat expressios (" v) The Notio of a Stage I the itroductio, we gave the ituitive explaatio for a stage. After presetig the sematics for MetaML, we ca ow provide a reasoable formal deitio. We dee (the trace of) a stage as the derivatio tree geerated by the ivocatio of the derivatio ` e 1,! v 1 (cf. Ru 0 rule). Note that while the otio of a level is deed with respect to sytax, the otio of a stage is deed with respect to a trace of a operatioal sematics. Although quite ituitive, this distictio was ot always clear to us, especially that there does ot seem to be ay comparable deitio i the literature with respect to a operatioal sematics. The levels of subterms i a program, ad the stages ivolved i the executio of a program ca be quite urelated. A program <1+ ru <4+2>> has expressios at levels 0, 1, ad 2. If we dee the \level of a program" as the maximum level of ay of its subexpressios, the this is a 2-level program. The evaluatio of this expressio (which just ivolves rebuildig it), ivolves o derivatios ` e 1,! v 1. O the other had, the evaluatio of slightly modied 2-level program ru <1 + ru <4+2>> ivolves two stages. To further illustrate the distictio betwee levels ad stages, let us dee the \umber of stages" of a program as the umber of times the ` e 1,! v 1 is used i the derivatio ivolved i its evaluatio. Cosider: (f x => if P the x else lift (ru x)) <1+2> where P is a arbitrary problem. The umber of stages i this program is ot statically decidable. Furthermore, we caot say, i geeral, which occurrece of Ru will be ultimately resposible for triggerig the computatio of the additio i expressio <1+2>. Recogizig this mismatch was a key step towards dig a type-system for MetaML, which, ituitively, employs the static otio of level to approximate the dyamic otio of stage Why is -Abstractio ot Eough? It may appear that stagig requires oly -abstractio, ad its dual operatio, applicatio. While this may be true for certai applicatios, for the domai of program geeratio there are two additioal capabilities that are eeded: First, a delayed computatio must be observable, so that users ca ispect the code produced by their geerators, ad so that it ca be prited ad fed ito compilers. I a compiled implemetatio, -abstractios lose their high-level itesioal represetatio, ad either of these is possible. Secod, geerators ofte eed to perform \reductios uder Lambda's". This is ecessary for almost ay staged applicatio that performs some kid of ufoldig, ad is used i fuctios like back. Although we caot prove it, the eect of

24 Escape (uder lambda) caot be imitated i the call-by-value -calculus without extedig it with additioal costructs. To further explai this poit, we will show a example of the result of ecodig of the operatioal sematics of MetaML i SML/NJ. The essetial igrediets of a program that requires more tha lambda ad applicatio for stagig are Brackets, dyamic (o-level 0) abstractios, ad Escapes. Lambda-abstractio over uit ca be used to ecode Bracket, ad applicatio to uit to ecode Ru. However, Escape is cosiderably more dicult. I particular, the expressio iside a Escape has to be executed before the surroudig delayed computatio (closure) is costructed. This becomes a problem whe variables itroduced iside the delayed expressio occur i the Escaped expressio. For example: <f x => ~(f <x>)>. Oe way to imitate this behavior uses two o-pure SML features. Refereces allow reductios uder lambda, ad exceptios allow the creatio of uiitialized referece cells. Cosider the followig sequece of MetaML declaratios: fu G f = <f x => ~(f <x>)> val pc = G (f xc => <(~xc,~xc)>) val p5 = (ru pc) 5 The correspodig imitatio i SML would be: exceptio ot_yet_defied val udefied = (f () => (raise ot_yet_defied)) fu G f = let val xh = ref udefied val xc = f () =>!xh () val c = f xc i f () => f x => (xh:=(f () => x) c ()) ed val pc = G (f xc => f () => (xc(),xc())) val p5 = (pc ()) 5 I this traslatio, values of type <> are ecoded by delayed computatios of type uit!. We begi by assigig a lifted udeed value to udefied. Now we are ready to write the aalog of the fuctio G. Give a fuctio f, the fuctio G rst creates a uiitialized referece cell xh. This referece cell correspods to the occurreces of x i the applicatio f <x> i the MetaML deitio of G. Ituitively, the fact that xh is uiitialized correspods to the fact that x will ot yet be boud to a xed value whe the applicatio f <x> is to be performed. This facility isvery importat i MetaML, as it allows us to ufold fuctios like f o \dummy" variables like x. The expressio f () =>!xh () is a delayed lookup of xh. This correspods to the Brackets surroudig x i the expressio f <x>. Now, we simply perform the applicatio of the fuctio f to this delayed costructio. It is importat to ote here that we are applyig f as it is passed to the fuctio G, before we kowwhatvalue x is boud to.