What s a Compler??? Sprng 2005 2 CSc 520 Prncples of Programmng Languages 2: Translators Chrstan Collberg collberg@cs.arzona.edu Department of Computer Scence Unversty of Arzona Copyrght c 2004 Chrstan Collberg [] 520 Sprng 2005 2 At the very basc level a compler translates a computer program from source code to some knd of executable code: x.c x.java source Compler errors x.s asm x.class Often the source code s smply a text fle and the executable code s a resultng assembly language program: gcc -S x.c reads the C source fle x.c and generates an assembly code fle x.s. Or the output can be a vrtual machne code: javac x.java produces x.class. [2] What s a Language Translator??? Example Language Translators A compler s really a specal case of a language translator. A translator s a program that transforms a program P wrtten n a language L nto a program P 2 wrtten n another language L 2. Typcally, we desre P and P 2 to be semantcally equvalent,.e. they should behave dentcally. source language translator target language L A T E X Postscrpt FORTRAN C++ C.class x86 bnary latex2html ps2asc f2c cfront gcc SourceAgan fx32 html text C C assembly Java Alpha bnary
Compler Input Compler Output Text Fle Common on Unx. Syntax Tree A structure edtor uses ts knowledge of the source language syntax to help the user edt & run the program. It can send a syntax tree to the compler, relevng t of lexng & parsng. Assembly Unx complers do ths. Slow, but easy for the compler. Object.o-fles on Unx. Faster, snce we don t have to call the assembler. Executable Called a load-and-go-compler. Abstract Machne Serves as nput to an nterpreter. Fast turnaround tme. C-code Good for portablty. Sprng 2005 2 [5] 520 Sprng 2005 2 [6] Compler Tasks Compler Phases Statc Semantc Analyss Is the program (statcally) correct? If not, produce error messages to the user. Generaton The compler must produce code that can be executed. Symbolc Debug Informaton The compler should produce a descrpton of the source program needed by symbolc debuggers. Try man gdb. Cross References The compler may produce cross-referencng nformaton. Where are dentfers declared & referenced? Profler Informaton Where does my program spend most of ts executon tme? Try man gprof. A N A L Y S I S Lexcal Analyss Syntactc Analyss Semantc Analyss S Y N T H E S I S Intermedate Generaton Optmzaton Machne Generaton
Compler Phases Lexcal analyss Lexcal analyss Example The lexer reads the source fle and dvdes the text nto lexcal unts (tokens), such as: Reserved words BEGIN, IF,... dentfers x, StrngTokenzer,... specal characters +,,, ˆ,... numbers 30, 3.4,... comments ( text ), strngs "text". Lexcal errors (such as llegal character, undelmted character strng, comment wthout end ) are reported. Lexcal Analyss of Englsh The sentence The boy s cowbell won t play. would be translated to the lst of tokens the, boy+possessve, cowbell, Lexcal Analyss of Java The sentence x = 3.4 (9.0+y); would be translated to the lst of tokens wll, not, play <ID,x>, EQ, <FLOAT,3.4>, STAR, LPAREN, <FLOAT,9.0>, PLUS, <ID,y>, RPAREN, SEMICOLON Sprng 2005 2 [9] 520 Sprng 2005 2 [0] Compler Phases Syntactc analyss Syntactc analyss Example Syntax analyss ( parsng) determnes the structure of a sentence. The compler reads the tokens produced durng lexcal analyss and bulds an abstract syntax tree (AST), whch reflects the herarchcal structure of the nput program. Syntactc errors are reported to the user: mssng ;, BEGIN wthout END Syntactc Analyss of Englsh The sentence The boy plays cowbell. would be parsed nto the tree NP S Det N V VP NP N The boy plays cowbell S=sentence, NP=noun phrase, VP=verb phrase, Det=determner, N=noun, V=verb.
Syntactc analyss Example Compler Phases Semantc analyss Syntactc Analyss of Java The sentence x = 3.4 (9.0+y); would be parsed nto the AST ID x Assgn FLOAT 3.4 MUL FLOAT 9.0 ADD ID y The AST produced durng syntactc analyss s decorated wth attrbutes, whch are then evaluated. The attrbutes can represent any knd of nformaton such as expresson types. The compler also collects nformaton useful for future phases. Semantc errors are reported: dentfer not declared, nteger type expected. Sprng 2005 2 [3] 520 Sprng 2005 2 [4] Semantc analyss Example Semantc Analyss of Englsh In the sentence The boy plays hs cowbell. we determne that hs refers to the boy. In the sentence Semantc Analyss of Java statc float luftballons = 0; vod P() {nt luftballons = 99; System.out.prntln(luftballons); the compler must determne whch luftballons the prnt-statement refers to, that float luftballons=0 has the wrong type. Compler Phases IR Generaton From the decorated AST ths phase generates ntermedate code (IR). The IR adds an extra a level of abstracton between the hgh level AST and the very low level assembly code we want to generate. Ths smplfes code generaton. A carefully desgned IR allows us to buld complers for a number of languages and machnes, by mxng and matchng front-ends and back-ends.
IR Generaton Example Compler Phases Optmzaton IR Generaton of Englsh From the sentence Every boy has a cowbell. we could generate x; boy(x) has-cowbell(x) IR Generaton of Java From the sentence x = 3.4 (9.0+y); the compler could generate the (stack-based) IR code pusha x, pushf 3.4, pushf 9.0, push y, add, mul, assgn The (often optonal) code optmzaton phase transforms an IR program nto an equvalent but more effcent program. Typcal transformatons nclude common subexpresson elmnaton only compute an expresson once, even f t occurs more than once n the source), nlne expanson nsert a procedure s code at the call ste to reduce call overhead, algebrac transformatons A := A + A s faster than A := A 2. Sprng 2005 2 [7] 520 Sprng 2005 2 [8] Compler Phases Generaton Mult-pass Complaton The last complaton phase transforms the ntermedate code nto machne code, usually assembly code or lnk modules. Alternatvely, the compler generates Vrtual Machne (),.e. code for a software defned archtecture. Java complers, for example, generate class fles contanng bytecodes for the Java. The next slde shows the outlne of a typcal compler. In a unx envronment each pass could be a stand-alone program, and the passes could be connected by ppes: lex x.c parse sem r opt codegen > x.s For performance reasons the passes are usually ntegrated: front x.c > x.r back x.r > x.s The front-end does all analyss and IR generaton. The back-end optmzes and generates code.
Mult-pass Complaton... Mx-and-Match Complers source Lexer errors tokens errors Parser AST F R O N T Ada Pascal Modula 2 C++ E N D IR Optmze Machne Gen IR Gen Interm. Gen AST Semantc Analyser errors B A C K Sparc Mps 68000 IBM/370 E N D asm Ada Mps compler Pascal Mps compler Pascal 68k compler Sprng 2005 2 [2] 520 Sprng 2005 2 [22] Example Let s comple the procedure Foo, from start to fnsh: PROCEDURE Foo (); VAR : EGER; BEGIN := ; WHILE < 20 DO PR 2; := 2 + ; ENDDO; END Foo; The complaton phases are: Lexal Analyss Syntactc Analyss Semantc Analyss Intermedate code generaton Optmzaton Machne code generaton. Example Lexcal Analyss Break up the source code (a text fle) and nto tokens. Source Stream of Tokens PROCEDURE Foo (); PROCEDURE, <d,foo>, LPAR, RPAR, SC, VAR : EGER; VAR, <d,>, COLON, <d,eger>,sc, BEGIN := ; BEGIN, <d,>,ceq,<nt,>,sc, WHILE, <d,>, LT, <nt,20>,do, WHILE < 20 DO PR 2; := 2 + ; PR, <d,>, MUL, <nt,2>, SC, <d,>, CEQ, <d,>, MUL, <nt,2>, PLUS, <nt,>, SC, ENDDO, SC, END, <d,foo>, SC ENDDO; END Foo;
Example Lexcal Analyss... Example Syntactc Analyss We defned the followng set of tokens: TOKEN STRING TOKEN STRING PROCEDURE "PROCEDURE" <nt,> nteger lteral <d,foo> dentfer WHILE "WHILE" LPAR "(" LT "<" RPAR ")" DO "DO" SC ";" PR "PR" VAR "VAR" MUL "" COLON ":" PLUS "+" BEGIN "BEGIN" ENDDO "ENDDO" CEQ ":=" END "END" Stream of Tokens PROCEDURE, <d,foo>, LPAR,RPAR,SC,VAR,<d,>, COLON,<d,EGER>,SC, BEGIN,<d,>,CEQ,<nt,>, SC,WHILE,<d,>,LT, <nt,20>,do,pr,<d,>, MUL,<nt,2>,SC,<d,>, CEQ,<d,>,MUL, <nt,2>,plus,<nt,>,sc, ENDDO,SC,END,<d,Foo>,SC Abstract Syntax Tre PROC DECL WHILE < 20 PR + 2 2 Sprng 2005 2 [25] 520 Sprng 2005 2 [26] Example Semantc Analyss PROC DECL d:foo Args Decls Stats Abstract Syntax Tree Decorated Abstract Syntax Tree PROC DECL type:eger STAT Des Expr WHILE STAT Expr Body PROC DECL WHILE val: op:< Rght val:20 PR STAT Expr op: Rght STAT Des Expr op:+ Rght < 20 WHILE PR < BOOL 20 PR + val:2 op: Rght val: 2 2 2 val:2
Example IR Generaton PROC DECL d:foo Args Decls Stats Decorated Abstract Syntax Tree Intermedate type:eger val: STAT Des Expr op:< type:bool Rght val:20 WHILE STAT Expr Body PR STAT Expr op: Rght val:2 STAT Des Expr op: Rght op:+ Rght val: PROC DECL WHILE < PR BOOL 20 [] [2] BRGE 20 [9] [3] MUL t 2 [4] PR t [5] MUL t 2 2 [6] ADD t 3 t 2 val:2 2 [7] t 3 [8] JUMP [2] [9] Sprng 2005 2 [29] 520 Sprng 2005 2 [30] Example IR Generaton... Example Optmzaton Intermedate [] [2] BRGE 20 [9] [3] MUL t 2 [4] PR t [5] MUL t 2 2 [6] ADD t 3 t 2 [7] t 3 [8] JUMP [2] Intermedate Defnton A, B A := B; BRGE A, B, C IF (A B) THEN contnue at nstructon C; MUL A, B, C A := B C; ADD A, B, C A := B + C; SHL A, B, C A:=shft B left C steps; PR A Prnt A and a newlne; Intermedate [] [2] BRGE 20 [9] [3] MUL t 2 [4] PR t [5] MUL t 2 2 [6] ADD t 3 t 2 [7] t 3 [8] JUMP [2] [9] Optmzed Intermedate [] [2] BRGE 20 [8] [3] SHL t [4] PR t [5] ADD t 2 t [6] t 2 [7] JUMP [2] [8] [9] JUMP A Contnue at nstructon A;
xample Machne Generaton Interpretaton Intermedate [] [2] BRGE 20 [8] [3] SHL t [4] PR t [5] ADD t 2 t [6] t 2 [7] JUMP [2] [8] MIPS Machne.data _:.word 0.text.globl man man: l $4, $32: bge $4, 20, $33 sll $a0, $4, l $v0, syscall addu $4, $a0, b $32 $33: sw $4, _ An nterpreter s lke a CPU, only n software. The compler generates vrtual machne () code rather than natve machne code. The nterpreter executes nstructons rather than natve machne code. Sprng 2005 2 [33] 520 Sprng 2005 2 [34] Interpretaton... Knds of Interpreters Interpreters are slow Often 0 00 tmes slower than executng machne code drectly. portable The vrtual machne code s not ted to any partcular archtecture. Interpreters work well wth very hgh-level, dynamc languages (APL,Prolog,ICON) where a lot s unknown at comple-tme (array bounds, etc). "APL/Prolog style (load and go/nteractve) nterpreter" Source Source Read, lex, parse, semantcs Read, lex, parse, semantcs Fle.vm Read read eval prnt Interpret Fle.vm read eval prnt "Java style nterpreter" Interpret
Knds of Interpreters... Actons n an Interpreter Source Fle.vm Just In Tme (JIT,Dynamc) Compler Read, lex, parse, semantcs Comple Natve Fle.vm Execute Internally, an nterpreter conssts of. The nterpreter engne, whch executes the nstructons. 2. Memory for storng user data. Often separated as a heap and a stack. 3. A stream of nstructons. Sprng 2005 2 [37] 520 Sprng 2005 2 [38] Actons n an Interpreter... Readngs and References Instruct onstream add store mul...... Memory Stack Heap I := Get next nstructon. Decode the nstructon. Op := the opcode Arg := st argument Arg2 := 2nd argument... Perform the functon of the opcode. "Hello!" Statc Data Read Louden: Introducton pp. 8, 2 27. or the Dragon Book: Introducton pp. 24 A Smple Compler pp. 25 82 Some Complers pp. 733 744 or the Tger Book: Introducton pp. 5
Summary Summary... The structure of a compler depends on. the complexty of the language we re workng on (hgher complexty more passes), 2. the qualty of the code we hope to produce (better code more passes), 3. the degree of portablty we hope to acheve (more portable better separaton between front- and back-ends). 4. the number of people workng on the compler (more people more ndependent modules). Some hghly retargetable complers for hgh-level languages produce C-code, rather than machne code. Ths C-code s then compled by the natve C compler to machne code. Some languages (APL, LISP, Smalltalk, Java, ICON, Perl, Awk) are tradtonally nterpreted (executed n software by an nterpreter) rather than compled to machne code. Sprng 2005 2 [4] 520 Sprng 2005 2 [42] Summary... Some nterpreters use dynamc complaton (or jttng), swtchng between. nterpretng the vrtual machne code, 2. translatng the vrtual machne code to natve machne code, 3. executng the natve machne code, 4. optmzng the natve and/or vrtual machne code, and 5. throwng natve code away f t s no longer needed or takes up too much room. All ths s done dynamcally at runtme.