Structure of a Typical Interpreter. Compiler. Scanning and Parsing. Lexical Analysis (Scanning) Interaction Between Scanning and Parsing

Transcription

1 Snning nd Prsing Announements Projet 1 is 5% of totl grde Projet 2 is 10% of totl grde Projet 3 is 15% of totl grde Projet 4 is 10% of totl grde Tody Outline of plnned topis for ourse Overll struture of ompiler Lexil nlysis (snning) Syntti nlysis (prsing) Struture of Typil Interpreter Anlysis hrter strem lexil nlysis s words syntti nlysis AST sentenes semnti nlysis nnotted AST interpreter Compiler Synthesis IR ode genertion IR optimiztion IR ode genertion trget lnguge CS553 Leture Snning nd Prsing 2 CS553 Leture Snning nd Prsing 3 Lexil Anlysis (Snning) Intertion Between Snning nd Prsing Brek hrter strem into s ( words ) Tokens, lexemes, nd ptterns Lexil nlyzers re usully utomtilly generted from ptterns (regulr expressions) (e.g., lex) Exmples reltion identier number string lexeme(s) <,<=,=,!=,... foo,index ,570 hi, mom pttern < <= =!=... [-za-z_][-za-z0-9_]* [0-9] [0-9]*.[0-9].* hrter strem Lexil nlyzer lexer.next() lexer.peek() Prser prse tree or AST pi := , identier(pi), ssign,number( ) CS553 Leture Snning nd Prsing 4 CS553 Leture Snning nd Prsing 5 1

2 Speying Tokens with SbleCC Reognizing Tokens with DFAs Theory meets prtie: Regulr expressions, forml lnguges, grmmrs, prsing i f t_ SbleCC exmple input file: Pkge minijv; Helpers ll = [0..0xFFFF]; r = 13; digit = ['0'..'9']; letter = [''..'z'] ['A'..'Z']; undersore = _ ; Tokens t_plus = ''; t_ = ''; t_id = letter (letter digit undersore)*; t_blnk = (' ' eol tb); t_omment = _omment line_omment; letter (letter digit)* letter 1 2 letter or digit t_id not_str = [ll - '*']; not_str_slsh = [not_str - '/']; _omment = '/*' not_str* ('*' (not_str_slsh not_str*)?)* '*/'; Ignored Tokens t_blnk, t_omment; Ambiguity due to mthing substrings Longest mth Rule priority CS553 Leture Snning nd Prsing 6 CS553 Leture Snning nd Prsing 7 Syntti Anlysis (Prsing) Intertion Between Snning nd Prsing Impose struture on strem Limited to syntti struture ( high-level) Struture usully represented with n bstrt syntx tree (AST) Prsers re usully utomtilly generted from ontext-free grmmrs (e.g., y, bison, up, jv, sble) Exmple for hrter strem Lexil nlyzer lexer.next() lexer.peek() Prser prse tree or AST for i = 1 to 10 do [i] = x * 5; i 1 10 sg rr tms i x 5 for id(i) equl number(1) to number(10) do id() lbrket id(i) rbrket equl id(x) times number(5) semi CS553 Leture Snning nd Prsing 8 CS553 Leture Snning nd Prsing 9 2

3 Bottom-Up Prsing: Sht-Redue Grmmer (2) E -> E T (3) E -> T (4) T -> id Rightmost derivtion: expnd rightmost non-terminls first SbleCC, y, nd bison generte -redue prsers: LALR(1): look-hed, left-to-right, rightmost derivtion in reverse, 1 symbol lookhed LALR is prsing tble rution method, smller tbles thn nonil LR b S -> E -> E T -> E id -> E T id -> E id id -> T id id -> id id id CS553 Leture Snning nd Prsing 10 Sht-Redue Prsing Exmple $ Stk Input Ation $ $ T b T T $ S b b b b b redue (3) redue (2) redue (2) redue (1) ept (2) E -> E T (3) E -> T (4) T -> id CS553 Leture Snning nd Prsing 11 Sht-Redue Prsing Exmple (preedene problem) $ Stk Input Ation b * (2) E -> E T (3) E -> E * T (4) E -> T (5) T -> id CS553 Leture Snning nd Prsing 12 Syntx-direted Trnsltion: AST Constrution exmple Grmmer with prodution rules S: E { $$ = $1; }; E: E T { $$ = new node(, $1, $3); } T { $$ = $1; } ; T: { $$ = new lef( id, $1); }; Impliit prse tree for b E T E S E T T b AST for b CS553 Leture Snning nd Prsing 13 b 3

4 Using SbleCC to spey grmmr nd generte AST Prsing Terms Produtions st_progrm {-> progrm} = st_min_lss st_lss_del* {-> New progrm(st_min_lss.min_lss,[st_lss_del.lss_del])} ; st_exp_list {-> exp* } = {mny_rule} st_exp st_exp_rest* {-> [st_exp.exp, st_exp_rest.exp] } {empty_rule} {-> [] } ; st_exp_rest {-> exp* } = t_omm st_exp {-> [st_exp.exp] }; Abstrt Syntx Tree CFG (Context-free Grmmer) prodution rule terminl nonterminl FOLLOW(X): the set of terminls tht n immeditely follow X BNF (Bkus-Nur Form) nd EBNF (Extended BNF): equivlent to CFGs progrm = min_lss [lss_dels]:lss_del*; exp = {ll} exp t_id [rgs]:exp*... CS553 Leture Snning nd Prsing 14 CS553 Leture Snning nd Prsing 15 Prsing Terms ont Top-down prsing LL(1): left-to-right reding of s, leftmost derivtion, 1 symbol look-hed Preditive prser: n effiient non-bktrking top-down prser tht n hndle LL(1) More generlly reursive desent prsing my involve bktrking Bottom-up Prsing LR(1): left-to-right reding of s, rightmost derivtion in reverse, 1 symbol lookhed Sht-redue prsers: for exmple, bison, y, nd SbleCC generted prsers Methods for produing n LR prsing tble SLR, simple LR Cnonil LR, most powerful LALR(1) Conepts Compiltion stges in ompiler Snning, prsing, semnti nlysis, intermedite ode genertion, optimiztion, ode genertion Lexil nlysis or snning Tools: SbleCC, lex, flex, et. Syntti nlysis or prsing Tools: SbleCC, y, bison, et. CS553 Leture Snning nd Prsing 16 CS553 Leture Snning nd Prsing 17 4

5 Next Time Leture More undergrdute ompilers review CS553 Leture Snning nd Prsing 18 Lnguge Implementtion Timeline For entertinment purposes only! A-0 [Hopper] Fortrn [Bkus] Algol [Comm.] LISP [MCrthy] COBOL [Short Rnge Comm.] Prser genertors Simul [Dhl & Nygrd] BASIC [Kemeny & Kurtz] Dep. vetors [Krp et l.] Smlltlk [Ky] & PFC [Kennedy] Tre shed. [Fisher] Coloring reg. llo. [Chitin] 1 st RISC (IBM 801), Wolfe s thesis C [Stroustrup] Oml [INRIA] Drgon book [ASU] PDG [Fernte] Perl [Wll] SW pipelining [Lm] SSA [Cytron] 486 w/ he Sprse ond.. [Wegmn&Zdek] Superblok sheduling [Hwu] Jv [Gosling&Sun] Psl [Wirth] & 1 st upro [4004] C [Rithie] & ML [Milner et l.] Prolog [Colmerurer] Vlue numbering [Coke&Shwrtz] Copying GC [Cheney] Itnium ships & Jikes RVM [IBM] Modern DFA [Kildll] & Lmport s prllelism Lex & YACC [Johnson] GCD test [Bnerjee & Towle] Prfrse [Kuk] My v. must [Brth] Flow-sens. defined [Bnning] PRE [Morel et l.] CSU CS553 Leture Snning nd Prsing 19 5