Compilation: a bit about the target architecture

Transcription

1 Compilation: a bit about the target architecture MIF08 Laure Gonnord Laure.Gonnord@univ-lyon1.fr

2 Plan 1 The LEIA architecture in a nutshell 2 One example Laure Gonnord (Lyon1/FST) Compilation: a bit about the target architecture 2 / 12

3 The LEIA architecture in a nutshell Outline 1 The LEIA architecture in a nutshell 2 One example Laure Gonnord (Lyon1/FST) Compilation: a bit about the target architecture 3 / 12

4 The LEIA architecture in a nutshell Our target machine : LEIA Memory: bits words. Instructions are also encoded in 16-bits words. Registers PC, IR + 16 general purpose registers R0,...,R15. We will use only the 8 first registers We have a assembler and a simulator (see Lab 1) Laure Gonnord (Lyon1/FST) Compilation: a bit about the target architecture 4 / 12

5 The LEIA architecture in a nutshell LEIA ISA See companion document. Laure Gonnord (Lyon1/FST) Compilation: a bit about the target architecture 5 / 12

6 The LEIA architecture in a nutshell Example : ADD instruction add dr sr1 sr2, does dr <- sr1 + sr2. All operands are registers. Example : add r1 r2 r3 executes r1 <- r2 + r3. add dr sr1 imm4, does dr <- sr + imm4. The last operand is an immediate value (on 4bits) encoded in the instruction. Example : add r1 r2 5 executes r1 <- r Laure Gonnord (Lyon1/FST) Compilation: a bit about the target architecture 6 / 12

7 The LEIA architecture in a nutshell LEIA ADD : encoding A bit specifies the addressing mode: class action ADD reg r d r i + r j d i j ADD imm4 r d r i + ext(j) d i j Example: assemble add r1 r2 5 Laure Gonnord (Lyon1/FST) Compilation: a bit about the target architecture 7 / 12

8 The LEIA architecture in a nutshell LEIA Memory access instructions store wmem sr [dr], does mem[dr] <- sr. load rmem dr [sr], does dr <- mem[sr]. indirect addressing: the address must be stored in a register mem[rj] the value at the address stored in r j. class action wmem mem[r j ] r i i j rmem r d mem[r j ] d j See also the copy instruction. Laure Gonnord (Lyon1/FST) Compilation: a bit about the target architecture 8 / 12

9 The LEIA architecture in a nutshell LEIA: branching Unconditional branching: jump c, does PC <- a + c where a is the current instruction address. class action jump jump c Test and branch: snif r1 op r2, skip the next instruction if the test is true class action snif /0 condition i j Laure Gonnord (Lyon1/FST) Compilation: a bit about the target architecture 9 / 12

10 One example Outline 1 The LEIA architecture in a nutshell 2 One example Laure Gonnord (Lyon1/FST) Compilation: a bit about the target architecture 10 / 12

11 One example Ex : Assembly code - demo 1 ; ;; simple LEIA assembly demo leth r1 0 ; first op letl r1 5.set r3 b ; second, from memory 6 rmem r2 [r3] add r4 r1 r2 ; add and store in memory.set r3 resu wmem r4 [r3] 11 jump 0 ; stop b: resu:.word 2 16.reserve 1 ; reservation of a memory cell python3 asm.py demo.s && LEIA -s demo.obj Laure Gonnord (Lyon1/FST) Compilation: a bit about the target architecture 11 / 12

12 One example LEIA Exercises see TD sheet. Laure Gonnord (Lyon1/FST) Compilation: a bit about the target architecture 12 / 12

13 Introduction MIF08 Laure Gonnord

14 What s compilation? source language compiler targuet language errors Laure Gonnord (Lyon1/FST) Introduction 2 / 7

15 Compilation toward the machine language We immediatly think of the translation of a high-level language (C,Java,OCaml) into the machine language of a processor (Pentium, PowerPC... ) % gcc -o sum sum.c i n t main ( i n t argc, char argv ) { i n t i, s = 0; for ( i = 0; i <= 100; i ++) s += i i ; p r i n t f ( " = %d \ n ", s ) ; } But this is only one aspect, we will see more! Laure Gonnord (Lyon1/FST) Introduction 3 / 7

16 Course Objective Be familiar with the mecanisms inside a (simple) compiler. Beyond the scope: compilers optimisations of the 21 th century. Laure Gonnord (Lyon1/FST) Introduction 4 / 7

17 Course Content Syntax Analysis : lexing, parsing, AST Evaluators Code generation (Code Optimisation) Support language: Python 2.7 Frontend infrastructure : ANTLR 4. Laure Gonnord (Lyon1/FST) Introduction 5 / 7

18 Course Organization 4 TD groups: S. Brandel, L. Gonnord, Matthieu Moy, X.Urbain (@univ-lyon1.fr) 6 (or 7) TP groups: S. Brandel, T. Excoffier, J. Fernandez, G. Salagnac (Insa), G. Bouchard, M. Moy, X.Urbain. The official URL : Laure Gonnord (Lyon1/FST) Introduction 6 / 7

19 Evaluation One quick during an exercise session. Some of the lab exercises. A final exam. Laure Gonnord (Lyon1/FST) Introduction 7 / 7

20 Syntax Analysis MIF08 Laure Gonnord

21 Goal of this chapter Understand the syntaxic structure of a language; Separate the different steps of syntax analysis; Be able to write a syntax analysis tool for a simple language; Remember: syntax semantics. Laure Gonnord (Lyon1/FST) Syntax Analysis 2 / 29

22 Syntax analysis steps How do you read text? Text=a sequence of symbols (letters, spaces, punctuation); Group symbols into tokens: Words: groups of letters; Punctuation; Spaces. Laure Gonnord (Lyon1/FST) Syntax Analysis 3 / 29

23 Syntax analysis steps How do you read text? Text=a sequence of symbols (letters, spaces, punctuation); Group symbols into tokens: Lexical analysis Words: groups of letters; Punctuation; Spaces. Laure Gonnord (Lyon1/FST) Syntax Analysis 3 / 29

24 Syntax analysis steps How do you read text? Text=a sequence of symbols (letters, spaces, punctuation); Group symbols into tokens: Lexical analysis Words: groups of letters; Punctuation; Spaces. Group tokens into: Propositions; Sentences. Laure Gonnord (Lyon1/FST) Syntax Analysis 3 / 29

25 Syntax analysis steps How do you read text? Text=a sequence of symbols (letters, spaces, punctuation); Group symbols into tokens: Lexical analysis Words: groups of letters; Punctuation; Spaces. Group tokens into: Parsing Propositions; Sentences. Laure Gonnord (Lyon1/FST) Syntax Analysis 3 / 29

26 Syntax analysis steps How do you read text? Text=a sequence of symbols (letters, spaces, punctuation); Group symbols into tokens: Lexical analysis Words: groups of letters; Punctuation; Spaces. Group tokens into: Parsing Propositions; Sentences. Then proceed with word meanings: Definition of each word. ex: a dog is a hairy mammal, that barks and... Role in the phrase: verb, subject,... Laure Gonnord (Lyon1/FST) Syntax Analysis 3 / 29

27 Syntax analysis steps How do you read text? Text=a sequence of symbols (letters, spaces, punctuation); Group symbols into tokens: Lexical analysis Words: groups of letters; Punctuation; Spaces. Group tokens into: Parsing Propositions; Sentences. Then proceed with word meanings: Semantics Definition of each word. ex: a dog is a hairy mammal, that barks and... Role in the phrase: verb, subject,... Laure Gonnord (Lyon1/FST) Syntax Analysis 3 / 29

28 Syntax analysis steps How do you read text? Text=a sequence of symbols (letters, spaces, punctuation); Group symbols into tokens: Lexical analysis Words: groups of letters; Punctuation; Spaces. Group tokens into: Parsing Propositions; Sentences. Then proceed with word meanings: Semantics Definition of each word. ex: a dog is a hairy mammal, that barks and... Role in the phrase: verb, subject,... Syntax analysis=lexical analysis+parsing Laure Gonnord (Lyon1/FST) Syntax Analysis 3 / 29

29 Lexical Analysis aka Lexing Outline 1 Lexical Analysis aka Lexing 2 Parsing Laure Gonnord (Lyon1/FST) Syntax Analysis 4 / 29

30 Lexical Analysis aka Lexing What for? int y = *x ; = [TINT, VAR("y"), EQ, INT(12), PLUS, INT(4), TIMES, VAR("x"), SCOL] Group characters into a list of tokens, e.g.: The word int stands for type integer; A sequence of letters stands for a variable; A sequence of digits stands for an integer;... Laure Gonnord (Lyon1/FST) Syntax Analysis 5 / 29

31 Lexical Analysis aka Lexing What s behind From a Regular Language, produce a Finite State Machine (see LIF15) Laure Gonnord (Lyon1/FST) Syntax Analysis 6 / 29

32 Lexical Analysis aka Lexing Tools: lexical analyzer constructors Lexical analyzer constructor: builds an automaton from a regular language definition; Ex: Lex (C), JFlex (Java), OCamllex, ANTLR (multi),... input: a set of regular expressions with actions (Toto.g4); output: a file(s) (Toto.java) that contains the corresponding automaton. Laure Gonnord (Lyon1/FST) Syntax Analysis 7 / 29

33 Lexical Analysis aka Lexing Analyzing text with the compiled lexer The input of the lexer is a text file; Execution: Checks that the input is accepted by the compiled automaton; Executes some actions during the automaton traversal. Laure Gonnord (Lyon1/FST) Syntax Analysis 8 / 29

34 Lexical Analysis aka Lexing Lexing tool for Java: ANTLR The official webpage : (BSD license); ANTLR is both a lexer and a parser; ANTLR is multi-language (not only Java). During the labs; we will use the Python back-end (here, demo in java) Laure Gonnord (Lyon1/FST) Syntax Analysis 9 / 29

35 Lexical Analysis aka Lexing ANTLR lexer format and compilation.g4 grammar { // Some init code... { // Some global variables } // More optional blocks are available --->> lex rules Compilation: antlr4 Toto.g4 javac *.java grun Toto r // produces several Java files // compiles into xx.class files // Run analyzer with starting rule r Laure Gonnord (Lyon1/FST) Syntax Analysis 10 / 29

36 Lexical Analysis aka Lexing Lexing with ANTLR: example Lexing rules: Must start with an upper-case letter; Follow the usual extended regular-expressions syntax (same as egrep, sed... A simple example grammar H e l l o ; / / This r u l e i s a c t u a l l y a parsing r u l e r : HELLO ID ; / / match " h e l l o " f o l l o w e d by an i d e n t i f i e r HELLO : 'hello ' ; / / beware the s i n g l e quotes ID : [ a z ]+ ; / / match lower case i d e n t i f i e r s WS : [ \ t \ r \ n ]+ > s kip ; / / skip spaces, t a b s, newlines Laure Gonnord (Lyon1/FST) Syntax Analysis 11 / 29

37 Lexical Analysis aka Lexing Lexing - more than regular languages Counting in ANTLR - CountLines.g4 l e x e r grammar CountLines; / / Members can be accessed i n any r u l { i n t nblines=0 ; } NEWLINE : [ \ r \ n ] { nblines++ ; System. out. p r i n t l n ( " Current l i n e s : " +nblines ) ; } ; SK : ( [ a z ]+ [ \ t ] + ) > s kip ; antlr4 Toto.g4 javac *.java grun Toto 'tokens' // produces several Java files // compiles into xx.class files // Run the lexical analyser only Laure Gonnord (Lyon1/FST) Syntax Analysis 12 / 29

38 Parsing Outline 1 Lexical Analysis aka Lexing 2 Parsing Semantic actions / Attributes Laure Gonnord (Lyon1/FST) Syntax Analysis 13 / 29

39 Parsing What s Parsing? Relate tokens by structuring them. Flat tokens [TINT, VAR("y"), EQ, INT(12), PLUS, INT(4), TIMES, VAR("x"), SCOL] Parsing Yes/No + Structured tokens = int y + 12 * 4 x int Laure Gonnord (Lyon1/FST) Syntax Analysis 14 / 29

40 Parsing Analysis Phases source code lexical analysis sequence of lexems (tokens) syntactic analysis (Parsing) abstract syntax tree (AST ) semantic analysis abstract syntax (+ symbol table) Laure Gonnord (Lyon1/FST) Syntax Analysis 15 / 29

41 Parsing What s behind? From a Context-free Grammar, produce a Stack Automaton (see LIF15). Laure Gonnord (Lyon1/FST) Syntax Analysis 16 / 29

42 Parsing Tools: parser generators Parser generator: builds a stack automaton from a grammar definition; Ex: yacc(c), javacup (Java), OCamlyacc, ANTLR,... input : a set of grammar rules with actions (Toto.g4); output : a file(s) (Toto.java) that contains the corresponding stack automaton. Laure Gonnord (Lyon1/FST) Syntax Analysis 17 / 29

43 Parsing Lexing vs Parsing Lexing supports ( regular) languages; We want more (general) languages rely on context-free grammars; To that intent, we need a way: To declare terminal symbols (tokens); To write grammars. Use both Lexing rules and Parsing rules. Laure Gonnord (Lyon1/FST) Syntax Analysis 18 / 29

44 Parsing From a grammar to a parser The grammar must be context-free: S-> asb S-> eps The grammar rules are specified as Parsing rules; a and b are terminal tokens, produced by Lexing rules. On board: notion of derivation tree (see also exercise session2) Laure Gonnord (Lyon1/FST) Syntax Analysis 19 / 29

45 Parsing Parsing with ANTLR: example 1/2 AnBnLexer.g4 lexer grammar AnBnLexer; // Lexing rules: recognize tokens A: 'a ' ; B: 'b ' ; WS : [\ t \ r \ n ]+ -> skip ; // skip spaces, tabs, newlines Laure Gonnord (Lyon1/FST) Syntax Analysis 20 / 29

46 Parsing Parsing with ANTLR: example 2/2 AnBnParser.g4 parser grammar A nbnparser; options { tokenvocab = AnBnLexer; } // extern tokens definition // Parsing rules: structure tokens together prog : s EOF ; // EOF: predefined end - of - file token s : A s B ; // nothing for empty alternative Laure Gonnord (Lyon1/FST) Syntax Analysis 21 / 29

47 Parsing ANTLR expressivity LL(*) At parse-time, decisions gracefully throttle up from conventional fixed k 1 lookahead to arbitrary lookahead. Further reading (PLDI 11 paper, T. Parr, K. Fisher) Laure Gonnord (Lyon1/FST) Syntax Analysis 22 / 29

48 Parsing Left recursion ANTLR permits left recursion: a: a b; But not indirect left recursion. X 1... X n There exist algorithms to eliminate indirect recursions. Laure Gonnord (Lyon1/FST) Syntax Analysis 23 / 29

49 Parsing Lists ANTLR permits lists: prog: statement + ; Read the documentation! https: //github.com/antlr/antlr4/blob/master/doc/index.md Laure Gonnord (Lyon1/FST) Syntax Analysis 24 / 29

50 Parsing Semantic actions / Attributes Outline 1 Lexical Analysis aka Lexing 2 Parsing Semantic actions / Attributes Laure Gonnord (Lyon1/FST) Syntax Analysis 25 / 29

51 Parsing Semantic actions / Attributes Semantic actions Semantic actions: code executed each time a grammar rule is matched. Printing as a semantic action in ANTLR s : A s B { System. out. println (" rule s"); } s : A s B { print (" rule s"); } // python Right rule : Python/Java/C++, depending on the back-end antlr4 -Dlanguage=Python3 We can do more than acceptors. Laure Gonnord (Lyon1/FST) Syntax Analysis 26 / 29

52 Parsing Semantic actions / Attributes Semantic actions - attributes An attribute is a set attached to non-terminals/terminals of the grammar They are usually of two types: synthetized: sons father. inherited: the converse. Laure Gonnord (Lyon1/FST) Syntax Analysis 27 / 29

53 Parsing Semantic actions / Attributes Semantic attributes for numerical expressions 1/2 e ::= c constant x variable e + e add e e mult... Let s come to an attribution. On board. Laure Gonnord (Lyon1/FST) Syntax Analysis 28 / 29

54 Parsing Semantic actions / Attributes Semantic attributes 2/2 : Implem Implementation of the former actions (java): ArithExprParser.g4 parser grammar A r i t h E x p r P a r s e r ; options {tokenvocab=arithexprlexer; } prog : expr EOF { System. out. p r i n t l n ( " R e s u l t : " +$expr. v a l ) ; } ; expr r e t u r n s [ i n t v a l ] : / / expr has an i n t e g e r a t t r i b u t e LPAR e=expr RPAR { $val=$e. v a l ; } INT { $val=$int. i n t ; } / / i m p l i c i t a t t r i b u t e f o r INT e1=expr PLUS e2=expr / / name sub p a r t s { $val=$e1. v a l +$e2. v a l ; } / / access a t t r i b u t e s e1=expr MINUS e2=expr { $val=$e1. val $e2. v a l ; } ; Laure Gonnord (Lyon1/FST) Syntax Analysis 29 / 29