Compiler and Language Processing Tools

Transcription

1 Compiler and Language Processing Tools Summer Term 2011 Introduction Prof. Dr. Arnd Poetzsch-Heffter Software Technology Group TU Kaiserslautern Prof. Dr. Arnd Poetzsch-Heffter Compilers 1

2 Outline Introduction 1. Introduction Language Processing Tools Application Domains Tasks of Language-Processing Tools Examples 2. Language Processing Terminology and Requirements Compiler Architecture 3. Compiler Construction Prof. Dr. Arnd Poetzsch-Heffter Compilers 2

3 Introduction Language processing tools Language Processing Tools Processing of source texts in (source) languages Analysis of (source) texts Translation to target languages Prof. Dr. Arnd Poetzsch-Heffter Compilers 3

4 Introduction Language processing tools (2) Language Processing Tools Typical source languages Programming languages: C, C++, C#, Java, Scala, Haskell, ML, Smalltalk, Prolog Script languages: JavaScript, bash Languages for configuration management: make, ant Application and tool-specific languages: Excel, JFlex, CUPS Specification languages: Z, CASL, Isabelle/HOL Formatting and data description languages: LaTeX, HTML, XML Design and architecture description languages: UML, SDL, VHDL, Verilog Prof. Dr. Arnd Poetzsch-Heffter Compilers 4

5 Introduction Language processing tools (3) Language Processing Tools Typical target languages Assembly, machine, and bytecode languages Programming language Data and layout description languages Languages for printer control... Prof. Dr. Arnd Poetzsch-Heffter Compilers 5

6 Introduction Language processing tools (4) Language Processing Tools Language implementation tasks Tool support for language processing Integration into existing systems Connection to other systems Prof. Dr. Arnd Poetzsch-Heffter Compilers 6

7 Application domains Introduction Application Domains Programming environments Context-sensitive editors, class browers Graphical programming tools Pre-processors Compilers Interpreters Debuggers Run-time environments (loading, linking, execution, memory management) Prof. Dr. Arnd Poetzsch-Heffter Compilers 7

8 Application domains (2) Introduction Application Domains Generation of programs from design documents (UML) Program comprehension, re-engineering Design and implementation of domain-specific languages Robot control Simulation tools Spread sheets, active documents Web technology Analysis of Web sites Active Web sites (with integrated functionality) Abstract platforms, e.g. JVM,.NET Optimization of caching Prof. Dr. Arnd Poetzsch-Heffter Compilers 8

9 Introduction Application Domains Related fields Formal languages, language specification and design Programming and specification languages Programming, software engineering, software generation, software architecture System software, computer architecture Prof. Dr. Arnd Poetzsch-Heffter Compilers 9

10 Introduction Tasks of Language-Processing Tools Tasks of Language-Processing Tools Source Code Source Code Source Code Input Data Analyser Translation Interpreter Analysis Results Target Code Output Data Analysis, translation and interpretation are often combined. Prof. Dr. Arnd Poetzsch-Heffter Compilers 10

11 Introduction Tasks of Language-Processing Tools Tasks of Language-Processing Tools (2) 1. Translation Compiler implements analysis and translation OS and real machine implement interpretation Pros: Most efficient solution One interpreter for different programming languages Prerequisite for other solutions Prof. Dr. Arnd Poetzsch-Heffter Compilers 11

12 Introduction Tasks of Language-Processing Tools Tasks of Language-Processing Tools (3) 2. Direct interpretation Interpreter implements all tasks. Examples: JavaScript, command line languages (bash) Pros: No translation necessary (but analysis at run-time) Prof. Dr. Arnd Poetzsch-Heffter Compilers 12

13 Introduction Tasks of Language-Processing Tools Tasks of Language-Processing Tools (4) 3. Abstract and virtual machines Compiler implements analysis and translation to abstract machine code Abstract machine works as interpreter Examples: Java/JVM, C#,.NET Pros: Platform independent (portability, mobile code) Self-modifing programs possible 4. Other combinations Prof. Dr. Arnd Poetzsch-Heffter Compilers 13

14 Example: Analysis Introduction Beispiel: (Analyse) Examples package b1_1 ; class Weltklasse extends Superklasse implement BesteBohnen {Qualifikation studieren ( Arbeit schweiss) { return new Qualifikation ();}} Superklasse.class BesteBohnen.class Qualifikation.class Arbeit.class... javac-analysator b1_1/weltklasse.java:4: '{' expected. extends Superklasse ^ 1 error Prof. Dr. Arnd Poetzsch-Heffter Compilers 14

15 Example: Translation Introduction Examples Beispiel 1: (Übersetzung) package b1_1; class Weltklasse extends Superklasse implements BesteBohnen { Qualifikation studieren ( Arbeit schweiss ) { return new Qualifikation(); }} Superklasse.class BesteBohnen.class Qualifikation.class Arbeit.class... javac Compiled from Weltklasse.java class b1_1/weltklasse extends... implements... { b1_1/weltklasse(); b1_1.qualifikation studieren(...); } Method b1_1/weltklasse()... Method b1_1.qualifikation studieren(...) A. Poetzsch-Heffter, TU Kaiserslautern 9 Prof. Dr. Arnd Poetzsch-Heffter Compilers 15

16 Introduction Example: Translation (2) Result of translation A. Poetzsch-Heffter, TU Kaiserslautern 10 Examples Compiled from Weltklasse.java class b1_1/weltklasse extends b1_1.superklasse implements b1_1.bestebohnen { b1_1/weltklasse(); b1_1.qualifikation studieren(b1_1.arbeit); } Method b1_1/weltklasse() 0 aload_0 1 invokespecial #6 <Method b1_1.superklasse()> 4 return Method b1_1.qualifikation studieren(b1_1.arbeit) 0 new #2 <Class b1_1.qualifikation> 3 dup 4 invokespecial #5 <Method b1_1.qualifikation()> 7 areturn Prof. Dr. Arnd Poetzsch-Heffter Compilers 16 Beispiel 1: (Fortsetzung)

17 Example 2: Translation Introduction Examples Beispiel 2: (Übersetzung) int main() { printf("willkommen zur Vorlesung!"); return 0; } gcc.file "hello_world.c".version "01.01" gcc2_compiled.:.section.rodata.lc0:.string "Willkommen zur Vorlesung!".text.align 16.globl main.type main,@function main: pushl %ebp movl %esp,%ebp subl $8,%esp A. Poetzsch-Heffter, TU Kaiserslautern 11 Prof. Dr. Arnd Poetzsch-Heffter Compilers 17

18 Introduction Example 2: Translation (2) Result of translation Beispiel 2: (Fortsetzung) Examples.file "hello_world.c".version "01.01" gcc2_compiled.:.section.rodata.lc0:.string "Willkommen zur Vorlesung!".text.align 16.globl main.type main: pushl %ebp movl %esp,%ebp subl $8,%esp addl $-12,%esp pushl $.LC0 call printf addl $16,%esp xorl %eax,%eax jmp.l2.p2align 4,,7.L2: movl %ebp,%esp popl %ebp ret.lfe1:.size main,.lfe1-main.ident "GCC: (GNU) (release)" A. Poetzsch-Heffter, TU Kaiserslautern 12 Prof. Dr. Arnd Poetzsch-Heffter Compilers 18

19 Example 3: Translation Beispiel 3: Introduction Examples (Übersetzung) groovy.tex (104 bytes) \documentclass{article} \begin{document} \vspace*{7cm} \centerline{\huge\bf It s groovy} \end{document} latex groovy.dvi (207 bytes, binary)... dvips groovy.ps (7136 bytes) %!PS-Adobe-2.0 %%Creator: dvips(k) %%Title: groovy.dvi... Prof. Dr. Arnd Poetzsch-Heffter A. Poetzsch-Heffter, Compilers TU Kaiserslautern 13 19

20 Example: Interpretation Introduction Examples.class File.class-Datei iload_1 15 iload_2 16 idiv 17 istore_3... Eingabedaten Input Data iload_1 15 iload_2 16 idiv 17 istore_3... Java Virtual Machine (JVM) Output Ausgabedaten Data Prof. Dr. Arnd Poetzsch-Heffter Compilers 20

21 Introduction Examples Example: Combined technique Java implementation with just-in-time (JIT) compiler Java-Überset- Source zungseinheit Code Unit (JIT=Just in time) javac Analysator Analyzer Translator Übersetzer.class-Datei file Eingabedaten Input Data Java Byte Code JIT-Übersetzer Translator JVM Maschinencode Machine Code reale Real Machine Maschine/Hardware / Ausgabedaten Output Data Prof. Dr. Arnd Poetzsch-Heffter Compilers 21

22 Language Processing Terminology and Requirements Language processing: The task of translation Source Code Translator Error Message or Target Code Translator (in a broader sense): Analysis, optimization and translation Source code: Input (string) for translator in syntax of source language (SL) Target Code: Output (string) of translator in syntax of target language (TL) Prof. Dr. Arnd Poetzsch-Heffter Compilers 22

23 Language Processing Phases of language processing Terminology and Requirements Analysis of input: Program text Specification Diagrams Dependant on target of implementation Transformation (XSLT, refactoring) Pretty printing, formatting Semantic analysis (program comprehension) Optimization (Actual) translation Prof. Dr. Arnd Poetzsch-Heffter Compilers 23

24 Language Processing Compile time vs. run-time Terminology and Requirements Compile time: during run-time of compiler/translator Static: All information/aspects known at compile time, e.g.: Type checks Evaluation of constant expressions Relative addresses Run-time: during run-time of compiled program Dynamic: All information that are not statically known, e.g.: Allocation of dynamic arrays Bounds check of arrays Dynamic binding of methods Memory management of recursive procedures For dynamic aspects that cannot be handled at compile time, the compiler generates code that handles these aspects at run-time. Prof. Dr. Arnd Poetzsch-Heffter Compilers 24

25 Language Processing Terminology and Requirements What is a good compiler? Prof. Dr. Arnd Poetzsch-Heffter Compilers 25

26 Language Processing Terminology and Requirements Requirements for translators Error handling (static/dynamic) Efficient target code Choice: Fast translation with slow code vs. slow translation with fast code Semantically correct translation Prof. Dr. Arnd Poetzsch-Heffter Compilers 26

27 Language Processing Terminology and Requirements Semantically correct translation Intuitive definition: Compiled program behaves according to language definition of source language. Formal definition: semsl: SL_Program SL_Data SL_Data semtl: TL_Program TL_Data TL_Data compile: SL_Program TL_Program code: SL_Data TL_Data decode: TL_Data SL_Data Semantic correctness: semsl(p,d) = decode(semtl(compile(p), code(d))) Prof. Dr. Arnd Poetzsch-Heffter Compilers 27

28 Language Processing Compiler Architecture Compiler Architecture Source Code as String Analysis Token Stream Scanner Parser Decorated Syntax Tree (Close to SL) Translator Intermediate Language Synthesis Attribution & Optimization Attribution & Optimization Syntax Tree Name and Type Analysis Code Generator Target Code as String Peep Hole Optimization Prof. Dr. Arnd Poetzsch-Heffter Compilers 28

29 Language Processing Compiler Architecture Properties of compiler architectures Phases are conceptual units of translation Phases can be interleaved Design of phases depends on source language, target language and design decisions Phase vs. pass (phase can comprise more than one pass.) Separate translation of pogram parts (Interface information must be accessible.) Combination with other architecture decisions: Common intermediate language Prof. Dr. Arnd Poetzsch-Heffter Compilers 29

30 Language Processing Common intermediate language Compiler Architecture Source Language 1 Source Language 2... Source Language n Intermediate Language Target Language 1 Target Language 2... Target Language m Prof. Dr. Arnd Poetzsch-Heffter Compilers 30

31 Language Processing Compiler Architecture Dimensions of compiler construction Programming languages Sequential procedural, imperative, OO-languages Functional, logical languages Parallel languages/language constructs Target languages/machines Code for abstract machines Assembler Machine languages (CISC, RISC,...) Multi-processor/multi-core architectures Memory hierarchy Translation tasks: analysis, optimization, synthesis Construction techniques and tools: bootstrapping, generators Portability, specification, correctness Prof. Dr. Arnd Poetzsch-Heffter Compilers 31

32 Compiler Construction Compiler construction techniques 1. Stepwise construction Construction with compiler for different language Construction with compiler for different machine Bootstrapping 2. Compiler-compiler: Tools for compiler generation Scanner generators (regular expressions) Parser generators (context-free grammars) Attribute evaluation generators (attribute grammar) Code generator generators (machine specification) Interpreter generators (semantics of language) Other phase-specific tools 3. Special programming techniques General technique: syntax-driven Special technique: recursive descend Prof. Dr. Arnd Poetzsch-Heffter Compilers 32

33 Compiler Construction Stepwise construction Programming typically depends on an existing compiler for the implementation language. For compiler construction, this does not hold in general. Source, target, and implementation languages of compilers can be denoted in T-diagrams. SL TL CL T-diagram denotes compiler from source language SL to target language TL (SL TL compiler) written in language CL. Prof. Dr. Arnd Poetzsch-Heffter Compilers 33

34 Compiler Construction Construction with compiler for different language Given: C ML (machine language) compiler in ML Construct: SL ML compiler in ML Solution: Develop SL ML compiler in C, translate that compiler from C ML by using the existing C ML compiler to be developed by translation SL ML SL ML C C ML ML ML existing Prof. Dr. Arnd Poetzsch-Heffter Compilers 34

35 Compiler Construction Construction with compiler for different machine Construct: C ML 1 compiler in ML 1 Given 1. C ML 1 compiler in C 2. C ML 2 compiler in ML 2 Method: construct cross compiler First step given C ML 1 cross compiler C ML 1 C C ML 2 ML 2 ML 2 given Prof. Dr. Arnd Poetzsch-Heffter Compilers 35

36 Compiler Construction Construction with compiler for different machine (2) Second step given C ML 1 resulting compiler C ML 1 C C ML 1 ML 1 ML 2 cross compiler Prof. Dr. Arnd Poetzsch-Heffter Compilers 36

37 Compiler Construction Bootstrapping Construct: SL ML compiler in ML Suppose: yet no compiler exists Method: 1. Construct partial language SL i of SL such that SL 0 SL 1 SL 2... SL 2. Implement SL 0 compiler for ML in ML 3. Implement SL i+1 compiler for ML in SL i 4. Create SL i+1 compiler for ML in ML Prof. Dr. Arnd Poetzsch-Heffter Compilers 37

38 Bootstrapping (2) Compiler Construction by extension SL ML SL ML SL 2 ML SL 2 SL 2 ML ML SL 1 ML SL 1 SL 1 ML ML SL 0 SL 0 ML ML ML by translation manually Prof. Dr. Arnd Poetzsch-Heffter Compilers 38

39 Compiler Construction Recommended reading Wilhelm, Maurer: Chap. 1, Introduction (pp. 1 5) Chap. 6, Structure of Compilers (pp ) Appel Chap. 1, Introduction (pp. 3 14) Prof. Dr. Arnd Poetzsch-Heffter Compilers 39