Compiler and Language Processing Tools

Transcription

1 Compiler and Language Processing Tools Summer Term 2009 Introduction Dr.-Ing. Ina Schaefer Software Technology Group TU Kaiserslautern Ina Schaefer Compilers 1

2 Outline Introduction 1. Introduction Overview and Application Domains Tasks of Language-Processing Tools Examples 2. Language Processing Terminology and Requirements Compiler Architecture 3. Compiler Construction Ina Schaefer Compilers 2

3 Introduction Language Processing Tools Overview and Application Domains Processing of Source Texts in Source Languages Analysis of Source Texts Translation to Target Languages Ina Schaefer Compilers 3

4 Introduction Language Processing Tools (2) Overview and Application Domains Typical Source Languages Programming Languages: C, C++, C#, Java, ML, Smalltalk, Prolog, Script Languages (JavaScript), bash Languages for Configuration Management: make, ant Application and Tool-Specific Languages: Excel, JFlex, CUPS Specification Languages: Z, CASL, Isabelle/HOL Formatting and Data Description Languages: LaTeX, HTML, XML Design and Architecture Description Languages: UML, SDL, VHDL, Verilog Ina Schaefer Compilers 4

5 Introduction Language Processing Tools (3) Overview and Application Domains Typical Target Languages Assembly and Machine Languages Programming Language Data and layout Description Languages Languages for Printer Control Ina Schaefer Compilers 5

6 Introduction Language Processing Tools (4) Overview and Application Domains Language Implementation Tasks Tool Support for Language Processing Integration into Existing Systems Connection to Other Systems Ina Schaefer Compilers 6

7 Application Domains Introduction Overview and Application Domains Programming Environments Context-sensitive Editors, Class Browers Graphical Programming Tools Pre-Processors Compilers Interpreters Debuggers Run-time Environments (loading, linking, execution, memory management) Ina Schaefer Compilers 7

8 Introduction Application Domains (2) Overview and Application Domains Generation of Programs from Design Documents (UML) Program Comprehension, Re-engineering Design and Implementation of Application-specific Languages Robot Control Simulation Tools Spread Sheets, Active Documents Web Technology Analysis of Web Sites Active Websites (with integrated functionality) Abstract Platforms, e.g. JVM,.NET Optimization of Caching Ina Schaefer Compilers 8

9 Introduction Overview and Application Domains Related Fields Formal Languages, Language Specification and Design Programming and Specification Languages Programming, Software Engineering, Sotware Generation, Software Architecture System Software, Computer Architecture Ina Schaefer Compilers 9

10 Introduction Tasks of Language-Processing Tools Tasks of Language-Processing Tools Source Code Source Code Source Code Input Data Analyser Translation Interpreter Analysis Results Target Code Output Data Analysis, Translation and Interpretation are often combined. Ina Schaefer Compilers 10

11 Introduction Tasks of Language-Processing Tools Tasks of Language-Processing Tools (2) 1. Translation Compiler implements Analysis and Translation OS and Real Machine implement Interpretation Pros: Most efficient solution One interpreter for all programming languages Prerequisite for other solutions Ina Schaefer Compilers 11

12 Introduction Tasks of Language-Processing Tools Tasks of Language-Processing Tools (3) 2. Direct Interpretation Interpreter implements all tasks. Examples: Java Script, Command Line Languages (bash) Pros: No translation necessary (but analysis at run-time) Ina Schaefer Compilers 12

13 Introduction Tasks of Language-Processing Tools Tasks of Language-Processing Tools (4) 3. Abstract and Virtual Machines Compiler implements Analysis and Translation to Abstract Machine Code Abstract Machine works as Interpreter Examples: Java/JVM, C#ṄET Pros: Platform independent (portability, mobile code) Self-modifing programs possible 4. Other Combinations Ina Schaefer Compilers 13

14 Example: Analysis Introduction Beispiel: (Analyse) Examples package b1_1 ; class Weltklasse extends Superklasse implement BesteBohnen {Qualifikation studieren ( Arbeit schweiss) { return new Qualifikation ();}} Superklasse.class BesteBohnen.class Qualifikation.class Arbeit.class... javac-analysator b1_1/weltklasse.java:4: '{' expected. extends Superklasse ^ 1 error Ina Schaefer Compilers 14

15 Example: Translation Introduction Examples Beispiel 1: (Übersetzung) package b1_1; class Weltklasse extends Superklasse implements BesteBohnen { Qualifikation studieren ( Arbeit schweiss ) { return new Qualifikation(); }} Superklasse.class BesteBohnen.class Qualifikation.class Arbeit.class... javac Compiled from Weltklasse.java class b1_1/weltklasse extends... implements... { b1_1/weltklasse(); b1_1.qualifikation studieren(...); } Method b1_1/weltklasse()... Method b1_1.qualifikation studieren(...) A. Poetzsch-Heffter, TU Kaiserslautern 9 Ina Schaefer Compilers 15

16 Introduction Example: Translation (2) Result of Translation A. Poetzsch-Heffter, TU Kaiserslautern 10 Examples Compiled from Weltklasse.java class b1_1/weltklasse extends b1_1.superklasse implements b1_1.bestebohnen { b1_1/weltklasse(); b1_1.qualifikation studieren(b1_1.arbeit); } Method b1_1/weltklasse() 0 aload_0 1 invokespecial #6 <Method b1_1.superklasse()> 4 return Method b1_1.qualifikation studieren(b1_1.arbeit) 0 new #2 <Class b1_1.qualifikation> 3 dup 4 invokespecial #5 <Method b1_1.qualifikation()> 7 areturn Ina Schaefer Compilers 16 Beispiel 1: (Fortsetzung)

17 Example 2: Translation Introduction Examples Beispiel 2: (Übersetzung) int main() { printf("willkommen zur Vorlesung!"); return 0; } gcc.file "hello_world.c".version "01.01" gcc2_compiled.:.section.rodata.lc0:.string "Willkommen zur Vorlesung!".text.align 16.globl main.type main,@function main: pushl %ebp movl %esp,%ebp subl $8,%esp A. Poetzsch-Heffter, TU Kaiserslautern 11 Ina Schaefer Compilers 17

18 Introduction Example 2: Translation (2) Result of Translation Beispiel 2: (Fortsetzung) Examples.file "hello_world.c".version "01.01" gcc2_compiled.:.section.rodata.lc0:.string "Willkommen zur Vorlesung!".text.align 16.globl main.type main: pushl %ebp movl %esp,%ebp subl $8,%esp addl $-12,%esp pushl $.LC0 call printf addl $16,%esp xorl %eax,%eax jmp.l2.p2align 4,,7.L2: movl %ebp,%esp popl %ebp ret.lfe1:.size main,.lfe1-main.ident "GCC: (GNU) (release)" A. Poetzsch-Heffter, TU Kaiserslautern 12 Ina Schaefer Compilers 18

19 Example 3: Translation Beispiel 3: Introduction Examples (Übersetzung) groovy.tex (104 bytes) \documentclass{article} \begin{document} \vspace*{7cm} \centerline{\huge\bf It s groovy} \end{document} latex groovy.dvi (207 bytes, binary)... dvips groovy.ps (7136 bytes) %!PS-Adobe-2.0 %%Creator: dvips(k) %%Title: groovy.dvi... Ina Schaefer A. Poetzsch-Heffter, Compilers TU Kaiserslautern 13 19

20 Example: Interpretation Introduction Examples.class File.class-Datei iload_1 15 iload_2 16 idiv 17 istore_3... Eingabedaten Input Data iload_1 15 iload_2 16 idiv 17 istore_3... Java Virtual Machine (JVM) Output Ausgabedaten Data Ina Schaefer Compilers 20

21 Introduction Examples Example: Combined Technique Java Implementation with Just-In-time (JIT) Compiler Java-Überset- Source zungseinheit Code Unit (JIT=Just in time) javac Analysator Analyzer Translator Übersetzer.class-Datei file Eingabedaten Input Data Java Byte Code JIT-Übersetzer Translator JVM Maschinencode Machine Code reale Real Machine Maschine/Hardware / Ausgabedaten Output Data Ina Schaefer Compilers 21

22 Language Processing Terminology and Requirements Language Processing: The Translation Task Source Code Translator Error Message or Target Code Translator (in a broader sense): Analysis, Optimization and Translation Souce Code: Input (String) for Translator in Syntax of Source Language (SL) Target Code: Output (String) of Translator in Syntax of Target Language (TL) Ina Schaefer Compilers 22

23 Language Processing Phases of Language Processing Terminology and Requirements Analysis of Input: Program Text Specification Diagrams Dependant on Target of Implementation Transformation (XSLT, Refactoring) Pretty Printing, Formatting Semantic Analysis (Program Comprehension) Optimization (Actual) Translation Ina Schaefer Compilers 23

24 Language Processing Compile Time vs. Run-time Terminology and Requirements Compile Time: during Run-time of Compiler/Translator Static: All Information/Aspects known at Compile Time, e.g. Type Checks Evaluation of Constant Expressions Relative Adresses Run Time: during Run-time of Compiled Program Dynamic: All Information that are not statically known, e.g. Allocation of Dynamic Arrays Bounds Check of Arrays Dynamic Binding of Methods Memory Management of Recursive Procedures For dynamic aspects that cannot be handled at compile time, the compiler generates code that handles these aspects at runtime. Ina Schaefer Compilers 24

25 Language Processing Terminology and Requirements What is a good compiler? Ina Schaefer Compilers 25

26 Language Processing Terminology and Requirements Requirements for Translators Error Handling (Static/Dynamic) Efficient Target Code Choice: Fast Translation with Slow Code vs. Slow Translation with Fast Code Semantically Correct Translation Ina Schaefer Compilers 26

27 Language Processing Terminology and Requirements Semantically Correct Translation Intuitive Definition: Compiled Program behaves according to Language Definition of Source Language. Formal Definition: semsl: SL_Program SL_Data SL_Data semtl: TL_Program TL_Data TL_Data compile: SL_Program TL_Program code: SL_Data TL_Data decode: TL_Data SL_Data Semantic Correctness: semsl(p,d) = decode(semtl(compile(p), code(d))) Ina Schaefer Compilers 27

28 Language Processing Compiler Architecture Compiler Architecture Source Code as String Analysis Token Stream Scanner Parser Decorated Syntax Tree (Close to SL) Translator Intermediate Language Synthesis Attribution & Optimization Attribution & Optimization Syntax Tree Name and Type Analysis Code Generator Target Code as String Peep Hole Optimization Ina Schaefer Compilers 28

29 Language Processing Compiler Architecture Properties of Compiler Architectures Phases represent Concepts. Phases can be interleaved. Concrete Layout of Phases depends on Source Language, Target Language and Concrete Design Decisions. Phase vs. Pass (Phase can comprise more than one pass.) Separate Translation of Program Parts (Interface information must be accessible.) Combination with other Architecture Decisions: Common Intermediate Language Ina Schaefer Compilers 29

30 Language Processing Compiler Architecture Common Intermediate Language Source Language 1 Source Language 2... Source Language n Intermediate Language Target Language 1 Target Language 2... Target Language m Ina Schaefer Compilers 30

31 Language Processing Compiler Architecture Dimensions of Compiler Construction Programming Languages Sequential Procedural, Imperative, OO-Languages Functional, Logical Languages Parallel Languages/Language Constructs Target Languages/Machines Code for Abstract Machines Assembler Machine Languages (CISC, RISC,...) Multi Processor Architectures Memory Hierarchy Translation Tasks: Analysis, Optimization, Synthesis Construction Techniques and Tools: Bootstrapping, Generators Portability, Specification, Correctness Ina Schaefer Compilers 31

32 Compiler Construction Compiler Construction Techniques 1. Stepwise Construction Construction with compiler for different language Construction with compiler for different machine Bootstrapping 2. Compiler - Compiler: Tools for Compiler Generation Scanner Generators (regular expressions) Parser Generators (context-free grammars) Attribute Evaluation Generators (attribute grammar) Code Generator Generators (machine specification) Interpreter Generators (semantics of language) Other Phase-specific Tools 3. Special Programming Techniques General Technique: syntax-driven Special Technique: recursive descend Ina Schaefer Compilers 32

33 Compiler Construction Stepwise Construction ogrammierung basiert üblicherweise auf Exis es Übersetzers für die Implementierungsspr Programming typically depends on existing compiler for implementation language. For compiler construction, this does not hold in general. i Übersetzerkonstruktion kann davon im Allg Source, target and implementation languages of compilers can be sgegangen denoted in T-Diagrams. werden; stehe QS ZS PS T-diagram denotes compiler from QS to ZS written in PS. einen Übersetzer der in Sprache PS geschr und QS in ZS übersetzt. Ina Schaefer Compilers 33

34 Compiler Construction Construction with compiler for different language Given: SL Compiler Construct: Compiler for machine language (MS) in MS Suppose: C Compiler exists on platform with machine language QS SL MS QS SL MS to zu be entwickeln developed C existing existiert C MS MS MS by durch translation Übersetzung Ina Schaefer Compilers 34

35 Compiler Construction Construction with compiler for different machine Construct: C Compiler for M 1 in M 1 Suppose: C Compiler exists for M 2 in M 2 Method: Construct Cross Compiler first First Step C MS 1 C MS 1 C C MS 2 MS 2 MS 2 Cross Compiler Cross Compiler Ina Schaefer Compilers 35

36 Compiler Construction Construction with C compiler C forms MS different 2 2 machine (2) MS 2 Second Step Cross Compiler Cross Compiler C MS 1 C MS 1 C C MS 1 MS 1 MS 2 Ina Schaefer Compilers 36

37 Compiler Construction Bootstrapping Construct: QS Compiler for MS in MS Suppose: yet no compiler exists Method: 1. Construct partial language QS i of QS such that QS 0 QS 1 QS 2... QS 2. Implement QS 0 Compiler for MS in MS 3. Implement QS i+1 Compiler for MS in QS i 4. Create QS i+1 Compiler for MS in MS Ina Schaefer Compilers 37

38 Bootstrapping (2) by extension durch Erweiterung Compiler Construction QS MS QS MS QS 2 MS QS 2 QS 2 MS MS QS 1 MS QS 1 QS 1 MS MS QS 0 QS 0 MS MS MS manually von Hand by translation durch Übersetzung Ina Schaefer Compilers 38

39 Compiler Construction Recommended Reading Wilhelm, Maurer: Chap. 1, Introduction (pp. 1 5) Chap. 6, Structure of Compilers (pp ) Appel Chap. 1, Introduction (pp. 3 14) Ina Schaefer Compilers 39