G52LAC Languages and Computation Lecture 6
|
|
|
- Andrea Johns
- 5 years ago
- Views:
Transcription
1 G52LAC Languages and Computation Lecture 6 Equivalence of Regular Expression and Finite Automata Henrik Nilsson University of Nottingham G52LACLanguages and ComputationLecture 6 p.1/28
2 This Lecture (1) G52LACLanguages and ComputationLecture 6 p.2/28
3 This Lecture (1) We have now seen three ways of formally describing potentially infinite languages: - Deterministic Finite Automata (DFA) - Nondeterministic Finite Automata (NFA) - Regular Expressions (RE) G52LACLanguages and ComputationLecture 6 p.2/28
4 This Lecture (1) We have now seen three ways of formally describing potentially infinite languages: - Deterministic Finite Automata (DFA) - Nondeterministic Finite Automata (NFA) - Regular Expressions (RE) Because - a DFA is a special case of an NFA - any NFA can be converted into an equivalent DFA DFAs and NFAs describe the same class of languages: the Regular languages. G52LACLanguages and ComputationLecture 6 p.2/28
5 This Lecture (2) So, what class of languages do the REs describe? Smaller? Larger? Completely different? G52LACLanguages and ComputationLecture 6 p.3/28
6 This Lecture (2) So, what class of languages do the REs describe? Smaller? Larger? Completely different? In fact: Regular Expressions describe the Regular Languages G52LACLanguages and ComputationLecture 6 p.3/28
7 This Lecture (2) So, what class of languages do the REs describe? Smaller? Larger? Completely different? In fact: Regular Expressions describe the Regular Languages Proof: translation between RE and FA G52LACLanguages and ComputationLecture 6 p.3/28
8 This Lecture (2) So, what class of languages do the REs describe? Smaller? Larger? Completely different? In fact: Regular Expressions describe the Regular Languages Proof: translation between RE and FA This lecture: translation of RE into NFA G52LACLanguages and ComputationLecture 6 p.3/28
9 This Lecture (2) So, what class of languages do the REs describe? Smaller? Larger? Completely different? In fact: Regular Expressions describe the Regular Languages Proof: translation between RE and FA This lecture: translation of RE into NFA Will start by a motivating example. G52LACLanguages and ComputationLecture 6 p.3/28
10 This Lecture (2) So, what class of languages do the REs describe? Smaller? Larger? Completely different? In fact: Regular Expressions describe the Regular Languages Proof: translation between RE and FA This lecture: translation of RE into NFA Will start by a motivating example. Time permitting, brief look at another application: scanners. Study details in your own time if of interest. G52LACLanguages and ComputationLecture 6 p.3/28
11 Applications (1) RE to NFA conversion has important practical applications. The following is a very nice, practically oriented article you should be able to fully appreciate based on what you have learned in G52MAL thus far: Russ Cox. Regular Expression Matching Can Be Simple And Fast (but is slow in Java, Perl, PHP, Python, Ruby,...), January G52LACLanguages and ComputationLecture 6 p.4/28
12 Applications (2) Underlying message: if you re ignorant about CS theory, your code can perform really poorly. Example from the paper: Time to match (a+ǫ) n a n against a n G52LACLanguages and ComputationLecture 6 p.5/28
13 Applications (2) Underlying message: if you re ignorant about CS theory, your code can perform really poorly. Example from the paper: Time to match (a+ǫ) n a n against a n Note difference of time scale: 60 s vs. 60µs! s_construction G52LACLanguages and ComputationLecture 6 p.5/28
14 Applications (3) To quantify: G52LACLanguages and ComputationLecture 6 p.6/28
15 Applications (3) To quantify: Thompson NFA implementation a million times faster than Perl (5.8.7) when running on a 29-character string. G52LACLanguages and ComputationLecture 6 p.6/28
16 Applications (3) To quantify: Thompson NFA implementation a million times faster than Perl (5.8.7) when running on a 29-character string. Thompson NFA handles a 100-character string in under 200 microseconds; Perl would require over years. G52LACLanguages and ComputationLecture 6 p.6/28
17 Applications (3) To quantify: Thompson NFA implementation a million times faster than Perl (5.8.7) when running on a 29-character string. Thompson NFA handles a 100-character string in under 200 microseconds; Perl would require over years. How old is the universe? G52LACLanguages and ComputationLecture 6 p.6/28
18 Applications (3) To quantify: Thompson NFA implementation a million times faster than Perl (5.8.7) when running on a 29-character string. Thompson NFA handles a 100-character string in under 200 microseconds; Perl would require over years. How old is the universe? Current best estimate: 13.8 billion years... G52LACLanguages and ComputationLecture 6 p.6/28
19 Applications (3) To quantify: Thompson NFA implementation a million times faster than Perl (5.8.7) when running on a 29-character string. Thompson NFA handles a 100-character string in under 200 microseconds; Perl would require over years. How old is the universe? Current best estimate: 13.8 billion years... or about years years is a looong time... G52LACLanguages and ComputationLecture 6 p.6/28
20 Recap: Syntax of Regular Expressions 1. is an RE 2. ǫ is an RE 3. For all x Σ, x is an RE (Handwriting convention: x is an RE) 4. If E and F are REs, so is E +F 5. If E and F are REs, so is EF 6. If E is an REs, so is E 7. If E is an REs, so is (E) These are all regular expressions. G52LACLanguages and ComputationLecture 6 p.7/28
21 Recap: Semantics of Regular Expr. 1. L( ) = 2. L(ǫ) = {ǫ} 3. For all x Σ, L(x) = {x} 4. L(E +F) = L(E) L(F) 5. L(EF) = L(E)L(F) 6. L(E ) = L(E) 7. L( (E) ) = L(E) G52LACLanguages and ComputationLecture 6 p.8/28
22 Translating RE to NFA (1) We are going to detail a Graphical Construction for converting an RE to an NFA that is suitable for carrying out by hand. It can be further refined into a fully formal algorithm: see the lecture notes for details. G52LACLanguages and ComputationLecture 6 p.9/28
23 Translating RE to NFA (1) We are going to detail a Graphical Construction for converting an RE to an NFA that is suitable for carrying out by hand. It can be further refined into a fully formal algorithm: see the lecture notes for details. (Our Graphical Construction is a variation of Thompson s Construction. The latter translates into NFA ǫ : a variation of NFA with a special ǫ-move that does not consume any input. We don t cover NFA ǫ in this module.) G52LACLanguages and ComputationLecture 6 p.9/28
24 Translating RE to NFA (2) Specification: Let N(E) denote the NFA that results by applying the graphical construction to an RE E. Then the following equation must hold: L(E) = L(N(E)) (Note that L is overloaded: the language of an RE to the left, the language of an NFA to the right.) We proceed case by case according to the structure of the syntax of REs. G52LACLanguages and ComputationLecture 6 p.10/28
25 RE to NFA, Case Recall: L( ) = N( ): Note: L(N( )) = = L( ); specification satisfied in this case. Note: States are given without names for simplicity. Suffice as construction is graphical; states to be named at the end. G52LACLanguages and ComputationLecture 6 p.11/28
26 RE to NFA, Caseǫ Recall: L(ǫ) = {ǫ} N(ǫ): Note: L(N(ǫ)) = {ǫ} = L(ǫ); specification satisfied in this case. G52LACLanguages and ComputationLecture 6 p.12/28
27 RE to NFA, Casexforx Σ Recall: For each x Σ,L(x) = {x} N(x): x Note: L(N(x)) = {x} = L(x); specification satisfied in this case. G52LACLanguages and ComputationLecture 6 p.13/28
28 RE to NFA, CaseE +F (1) Recall: L(E +F) = L(E) L(F) N(E +F): N ( E ) N ( F ) The NFAs N(E) and N(F) in parallel. The initial states of N(E +F) are the union of the initial states of N(E) and N(F). G52LACLanguages and ComputationLecture 6 p.14/28
29 RE to NFA, CaseE +F (2) Note: Assuming specification holds for E and F, L(N(E +F)) = L(N(E)) L(N(F)) = L(E) L(F) = L(E +F) Thus, specification holds in this case. (This is an inductive case.) G52LACLanguages and ComputationLecture 6 p.15/28
30 RE to NFA, CaseEF (1) Sub-case 1: No initial state of N(E) is accepting; i.e. ǫ / L(N(E)) (Recall: L(EF) = L(E)L(F)) N ( E ) a N ( F ) b c G52LACLanguages and ComputationLecture 6 p.16/28
31 RE to NFA, CaseEF (2) N ( E ) a N ( E F ) a a N ( F ) b b c b c c G52LACLanguages and ComputationLecture 6 p.17/28
32 RE to NFA, CaseEF (3) Sub-case 2: Some initial states of N(E) are accepting; i.e. ǫ L(N(E)) N ( E ) a N ( F ) b c G52LACLanguages and ComputationLecture 6 p.18/28
33 RE to NFA, CaseEF (4) N ( E ) a N ( E F ) a a N ( F ) b b c b c c G52LACLanguages and ComputationLecture 6 p.19/28
34 RE to NFA, CaseEF (5) Note: Assuming specification holds for E and F, L(N(EF)) = L(N(E))L(N(F)) = L(E)L(F) = L(EF) Thus, specification holds in this case. (This is an inductive case.) G52LACLanguages and ComputationLecture 6 p.20/28
35 RE to NFA, CaseE (1) (Recall: L(E ) = L(E) ) N ( E ) a * b c G52LACLanguages and ComputationLecture 6 p.21/28
36 RE to NFA, CaseE (2) N ( E *) a b N ( E ) a b a c b c c Note the additional initial and accepting state that ensures the empty word is accepted. G52LACLanguages and ComputationLecture 6 p.22/28
37 RE to NFA, CaseE (3) Note: Assuming specification holds for E, L(N(E )) = L(N(E)) = L(E) = L(E ) Thus, specification holds in this case. (This is an inductive case.) G52LACLanguages and ComputationLecture 6 p.23/28
38 RE to NFA, Case(E) (Recall: L( (E) ) = L(E)) N( (E) ) = N(E) Note: Assuming specification holds for E, L(N( (E) )) = L(N(E)) = L(E) = L( (E) ) Thus, specification holds in this case. (This is an inductive case.) G52LACLanguages and ComputationLecture 6 p.24/28
39 Example Systematically construct an NFA for the regular expression: (a+b) c ( zero or more as or bs, followed by a single c ) Use the graphical construction. On the white board. G52LACLanguages and ComputationLecture 6 p.25/28
40 Scanning (1) The first stage of many real-world language processing tasks, such as a compiler, is to group individual characters into languagespecific symbols called Lexemes or Tokens: - Keywords (likeif,then,while) - Literals (like42,3.14, A,"abc") - Special symbols and separators (like:=, (,;) -... G52LACLanguages and ComputationLecture 6 p.26/28
41 Scanning (1) The first stage of many real-world language processing tasks, such as a compiler, is to group individual characters into languagespecific symbols called Lexemes or Tokens: - Keywords (likeif,then,while) - Literals (like42,3.14, A,"abc") - Special symbols and separators (like:=, (,;) -... This process is called Lexical Analysis or Scanning, and is performed by a Scanner. G52LACLanguages and ComputationLecture 6 p.26/28
42 Scanning (2) Commonly, white space and comments are understood as token separators. G52LACLanguages and ComputationLecture 6 p.27/28
43 Scanning (2) Commonly, white space and comments are understood as token separators. An additional task of the scanner is often to discard white space and comments as they usually serve no purpose after the scanning. G52LACLanguages and ComputationLecture 6 p.27/28
44 Scanning (2) Commonly, white space and comments are understood as token separators. An additional task of the scanner is often to discard white space and comments as they usually serve no purpose after the scanning. Regular expressions is the most commonly used formalism for describing the Lexical Syntax of a language; i.e. the syntax of the tokes, white space, and comments. G52LACLanguages and ComputationLecture 6 p.27/28
45 Scanning (2) Commonly, white space and comments are understood as token separators. An additional task of the scanner is often to discard white space and comments as they usually serve no purpose after the scanning. Regular expressions is the most commonly used formalism for describing the Lexical Syntax of a language; i.e. the syntax of the tokes, white space, and comments. In essence, a scanner is thus a finite automaton. G52LACLanguages and ComputationLecture 6 p.27/28
46 Scanning (3) There are many famous so called scanner generators; e.g. Lex, Flex: given regular expressions describing the lexical syntax, they produce a scanner for the language. G52LACLanguages and ComputationLecture 6 p.28/28
47 Scanning (3) There are many famous so called scanner generators; e.g. Lex, Flex: given regular expressions describing the lexical syntax, they produce a scanner for the language. Internally, they use Thompson s construction (or similar). G52LACLanguages and ComputationLecture 6 p.28/28
CSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION Spring 2017 http://cseweb.ucsd.edu/classes/sp17/cse105-ab/ Today's learning goals Sipser Ch 1.2, 1.3 Design NFA recognizing a given language Convert an NFA (with or without
CS Lecture 2. The Front End. Lecture 2 Lexical Analysis
CS 1622 Lecture 2 Lexical Analysis CS 1622 Lecture 2 1 Lecture 2 Review of last lecture and finish up overview The first compiler phase: lexical analysis Reading: Chapter 2 in text (by 1/18) CS 1622 Lecture
CS415 Compilers. Lexical Analysis
CS415 Compilers Lexical Analysis These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Lecture 7 1 Announcements First project and second homework
Implementation of Lexical Analysis
Implementation of Lexical Analysis Outline Specifying lexical structure using regular expressions Finite automata Deterministic Finite Automata (DFAs) Non-deterministic Finite Automata (NFAs) Implementation
CSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION Spring 2017 http://cseweb.ucsd.edu/classes/sp17/cse105-ab/ Today's learning goals Sipser Ch 1.2, 1.3 Decide whether or not a string is described by a given regular expression
Lexical Analysis. Lecture 2-4
Lexical Analysis Lecture 2-4 Notes by G. Necula, with additions by P. Hilfinger Prof. Hilfinger CS 164 Lecture 2 1 Administrivia Moving to 60 Evans on Wednesday HW1 available Pyth manual available on line.
Implementation of Lexical Analysis
Implementation of Lexical Analysis Outline Specifying lexical structure using regular expressions Finite automata Deterministic Finite Automata (DFAs) Non-deterministic Finite Automata (NFAs) Implementation
Lexical Analysis. Introduction
Lexical Analysis Introduction Copyright 2015, Pedro C. Diniz, all rights reserved. Students enrolled in the Compilers class at the University of Southern California have explicit permission to make copies
Lexical Analysis. Chapter 2
Lexical Analysis Chapter 2 1 Outline Informal sketch of lexical analysis Identifies tokens in input string Issues in lexical analysis Lookahead Ambiguities Specifying lexers Regular expressions Examples
Implementation of Lexical Analysis
Implementation of Lexical Analysis Lecture 4 (Modified by Professor Vijay Ganesh) Tips on Building Large Systems KISS (Keep It Simple, Stupid!) Don t optimize prematurely Design systems that can be tested
Introduction to Lexical Analysis
Introduction to Lexical Analysis Outline Informal sketch of lexical analysis Identifies tokens in input string Issues in lexical analysis Lookahead Ambiguities Specifying lexical analyzers (lexers) Regular
Lexical Analysis. Lecture 3-4
Lexical Analysis Lecture 3-4 Notes by G. Necula, with additions by P. Hilfinger Prof. Hilfinger CS 164 Lecture 3-4 1 Administrivia I suggest you start looking at Python (see link on class home page). Please
Interpreter. Scanner. Parser. Tree Walker. read. request token. send token. send AST I/O. Console
Scanning 1 read Interpreter Scanner request token Parser send token Console I/O send AST Tree Walker 2 Scanner This process is known as: Scanning, lexing (lexical analysis), and tokenizing This is the
Announcements! P1 part 1 due next Tuesday P1 part 2 due next Friday
Announcements! P1 part 1 due next Tuesday P1 part 2 due next Friday 1 Finite-state machines CS 536 Last time! A compiler is a recognizer of language S (Source) a translator from S to T (Target) a program
Converting a DFA to a Regular Expression JP
Converting a DFA to a Regular Expression JP Prerequisite knowledge: Regular Languages Deterministic Finite Automata Nondeterministic Finite Automata Regular Expressions Conversion of Regular Expression
Computer Science Department Carlos III University of Madrid Leganés (Spain) David Griol Barres
Computer Science Department Carlos III University of Madrid Leganés (Spain) David Griol Barres dgriol@inf.uc3m.es Introduction: Definitions Lexical analysis or scanning: To read from left-to-right a source
UNIT -2 LEXICAL ANALYSIS
OVER VIEW OF LEXICAL ANALYSIS UNIT -2 LEXICAL ANALYSIS o To identify the tokens we need some method of describing the possible tokens that can appear in the input stream. For this purpose we introduce
Lexical Analysis. Lecture 3. January 10, 2018
Lexical Analysis Lecture 3 January 10, 2018 Announcements PA1c due tonight at 11:50pm! Don t forget about PA1, the Cool implementation! Use Monday s lecture, the video guides and Cool examples if you re
Figure 2.1: Role of Lexical Analyzer
Chapter 2 Lexical Analysis Lexical analysis or scanning is the process which reads the stream of characters making up the source program from left-to-right and groups them into tokens. The lexical analyzer
Administrivia. Lexical Analysis. Lecture 2-4. Outline. The Structure of a Compiler. Informal sketch of lexical analysis. Issues in lexical analysis
dministrivia Lexical nalysis Lecture 2-4 Notes by G. Necula, with additions by P. Hilfinger Moving to 6 Evans on Wednesday HW available Pyth manual available on line. Please log into your account and electronically
Compiler course. Chapter 3 Lexical Analysis
Compiler course Chapter 3 Lexical Analysis 1 A. A. Pourhaji Kazem, Spring 2009 Outline Role of lexical analyzer Specification of tokens Recognition of tokens Lexical analyzer generator Finite automata
David Griol Barres Computer Science Department Carlos III University of Madrid Leganés (Spain)
David Griol Barres dgriol@inf.uc3m.es Computer Science Department Carlos III University of Madrid Leganés (Spain) OUTLINE Introduction: Definitions The role of the Lexical Analyzer Scanner Implementation
Compiler phases. Non-tokens
Compiler phases Compiler Construction Scanning Lexical Analysis source code scanner tokens regular expressions lexical analysis Lennart Andersson parser context free grammar Revision 2011 01 21 parse tree
Regular Expressions. Agenda for Today. Grammar for a Tiny Language. Programming Language Specifications
Agenda for Today Regular Expressions CSE 413, Autumn 2005 Programming Languages Basic concepts of formal grammars Regular expressions Lexical specification of programming languages Using finite automata
Chapter 3 Lexical Analysis
Chapter 3 Lexical Analysis Outline Role of lexical analyzer Specification of tokens Recognition of tokens Lexical analyzer generator Finite automata Design of lexical analyzer generator The role of lexical
CMSC 350: COMPILER DESIGN
Lecture 11 CMSC 350: COMPILER DESIGN see HW3 LLVMLITE SPECIFICATION Eisenberg CMSC 350: Compilers 2 Discussion: Defining a Language Premise: programming languages are purely formal objects We (as language
CSE450. Translation of Programming Languages. Lecture 20: Automata and Regular Expressions
CSE45 Translation of Programming Languages Lecture 2: Automata and Regular Expressions Finite Automata Regular Expression = Specification Finite Automata = Implementation A finite automaton consists of:
CS 403 Compiler Construction Lecture 3 Lexical Analysis [Based on Chapter 1, 2, 3 of Aho2]
![CS 403 Compiler Construction Lecture 3 Lexical Analysis [Based on Chapter 1, 2, 3 of Aho2]](/thumbs/78/76928449.jpg "CS 403 Compiler Construction Lecture 3 Lexical Analysis [Based on Chapter 1, 2, 3 of Aho2]")
CS 403 Compiler Construction Lecture 3 Lexical Analysis [Based on Chapter 1, 2, 3 of Aho2] 1 What is Lexical Analysis? First step of a compiler. Reads/scans/identify the characters in the program and groups
Prof. Mohamed Hamada Software Engineering Lab. The University of Aizu Japan
Compilers Prof. Mohamed Hamada Software Engineering Lab. The University of Aizu Japan Lexical Analyzer (Scanner) 1. Uses Regular Expressions to define tokens 2. Uses Finite Automata to recognize tokens
CSE 413 Programming Languages & Implementation. Hal Perkins Winter 2019 Grammars, Scanners & Regular Expressions
CSE 413 Programming Languages & Implementation Hal Perkins Winter 2019 Grammars, Scanners & Regular Expressions 1 Agenda Overview of language recognizers Basic concepts of formal grammars Scanner Theory
Last lecture CMSC330. This lecture. Finite Automata: States. Finite Automata. Implementing Regular Expressions. Languages. Regular expressions
Last lecture CMSC330 Finite Automata Languages Sets of strings Operations on languages Regular expressions Constants Operators Precedence 1 2 Finite automata States Transitions Examples Types This lecture
Formal Languages and Compilers Lecture VI: Lexical Analysis
Formal Languages and Compilers Lecture VI: Lexical Analysis Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/ artale/ Formal
Compiler Construction
Compiler Construction Thomas Noll Software Modeling and Verification Group RWTH Aachen University https://moves.rwth-aachen.de/teaching/ss-16/cc/ Conceptual Structure of a Compiler Source code x1 := y2
Implementation of Lexical Analysis. Lecture 4
Implementation of Lexical Analysis Lecture 4 1 Tips on Building Large Systems KISS (Keep It Simple, Stupid!) Don t optimize prematurely Design systems that can be tested It is easier to modify a working
Finite automata. We have looked at using Lex to build a scanner on the basis of regular expressions.
Finite automata We have looked at using Lex to build a scanner on the basis of regular expressions. Now we begin to consider the results from automata theory that make Lex possible. Recall: An alphabet
CSE302: Compiler Design
CSE302: Compiler Design Instructor: Dr. Liang Cheng Department of Computer Science and Engineering P.C. Rossin College of Engineering & Applied Science Lehigh University February 01, 2007 Outline Recap
CS 432 Fall Mike Lam, Professor. Finite Automata Conversions and Lexing
CS 432 Fall 2017 Mike Lam, Professor Finite Automata Conversions and Lexing Finite Automata Key result: all of the following have the same expressive power (i.e., they all describe regular languages):
Lexical Analysis. Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice.
Lexical Analysis Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice. Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved.
Lexical Analysis. Lexical analysis is the first phase of compilation: The file is converted from ASCII to tokens. It must be fast!
Lexical Analysis Lexical analysis is the first phase of compilation: The file is converted from ASCII to tokens. It must be fast! Compiler Passes Analysis of input program (front-end) character stream
The Front End. The purpose of the front end is to deal with the input language. Perform a membership test: code source language?
The Front End Source code Front End IR Back End Machine code Errors The purpose of the front end is to deal with the input language Perform a membership test: code source language? Is the program well-formed
Finite Automata Theory and Formal Languages TMV027/DIT321 LP4 2016
Finite Automata Theory and Formal Languages TMV027/DIT321 LP4 2016 Lecture 15 Ana Bove May 23rd 2016 More on Turing machines; Summary of the course. Overview of today s lecture: Recap: PDA, TM Push-down
for (i=1; i<=100000; i++) { x = sqrt (y); // square root function cout << x+i << endl; }
Ex: The difference between Compiler and Interpreter The interpreter actually carries out the computations specified in the source program. In other words, the output of a compiler is a program, whereas
Lexical Analysis. Implementation: Finite Automata
Lexical Analysis Implementation: Finite Automata Outline Specifying lexical structure using regular expressions Finite automata Deterministic Finite Automata (DFAs) Non-deterministic Finite Automata (NFAs)
Front End: Lexical Analysis. The Structure of a Compiler
Front End: Lexical Analysis The Structure of a Compiler Constructing a Lexical Analyser By hand: Identify lexemes in input and return tokens Automatically: Lexical-Analyser generator We will learn about
Implementation of Lexical Analysis
Outline Implementation of Lexical nalysis Specifying lexical structure using regular expressions Finite automata Deterministic Finite utomata (DFs) Non-deterministic Finite utomata (NFs) Implementation
COMP-421 Compiler Design. Presented by Dr Ioanna Dionysiou
COMP-421 Compiler Design Presented by Dr Ioanna Dionysiou Administrative! [ALSU03] Chapter 3 - Lexical Analysis Sections 3.1-3.4, 3.6-3.7! Reading for next time [ALSU03] Chapter 3 Copyright (c) 2010 Ioanna
CS308 Compiler Principles Lexical Analyzer Li Jiang
CS308 Lexical Analyzer Li Jiang Department of Computer Science and Engineering Shanghai Jiao Tong University Content: Outline Basic concepts: pattern, lexeme, and token. Operations on languages, and regular
Structure of Programming Languages Lecture 3
Structure of Programming Languages Lecture 3 CSCI 6636 4536 Spring 2017 CSCI 6636 4536 Lecture 3... 1/25 Spring 2017 1 / 25 Outline 1 Finite Languages Deterministic Finite State Machines Lexical Analysis
CSE 413 Programming Languages & Implementation. Hal Perkins Autumn 2012 Grammars, Scanners & Regular Expressions
CSE 413 Programming Languages & Implementation Hal Perkins Autumn 2012 Grammars, Scanners & Regular Expressions 1 Agenda Overview of language recognizers Basic concepts of formal grammars Scanner Theory
Implementation of Lexical Analysis
Written ssignments W assigned today Implementation of Lexical nalysis Lecture 4 Due in one week :59pm Electronic hand-in Prof. iken CS 43 Lecture 4 Prof. iken CS 43 Lecture 4 2 Tips on uilding Large Systems
Decidable Problems. We examine the problems for which there is an algorithm.
Decidable Problems We examine the problems for which there is an algorithm. Decidable Problems A problem asks a yes/no question about some input. The problem is decidable if there is a program that always
Dr. D.M. Akbar Hussain
1 2 Compiler Construction F6S Lecture - 2 1 3 4 Compiler Construction F6S Lecture - 2 2 5 #include.. #include main() { char in; in = getch ( ); if ( isalpha (in) ) in = getch ( ); else error (); while
Lexical Analysis. Dragon Book Chapter 3 Formal Languages Regular Expressions Finite Automata Theory Lexical Analysis using Automata
Lexical Analysis Dragon Book Chapter 3 Formal Languages Regular Expressions Finite Automata Theory Lexical Analysis using Automata Phase Ordering of Front-Ends Lexical analysis (lexer) Break input string
Zhizheng Zhang. Southeast University
Zhizheng Zhang Southeast University 2016/10/5 Lexical Analysis 1 1. The Role of Lexical Analyzer 2016/10/5 Lexical Analysis 2 2016/10/5 Lexical Analysis 3 Example. position = initial + rate * 60 2016/10/5
Wednesday, September 3, 14. Scanners
Scanners Scanners Sometimes called lexers Recall: scanners break input stream up into a set of tokens Identifiers, reserved words, literals, etc. What do we need to know? How do we define tokens? How can
CS 301. Lecture 05 Applications of Regular Languages. Stephen Checkoway. January 31, 2018
CS 301 Lecture 05 Applications of Regular Languages Stephen Checkoway January 31, 2018 1 / 17 Characterizing regular languages The following four statements about the language A are equivalent The language
Lecture 5: Regular Expression and Finite Automata
Lecture 5: Regular Expression and Finite Automata Dr Kieran T. Herley Department of Computer Science University College Cork 2017-2018 KH (28/09/17) Lecture 5: Regular Expression and Finite Automata 2017-2018
Finite Automata and Scanners
Finite Automata and Scanners A finite automaton (FA) can be used to recognize the tokens specified by a regular expression. FAs are simple, idealized computers that recognize strings belonging to regular
Outline. 1 Scanning Tokens. 2 Regular Expresssions. 3 Finite State Automata
Outline 1 2 Regular Expresssions Lexical Analysis 3 Finite State Automata 4 Non-deterministic (NFA) Versus Deterministic Finite State Automata (DFA) 5 Regular Expresssions to NFA 6 NFA to DFA 7 8 JavaCC:
CSEP 501 Compilers. Languages, Automata, Regular Expressions & Scanners Hal Perkins Winter /8/ Hal Perkins & UW CSE B-1
CSEP 501 Compilers Languages, Automata, Regular Expressions & Scanners Hal Perkins Winter 2008 1/8/2008 2002-08 Hal Perkins & UW CSE B-1 Agenda Basic concepts of formal grammars (review) Regular expressions
Lexical Analysis 1 / 52
Lexical Analysis 1 / 52 Outline 1 Scanning Tokens 2 Regular Expresssions 3 Finite State Automata 4 Non-deterministic (NFA) Versus Deterministic Finite State Automata (DFA) 5 Regular Expresssions to NFA
for (i=1; i<=100000; i++) { x = sqrt (y); // square root function cout << x+i << endl; }
Ex: The difference between Compiler and Interpreter The interpreter actually carries out the computations specified in the source program. In other words, the output of a compiler is a program, whereas
COLLEGE OF ENGINEERING, NASHIK. LANGUAGE TRANSLATOR
Pune Vidyarthi Griha s COLLEGE OF ENGINEERING, NASHIK. LANGUAGE TRANSLATOR By Prof. Anand N. Gharu (Assistant Professor) PVGCOE Computer Dept.. 22nd Jan 2018 CONTENTS :- 1. Role of lexical analysis 2.
Compiler Construction
Compiler Construction Lecture 2: Lexical Analysis I (Introduction) Thomas Noll Lehrstuhl für Informatik 2 (Software Modeling and Verification) noll@cs.rwth-aachen.de http://moves.rwth-aachen.de/teaching/ss-14/cc14/
Regular Languages. MACM 300 Formal Languages and Automata. Formal Languages: Recap. Regular Languages
Regular Languages MACM 3 Formal Languages and Automata Anoop Sarkar http://www.cs.sfu.ca/~anoop The set of regular languages: each element is a regular language Each regular language is an example of a
CS321 Languages and Compiler Design I. Winter 2012 Lecture 4
CS321 Languages and Compiler Design I Winter 2012 Lecture 4 1 LEXICAL ANALYSIS Convert source file characters into token stream. Remove content-free characters (comments, whitespace,...) Detect lexical
Concepts Introduced in Chapter 3. Lexical Analysis. Lexical Analysis Terms. Attributes for Tokens
Concepts Introduced in Chapter 3 Lexical Analysis Regular Expressions (REs) Nondeterministic Finite Automata (NFA) Converting an RE to an NFA Deterministic Finite Automatic (DFA) Lexical Analysis Why separate
Lexical Analysis. COMP 524, Spring 2014 Bryan Ward
Lexical Analysis COMP 524, Spring 2014 Bryan Ward Based in part on slides and notes by J. Erickson, S. Krishnan, B. Brandenburg, S. Olivier, A. Block and others The Big Picture Character Stream Scanner
Monday, August 26, 13. Scanners
Scanners Scanners Sometimes called lexers Recall: scanners break input stream up into a set of tokens Identifiers, reserved words, literals, etc. What do we need to know? How do we define tokens? How can
Scanners. Xiaokang Qiu Purdue University. August 24, ECE 468 Adapted from Kulkarni 2012
Scanners Xiaokang Qiu Purdue University ECE 468 Adapted from Kulkarni 2012 August 24, 2016 Scanners Sometimes called lexers Recall: scanners break input stream up into a set of tokens Identifiers, reserved
Implementation of Lexical Analysis
Written ssignments W assigned today Implementation of Lexical nalysis Lecture 4 Due in one week y 5pm Turn in In class In box outside 4 Gates Electronically Prof. iken CS 43 Lecture 4 Prof. iken CS 43
Alternation. Kleene Closure. Definition of Regular Expressions
Alternation Small finite sets are conveniently represented by listing their elements. Parentheses delimit expressions, and, the alternation operator, separates alternatives. For example, D, the set of
Lecture 2 Finite Automata
Lecture 2 Finite Automata August 31, 2007 This lecture is intended as a kind of road map to Chapter 1 of the text just the informal examples that I ll present to motivate the ideas. 1 Expressions without
Decision, Computation and Language
Decision, Computation and Language Regular Expressions Dr. Muhammad S Khan (mskhan@liv.ac.uk) Ashton Building, Room G22 http://www.csc.liv.ac.uk/~khan/comp218 Regular expressions M S Khan (Univ. of Liverpool)
Lecture 3: Lexical Analysis
Lecture 3: Lexical Analysis COMP 524 Programming Language Concepts tephen Olivier January 2, 29 Based on notes by A. Block, N. Fisher, F. Hernandez-Campos, J. Prins and D. totts Goal of Lecture Character
CSE302: Compiler Design
CSE302: Compiler Design Instructor: Dr. Liang Cheng Department of Computer Science and Engineering P.C. Rossin College of Engineering & Applied Science Lehigh University February 13, 2007 Outline Recap
Lexical Analysis - An Introduction. Lecture 4 Spring 2005 Department of Computer Science University of Alabama Joel Jones
Lexical Analysis - An Introduction Lecture 4 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved.
CSE450. Translation of Programming Languages. Automata, Simple Language Design Principles
CSE45 Translation of Programming Languages Automata, Simple Language Design Principles Finite Automata State Graphs A state: The start state: An accepting state: A transition: a A Simple Example A finite
Where We Are. CMSC 330: Organization of Programming Languages. This Lecture. Programming Languages. Motivation for Grammars
CMSC 330: Organization of Programming Languages Context Free Grammars Where We Are Programming languages Ruby OCaml Implementing programming languages Scanner Uses regular expressions Finite automata Parser
Concepts. Lexical scanning Regular expressions DFAs and FSAs Lex. Lexical analysis in perspective
Concepts Lexical scanning Regular expressions DFAs and FSAs Lex CMSC 331, Some material 1998 by Addison Wesley Longman, Inc. 1 CMSC 331, Some material 1998 by Addison Wesley Longman, Inc. 2 Lexical analysis
CS2 Language Processing note 3
CS2 Language Processing note 3 CS2Ah 5..4 CS2 Language Processing note 3 Nondeterministic finite automata In this lecture we look at nondeterministic finite automata and prove the Conversion Theorem, which
Chapter 4. Lexical analysis. Concepts. Lexical scanning Regular expressions DFAs and FSAs Lex. Lexical analysis in perspective
Chapter 4 Lexical analysis Lexical scanning Regular expressions DFAs and FSAs Lex Concepts CMSC 331, Some material 1998 by Addison Wesley Longman, Inc. 1 CMSC 331, Some material 1998 by Addison Wesley
(Refer Slide Time: 0:19)
Theory of Computation. Professor somenath Biswas. Department of Computer Science & Engineering. Indian Institute of Technology, Kanpur. Lecture-15. Decision Problems for Regular Languages. (Refer Slide
Lexical Analysis. Finite Automata. (Part 2 of 2)
# Lexical Analysis Finite Automata (Part 2 of 2) PA0, PA Although we have included the tricky file ends without a newline testcases in previous years, students made good cases against them (e.g., they
CSc 453 Lexical Analysis (Scanning)
CSc 453 Lexical Analysis (Scanning) Saumya Debray The University of Arizona Tucson Overview source program lexical analyzer (scanner) tokens syntax analyzer (parser) symbol table manager Main task: to
1. (10 points) Draw the state diagram of the DFA that recognizes the language over Σ = {0, 1}
CSE 5 Homework 2 Due: Monday October 6, 27 Instructions Upload a single file to Gradescope for each group. should be on each page of the submission. All group members names and PIDs Your assignments in
Definition: A context-free grammar (CFG) is a 4- tuple. variables = nonterminals, terminals, rules = productions,,
CMPSCI 601: Recall From Last Time Lecture 5 Definition: A context-free grammar (CFG) is a 4- tuple, variables = nonterminals, terminals, rules = productions,,, are all finite. 1 ( ) $ Pumping Lemma for
Languages and Compilers
Principles of Software Engineering and Operational Systems Languages and Compilers SDAGE: Level I 2012-13 4. Lexical Analysis (Scanning) Dr Valery Adzhiev vadzhiev@bournemouth.ac.uk Office: TA-121 For
2068 (I) Attempt all questions.
2068 (I) 1. What do you mean by compiler? How source program analyzed? Explain in brief. 2. Discuss the role of symbol table in compiler design. 3. Convert the regular expression 0 + (1 + 0)* 00 first
Formal Languages and Compilers Lecture IV: Regular Languages and Finite. Finite Automata
Formal Languages and Compilers Lecture IV: Regular Languages and Finite Automata Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/
Lexical Analysis. Chapter 1, Section Chapter 3, Section 3.1, 3.3, 3.4, 3.5 JFlex Manual
Lexical Analysis Chapter 1, Section 1.2.1 Chapter 3, Section 3.1, 3.3, 3.4, 3.5 JFlex Manual Inside the Compiler: Front End Lexical analyzer (aka scanner) Converts ASCII or Unicode to a stream of tokens
CS5371 Theory of Computation. Lecture 8: Automata Theory VI (PDA, PDA = CFG)
CS5371 Theory of Computation Lecture 8: Automata Theory VI (PDA, PDA = CFG) Objectives Introduce Pushdown Automaton (PDA) Show that PDA = CFG In terms of descriptive power Pushdown Automaton (PDA) Roughly
CS 536 Introduction to Programming Languages and Compilers Charles N. Fischer Lecture 3
CS 536 Introduction to Programming Languages and Compilers Charles N. Fischer Lecture 3 CS 536 Spring 2015 1 Scanning A scanner transforms a character stream into a token stream. A scanner is sometimes
Nondeterministic Finite Automata (NFA): Nondeterministic Finite Automata (NFA) states of an automaton of this kind may or may not have a transition for each symbol in the alphabet, or can even have multiple
Chapter 2 :: Programming Language Syntax
Chapter 2 :: Programming Language Syntax Michael L. Scott kkman@sangji.ac.kr, 2015 1 Regular Expressions A regular expression is one of the following: A character The empty string, denoted by Two regular
Lexical Analysis - 2
Lexical Analysis - 2 More regular expressions Finite Automata NFAs and DFAs Scanners JLex - a scanner generator 1 Regular Expressions in JLex Symbol - Meaning. Matches a single character (not newline)
Part 5 Program Analysis Principles and Techniques
1 Part 5 Program Analysis Principles and Techniques Front end 2 source code scanner tokens parser il errors Responsibilities: Recognize legal programs Report errors Produce il Preliminary storage map Shape
Finite Automata. Dr. Nadeem Akhtar. Assistant Professor Department of Computer Science & IT The Islamia University of Bahawalpur
Finite Automata Dr. Nadeem Akhtar Assistant Professor Department of Computer Science & IT The Islamia University of Bahawalpur PhD Laboratory IRISA-UBS University of South Brittany European University
Compiler Construction LECTURE # 3
Compiler Construction LECTURE # 3 The Course Course Code: CS-4141 Course Title: Compiler Construction Instructor: JAWAD AHMAD Email Address: jawadahmad@uoslahore.edu.pk Web Address: http://csandituoslahore.weebly.com/cc.html
Formal Languages and Grammars. Chapter 2: Sections 2.1 and 2.2
Formal Languages and Grammars Chapter 2: Sections 2.1 and 2.2 Formal Languages Basis for the design and implementation of programming languages Alphabet: finite set Σ of symbols String: finite sequence