Bayesian Networks and Decision Graphs

Size: px
Start display at page:

Download "Bayesian Networks and Decision Graphs"

Transcription

1 ayesian Networks and ecision raphs hapter 7 hapter 7 p. 1/27

2 Learning the structure of a ayesian network We have: complete database of cases over a set of variables. We want: ayesian network structure representing the independence properties in the database. ases Pr t Ut 1. yes pos pos 2. yes neg pos 3. yes pos neg 4. yes pos neg 5. no neg neg t Pr Ut hapter 7 p. 2/27

3 ayesian networks from databases I xample: Transmission of symbol strings, P (U): irst 2 Last 3 aaa aab aba abb baa bab bba bbb aa ab ba bb onsider the model N as representing the database: T1 T2 T3 T4 T5 Simpler than the database. representation of the database. The chain rule (yields the joint probability which we can compare to the actual database): P N (U) = P(T 1, T 2, T 3, T 4, T 5 ) = P(T 1 )P(T 2 T 1 )P(T 3 T 2 )P(T 4 T 3 )P(T 5 T 4 ). hapter 7 p. 3/27

4 ayesian networks from databases II xample: Transmission of symbol strings, P (U): irst 2 Last 3 aaa aab aba abb baa bab bba bbb aa ab ba bb onsider the model N as representing the database: T1 T2 T3 T4 T5 irst 2 Last 3 aaa aab aba abb baa bab bba bbb aa ab ba bb re P N (U) and P (U) sufficiently identical? hapter 7 p. 4/27

5 (naïve) way to look at it Some agent produces samples of cases from a ayesian network M over the universe U. These cases are handed over to you, and you should now reconstruct M from the cases. ssumptions: The sample is fair (P (U) reflects the distribution determined by M). ll links in M are essential. naïve procedure: or each ayesian network structure N: - alculate the distance between P N (U) and P (U). Return the network N that minimizes the distance, and where all links are essential. ut this is hardly feasible! hapter 7 p. 5/27

6 The space of network structures is huge! The number of structures (as a function of the number of nodes): f(n) = nx ( 1) i+1 n! (n i)!i! 2i(n i) f(n i). i=1 Some example calculations: Nodes Number of s Nodes Number of s hapter 7 p. 6/27

7 Two approaches to structural learning Score based learning: Produces a series of candidate structures. Returns the structure with highest score. onstraint based learning: stablishes a set of conditional independence statements for the data. uilds a structure with d-separation properties corresponding to the independence statements found. hapter 7 p. 7/27

8 onstraint based learning Some notation: To denote that is conditionally independent of given X in the database we shall use I(,, X). Some assumptions: The database is a faithful sample from a ayesian network M: and are d-separated given X in M if and only if I(,, X). We have an oracle that correctly answers questions of the type: Is I(,, X)? The algorithm: Use the oracle s answers to first establish a skeleton of a ayesian network: The skeleton is the undirected graph obtained by removing directions on the arcs. Next, when the skeleton is found we then start looking for the directions on the arcs. hapter 7 p. 8/27

9 inding the skeleton I The idea: if there is a link between and in M then they cannot be d-separated, and as the data is faithful it can be checked by asking questions to the oracle: The link is part of the skeleton if and only if I(,, X), for all X. ssume that the only conditional independence found is I(, ): The skeleton The only possible ssume that the conditional independences found are I(,, ) and I(,, {, }): hapter 7 p. 9/27

10 inding the skeleton II The idea: if there is a link between and in M then they cannot be d-separated, and as the data is faithful it can be checked by asking questions to the oracle: The link is part of the skeleton if and only if I(,, X), for all X. ssume that the only conditional independence found is I(, ): The skeleton The only possible ssume that the conditional independences found are I(,, ) and I(,, {, }): hapter 7 p. 10/27

11 Setting the directions on the links I Rule 1: If you have three nodes,,, such that and, but not, then introduce the v-structure if there exists an X (possibly empty) such that I(,, X) and X. xample: ssume that we get the independences I(, ), I(, ), I(,, ), I(,, ), I(,, ), I(,, {, }), I(,, {, }), I(,, {, }), I(,, {, }), I(,, {,, }), I(,, {,, }), I(,, {,, }). hapter 7 p. 11/27

12 Setting the directions on the links I Rule 1: If you have three nodes,,, such that and, but not, then introduce the v-structure if there exists an X (possibly empty) such that I(,, X) and X. xample: ssume that we get the independences I(, ), I(, ), I(,, ), I(,, ), I(,, ), I(,, {, }), I(,, {, }), I(,, {, }), I(,, {, }), I(,, {,, }), I(,, {,, }), I(,, {,, }). hapter 7 p. 12/27

13 Setting the directions on the links II Rule 2 [void new v-structures]: When Rule 1 has been exhausted, and you have (and no link between and ), then direct. Rule 3 [void cycles]: If introduces a directed cycle in the graph, then do Rule 4 [hoose randomly]: If none of the rules 1-3 can be applied anywhere in the graph, choose an undirected link and give it an arbitrary direction. xample: Skeleton Rule 1 Rule 4 hapter 7 p. 13/27

14 The rules: an overview Rule 1 [ind v-structures]: If you have three nodes,,, such that and, but not, then introduce the v-structure if there exists an X (possibly empty) such that I(,, X) and X. Rule 2 [void new v-structures]: When Rule 1 has been exhausted, and you have (and no link between and ), then direct. Rule 3 [void cycles]: If introduces a directed cycle in the graph, then do Rule 4 [hoose randomly]: If none of the rules 1-3 can be applied anywhere in the graph, choose an undirected link and give it an arbitrary direction. hapter 7 p. 14/27

15 nother example onsider the graph: pply the four rules to learn a ayesian network structure hapter 7 p. 15/27

16 nother example I Step 1: Rule 1 hapter 7 p. 16/27

17 nother example I Step 1: Rule 1 Step 2: Rule 2 hapter 7 p. 16/27

18 nother example I Step 1: Rule 1 Step 2: Rule 2 Step 3: Rule 4 hapter 7 p. 16/27

19 nother example I Step 1: Rule 1 Step 2: Rule 2 Step 3: Rule 4 Step 4: Rule 2 hapter 7 p. 16/27

20 nother example I Step 1: Rule 1 Step 2: Rule 2 Step 3: Rule 4 Step 4: Rule 2 Step 5: Rule 2 hapter 7 p. 16/27

21 nother example I Step 1: Rule 1 Step 2: Rule 2 Step 3: Rule 4 Step 4: Rule 2 Step 5: Rule 2 Step 6: Rule 4 However, we are not guaranteed a unique solution! hapter 7 p. 16/27

22 nother example II Step 1: Rule 1 Step 2: Rule 2 hapter 7 p. 17/27

23 nother example II Step 1: Rule 1 Step 2: Rule 2 Step 3: Rule 4 hapter 7 p. 17/27

24 nother example II Step 1: Rule 1 Step 2: Rule 2 Step 3: Rule 4 Step 4: Rule 4 hapter 7 p. 17/27

25 nother example II Step 1: Rule 1 Step 2: Rule 2 Step 3: Rule 4 Step 4: Rule 4 Step 5: Rule 2 hapter 7 p. 17/27

26 nother example II Step 1: Rule 1 Step 2: Rule 2 Step 3: Rule 4 Step 4: Rule 4 Step 5: Rule 2 Step 6: Rule 2+3 lthough the solution is not necessarily unique, all solutions have the same d-separation properties! hapter 7 p. 17/27

27 rom independence tests to skeleton Until now, we have assumed that all questions of the form Is I(,, X)? can be answered (allowing us to establish the skeleton). However, questions come at a price, and we would there like to ask as few questions as possible. To reduce the number of questions we exploit the following property: Theorem: The nodes and are not linked if and only if I(,, pa()) or I(,, pa()). It is sufficient to ask questions I(,, X), where X is a subset of s or s neighbors. H n active path from to must go through a parent of. hapter 7 p. 18/27

28 The P algorithm The P algorithm: 1. Start with the complete graph; 2. i := 0; 3. while a node has at least i + 1 neighbors - for all nodes with at least i + 1 neighbors - for all neighbors of - for all neighbor sets X such that X = i and X (nb() \ {}) - if I(,, X) then remove the link and store "I(,, X)" - i := i + 1 hapter 7 p. 19/27

29 xample We start with the complete graph and ask the questions I(, )?, I(, )?, I(, )?, I(, )?, I(, )?, I(, )?, I(, )?, I(, )?, I(, )?, I(, )? The original model The complete graph hapter 7 p. 20/27

30 xample We start with the complete graph and ask the questions I(, )?, I(, )?, I(, )?, I(, )?, I(, )?, I(, )?, I(, )?, I(, )?, I(, )?, I(, )?. The original model The complete graph We get a yes for I(, )? and I(, )?: the links and are therefore removed. hapter 7 p. 21/27

31 xample We now condition on one variable and ask the questions I(,, )?, I(,, )?, I(,, )?, I(,, )?, I(,, )?, I(,, )?, I(,, )?, I(,, )?, I(,, )?,...,I(,, )?, I(,, )?. The original model fter one iteration hapter 7 p. 22/27

32 xample We now condition on one variable and ask the questions I(,, )?, I(,, )?, I(,, )?, I(,, )?, I(,, )?, I(,, )?, I(,, )?, I(,, )?, I(,, )?,...,I(,, )?, I(,, )?. The original model fter one iteration The question I(,, )? has the answer "yes": we therefore remove the link. hapter 7 p. 23/27

33 xample We now condition on two variable and ask questions like I(,, {, })?. The original model fter two iterations The questions I(,, {, })? and I(,, {, })? have the answer yes : we therefore remove the links and. hapter 7 p. 24/27

34 xample We now condition on three variables, but since no nodes have four neighbors we are finished. The original model fter three iterations The identified set of independence statements are then I(, ), I(, ), I(,, ), I(,, {, }), and I(,, {, }). They are sufficient for applying rules 1-4. hapter 7 p. 25/27

35 Real world data The oracle is a statistical test, e.g. conditional mutual information: (, X) = X X P(X) X, P(, X)log P(, X) P( X)P( X). I(,, X) (, X) = 0. However, all tests have false positives and false negatives, which may provide false results/causal relations! Similarly, false results may also be caused to hidden variables: hapter 7 p. 26/27

36 Properties If the database is a faithful sample from a ayesian network and the oracle is reliable, then a solution exists. You can only infer causal relations if no hidden variables exists. hapter 7 p. 27/27

Data Compression Fundamentals

Data Compression Fundamentals 1 Data Compression Fundamentals Touradj Ebrahimi Touradj.Ebrahimi@epfl.ch 2 Several classifications of compression methods are possible Based on data type :» Generic data compression» Audio compression»

More information

Automata Theory TEST 1 Answers Max points: 156 Grade basis: 150 Median grade: 81%

Automata Theory TEST 1 Answers Max points: 156 Grade basis: 150 Median grade: 81% Automata Theory TEST 1 Answers Max points: 156 Grade basis: 150 Median grade: 81% 1. (2 pts) See text. You can t be sloppy defining terms like this. You must show a bijection between the natural numbers

More information

ITEC2620 Introduction to Data Structures

ITEC2620 Introduction to Data Structures ITEC2620 Introduction to Data Structures Lecture 9b Grammars I Overview How can a computer do Natural Language Processing? Grammar checking? Artificial Intelligence Represent knowledge so that brute force

More information

Markov Random Fields

Markov Random Fields 3750 Machine earning ecture 4 Markov Random ields Milos auskrecht milos@cs.pitt.edu 5329 ennott quare 3750 dvanced Machine earning Markov random fields Probabilistic models with symmetric dependences.

More information

Compiler Construction

Compiler Construction Compiler Construction Exercises 1 Review of some Topics in Formal Languages 1. (a) Prove that two words x, y commute (i.e., satisfy xy = yx) if and only if there exists a word w such that x = w m, y =

More information

10708 Graphical Models: Homework 2

10708 Graphical Models: Homework 2 10708 Graphical Models: Homework 2 Due October 15th, beginning of class October 1, 2008 Instructions: There are six questions on this assignment. Each question has the name of one of the TAs beside it,

More information

2017 ACM ICPC ASIA, INDIA REGIONAL ONLINE CONTEST

2017 ACM ICPC ASIA, INDIA REGIONAL ONLINE CONTEST Official Problem Set 017 ACM ICPC ASIA, INDIA REGIONAL ONLINE CONTEST 1 Problem code: EQUALMOD Problem name: Modulo Equality You have two arrays A and B, each containing N integers. Elements of array B

More information

Cheng Soon Ong & Christian Walder. Canberra February June 2018

Cheng Soon Ong & Christian Walder. Canberra February June 2018 Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 Outlines Overview Introduction Linear Algebra Probability Linear Regression

More information

Machine Learning Lecture 16

Machine Learning Lecture 16 ourse Outline Machine Learning Lecture 16 undamentals (2 weeks) ayes ecision Theory Probability ensity stimation Undirected raphical Models & Inference 28.06.2016 iscriminative pproaches (5 weeks) Linear

More information

Glynda, the good witch of the North

Glynda, the good witch of the North Strings and Languages It is always best to start at the beginning -- Glynda, the good witch of the North What is a Language? A language is a set of strings made of of symbols from a given alphabet. An

More information

Lec-5-HW-1, TM basics

Lec-5-HW-1, TM basics Lec-5-HW-1, TM basics (Problem 0)-------------------- Design a Turing Machine (TM), T_sub, that does unary decrement by one. Assume a legal, initial tape consists of a contiguous set of cells, each containing

More information

On the Parikh-de-Bruijn grid

On the Parikh-de-Bruijn grid On the Parikh-de-Bruijn grid Péter Burcsi Zsuzsanna Lipták W. F. Smyth ELTE Budapest (Hungary), U of Verona (Italy), McMaster U (Canada) & Murdoch U (Australia) Words & Complexity 2018 Lyon, 19-23 Feb.

More information

Chapter 4: Regular Expressions

Chapter 4: Regular Expressions CSI 3104 /Winter 2011: Introduction to Formal Languages What are the languages with a finite representation? We start with a simple and interesting class of such languages. Dr. Nejib Zaguia CSI3104-W11

More information

Proof Techniques Alphabets, Strings, and Languages. Foundations of Computer Science Theory

Proof Techniques Alphabets, Strings, and Languages. Foundations of Computer Science Theory Proof Techniques Alphabets, Strings, and Languages Foundations of Computer Science Theory Proof By Case Enumeration Sometimes the most straightforward way to prove that a property holds for all elements

More information

CMPSCI 250: Introduction to Computation. Lecture #28: Regular Expressions and Languages David Mix Barrington 2 April 2014

CMPSCI 250: Introduction to Computation. Lecture #28: Regular Expressions and Languages David Mix Barrington 2 April 2014 CMPSCI 250: Introduction to Computation Lecture #28: Regular Expressions and Languages David Mix Barrington 2 April 2014 Regular Expressions and Languages Regular Expressions The Formal Inductive Definition

More information

COMP90051 Statistical Machine Learning

COMP90051 Statistical Machine Learning COMP90051 Statistical Machine Learning Semester 2, 2016 Lecturer: Trevor Cohn 21. Independence in PGMs; Example PGMs Independence PGMs encode assumption of statistical independence between variables. Critical

More information

CS402 Theory of Automata Solved Subjective From Midterm Papers. MIDTERM SPRING 2012 CS402 Theory of Automata

CS402 Theory of Automata Solved Subjective From Midterm Papers. MIDTERM SPRING 2012 CS402 Theory of Automata Solved Subjective From Midterm Papers Dec 07,2012 MC100401285 Moaaz.pk@gmail.com Mc100401285@gmail.com PSMD01 MIDTERM SPRING 2012 Q. Point of Kleen Theory. Answer:- (Page 25) 1. If a language can be accepted

More information

Computer Vision Group Prof. Daniel Cremers. 4. Probabilistic Graphical Models Directed Models

Computer Vision Group Prof. Daniel Cremers. 4. Probabilistic Graphical Models Directed Models Prof. Daniel Cremers 4. Probabilistic Graphical Models Directed Models The Bayes Filter (Rep.) (Bayes) (Markov) (Tot. prob.) (Markov) (Markov) 2 Graphical Representation (Rep.) We can describe the overall

More information

CHAPTER TWO LANGUAGES. Dr Zalmiyah Zakaria

CHAPTER TWO LANGUAGES. Dr Zalmiyah Zakaria CHAPTER TWO LANGUAGES By Dr Zalmiyah Zakaria Languages Contents: 1. Strings and Languages 2. Finite Specification of Languages 3. Regular Sets and Expressions Sept2011 Theory of Computer Science 2 Strings

More information

CSCE 478/878 Lecture 6: Bayesian Learning and Graphical Models. Stephen Scott. Introduction. Outline. Bayes Theorem. Formulas

CSCE 478/878 Lecture 6: Bayesian Learning and Graphical Models. Stephen Scott. Introduction. Outline. Bayes Theorem. Formulas ian ian ian Might have reasons (domain information) to favor some hypotheses/predictions over others a priori ian methods work with probabilities, and have two main roles: Optimal Naïve Nets (Adapted from

More information

D-Separation. b) the arrows meet head-to-head at the node, and neither the node, nor any of its descendants, are in the set C.

D-Separation. b) the arrows meet head-to-head at the node, and neither the node, nor any of its descendants, are in the set C. D-Separation Say: A, B, and C are non-intersecting subsets of nodes in a directed graph. A path from A to B is blocked by C if it contains a node such that either a) the arrows on the path meet either

More information

Notes for Comp 454 Week 2

Notes for Comp 454 Week 2 Notes for Comp 454 Week 2 This week we look at the material in chapters 3 and 4. Homework on Chapters 2, 3 and 4 is assigned (see end of notes). Answers to the homework problems are due by September 10th.

More information

Part II. C. M. Bishop PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS

Part II. C. M. Bishop PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS Part II C. M. Bishop PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS Converting Directed to Undirected Graphs (1) Converting Directed to Undirected Graphs (2) Add extra links between

More information

XVIII Open Cup named after E.V. Pankratiev Stage 10, Grand Prix of Korea, Sunday, February 4, 2018

XVIII Open Cup named after E.V. Pankratiev Stage 10, Grand Prix of Korea, Sunday, February 4, 2018 Problem A. Donut file: XVIII Open Cup named after E.V. Pankratiev 3 seconds 1024 mebibytes In this problem, we will use Chebyshev distance on a Cartesian plane. The Chebyshev distance between two points

More information

Undirected graph is a special case of a directed graph, with symmetric edges

Undirected graph is a special case of a directed graph, with symmetric edges S-6S- ijkstra s lgorithm -: omputing iven a directed weighted graph (all weights non-negative) and two vertices x and y, find the least-cost path from x to y in. Undirected graph is a special case of a

More information

CMPSCI 250: Introduction to Computation. Lecture #36: State Elimination David Mix Barrington 23 April 2014

CMPSCI 250: Introduction to Computation. Lecture #36: State Elimination David Mix Barrington 23 April 2014 CMPSCI 250: Introduction to Computation Lecture #36: State Elimination David Mix Barrington 23 April 2014 State Elimination Kleene s Theorem Overview Another New Model: The r.e.-nfa Overview of the Construction

More information

1.3 Functions and Equivalence Relations 1.4 Languages

1.3 Functions and Equivalence Relations 1.4 Languages CSC4510 AUTOMATA 1.3 Functions and Equivalence Relations 1.4 Languages Functions and Equivalence Relations f : A B means that f is a function from A to B To each element of A, one element of B is assigned

More information

Introduction to Graphical Models

Introduction to Graphical Models Robert Collins CSE586 Introduction to Graphical Models Readings in Prince textbook: Chapters 10 and 11 but mainly only on directed graphs at this time Credits: Several slides are from: Review: Probability

More information

Lecture 5: Exact inference. Queries. Complexity of inference. Queries (continued) Bayesian networks can answer questions about the underlying

Lecture 5: Exact inference. Queries. Complexity of inference. Queries (continued) Bayesian networks can answer questions about the underlying given that Maximum a posteriori (MAP query: given evidence 2 which has the highest probability: instantiation of all other variables in the network,, Most probable evidence (MPE: given evidence, find an

More information

Inference Complexity As Learning Bias. The Goal Outline. Don t use model complexity as your learning bias

Inference Complexity As Learning Bias. The Goal Outline. Don t use model complexity as your learning bias Inference Complexity s Learning Bias Daniel Lowd Dept. of Computer and Information Science University of Oregon Don t use model complexity as your learning bias Use inference complexity. Joint work with

More information

TCOM 501: Networking Theory & Fundamentals. Lecture 11 April 16, 2003 Prof. Yannis A. Korilis

TCOM 501: Networking Theory & Fundamentals. Lecture 11 April 16, 2003 Prof. Yannis A. Korilis TOM 50: Networking Theory & undamentals Lecture pril 6, 2003 Prof. Yannis. Korilis 2 Topics Routing in ata Network Graph Representation of a Network Undirected Graphs Spanning Trees and Minimum Weight

More information

These definitions cover some basic terms and concepts of graph theory and how they have been implemented as C++ classes.

These definitions cover some basic terms and concepts of graph theory and how they have been implemented as C++ classes. hapter 5. raph Theory *raph Theory, *Mathematics raph theory is an area of mathematics which has been incorporated into IS to solve some specific problems in oolean operations and sweeping. It may be also

More information

Graphical Models. David M. Blei Columbia University. September 17, 2014

Graphical Models. David M. Blei Columbia University. September 17, 2014 Graphical Models David M. Blei Columbia University September 17, 2014 These lecture notes follow the ideas in Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. In addition,

More information

Computer Vision Group Prof. Daniel Cremers. 4. Probabilistic Graphical Models Directed Models

Computer Vision Group Prof. Daniel Cremers. 4. Probabilistic Graphical Models Directed Models Prof. Daniel Cremers 4. Probabilistic Graphical Models Directed Models The Bayes Filter (Rep.) (Bayes) (Markov) (Tot. prob.) (Markov) (Markov) 2 Graphical Representation (Rep.) We can describe the overall

More information

Counting Faces of Polytopes

Counting Faces of Polytopes Counting Faces of Polytopes Carl Lee University of Kentucky James Madison University March 2015 Carl Lee (UK) Counting Faces of Polytopes James Madison University 1 / 36 Convex Polytopes A convex polytope

More information

Computer vision: models, learning and inference. Chapter 10 Graphical Models

Computer vision: models, learning and inference. Chapter 10 Graphical Models Computer vision: models, learning and inference Chapter 10 Graphical Models Independence Two variables x 1 and x 2 are independent if their joint probability distribution factorizes as Pr(x 1, x 2 )=Pr(x

More information

Lecture 3: Conditional Independence - Undirected

Lecture 3: Conditional Independence - Undirected CS598: Graphical Models, Fall 2016 Lecture 3: Conditional Independence - Undirected Lecturer: Sanmi Koyejo Scribe: Nate Bowman and Erin Carrier, Aug. 30, 2016 1 Review for the Bayes-Ball Algorithm Recall

More information

Section 1.7 Sequences, Summations Cardinality of Infinite Sets

Section 1.7 Sequences, Summations Cardinality of Infinite Sets Section 1.7 Sequences, Summations Cardinality of Infinite Sets Definition: A sequence is a function from a subset of the natural numbers (usually of the form {0, 1, 2,... } to a set S. Note: the sets and

More information

Regular Expressions. Lecture 10 Sections Robb T. Koether. Hampden-Sydney College. Wed, Sep 14, 2016

Regular Expressions. Lecture 10 Sections Robb T. Koether. Hampden-Sydney College. Wed, Sep 14, 2016 Regular Expressions Lecture 10 Sections 3.1-3.2 Robb T. Koether Hampden-Sydney College Wed, Sep 14, 2016 Robb T. Koether (Hampden-Sydney College) Regular Expressions Wed, Sep 14, 2016 1 / 23 Outline 1

More information

Models and Algorithms for Complex Networks

Models and Algorithms for Complex Networks Models and Algorithms for Complex Networks with network with parametrization, elements maintaining typically characteristic local profiles with categorical attributes [C. Faloutsos MMDS8] Milena Mihail

More information

Assignment No.4 solution. Pumping Lemma Version I and II. Where m = n! (n-factorial) and n = 1, 2, 3

Assignment No.4 solution. Pumping Lemma Version I and II. Where m = n! (n-factorial) and n = 1, 2, 3 Assignment No.4 solution Question No.1 a. Suppose we have a language defined below, Pumping Lemma Version I and II a n b m Where m = n! (n-factorial) and n = 1, 2, 3 Some strings belonging to this language

More information

1. Which of the following regular expressions over {0, 1} denotes the set of all strings not containing 100 as a sub-string?

1. Which of the following regular expressions over {0, 1} denotes the set of all strings not containing 100 as a sub-string? Multiple choice type questions. Which of the following regular expressions over {, } denotes the set of all strings not containing as a sub-string? 2. DFA has a) *(*)* b) ** c) ** d) *(+)* a) single final

More information

Section 2.4 Sequences and Summations

Section 2.4 Sequences and Summations Section 2.4 Sequences and Summations Definition: A sequence is a function from a subset of the natural numbers (usually of the form {0, 1, 2,... } to a set S. Note: the sets and {0, 1, 2, 3,..., k} {1,

More information

The Basics of Graphical Models

The Basics of Graphical Models The Basics of Graphical Models David M. Blei Columbia University September 30, 2016 1 Introduction (These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan.

More information

Accessing Variables. How can we generate code for x?

Accessing Variables. How can we generate code for x? S-322 Register llocation ccessing Variables How can we generate code for x? a := x + y The variable may be in a register:,r x, The variable may be in a static memory location: ST x,r w work register L

More information

Chapter 8 of Bishop's Book: Graphical Models

Chapter 8 of Bishop's Book: Graphical Models Chapter 8 of Bishop's Book: Graphical Models Review of Probability Probability density over possible values of x Used to find probability of x falling in some range For continuous variables, the probability

More information

CS 403 Compiler Construction Lecture 3 Lexical Analysis [Based on Chapter 1, 2, 3 of Aho2]

CS 403 Compiler Construction Lecture 3 Lexical Analysis [Based on Chapter 1, 2, 3 of Aho2] CS 403 Compiler Construction Lecture 3 Lexical Analysis [Based on Chapter 1, 2, 3 of Aho2] 1 What is Lexical Analysis? First step of a compiler. Reads/scans/identify the characters in the program and groups

More information

Machine Learning

Machine Learning Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University October 2, 2012 Today: Graphical models Bayes Nets: Representing distributions Conditional independencies

More information

Lecture 5: Exact inference

Lecture 5: Exact inference Lecture 5: Exact inference Queries Inference in chains Variable elimination Without evidence With evidence Complexity of variable elimination which has the highest probability: instantiation of all other

More information

Machine Learning

Machine Learning Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University February 18, 2015 Today: Graphical models Bayes Nets: Representing distributions Conditional independencies

More information

The angle measure at for example the vertex A is denoted by m A, or m BAC.

The angle measure at for example the vertex A is denoted by m A, or m BAC. MT 200 ourse notes on Geometry 5 2. Triangles and congruence of triangles 2.1. asic measurements. Three distinct lines, a, b and c, no two of which are parallel, form a triangle. That is, they divide the

More information

Power Set of a set and Relations

Power Set of a set and Relations Power Set of a set and Relations 1 Power Set (1) Definition: The power set of a set S, denoted P(S), is the set of all subsets of S. Examples Let A={a,b,c}, P(A)={,{a},{b},{c},{a,b},{b,c},{a,c},{a,b,c}}

More information

CS 343: Artificial Intelligence

CS 343: Artificial Intelligence CS 343: Artificial Intelligence Bayes Nets: Independence Prof. Scott Niekum The University of Texas at Austin [These slides based on those of Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley.

More information

CAD Algorithms. Shortest Path

CAD Algorithms. Shortest Path lgorithms Shortest Path lgorithms Mohammad Tehranipoor epartment September 00 Shortest Path Problem: ind the best way of getting from s to t where s and t are vertices in a graph. est: Min (sum of the

More information

CS201 Discussion 7 MARKOV AND RECURSION

CS201 Discussion 7 MARKOV AND RECURSION CS201 Discussion 7 MARKOV AND RECURSION Before we begin Any questions about the midterm solutions? Making a Markov Map Recall that in Markov, we re trying to make a map of all k-grams to all k-grams that

More information

Reading. Graph Terminology. Graphs. What are graphs? Varieties. Motivation for Graphs. Reading. CSE 373 Data Structures. Graphs are composed of.

Reading. Graph Terminology. Graphs. What are graphs? Varieties. Motivation for Graphs. Reading. CSE 373 Data Structures. Graphs are composed of. Reading Graph Reading Goodrich and Tamassia, hapter 1 S 7 ata Structures What are graphs? Graphs Yes, this is a graph. Graphs are composed of Nodes (vertices) dges (arcs) node ut we are interested in a

More information

Problem Set 3 Due: 11:59 pm on Thursday, September 25

Problem Set 3 Due: 11:59 pm on Thursday, September 25 CS235 Languages and Automata Handout # 6 Prof. Lyn Turbak September 18, 2008 Wellesley College Problem Set 3 Due: 11:59 pm on Thursday, September 25 Reading: Handout #4: Standard ML of New Jersey (SML/NJ)

More information

FMA901F: Machine Learning Lecture 6: Graphical Models. Cristian Sminchisescu

FMA901F: Machine Learning Lecture 6: Graphical Models. Cristian Sminchisescu FMA901F: Machine Learning Lecture 6: Graphical Models Cristian Sminchisescu Graphical Models Provide a simple way to visualize the structure of a probabilistic model and can be used to design and motivate

More information

Machine Learning. Sourangshu Bhattacharya

Machine Learning. Sourangshu Bhattacharya Machine Learning Sourangshu Bhattacharya Bayesian Networks Directed Acyclic Graph (DAG) Bayesian Networks General Factorization Curve Fitting Re-visited Maximum Likelihood Determine by minimizing sum-of-squares

More information

Bayesian Classification Using Probabilistic Graphical Models

Bayesian Classification Using Probabilistic Graphical Models San Jose State University SJSU ScholarWorks Master's Projects Master's Theses and Graduate Research Spring 2014 Bayesian Classification Using Probabilistic Graphical Models Mehal Patel San Jose State University

More information

Building Classifiers using Bayesian Networks

Building Classifiers using Bayesian Networks Building Classifiers using Bayesian Networks Nir Friedman and Moises Goldszmidt 1997 Presented by Brian Collins and Lukas Seitlinger Paper Summary The Naive Bayes classifier has reasonable performance

More information

QUESTION BANK. Unit 1. Introduction to Finite Automata

QUESTION BANK. Unit 1. Introduction to Finite Automata QUESTION BANK Unit 1 Introduction to Finite Automata 1. Obtain DFAs to accept strings of a s and b s having exactly one a.(5m )(Jun-Jul 10) 2. Obtain a DFA to accept strings of a s and b s having even

More information

ASSIGNMENT 3. COMP-202, Fall 2014, All Sections. Due: October 24 th, 2014 (23:59)

ASSIGNMENT 3. COMP-202, Fall 2014, All Sections. Due: October 24 th, 2014 (23:59) ASSIGNMENT 3 COMP-202, Fall 2014, All Sections Due: October 24 th, 2014 (23:59) Please read the entire pdf before starting. You must do this assignment individually and, unless otherwise specified, you

More information

A Brief Introduction to Bayesian Networks AIMA CIS 391 Intro to Artificial Intelligence

A Brief Introduction to Bayesian Networks AIMA CIS 391 Intro to Artificial Intelligence A Brief Introduction to Bayesian Networks AIMA 14.1-14.3 CIS 391 Intro to Artificial Intelligence (LDA slides from Lyle Ungar from slides by Jonathan Huang (jch1@cs.cmu.edu)) Bayesian networks A simple,

More information

Twin Binary Sequences: A Non-redundant Representation for General Non-slicing Floorplan

Twin Binary Sequences: A Non-redundant Representation for General Non-slicing Floorplan Twin inary Sequences: Non-redundant Representation for General Non-slicing loorplan vangeline.y. Young, hris.n. hu and Zion ien Shen bstract The efficiency and effectiveness of many floorplanning methods

More information

Warm-up as you walk in

Warm-up as you walk in arm-up as you walk in Given these N=10 observations of the world: hat is the approximate value for P c a, +b? A. 1/10 B. 5/10. 1/4 D. 1/5 E. I m not sure a, b, +c +a, b, +c a, b, +c a, +b, +c +a, b, +c

More information

Chapter 18: Decidability

Chapter 18: Decidability Chapter 18: Decidability Peter Cappello Department of Computer Science University of California, Santa Barbara Santa Barbara, CA 93106 cappello@cs.ucsb.edu Please read the corresponding chapter before

More information

Multiple Choice Questions

Multiple Choice Questions Techno India Batanagar Computer Science and Engineering Model Questions Subject Name: Formal Language and Automata Theory Subject Code: CS 402 Multiple Choice Questions 1. The basic limitation of an FSM

More information

Differential Cryptanalysis

Differential Cryptanalysis Differential Cryptanalysis See: Biham and Shamir, Differential Cryptanalysis of the Data Encryption Standard, Springer Verlag, 1993. c Eli Biham - March, 28 th, 2012 1 Differential Cryptanalysis The Data

More information

Introduction to Syntax Analysis. The Second Phase of Front-End

Introduction to Syntax Analysis. The Second Phase of Front-End Compiler Design IIIT Kalyani, WB 1 Introduction to Syntax Analysis The Second Phase of Front-End Compiler Design IIIT Kalyani, WB 2 Syntax Analysis The syntactic or the structural correctness of a program

More information

Learning DAGs from observational data

Learning DAGs from observational data Learning DAGs from observational data General overview Introduction DAGs and conditional independence DAGs and causal effects Learning DAGs from observational data IDA algorithm Further problems 2 What

More information

CS 3100 Models of Computation Fall 2011 This assignment is worth 8% of the total points for assignments 100 points total.

CS 3100 Models of Computation Fall 2011 This assignment is worth 8% of the total points for assignments 100 points total. CS 3100 Models of Computation Fall 2011 This assignment is worth 8% of the total points for assignments 100 points total September 7, 2011 Assignment 3, Posted on: 9/6 Due: 9/15 Thursday 11:59pm 1. (20

More information

Summary. Agenda. Search. Search is a common technique in problem solving. Search Formulation Tree Search Uniform cost search A* Search.

Summary. Agenda. Search. Search is a common technique in problem solving. Search Formulation Tree Search Uniform cost search A* Search. ummary rtificial Intelligence and its applications ecture earch earch onstraint atisfaction Problems rom start state to goal state onsider constraints Professor aniel Yeung danyeung@ieee.org r. Patrick

More information

What are graphs? (Take 1)

What are graphs? (Take 1) Lecture 22: Let s Get Graphic Graph lgorithms Today s genda: What is a graph? Some graphs that you already know efinitions and Properties Implementing Graphs Topological Sort overed in hapter 9 of the

More information

Bayesian Networks. A Bayesian network is a directed acyclic graph that represents causal relationships between random variables. Earthquake.

Bayesian Networks. A Bayesian network is a directed acyclic graph that represents causal relationships between random variables. Earthquake. Bayes Nets Independence With joint probability distributions we can compute many useful things, but working with joint PD's is often intractable. The naïve Bayes' approach represents one (boneheaded?)

More information

Section 6.3: Further Rules for Counting Sets

Section 6.3: Further Rules for Counting Sets Section 6.3: Further Rules for Counting Sets Often when we are considering the probability of an event, that event is itself a union of other events. For example, suppose there is a horse race with three

More information

UNIT I PART A PART B

UNIT I PART A PART B OXFORD ENGINEERING COLLEGE (NAAC ACCREDITED WITH B GRADE) DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING LIST OF QUESTIONS YEAR/SEM: III/V STAFF NAME: Dr. Sangeetha Senthilkumar SUB.CODE: CS6503 SUB.NAME:

More information

Minimum Spanning Trees and Shortest Paths

Minimum Spanning Trees and Shortest Paths Minimum Spanning Trees and Shortest Paths Prim's algorithm ijkstra's algorithm November, 017 inda eeren / eoffrey Tien 1 Recall: S spanning tree Starting from vertex 16 9 1 6 10 13 4 3 17 5 11 7 16 13

More information

CS2210 Data Structures and Algorithms

CS2210 Data Structures and Algorithms S1 ata Structures and Algorithms Lecture 1 : Shortest Paths A 4 1 5 5 3 4 Goodrich, Tamassia Outline Weighted Graphs Shortest Paths Algorithm (ijkstra s) Weighted Graphs ach edge has an associated numerical

More information

Introduction to Syntax Analysis

Introduction to Syntax Analysis Compiler Design 1 Introduction to Syntax Analysis Compiler Design 2 Syntax Analysis The syntactic or the structural correctness of a program is checked during the syntax analysis phase of compilation.

More information

Undirected graph is a special case of a directed graph, with symmetric edges

Undirected graph is a special case of a directed graph, with symmetric edges ijkstra s lgorithm 7-: omputing iven a directed weighted graph(all weights non-negative) and two verticesxand y, find the least-cost path fromxtoy in. Undirected graph is a special case of a directed graph,

More information

UNION-FREE DECOMPOSITION OF REGULAR LANGUAGES

UNION-FREE DECOMPOSITION OF REGULAR LANGUAGES UNION-FREE DECOMPOSITION OF REGULAR LANGUAGES Thesis submitted in partial fulfillment of the requirements for the award of degree of Master of Engineering in Computer Science and Engineering Submitted

More information

Outline. Solving Problems by Searching. Introduction. Problem-solving agents

Outline. Solving Problems by Searching. Introduction. Problem-solving agents Outline Solving Problems by Searching S/ University of Waterloo Sept 7, 009 Problem solving agents and search Examples Properties of search algorithms Uninformed search readth first Depth first Iterative

More information

Joint Entity Resolution

Joint Entity Resolution Joint Entity Resolution Steven Euijong Whang, Hector Garcia-Molina Computer Science Department, Stanford University 353 Serra Mall, Stanford, CA 94305, USA {swhang, hector}@cs.stanford.edu No Institute

More information

CS412/413. Introduction to Compilers Tim Teitelbaum. Lecture 2: Lexical Analysis 23 Jan 08

CS412/413. Introduction to Compilers Tim Teitelbaum. Lecture 2: Lexical Analysis 23 Jan 08 CS412/413 Introduction to Compilers Tim Teitelbaum Lecture 2: Lexical Analysis 23 Jan 08 Outline Review compiler structure What is lexical analysis? Writing a lexer Specifying tokens: regular expressions

More information

1/19/2010. Usually for static, deterministic environment

1/19/2010. Usually for static, deterministic environment To download the discussion slides, go to http://april.eecs.umich.edu/courses/eecs492_w10/wiki/index.php/iscussion_lides eterministic, tochastic and trategic nvironment eterministic ompletely predictable

More information

Machine Learning

Machine Learning Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University April 1, 2019 Today: Inference in graphical models Learning graphical models Readings: Bishop chapter 8 Bayesian

More information

Introduction to Computer Architecture

Introduction to Computer Architecture Boolean Operators The Boolean operators AND and OR are binary infix operators (that is, they take two arguments, and the operator appears between them.) A AND B D OR E We will form Boolean Functions of

More information

1 : Introduction to GM and Directed GMs: Bayesian Networks. 3 Multivariate Distributions and Graphical Models

1 : Introduction to GM and Directed GMs: Bayesian Networks. 3 Multivariate Distributions and Graphical Models 10-708: Probabilistic Graphical Models, Spring 2015 1 : Introduction to GM and Directed GMs: Bayesian Networks Lecturer: Eric P. Xing Scribes: Wenbo Liu, Venkata Krishna Pillutla 1 Overview This lecture

More information

ALGORITHMS FOR DECISION SUPPORT

ALGORITHMS FOR DECISION SUPPORT GORIHM FOR IION UOR ayesian networks Guest ecturer: ilja Renooij hanks to hor Whalen or kindly contributing to these slides robabilistic Independence onditional Independence onditional Independence hain

More information

Clustering Using Graph Connectivity

Clustering Using Graph Connectivity Clustering Using Graph Connectivity Patrick Williams June 3, 010 1 Introduction It is often desirable to group elements of a set into disjoint subsets, based on the similarity between the elements in the

More information

DVA337 HT17 - LECTURE 4. Languages and regular expressions

DVA337 HT17 - LECTURE 4. Languages and regular expressions DVA337 HT17 - LECTURE 4 Languages and regular expressions 1 SO FAR 2 TODAY Formal definition of languages in terms of strings Operations on strings and languages Definition of regular expressions Meaning

More information

NP-Hardness. We start by defining types of problem, and then move on to defining the polynomial-time reductions.

NP-Hardness. We start by defining types of problem, and then move on to defining the polynomial-time reductions. CS 787: Advanced Algorithms NP-Hardness Instructor: Dieter van Melkebeek We review the concept of polynomial-time reductions, define various classes of problems including NP-complete, and show that 3-SAT

More information

Lecture 4: Undirected Graphical Models

Lecture 4: Undirected Graphical Models Lecture 4: Undirected Graphical Models Department of Biostatistics University of Michigan zhenkewu@umich.edu http://zhenkewu.com/teaching/graphical_model 15 September, 2016 Zhenke Wu BIOSTAT830 Graphical

More information

CS-171, Intro to A.I. Mid-term Exam Summer Quarter, 2016

CS-171, Intro to A.I. Mid-term Exam Summer Quarter, 2016 CS-171, Intro to A.I. Mid-term Exam Summer Quarter, 2016 YOUR NAME: YOUR ID: ID TO RIGHT: ROW: SEAT: The exam will begin on the next page. Please, do not turn the page until told. When you are told to

More information

Technische Universität München Zentrum Mathematik

Technische Universität München Zentrum Mathematik Technische Universität München Zentrum Mathematik Prof. r. r. Jürgen Richter-ebert, ernhard Werner Projective eometry SS 8 https://www-m.ma.tum.de/bin/view/lehre/ss8/pss8/webome Solutions for Worksheet

More information

Graphical Models & HMMs

Graphical Models & HMMs Graphical Models & HMMs Henrik I. Christensen Robotics & Intelligent Machines @ GT Georgia Institute of Technology, Atlanta, GA 30332-0280 hic@cc.gatech.edu Henrik I. Christensen (RIM@GT) Graphical Models

More information

A Randomized Algorithm for Minimum Cuts

A Randomized Algorithm for Minimum Cuts Randomized lgorithm for Minimum uts ndreas Klappenecker randomized algorithm is an algorithm that receives, in addition to its input, a stream of random bits which is used to make random choices. The random

More information

CS242: Probabilistic Graphical Models Lecture 3: Factor Graphs & Variable Elimination

CS242: Probabilistic Graphical Models Lecture 3: Factor Graphs & Variable Elimination CS242: Probabilistic Graphical Models Lecture 3: Factor Graphs & Variable Elimination Instructor: Erik Sudderth Brown University Computer Science September 11, 2014 Some figures and materials courtesy

More information

CS 161 Lecture 11 BFS, Dijkstra s algorithm Jessica Su (some parts copied from CLRS) 1 Review

CS 161 Lecture 11 BFS, Dijkstra s algorithm Jessica Su (some parts copied from CLRS) 1 Review 1 Review 1 Something I did not emphasize enough last time is that during the execution of depth-firstsearch, we construct depth-first-search trees. One graph may have multiple depth-firstsearch trees,

More information