Learning Bayesian Networks via Edge Walks on DAG Associahedra
|
|
- Marian Morton
- 5 years ago
- Views:
Transcription
1 Learning Bayesian Networks via Edge Walks on DAG Associahedra Liam Solus Based on work with Lenka Matejovicova, Adityanarayanan Radhakrishnan, Caroline Uhler, and Yuhao Wang KTH Royal Institute of Technology 8 January 207 Workshop on Convex Polytopes, Osaka University Liam Solus (KTH) Learning Bayesian Networks 8 January 207 / 34
2 Bayesian Basics Definitions Directed Acyclic Graph (DAG) Models G = ([n], A) a directed acyclic graph (DAG) 2 3 The node i associates to a random variable X i 4 Markov Assumption (MA) The nonedges of G encode conditional independence (CI) relations capturing cause-effect relationships: X i X nondes(i)\ pa(i) X pa(i) pa(i) := the collection of parents of i. nondes(i) := the nondescendents of i = [n]\ des(i) {i}. The CI relations implied by the MA are captured by d-separation in G. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
3 Bayesian Basics Definitions Directed Acyclic Graph (DAG) Models Let A, B, C [n] disjoint with A, B. We say that C d-connects A and B in G if there is an undirected path U from A to B such that every collider on U has a descendant in C, and 2 no other node is in C. i i i In there is no such path U, we say A and B are d-separated by C. The Global Markov Property A probability distribution P obeys MA for a DAG G if and only if X A X B X C for all A, B, C for which C d-separates A and B in G. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
4 Bayesian Basics Definitions Directed Acyclic Graph (DAG) Models Bio-Mom Bio-Dad Bio-Mom Bio-Grandchild Bio-Child Bio-Child Bio-Mom Bio-Dad Bio-Child Bio-Grandchild General Goal Suppose we obtain data from an unknown DAG G, from which we infer a collection of CI relations C. Can we learn the DAG G from C? Algorithms? Consistency gaurantees? Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
5 Bayesian Basics Algorithms The PC and SP Algorithms The PC-Algorithm: (Spirtes, Glymour, and Scheines, 200) Identify undirected graph. 2 Then orient the edges. The SP-Algorithm: (Uhler and Raskutti, 204) To each permutation π = π π 2... π n construct a permutation DAG G π with arrows (π i, π j ) E(G π ) if and only if i < j and π i π j {π,..., π j }\{π i }. 2 Choose a sparsest permutation π ; i.e. a permutation for which G π has the fewest edges. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
6 Bayesian Basics Algorithms The SP-Algorithm n = 3 and C = { 3} : i < j and π i π j {π,..., π j }\{π i }. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
7 Bayesian Basics Algorithms Consistency Guarantees Faithfulness Assumption A probability distribution P is faithful to a DAG G if the only CI relations satisfied by P are those entailed by the MA. Restricted Faithfulness Assumption P satisfies the restricted faithfulness assumption with respect to a DAG G = ([n], A) if it satisfies the following two conditions: Adjacency Faithfulness Faithfulness for all arrows i j A. 2 Orientation Faithfulness Faithfulness for all triples (i, j, k). i j i j k k Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
8 Bayesian Basics Algorithms Consistency Guarantees SMR Assumption P satisfies the SMR assumption with respect to a DAG G if it satisfies the MA with respect to G and G < H for every DAG H such that P satisfies the MA for H and H is not Markov equivalent to G. Theorem (Uhler and Raskutti, 204) The SP-algorithm is consistent under the SMR assumption which is strictly weaker than restricted faithfulness. Downside: SP-algorithm search space is factorial in size! Can we more efficiently search through the permutations S n? Can we shrink the search space of the SP-algorithm while maintaining consistency gaurantees? Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
9 Some Geometry Permutohedra Edge Walks on a Permutohedron P n ron Each vertex corresponds to a permutation DAG G π. Edges correspond to flipping adjacent transpositions: Can we take a greedy walk along the edges of P n? I.e. walk from G π to G τ whenever E(G π ) > E(G τ )? (One) Problem: Two permutations can have the same permutation DAGs. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
10 Some Geometry DAG Associahedra DAG Associahedra (Mohammadi, Uhler, Wang, Yu, 206) DAG associahedron CI relations:?? 2,?? 4 3,?? 4 {2, 3} 2?? 4 3, 2?? 4 {, 3} Associate CI relations to edges of Pn with respect to the dependence relations i < j and πi 6 πj {π,..., πj }\{πi } : 2 no elements in conditioning set: edges with nothing before {, 2}: and {3} conditioning set is {3}: edges with 3 before 2 and 4: Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
11 Some Geometry DAG Associahedra DAG Associahedra DAG associahedron(mohammadi, Uhler, Wang, Yu, 206) Caroline (MIT) Liam Solus Uhler (KTH) CI relations:?? 2,?? 4 3,?? 4 {2, 3} 2?? 4 3, 2?? 4 {, 3} Sparsest permutations Learning Bayesian Networks Munich, Oct January /4/ 34
12 Some Geometry DAG Associahedra DAG Associahedra (Mohammadi, Uhler, Wang, Yu, 206) DAG associahedron Theorem (Mohammadi, Uhler, C. Wang, Yu, 206) CI relations:??are 2, labeled?? 4 by 3, the??different 4 {2, 3} Pn (C) is a convex polytope whose vertices 4 3 2?? 4 3, 2?? 4 {, 3} possible permutation DAGs for C. 2 Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
13 Greedy SP Algorithms Greedy SP Algorithms The vertices of the DAG Associahedron, P n (C), serve as a reduced search space for the SP-algorithm. Edge SP Algorithm Input: C and a permutation π S n. Take a nonincreasing edge walk along the edges of the DAG associahedron P n (C). When is this algorithm consistent? Under the faithfulness assumption? Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
14 Greedy SP Algorithms Geometric Aspects of Faithfulness Geometric Aspects of Faithfulness A covered edge in a DAG G is any edge i j such that pa(j) = pa(i) {i}. Revisiting Our SP-algorithm Example n = 3 and C = { 3} : π = 23 2 τ = 32 i < j and π i π j {π,..., π j }\{π i }. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
15 Greedy SP Algorithms Geometric Aspects of Faithfulness Geometric Aspects of Faithfulness Theorem (Matejovicova, LS, Uhler, Y. Wang, 207) Under the faithfulness assumption, each edge e corresponds to flipping a covered edge in one of the DAGs associated to the endpoints of e. Triangle SP Algorithm Input: C and a permutation π S n. Take a nonincreasing edge walk along the edges of the DAG associahedron P n (C) that correspond to covered edge reversals. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
16 Greedy SP Algorithms Geometric Aspects of Faithfulness Edge Walks and Independence Maps A DAG H is an independence map of another DAG G, written G H, if any CI relation entailed by H is also entailed by G. Theorem (Matejovicova, LS, Uhler, Y. Wang, 207) A positive probability distribution P is faithful to a sparsest DAG G π and only if G π G π for all permutations π. if Result of Chickering 2002 implies that under faithfulness we can always find an sequence of independence maps G π =: G 0 G G 2 G N := G π. Can always find a sequence that coincides with a nonincreasing edge walk! Theorem (Matejovicova, LS, Uhler, Y. Wang, 207) The Triangle SP Algorithm is consistent under the faithfulness assumption. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
17 Greedy SP Algorithms Parsing out the Assumptions ESP Assumption: The assumption guaranteeing consistency of Edge SP. TSP Assumption: The assumption guaranteeing consistency of Triangle SP. A B = A is strictly weaker than B Theorem (Matejovicova, LS, Uhler, Y. Wang, 207) SMR ESP TSP faithfulness. What about restricted faithfulness? Theorem (Matejovicova, LS, Uhler, Y. Wang, 207) Consistency under the TSP assumption implies adjacency faithfulness, but not orientation faithfulness. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
18 Greedy SP Algorithms The Good News The vertices of the DAG Associahedron serve as a reduced search space for the SP Algorithm. In this way, we can execute the SP algorithm in a reduced search space (less than n! elements). The ESP and TSP Algorithms can greedily search over the vertices of a DAG associahedron. The TSP Algorithm is consistent under faithfulness. We understand the relationships amongst the SMR, ESP, TSP, faithfulness, and restricted faithfulness assumptions. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
19 Greedy SP Algorithms Markov Equivalence The Bad News: Markov Equivalence of DAGs Our search along the edges is not truly greedy... At times we are required to move between DAGs G π and G τ where G π = G τ Two DAGs that differ only by a covered edge reversal entail the same set of CI relations. We call any two such DAGs Markov equivalent. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
20 Greedy SP Algorithms Markov Equivalence The Bad News: The Problem of Markov Equivalence Problem: The algorithm may search through large portions of a Markov Equivalence Class (MEC) before finding a neighboring DAG with fewer edges. To terminate it must search the ENTIRE MEC of the sparsest DAGs! This motivates two enumerative questions: For a fixed set of graph parameters, how many MECs are there? 2 What are the sizes of these MECs? Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
21 Greedy SP Algorithms Markov Equivalence Markov Equivalence of DAGs A collider that is not in a triangle is called an immorality Immorality Not an immorality Theorem (Verma and Pearl, 992) Two DAGs are Markov equivalent if and only if they have the same skeleton and the same set of immoralities. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
22 Greedy SP Algorithms Markov Equivalence Markov Equivalence of DAGs An MEC with three elements: [G] = the MEC containing the DAG G. [G] has an associated partially directed graph called the essential graph of [G]. chain components of [G] = undirected connected components of Ĝ. essential components of [G] = directed connected components of Ĝ. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
23 Greedy SP Algorithms Markov Equivalence The Two Enumerative Questions Previous work: (Gillespie and Perlman, 200) Computer enumeration of all MECs on p 0 nodes. 2 (Gillespie 06, Steinsky 03, Wagner 3) Enumeration of MECs of a size: Formulas only for small class sizes (size one, two, and three), or restricted chordal components. 2 Inclusion-Exclusion arguments on essential graphs. A new approach: Greedy SP algorithm must search ENTIRE true MEC to terminate 2 when can we solve the enumeration problem for a fixed skeleton? Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
24 The Problem of Markov Equivalence Combinatorial Enumeration by Skeleton A New Instance of an Old Combinatorial Approach Restrict to a type of skeleton and solve the enumeration problem here. + I p := the path on p vertices. M(G) := number of MECs with skeleton G. M(I p ) = M(I p ) + M(I p 2 ) and M(I ) =, M(I 2 ) = M(I p ) = F p, the (p ) st Fibonacci number. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
25 The Problem of Markov Equivalence Combinatorial Enumeration by Skeleton A Second Proof: More Information An independent set in G is a subset of mutually non-adjacent nodes: α k (G) := number of independent sets in G of size k I (G; x) := k 0 α k(g)x k ; the independence polynomial of G. 2 m k (G) := number of MECs on G with k immoralities. k 0 m k(i p )x k = I (I p 2 ; x) = F p (x), the (p ) st Fibonacci Polynomial Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
26 The Problem of Markov Equivalence Combinatorial Enumeration by Skeleton A Second Proof: More Information s l (G) := the number of MECs on G of size l. A composition of p into k parts is an ordered sum of k positive integers the value of which is p: c + c c k = p. s l (I p ) is the number of compositions of p k into k + parts over all k for which k+ j= c j = l. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
27 The Problem of Markov Equivalence Combinatorial Enumeration by Skeleton Combinatorial Statistics: Refining the Problem M(G) = the total number of MECs on G. m k (G) = the number of MECs on G with precisely k immoralities. m(g) = the maximum number of immoralities within an MEC on G. s k (G) = the number of MECs on G of size k. The first three statistics combine into the polynomial generating function: M(G; x) := m(g) k=0 m k (G)x k Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
28 The Problem of Markov Equivalence Combinatorial Enumeration by Skeleton Some Further Examples The complete set of these statistics are recoverable for some other important graphs including: S p K,p C p K 2,p Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
29 The Problem of Markov Equivalence Combinatorial Enumeration by Skeleton Sparse Examples: Bounds for Trees A classical bound on the number of independent sets a tree also holds for the number of MECs on a tree: Theorem (Radhakrishnan, LS, Uhler, 206) Let T p be an undirected tree on p nodes. Then F p = M(I p ) M(T p ) M(S p ) = 2 p p +. We can also bound the size of an MEC: Theorem (Radhakrishnan, LS, Uhler, 206) Let T p be a directed tree on p nodes whose essential graph has l > 0 chain components m 0 essential components. Then ( ) p m l 2 l #[T p ]. l Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
30 Greedy SP-Algorithms An Implementable Algorithm Overcoming Exponentiality: An Implementable Algorithm We need a version of the TSP-Algorithm that avoids the problem of exponentially-sized Markov equivalence classes of permutation DAGs. Solution: Introduce a search-depth bound d and a fixed number of runs r. Triangle SP Algorithm with depth and run bounds Input: C and two positive integers d and r. Pick a permutation DAG G π and do a depth-first search along the edges of P n (C) with depth bound d, searching for a sparser permutation DAG. Repeat search until no sparser DAG is found, and return the last DAG visited. 2 Do step r times and then select the sparsest of the r permutation DAGs recovered. Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
31 Greedy SP-Algorithms An Implementable Algorithm A Sample of Some Simulations Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
32 Greedy SP-Algorithms An Implementable Algorithm A Sample of Some Simulations Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
33 Moral of the Story Combinatorial convex geometry of Generalized Permutohedra can provide DAG model learning algorithms useful in causal inference! Combinatorics of DAG associahedra provide a graphical version of these algorithms that are implementable and are consistent under common identifiability assumptions! The number and size of Markov equivalence classes is important to understanding the efficiency of algorithms searching over a space of DAGs. Connections to classic combinatorial optimization problems yield FUN problems and reveal that MECs can be large even for sparse graphs! Adjusting algorithms with fixed search-depth and run bounds results in algorithms that are efficient and more reliable than the PC-algorithm! Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
34 Thank You! (Preprints available on the ArXiv!) Liam Solus (KTH) Learning Bayesian Networks 8 January / 34
arxiv: v2 [math.pr] 14 Aug 2017
A NOTE ON SAMPLING GRAPHICAL MARKOV MODELS MEGAN BERNSTEIN AND PRASAD TETALI arxiv:1705.09717v [math.pr] 14 Aug 017 Abstract. We consider sampling and enumeration problems for Markov equivalence classes.
More informationLecture 4: Undirected Graphical Models
Lecture 4: Undirected Graphical Models Department of Biostatistics University of Michigan zhenkewu@umich.edu http://zhenkewu.com/teaching/graphical_model 15 September, 2016 Zhenke Wu BIOSTAT830 Graphical
More informationCounting and Exploring Sizes of Markov Equivalence Classes of Directed Acyclic Graphs
Journal of Machine Learning Research 16 (2015) 2589-2609 Submitted 9/14; Revised 3/15; Published 12/15 Counting and Exploring Sizes of Markov Equivalence Classes of Directed Acyclic Graphs Yangbo He heyb@pku.edu.cn
More informationResearch Article Structural Learning about Directed Acyclic Graphs from Multiple Databases
Abstract and Applied Analysis Volume 2012, Article ID 579543, 9 pages doi:10.1155/2012/579543 Research Article Structural Learning about Directed Acyclic Graphs from Multiple Databases Qiang Zhao School
More informationA Transformational Characterization of Markov Equivalence for Directed Maximal Ancestral Graphs
A Transformational Characterization of Markov Equivalence for Directed Maximal Ancestral Graphs Jiji Zhang Philosophy Department Carnegie Mellon University Pittsburgh, PA 15213 jiji@andrew.cmu.edu Abstract
More informationLearning DAGs from observational data
Learning DAGs from observational data General overview Introduction DAGs and conditional independence DAGs and causal effects Learning DAGs from observational data IDA algorithm Further problems 2 What
More informationCounting and Exploring Sizes of Markov Equivalence Classes of Directed Acyclic Graphs
Counting and Exploring Sizes of Markov Equivalence Classes of Directed Acyclic Graphs Yangbo He heyb@pku.edu.cn Jinzhu Jia jzjia@math.pku.edu.cn LMAM, School of Mathematical Sciences, LMEQF, and Center
More informationTheorem 2.9: nearest addition algorithm
There are severe limits on our ability to compute near-optimal tours It is NP-complete to decide whether a given undirected =(,)has a Hamiltonian cycle An approximation algorithm for the TSP can be used
More informationA note on the pairwise Markov condition in directed Markov fields
TECHNICAL REPORT R-392 April 2012 A note on the pairwise Markov condition in directed Markov fields Judea Pearl University of California, Los Angeles Computer Science Department Los Angeles, CA, 90095-1596,
More informationMinimal I-MAP MCMC for Scalable Structure Discovery in Causal DAG Models
for Scalable Structure Discovery in Causal DAG Models Raj Agrawal 123 Tamara Broderick 12 Caroline Uhler 23 Abstract Learning a Bayesian network (BN) from data can be useful for decision-making or discovering
More informationThese notes present some properties of chordal graphs, a set of undirected graphs that are important for undirected graphical models.
Undirected Graphical Models: Chordal Graphs, Decomposable Graphs, Junction Trees, and Factorizations Peter Bartlett. October 2003. These notes present some properties of chordal graphs, a set of undirected
More informationA Discovery Algorithm for Directed Cyclic Graphs
A Discovery Algorithm for Directed Cyclic Graphs Thomas Richardson 1 Logic and Computation Programme CMU, Pittsburgh PA 15213 1. Introduction Directed acyclic graphs have been used fruitfully to represent
More informationGraphical Models and Markov Blankets
Stephan Stahlschmidt Ladislaus von Bortkiewicz Chair of Statistics C.A.S.E. Center for Applied Statistics and Economics Humboldt-Universität zu Berlin Motivation 1-1 Why Graphical Models? Illustration
More informationChapter 2 PRELIMINARIES. 1. Random variables and conditional independence
Chapter 2 PRELIMINARIES In this chapter the notation is presented and the basic concepts related to the Bayesian network formalism are treated. Towards the end of the chapter, we introduce the Bayesian
More informationCME 305: Discrete Mathematics and Algorithms Instructor: Reza Zadeh HW#3 Due at the beginning of class Thursday 02/26/15
CME 305: Discrete Mathematics and Algorithms Instructor: Reza Zadeh (rezab@stanford.edu) HW#3 Due at the beginning of class Thursday 02/26/15 1. Consider a model of a nonbipartite undirected graph in which
More informationSeparators and Adjustment Sets in Markov Equivalent DAGs
Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16) Separators and Adjustment Sets in Markov Equivalent DAGs Benito van der Zander and Maciej Liśkiewicz Institute of Theoretical
More informationA combinatorial interpretation of the h- and γ-vectors of the cyclohedron
A combinatorial interpretation of the h- and γ-vectors of the cyclohedron Pol Gómez Riquelme under the direction of Pavel Galashin MIT Mathematics Department Research Science Institute July 8, 015 Abstract
More informationStat 5421 Lecture Notes Graphical Models Charles J. Geyer April 27, Introduction. 2 Undirected Graphs
Stat 5421 Lecture Notes Graphical Models Charles J. Geyer April 27, 2016 1 Introduction Graphical models come in many kinds. There are graphical models where all the variables are categorical (Lauritzen,
More informationarxiv: v2 [stat.ml] 27 Sep 2013
Order-independent causal structure learning Order-independent constraint-based causal structure learning arxiv:1211.3295v2 [stat.ml] 27 Sep 2013 Diego Colombo Marloes H. Maathuis Seminar for Statistics,
More informationChordal graphs MPRI
Chordal graphs MPRI 2017 2018 Michel Habib habib@irif.fr http://www.irif.fr/~habib Sophie Germain, septembre 2017 Schedule Chordal graphs Representation of chordal graphs LBFS and chordal graphs More structural
More information10708 Graphical Models: Homework 2
10708 Graphical Models: Homework 2 Due October 15th, beginning of class October 1, 2008 Instructions: There are six questions on this assignment. Each question has the name of one of the TAs beside it,
More informationarxiv: v1 [cs.cg] 7 Oct 2017
A Proof of the Orbit Conjecture for Flipping Edge-Labelled Triangulations Anna Lubiw 1, Zuzana Masárová 2, and Uli Wagner 2 arxiv:1710.02741v1 [cs.cg] 7 Oct 2017 1 School of Computer Science, University
More informationSurvey of contemporary Bayesian Network Structure Learning methods
Survey of contemporary Bayesian Network Structure Learning methods Ligon Liu September 2015 Ligon Liu (CUNY) Survey on Bayesian Network Structure Learning (slide 1) September 2015 1 / 38 Bayesian Network
More informationSummary: A Tutorial on Learning With Bayesian Networks
Summary: A Tutorial on Learning With Bayesian Networks Markus Kalisch May 5, 2006 We primarily summarize [4]. When we think that it is appropriate, we comment on additional facts and more recent developments.
More informationLearning Causal Graphs with Small Interventions
Learning Causal Graphs with Small Interventions Karthieyan Shanmugam 1, Murat Kocaoglu 2, Alexandros G. Dimais 3, Sriram Vishwanath 4 Department of Electrical and Computer Engineering The University of
More informationOrder-Independent Constraint-Based Causal Structure Learning
Journal of Machine Learning Research 15 (2014) 3741-3782 Submitted 9/13; Revised 7/14; Published 11/14 Order-Independent Constraint-Based Causal Structure Learning Diego Colombo Marloes H. Maathuis Seminar
More informationCausal inference and causal explanation with background knowledge
403 Causal inference and causal explanation with background knowledge Christopher Meek Department of Philosophy Carnegie Mellon University Pittsburgh, PA 15213* Abstract This paper presents correct algorithms
More informationLecture 5: Exact inference. Queries. Complexity of inference. Queries (continued) Bayesian networks can answer questions about the underlying
given that Maximum a posteriori (MAP query: given evidence 2 which has the highest probability: instantiation of all other variables in the network,, Most probable evidence (MPE: given evidence, find an
More informationBest known solution time is Ω(V!) Check every permutation of vertices to see if there is a graph edge between adjacent vertices
Hard Problems Euler-Tour Problem Undirected graph G=(V,E) An Euler Tour is a path where every edge appears exactly once. The Euler-Tour Problem: does graph G have an Euler Path? Answerable in O(E) time.
More informationBayesian Networks, Winter Yoav Haimovitch & Ariel Raviv
Bayesian Networks, Winter 2009-2010 Yoav Haimovitch & Ariel Raviv 1 Chordal Graph Warm up Theorem 7 Perfect Vertex Elimination Scheme Maximal cliques Tree Bibliography M.C.Golumbic Algorithmic Graph Theory
More informationRobust Independence-Based Causal Structure Learning in Absence of Adjacency Faithfulness
Robust Independence-Based Causal Structure Learning in Absence of Adjacency Faithfulness Jan Lemeire Stijn Meganck Francesco Cartella ETRO Department, Vrije Universiteit Brussel, Belgium Interdisciplinary
More information2.1 Greedy Algorithms. 2.2 Minimum Spanning Trees. CS125 Lecture 2 Fall 2016
CS125 Lecture 2 Fall 2016 2.1 Greedy Algorithms We will start talking about methods high-level plans for constructing algorithms. One of the simplest is just to have your algorithm be greedy. Being greedy,
More informationMonotone Paths in Geometric Triangulations
Monotone Paths in Geometric Triangulations Adrian Dumitrescu Ritankar Mandal Csaba D. Tóth November 19, 2017 Abstract (I) We prove that the (maximum) number of monotone paths in a geometric triangulation
More informationChapter 4. Relations & Graphs. 4.1 Relations. Exercises For each of the relations specified below:
Chapter 4 Relations & Graphs 4.1 Relations Definition: Let A and B be sets. A relation from A to B is a subset of A B. When we have a relation from A to A we often call it a relation on A. When we have
More informationLearning Equivalence Classes of Bayesian-Network Structures
Journal of Machine Learning Research 2 (2002) 445-498 Submitted 7/01; Published 2/02 Learning Equivalence Classes of Bayesian-Network Structures David Maxwell Chickering Microsoft Research One Microsoft
More informationDiscrete mathematics , Fall Instructor: prof. János Pach
Discrete mathematics 2016-2017, Fall Instructor: prof. János Pach - covered material - Lecture 1. Counting problems To read: [Lov]: 1.2. Sets, 1.3. Number of subsets, 1.5. Sequences, 1.6. Permutations,
More informationIntroduction to Graph Theory
Introduction to Graph Theory Tandy Warnow January 20, 2017 Graphs Tandy Warnow Graphs A graph G = (V, E) is an object that contains a vertex set V and an edge set E. We also write V (G) to denote the vertex
More informationNOTICE WARNING CONCERNING COPYRIGHT RESTRICTIONS: The copyright law of the United States (title 17, U.S. Code) governs the making of photocopies or
NOTICE WARNING CONCERNING COPYRIGHT RESTRICTIONS: The copyright law of the United States (title 17, U.S. Code) governs the making of photocopies or other reproductions of copyrighted material. Any copying
More informationFigure 4.1: The evolution of a rooted tree.
106 CHAPTER 4. INDUCTION, RECURSION AND RECURRENCES 4.6 Rooted Trees 4.6.1 The idea of a rooted tree We talked about how a tree diagram helps us visualize merge sort or other divide and conquer algorithms.
More informationThe Fibonacci hypercube
AUSTRALASIAN JOURNAL OF COMBINATORICS Volume 40 (2008), Pages 187 196 The Fibonacci hypercube Fred J. Rispoli Department of Mathematics and Computer Science Dowling College, Oakdale, NY 11769 U.S.A. Steven
More informationSpanning trees and orientations of graphs
Journal of Combinatorics Volume 1, Number 2, 101 111, 2010 Spanning trees and orientations of graphs Carsten Thomassen A conjecture of Merino and Welsh says that the number of spanning trees τ(g) of a
More informationTreewidth and graph minors
Treewidth and graph minors Lectures 9 and 10, December 29, 2011, January 5, 2012 We shall touch upon the theory of Graph Minors by Robertson and Seymour. This theory gives a very general condition under
More informationarxiv: v1 [stat.ot] 20 Oct 2011
Markov Equivalences for Subclasses of Loopless Mixed Graphs KAYVAN SADEGHI Department of Statistics, University of Oxford arxiv:1110.4539v1 [stat.ot] 20 Oct 2011 ABSTRACT: In this paper we discuss four
More informationDirected Graphical Models (Bayes Nets) (9/4/13)
STA561: Probabilistic machine learning Directed Graphical Models (Bayes Nets) (9/4/13) Lecturer: Barbara Engelhardt Scribes: Richard (Fangjian) Guo, Yan Chen, Siyang Wang, Huayang Cui 1 Introduction For
More informationUnit 8: Coping with NP-Completeness. Complexity classes Reducibility and NP-completeness proofs Coping with NP-complete problems. Y.-W.
: Coping with NP-Completeness Course contents: Complexity classes Reducibility and NP-completeness proofs Coping with NP-complete problems Reading: Chapter 34 Chapter 35.1, 35.2 Y.-W. Chang 1 Complexity
More informationCME 305: Discrete Mathematics and Algorithms Instructor: Reza Zadeh HW#3 Due at the beginning of class Thursday 03/02/17
CME 305: Discrete Mathematics and Algorithms Instructor: Reza Zadeh (rezab@stanford.edu) HW#3 Due at the beginning of class Thursday 03/02/17 1. Consider a model of a nonbipartite undirected graph in which
More informationCSE 431/531: Algorithm Analysis and Design (Spring 2018) Greedy Algorithms. Lecturer: Shi Li
CSE 431/531: Algorithm Analysis and Design (Spring 2018) Greedy Algorithms Lecturer: Shi Li Department of Computer Science and Engineering University at Buffalo Main Goal of Algorithm Design Design fast
More informationwith Dana Richards December 1, 2017 George Mason University New Results On Routing Via Matchings Indranil Banerjee The Routing Model
New New with Dana Richards George Mason University richards@gmu.edu December 1, 2017 GMU December 1, 2017 1 / 40 New Definitions G(V, E) is an undirected graph. V = {1, 2, 3,..., n}. A pebble at vertex
More informationIntroductory Combinatorics
Introductory Combinatorics Third Edition KENNETH P. BOGART Dartmouth College,. " A Harcourt Science and Technology Company San Diego San Francisco New York Boston London Toronto Sydney Tokyo xm CONTENTS
More informationIntegrating locally learned causal structures with overlapping variables
Integrating locally learned causal structures with overlapping variables Robert E. Tillman Carnegie Mellon University Pittsburgh, PA rtillman@andrew.cmu.edu David Danks, Clark Glymour Carnegie Mellon University
More informationLecture 5: Exact inference
Lecture 5: Exact inference Queries Inference in chains Variable elimination Without evidence With evidence Complexity of variable elimination which has the highest probability: instantiation of all other
More informationCSE 431/531: Analysis of Algorithms. Greedy Algorithms. Lecturer: Shi Li. Department of Computer Science and Engineering University at Buffalo
CSE 431/531: Analysis of Algorithms Greedy Algorithms Lecturer: Shi Li Department of Computer Science and Engineering University at Buffalo Main Goal of Algorithm Design Design fast algorithms to solve
More information6. Lecture notes on matroid intersection
Massachusetts Institute of Technology 18.453: Combinatorial Optimization Michel X. Goemans May 2, 2017 6. Lecture notes on matroid intersection One nice feature about matroids is that a simple greedy algorithm
More informationDistance Sequences in Locally Infinite Vertex-Transitive Digraphs
Distance Sequences in Locally Infinite Vertex-Transitive Digraphs Wesley Pegden July 7, 2004 Abstract We prove that the out-distance sequence {f + (k)} of a vertex-transitive digraph of finite or infinite
More information2. Graphical Models. Undirected graphical models. Factor graphs. Bayesian networks. Conversion between graphical models. Graphical Models 2-1
Graphical Models 2-1 2. Graphical Models Undirected graphical models Factor graphs Bayesian networks Conversion between graphical models Graphical Models 2-2 Graphical models There are three families of
More information5. Lecture notes on matroid intersection
Massachusetts Institute of Technology Handout 14 18.433: Combinatorial Optimization April 1st, 2009 Michel X. Goemans 5. Lecture notes on matroid intersection One nice feature about matroids is that a
More informationFinal. Name: TA: Section Time: Course Login: Person on Left: Person on Right: U.C. Berkeley CS170 : Algorithms, Fall 2013
U.C. Berkeley CS170 : Algorithms, Fall 2013 Final Professor: Satish Rao December 16, 2013 Name: Final TA: Section Time: Course Login: Person on Left: Person on Right: Answer all questions. Read them carefully
More informationSub-Local Constraint-Based Learning of Bayesian Networks Using A Joint Dependence Criterion
Journal of Machine Learning Research 14 (2013) 1563-1603 Submitted 11/10; Revised 9/12; Published 6/13 Sub-Local Constraint-Based Learning of Bayesian Networks Using A Joint Dependence Criterion Rami Mahdi
More informationMinimal Dominating Sets in Graphs: Enumeration, Combinatorial Bounds and Graph Classes
Minimal Dominating Sets in Graphs: Enumeration, Combinatorial Bounds and Graph Classes J.-F. Couturier 1 P. Heggernes 2 D. Kratsch 1 P. van t Hof 2 1 LITA Université de Lorraine F-57045 Metz France 2 University
More information3 No-Wait Job Shops with Variable Processing Times
3 No-Wait Job Shops with Variable Processing Times In this chapter we assume that, on top of the classical no-wait job shop setting, we are given a set of processing times for each operation. We may select
More informationComplexity of Disjoint Π-Vertex Deletion for Disconnected Forbidden Subgraphs
Journal of Graph Algorithms and Applications http://jgaa.info/ vol. 18, no. 4, pp. 603 631 (2014) DOI: 10.7155/jgaa.00339 Complexity of Disjoint Π-Vertex Deletion for Disconnected Forbidden Subgraphs Jiong
More informationCh9: Exact Inference: Variable Elimination. Shimi Salant, Barak Sternberg
Ch9: Exact Inference: Variable Elimination Shimi Salant Barak Sternberg Part 1 Reminder introduction (1/3) We saw two ways to represent (finite discrete) distributions via graphical data structures: Bayesian
More informationChapter 6. Dynamic Programming
Chapter 6 Dynamic Programming CS 573: Algorithms, Fall 203 September 2, 203 6. Maximum Weighted Independent Set in Trees 6..0. Maximum Weight Independent Set Problem Input Graph G = (V, E) and weights
More informationEdge intersection graphs. of systems of grid paths. with bounded number of bends
Edge intersection graphs of systems of grid paths with bounded number of bends Andrei Asinowski a, Andrew Suk b a Caesarea Rothschild Institute, University of Haifa, Haifa 31905, Israel. b Courant Institute,
More informationRecitation 4: Elimination algorithm, reconstituted graph, triangulation
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.438 Algorithms For Inference Fall 2014 Recitation 4: Elimination algorithm, reconstituted graph, triangulation
More informationLecture 3: Graphs and flows
Chapter 3 Lecture 3: Graphs and flows Graphs: a useful combinatorial structure. Definitions: graph, directed and undirected graph, edge as ordered pair, path, cycle, connected graph, strongly connected
More information4.1.2 Merge Sort Sorting Lower Bound Counting Sort Sorting in Practice Solving Problems by Sorting...
Contents 1 Introduction... 1 1.1 What is Competitive Programming?... 1 1.1.1 Programming Contests.... 2 1.1.2 Tips for Practicing.... 3 1.2 About This Book... 3 1.3 CSES Problem Set... 5 1.4 Other Resources...
More information1 The Traveling Salesman Problem
Comp 260: Advanced Algorithms Tufts University, Spring 2018 Prof. Lenore Cowen Scribe: Duc Nguyen Lecture 3a: The Traveling Salesman Problem 1 The Traveling Salesman Problem The Traveling Salesman Problem
More informationLearning Bounded Treewidth Bayesian Networks
Journal of Machine Learning Research 9 (2008) 2287-2319 Submitted 5/08; Published 10/08 Learning Bounded Treewidth Bayesian Networks Gal Elidan Department of Statistics Hebrew University Jerusalem, 91905,
More informationInfo 2950, Lecture 16
Info 2950, Lecture 16 28 Mar 2017 Prob Set 5: due Fri night 31 Mar Breadth first search (BFS) and Depth First Search (DFS) Must have an ordering on the vertices of the graph. In most examples here, the
More informationCluster algebras and infinite associahedra
Cluster algebras and infinite associahedra Nathan Reading NC State University CombinaTexas 2008 Coxeter groups Associahedra and cluster algebras Sortable elements/cambrian fans Infinite type Much of the
More informationD-Separation. b) the arrows meet head-to-head at the node, and neither the node, nor any of its descendants, are in the set C.
D-Separation Say: A, B, and C are non-intersecting subsets of nodes in a directed graph. A path from A to B is blocked by C if it contains a node such that either a) the arrows on the path meet either
More informationFMA901F: Machine Learning Lecture 6: Graphical Models. Cristian Sminchisescu
FMA901F: Machine Learning Lecture 6: Graphical Models Cristian Sminchisescu Graphical Models Provide a simple way to visualize the structure of a probabilistic model and can be used to design and motivate
More informationSTAT 598L Learning Bayesian Network Structure
STAT 598L Learning Bayesian Network Structure Sergey Kirshner Department of Statistics Purdue University skirshne@purdue.edu November 2, 2009 Acknowledgements: some of the slides were based on Luo Si s
More information6. Algorithm Design Techniques
6. Algorithm Design Techniques 6. Algorithm Design Techniques 6.1 Greedy algorithms 6.2 Divide and conquer 6.3 Dynamic Programming 6.4 Randomized Algorithms 6.5 Backtracking Algorithms Malek Mouhoub, CS340
More informationProper Partitions of a Polygon and k-catalan Numbers
Proper Partitions of a Polygon and k-catalan Numbers Bruce E. Sagan Department of Mathematics Michigan State University East Lansing, MI 48824-1027 USA sagan@math.msu.edu July 13, 2005 Abstract Let P be
More informationComputer Vision Group Prof. Daniel Cremers. 4. Probabilistic Graphical Models Directed Models
Prof. Daniel Cremers 4. Probabilistic Graphical Models Directed Models The Bayes Filter (Rep.) (Bayes) (Markov) (Tot. prob.) (Markov) (Markov) 2 Graphical Representation (Rep.) We can describe the overall
More informationOptimal Region for Binary Search Tree, Rotation and Polytope
Optimal Region for Binary Search Tree, Rotation and Polytope Kensuke Onishi Mamoru Hoshi 2 Department of Mathematical Sciences, School of Science Tokai University, 7 Kitakaname, Hiratsuka, Kanagawa, 259-292,
More informationGraph Algorithms. Chromatic Polynomials. Graph Algorithms
Graph Algorithms Chromatic Polynomials Graph Algorithms Chromatic Polynomials Definition G a simple labelled graph with n vertices and m edges. k a positive integer. P G (k) number of different ways of
More informationGraph Algorithms. Chapter 22. CPTR 430 Algorithms Graph Algorithms 1
Graph Algorithms Chapter 22 CPTR 430 Algorithms Graph Algorithms Why Study Graph Algorithms? Mathematical graphs seem to be relatively specialized and abstract Why spend so much time and effort on algorithms
More informationFrom NP to P Musings on a Programming Contest Problem
From NP to P Musings on a Programming Contest Problem Edward Corwin Antonette Logar Mathematics and CS SDSM&T Rapid City, SD 57701 edward.corwin@sdsmt.edu ABSTRACT A classic analysis of algorithms problem
More informationThe Associahedra and Permutohedra Yet Again
The Associahedra and Permutohedra Yet Again M. D. Sheppeard Abstract The associahedra and permutohedra polytopes are redefined as subsets of discrete simplices, associated respectively to the commutative
More informationPebble Sets in Convex Polygons
2 1 Pebble Sets in Convex Polygons Kevin Iga, Randall Maddox June 15, 2005 Abstract Lukács and András posed the problem of showing the existence of a set of n 2 points in the interior of a convex n-gon
More informationarxiv: v1 [cs.ai] 11 Oct 2015
Journal of Machine Learning Research 1 (2000) 1-48 Submitted 4/00; Published 10/00 ParallelPC: an R package for efficient constraint based causal exploration arxiv:1510.03042v1 [cs.ai] 11 Oct 2015 Thuc
More informationDirected Graphical Models
Copyright c 2008 2010 John Lafferty, Han Liu, and Larry Wasserman Do Not Distribute Chapter 18 Directed Graphical Models Graphs give a powerful way of representing independence relations and computing
More information1. A busy airport has 1500 takeo s perday. Provetherearetwoplanesthatmusttake o within one minute of each other. This is from Bona Chapter 1 (1).
Math/CS 415 Combinatorics and Graph Theory Fall 2017 Prof. Readdy Homework Chapter 1 1. A busy airport has 1500 takeo s perday. Provetherearetwoplanesthatmusttake o within one minute of each other. This
More informationNew algorithms for sampling closed and/or confined equilateral polygons
New algorithms for sampling closed and/or confined equilateral polygons Jason Cantarella and Clayton Shonkwiler University of Georgia BIRS Workshop Entanglement in Biology November, 2013 Closed random
More informationSmall Survey on Perfect Graphs
Small Survey on Perfect Graphs Michele Alberti ENS Lyon December 8, 2010 Abstract This is a small survey on the exciting world of Perfect Graphs. We will see when a graph is perfect and which are families
More informationInteraction Between Input and Output-Sensitive
Interaction Between Input and Output-Sensitive Really? Mamadou M. Kanté Université Blaise Pascal - LIMOS, CNRS Enumeration Algorithms Using Structure, Lorentz Institute, August 26 th, 2015 1 Introduction
More informationPractice Problems for the Final
ECE-250 Algorithms and Data Structures (Winter 2012) Practice Problems for the Final Disclaimer: Please do keep in mind that this problem set does not reflect the exact topics or the fractions of each
More informationMachine Learning. Sourangshu Bhattacharya
Machine Learning Sourangshu Bhattacharya Bayesian Networks Directed Acyclic Graph (DAG) Bayesian Networks General Factorization Curve Fitting Re-visited Maximum Likelihood Determine by minimizing sum-of-squares
More informationBMI/STAT 768: Lecture 06 Trees in Graphs
BMI/STAT 768: Lecture 06 Trees in Graphs Moo K. Chung mkchung@wisc.edu February 11, 2018 Parts of this lecture is based on [3, 5]. Many objects and data can be represented as networks. Unfortunately networks
More informationA Note on Polyhedral Relaxations for the Maximum Cut Problem
A Note on Polyhedral Relaxations for the Maximum Cut Problem Alantha Newman Abstract We consider three well-studied polyhedral relaxations for the maximum cut problem: the metric polytope of the complete
More information1 Variations of the Traveling Salesman Problem
Stanford University CS26: Optimization Handout 3 Luca Trevisan January, 20 Lecture 3 In which we prove the equivalence of three versions of the Traveling Salesman Problem, we provide a 2-approximate algorithm,
More informationPaths, Flowers and Vertex Cover
Paths, Flowers and Vertex Cover Venkatesh Raman M. S. Ramanujan Saket Saurabh Abstract It is well known that in a bipartite (and more generally in a König) graph, the size of the minimum vertex cover is
More information3 : Representation of Undirected GMs
0-708: Probabilistic Graphical Models 0-708, Spring 202 3 : Representation of Undirected GMs Lecturer: Eric P. Xing Scribes: Nicole Rafidi, Kirstin Early Last Time In the last lecture, we discussed directed
More informationGraph Theory using Sage
Introduction Student Projects My Projects Seattle, August 2009 Introduction Student Projects My Projects 1 Introduction Background 2 Student Projects Conference Graphs The Matching Polynomial 3 My Projects
More informationAlgorithm and Complexity of Disjointed Connected Dominating Set Problem on Trees
Algorithm and Complexity of Disjointed Connected Dominating Set Problem on Trees Wei Wang joint with Zishen Yang, Xianliang Liu School of Mathematics and Statistics, Xi an Jiaotong University Dec 20, 2016
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS Bayesian Networks Directed Acyclic Graph (DAG) Bayesian Networks General Factorization Bayesian Curve Fitting (1) Polynomial Bayesian
More informationProbabilistic Graphical Models
Overview of Part One Probabilistic Graphical Models Part One: Graphs and Markov Properties Christopher M. Bishop Graphs and probabilities Directed graphs Markov properties Undirected graphs Examples Microsoft
More information