Bayesian Networks, Winter Yoav Haimovitch & Ariel Raviv

Transcription

1 Bayesian Networks, Winter Yoav Haimovitch & Ariel Raviv 1

2 Chordal Graph Warm up Theorem 7 Perfect Vertex Elimination Scheme Maximal cliques Tree Bibliography M.C.Golumbic Algorithmic Graph Theory and Perfect Graphs. Second edition Pearl, J. Probabilistic Reasoning in Intelligent Systems. 2

3 Definition: Undirected graph G is chordal if every cycle of length greater than 3 has a chord. Remark: Chordality is a hereditary property, All induced subgraphs of a chordal graph are chordal! Chordal graphs are also called triangulated and perfect elimination graphs. 3

4 4

5 A vertex x of G is called simplicial if its adjacency set Adj(x) is a clique (not necessarily maximal) Lemma 1[Dirac 1961]: Each chordal graph has a simplicial vertex and if G is not a clique it has two non adjacent simplicial vertices. 5

6 If G is a clique, done. If not, assume that G has two non-adjacent nodes a and b and that the lemma is true to all graphs with fewer vertices than G. Let S be a minimal vertex separator for a and b. Let G a and G b be the connected components of a and b respectively. 6

7 S is a clique if not, then there exists x and y such that there is no edge between them, and because S is minimal vertex separator there is a cycle <a,,x,,b,,y,,a>. It is easy to show that there exist a minimal cycle with length no less then 4 with no chords in it, contradiction. 7

8 S is a clique. G A +S is smaller than G therefore by induction the lemma holds, i.e. G A +S is a clique or has two non adjacent simplicial vertices, one of which must be in G A. Any simplicial vertex in G A is a simplicial vertex in G because all elements of Adj(a) are inside G A +S. Thus from G A and G B we get two simplicial vertices in G. 8

9 Definition: The following are equivalent: 1. G is chordal 2. The edges of G can be directed acyclically so that every pair of converging arrows comes from two adjacent vertices. 3. G has a perfect vertex elimination scheme. 4. There is a tree T with maximal cliques of G as vertices. Any two cliques containing v are either adjacent in T or connected by a path of cliques that contain v. 9

10 By lemma 1 every chordal graph has a simplicial vertex Direct all of its edges to it Repeat the process with the rest of the graph (can be done because being chordal is hereditary) From the definition of simplicial vertex corollary that each two converging edges coming from adjacent vertices. 10

11 Let there be a cycle of size larger than 3. There exist a valid direction which is acyclic There are two converging edges, which come from two adjacent vertices The cycle has a chord. 11

12 Given a Markov network, we would like to calculate a probability function over its variables What if we want to calculate the probability of one variable? 12

13 P(X 1,X 2,X 3,X 4,X 5 )=K f 1 (X 1,X 2 )f 2 (X 2,X 3 )f 3 (X 3,X 4 )f 4 (X 4,X 5 )f 5 (X 5,X 1 ) P(X 1,X 2,X 3,X 4 )=Σ X5 K f 1 (X 1,X 2 )f2(x 2,X 3 )f 3 (X 3,X 4 )f 4 (X 4,X 5 )f 5 (X 5,X 1 )= =K f 1 (X 1,X 2 )f 2 (X 2,X 3 )f 3 (X 3,X 4 ) Σ X5 f 4 (X 4,X 5 )f 5 (X 5,X 1 ) = K` f 1 (X 1,X 2 )f 2 (X 2,X 3 )f 3 (X 3,X 4 ) F(X 4, X 1 ) Explanation: Each time we sum over a variable we automatically add edges between all its adjacent vertices (creating a clique) 13

14 Here is an example of how bad (i.e. big cliques) things can happen with incorrect order of summing P(X 1,X 2,X 3,X 4,X 5 )=Σ X0 K f 1 (X 0,X 1 )f 2 (X 0,X 2 )f 3 (X 0,X 3 )f 4 (X 0,X 4 )f 5 (X 0,X 5 ) = K` F(X 1,X 2,X 3,X 4,X 5 ) Explanation: For wrong order of summing we get big cliques and thus huge tables of probabilities to store, which are exponential in number of variables. 14

15 Let G=(V,E) be an undirected graph and let σ=[v 1,v 2,,v n ] be an ordering of the vertices. We say that σ is a perfect vertex elimination scheme if for each i: X i = { v j in Adj(v i ) j>i } is a clique. 15

16 Definition: The following are equivalent: 1. G is chordal 2. The edges of G can be directed acyclically so that every pair of converging arrows comes from two adjacent vertices. 3. G has a perfect vertex elimination scheme. 4. There is a tree T with maximal cliques of G as vertices. Any two cliques containing v are either adjacent in T or connected by a path of cliques that contain v. 16

17 By lemma 1, G has a simplicial vertex. The subgraph induced after removing this vertex is also chordal. By induction, there is an elimination order. 17

18 Let us assume a cycle of length 4 or greater. Let x be the first vertex on the elimination scheme of this cycle. x is simplicial at this point. Two vertices adjacent to x from the cycle are also adjacent between themselves. 18

19 3. All vertices of G can be deleted by arranging them in separate piles, one for each clique, and then repeatedly applying the following two operations: Delete a vertex that occurs in only one pile. Delete a pile contained in another pile 19

20 A B C D E B A C B C D D E 20

21 A B C A B A C D E B D C D D E 21

22 Definition: The following are equivalent: 1. G is chordal 2. The edges of G can be directed acyclically so that every pair of converging arrows comes from two adjacent vertices. 3. G has a perfect vertex elimination scheme. 4. There is a tree T with maximal cliques of G as vertices. Any two cliques containing v are connected by a path of cliques that contain v. 22

23 a b a c C 1 b C 1 c C 2 C 2 C 3 b c e d d b C 3 e d 23

24 C 1 a b a C 1 b C 2 C 4 C 2 C 4 a b c d c d C 3 c d C 3 24

25 Lemma 2: A vertex is simplicial iff it belongs to exactly one maximal clique. Let G =(V,E) be a chordal graph and V =k Proof by induction: Assume the claim holds for all graphs of size < k. By Lemma 1 G has a simplicial vertex v. By Lemma 2 v belongs to exactly one clique C. 25

26 For G =(V\{v},E ) there exists a tree T which satisfies the claim. Split into two cases: C = C {v} is maximal in G. Add v to C to build T from T. T is a Clique-Tree as needed. b e C 2 a C 1 c d b a c C 1 b e d C 2 26

27 C = C {v} is not maximal in G. There is a maximal clique P in G such that C P. Add C to T to build T. T is a Clique-Tree as needed. a C 2 b a C 1 a b C 2 c C 1 d c d d 27

28 Let T be a Clique-Tree of graph G=(V,E) and V =k. Proof by induction: Assume the claim holds for all graphs of size < k. Let L be a leaf in T, and P its parent in T. Let v L\P (exists from maximality). v cannot be in any other clique. by Lemma 2, v is simplicial. 28

29 Let T be T with v removed. T is a Clique- Tree. By assumption G =(V\{v},E ) is chordal. By part 3 of Theorem 7, there exists σ, a Perfect Elimination Order for G. Let σ = [v, σ ] be a perfect elimination order for G. By the converse of part 3, we can conclude that G is chordal 29