EE96 Pat. Recog. II: Introduction to Graphical Models University of Washington Spring 00 Dept. of Electrical Engineering Lecture : May 0, 00 Lecturer: Jeff Bilmes Scribe: Arindam Mandal, David Palmer(000). Graph Theory (review from last time) (decomposable) (chordal) (every min. α β-separation is complete) (there exists a junction tree of cliques) We want to try to get any graph into this family of equivalences, so that we can then construct a junction tree from the graph. Next time we will relate this to independence and inference. Running Intersection Property (RIP): A sequence (C, C,..., C n )of sets has the RIP if j, S j C i for some i < j, where S j = H j C j (the separator set) H j = C C... C j (the history up to and including j) R j = C j \ H j (the residual or innovation) The sequence of cliques formed by a well-ordering (e.g., breadth-first or depth-first search order) of a junction tree has the RIP. Intuitively, S j = H j C j (by definition). Then using the definition of a junction tree, C pj C j C pj for any p j < j which is a neighbor of node j in the junction tree (according to the search order). However, since p j is a neighbor of j, C pj contains everything in H j. Hence, S j = C pj C j S j C pj. Example: Cpj Cj ( ) Figure.: C j C k C pj ; C j k<j C k C pj -
Lecture : May 0, 00 -. Constructing a Junction Tree Algorithm: constructing a junction tree (see Fig. ) input: C, C,..., C p (the cliques of a chorded graph satisfying RIP) step : associate each node in the graph with each clique step : for j =,..., p add an edge between C i and C j for some i < j such that S j C i (S j = H j C j ) Ci Sj Cj v v v Figure.: S j C i Intuition: Why does it work? For a junction tree, for any node v, cliques containing v must induce a connected tree, and there are two cases: Case : v S j, and we know that S j C i and S j C j, which means that v C i and v C j Case : v S j, which means that v R j = C j \ H j Note: node j is a candidate for some later node that has v. See [SL96] for the formal proof.. More Graph Theory Basics Definition: Perfect DAG - a DAG is perfect if the parents of every node form a complete set. A Perfect DAG NOT a Perfect DAG Figure.: Examples of perfect and non-perfect DAGS Definition: Perfect Numbering -a node numbering (v, v,..., v k ) of an UGM is perfect if all lower numbered neighbors of every node are complete, i.e., ne(v j ) {v, v,..., v j } is complete. Examples are shown in Fig...
Lecture : May 0, 00 - B D E A C Figure.: ABCDEF is a perfect numbering but ABCEFD is not. F Figure.: No perfect numbering possible Note: Any ordering of a complete graph is perfect. Perfect DAG moralized graph. Given a well-ordered (e.g., topological order) (v, v,..., v k ) perfect DAG G, its undirected version G is chordal with v, v,..., v k a perfect numbering Proof: G is decomposable: j, (W j, V j, S j ) form a decomposition, where: W j = cl (v j ) V j (note that cl denotes closure w.r.t. the undirected graph) S j = W j V j is a separator V j = {v, v,..., v k } (W j, V j, S j ) = (cl (V j ) {v, v,..., v k }, {v, v,..., v j, }, cl (V j ) {v, v,..., v j }) Additionally, cl (V j ) = V j pa(vj ) since G is a DAG. Also, pa(v j ) are complete because G is a perfect DAG. Hence cl (V j ) A is complete where A V. Thus,in (W j, V j, S j ) The first term cl (V j ) {v, v,..., v k } is complete and therefore decomposable. The second term {v, v,..., v j } is decomposable by induction. The third term cl (V j ) {v, v,..., v j } is complete. Suppose we are given a perfect numbering (v, v,..., v k ) of an UGM. We can produce a perfect DAG by pointing lower numbered vertices to higher numbered vertices. This implies that the UGM is chordal. 6 6 Figure.6: Producing a perfect DAG from an UGM. (from [SL96].7) An UGM is chordal iff it admits a perfect numbering.
Lecture : May 0, 00 - Summary of equivalent conditions on graphs: decomposable chordal every minimal (α, β)-separator is complete. there is a junction tree of cliques the graph admits a perfect numbering But how do we tell if a graph is chordal? Two important definitions: Definition: simplicial - α is simplicial if bd(α) is complete (all neighbors are joined in UGM) α α Not simplicial Simplicial Figure.7: Examples of simplicial and non-simplicial nodes. Definition: eliminatable - a graph G is said to be eliminatable if all nodes can be successfully removed by the elimination algorithm without adding any edges Remember elimination algorithm (from first lectures):. choose an ordering. pick nodes in order, join all neighbors. remove node, repeat Note: Running elimination on a perfect numbering produces a chordal graph. Any eliminatable graph is chordal. Proof: By induction Base case: node is obvious Assume true for N, show N+ Given an eliminatable graph with N+ nodes, we know that it has at least one simplicial node A G \ {A} is triangulated by induction (i.e. since G \ {A} is eliminatable also) but A doesn t produce any chordless cycles because it is simplicial G is chordal Note: It ll be proved later that any chordal is also eliminatable. But how do we know if a graph is eliminatable? We run the elimination algorithm many times for all possible orderings to realize an elimination ordering for which no edges are added which would imply a chordal graph. However, this would be prohibitively expensive. Alternatively, we could start by finding a simplicial node and try to eliminate it. If we can eliminate the node, it implies that we have found an elimination order that produces a perfect numbering of
Lecture : May 0, 00 - nodes. If we cannot eliminate the simplicial node, it implies the graph is not chordal. This alternate procedure is also computationally expensive. Thus an efficient procedure is needed for testing the chordality of a graph.. Maximum Cardinality Search Maximum Cardinality Search (MCS) is an algorithm that tests an UGM for chordality in O( V + E ) time. First published by Tarjan and Yannakakis [TY8], it is a very important algorithm for Bayesian Networks. Algorithm: Maximum Cardinality Search Input: undirected graph G Outputs: ) whether G is chordal or not ) a perfect numbering for G, if it is chordal set output = chordal set i = set L = set c(v) = 0 v V while (L V ) end U = V \ L v = argmax v U v i = v ;; i is a counter ;; L is the set of previously labeled vertices c(v) ;; array counting number of previously labeled neighbors of each node v ;; current set of unlabeled vertices ;;find, from the unlabeled nodes, one with the greatest number of previously labeled neighbors, breaking ties arbitrarily. Note that the name of the algorithm is derived from the fact that the next node we choose to be label is the one with the maximum cardiality of the set of previously labeled neighbors. ;; rename so v gets the name v i if ne(v i ) L is not complete ;; output = not chordal π vi. = ne(vi ) L ;; set of prev. labeled neighbors of v i c(w) = c(w) +, w (ne(v i ) U) L = L {v i } i = i + if set of previously labeled neighbors is not complete, not chordal. ;; increment counts of all unlabeled neighbors of this newly labeled node. Later we pick one with many neighbors. The idea is that we maximize the cardinality of the set of previously labeled neighbors, i.e., ne(v i ) L. The cardinality is increased by increasing c(w), so next node we choose will have many labeled neighbors. We can prove that if ever such a set is not complete, then it is not chordal. Thus, in a nutshell, the algorithm chooses and labels a node that has the greatest number of previously labeled neighbors. Table. lists the values of the variables i, c, L and U used in the MCS algorithm for the graph shown in Fig. 8.
Lecture : May 0, 00-6 Table.: MCS algorithm for graph of Fig. 8 i c 6 L U 0 0 0 0 0 0 :6 0 0 0 0 :6 0 : :6 0 : :6 0 :,,6 6 0 :,:6 x 6 x x6 x x x Figure.8: Node ordering produced by MCS algorithm: (,,,6,,). Note: The first node in the ordering is chosen arbitrarily, since c(v) = 0 for all nodes during the first iteration. The resulting labels (v, v,..., v k ) constitute a vertex ordering. This ordering will be perfect (if the graph is chordal) since for any node v, π v will be complete.. Finding Cliques Satisfying RIP Key idea: given the ordering produced by MCS, we can find a clique ordering satisfying RIP. Algorithm: Find cliques in RIP order / Ladder-based clique construction start with (v, v,..., v k ) in RIP order (from MCS) π. = π vi ;; cardinality of sets of previously labeled neighbors define i as ladder node if π i π i+ ORi = k. = j th ladder node λ j c j. = {λj } π λi ;;λ j and set of previously run nodes from MCS c j are the cliques of G and clique ordering (c, c,...) satisfies the RIP. ;; cardinality is decreasing or staying the same as i increases.
Lecture : May 0, 00-7 Proof: see [CDLS99]. Table. shows the result of running the ladder-based clique construction algorithm on the node ordering produced by the MCS algorithm. Table.: Ladder-based Clique Construction. Labeling 6 π vi {} {,} {,} {,} {} π i 0 Ladder? N N Y Y Y Y The clique ordering produced by this algorithm is: {,,}, {,,}, {6,,}, {,}. Note: We can use these cliques to construct a junction tree using the procedure outlined in section.. References [TY8] R.E. TARJAN, M. YANNAKAKIS, Simple Linear-Time Algorithms to Test Chordality of Graphs, Test Acyclicity of Hypergraphs, and Selectively Reduce Acyclic Hypergraphs, SIAM Journal on Computing, Volume, Number, August 98, pp. 66-79. [SL96] S. L. LAURITZEN, Graphical Models, Oxford Science Publications, 996. [CDLS99] R. G. COWELL, A. P. DAWID, S. L. LAURITZEN, and D. J. SPIEGELHALTER, Probabilistic Networks and Expert Systems, Springer, 999.