Bayesian Networks. 2. Separation in Graphs

Size: px

Start display at page:

Download "Bayesian Networks. 2. Separation in Graphs"

Melanie Poole
6 years ago
Views:

1 ayesian Netwks ayesian Netwks 2. Separation in raphs Lars Schmidt-Thieme Infmation Systems and Machine Learning Lab (ISMLL) Institute f usiness conomics and Infmation Systems & Institute f omputer Science University of Hildesheim ourse on ayesian Netwks, summer term /30 ayesian Netwks 1. Separation in Undirected raphs 2. Properties of Ternary Relations on Sets 3. Separation in irected raphs ourse on ayesian Netwks, summer term /30

2 ayesian Netwks / 1. Separation in Undirected raphs raphs efinition 1. Let V be any set and P 2 (V ) := {{x, y} x, y V } be a subset of sets of undered pairs of V. Then := (V, ) is called an undirected graph. The elements of V are called vertices nodes, the elements of edges. Let e = {x, y} be an edge, then we call the vertices x, y incident to the edge e. We call two vertices x, y V adjacent, if there is an edge {x, y}. The set of all vertices adjacent with a given vertex x V is called its fan: fan(x) := {y V {x, y} } H igure 1: xample graph. ourse on ayesian Netwks, summer term /30 ayesian Netwks / 1. Separation in Undirected raphs Paths on graphs efinition 2. Let V be a set. We call V := i N V i the set of finite sequences in V. The length of a sequence s V is denoted by s. Let = (V, ) be a graph. We call := V := {p V {p i, p i+1 }, i = 1,..., p 1} the set of paths on. ny contiguous subsequence of a path p is called a subpath of p, i.e. any path (p i, p i+1,..., p j ) with 1 i j n. The subpath (p 2, p 3,..., p n 1 ) is called the interi of p. path of length p 2 is called proper. H igure 2: xample graph. The sequences (,,, H) (,,, ) ( ) are paths on, but the sequences (,,, ) (, H,, ) are not. ourse on ayesian Netwks, summer term /30

3 ayesian Netwks / 1. Separation in Undirected raphs Separation in graphs (u-separation) efinition 3. Let := (V, ) be a graph. Let Z V be a subset of vertices. We say, two vertices x, y V are u- separated by Z in, if every path from x to y contains some vertex of Z ( p : p 1 = x, p p = y i {1,..., n} : p i Z). We write I (X, Y Z) f the statement, that X and Y are u-separated by Z in. I is called u-separation relation in. Let X, Y, Z V be three disjoint subsets of vertices. We say, the vertices X and Y are u-separated by Z in, if every path from any vertex from X to any vertex from Y is separated by Z, i.e., contains some vertex of Z. igure 3: p. 179]. (a) I(, I I) xample f u-separation [H97, (b) ourse on ayesian Netwks, summer term /30 ayesian Netwks / 1. Separation in Undirected raphs Separation in graphs (u-separation) (a) I(, I I) (b)(,ii) (c) I( {, }, {, H} I {, }) (d)({, ) (a) I(, I I) (a) I(, I I) (b)(,ii) (b)(,ii) (c) I( {, }, {, H} I {, }) (d)({, ), {,H} (c) I( {, }, {, H} I {, }) (d)({, ), {,H} I {,I}), }, {, igure H} I {, 4: }) Me examples (d)({, f ), u-separation {,H} I {,I})[H97, p. 179]. ourse on ayesian Netwks, summer term /30

4 ayesian Netwks / 1. Separation in Undirected raphs Properties of u-separation / no chardality u-separation the chdality property does not hold (in general). (e) Weak Transitivity & :::} (d) hdality igure 5: ounterexample f chdality in undirected graphs (u-separation) [H97, p. 189]. ourse on ayesian Netwks, summer term /30 ayesian Netwks / 1. Separation in Undirected raphs Properties of u-separation symmetry decomposition composition strong union weak union contraction intersection strong transitivity weak transitivity chdality relation u-separation ourse on ayesian Netwks, summer term /30

5 ayesian Netwks / 1. Separation in Undirected raphs hecking u-separation To test, if f a given graph = (V, ) two given sets X, Y V of vertices are u-separated by a third given set Z V of vertices, we may use standard breadth-first search to compute all vertices that can be reached from X (see, e.g., [OW02], [LR90]). 1 breadth-first search(, X) : 2 bder := X 3 reached := 4 while bder do 5 reached := reached bder 6 bder := fan (bder) \ reached 7 od 8 return reached checking u-separation we have to tweak the algithm 1. not to add vertices from Z to the bder and 2. to stop if a vertex of Y has been reached. 1 check-u-separation(, X, Y, Z) : 2 bder := X 3 reached := 4 while bder do 5 reached := reached bder 6 bder := fan (bder) \ reached \ Z 7 if bder Y 8 return false 9 fi 10 od 11 return true igure 6: readth-first search algithm f enumerating all vertices reachable from X. igure 7: readth-first search algithm f checking u-separation of X and Y by Z. ourse on ayesian Netwks, summer term /30 ayesian Netwks 1. Separation in Undirected raphs 2. Properties of Ternary Relations on Sets 3. Separation in irected raphs ourse on ayesian Netwks, summer term /30

6 ayesian Netwks / 2. Properties of Ternary Relations on Sets Symmetry efinition 4. Let V be any set and I a ternary relation on P(V ), i.e., I (P(V )) 3. I is called symmetric, if I(X, Y Z) I(Y, X Z) uilding Probabilistic Models (a) Symmetry igure 8: xamples f symmetry [H97, p. 186]. (b) ecomposition ourse on ayesian Netwks, summer term /30 ayesian Netwks / 2. Properties of Ternary Relations on Sets =? ecomposition and omposition (c) Weak Union efinition 5. I is called (right-)decomposable, if I(X, Y Z) I(X, Y Z) f any Y Y I is called (right-)composable, if I(X, Y Z) and I(X, Y Z) I(X, Y(d) ontraction Y Z) uilding Probabilistic Models =? (e) Intersection PTnTTl' ~ ~ " (a) Symmetry (b) ecomposition igure 9: xamples f decomposition [H97, p. 186]. ourse on ayesian Netwks, summer term /30

7 ayesian Netwks / 2. Properties of Ternary Relations on Sets Union efinition 6. I is called strongly unionable, if I(X, Y Z) I(X, Y Z Z ) uilding Probabilistic Models f all Z disjunct with X, Y I is called (right-)weakly unionable, if I(X, Y Z) I(X, Y (Y \ Y ) Z) f any Y Y (a) Symmetry (b) ecomposition uilding Probabilistic Models =? (a) Strang Union 6 5. uilding Probabilistic Models (b) Strang Transitivity (a) Symmetry (c) Weak Union igure 10: xamples f a) strong union and b) weak union [H97, p. 186,189]. ourse on ayesian Netwks, summer term /30 ayesian Netwks / 2. Properties of Ternary Relations on Sets ontraction and Intersection =? (b) ecomposition (a) Symmetry efinition 7. I is (e) called Weak Transitivity (right-)contractable, (d) ontraction (b) ecomposition (c) Weak if Union I(X, Y Z) and I(X, Y Y Z) I(X, Y Y Z) =? I is called (right-)intersectable, if & :::} =? I(X, Y Y Z) and I(X, Y Y Z) I(X, Y Y Z) (c) Weak Union PTnTTl' ~ ~ " (e) Intersection (d) ontraction (d) hdality =? (d) ontraction PTnTTl' ~ ~ " (e) Intersection igure 11: xamples f a) contraction and b) intersection [H97, p. 186]. ourse on ayesian Netwks, summer term /30

8 ayesian Netwks / 2. Properties of Ternary Relations on Sets Transitivity efinition 8. I is called strongly transitive, if I(X, Y Z) I(X, {v} Z) I({v}, Y Z) v V \ Z I is called weakly transitive, if I(X, Y Z) and I(X, Y Z {v}) I(X, {v} Z) I({v}, Y Z) v V \Z (a) Strang Union (b) Strang Transitivity (e) Weak Transitivity (b) Strang Transitivity (e) Weak Transitivity igure 12: xamples f a) strong transitivity and b) weak transitivity. [H97, p. 189] ourse on ayesian Netwks, summer term /30 ayesian Netwks / 2. Properties of Ternary Relations on Sets hdality efinition 9. I is called chdal, if & :::} I({a}, {c} {b, d}) and I({b}, {d} {a, c}) I({a}, {c} {b}) I({a}, {c} {d}) (d) hdality :::} a a a a b d & b d b d b d c c igure 13: xample f chdality. c c (d) hdality ourse on ayesian Netwks, summer term /30

9 ayesian Netwks 1. Separation in Undirected raphs 2. Properties of Ternary Relations on Sets 3. Separation in irected raphs ourse on ayesian Netwks, summer term /30 ayesian Netwks / 3. Separation in irected raphs irected graphs efinition 10. Let V be any set and V V be a subset of sets of dered pairs of V. Then := (V, ) is called a directed graph. The elements of V are called vertices nodes, the elements of edges. The set of all vertices with an edge to a given vertex x V is called its fanin: fanin(x) := {y V (y, x) } I J H Let e = (x, y) be an edge, then we call the vertices x, y incident to the edge e. We call two vertices x, y V adjacent, if there is an edge (x, y) (y, x). The set of all vertices with an edge from a given vertex x V is called its fanout: fanout(x) := {y V (x, y) } K igure 14: anin (ange) and fanout (green) of a node (blue). ourse on ayesian Netwks, summer term /30 L

10 ayesian Netwks / 3. Separation in irected raphs Paths on directed graphs efinition 11. Let = (V, ) be a directed graph. We call := V := {p V (p i, p i+1 ), i = 1,..., p 1} the set of paths on. two vertices x, y V we denote by [x,y] := {p V p 1 = x, p p = y} the set of paths from x to y. The notions of subpath, interi, and proper path carry over to directed graphs. igure 15: xample f a cycle. proper path p = (p 1,..., p n ) with p 1 = p n is called cyclic. path without cyclic subpath is called a simple path. graph without a cyclic path is called directed acyclig graph (). ourse on ayesian Netwks, summer term /30 ayesian Netwks / 3. Separation in irected raphs Paths on directed graphs (2/2) efinition 12. a vertices of the fanout are also called children child(x) := fanout(x) := {y V (x, y) } and the vertices of the fanin parents: pa(x) := fanin(x) := {y V (y, x) } Vertices y with a proper path from y to x are called ancests of x: anc(x) := {y V p : p 2, p 1 = y, p p = x} Vertices y with a proper path from x to y are called descendents of x: desc(x) := {y V p : p 2, p 1 = x, p p = y} Vertices that are not a descendent of x are called nondescendents of x. I K igure 16: Parents/anin (ange) and additional ancests (light ange), children/fanout (green) and additional descendants (light green) of a node (blue). ourse on ayesian Netwks, summer term /30 J H L

11 ayesian Netwks / 3. Separation in irected raphs hains efinition 13. Let := (V, ) be a directed graph. We can construct an undirected skeleton u() := (V, u()) of by dropping the directions of the edges: u() := {{x, y} (x, y) (y, x) } The paths on u() are called chains of : := u() i.e., a chain is a sequence of vertices that are linked by a fward a backward edge. If we want to stress the directions of the linking edges, we denote a chain p = (p 1,..., p n ) by igure 17: hain (,,,, ) on directed graph and path on undirected skeleton. p 1 p 2 p 3 p n 1 p n The notions of length, subchain, interi and proper carry over from undirected paths to chains. ourse on ayesian Netwks, summer term /30 ayesian Netwks / 3. Separation in irected raphs locked chains efinition 14. Let := (V, ) be a directed graph. We call a chain p 1 p 2 p 3 a head-to-head meeting. Let Z V be a subset of vertices. Then a chain p is called blocked at position i by Z, if f its subchain (p i 1, p i, p i+1 ) there is { p i Z, if not p i 1 p i p i+1 p i Z anc(z), else igure 18: hain (,,,, ) is blocked by Z = {} at 2. ourse on ayesian Netwks, summer term /30

12 ayesian Netwks / 3. Separation in irected raphs locked chains / me examples igure 19: hain (,,,, ) is blocked by Z = at 3. igure 20: hain (,,,, ) is not blocked by Z = {} at 3. ourse on ayesian Netwks, summer term /30 ayesian Netwks / 3. Separation in irected raphs locked chains / rationale The notion of blocking is choosen in a way so that chains model "flow of causal influence" through a causal netwk where the states of the vertices Z are already know. 2) iverging connection / common cause: flu flu nausea fever nausea fever 1) Serial connection / intermediate cause: flu flu 3) onverging connection / common effect: flu salmonella flu salmonella nausea nausea nausea nausea pal pal pal pal Models "discounting" [Nea03, p. 51]. ourse on ayesian Netwks, summer term /30

13 ayesian Netwks / 3. Separation in irected raphs The mal graph efinition 15. Let := (V, ) be a. s the mal graph of we denote the undirected skeleton graph of plus additional edges between each two parents of a vertex, i.e. mal() := (V, ) with := u() {{x, y} z V : x, y pa(z)} igure 22: and its mal graph. ourse on ayesian Netwks, summer term /30 ayesian Netwks / 3. Separation in irected raphs Separation in s (d-separation) Let := (V, ) be a. Let X, Y, Z V be three disjoint subsets of vertices. We say, the vertices X and Y are separated by Z in, if (i) every chain from any vertex from X to any vertex from Y is blocked by Z equivalently (ii) X and Y are u-separated by Z in the mal graph of the ancestral hull of X Y Z. We write I (X, Y Z) f the statement, that X and Y are separated by Z in. igure 23: re the vertices and separated by in? ourse on ayesian Netwks, summer term /30

14 ayesian Netwks / 3. Separation in irected raphs Separation in s (d-separation) / examples igure 24: and are separated by in. ourse on ayesian Netwks, summer term /30 ayesian Netwks / 3. Separation in irected raphs Separation in s (d-separation) / me examples igure 25: and are not separated by {, } in. ourse on ayesian Netwks, summer term /30

15 ayesian Netwks / 3. Separation in irected raphs hecking d-separation To test, if f a given graph = (V, ) two given sets X, Y V of vertices are d-separated by a third given set Z V of vertices, we may build the mal graph of the ancestral hull and apply the u-separation criterion. 1 check-d-separation(, X, Y, Z) : 2 := mal(anc (X Y Z)) 3 return check-u-separation(, X, Y, Z) igure 26: lgithm f checking d-separation via u-separation in the mal graph. drawback of this algithm is that we have to rebuild the mal graph of the ancestral hull whenever X Y changes. ourse on ayesian Netwks, summer term /30 ayesian Netwks / 3. Separation in irected raphs hecking d-separation Instead of constructing a mal graph, we can modify a breadth-first search f chains to find all vertices not d- separated from X by Z in. The breadth-first search must not hop over head-to-head meetings with the middle vertex not in Z n having an descendent in Z. 1 enumerate-d-separation( = (V, ), X, Z) : 2 bderward := 3 bderackward := X \ Z 4 reached := 5 while bderward bderackward do 6 reached := reached (bderward \ Z) bderackward x y z fanout(y) if y Z anc(z) z fanin(y) igure 27: Restricted breadth-first search of non-blocked chains. 7 bderward := fanout (bderackward (bderward \ Z)) \ reached 8 bderackward := fanin (bderackward (bderward (Z anc(z)))) \ Z \ reached 9 od 10 return V \ reached igure 28: lgithm f enumerating all vertices d-separated from X by Z in via restricted breadth-first search (see [Nea03, p ] f another fmulation). ourse on ayesian Netwks, summer term /30

16 ayesian Netwks / 3. Separation in irected raphs Properties of d-separation / no strong union d-separation the strong union property does not hold. I is called strongly unionable, if I(X, Y Z) I(X, Y Z Z ) f all Z disjunct with X, Y X Y X Y Z Z (a) Strang Union igure 29: xample f strong union in undirected graphs (u-separation) [H97, p. 189]. igure 30: ounterexample f strong unions in s (d-separation). (b) Strang Transitivity ourse on ayesian Netwks, summer term /30 (e) Weak Transitivity ayesian Netwks / 3. Separation in irected raphs Properties of d-separation / no strong transitivity d-separation the strong transitivity property does not hold. I is called strongly transitive, if & I(X, Y Z) I(X, {v} Z) I({v}, Y Z) :::} v V \ Z x (d) hdality z y ion (b) Strang Transitivity igure 31: xample f strong transitivity in undirected graphs (u-separation) [H97, p. 189]. igure 32: ounterexample f strong transitivity in s (d-separation). ourse on ayesian Netwks, summer term /30 v

17 ayesian Netwks / 3. Separation in irected raphs Properties of d-separation symmetry decomposition composition strong union weak union contraction intersection strong transitivity weak transitivity chdality relation u-separation d-separation ourse on ayesian Netwks, summer term /30 ayesian Netwks / 3. Separation in irected raphs References [H97] nrique astillo, José Manuel utiérrez, and li S. Hadi. xpert Systems and Probabilistic Netwk Models. Springer, New Yk, [LR90] Thomas H. men, harles. Leiserson, and Ronald L. Rivest. lgithms. MIT Press, ambridge, Massachusetts, Introduction to [Nea03] Richard. Neapolitan. Learning ayesian Netwks. Prentice Hall, [OW02] [VP90] Thomas Ottmann and Peter Widmayer. lgithmen und atenstrukturen. Spektrum Verlag, Heidelberg, Thomas Verma and Judea Pearl. ausal netwks: semantics and expressiveness. In Ross. Shachter, Tod S. Levitt, Laveen N. Kanal, and John. Lemmer, edits, Uncertainty in rtificial Intelligence 4, pages Nth-Holland, msterdam, ourse on ayesian Netwks, summer term /30

Machine Learning Lecture 16

ourse Outline Machine Learning Lecture 16 undamentals (2 weeks) ayes ecision Theory Probability ensity stimation Undirected raphical Models & Inference 28.06.2016 iscriminative pproaches (5 weeks) Linear