Learning DAGs from observational data

Transcription

1 Learning DAGs from observational data

2 General overview Introduction DAGs and conditional independence DAGs and causal effects Learning DAGs from observational data IDA algorithm Further problems 2

3 What can we do when the DAG is unknown? Knowing the DAG is unrealistic in high-dimensional settings. So we assume that the data come from an unknown DAG. 3

4 What can we do when the DAG is unknown? Knowing the DAG is unrealistic in high-dimensional settings. So we assume that the data come from an unknown DAG. A DAG encodes conditional independence relationships. So given all conditional independence relationships in the observational distribution, can we learn the DAG? 3

5 What can we do when the DAG is unknown? Knowing the DAG is unrealistic in high-dimensional settings. So we assume that the data come from an unknown DAG. A DAG encodes conditional independence relationships. So given all conditional independence relationships in the observational distribution, can we learn the DAG? Almost... several DAGs can encode the same conditional independence relationships. They are Markov equivalent. 3

6 What can we do when the DAG is unknown? Knowing the DAG is unrealistic in high-dimensional settings. So we assume that the data come from an unknown DAG. A DAG encodes conditional independence relationships. So given all conditional independence relationships in the observational distribution, can we learn the DAG? Almost... several DAGs can encode the same conditional independence relationships. They are Markov equivalent. Example: X 1 X 3 X 1 X 3 X 2 X 1 X 2 X 3 false true X 1 X 2 X 3 false true X 1 X 2 X 3 false true X 1 X 2 X 3 true false 3

7 What can we do when the DAG is unknown? Knowing the DAG is unrealistic in high-dimensional settings. So we assume that the data come from an unknown DAG. A DAG encodes conditional independence relationships. So given all conditional independence relationships in the observational distribution, can we learn the DAG? Almost... several DAGs can encode the same conditional independence relationships. They are Markov equivalent. Example: X 1 X 3 X 1 X 3 X 2 X 1 X 2 X 3 false true X 1 X 2 X 3 false true X 1 X 2 X 3 false true X 1 X 2 X 3 true false no v-structure v-structure 3

8 What can we do when the DAG is unknown? Knowing the DAG is unrealistic in high-dimensional settings. So we assume that the data come from an unknown DAG. A DAG encodes conditional independence relationships. So given all conditional independence relationships in the observational distribution, can we learn the DAG? Almost... several DAGs can encode the same conditional independence relationships. They are Markov equivalent. Example: X 1 X 3 X 1 X 3 X 2 X 1 X 2 X 3 false true X 1 X 2 X 3 false true X 1 X 2 X 3 false true X 1 X 2 X 3 true false no v-structure v-structure A v-structure is a triple i j k where i and k are not adjacent 3

9 Markov equivalence class All DAGs in a Markov equivalence class have the same skeleton and the same v-structures (Verma and Pearl, 1990) 4

10 Markov equivalence class All DAGs in a Markov equivalence class have the same skeleton and the same v-structures (Verma and Pearl, 1990) They can be uniquely represented by a CPDAG: edge between X and Y iff X and Y are d-connected given S for all subsets S of the remaining variables (edges are stronger than in CIGs/Gaussian Graphical Models) X Y iff X Y in all DAGs in the equivalence class (direct causal effect) X Y iff there is a DAG in the equivalence class with X Y and one with X Y (unidentifiable orientations) 4

11 Markov equivalence class All DAGs in a Markov equivalence class have the same skeleton and the same v-structures (Verma and Pearl, 1990) They can be uniquely represented by a CPDAG: edge between X and Y iff X and Y are d-connected given S for all subsets S of the remaining variables (edges are stronger than in CIGs/Gaussian Graphical Models) X Y iff X Y in all DAGs in the equivalence class (direct causal effect) X Y iff there is a DAG in the equivalence class with X Y and one with X Y (unidentifiable orientations) Example: CPDAG 4

12 Markov equivalence class All DAGs in a Markov equivalence class have the same skeleton and the same v-structures (Verma and Pearl, 1990) They can be uniquely represented by a CPDAG: edge between X and Y iff X and Y are d-connected given S for all subsets S of the remaining variables (edges are stronger than in CIGs/Gaussian Graphical Models) X Y iff X Y in all DAGs in the equivalence class (direct causal effect) X Y iff there is a DAG in the equivalence class with X Y and one with X Y (unidentifiable orientations) Example: CPDAG DAG 1 DAG 2 DAG 3 DAG 4 4

13 Markov equivalence class All DAGs in a Markov equivalence class have the same skeleton and the same v-structures (Verma and Pearl, 1990) They can be uniquely represented by a CPDAG: edge between X and Y iff X and Y are d-connected given S for all subsets S of the remaining variables (edges are stronger than in CIGs/Gaussian Graphical Models) X Y iff X Y in all DAGs in the equivalence class (direct causal effect) X Y iff there is a DAG in the equivalence class with X Y and one with X Y (unidentifiable orientations) Example: CPDAG DAG 1 DAG 2 DAG 3 DAG 4 4

14 Markov equivalence class All DAGs in a Markov equivalence class have the same skeleton and the same v-structures (Verma and Pearl, 1990) They can be uniquely represented by a CPDAG: edge between X and Y iff X and Y are d-connected given S for all subsets S of the remaining variables (edges are stronger than in CIGs/Gaussian Graphical Models) X Y iff X Y in all DAGs in the equivalence class (direct causal effect) X Y iff there is a DAG in the equivalence class with X Y and one with X Y (unidentifiable orientations) Example: CPDAG DAG 1 DAG 2 DAG 3 DAG 4 4

15 Causal structure learning Learning (Markov equivalence classes of) DAGs is challenging. Main methods: Score-based methods: e.g. Greedy Equivalence Search (Chickering, 2002) Constraint-based methods: e.g. PC algorithm (Spirtes et al, 2000) 5

16 Causal structure learning Learning (Markov equivalence classes of) DAGs is challenging. Main methods: Score-based methods: e.g. Greedy Equivalence Search (Chickering, 2002) Constraint-based methods: e.g. PC algorithm (Spirtes et al, 2000) Fast Consistent for high-dimensional sparse graphs (Kalisch & Bühlmann, 2007) 5

17 Causal structure learning Learning (Markov equivalence classes of) DAGs is challenging. Main methods: Score-based methods: e.g. Greedy Equivalence Search (Chickering, 2002) Constraint-based methods: e.g. PC algorithm (Spirtes et al, 2000) Fast Consistent for high-dimensional sparse graphs (Kalisch & Bühlmann, 2007) Restricted structural equation models: e.g. LiNGAM (Shimizu et al, 2006; Tübingen group) DAG is identifiable! 5

18 Faithfulness Constraint-based methods require a faithfulness assumption: the conditional independencies in the distribution exactly equal the ones encoded in the DAG via d-separation Example of a distribution that is not faithful to its generating DAG: 1 X 2 X 1 ɛ 1-1 X 2 X 1 +ɛ 2 X 1 1 X 3 X 3 X 1 X 2 +ɛ 3 6

19 Faithfulness Constraint-based methods require a faithfulness assumption: the conditional independencies in the distribution exactly equal the ones encoded in the DAG via d-separation Example of a distribution that is not faithful to its generating DAG: 1 X 2 X 1 ɛ 1-1 X 2 X 1 +ɛ 2 X 1 1 X 3 X 3 X 1 X 2 +ɛ 3 X 1 and X 3 are not d-separated by the empty set 6

20 Faithfulness Constraint-based methods require a faithfulness assumption: the conditional independencies in the distribution exactly equal the ones encoded in the DAG via d-separation Example of a distribution that is not faithful to its generating DAG: 1 X 2 X 1 ɛ 1-1 X 2 X 1 +ɛ 2 X 1 1 X 3 X 3 X 1 X 2 +ɛ 3 X 1 and X 3 are not d-separated by the empty set But: X 1 = ɛ 1, X 2 = ɛ 1 +ɛ 2, X 3 = ɛ 1 (ɛ 1 +ɛ 2 )+ɛ 3 = ɛ 2 +ɛ 3. Hence, X 1 and X 3 are independent. 6

21 Skeleton of a DAG Under the faithfulness assumption: There is an edge between X i and X j in the DAG if and only if X i and X j are dependent given every subset of the remaining variables 7

22 Skeleton of a DAG Under the faithfulness assumption: There is an edge between X i and X j in the DAG if and only if X i and X j are dependent given every subset of the remaining variables This means that the skeleton of a DAG is determined uniquely by conditional independence relationships 7

23 Skeleton of a DAG Under the faithfulness assumption: There is an edge between X i and X j in the DAG if and only if X i and X j are dependent given every subset of the remaining variables This means that the skeleton of a DAG is determined uniquely by conditional independence relationships But the directions of the edges are generally not uniquely determined 7

24 PC algorithm Assuming faithfulness, a CPDAG can be estimated by the PC-algorithm of Peter Spirtes and Clark Glymour (2000): Determine the skeleton Determine the v-structures Direct as many of the remaining edges as possible 8

25 PC algorithm Assuming faithfulness, a CPDAG can be estimated by the PC-algorithm of Peter Spirtes and Clark Glymour (2000): Determine the skeleton No edge between X i and X j X i X j S for some subset S of the remaining variables X i X j S for some subset S of adj(x i ) or of adj(x j ) Start with the complete graph For k = 0,1,...: Consider all pairs of adjacent vertices (X i,x j ), and remove edge if they are conditionally independent given some subset of size k of adj(x i ) or of adj(x j ) Determine the v-structures Direct as many of the remaining edges as possible 8

26 PC algorithm - oracle version Assume faithfulness and an oracle that tells us whether or not X Y S for any triple (X,Y,S). Then a CPDAG can be estimated by the PC-algorithm of Peter Spirtes and Clark Glymour (2000): Determine the skeleton Determine the v-structures Direct as many of the remaining edges as possible 9

27 PC algorithm - oracle version Assume faithfulness and an oracle that tells us whether or not X Y S for any triple (X,Y,S). Then a CPDAG can be estimated by the PC-algorithm of Peter Spirtes and Clark Glymour (2000): Determine the skeleton Determine the v-structures Direct as many of the remaining edges as possible Fast implementation in the R-package pcalg (Kalisch et al., 2012) Consistent in sparse high-dimensional settings (Kalisch and Bühlmann, 2007) 9

28 PC algorithm - sample version Instead of the oracle, we perform conditional independence tests 10

29 PC algorithm - sample version Instead of the oracle, we perform conditional independence tests In the multivariate Gaussian setting, this is equivalent to testing for zero partial correlation: H 0 : ρ ij S = 0 versus H a : ρ ij S 0. 10

30 PC algorithm - sample version Instead of the oracle, we perform conditional independence tests In the multivariate Gaussian setting, this is equivalent to testing for zero partial correlation: H 0 : ρ ij S = 0 versus H a : ρ ij S 0. Partial correlations can be computed via regression, inversion of parts of the covariance matrix, or a recursive formula 10

31 PC algorithm - sample version Instead of the oracle, we perform conditional independence tests In the multivariate Gaussian setting, this is equivalent to testing for zero partial correlation: H 0 : ρ ij S = 0 versus H a : ρ ij S 0. Partial correlations can be computed via regression, inversion of parts of the covariance matrix, or a recursive formula For testing, it is helpful to use Fisher s Z-transform: ẑ ij S = 1 ( ) 1+ ρij S 2 log. 1 ρ ij S Under H 0, n S 3ẑ ij S N(0,1). 10

32 PC algorithm - sample version Instead of the oracle, we perform conditional independence tests In the multivariate Gaussian setting, this is equivalent to testing for zero partial correlation: H 0 : ρ ij S = 0 versus H a : ρ ij S 0. Partial correlations can be computed via regression, inversion of parts of the covariance matrix, or a recursive formula For testing, it is helpful to use Fisher s Z-transform: ẑ ij S = 1 ( ) 1+ ρij S 2 log. 1 ρ ij S Under H 0, n S 3ẑ ij S N(0,1). Hence, we reject H 0 versus H a if n S 3 ẑ ij S > Φ 1 (1 α/2) 10

33 PC algorithm - sample version Instead of the oracle, we perform conditional independence tests In the multivariate Gaussian setting, this is equivalent to testing for zero partial correlation: H 0 : ρ ij S = 0 versus H a : ρ ij S 0. Partial correlations can be computed via regression, inversion of parts of the covariance matrix, or a recursive formula For testing, it is helpful to use Fisher s Z-transform: ẑ ij S = 1 ( ) 1+ ρij S 2 log. 1 ρ ij S Under H 0, n S 3ẑ ij S N(0,1). Hence, we reject H 0 versus H a if n S 3 ẑ ij S > Φ 1 (1 α/2) The significance level α serves as a tuning parameter for the PC algorithm 10

34 PC algorithm - sample version Instead of the oracle, we perform conditional independence tests In the multivariate Gaussian setting, this is equivalent to testing for zero partial correlation: H 0 : ρ ij S = 0 versus H a : ρ ij S 0. Partial correlations can be computed via regression, inversion of parts of the covariance matrix, or a recursive formula For testing, it is helpful to use Fisher s Z-transform: ẑ ij S = 1 ( ) 1+ ρij S 2 log. 1 ρ ij S Under H 0, n S 3ẑ ij S N(0,1). Hence, we reject H 0 versus H a if n S 3 ẑ ij S > Φ 1 (1 α/2) The significance level α serves as a tuning parameter for the PC algorithm We perform many many tests during the algorithm. Can we obtain consistency results? 10

35 High-dimensional asymptotic framework Since typical datasets in biology contain many more variables than observations, we consider a framework in which the graph is allowed to grow with the sample size n: DAG: G n Number of variables: p n Variables: X n1,...,x npn Distribution: P n Parial correlations: ρ nij S 11

36 Assumptions P n is multivariate Gaussian and faithful to the true unknown causal DAG G n 12

37 Assumptions P n is multivariate Gaussian and faithful to the true unknown causal DAG G n High-dimensionality and sparseness: p n = O(n a ), for some 0 a < Maximum number of neighbors in G n is q n = O(n 1 b ), for some 0 < b 1 12

38 Assumptions P n is multivariate Gaussian and faithful to the true unknown causal DAG G n High-dimensionality and sparseness: p n = O(n a ), for some 0 a < Maximum number of neighbors in G n is q n = O(n 1 b ), for some 0 < b 1 Regularity conditions on partial correlations: sup n,i j,s ρ nij S M for some M < 1, where S {X n1,...,x npn }\{X ni,x nj } with S q n inf i,j,s { ρ nij S : ρ nij S 0} c n, where S {X n1,...,x npn }\{X ni,x nj } with S q n and c 1 n = O(n d ) for some 0 < d < b/2 12

39 High dimensional consistency Denote the estimated CPDAG by Ĉn(α n ) and the true CPDAG by C. Then there exists a sequence α n 0 such that P(Ĉn(α n ) = C) = 1 O(exp( Cn 1 2d )), for some C > 0 and d as in the assumptions (Kalisch & Bühlmann, 2007) 13

40 Sketch of proof Sketch of the proof: E nij S is event of a type I/II error when testing for ρ nij S = 0 Let PC qn denote the PC algorithm where we test conditional independencies up to level q n Choose α n st P(E nij S ) = O(nexp( C(n q n )c 2 n)) if S q n Then P(error occurs in PC qn (α n )) P( i,j,s: S qn E nij S ) i,j,s: S q n P(E nij S ) O(p q n+2 n )O(nexp( C(n q n )c 2 n)) = O(exp(q n log(p n )+log(n) C(n q n )c 2 n)) = O(exp(n 1 b alog(n)+log(n) Cn 1 2d +Cn 1 2d b ) 0 14

41 Summary: learning DAGs from observational data Markov equivalence class Faithfulness PC algorithm Consistency in high-dimensional settings 15