Approximate Bayesian Computation. Alireza Shafaei - April 2016

Transcription

1 Approximate Bayesian Computation Alireza Shafaei - April 2016

2 The Problem Given a dataset, we are interested in.

3 The Problem Given a dataset, we are interested in.

4 The Problem Given a dataset, we are interested in.

5 The Problem - A Review Previously we looked at the general problem of handling high-dimensional integrals and unnormalized probability functions.

6 The Problem - A Review Rejection Sampling Given Accept x with probability

7 The Problem - A Review Importance Sampling

8 The Problem - A Review Importance Sampling

9 The Problem - A Review Importance Sampling

10 The Problem - A Review Importance Sampling

11 The Problem - A Review Markov chain Monte Carlo

12 The Problem - A Review Markov chain Monte Carlo Use a transition to move in the space.

13 The Problem - A Review Markov chain Monte Carlo Use a transition to move in the space. Gibbs sampling.

14 The Problem - A Review Markov chain Monte Carlo Use a transition to move in the space. Gibbs sampling. Metropolis-Hastings algorithm.

15 The Problem - A Review Markov chain Monte Carlo Use a transition to move in the space. Gibbs sampling. Metropolis-Hastings algorithm. Reversible Jump MCMC (non-parametric)

16 The Problem Given a dataset, we are interested in.

17 The Problem Given a dataset, we are interested in. What if we can t calculate?

18 The Problem Apparently applies to a lot of problems in biology.

19 The Problem Apparently applies to a lot of problems in biology. Given a parameter you can simulate the execution.

20 The Problem Apparently applies to a lot of problems in biology. Given a parameter you can simulate the execution. Could be intractable or simply no mathematical derivation of it exists.

21 Approximate Bayesian Computation 1. Draw

22 Approximate Bayesian Computation 1. Draw 2. Simulate

23 Approximate Bayesian Computation 1. Draw 2. Simulate 3. Accept if

24 Approximate Bayesian Computation 1. Draw 2. Simulate 3. Accept if

25 Approximate Bayesian Computation 1. Draw 2. Simulate 3. Accept if

26 Approximate Bayesian Computation 1. Draw 2. Simulate 3. Accept if

27 Discussion Randomly sampling from the prior each time is too wasteful. We want to explore the space to accept more often.

28 Discussion Randomly sampling from the prior each time is too wasteful. We want to explore the space to accept more often. Sampling from the prior does not incorporate current observations.

29 Discussion Randomly sampling from the prior each time is too wasteful. We want to explore the space to accept more often. Sampling from the prior does not incorporate current observations. How do we choose?

30 Approximate MCMC 1. Propose

31 Approximate MCMC 1. Propose 2. Simulate

32 Approximate MCMC 1. Propose 2. Simulate 3. If a. Accept with probability

33 Approximate Gibbs Let s assume is known. is unknown.

34 Approximate Gibbs Let s assume is known. is unknown. 1.

35 Approximate Gibbs Let s assume is known. is unknown

36 Approximate Gibbs Let s assume is known. is unknown

37 Approximate Gibbs Let s assume is known. is unknown else go to 2.

38 Discussion Pros Likelihood is not needed.

39 Discussion Pros Likelihood is not needed. Easy to implement and parallelize.

40 Discussion Pros Likelihood is not needed. Easy to implement and parallelize. Cons Lot s of tuning.

41 Discussion Pros Likelihood is not needed. Easy to implement and parallelize. Cons Lot s of tuning. For complex problems, sampling from the prior is frustrating because it does not incorporate the evidence.

42 Discussion Pros Likelihood is not needed. Easy to implement and parallelize. Cons Lot s of tuning. For complex problems, sampling from the prior is frustrating because it does not incorporate the evidence. How good is our approximation?

43 Thank you!

44 References 1. Wilkinson, Richard, and Simon Tavaré. "Approximate Bayesian Computation: a simulation based approach to inference." Barber, David. Bayesian reasoning and machine learning. Cambridge University Press,