Math 494: Mathematical Statistics

Size: px

Start display at page:

Download "Math 494: Mathematical Statistics"

Daniela Dickerson
5 years ago
Views:

1 Math 494: Mathematical Statistics Instructor: Jimin Ding Department of Mathematics Washington University in St. Louis Class materials are available on course website ( jmding/math494/ ) Spring 2018 Jimin Ding, Math WUSTL Math 494 Spring / 8

2 Monte Carlo Method Jimin Ding, Math WUSTL Math 494 Spring / 8

3 Monte Carlo Method A class of computational algorithms based on repeated random sampling. To obtain numerical results when closed-form solution is not available or hard to track down. Before MC method was developed, simulations were used to test a known deterministic problem, and statistical sampling was used to estimate uncertainties in simulations. But MC simulations invert the process and solve deterministic problems using probabilistic simulations. Invented by Stanislaw Ulam and von Neumann in the late 1940s. Named by Nicholas Metropolis, after the Monte Carlo casino at Monaco. Extension: Markov Chain Monte Carlo (MCMC) Jimin Ding, Math WUSTL Math 494 Spring / 8

4 Example 1: Estimation of π (Example 4.8.1) Let U 1, U 2 iid U(0, 1). Set X = { 1, U U 2 2 < 1 0, U U Then E(X) = P (X = 1) = P (U U 2 2 < 1) = Jimin Ding, Math WUSTL Math 494 Spring / 8

5 Example 1: Estimation of π (Example 4.8.1) { iid 1, U 2 Let U 1, U 2 U(0, 1). Set X = 1 + U 2 2 < 1 0, U1 2 + U Then E(X) = P (X = 1) = P (U1 2 + U 2 2 < 1) = π/4 Jimin Ding, Math WUSTL Math 494 Spring / 8

6 Example 1: Estimation of π (Example 4.8.1) { iid 1, U 2 Let U 1, U 2 U(0, 1). Set X = 1 + U 2 2 < 1 0, U1 2 + U Then E(X) = P (X = 1) = P (U1 2 + U 2 2 < 1) = π/4 So one can estimate π by 4E(X) or 4 X. Algorithm: Generate u 11,, u 1n and u 21,, u 2n from U(0, 1) Set x 1,, x n. Find x and use 4 x to estimate π. Construct a 95% CI for π: x ± 1.96 x(1 x)/n Jimin Ding, Math WUSTL Math 494 Spring / 8

7 Example 2: Monte Carlo Integration (Example 4.8.4) Recall that π = x 2 dx = Jimin Ding, Math WUSTL Math 494 Spring / 8

8 Example 2: Monte Carlo Integration (Example 4.8.4) Recall that π = x 2 dx = E(g(X)),where X U(0, 1) and g(x) = 4 1 x 2. So one can estimate π by g(x). Jimin Ding, Math WUSTL Math 494 Spring / 8

9 Example 2: Monte Carlo Integration (Example 4.8.4) Recall that π = x 2 dx = E(g(X)),where X U(0, 1) and g(x) = 4 1 x 2. So one can estimate π by g(x). Algorithm: Generate u 1,, u n from U(0, 1) Calculate g(u 1 ),, g(u n ). Estimate π by ḡ = n i=1 g(u i)/n. Construct a 95% CI for π: denote s g = 1 n 1 n i=1 [g(u i) ḡ] 2, ḡ ± 1.96s g / n Jimin Ding, Math WUSTL Math 494 Spring / 8

10 Example 2: Monte Carlo Integration (Example 4.8.4) Recall that π = x 2 dx = E(g(X)),where X U(0, 1) and g(x) = 4 1 x 2. So one can estimate π by g(x). Algorithm: Generate u 1,, u n from U(0, 1) Calculate g(u 1 ),, g(u n ). Estimate π by ḡ = n i=1 g(u i)/n. Construct a 95% CI for π: denote s g = 1 n 1 n i=1 [g(u i) ḡ] 2, ḡ ± 1.96s g / n In general, b a g(x)dx = (b a) b g(x) a b a dx = (b a)e(g(x)), where X U[a, b], hence can be estimated by (b a)g(x) Jimin Ding, Math WUSTL Math 494 Spring / 8

11 Example 3: Generate a r.v. from F 1 (U) Theorem Ex 4.8.1: If X F, then F (X) U[0, 1]. And Y = F 1 (U) F. Jimin Ding, Math WUSTL Math 494 Spring / 8

12 Example 3: Generate a r.v. from F 1 (U) Theorem Ex 4.8.1: If X F, then F (X) U[0, 1]. And Y = F 1 (U) F. For example, to generate X from f X (x) = exp{x e x }, x R: (Extreme value distribution, log of Weibull distribution) Jimin Ding, Math WUSTL Math 494 Spring / 8

13 Example 3: Generate a r.v. from F 1 (U) Theorem Ex 4.8.1: If X F, then F (X) U[0, 1]. And Y = F 1 (U) F. For example, to generate X from f X (x) = exp{x e x }, x R: (Extreme value distribution, log of Weibull distribution) It is easy to show the cdf is F X (x) = 1 exp{ e x }, x R Jimin Ding, Math WUSTL Math 494 Spring / 8

14 Example 3: Generate a r.v. from F 1 (U) Theorem Ex 4.8.1: If X F, then F (X) U[0, 1]. And Y = F 1 (U) F. For example, to generate X from f X (x) = exp{x e x }, x R: (Extreme value distribution, log of Weibull distribution) It is easy to show the cdf is F X (x) = 1 exp{ e x }, x R So F 1 (u) = log( log(1 u)), u (0, 1). Jimin Ding, Math WUSTL Math 494 Spring / 8

15 Example 3: Generate a r.v. from F 1 (U) Theorem Ex 4.8.1: If X F, then F (X) U[0, 1]. And Y = F 1 (U) F. For example, to generate X from f X (x) = exp{x e x }, x R: (Extreme value distribution, log of Weibull distribution) It is easy to show the cdf is F X (x) = 1 exp{ e x }, x R So F 1 (u) = log( log(1 u)), u (0, 1). Hence generate U U(0, 1), then X = log( log(1 U)) follows extreme value distribution with above pdf. Jimin Ding, Math WUSTL Math 494 Spring / 8

16 Example 4: Monte Carlo CI Let X 1,, X n iid N(µ, σ 2 ). A 95% CI for µ: x ± t 0.025,n 1 s/ n. This can be explained as if one construct 100 CIs, 95 of them will contain the true mean µ. Here 95% is called nominal coverage probability. Now we can construct a Monte Carlo simulation to check the actual coverage probability, and compare it with 95% nominal coverage probability. Algorithm: 1. Set k = 1 2. Generate X 1,, X n iid N(µ, σ 2 ) 3. Construct a 95% CI: x ± t 0.025,n 1 s/ n. 4. If k = N, go to step 5; otherwise, k = k + 1, and go to step Count the number of CIs containing µ and denote as I. Then the actual coverage probability is 1 ˆα = I/N. Furthermore, the approximation error is 1.96 ˆα(1 ˆα)/N. Jimin Ding, Math WUSTL Math 494 Spring / 8

17 Remarks What happens if data were not from N(µ, σ 2 )? Check the robustness of your CI. See example for Monte Carlo tests. Jimin Ding, Math WUSTL Math 494 Spring / 8

Chapter 1. Introduction

Chapter 1. Introduction Chapter 1 Introduction A Monte Carlo method is a compuational method that uses random numbers to compute (estimate) some quantity of interest. Very often the quantity we want to compute is the mean of