UP School of Statistics Student Council Education and Research

Transcription

1 w UP School of Statistics Student Council Education and Research erho.weebly.com 0 erhomyhero@gmail.com f /erhoismyhero S133_HOA_001 Statistics 133 Bayesian Statistical Inference Use of R for Bayesian Analysis 1. Let X 1, X 2,, X n ~Be ( p) and suppose p ~Beta(1, 1). Find the MSE of the following estimators for the target parameter p : a. ^p MLE b. ^p B Note: Some lines of code may appear as two lines in this handout but they must be coded in R as one line #Create a sequence containing values of p for plotting p<-seq(from=0,to=1,by=0.01) #Set n to a value of 10 n<-10 #From the formula of the MSE of the MLE derived above mse.mle<-p*(1-p)/n #From the formula of the MSE of the Bayes estimator derived above mse.bayes<-n*p*(1-p)/(n+2)^2+((1-2*p)/(n+2))^2 Plotting Codes are displayed chronologically a. Method 1 plot(x=p,y=mse.mle,type='l',col='1',lwd="2",xlab="p",ylab="ms E",las=1) i. x = specify values on x-axis ii. y = specify values on y-axis iii. type = specify plot type (type?plot in R to see different plot types). Specified above is l (small letter L) indicating line type plot iv. lwd = specifiy line thickness v. col = specifies color of plotting vi. xlab = label of x-axis 1

2 vii. ylab = label of y-axis viii. las = 1 makes the y-values in the y-axis appear horizontally lines(x=p,y=mse.bayes,col='2',lwd="2",lty=2) lines function allows one to superimpose a plot to a pre-existing plot. lty = specifies the line type. lty = 2 means the dashed line is used type?par and look for lty to explore different line types Resulting plot b. Method 2 plot(x=p,y=mse.mle,type="l",axes="f",xlab="p",ylab="mse",lwd= "2") Axes = F means that the plot will appear without x and y axes as we will customize the tick marks of the axes to be shown below lines(x=p,y=mse.bayes,col='2',lwd="2",lty=2) This is the same lines function as the first method. This will superimpose the Bayes MSE graph to the MLE MSE graph box() axis(side=1,at=seq(from=0,to=1,by=0.1)) axis(side=2,at=seq(from=0,to=0.05,by=0.005),las=1) i. box() function wraps the plot around a box since the axes = F option in plot removes the box. ii. axis function creates customizable axes 2

3 iii. side=1 creates the x-axis or places tick marks on the bottom of the box and side=2 creates the y- axis or places tick marks on the left of the box. (side = 3 and side = 4 puts tick marks on the top and right, respectively) iv. at option defines the sequence of numbers we wish to put in the axes Resulting plot We now have more values in the x-axis compared to the previous plot. c. Adding legends (Optional) legend(x=0.35,y=0.01,legend=c("mle","bayes"),col=c(1,2),lty=c (1,2),lwd="1") i. place the legend function after the plotting functions such as plot, lines, box and axes ii. x = and y = specifies the coordinates of the top left of the legend box iii. legend = specifies the legend labels iv. col = specifies the color of the legend lines v. lty = specifies the line type of the legend lines vi. lwd = specifies the thickness of the legend lines vii. legend label MSE will have color 1 and lty 1 while label Bayes will have color 2 and lty 2 Using plotting method 2, we add labels p<-seq(from=0,to=1,by=0.01) n<-10 mse.mle<-p*(1-p)/n mse.bayes<-n*p*(1-p)/(n+2)^2+((1-2*p)/(n+2))^2 plot(x=p,y=mse.mle,type="l",axes="f",xlab="p",ylab="mse",lwd= "2") lines(x=p,y=mse.bayes,col='2',lwd="2",lty=2) box() axis(side=1,at=seq(from=0,to=1,by=0.1)) 3

4 axis(side=2,at=seq(from=0,to=0.05,by=0.005),las=1) legend(x=0.35,y=0.01,legend=c("mle","bayes"),col=c(1,2),lty=c (1,2),lwd="1") Resulting plot 2. Let X 1, X 2,, X n ~ Po(θ ) and suppose θ ~Gamma(r, λ ). Find the MSE of the following estimators for the target parameter θ : a. ^θ MLE b. ^θ B Suppose we use Ga(2, 1) as prior Codes theta<-seq(from=0,to=5,by=0.5) n<-6 lambda<-1 r<-2 mse.mle<-theta/n mse.bayes<-n*theta/(lambda+n)^2+((r-(lambda*theta))/ (lambda+n))^2 plot(x=theta,y=mse.bayes,type="l",lwd=2,col=1,xlab='theta',ylab= 'mse',las=1) lines(x=theta,y=mse.mle,lwd=2,col=1,lty=2) legend(x=0.1,y=0.7,legend=c("bayes","frequentist"),lty=c(1,2),lw d=2) 4

5 Resulting plot Bayesian Confidence Interval Estimation Note: Check if the following packages are installed: TeachingDemos, coda, VGAM Use of the Special Parametric Family of Distributions dbeta(x,shape1,shape2) returns the density or point in the graph provided the quantile and parameters x specifies quantile where we want to find the density shape1 specifies alpha parameter in Beta distribution shape2 specifies beta parameter in Beta distribution pbeta(q,shape1,shape2,lower.tail = TRUE) returns the probability or area to the left of the specified quantile q specifies quantile qbeta(p,shape1,shape2,lower.tail = TRUE) the inverse of the pbeta function. Returns the quantile for the specified left-tail probability p specifies the lower tail probability rbeta(n,shape1,shape2) Creates a random sample of size n for a Beta distribution with paramters shape1 and shape2 dnorm, pnorm, qnorm, rnorm are used for the normal distribution dgamma, pgamma, qgamma, rgamma are used for the gamma distribution 5

6 Graphical Presentation of the Special Parametric Distributions Consider a normal distribution with mean 0 and variance 1. dnorm(x = 1, mean = 0, sd = 1) Result: dnorm returns this value ( ) Quantile (x) = 1 pnorm(q = 1, mean = 0, sd =1, lower.tail = TRUE) Result: pnorm returns this area ( ) Quantile (q) = 1 6

7 qnorm(p = 0.95, mean = 0, sd = 1, lower.tail = TRUE) Result: (approx ) p = 0.95 qnorm returns this value ( ) Equal Tail Credible Intervals Suppose that the probability of contracting a disease has a Beta(19, 133) posterior distribution. Give the 95% equal-tail credible interval for the estimate of the probability of contracting a disease. qbeta(p = c(0.025,0.975), shape1 = 19, shape2 = 133) HPD Credible Intervals For the same example above hpd(posterior.icdf = qbeta,shape1 = 19, shape2 = 133, conf = 0.95) Note: hpd function is under the TeachingDemos package 7

8 Monte Carlo Integration Syntax mc.integration<-function(lower,upper,incr,n,term){ temp.upper<-lower+incr sum=0 while(temp.upper<=upper){ ran.values<-runif(n = n, min = lower, max = temp.upper) sum=sum + mean(term(ran.values))*incr lower<-lower + incr temp.upper<-temp.upper+incr } return(sum) } When to use? If the prior is not a conjugate distribution of the process or if the posterior is not a special distribution Monte Carlo HPD Credible Interval R function: HPDinterval(as.mcmc(),prob) HPDinterval function is under coda package Requires a random sample from the posterior distribution. A vector containing the random sample is placed inside the parentheses of the as.mcmc() argument prob specifies level of confidence Monte Carlo Equal Tail Credible Interval quantile(x,probs) x specifies random sample probs specifies tail probabilities Example: Approximate credible intervals for a Beta(19, 133) posterior distribution set.seed(22) sample<-rbeta(n = 1000,shape1 = 19,shape2 = 133) #Equal tail quantile(x = sample,probs = c(0.025,0.975) #HPD HPDinterval(as.mcmc(sample),prob = 0.95)