Dynamic Documents. David Allen University of Kentucky. August 15, 2014

Transcription

1 Dynamic Documents David Allen University of Kentucky August 15, 2014 The term dynamic document covers a wide assortment of documents. See Wikipedia for a general definition. This discussion involves a situation where the document is a report including statistical analyses and graphical displays based on data. The data are continually being augmented or replaced. What is needed a way to automate the revision of the document when the data changes. The approach here is to use R to do the statistical calculations and graphics, tikzdevice (an R package) to output the graphics to L A TEX, and knitr (also an R package) to process an input file to a L A TEX file. 1 The Kentucky Senate Race On November 4, 2014, the Commonwealth of Kentucky will elect a United States Senator. The candidates are Alison Lundergan Grimes and Mitch McConnell. This race has high national impact and is closely watched. A poll yields the number of people in a sample, from a population of potential voters, favoring each candidate. The proportion of the sample favoring Alison (or Mitch) is reported. However, this provides no indication of the sampling variability. The parameter of interest is the population proportion favoring Alison. A credible interval is such that the parameter lies within the interval with high probability. A credible interval is a more informative mode of presentation, as it conveys the uncertainty of knowledge about the parameter. Calculation of the credible interval is the statistical analysis portion of the report. Denote the proportion of the population favoring Alison by p. The first step in calculating a credible interval is finding the posterior density function 1

2 of p given the sample results. One needs to select a level of credibility. The value 0.95 has a strong tradition and is used here. The 0.95 credibility interval is an interval (p 1, p 2 ) where P (p 1 < p < p 2 ) = There are multiple intervals satisfying the probability statement. The interval having minimal length is usually used. An example assuming a sample with 55 favoring Alison and 45 favoring Mitch is shown in the following graph. The 0.95 credible interval (0.4528, ) is the base of the shaded region. Posterior Density Proportion Favoring Alison A report might say something like A current poll produced 55 potential voters favoring Alison and 45 favoring Mitch. These results give a 0.95 credible interval for the proportion favoring Alison of (0.4528, ). The report might also include the above graph. The preceding graphic and the credible interval were produced with R. The output from R was then transcribed to a L A TEX file to produce the report. Polling will be a continuing activity from now until election day. Rerunning R and cutting and pasting output into a L A TEX document is tedious and error prone. Subsequent sections show how to automate the process. 2 TikZ graphics TikZ is a graphics package used in conjunction with TEX. It is included with most distributions of TEX, but may be downloaded at 2

3 net/projects/pgf/. A large selection of examples of TikZ graphics are posted at I think it likely that most TUGboat readers are familiar with TikZ. I give just two examples I have composed. This graphic was hand-coded in Sketch, and then processed into TikZ. This example was hand-coded directly in TikZ. θ 2 θ 1 GI tract Plasma Other θ 4 θ 3 3 An overview of R R is a language and environment for statistical computing and graphics. Its home page is R is a free software project. It compiles and runs on a wide variety of Unix platforms and similar systems (including FreeBSD and GNU/Linux), Windows and MacOSX. R is often the vehicle of choice for research in statistical methodology, and it provides an open source route to participation in that activity. R provides a wide variety of statistical techniques including linear and nonlinear modeling, classical statistical tests, time-series analysis, classification, and clustering. R is highly extensible and contains a rich collection of graphical techniques. One of R s strengths is the ease with which welldesigned publication-quality plots can be produced, including mathematical 3

4 symbols and formulas where needed. Defaults for the minor design choices in graphics have been carefully considered, but the user retains full control. 4 TikzDevice TikzDevice enables L A TEX-ready output from R graphics functions. This is done by producing code that can be understood by the TikZ graphics language. All text in a graphic output with the tikz() function will be typeset by L A TEX and therefore will match whatever fonts are currently used in the document. This also means that L A TEX mathematics can be typeset directly into labels and annotations! Graphics produced this way can also be annotated with custom TikZ commands. An example R Graphic output using tikzdevice is y y = 10 + (x 5) x The program that produced the preceding graph is setwd("~/tug2014") source("quadratic-data.r") source("quadratic-graph.r") The source command executes the statements in the named file. Here I group code into small parts to focus discussion. The file quadratic-data.r contains the data generation code x <- (0:100)/10 y < (x-5)^2 4

5 R formulas are different from standard mathematical formulas. A function call operates on every element of a vector. When vectors of different lengths are added or subtracted, the shorter one is recycled in an effort to make them the same length. The file quadratic-graph.r contains the graphics code require(tikzdevice) tikz("quadratic-graph.tex",standalone=false, width=4.5, height=2.5) par(mex=0.6, mar=c(4.5,5,0,0)+0.1) plot(x, y, type= l, xlab="$x$",ylab="$y$") text(5, 25, "$y = 10+(x-5)^2$") dev.off() Most of this code is understandable by comparing it to the resulting graph. An exception might be the par function used to change graphic parameters from their default values. The ones used in this example are Arg mex mar Description A character size expansion factor used to describe coordinates in the margins of plots. Vector of the form c(bottom, left, top, right) which gives the number of lines of margin to be specified on the four sides. 5 Implementation This section implements a dynamic document that facilitates reporting the current status of the race between Alison and Mitch. The document has a title, a graph of the posterior density, and a short statistical report. The data file for this senate race is senate.dat and contains just two numbers, the number in the sample that favor Alison, and the number that favor Mitch. This two number file can be updated with an editor. For more complicated situations there might be a program that updates a data file. Knitr is an R package containing a function knit. The function knit takes a file name with an extension.rnw as an argument. An.Rnw file is like a L A TEX file with inter-dispersed R chunks. The output is a pure L A TEX file containing the output from running the R chunks. Documentation for knitr is available online and in Yihui Xie s book, [1]. 5

6 R is an implemention of a language S. There is a function \Sexpr(), for S expression, that may be placed in the TEX portion of the file. \Sexpr() takes an R expression as an argument. The expression is evaluated, converted to text, and passed into the L A TEX output. The content of senate.rnw is \documentclass[12pt]{article} \begin{document} <<setup,echo=false>>= \title{alison Versus Mitch} \author{david Allen\\University of Kentucky} \maketitle \thispagestyle{empty} % <<params,echo=false>>= A poll released July 28, 2014 produced \Sexpr{a-2} potential voters favoring Alison and \Sexpr{b-2} favoring Mitch. These results give a \Sexpr{level} credible interval for the proportion favoring Alison of (\Sexpr{p1}, \Sexpr{p2}). % The posterior density function, with the area over the credible interval shaded, is <<label="density",dev= tikz,echo=false,fig.width=4,fig.height=2.75,fig.align= cen \end{document} The R chunks have a header of the form <<... >>= where commands are inside the double brackets. A chunk ends with The content of chunk1.r is setwd("~/tug2014") 6

7 interval.length <- function(p1, a, b, level=0.95) { q <- qbeta(1-level, a,b) if( p1 > q ) return(1 - q) if( p1 < 0 ) return(qbeta(level, a, b)) p2 <- qbeta(pbeta(p1, a, b) + level, a, b) return(p2-p1) } The function interval.length is a function I wrote that gives the length of an interval starting at p 1. The posterior distribution of p is the beta distribution. qbeta is a built-in function giving quantiles of the beta distribution. The content of chunk2 is vote <- vector(mode="numeric") vote <- scan(file="senate.dat") a <- vote[1] + 2 b <- vote[2] + 2 level < p1 <- optimize(f = interval.length, interval = c(0, qbeta(1-level, a,b)), a=a, b=b, level=level)$minimum p2 <- qbeta(pbeta(p1, a, b) + level, a, b) Here the built-in optimize function is used to find the value of p 1 associated with the shortest interval. Then the corresponding p 2 is computed. The content of chunk3.r is left <- (1:80)/80*p1 interval <- p1 + (1:80)/80*(p2-p1) right <- p2 + (1:80)/80*(1-p2) domain <- c(left, interval, right) range <- dbeta(domain, a, b) par(mex=0.6, mar=c(4.5,5,0,0)+0.1) plot(c(0,1), c(0, max(range)), type="n", xlab="proportion Favoring Alison", ylab="density",yaxt= n ) 7

8 polygon(c(interval, p2, p1), c(dbeta(interval, a, b), 0, 0), col=27) lines(domain, range); lines(c(0,1),c(0,0)) lines(c(0.5,0.5),c(0,max(range))) This code produces the graph. After each data update run the following command in a terminal. Rscript -e "library(knitr);knit( senate.rnw )" A L A TEX file is produced that is processed in the usual ways. Rscript is just the command line version of R. The -e option means the following are statements to be run, as opposed to a file containing statements. I conclude with an exercise. A poll was released on July 28, 2014 (the first full day of the conference in Portland) showing 321 for Alison and 336 for Mitch. I invite you to prepare a data file, senate.dat, containing Then knit the.rnw file and L A TEX the resulting senate.tex References [1] Yihui Xie. Dynamic Documents with R and knitr. Chapman & Hall/CRC Press, ISBN