Reproducible research and knitr

Transcription

1 Reproducible research and knitr Friedrich Schuster 1 February 24, Thanks to Yihui Xie, author of knitr. Friedrich Schuster () knitr February 24, / 30

2 Background, expectations Background: Senior software developer at HMS Analytical Software GmbH, Heidelberg Not working in scientific research Limited experience with Sweave New to knitr Friedrich Schuster () knitr February 24, / 30

3 Agenda Reproducible research Background and expectations Sweave and knitr Chunks, options, hooks etc. Other features Friedrich Schuster () knitr February 24, / 30

4 Reporting is important Trivial: Sharing work with others and presenting it is important. Not so trivial: Doing it right. Different requirements exist: documentation, scientific reports, presentations, web publishing, books, or all of the above. Requirements determine (for example) the export format(s): Excel, Word, PDF, PNG, HTML... Friedrich Schuster () knitr February 24, / 30

5 Reproducible research The term reproducible research refers to the idea that the ultimate product of research is the paper along with the full computational environment used to produce the results in the paper such as the code, data, etc. necessary for reproduction of the results and building upon the research. (Wikipedia) Some journals even seem to require a certain level of reproducability: Friedrich Schuster () knitr February 24, / 30

6 Reproducible research What: Why: Code, data, results etc. in one collection of files or a single package. Automate computations and report generation with a script. Present figures, tables and code in a single, coherent document. Transparency: full code, data and results. Reproducibility: do it again or let others repeat your steps. Effort: concentrate on research, not editing a report. Literate programming : related, but not exactly the same: concentrates on readable code and code documentation, not report generation. Friedrich Schuster () knitr February 24, / 30

7 Quality vs time Friedrich Schuster () knitr February 24, / 30

8 CRAN task view reproducible research Contains and is structured by: Packages supporting different types of file formats (HTML, LaTeX, Microsoft formats, Open document formats, plain text formats etc.) Packages for specific types of analysis or common tasks (e.g. buffering of R objects, syntax highlighting) A list of known packages for reproducible research. A short list of related links. Friedrich Schuster () knitr February 24, / 30

9 What you can do with knitr Create a PDF from R code directly: with stich() Run R script and create a simple document: with spin() Use a Markdown file with R as input. Create a PDF report with code, output and text. Create a PDF report from a Sweave file: Sweave2knitr(). Create a beamer presentation. Publish to the web: create HTML, upload images, create wordpress blog entries. Friedrich Schuster () knitr February 24, / 30

10 Elements of a report Common elements of a report, e.g. in the stats homework example: Formatted text (with or without embedded result values) Source code (syntax highlighted) Figures, plots, mathematical expressions Program results Program errors and warnings Friedrich Schuster () knitr February 24, / 30

11 Ideal workflow PDF Code R Results In Trans HTML Other Friedrich Schuster () knitr February 24, / 30

12 Sweave, knitr Similar to Sweave, the knitr package allows you to embed R code and figures in L A TEX documents. Friedrich Schuster () knitr February 24, / 30

13 How does it really work? R Code Results Tex+R Trans Tex+Result PDF Friedrich Schuster () knitr February 24, / 30

14 How does it work? The input document is a template. Sweave or knitr do not format the output, they handle the execution of R code and replace the source code with results. The output itself is then created by other packages and programs (e.g. package R2HTML or latex). Formatting the template is up to you. For larger PDF documents Tex document classes for articles exist (also for presentations : beamer class). Latex-Editors support formatting: for example Lyx User interfaces for R (especially Rstudio, but also eclipse and other tools) simplify the task of writing and running R scripts for reproducible research. Friedrich Schuster () knitr February 24, / 30

15 Terminology Tangling: extracting R code from the input file. ( Sweave: Stangle() ) Weaving: running R code through R an replacing the code chunks with the results. (in knitr: knitting ). ( Sweave: Sweave() ) TeX, LaTeX: an open source typsetting framework that can be used to produce complex technical documents. Code type: code chunks or inline expressions. Chunks: separate code segments of R code or other languages (Python, Awk, Ruby, Haskell, Bash, Perl, Graphviz, Tikz, SAS, CoffeeScript). Chunks are separated from the formatted text by chunk delimiters. Inline expressions: R expressions embedded in the formatted text. Hooks (knitr): Code that is executed before or after code chunks. Six output types (in knitr): output from code chunks or inline expressions: source code itself, formatted text output, messages from R, warnings from R, errors from R, plots and graphics. Friedrich Schuster () knitr February 24, / 30

16 Why knitr? Compared to Sweave... More input and output formats. knitr tries to simplify work with different input formats: determines the format from the source file. Multiple plots per code chunk are supported. Integrates different features that require extra packages in Sweave: syntax highlighting, caching of R objects, list formatting etc. knitr tries hard not to be incompatible to Sweave. In fact it is possible to use Sweave files with knitr. Friedrich Schuster () knitr February 24, / 30

17 Source file examples Sweave: file:///home/schuster/workspace/rug_2013_01_ ReprodResearch/src/Rnw/sweave-example-1.Rnw knitr: file:///home/schuster/workspace/rug_2013_01_ ReprodResearch/src/Rnw/002-minimal.Rnw Friedrich Schuster () knitr February 24, / 30

18 Installation # install.packages('knitr') Friedrich Schuster () knitr February 24, / 30

19 Calling knitr stitch(), spin() stitch("knitr-spin.r") tools::texi2dvi(paste(fn, "tex", sep = "."), pdf = TRUE) # stitch_rhtml() stitch_rmd() spin("knitr-spin.r") knitr-spin.pdf knitr-spin.html Convert Sweave file to knitr, then run knitr with PDF output: require("knitr") require("markdown") require("tools") require("rgl") Sweave2knitr("sweave-example-1.Rnw", output = "Sweave2knitr.Rnw") knit(input = "Sweave2knitr.Rnw", output = "Sweave2knitr.tex") tools::texi2dvi("sweave2knitr.tex", pdf = TRUE) sweave-example-1.pdf Friedrich Schuster () knitr February 24, / 30

20 Options Output types: six types: from code chunks and inline expressions 1 source code: chunk option echo : FALSE, TRUE 2 text output: chunk option results : markup, asis, hide 3 messages: chunk option message : FALSE, TRUE 4 warnings: chunk option warning : FALSE, TRUE 5 errors: chunk option error : FALSE, TRUE 6 plots: chunk option fig.keep : none, low, high All options can take values from R expressions: conditional evaluation is possible. Friedrich Schuster () knitr February 24, / 30

21 Code chunks: text and result options # <<knitr-installation,echo=true, results=false, message=false, # warning=false, error=false >>= # R.Version() Link Stat546-HW4-Xie.pdf Link Stat546-HW4-Xie.Rnw Friedrich Schuster () knitr February 24, / 30

22 Code chunks: example letters ## [1] "a" "b" "c" "d" "e" "f" "g" "h" "i" "j" "k" "l" "m" "n" "o" "p" "q" ## [18] "r" "s" "t" "u" "v" "w" "x" "y" "z" par(mar = c(4, 4, 0.2, 0.2)) plot(rnorm(100)) rnorm(100) Friedrich Schuster () knitr February 24, / 30

23 Code chunks: Some plot options More than 20 grapics devices included: specified with the dev= option. E.g.: tikz, CairoPNG... knitr uses the evaluate package to record plots. It saves a snapshot when plot.new() or grid.newpage() is calld and when a complete expression is evaluated. What is discarded or retained is controlle with the option fig.keep= all. With fig.keep= high only the completed and different plots will be kept. With fig.keep= last only the very last plot kept. with the option fig.show= asis the process of plotting can be shown step by step. Even animations are possible: fig.show= animate file:///home/schuster/workspace/rug_2013_01_reprodresearch/ doc/knitr-graphics.pdf fig.align= center - alignment Plot size: e.g. fig.fig.height=4 fig.cap=na - disable figure caption, or fig.cap= hello Friedrich Schuster () knitr February 24, / 30

24 Global options options(tikzdefaultengine = xetex ) Friedrich Schuster () knitr February 24, / 30

25 Hooks Chunk hooks: Chunk hooks are functions to be called before or after a code chunk. Output hooks: Output hooks are used to customize and polish the raw output from chunks. There are 8 output hooks in all to deal with different types of output: source code, ordinary R output, warnings, messages, errors, plots, inline code, all output from a chunk, and the output of the whole document. see: Friedrich Schuster () knitr February 24, / 30

26 Plain text formats Markdown: is a lightweight markup language, allowing people to write using an easy-to-read, easy-to-write plain text format, then convert it to... other formats. restructuredtext Friedrich Schuster () knitr February 24, / 30

27 Web publishing HTML Wordpress blog post: works with the RWordPress package and allows to publish HTML fragments as new Wordpress blog posts. Knitr creates the HTML fragements with R Markdown and the markdown package. Image upload to an image host Friedrich Schuster () knitr February 24, / 30

28 Child documents and code Import code files Import sub documents Knit child documents in standalone mode Friedrich Schuster () knitr February 24, / 30

29 Other tools Pandoc Pandoc can convert documents in markdown, restructuredtext, textile, HTML, DocBook, LaTeX, or MediaWiki markup to HTML formats: XHTML, HTML5, and HTML slide shows using Slidy, Slideous, S5, or DZSlides. Word processor formats: Microsoft Word docx, OpenOffice/LibreOffice ODT, OpenDocument XML Ebooks: EPUB version 2 or 3, FictionBook2 Documentation formats: DocBook, GNU TexInfo, Groff man pages TeX formats: LaTeX, ConTeXt, LaTeX Beamer slides PDF via LaTeX Lightweight markup formats: Markdown, restructuredtext, AsciiDoc, MediaWiki markup, Emacs Org-Mode, Textile Friedrich Schuster () knitr February 24, / 30

30 Thank you Homepage: Examples: Github repository (more examples): Thank you! Friedrich Schuster () knitr February 24, / 30