R Bootcamp Part I (B)

Transcription

1 R Bootcamp Part I (B) An R Script is available to make it easy for you to copy/paste all the tutorial commands into RStudio: Preliminaries: Skip this during Bootcamp presentation The Dataset Used in This Part of the R Bootcamp: The data are a modification of the survey dataset in the MASS package. For the record, below shows exactly how we created the data for the tutorial file datafile.csv. You can ignore this stuff and when you get to Step (2) below, just read the already-modified data file into R. library(mass) help(survey) # remove missing values for this example to illustrate basic functions mydata <- na.omit(survey) n <- nrow(mydata) # rename variables to allow "generic" code examples for the tutorial set.seed( ) library(tidyverse) mydata <- transmute(mydata, y = Pulse, x = *y + rnorm(n, m=0, s=7), x1 = Wr.Hnd, x2 = NW.Hnd, x3 = Height, g = Clap, g1 = Exer, g2 = Sex ) The Variable Names y is a quantitative (perhaps response) variable (plotted on y-axis) x, x1, x2, x3 are quantitative (perhaps explanatory/predictor) variables (plotted on x-axis) g, g1, g2 are categorical (perhaps explanatory/predictor) variables (g stands for group ) Starting any Data Analysis (1) Skip this step if using rstudio.uchicago.edu Load extra packages every time you start RStudio installed on your own computer If you are using packages are automatically loaded for you and you can skip down to step (2). Yay! If you are using RStudio on your own computer, you should enter the following into the Console each time you start up the software. This loads up many usefuel extra R functions. library(mass) library(ggally) library(openintro) library(mosaic) library(knitr) library(tidyverse) library(ggformula) R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 1 of 13

2 library(gridextra) library(broom) Caution: A lot of red text will flash by as various packages are loaded up. This is normal....unless you see messages labeled Error. The most common error is "there is no package called..." This means that you never installed the package. Go back the the Canvas R Help page and find the installation instructions ( Follow the 3rd step of the instructions: Install add-on packages. (2) Download, Upload, then Read in the data. Your instructor will likely provide you with clean, comma-separated datasets, with variable names already included. For example, Dr. Collins plans to store STAT 234 datasets at Many instructors post data files on their Canvas website. For this tutorial, we put the data at this location: 1. Go to and download datafile.csv 2. Upload the file to rstudio.uchicago.edu 3. Read the data file into R/RStudio mydata <- read.csv("datafile.csv") (3) Check the data. Explore the data structure. Always make a quick check of the data you just loaded into R. The following functions will help. # in the tibble pkg within the tidyverse set of pkgs glimpse(mydata) # info on variable names, types, and values # base R functions str(mydata) # info on variable names, types, and values summary(mydata) # numerical summaries of the values in each variable head(mydata, 10) # see the first 10 rows, cases, observations tail(mydata, 7) # see the last 7 rows, cases, observations names(mydata) # see only the names of the dataset columns, variables R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 2 of 13

3 View(mydata) # view the data in a spreadsheet-like format Just one or two of these functions should suffice to get the information you need to move forward with your data analysis. Which function(s) depends on the type of information you are looking for. End of Preliminaries: Start Bootcamp presentation here Learn by Doing: Follow Along On Your Computer Right Now Start RStudio or just log on at Workflow: The Source Pane in RStudio An R Script is available to make it easy for you to copy/paste all the tutorial commands into RStudio: Download RBootcamp1B Rcode.R and upload to rstudio.uchicago.edu Go to the Files tab and double-click on the filename. To run code, highlight line(s) of R code and click CTL-Enter (Windows) or CMD-Enter (Mac). Read in the file datafile.csv mydata <- read.csv("datafile.csv") and then run the R code below from the R Script RBootcamp1B Rcode.R. OK. Here we go! Every command in R is a function: mean(), histogram(), var(),... The R function syntax for indicating y-axis and x-axis variables is typically either 1. a list of variables: functionname(x,y) or 2. a formula y = f(x), entered like this: functionname(y ~ x). The symbol is called a tilde or a squiggle. On your keyboard, upper left under the esc key. Formula-based notation makes it easy to summarize results by groups (categories). We recommend using the following formula-based packages for Numerical summaries: mosaic package (augments many base R functions) Graphical summaries: ggformula package (formula-based interface for ggplot2 graphics). Example: Convincing you that formula-based functions are easier Here is an example of getting the mean of a quantitative variable (y) across several groups (a categorical variable, g). The categorical variable g has three levels ("Left", "Right", "Neither"). Using base R things look rather ugly: R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 3 of 13

4 head(mydata, 12) mean(mydata$y[ mydata$g == "Left" ]) mean(mydata$y[ mydata$g == "Right" ]) mean(mydata$y[ mydata$g == "Neither" ]) Of course, you could Google and find this (opaque, but shorter) code using base R: tapply(mydata$y, mydata$g, mean) Strong recommendation: Use the mosaic package instead, which elegantly extends the abilities of base R functions: mean(y ~ g, data = mydata) This is transparent. The mean of y for each group in g: easier to read, understand, and remember. Numerical Summaries The mosaic package augments most of the basic numerical summaries to allow formula-based input. library(mosaic) # sample average, sd, and variance mean(~ y, data = mydata) sd(~ y, data = mydata) var(~ y, data = mydata) # five-number summary: min, Q1, median, Q3, max min(~ y, data = mydata) quantile(~ y, data = mydata, probs = 0.25) median(~ y, data = mydata) quantile(~ y, data = mydata, probs = 0.75) max(~ y, data = mydata) # another way to get the median quantile(~ y, data = mydata, probs = 0.50) # getting the five-number summary from one function fivenum(~ y, data = mydata) quantile(~ y, data = mydata, probs = c(0, 0.25, 0.50, 0.75, 1)) # favstats is a function in the mosaic pkg (not available in base R) favstats(~ y, data = mydata) # separate summaries of a quantitative variable by groups/categories mean(~ y g, data = mydata) sd(~ y g, data = mydata) favstats(~ y g, data = mydata) R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 4 of 13

5 # summaries of y for each combination of values from two sets of groups/categories mean(~ y g1 + g2, data = mydata) sd(~ y g1 + g2, data = mydata) favstats(~ y g1 + g2, data = mydata) Graphical Summaries We highly recommend using the ggformula package for plotting. Some ggformula tutorials are available, but in this R Bootcamp you will see pretty much all syntax you will need for intro stats. The ggformula authors provide a tutorial: org/web/packages/ggformula/vignettes/ggformula.html After demonstrating ggformula plots, we will recreate some of the same plots using base R. We think you should at least know that base R does have plotting capabilities, because if you ask someone for help iwht R, they may not be familiar with recent developments (like the ggformula package). However, you ll see pretty quickly from the base R examples, that ggformula plotting is much, much simpler and feature-rich. Basic syntax for ggformula functions: library(ggformula) # This is not real R code. It will not work. It's just a template. gf_plottype(formula, data = mydata) %>% gf_plotdecorator1( options ) %>% gf_plotdecorator2( options ) The elements of the plot are chained together by the %>% operator plottype is a keyword for a type of plot, for example, dotplot (common options: binwidth, dotsize, fill = varname) boxplot (common option: outlier.size = 0.5 or outlier.size = 1.5) bar (a bar plot of counts for categorical variables) histogram (common options: bins = 12, color = "white") point (a scatterplot with nifty options: color = g, size = x) qq (quantile plot with default option: distribution = qnorm) qqline (quantile plot reference line with default option: distribution = qnorm) and common plotdecorator functions modify size and color, add lines and text: gf labs(x = "x-axis label", y = "y-axis label") gf lims(x = c(1, 5), y = c(10, 40)) (set the limits of the axes) gf hline(yintercept = 7, linetype = "dashed") (add a dashed horizontal line at y=7) gf vline(xintercept = 4, linetype = "dotted", size = 2) (add a thick, dotted vertical line) gf abline(intercept = 3, slope = 2) (add a line with intercept and slope specified R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 5 of 13

6 gf coefline(model = lm(y and more... x, data = mydata)) (add a least-squares regression line) Reminder: Always include clarifying axis labels on all graphs using options like: gf_labs(x = "x-axis label (with units)", y = "y-axis label (with units)") Dotplots and Boxplots # dotplot colored by group membership gf_dotplot(~ y, data = mydata, fill = ~ g2, binwidth = 3, binpositions = "all") # separate boxplots for each group/category gf_boxplot(y ~ g, data = mydata, outlier.size = 0.75) %>% gf_labs(x = "x-axis label", y = "y-axis label (with units)") # swap (flip) the x and y axes (horizontal boxplots) gf_boxplot(y ~ g, data = mydata, outlier.size = 1.5) %>% gf_labs(x = "y-axis label", y = "x-axis label (with units)") + coord_flip() # Note: used "+" not "%>%" to "chain" since coord_flip() is in ggplot2, but not in ggformula We do NOT recommend it, but someone will likely try to help you with dotplots and boxplots by showing you base R plotting with stripchart() and boxplot(). We demonstrate these functions at the end of this document. Barplots # barplot of counts for categorical variable gf_bar(~ g, data = mydata) # separate barplots by categories of another variable, colored by a 3rd variable gf_bar(~ g g1, data = mydata, fill = ~ g2, position = "stack") gf_bar(~ g g1, data = mydata, fill = ~ g2, position = "dodge") gf_bar(~ g g1, data = mydata, fill = ~ g2, position = "fill") We do NOT recommend it, but someone will likely try to help you with barplots by showing you base R plotting with barplot(). Consider trying to recreate a plot like this in base R: gf_bar(~ g g1, data = mydata, fill = ~ g2, position = "stack") See the base R barplot difficulty level yourself at Histograms R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 6 of 13

7 # histogram of counts for quantitative variable gf_histogram(~ x, data = mydata, bins = 20, color = "white") # or, more explicitly asking for counts... gf_histogram(..count.. ~ x, data = mydata, bins = 20, color = "white") # separate histograms for each group/category gf_histogram(~ x g, data = mydata, bins = 15, color = "white") # or stack the histograms for easier comparison gf_histogram(~ x g, data = mydata, bins = 15, color = "white") + facet_wrap(~ g, ncol=1) # Note: used "+" not "%>%" to "chain" since facet_wrap() is in ggplot2, but not in ggformula # ggformula has not translated *everything* you might want from ggplot2 #...and yes, we had to Google to even find this syntax We do NOT recommend it, but someone will likely try to help you with histograms by showing you base R plotting with hist(). Consider trying to re-create a plot like this in base R: gf_histogram(~ x g, data = mydata, bins = 15, color = "white") See the base R histogram difficulty yourself at Scatterplots # scatterplot of y vs. x gf_point(y ~ x, data = mydata) %>% gf_labs(x = "x-axis label (with units)", y = "y-axis label (with units)") # separate scatterplots for each group/category gf_point(y ~ x g, data = mydata) # one plot, color indicates group/catetory gf_point(y ~ x, color = ~ g, data = mydata) # one plot, color indicates group (categorical), size indicate a 3rd quantitative variable gf_point(y ~ x, size = ~ x3, color = ~ g, alpha = 0.5, data = mydata) # add horizontal, vertical, and intercept/slope lines to a scatterplot gf_point(y ~ x, data = mydata) %>% gf_hline(yintercept = mean(~ y, data = mydata), linetype = "dotted") %>% gf_vline(xintercept = mean(~ x, data = mydata), size = 1.5) %>% gf_abline(intercept = 41, slope = 0.09, linetype = "dashed") R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 7 of 13

8 # For illustraton, get the point (0,0) onto the graph to see the intercept = 40 gf_point(y ~ x, data = mydata) %>% gf_lims(x = c(0, max(~ x, data = mydata)), y = c(0, max(~ y, data = mydata))) %>% gf_abline(intercept = 41, slope = 0.09, linetype = "dashed") # add a least-squares regression line to a scatterplot gf_point(y ~ x, data = mydata) %>% gf_abline(intercept = 41, slope = 0.09, linetype = "dashed") %>% gf_coefline(model = lm(y ~ x, data = mydata), size = 1.5) We do NOT recommend it, but someone will likely try to help you with scatterplots by showing you base R plotting with plot(). We demonstrate these functions at the end of this document. Displaying Several Plots Together The gridextra package has a function called grid.arrange that works nicely. Save each plot first, then display them with the grid.arrange function. plot1 <- gf_histogram(~ y g2, data = mydata, bins = 12, color = "white") + facet_wrap(~ g2, ncol = 1) plot2 <- gf_histogram(~ x g2, data = mydata, bins = 8, color = "white") plot3 <- gf_dotplot(~ x3, data = mydata, binwidth = 2, dotsize = 0.8, fill = ~ g2, binpositions = "all") plot4 <- gf_point(y ~ x, size = ~ x3, color = ~ g2, alpha = 0.7, data = mydata) library(gridextra) grid.arrange(plot1, plot2, ncol = 2) grid.arrange(plot3, plot4, nrow = 2) STOP R Bootcamp Part I(B) Presentation Here Here are more advanced plot types you for your reference later in an introductory statistics course. Then we show examples of some of the plots in this document created using base R plotting functions. Quantile Plots # normal quantile plot gf_qq(~ y, data = mydata) %>% gf_qqline() R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 8 of 13

9 # or, equivalently gf_qq(~ y, data = mydata, distribution = qnorm) %>% gf_qqline(distribution = qnorm) # standardize the data (a good idea to make normal quantile plots comparable) mydata <- mutate(mydata, z = (y - mean(y)) / sd(y)) # normal quantile plot with data standardized gf_qq(~ z, data = mydata, distribution = qnorm) %>% gf_qqline(distribution = qnorm) # or compare the data to a different distribution: here, t-distn (df = 5) gf_qq(~ z, data = mydata, distribution = qt, dparams = list(df = 5)) %>% gf_qqline(distribution = qt, dparams = list(df = 5)) # separate normal quantile plots for each group/category gf_qq(~ z g, data = mydata, distribution = qnorm) %>% gf_qqline(distribution = qnorm) We do NOT recommend it, but someone will likely try to help you with normal quantile plots by showing you base R plotting with qqnorm(). Getting quantile plots for other (non-normal) distributions (like the t-distribution demonstrated above) requires other base R functions (like qqplot) and the set-up of temporary variables (it s not pretty). We demonstrate the qqnorm function at the end of this document. Simple Linear Regression Plots It s OK if you don t know what linear regression is...yet # find the least squares regression equation and save the results myfit <- lm(y ~ x, data = mydata) # examine the results summary(myfit) coef(myfit) library(broom) tidy(myfit, conf.int = TRUE, conf.level = 0.95) glance(summary(myfit)) # find the SSE for a model: Recall sigmahat = SSE / df sigmahat <- as.numeric( glance(summary(myfit))[3] ) df <- as.numeric( glance(summary(myfit))[6] ) R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 9 of 13

10 SSE <- sigmahat * df c(sigmahat, df, SSE) # view the results data.frame(sigmahat, df, SSE) # fancier view of the results # make a scatterplot of y vs. x and add the regression line to the plot gf_point(y ~ x, data = mydata) %>% gf_coefline(model = myfit) # add the fitted values, residuals, and stndardized residuals to dataset mydata <- mutate(mydata, fitted = fitted(myfit), resids = residuals(myfit), sresids = rstandard(myfit) ) # plot the residuals vs. fitted values (with dashed line at y = 0) gf_point(resids ~ fitted, data = mydata) %>% gf_hline(yintercept = 0, linetype = "dashed") %>% gf_labs(x = "fitted values", y = "residuals") # plot the standardized residuals vs. fitted values (with dashed lines at y = -2, 0, 2) gf_point(sresids ~ fitted, data = mydata) %>% gf_hline(yintercept = c(-2, 0, 2), linetype = "dashed") %>% gf_labs(x = "fitted values", y = "studentized residuals") NOT recommended: base R plots Here, we try to recreate some of the plots above using base R. If time, some plots will be demonstrated during the R Bootcamp especially consider that we gave up on creating a three-variable barplot and we encountered complexities for separate histograms by category below. Of course, you can try these plots later, on your own. We are confident you will quickly get the idea that ggformula plotting is so much easier and allows greater complexity and flexibility in plotting. Take-Home Message: Use ggformula plotting, not base R. Base R Dotplots and Boxplots There is no dotplot function in base R, but We Googled and found stripchart() in base R. # dotplot separated by group membership (We didn't succeed at matching gf_dotplot output) stripchart(mydata$y ~ mydata$g, pch=20, method="stack", horizontal=true, xlab = "x-axis label") R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 10 of 13

11 # separate boxplots for each group/category boxplot(mydata$y ~ mydata$g, pch=20, cex=1.5, xlab = "x-axis label") # swap (flip) the x and y axes (horizontal boxplots) boxplot(mydata$y ~ mydata$g, pch=20, cex=1.5, xlab = "x-axis label", horizontal=true) Base R Barplots and Histograms # barplot of counts for categorical variable counts <- table(mydata$g) barplot(counts) # separate barplots by categories of another variable, colored by a 3rd variable counts <- table(mydata$g, mydata$g1) barplot(counts, legend=unique(mydata$g), col=unique(mydata$g)) We re not even going to try adding a third variable to a barplot in base R. See the difficulty level yourself at # histogram of counts for quantitative variable hist(mydata$x, xlab = "x-axis label", breaks = 20, col = "grey", main="") # or, more explicitly asking for counts... hist(mydata$x, freq = TRUE, xlab = "x-axis label", breaks = 20, col = "grey", main="") Separate histograms for each group/category table(mydata$g) # Just reminding myself of the categories in variable g par(mfrow=c(1,3)) # set up plotting space for 1 row of 3 plots hist(mydata$x[mydata$g == "Left"], xlab = "Left", breaks = 12, col = "grey", main="") hist(mydata$x[mydata$g == "Neither"], xlab = "Neither", breaks = 12, col = "grey", main="") hist(mydata$x[mydata$g == "Right"], xlab = "Right", breaks = 12, col = "grey", main="") par(mfrow=c(1,1)) # return the plotting space to 1 plot per page Unfortunately, these plots are not on the same scale....and whenever you are asked to compare graphs, you are required to display them using the same scale (or you won t get full credit on homework). So, we ll try to fix the problem in Base R. After looking at the 3 histograms above, now that we can see about how tall the tallest plot is. We ll use that information to make all 3 plots that tall. R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 11 of 13

12 par(mfrow=c(1,3)) # set up plotting space for 1 row of 3 plots hist(mydata$x[mydata$g == "Left"], xlab = "Left", breaks = 12, col = "grey", main="", ylim = c(0, 15)) hist(mydata$x[mydata$g == "Neither"], xlab = "Neither", breaks = 12, col = "grey", main="", ylim = c(0, 15)) hist(mydata$x[mydata$g == "Right"], xlab = "Right", breaks = 12, col = "grey", main="", ylim = c(0, 15)) par(mfrow=c(1,1)) # return the plotting space to 1 plot per page Reminder: Separate histograms for each group/catetory is just one line using ggformula. Plus, all three plots are automatically on the same scale for easy comparison. gf_histogram(~ x g, data = mydata, bins = 12, color = "white") # or stack the histograms for easier comparison par(mfrow=c(3,1)) # set up plotting space for 3 rows of 1 plot each hist(mydata$x[mydata$g == "Left"], xlab = "Left", breaks = 12, col = "grey", main="", ylim = c(0, 15)) hist(mydata$x[mydata$g == "Neither"], xlab = "Neither", breaks = 12, col = "grey", main="", ylim = c(0, 15)) hist(mydata$x[mydata$g == "Right"], xlab = "Right", breaks = 12, col = "grey", main="", ylim = c(0, 15)) par(mfrow=c(1,1)) # return the plotting space to 1 plot per page Base R Scatterplots # scatterplot of y vs. x plot(mydata$x, mydata$y, pch=20, xlab = "x-axis label (with units)", ylab = "y-axis label (with units)") # separate scatterplots for each group/category table(mydata$g) # Just reminding myself of the category names par(mfrow=c(1,3)) # set up plotting space for 1 row of 3 plots plot(mydata$x[mydata$g == "Left"], mydata$y[mydata$g == "Left"], pch=20, xlab = "x-axis label (with units)", ylab = "y-axis label (with units)", main = "Left") plot(mydata$x[mydata$g == "Neither"], mydata$y[mydata$g == "Neither"], pch=20, xlab = "x-axis label (with units)", ylab = "y-axis label (with units)", main = "Neither") plot(mydata$x[mydata$g == "Right"], mydata$y[mydata$g == "Right"], pch=20, xlab = "x-axis label (with units)", ylab = "y-axis label (with units)", main = "Right") par(mfrow=c(1,1)) # return the plotting space to 1 plot per page Again, the plots are not on the same scale and so not directly comparable (and this would not receive full credit on homework). This is crazy! First, we had to type in the values of the categorical variable R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 12 of 13

13 myself; typed 3 times for each value! Then, the plots are not even on the same scale, which does not get you full credit on homework. Recall that with ggformula, just one line to do it all! gf point(y x g, data = mydata) We re going to stop here and not bother trying any of the more difficult scatterplots using just base R. We ll stick with ggformula plotting and we strongly recommend you do the same. Base R Quantile Plots # normal quantile plot qqnorm(mydata$y, pch = 20) qqline(mydata$y) # standardize the data (a good idea to make normal quantile plots comparable) mydata <- mutate(mydata, z = (y - mean(y)) / sd(y)) # normal quantile plot with data standardized qqnorm(mydata$z, pch = 20) qqline(mydata$z) You already know, from demonstrations of hist() and plot() above, that plotting separately by group is difficult in base R. We ll not attempt it for quantile plots. Base R Displaying Several Plots of Different Types Together We re not sure how to do this in base R. Can it be done??? grid.arrange() does not work with the plot types created by base R. We could Google, but the answer would be so difficult, we would lose interest. We ll stick with ggformula plots and use the grid.arrange function to display them together. Base R Simple Linear Regression Plots We are tiring of base R plotting at this point, so we ll stop now. Take-Home Message: Use ggformula plotting, not base R. R Bootcamp: Part 1(B) Revised: March 30, 2018 (2:26pm) 13 of 13