BIOSTAT640 R Lab1 for Spring 2016

Transcription

1 BIOSTAT640 R Lab1 for Spring 2016 Minming Li & Steele H. Valenzuela Feb.1, 2016 This is the first R lab session of course BIOSTAT640 at UMass during the Spring 2016 semester. I, Minming (Matt) Li, am going to use the R codes from some questions of the first 3 homework sets as examples for the R lab session. For Amherst class on Wed (02/03/2016), I am going to present this document, and during this talk, I am also going to cover the key points and personal suggestions in using RStudio, R Markdown, etc. Starting from Page14, it was written by Steele Valenzuela in 2015, which I also combined into this file. For HW1 questions after Q4, please refer to the other version of solution by Prof. Carol Bigelow. Here we will mainly use R to solve HW1: Q1-Q4. Q1: a <- c(5, 10, 6, 11, 5, 14, 30, 11, 17, 3, 9, 3, 8, 8, 5, 5, 7, 4, 3, 7, 9, 11, 11, 9, 4) breaks <- seq(0, 30, by=5) # set intervals # breaks a.cut <- cut(a, breaks, right=true) a.freq <- table(a.cut) # a.freq a.cumfreq <- cumsum(a.freq) # cumsum means the cumulative sums # a.cumfreq a.relfreq <- a.freq/length(a) # a.relfreq a.cumrelfreq <- cumsum(a.relfreq) # a.cumrelfreq Q1table <- cbind(a.freq, a.cumfreq, a.relfreq, a.cumrelfreq) colnames(q1table) <- c("freq", "Cum Freq", "Rel Freq", "Cum Rel Freq") Q1table Freq Cum Freq Rel Freq Cum Rel Freq (0,5] (5,10] (10,15] (15,20] (20,25] (25,30] Q2: Boxplot for cholesterol levels (mg/dl) for two groups of men 1

2 grp1 <- c(233, 291, 312, 250, 246, 197, 268, 224, 239, 239, 254, 276, 234, 181, 248, 252, 202, 218, 212, 325) grp2 <- c(344, 185, 263, 246, 224, 212, 188, 250, 148, 169, 226, 175, 242, 252, 153, 183, 137, 202, 194, 213) cholesterol <- cbind(grp1, grp2) # cholesterol # median(grp1) #242.5 # median(grp2) #207 # 2a: Side-by-side box plot boxplot(grp1, grp2) # 2b. Side-by-side histograms with same definitions (starting value, ending value, tick marks, etc) of t # summary(grp1) # summary(grp2) par(mfrow=c(1,2)) hist(grp1, breaks=seq(120, 360, by=20)) hist(grp2, breaks=seq(120, 360, by=20)) 2

3 Histogram of grp1 Histogram of grp2 Frequency Frequency grp grp2 Comparing the two distributions, the first group of men has higher median, Q1, Q3 and Min values than the second group, but has smaller Max than 2nd group. Q3: Prob(11 or more of the 64 exposed firefighters reporting breath shortness): 1-pbinom(10, 64, 1/22) [1] # OR WE CAN USE: sum(dbinom(11:64, 64, 1/22)) # Q4: 4a. What proportion of women have weights that are outside the ACES-II ejection seat acceptable range? pnorm(140, mean=143, sd=29) + (1-pnorm(211, mean=143, sd=29)) [1] b. In a sample of 1000 women, how many are expected to have weights below the 140 lb threshold? 3

4 1000*pnorm(140, mean=143, sd=29) [1] So, there are about 459 women that are expected to have weights below the 140 lb threshold. For HW2 Q1 and Q2, please refer to the other version of solution by Prof. Carol Bigelow. Here we will mainly use R to solve HW2-Q3. Q3: Let s download the dataset. library(foreign) url <- " dat <- read.dta(file = url) # read.dta: Read Stata Binary Files # dat head(dat) temp boiling x <- dat$temp y1 <- dat$boiling m1 <- lm(y1 ~ x) # OR: m1 <- lm(boiling~temp, data=dat) # OR: m1 <- lm(dat$boiling ~ dat$temp) summary(m1) Call: lm(formula = y1 ~ x) Residuals: Min 1Q Median 3Q Max

5 Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-14 *** x < 2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 29 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 29 DF, p-value: < 2.2e-16 m1$coefficients # output the intercept and slope (Intercept) x # confint(m1) # output the CI for the parameters in the fitted model anova(m1) Analysis of Variance Table Response: y1 Df Sum Sq Mean Sq F value Pr(>F) x < 2.2e-16 *** Residuals Signif. codes: 0 *** ** 0.01 * plot(x, y1, main="simple Linear Regression of Y=boiling on X=temp", xlab="temp", ylab="boiling", pch=18) # pch means "plot character", can be 1 to 18; abline(m1, col="red", lty=1, lwd=2) # lty: "line type"; lwd: "line width". 5

6 Simple Linear Regression of Y=boiling on X=temp boiling temp From the above summary(m1) output, we know that: (a): Parameter Estimates: and 0.44, so regression line: Y= *X; (b): ANOVA (Analysis of Variance) table is obtained above using the anova(m1) code; (c): R-square=0.9207; (d): The scatterplot with the fitted line is plotted above. (2). Now, instead of Y=boiling, we want to use newy=100*log10(boiling) newy=100*log10(y1) m2 <- lm(newy ~ x) summary(m2) Call: lm(formula = newy ~ x) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) *** x e-15 *** --- 6

7 Signif. codes: 0 *** ** 0.01 * Residual standard error: on 29 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 29 DF, p-value: 3.623e-15 m2$coefficients # output the intercept and slope (Intercept) x # confint(m2) # output the CI for the parameters in the fitted model anova(m2) Analysis of Variance Table Response: newy Df Sum Sq Mean Sq F value Pr(>F) x e-15 *** Residuals Signif. codes: 0 *** ** 0.01 * plot(x, newy, main="simple Linear Regression of Y=100*log10(boiling) on X=temp", xlab="temp", ylab="100*log10(boiling)", pch=18) # pch means "plot character", can be 1 to 18; abline(m2, col="red", lwd=2) # lwd means "line width" Simple Linear Regression of Y=100*log10(boiling) on X=temp 100*log10(boiling) temp 7

8 From the above summary(m2) output, we know that: (a): Parameter Estimates: and 0.93, so regression line: 100log10(Y)= *X; (b): ANOVA (Analysis of Variance) table is obtained above using the anova(m2) code; (c): R-square=0.8852; (d): The scatterplot with the fitted line is plotted above. Solution (one paragraph of text that is interpretation of analysis): Did you notice that the scatter plot of these data reveal two outlying values? Their inclusion may or may not be appropriate. If all n=31 data points are included in the analysis, then the model that explains more of the variability in boiling point is Y=boiling point modeled linearly in X=temp. It has a greater Rˆ2 (92.07% vs %). Be careful - It would not make sense to compare the residual mean squares of the two models because the scales of measurement involved are di erent. For HW3 Q1, Q4, Q5, please refer to the other version of solution by Prof. Carol Bigelow. Here we will mainly use R to solve Q2,Q3. Q3: Use R to reproduce th anova tables you worked with in Q2. Let s download the dataset. library(foreign) url <- " dat <- read.dta(file = url) # read.dta: Read Stata Binary Files # dat head(dat) id y x1 x dim(dat) [1] 15 4 str(dat) 8

9 data.frame : 15 obs. of 4 variables: $ id: num $ y : num $ x1: num $ x2: num attr(*, "datalabel")= chr "PubHlth 640 Unit 2 Regression - Larvae data" - attr(*, "time.stamp")= chr "10 Feb :36" - attr(*, "formats")= chr "%9.0g" "%9.0g" "%9.0g" "%9.0g" - attr(*, "types")= int attr(*, "val.labels")= chr "" "" "" "" - attr(*, "var.labels")= chr "larva id" "log10(survival)" "log10(dose)" "log10(weight)" - attr(*, "expansion.fields")=list of 2..$ : chr "_dta" "note1" "\"Week 3 homework assignment exercises 2 and 3\""..$ : chr "_dta" "note0" "1" - attr(*, "version")= int 12 summary(dat) id y x1 x2 Min. : 1.0 Min. :2.351 Min. : Min. : st Qu.: 4.5 1st Qu.: st Qu.: st Qu.: Median : 8.0 Median :2.452 Median : Median : Mean : 8.0 Mean :2.567 Mean : Mean : rd Qu.:11.5 3rd Qu.: rd Qu.: rd Qu.: Max. :15.0 Max. :2.966 Max. : Max. : # model regression on x1 alone m1 <- lm(y ~ x1, data=dat) summary(m1) Call: lm(formula = y ~ x1, data = dat) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-15 *** x e-05 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 13 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 13 DF, p-value: 7.944e-05 m1$coefficients # output the intercept and slope (Intercept) x

10 confint(m1) # output the CI for the parameters in the fitted model 2.5 % 97.5 % (Intercept) x anova(m1) Analysis of Variance Table Response: y Df Sum Sq Mean Sq F value Pr(>F) x e-05 *** Residuals Signif. codes: 0 *** ** 0.01 * # model regression on x2 alone m2 <- lm(y ~ x2, data=dat) summary(m2) Call: lm(formula = y ~ x2, data = dat) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-13 *** x *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 13 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 13 DF, p-value: m2$coefficients # output the intercept and slope (Intercept) x confint(m2) # output the CI for the parameters in the fitted model 2.5 % 97.5 % (Intercept) x

11 anova(m2) Analysis of Variance Table Response: y Df Sum Sq Mean Sq F value Pr(>F) x *** Residuals Signif. codes: 0 *** ** 0.01 * # model regression on x1 and x2 m3 <- lm(y ~ x1 + x2, data=dat) summary(m3) Call: lm(formula = y ~ x1 + x2, data = dat) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-13 *** x e-05 *** x *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 12 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 2 and 12 DF, p-value: 6.085e-07 m3$coefficients # output the intercept and slope (Intercept) x1 x confint(m3) # output the CI for the parameters in the fitted model 2.5 % 97.5 % (Intercept) x x anova(m3) Analysis of Variance Table 11

12 Response: y Df Sum Sq Mean Sq F value Pr(>F) x e-07 *** x *** Residuals Signif. codes: 0 *** ** 0.01 * From above, you see that there are 3 times that we input almost exactly the same codes, but on m1, or m2, or m3. Here is a good chance to use Function in R. model.regress <- function(x) { a <- summary(x) b <- x$coefficients c <- confint(x) d <- anova(x) print(a); print(b); print(c); print(d) # the last line typically specifies what one wants the function to print } m1 <- lm(y ~ x1, data=dat) m2 <- lm(y ~ x2, data=dat) m3 <- lm(y ~ x1 + x2, data=dat) model.regress(m1) Call: lm(formula = y ~ x1, data = dat) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-15 *** x e-05 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 13 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 13 DF, p-value: 7.944e-05 (Intercept) x % 97.5 % (Intercept)

13 x Analysis of Variance Table Response: y Df Sum Sq Mean Sq F value Pr(>F) x e-05 *** Residuals Signif. codes: 0 *** ** 0.01 * # model.regress(m2) # model.regress(m3) 13

14 First Session Steele H. Valenzuela January 2015 Contents Downloading and Installing R and RStudio (for PC/Mac) If you have an Ubuntu operating system (OS) or any other OS aside from a PC/Mac, then you must love Linux and/or working in the terminal. That s impressive and you most likely can skip ahead to the section titled First Session In order to download R, please see their website. The website appears underdeveloped but you re at the right spot. Under the heading Download and Install R, you ll see the links to download R for your respective OS. In order to download RStudio (a fancy graphical user interface [GUI]), please see their website. This website is not underdeveloped but rather crisp. Moving on, under the heading Installers for ALL Platforms, you ll see the links to download RStudio for your respective OS. For your respective OS, it should be the first or the second link. The best explanation to give as to why you need both is this: RStudio is the fancy and friendly GUI that keeps you sane and organized as well as providing an inviting workplace, hence the very point of a GUI. You CANNOT use RStudio without R. R is what is under the hood, the engine per se, and RStudio is the interior and exterior of your car/truck/italian moped. I am automobile illiterate so please don t make me continue with this metaphor. Launching and Exiting RStudio Launching RStudio For PC users, please do the following: Start > All Programs > RStudio For Mac users, please do the following: Applications > RStudio; or Click on the RStudio icon shortcut on your dock; or In spotlight search (Cmd + Spacebar; or the magnifying glass in the upper right corner), enter RStudio and press return. Once in awhile, you will be asked to update RStudio. Please do so. Exiting RStudio From the toolbar at the top, at the far left: Click RStudio. From the drop down menu, click Quit RStudio. For PC, the shortcut is [insert shortcut here]. For Macs, the shortcut is Cmd + Q. Toolbar and Windows Upon starting RStudio, it will be intimidating, but do not fear, your loyal TA is here. There is a lot to grasp but getting your feet wet is the best way to progress with R. 1

15 Toolbar highlight. There are many options present in the toolbar, but for now, there are 4 commands we will The first command is under the following: File > New File From there, one should see R Script. Similar to Stata s do-file, a script allows you to write notes, commands, and anything else one may desire for a session. The second command is near the first: File > New Project From there, a box will pop up, asking you to Create a Project From. Choose the first option, New Directory, followed by Empty Directory, and give your project a name. For the sake of this class, name your project by the respective week of the class or the session number. The third command is under the following: Tools > Import Dataset > From Text File... /From Web URL... Uploading the data can be painful and if it takes you longer than 10 minutes, please consult me or any online 2

16 help forums. Unless one is scraping data from Twitter or some other complex method, it should be relatively easy. If need be, copy and paste into excel and save as a comma separated value (CSV) file and then import. You ll also notice the shortcut symbol for Import Dataset, a sheet and an arrow. The fourth command is under the following: Tools > Global Options For now, Appearance and Pane Layout are readily available to change the appearance of RStudio, such as the size of the text, the color scheme of the script and code, and various other options. 3

17 RStudio Layout Upon starting RStudio, you will see this similar layout (enter image). One half of the computer monitor is devoted to the source code, scripts, and the console, which runs the code. As for the other half of the monitor, you will see an environment where objects are stored, such as datasets and functions as well as the name of your files, plots, packages, and other tabs that you may explore. First Session Key to Colors The default color theme for R is TextMate. If you recall, you may change the theme under Tools > Global Options > Appearances. 4

18 Green- These are comments. In R, commands that begin with a number sign/pound sign/hashtag are comments. Blue- These are commands. Each command begins with an angle bracket ( > ). Black- This is R output. Compare it with the output you get. Preliminaries (working directory, start log, input data) In order to start or track a log, there is something called git. You may have heard of GitHub. We ll talk about that at a later time. As for now, simply start a new file or script to save your notes and commands. Your working directory is the default location to save and generate files, datasets, etc. Here are two helpful commands. # The following command displays your current working directory. getwd(...) # default location will be where project is specified 5

19 # The following command changes your current working directory to whatever you set it to. setwd(...) # this is a pain to write out but it ll become useful R has the ability to import data from a web url as well as a text file (CSV, comma delimited or tab files, etc.). R contains packages, a unique set of commands and datasets, which are developed by users from all over the world. Before uploading a package into your library, you must install the respective package as such: install.packages( package name ). In order to upload the same dataset from the Stata handout, the following commands will ensue. library(foreign) # a package that translates Stata, SAS, or SPSS data into R stata <- " # Web URL, note.../ivf.dta ivf <- read.dta(file = stata) # ivf is the name of the dataset whereas read.dta is the command As I m sure you ve noticed, the symbol <- is how objects are stored in R. View Data Structure Here are commands to view the structure of the data. str(ivf) # the structure of dataset ivf data.frame : 641 obs. of 6 variables: $ id : num $ matage : int $ hyp : int $ gestwks: num $ sex : Factor w/ 2 levels "male","female": $ bweight: int attr(*, "datalabel")= chr "In Vitro Fertilization data" - attr(*, "time.stamp")= chr "27 Aug :11" - attr(*, "formats")= chr "%9.0g" "%8.0g" "%8.0g" "%9.0g"... - attr(*, "types")= int attr(*, "val.labels")= chr "" "" "" ""... - attr(*, "var.labels")= chr "identity number" "maternal age (years)" "hypertension (1=yes, 0=no)" - attr(*, "version")= int 7 - attr(*, "label.table")=list of 1..$ sex: Named int attr(*, "names")= chr "male" "female" dim(ivf) # displays the dimensions of dataset ivf [1] names(ivf) # displays variable/column names [1] "id" "matage" "hyp" "gestwks" "sex" "bweight" summary(ivf) # summary of ivf 6

20 id matage hyp gestwks sex Min. : 1 Min. :23 Min. :0.000 Min. :24.7 male :326 1st Qu.:161 1st Qu.:31 1st Qu.: st Qu.:38.0 female:315 Median :321 Median :34 Median :0.000 Median :39.1 Mean :321 Mean :34 Mean :0.139 Mean :38.7 3rd Qu.:481 3rd Qu.:37 3rd Qu.: rd Qu.:40.1 Max. :641 Max. :43 Max. :1.000 Max. :42.4 NA s :2 bweight Min. : 630 1st Qu.:2850 Median :3200 Mean :3129 3rd Qu.:3550 Max. :4650 summary(ivf$sex) # factoral summary of categorical variable sex male female summary(ivf$bweight) # numerical summary of variable bweight Min. 1st Qu. Median Mean 3rd Qu. Max Examine Data The simplest way to view your data is with the following command: View(ivf) # will display a table in the same pane as scripts To view snippets of your data, implement the following commands: head(ivf) # default display is 6 rows id matage hyp gestwks sex bweight female female female male female male 3260 tail(ivf, 10) # displays tail end of ivf, more specifically, the last 10 rows id matage hyp gestwks sex bweight female female female male

21 male female male female female male 2920 Lastly, you may view specific rows and columns as such: ivf[10, 6] # 10th row, 6th column ivf[325, ] # 325th row, all columns ivf[, 4] # all 641 rows, 4th column Single Variable Description syntax: In R, if one would like to highlight a single variable, implement the following mean(ivf$bweight) # dataset followed by $ followed by variable name [1] 3129 If one would like to explore a single variable, here are some commands: min(ivf$matage); max(ivf$matage) # similar to SAS, you may implement the a semi-colon to separate comman [1] 23 [1] 43 range(ivf$gestwks) # range [1] mean(ivf$gestwks); median(ivf$gestwks) # mean and median [1] [1] sd(ivf$gestwks) # standard deviation [1] 2.33 table(ivf$sex) # frequency male female

22 The base graphics in R are really just that, basic, but in the coming weeks, new graphical packages will be explained and implemented. For now, here are a few commands: hist(ivf$bweight) # histogram Histogram of ivf$bweight Frequency boxplot(ivf$bweight) # boxplot ivf$bweight stem(ivf$bweight) # stem and leaf plot...not sure if you ll need to ever use this 9

23 The decimal point is 2 digit(s) to the right of the Two Variable Descriptives Here are a few commands for two variable descriptives: table(ivf$sex, ivf$hyp) # 2-way table 0 1 male female table(ivf$sex, ivf$hyp)/641 # R does not have a good way to display percentages so it must be done manua 0 1 male female And some graphical commands for two variable descriptives: boxplot(ivf$gestwks ~ ivf$sex) # one continuous variable by one descriptive variable 10

24 male female plot(ivf$gestwks, ivf$bweight, main = "insert main title here") # simple plot command insert main title here ivf$bweight ivf$gestwks plot(ivf$gestwks, ivf$bweight, xlab = "x-axis label here", ylab = "y-axis label here") 11

25 y axis label here x axis label here We can already see that in terms of summary statistics, Stata can be quite useful, so no hard feelings if you use Stata in the beginning. Very Important Information In RStudio, under one of the panes, you will see a tab for files. Depending on your working directory, you may see your own files from another project. DO NOT DELETE THESE FILES AS THEY WILL BE PERMANENTLY DELETED. This happened to a student from a class last semester... There is a lot I have not included because there is simply too much material. If you are ever confused about a command, type the following:?plot # this will direct you to a file under the help tab??plot # a more general search containing the words after?? Lastly, there are many resources online, such as stackoverflow or just to name a few. Professor Nicholas Reich taught a course last semester in R and his website is at this link. 12