Lecture 3: Basics of R Programming

Transcription

1 Lecture 3: Basics of R Programming This lecture introduces how to do things with R beyond simple commands. We will explore programming in R. What is programming? It is the act of instructing a computer to execute a task. That sounds pretty general, but today we ll see some specific examples of how to get R to do a task more complicated than rendering a graph or computing a mean. Outline: 1. R as a programming language 2. Grouping, loops and conditional execution 3. Creating your own functions Objectives By the end of this session students will be able to: 1. Perform basic data manipulations on vectors (numeric, logical, character); deal with missing values; index vectors; many-to-one, one-to-many merging 2. Use grouped expression and if-else statements: 3. Know how to write your own functions Trivia: In the U.S, what is the record snowfall in a 24-hour period? A couple odds and ends that may be of use to some of you: a) Reading in huge datasets b) Web Scraping c) Sorting use command sort(), e.g. > sort( c(1,55,-2,11) ) [1] d) Generating a random permutation of a set of data > sample(c("first","second","third","fourth"), replace=f) [1] "Fourth" "First" "Third" "Second" 1

2 e) Reading in individual xlsx sheets As mentioned, this can be a pain. Easiest method is to export as.csv, but another fine option is to use the XLConnect package: A Quick Review of Matrices and Data frames We discussed vectors, matrices and data structures in Lectures 1 and 2. Let us recall how to create a matrix of data from some given measurements, say heights and weights of 15 students. Suppose we have the data below: height weight We can read this into R using the following commands: > height = c(58,59,60,61,62,63,64,65,66,67,68,69,70,71,72) > weight = c(115,117,120,123,126,129,132,135,139,142,146,150,154,159,164) The above gives us two vectors with the height and weight. But it may be useful to have this as a matrix, so that each person s height and weight appear together. To do this, we can use the command: > htwtmatrix = matrix(c(height,weight),15,2) # what do 15 and 2 refer to? > htwtmatrix [,1] [,2] [1,] [2,] [3,] [4,] [5,] [6,] [7,] [8,] [9,] [10,] [11,] [12,] [13,] [14,] [15,]

3 What do you notice about how R creates a matrix from a vector? It constructs matrices column-wise by default, so if you want to create a matrix row-by-row, you need to give it an additional argument byrow=t. Exercise 1. How would you create a matrix that has height and weight as the two rows instead of columns? Look up the help on the matrix function if necessary. Now we have each person s height and weight together. However, for future reference, instead of storing the data as a matrix, it might be helpful to have column names with the data. Recall from lecture 1 that in order to assign column names, we first have to convert htwtmatrix to a data frame. A data frame has a unique set of row and column names. We use the command: > htwtdata = data.frame(htwtmatrix) > htwtdata X1 X (as.data.frame() works as well.) Notice that now the columns are named X1 and X2. We can now assign names to the columns by means of the names() command: > names(htwtdata) = c( height, weight ) 3

4 We can find the column names of a data frame, without opening up the whole data set, by typing in > names(htwtdata) [1] "height" "weight" Quick aside: Let s say we have a very large dataset and we don t want to search through the column names to find out the column number of a particular variable we re interested in. We can do this: > which(names(dataset.of.interest)=="column name") For example, with our two-column data frame htwtdata, to find the column number of weight, we can type > which(names(htwtdata)=="weight") [1] 2 > This is telling us that weight is the second column in the data frame. Let us recall how R operates on matrices, and how that compares to data frames. Recall that R evaluates functions over entire vectors (and matrices), avoiding the need for loops (more on this later). For example, what do the following commands do? > htwtmatrix*2 > htwtmatrix[,1]/12 # convert height in inches to feet > mean(htwtmatrix[,2]) To get the dimensions or number of rows or columns of a data frame, it is often useful to use one of the following commands: > dim(htwtdata) > nrow(htwtdata) > ncol(htwtdata) Exercise 2. What does the following R command do? 1 > htwtdata[,2]*703/htwtdata[,1]^2 1 See for a recent discussion on this health metric. 4

5 Exercise 3. How would you get R to give you the height and weight of the 8 th student in the data set? The 8 th and 10 th student? That was all a quick review. Now onto the new stuff: 3.2 Programming: loops, if-then/for/while statements So far we have mainly used R for performing one-line commands on vectors or matrices of data. One of the most powerful features of R is in being able to do programming, that is, automating a task. Today we will look at some simple yet powerful programming tools in R, such as loops, if-then and while statements. If/else statements In R, one can write a conditional statement as with syntax as follows: ifelse(condition on data, true value returned, false returned) The above expression reads: if condition assigned on the data is true, then do the true value operation, otherwise execute the false value. For example, > ifelse(3 > 4, x <- 5, x <- 6) > x [1] 6 The operators && and are often used to denote multiple conditions in an if statement. Whereas & (and) and (or) apply element-wise to vectors, && and apply to vectors of length one, and only evaluate their second argument in the sequence if necessary. Thus it is important to remember which logical operator to use in which situation. In general, operations across vectors will use the double-symbol convention, while loops will use the single-symbol convention. This will become clearer with some examples > hmean = mean(htwtdata$height) > wmean = mean(htwtdata$weight) > ifelse( hmean > 61 && wmean > 120, x <- 5, x <- 6) 5

6 > x [1] 5 Are hmean and wmean vectors of length 1? > htwt_cat<-ifelse (height>67 weight>150, high, low ) > htwt_cat [1] "low" "low" "low" "low" "low" "low" "low" "low" "low" "low" "high" [12] "high" "high" "high" "high" > htwt_cat<-ifelse (height>67 weight>150, high, low ) > htwt_cat [1] "low" (Notice that in the second ifelse statement only the first element in the series was computed.) If/else statements can be extended to include multiple conditions. Suppose we have the following data: final_score<- c(39, 51, 60, 65, 72, 78, 79, 83, 85, 85, 87, 89, 91, 95, 96, 97, 100, 100) passfail<-ifelse(final_score>=60, "pass", "fail") Suppose we want to create a variable called grades that is assigned as follows: F if final_score <60 D if 60 final_score<70 C if 70 final_score<80 B if 80 final_score<90 A if 90 final_score We can use a nested ifelse command as follows: grade <- ifelse(final_score<60,"f", ifelse (final_score<70,"d", ifelse(final_score<80,"c", ifelse (final_score<90,"b", "A")))) 6

7 If you have missing values in your vector (NA), it s not a problem. However (!), if you have some odd coding for missing values (-99 is common), what happens to grade for that value? This nested logical statements method is really useful for putting different colors in graphs for different conditions. Recall the Beijing air quality plot. 7

8 The code for the color section of the Beijing air graph in the plot() command reads: col = ifelse(pm25<=50,"green", ifelse(pm25<101,"yellow", ifelse(pm25<150,"orange", ifelse(pm25<201,"red", ifelse(pm25<301,"purple", "firebrick") ) ) ) ) Repetitive execution: for loops, repeat and while All of these examples are analogous to MACROS in SAS. We want to do something many times, without doing it by hand each time; we are automating the process. These methods can be greatly expanded to do dynamic, efficient, extremely complicated operations. The idea of this section is really just to get you familiar with how these programs do complicated operations. There is a for loop construction in R which has form > for (name in expr_1) execute expr_2 That means, for some subsection (name) of some set (expr_1), perform operation expr_2. Here is the simplest loop there is: > for(i in 1:12){print(i)} 8

9 Suppose, based on the ozone measurements in the airquality data set, we want to figure out which days were good air quality days (1) or bad air quality (0), based on a cutoff of ozone levels above 60ppb. Let us create a new vector called goodair, which stores the information on good and bad airquality days. We can do this using a for loop. > numdays = nrow(airquality) > numdays [1] 153 > goodair = numeric(numdays) # creates an object which will store the vector > for(i in 1:numdays) if (airquality$ozone[i] > 60) goodair[i] = 0 else goodair[i] = 1 ## (Notice that we have an if statement here within a for loop.) Does the command above work? Why/why not? Let us check the Ozone variable. What do you notice below? > airquality$ozone [1] NA NA [19] NA NA NA NA NA NA NA NA [37] NA 29 NA NA NA 23 NA NA NA NA NA [55] NA NA NA NA NA NA NA NA NA [73] NA NA NA [91] NA NA NA 22 [109] NA NA [127] [145] NA When there are missing values, many operations in R fail. One way to get around this is to create a new data frame that deletes all the rows corresponding to observations with missing rows. This can be done by means of the command na.omit > airqualfull = na.omit(airquality) > dim(airqualfull) [1] > dim(airquality) [1] # How many cases were deleted because of missing data? Sometimes deleting all cases with missing values is useful, and sometimes it is a horrible idea We could get around this without deleting missing cases with an ifelse statement within the for loop. (See R code for this method.) 9

10 Now let us try doing this again with the data with the complete cases. > numdays = nrow(airqualfull) > numdays [1] 111 > goodair = numeric(numdays) # initialize the vector > for(i in 1:numdays) if (airqualfull$ozone[i] >60) goodair[i] = 0 else goodair[i] = 1 > goodair [1] [38] [75] At this point we might be interested in which days were the ones with good air quality. The which command returns a set of indices corresponding to the condition specified. We can then use the indices to find the day of the month this corresponds to. > which(goodair == 1) ## notice the double = signs! > goodindices <- which(goodair == 1) > airqualfull[goodindices,] If we had wanted to keep the entire dataset airquality intact and not remove the observations with missing values, we could put a condition in the loop that deals with NA values. (Again: see R code.) Did we really need a loop?? I could have used an ifelse() statement instead of a loop to get the same results: > goodairifelse <- ifelse(is.na(airquality$ozone), NA, ifelse(airquality$ozone>60,1,0) 10

11 We can also index the rows as above in an easier fashion. We ll get to this in the section Conditional Indexing. Exercise 4. Suppose we want to define a day with good quality air (1) as one with ozone levels below 60ppb, and temperatures less than 80 degrees F. Use an ifelse() statement to do this with the airqualityfull dataset, and output the resulting subset (only values that meet these criteria) of the data to a file called goodquality.txt. Other looping options: WHILE & REPEAT Similar to a loop function, the while statement can be used to perform an operation while a given condition is true. For example: z <-0 while (z<5){ z<-z+2 print(z) } [1] 2 [1] 4 [1] 6 In the above while statement we initiate z to have a value of 0. We then state that as long as z is less than 5 we will continue to perform the following loop operation z<-z+2. Thus we have z <- 0+2 ##Initially z is 0, but after the first iteration of the loop the value of z is 2 z <- 2+2 ## After the second iteration the value of z is 4 z <- 4+2 ## After the third iteration the value of z is 6 The while statement stops here because now z is now bigger than 5. Another option for looping is the repeat function. An example follows: > i<-1 > repeat{ + print(i) + if( i == 15) break + i<-i+1 + } [1] 1 [1] 2 [1] 3 [1] 4 11

12 [1] 5 [1] 6 [1] 7 [1] 8 [1] 9 [1] 10 [1] 11 [1] 12 [1] 13 [1] 14 [1] 15 (I won t test you on any of this. I just want to give you a flavor for the options available.) Example of a useful LOOP (and a not-so-useful one): I m not sure if I can share this dataset with you or not, so I m just going to show you this code. > expression.data <- read.table("/users/avery/documents/classes/graduate/830 Microarray Data/data/data.cn.adh.dcis.clean.raw.txt", header=t, sep="\t") > dim(expression.data) #big data [1] There are 36 participants who have had genetic measurements (a microarray luminosity score) on 22k RNA expression levels. Note that rows and columns here are reversed from what we re used to: subjects are on the columns, not rows. The data of interest are at every third column, so I subset the data as follows: > col.signal <- seq(2,107,by=3) > data.signal <- data.frame(expression.data[, col.signal]) > dim(data.signal) [1] pl > > data.signal[10:15, 1:5] Person 1 Person 2 Person 3 Person 4 Person 5 Gene Gene Gene Gene Gene This is a made up example, but say we want to see if any of our participants were in fact just a replicate of Person 1. If any column was a replicate of column (person) 1, the expression levels 12

13 for each gene would be very, very close (about a 45 degree line on a plot). So we want to make a plot of every person s RNA expression level compared to person 1. (Again: this is a made up example, but serves the purposes of this lecture). We can run the following code to automate this: setwd("/users/avery/desktop/plots") for( i in 2:ncol(data.signal)){ png( paste("myplot_", i, ".png", sep="") ) plot(data.signal[,1],data.signal[,i], xlab="person 1 score", ylab=paste("person", i, sep=""),col="blue") dev.off() print(paste("plotting graph ", i)) } Don t pay attention to the specifics of the data. This is not a genetics class. The aim is just to have you see an automated script for generating multiple plots. (non-useful LOOP): Simulating # of ties in card game War (script not included in lecture) #when a loop is a big waste of effort > x<- seq(1:20) > x.sq<- numeric(20) > for(i in 1:20){ x.sq[i] <- x[i]^2 } > x2 <- x^2 Conditional Indexing Data frames and matrices allow for conditional indexing in R, which is often very useful. Instead of creating the goodair vector using a loop or ifelse() statement, we could directly extract the good air quality days using conditional indexing, using the single command below: > airqualfull[airqualfull$ozone < 60,] 13

14 It is worthwhile to keep in mind that many things in R can be done avoiding loops, and using conditional indexing can save a lot of time and effort! However, there are other times when using loops may be the best (or only) way to do things. Conditional indexing is also useful for extracting groups of data from data frames, or splitting data into groups according to some criterion. For example, to get sets of ozone measurements for days with temperatures higher and lower than 80 degrees F, we can use: > split(airqualfull$ozone, airqualfull$temp < 80) $`FALSE` [1] [19] [37] $`TRUE` [1] [19] [37] [55] Exercise 5. Using conditional indexing, write an R command to replicate the results of exercise 4. (Hint: use conditional indexing with a single &, not double &&.) 3.3 Merging and Sorting Dataframes This topic has a lot to it, so I only cover the basics. There are plenty of online resources to do this on your own. There won t be any homework questions on this section. Say you have two datasets and you want to merge them based on an ID number. A simple example of merging these by variable ID follows: > dataset.1<-matrix(c(1,13,12,1, 2,12,10,2, 3,13,9,1, 4,9,8,2, 5,3,7,3, 6,5,6,1, 7,6,5,2, 8,5,5,3), ncol=4, byrow=t) > > dataset.1<-data.frame(dataset.1) > names(dataset.1)<-c("id","read 1","read 2","read 3") > dataset.1 ID read 1 read 2 read

15 > dataset.2<-matrix(c(1,12, 2,13, 3,3, 4,15, 5,31, 6,15, 7,4, 8,6, 9,22), ncol=2, byrow=t) > > dataset.2<-data.frame(dataset.2) > names(dataset.2)<-c("id","read 4") > dataset.2 ID read > dataset.merged <- merge(dataset.1,dataset.2,by="id", all.y=true) > dataset.merged ID read 1 read 2 read 3 read NA NA NA 22 Now, a nice simple example of many-to-one merging: > library(reshape) > my.test.2<-matrix(c(1,13,12,1, 1,12,10,2, 2,13,9,1, 2,9,8,2, 2,3,7,3, 3,5,6,1, 3,6,5,2, 3,5,5,3), ncol=4, byrow=t) #create sample data > my.test.2<-as.data.frame(my.test.2) #convert to a dataframe, a more robust format 15

16 #add column names > names(my.test.2)<-c("id","read A","read B","visit") #print it to get a look. This is in "long" format > my.test.2 ID read A read B visit #above is a matrix of repeated measures on same individual, different visits. The function below coerces the data frame into a 'wide' format, with one row per individual, renaming variables to account for missing values. > wide_mytest <- reshape(my.test.2, direction="wide",idvar="id",timevar="visit") > > wide_mytest ID read A.1 read B.1 read A.2 read B.2 read A.3 read B NA NA Finally, imagine you have a matrix you want to sort from smallest to largest of a particular column, but you want to keep each row intact. Do the following: > setwd("/users/avery/desktop/classes/720 spring 2015/classes/wk3/sorting matrices") > ZZ<-read.table("ZZ") p result x1 x #note each row here starts with an assigned number; this is NOT an actual column of (1,2,3, ), it s just a row designation. #now sort it via column named "p" > ZZ[ order(zz[,1]), ] 16

17 #or: > attach(zz) > ZZ[order(p),] p result x1 x Now the matrix is sorted by column p while keeping the structure of the dataframe. There won t be any homework questions on this section! I just wanted to show you all some of these techniques as a reference for later on. 3.4 Writing simple functions in R: Why and How The R language allows the user to create objects of mode function. These are true R functions that are stored in a special internal form and may be used in future expressions. By this tool, the language gains enormously in power, convenience and elegance, and learning to write useful functions is one of the main ways to make your use of R comfortable and productive. It should be emphasized that most of the functions supplied as part of the R system, such as mean(), var(), dim() and so on, are themselves written in R and thus do not differ materially from user written functions. A function is defined by an assignment of the form > f.name <- function(arg_1, arg_2,...) expression The expression is an R expression, (usually a grouped expression), that uses the arguments, arg_i, to calculate some value. The value of the expression is the value returned by the function. A call to the function then usually takes the form f.name(expr_1, expr_2,...) and may occur anywhere a function call is legitimate. 17

18 Simple functions As a first example, consider a function to calculate a one-sample t-statistic to test the null hypothesis that in the height and weight data set, the mean population weight is x lb, where x can be specified by the user. This is an artificial example, of course, since there are other, simpler ways of achieving the same end (we ll do this next class). > onesam <- function(y1, x) { n1 <- length(y1) ##sample size yb1 <- mean(y1) ##mean of y1 s1 <- var(y1) ##variance of y1 tstat <- (yb1 - x)/sqrt(s1/n1) ##computing t-statistic = (mean-x)/se tstat } With this function defined, you could perform one-sample t-tests using a call such as > t.statistic <- onesam(htwtdata$weight, 130); t.statistic To check whether this function works, compare it to running the actual t-test function inbuilt in R: > t.test(htwtdata$weight-130) A function can be called within a loop, or can be applied to elements of a vector or matrix at once, making R very powerful. We will continue looking at similar examples throughout the course. Another example: min.max.range <- function(x){ minimum<- min(x) r <- max(x) - min(x) maximum <- max(x) print(minimum) print(maximum) print(r) } vec.1<- c(10, 20, 50) min.max.range(vec.1) [1] 10 [1] 50 18

19 [1] 40 Exercise 6. Write a function called summarystat, which returns the mean, median, and standard deviation of a set of numbers. Recap: Review of matrix/dataframe operations if statements, ifelse(), Loops Conditional indexing (the most useful topic of this class) Merging & sorting dataframes (not on homework but very useful in real applications) Creating functions Reading: VS. Chapter 8.1, 8.2, 9 and 10 Assignment: Homework 2 due, Homework 3 assigned. (With extra credit for those interested in simulation.) 19