LECTURE NOTES FOR ECO231 COMPUTER APPLICATIONS I. Part Two. Introduction to R Programming. RStudio. November Written by. N.

Transcription

1 LECTURE NOTES FOR ECO231 COMPUTER APPLICATIONS I Part Two Introduction to R Programming RStudio November 2016 Written by N.Nilgün Çokça

2

3 Introduction to R Programming 5 Installing R & RStudio 5 The R Studio Layout 7 History Tab 8 Environment (Workspace) Tab 8 R Script 8 File Tab 9 Plots Tab 9 Packages Tab 10 Help Tab 11 Viewer Tab 12 Changing the Working Directory 12 Using The Console for Basic Calculations 13 Vectors 15 What is Vector? 15 Summary-type Statistics 17 Matrices & Data Frames 18 Importing data from Excel to RStudio 20 Generic Summary of the Data 23 Use of Square Bracket 23 Multiple Logical Statement 24 Summary Statistics in R Studio 26 Plots in R Studio 27 Scatterplots 27 Barchart 29 Box Plots 30 Histogram in R Studio 31

4

5 Introduction to R Programming Installing R & RStudio R is the name of the programming language itself and R Studio is a convenient interface. RStudio is a free and open source integrated development environment (IDE) for R, a programming language for statistical computing and graphics. RStudio is available in two editions: RStudio Desktop, where the program is run locally as a regular desktop application; and RStudio Server, which allows accessing RStudio using a web browser while it is running on a remote Linux server. Prepackaged distributions of RStudio Desktop are available for Microsoft Windows, Mac OS X, and Linux. If you don t have either R or R Studio installed, please go to to download the appropriate version of R for your computer, and for R Studio. These are relatively large installs so they make take a few minutes to download and install. Both should still run smoothly even on older computers too. You can install RStudio to your computer after installing the R Project. And we already learn about how to install R (Lecturer Notes: Part One) The Official website for R Studio is At the end of this lecture you should be able to: ~ Open R Studio ~ Load data into R ~ Understand the different types of variables ~ Get some basic plots and summary statistics ~ Save your commands for later use.

6 Please select suitable version for your work. There is no difference installing RStudio to Mac or Windows. Select operating system, and install RStudio to your computer. 6

7 The R Studio Layout I. The Console is where you can type commands and see output II. The Environment (Workspace) tab shows all the active objects. The History tab shows a list of commands used so far. III. The Files tab shows all the files and folders in your default workspace as if you were on a PC/Mac window. The Plots tab will show all your graphs. The Packages tab will list a series of packages or add-ons needed to run certain processes. For additional info see the help tab. 7

8 RStudio has 3 windows, Console that we write functions and calculations. Environment shows values and results of Console. On the last window you could see the menu bars of Files, Plots, Packages, Help and Viewer. History Tab The history tab keeps a record of all previous commands. It helps when testing and running processes. Here you can either save the whole list or you can select the commands you want and send them to an R Script to keep track of your work. Environment (Workspace) Tab The workspace tab stores any object, value, function or anything you create during your R session. Here is another example on how the workspace looks like when more objects are added. R Script The usual R Studio screen has four windows: 1. Console. 2. Workspace and history. 3. Files, plots, packages and help. 4. The R script(s) and data view. The R script is where you keep a record of your work. For Stata users this would be like the dofile, for SPSS users is like the syntax and for SAS users the SAS program. 8

9 To create a new R script, you can either go to File -> New -> R script, or click on the icon with the + sign and select R Script, or simply press Ctrl+Shift+N. Make sure to save the script. File Tab The Files & directory structure of your current working directory. Plots Tab The plots tab will display the graphs. If you want to see more graph that you make, you can click to back arrow key on screen. To extract the graph, click on Export where you can save the file as an image (PNG, JPG, etc.) or as PDF, these options are useful when you only want to share the graph or use it in a LaTeX document. Probably, the easiest way to export a graph is by copying it to the clipboard and then paste it directly into your Word document. Export -> Copy to Clipboard Make sure to select Metafile 9

10 Paste it into your Word document. Your graph will look like this X D graphs will display on a separate screen. You won t be able to save it, but after moving it around, once you find the angle you want, you can screenshot it and paste it to Word document. In particular: Index 1. Ctrl-Enter (submit current line or selection in editor window to console) 2. Ctrl-1 (move cursor to editor window) 3. Ctrl-2 (move cursor to console) 4. Ctrl-6 (move cursor to plots) 5. Esc (interrupt currently executing command) Packages Tab The packages tab shows the list of add-ons included in the installation of RStudio. If checked, the package is loaded into R, if not, any command related to that package won t work, you will need select it. You can also install other add-ons by clicking on the Install Packages icon. 10

11 Installing a package; for example, installing the package rgl (useful to plot 3D images). It does not come with the original R install. Click on Install Packages, write the name in the pop-up window and click on Install. package. after installing R Studio is a computing environment where statistical techniques may be implemented. Packages are add-ons that can extend R s functionally and perform specific tasks covering a wide range of modern statistics packages are available in R Studio. > install.packages ( Ecdat ) Mirror Selection : 69 > install.packages () # will open packages lists. > library (Ecdat) If you type help for package > help (package=ecdat) # will open help menu for package on the right window. > remove.packages (Ecdat) # to remove package. By menu to install packages Tools Install package. Help Tab Where you can find the help pages for every function installed in your version of R. 11

12 Viewer Tab Used for displaying locally generated.html content. Changing the Working Directory Working directories are useful for keeping your work organized. A working directory is one spot (e.g. a folder) that you have created for saving all of your work. If you have different projects, you can change the working directory for that session. getwd ( ) setwd ( ) # shows the working directory (wd) # Changes the wd Every time you open RStudio, it goes to a default directory. You can change the default to a folder where you have your datafiles so you do not have to do it every time. In the menu go to Tools -> Options 12

13 Using The Console for Basic Calculations We will discuss assigning values to objects in R, basic arithmetic functions, and a few other handy things to know. RStudio is case sensitive like R Console. You will see that there are not many differences between R Console and RStudio. RStudio can be used as a calculator and also has a series of inbuilt functions. Note that any comment lines are prefixed with a hash symbol (#). R usually outputs any results prefixed by a number in square brackets. e.g. [1] Equal sign (=) or less than sign and dash (<-) are same for assigning values. Assigning 25 to x > x = 25 > x [1] 25 Assigning 9 to y > y <-9 Assigning x+y to z > z <-x+y > z [1] 34 As you see on Environment window the assigned letter shown. The environment window is Workspace of RStudio. 13

14 There are also a series of inbuilt functions in the R console, so let s have a look at some familiar ones to begin with. To find the square-root of a value, we can use the function sqrt ( ) or we could take it to the power of 0.5. Note that R is a case-sensitive language so sqrt is different to SQRT. sqrt(4) or alternatively 4^0.5 To take the power of something, we use the caret symbol (^) 2^3 Note that in the above, we passed the value 4 to the function sqrt by enclosing it in round brackets. this is how we pass data to a function in R. Spaces are typically not meaningful in R but the convention is to place these brackets directly after the function name, unlike in some command-line based functions. We also use square brackets [ ] and curly brackets { }. Similarly, there is an inbuilt function for finding the logarithm of a number. The default is to find the natural logarithm (i.e. ln) but we can change this by specifying the base as in the second example. log(8) [1] # Now try using here 2 log(8, 2) [1] 3 To see the variables in workspace you can use ls( ) list command. Or to clear any value you may type rm(z) to console. > ls () [1] x y z > rm(z) Object names in RStudio may include numbers but cannot start with numbers. You may name x1 but you cannot name 1x. Unexpected symbol in 1x Creating vectors, matrices, and performing some simple operations on them We may assign 11 to x, or we may create a vector x1 <- c(1,3,5,7,9) 14

15 Vectors Most of us are probably used to Excel where we can see all of the data by just scrolling down the spreadsheet. Unfortunately, Excel has strict limits on the size of your spreadsheet, whereas in R the only bounds are defined by the memory of your computer, and even then many packages have developed ways of enabling you to access dataset with millions of data points even on very limited computers. > x <-11 > x [1] 11 What is Vector? In R we often have data contained in vectors, and for these who start to get nervous around mathematics these are just a collection of values. More formally, they are objects in the Environment (or workspace) with are one-dimensional and all of the elements of the vector will be of the same data type. We can also think them as being analogous to a single column of data in an Excel spreadsheet. > x1 <- c(1, 3, 5, 7, 9) > x1 [1] A simple vector is the numbers 1 to 10, and we can obtain this vector just by going to the Console and typing: > 1:10 [1] Let s get a bit more serious now and move to the Script Window by entering Ctrl+1. Type the text x <- 1:10 then hit Ctrl+Enter. This will send the current line from the Script Window to the Console. # just display the second value in the vector x [2] [1] 2 15

16 # Show the first three values x[1:3] [1] > gender <-c( male, female ) > gender [1] male female We may create two vectors and work on these vectors X = 1, 2, 3, 4, 5 and Y = 1, 3, 5, 7, 9 > x <-1:5 > x [1] > > y<-c(1,3,5,7,9) > y [1] > > x+10 [1] > > x-10 [1] > > x*10 [1] > > x/2 [1] If two vectors are of the same length, we may add/substract/mult/div corresponding elements. We may add vectors or subtracts > x [1] > y [1] > > x+y [1] > > x-y [1]

17 > > x*y [1] > > x/y [1] Summary-type Statistics We can obtain the simple summary types of statistics for our vector x using some relatively intuitive commands. A few example is: max(x) min(x) sum(x) mean(x) median(x) Each number in the vector x is referred to as an element of x, and R is particularly useful for performing the same operation on every element within a vector, e.g. x/2 n <-length (x) y <- rep(0,n) # find the length of a vector # create an empty vector to hold the results Descriptive Statistics (location) Mean The mean is the sum of the observations divided by the total number of observations. X = X i n Excel R-R Studio =AVERAGE(range of cells) mean(x) Median The median is another measure of central tendency. To get the median you have to order the data from lowest to highest. The median is the number in the middle. If the number of cases is odd the median is the single value, for even number of cases the median is the average of the two numbers in the middle Excel R-R Studio =MEDIAN(range of cells) median(x) Descriptive Statistics (variability) Variance The variance measures the dispersion of the data from the mean. It is the simple mean of the squared distance from the mean. s 2 = (X i X ) 2 n 1 Excel R-R Studio =VAR(range of cells) var(x) 17

18 Standard Deviation The standard deviation is the squared root of the variance. Indicates have close the data is to the mean. Assuming a normal distribution:. 68% of the value are within 1 sd (.99). 95% within 2 sd (1.96). 99% within 3 sd (2.58) s = (X i X ) 2 n 1 Excel R-R Studio =STDEV(range of cells) sd(x) Range Range is a measure of dispersion. It is simple the difference between the largest and smallest value, max - min. range=(max(x)-min(x)) R-RStudio range Matrices & Data Frames Just as a vector was a collection of values of the same data types, a matrix is the same but with two dimensions. We can create a simple matrix by placing the following code in the Script window and hitting Ctrl+Enter. xmat <-matrix (1:20, ncol=4) Note that by specifying the number of columns the function automatically decided the number of rows, and that it filled the matrix down each column by default. Firstly, whilst a vector only has a length attribute, matrices have a length attribute and a dimension attribute. length(x) dim(x) length(xmat) dim(xmat) ıf we shift our attention to the Environment Tab in the upper-right window, we ll see that xmat is defined as integers but the square brackets now contain the values [1:15, 1:4]. The first values indicate the rows, whilst the second values denote the columns. Just like with a vector, we can extract individual rows, column or even single entries by using these indices. xmat[1, ] xmat[,1] xmat[2,3] # Show the first row # Show the first column # Show the 2 nd entry in the 3 rd column. 18

19 To create matrix, we use matrix function > matrix(c(1,2,3,4,5,6,7,8,9), nrow=3,byrow=true) [,1] [,2] [,3] [1,] [2,] [3,] > matrix(c(1,2,3,4,5,6,7,8,9), nrow=3,byrow=false) [,1] [,2] [,3] [1,] [2,] [3,] > mat<-matrix(c(1,2,3,4,5,6,7,8,9), nrow=3,byrow=true) > mat [,1] [,2] [,3] [1,] [2,] [3,] > mat[1,2] [1] 2 > mat[c(1,3),2] [1]

20 Importing data from Excel to RStudio R is a very powerful analytic environment with near limitless number of packages and functions. It s also a good idea to make sure you are now working in the Script Window Ctrl+1, and entering everything into the Console using Ctrl+Enter. To begin this section, open a new R Script File (Ctrl+Shift+N) and save it as ReadsData.R The steps outlined here are the same for most other spreadsheet applications. Two main options are to save data file as: Comma separated value: *.csv by default it will open in Excel Tab delimited text file: *.txt by default it will open in Text editor. Saving as.csv file easy way to go Let s save our Excel file to desktop as CSV file. To import data to RStudio open the program. Then you can write the command below: Name your dataset in RStudio as data1 >data1 <-read.csv(file.choose(), header=true) # with column headings Header=TRUE means R will take first row as header. >data1 The other way to write command as: >data2 <-read.table(file.choose(), header=t, sep=, ) 20

21 >data2 Let s save the same file as tab delimited text file. Save As Tab delimited text (.txt) In RStudio >data3 <-read.delim(file.choose(),header=t) >data3 You can still open this data as read.table command. >data4 <-read.table(file.choose(), header=t, sep= \t ) >data4 #Read in the Data Getting started with Data in RStudio RStudio let us to select file from ImportDataSet. By Import DataSet dialog box you can select your file. Name Heading Separator : Data name that you would like to select : If you have datanames in Excel file. : Let R know how data has separate. Whitespace Comma Semicolon Tab Decimal : which character will have specified decimal point. Period comma Quate : Double quate ( ) >view (DataSet) View command will open a spreadsheet view on top. You may remove other imported data sets (data1, data2, data3, ext.) >rm (data1) >rm (data2) 21

22 rm command will remove data set from workplace. Dim dimension of dataset >dim (DataSet) 725 rows 6 data >head (DataSet) >tail (DataSet) # first 6 rows # last 6 rows >DataSet [c(5,6,7,8,9), ] Show rows 5,6,7,8,9 and all columns >DataSet[5:9,] Show rows 5,6,7,8,9 and all columns >DataSet[-c(4:722),] Write all rows except 4 to 722 # Check the names >names (DataSet) Will give us name of variables in DataSet >mean (Age) error Error in mean(age) : object Age not found. That s because R doesn t recognize Age variable. There are two options here 1) >mean(dataset $Age) >DataSet$Age says RStudio where to find Age variable. 2) Attach(DataSet) Second way to work on DataSet easily, and you may use mean command: 3) >mean(age) >detach(dataset) will let you go back. It is personal preference to use any way to work on DataSet. > attach (LungCapData) > names (LungCapData) > class (LungCap) numeric 22

23 > class (Age) integer > class (Smoke) factor * * * > levels (Smoke) no yes Generic Summary of the Data > summary (LungCap) For numeric variables it gives mean, median, max etc. for factor variables summaries for frequency. > X <-c(0,1,1,1,0,0,0,0,0,0) > class (X) numeric > summary (X) min 1stQu Median Mean > X <-as.factor (X) > class (X) factor > summary (X) Use of Square Bracket > Age [11:14] # will give all observation for 11 up to 14 > LungCap [11:14,] # will give matrix of row 11 up to 14 for all columns. In RStudio = (equal sign) is used to assign values to objects. == (double equal sign) is used to represent the meaning of equality in a mathematical sense. We would like to calculate mean of Age we can use the square bracket to subset only females. > mean (Age[Gender00 female ]) > mean (Age [Gender== male ]) ) If we would like to create a dataset that contains only Female > FemData <-LungCapData[Gender== female, ] #space include all column 23

24 2) If we would like to create a dataset that contains only male > MaleData <-LungCapData[Gender== male, ] 3) Dimension of FemaleData > dim(femdata) ) Dimension of MaleData > dim (MaleData) > summary (Gender) Female Male > FemData [1:4,] # first 4 rows for all columns Let s take this one step further, we will subset data for male over 15 years old. > MaleOver15 <-LungCapData[Gender== male & Age>15, ] > dim (MaleOver15) 89 6 > MaleOver15[1:4, ] Multiple Logical Statement A very common question to ask of vector elements is whether they fit certain criteria. You could ask if the values are equal to a certain value, or if they are less than a certain value, and so on. Keeping with the idea of performing a simultaneous operation on each element of a vector, we can do this with our vectors. = or <- #assign something to a variable, e.g. x = c( a, b, b, e ) == #ask whether two things are equal, e.g. x = c(3,5,6,3) and then x == 3 gives T F F T #Then y[x == 3] gives those entries of y where x equals 3, i.e. the 1st and 4th entry of y > x = c(3,5,6,3) > x==3 24

25 [1] TRUE FALSE FALSE TRUE The following tables gives the six fundamental logical tests than can be performed on data. x<y x>y x<=y x>=y x==y x!=y One final logical test which can be extremely helpful for extracting data is coded using the command %in% which can be thought of as is in. It is uses a very similar syntax to the test above, but looks in a second vector for any values provided in the first. x%in%y This has given a logical vector with the values based on whether each specific value of x is in the vector y.!x%in%y # we have effectively asked, which values in x are not in y. Logic Statements, and a Few Other Random but Useful Commands in R Studio > Age[1:5] > temp <-Age>15 > temp [1:5] FALSE TRUE TRUE FALSE FALSE 0 : FALSE 1: TRUE > temp2 <-as.numeric(age>15) > temp2 [1:5] Female and Smoke to use > FemSmoke <-Gender== female & smoke== yes > FemSmoke [1:5] FALSE TRUE FALSE FALSE FALSE We can add this last statement to our DataSet. > MoreData <- cbind(lungcapdata, FemSmoke) > MoreData [1:5, ] e.g. 25

26 >mean (AGE[GENDER== female ]) [1] FemData <- example[gender== female,] Summary Statistics in R Studio Mean, Standard Deviation, Frequencies, etc. > mean(lungcap) > median(lungcap) > var(lungcap) > sd(lungcap) > sqrt(lungcap) > min(lungcap) > max(lungcap) > range(lungcap) > quantile(lungcap, probs=0,90) > quantile(lungcap, probs=c(0.20,0.5,0.9,1)) > sum(lungcap) > sum(lungcap)/725 > cor(lungcap, Age) > var(lungcap, Age) ANOVA is a parametric method appropriate for comparing the means for 2 or more independent populations. Pearson s correlation is used to examine the strength of the linear relationship between the 2 numeric variables. > cor(age, Height) 0.83 # fearly strong linear association. 26

27 Plots in R Studio An incredibly useful feature of the RStudio layout is the ability to scroll through all the plots you have created in your current session. Note the back arrow on the top left corner of the Plot window. We can use this, and the corresponding forwards arrow to look through everything we have done. When combined with the Export button at the top of this window this is pretty handy. Traditionally in R we save plots using code, and this is still a useful concept, particularly if you are developing plots from publication and issues like graphics resolution may require continued tweaking. However, clicking the Export button will give you the option of saving an image as a.pdf or any number of graphic formats. Traditionally we save in.png format and you can set the final image size in this window. Scatterplots Scatterplots are appropriate for examine the relationship between 2 numeric variables. By default, R creates a scatterplot with each x, y pair indicated by an open circle. The plot itself should appear under the plots tab of the lower right panel of RStudio. Notice that the command above again looks like a function, this time with two arguments separated by a comma. The first argument in the plot function specifies the variable for the x-axis and the second for the y-axis. If we wanted to connect the data points with lines, we could add a third argument, the letter l for line. > plot(x=lungdata$age, y=lungdata$heigth, type=l) To read what a function does and learn the arguments that are available to you, just type in a question mark followed by the name of the function that you are interested in?plot Notice that the help file replaces the plot in the lower right panel. You can toggle between plots and help files using the tabs at the top of that panel. Height and Age > attach(lungdata) > plot(age, Height) Height Age 27

28 > plot(age, Height, main= Scatterplot, xlab= AGE, ylab= Height ) > plot(age, Height, main= Scatterplot, xlab= AGE, ylab= Height, las=1) > plot(age, Height, main= Scatterplot, xlab= AGE, ylab= Height, las=1, xlim=c(0,25)) > plot(age, Height, main= Scatterplot, xlab= AGE, ylab= Height, las=1, xlim=c(0,25), cex=0,5) > plot(age, Height, main= Scatterplot, xlab= AGE, ylab= Height, las=1, xlim=c(0,25), pch=8, col=2) 80 Scatterplot Height > abline(lm(height~age)) AGE > abline(lm(height~age), col=4) 80 Scatterplot Height AGE 28

29 Barchart A bar chart is a visual display of the frequency for each category of a categorical variable or the relative frequency (%) for each category. > table (Gender) Gender female male > count <-table(gender) # to use later > table (Gender)/725 # percentage > percent <-table(gender)/725 # to use later > barplot(count) > barplot(percent) > barplot(percent, main= TITLE, xlab= Gender, ylab= % ) > barplot(percent, main= TITLE, xlab = Gender, ylab= %, las=1) > barplot(percent, main= TITLE, xlab = Gender, ylab= %, las=1, names.arg=c( Female, Male )) > barplot(percent, main= TITLE, xlab = Gender, ylab= %, las=1, names.arg=c( Female, Male ), horiz=true) TITLE Male % Female Gender 29

30 female male > pie(count) > pie(count, main= TITLE here ) Box Plots A boxplot is appropriate for summarizing the distribution of a numeric variable. It may be useful to compare all of the libraries visually using a boxplot. > boxplot(lungcap) > boxplot(lungcap, main= Boxplot, ylab= Lung Capacity ) > boxplot(lungcap, main= Boxplot, ylab= Lung Capacity, ylim=c(0,16)) > boxplot(lungcap, main= Boxplot, ylab= Lung Capacity, ylim=c(0,16), las=1) Boxplot 15 Lung Capacity > boxplot(lungcap ~ Gender) 30

31 > boxplot(lungcap ~ Gender, main= Boxplot by Gender ) Boxplot by Gender female male > boxplot(lungcap[gender== female ], LungCap[Gender== male ]) > quantile (LungCap, probs=c(0,0.25,0.5,0.75,1)) Histogram in R Studio A histogram is appropriate for summarizing the distribution of a numeric variable. R has default histogram() function. > hist(lungcap) > hist(lungcap, freq=false) > hist(lungcap, prob=t) > hist(lungcap, freq=false, ylim=c(0,0.2)) > hist(lungcap, freq=false, ylim=c(0,0.2), breaks=7) > hist(lungcap, freq=false, ylim=c(0,0.2), breaks=14) > hist(lungcap, freq=false, ylim=c(0,0.2), breaks=c(0,2,4,6,8,10,12,14,16)) > hist(lungcap, freq=false, ylim=c(0,0.2), breaks=seq(from=0, to=16, by=2)) > hist(lungcap, freq=false, ylim=c(0,0.2), breaks=14, main= Boxplot of Lung Capacity, xlab= Lung Capacity ) > hist(lungcap, freq=false, ylim=c(0,0.2), breaks=14, main= Boxplot of Lung Capacity, xlab= Lung Capacity, las=1) 31

32 Boxplot of Lung Capacity Density Lung Capacity > lines(density(lungcap)) > lines(density(lungcap), col=2, lwd=3) Boxplot of Lung Capacity Density Lung Capacity 32