Data Visualization in R

Transcription

1 Data Visualization in R L. Torgo ltorgo@fc.up.pt Faculdade de Ciências / LIAAD-INESC TEC, LA Universidade do Porto Aug, 2017 Introduction Motivation for Data Visualization Humans are outstanding at detecting patterns and structures with their eyes Data visualization methods try to explore these capabilities In spite of all advantages visualization methods also have several problems, particularly with very large data sets L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

2 Introduction Outline of what we will learn Tools for univariate data Tools for bivariate data Tools for multivariate data Multidimensional scaling methods L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 R Graphics Introduction L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

3 Introduction to Standard Graphics Standard Graphics (the graphics package) R standard graphics, available through package graphics, includes several functions that provide standard statistical plots, like: Scatterplots Boxplots Piecharts Barplots etc. These graphs can be obtained tyipically by a single function call Example of a scatterplot plot(1:10,sin(1:10)) These graphs can be easily augmented by adding several elements to these graphs (lines, text, etc.) L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Graphics Devices Introduction to Standard Graphics R graphics functions produce output that depends on the active graphics device The default and more frequently used device is the screen There are many more graphical devices in R, like the pdf device, the jpeg device, etc. The user just needs to open (and in the end close) the graphics output device she/he wants. R takes care of producing the type of output required by the device This means that to produce a certain plot on the screen or as a GIF graphics file the R code is exactly the same. You only need to open the target output device before! Several devices may be open at the same time, but only one is the active device L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

4 A few examples A scatterplot Introduction to Standard Graphics plot(seq(0,4*pi,0.1),sin(seq(0,4*pi,0.1))) The same but stored on a jpeg file jpeg('exp.jpg') plot(seq(0,4*pi,0.1),sin(seq(0,4*pi,0.1))) dev.off() And now as a pdf file pdf('exp.pdf',width=6,height=6) plot(seq(0,4*pi,0.1),sin(seq(0,4*pi,0.1))) dev.off() L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Package ggplot2 Introduction to ggplot Package ggplot2 implements the ideas created by Wilkinson (2005) on a grammar of graphics This grammar is the result of a theoretical study on what is a statistical graphic ggplot2 builds upon this theory by implementing the concept of a layered grammar of graphics (Wickham, 2009) The grammar defines a statistical graphic as: a mapping from data into aesthetic attributes (color, shape, size, etc.) of geometric objects (points, lines, bars, etc.) L. Wilkinson (2005). The Grammar of Graphics. Springer. H. Wickham (2009). A layered grammar of graphics. Journal of Computational and Graphical Statistics. L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

5 Introduction to ggplot The Basics of the Grammar of Graphics Key elements of a statistical graphic: data aesthetic mappings geometric objects statistical transformations scales coordinate system faceting L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Aesthetic Mappings Introduction to ggplot Controls the relation between data variables and graphic variables map the Temperature variable of a data set into the x variable in a scatter plot map the Species of a plant into the colour of dots in a graphic etc. L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

6 Geometric Objects Introduction to ggplot Controls what is shown in the graphics show each observation by a point using the aesthetic mappings that map two variables in the data set into the x, y variables of the plot etc. L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Introduction to ggplot Statistical Transformations Allows us to calculate and do statistical analysis over the data in the plot Use the data and approximate it by a regression line on the x, y coordinates Count occurrences of certain values etc. L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

7 Introduction to ggplot Scales Maps the data values into values in the coordinate system of the graphics device L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Coordinate System Introduction to ggplot The coordinate system used to plot the data Cartesian Polar etc. L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

8 Introduction to ggplot Faceting Split the data into sub-groups and draw sub-graphs for each group L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Introduction to ggplot A Simple Example data(iris) library(ggplot2) ggplot(iris, aes(x=petal.length, y=sepal.length, colour=species ) ) + geom_point() Petal.Length Sepal.Length Species setosa versicolor virginica L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

9 Univariate Plots Distribution of Values of Nominal Variables The Distribution of Values of Nominal Variables The Barplot The Barplot is a graph whose main purpose is to display a set of values as heights of bars It can be used to display the frequency of occurrence of different values of a nominal variable as follows: First obtain the number of occurrences of each value Then use the Barplot to display these counts L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Univariate Plots Distribution of Values of Nominal Variables Barplots in base graphics barplot(table(sales$insp), main='distribution of the Types of Inspection') Distribution of the Types of Inspection 1e+05 2e+05 3e+05 ok unkn fraud L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

10 Univariate Plots Distribution of Values of Nominal Variables Barplots in ggplot2 ggplot(sales,aes(x=insp)) + geom_bar() + ggtitle('distribution of the Types of Inspection') 4e+05 Distribution of the Types of Inspection 3e+05 count 2e+05 1e+05 ok unkn fraud Insp L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Univariate Plots Distribution of Values of Nominal Variables Pie Charts Pie charts serve the same purpose as bar plots but present the information in the form of a pie. pie(table(sales$insp), main='distribution of the Types of Inspection') Distribution of the Types of Inspection unkn ok fraud Note: L.Torgo Avoid (FCUP - LIAAD pie / charts! UP) Data Visualization Aug, / 38

11 Univariate Plots Distribution of Values of Nominal Variables Pie Charts in ggplot ggplot(sales,aes(x=factor(1),fill=insp)) + geom_bar(width=1) + ggtitle('distribution of the Types of Inspection') + coord_polar(theta="y") Distribution of the Types of Inspection 4e+05 1 Insp factor(1) 3e+05 1e+05 ok unkn fraud 2e+05 count L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Univariate Plots Distribution of the Values of a Continuous Variable The Distribution of Values of a Continuous Variable The Histogram The Histogram is a graph whose main purpose is to display how the values of a continuous variable are distributed It is obtained as follows: First the range of the variable is divided into a set of bins (intervals of values) Then the number of occurrences of values on each bin is counted Then this number is displayed as a bar L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

12 Univariate Plots Distribution of the Values of a Continuous Variable Two examples of Histograms in R hist(sales$,main='distribution of ', xlab='',ylab='nr. of Occurrencies') hist(sales$,main='distribution of ', xlab='',ylab='percentage',prob=true) Distribution of Distribution of Nr. of Occurrencies 1e+05 2e+05 3e+05 4e+05 Percentage 1e 08 2e 08 3e 08 4e 08 5e 08 1e+08 2e+08 3e+08 4e+08 1e+08 2e+08 3e+08 4e+08 L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Univariate Plots Distribution of the Values of a Continuous Variable Two examples of Histograms in ggplot2 filter(sales,prod == "p1",!is.na()) %>% ggplot(aes(x=)) + geom_histogram(binwidth=2000) + ggtitle("distribution of of Prod P1") + ylab("nr. of Occurrencies") filter(sales,prod == "p1",!is.na()) %>% ggplot(sales,aes(x=)) + geom_histogram(binwidth=2000,aes(y=..density..)) + ggtitle("distribution of of Prod P1") + ylab("percentage") Distribution of of Prod P1 Distribution of of Prod P1 4e 04 3e Nr. of Occurrencies 50 Percentage 2e 04 1e L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

13 Univariate Plots Distribution of the Values of a Continuous Variable Showing the Distribution of Values Box Plots Box plots provide interesting summaries of a variable distribution For instance, they inform us of the interquartile range and of the outliers (if any) L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Univariate Plots Distribution of the Values of a Continuous Variable An Example with base graphics boxplot(sales$,ylab='distribution of ities') Distribution of ities 1e+08 2e+08 3e+08 4e+08 L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

14 Univariate Plots Distribution of the Values of a Continuous Variable An Example with ggplot2 ggplot(sales,aes(x=factor(0),y=)) + geom_boxplot() + ylab("distribution of ities") + xlab("") + scale_x_discrete(breaks=null) 4e+08 Distribution of ities 3e+08 2e+08 1e+08 L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Conditioned Graphs Conditioned Graphs Data sets frequently have nominal variables that can be used to create sub-groups of the data according to these variables values e.g. the sub-group of male clients of a company Some of the visual summaries described before can be obtained on each of these sub-groups Conditioned plots allow the simultaneous presentation of these sub-group graphs to better allow finding eventual differences between the sub-groups Base graphics do not have conditioning but ggplot2 has it through the concept of facets L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

15 Conditioned Graphs Conditioned histograms Conditioned Histograms Goal: Constrast the distribution of data sub-groups dt <- filter(sales,prod %in% c("p1","p2","p3"),!is.na()) ggplot(dt,aes(x=)) + geom_histogram(binwidth=2000) + facet_wrap(~ Prod) + ggtitle("distribution of ity by Product") Distribution of ity by Product p1 p2 p3 100 count L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Conditioned Graphs Conditioned histograms Conditioned Histograms (2) dt <- filter(sales,prod %in% c("p226","p314","p319")) %>% mutate(unitprice=val/) ggplot(dt,aes(x=unitprice)) + geom_histogram(binwidth=10,aes(y=..density..)) + ggtitle("distribution of Unit Price") Distribution of Unit Price p226 p314 p density ok unkn fraud UnitPrice L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

16 Conditioned Graphs Conditioned Box Plots Conditioned Box Plots on base graphics boxplot( ~ Insp,sales, main='distribution of for the different transactions', xlab='type of Transaction',ylab='Distribution of ') ok unkn fraud 1e+08 2e+08 3e+08 4e+08 Distribution of for the different transactions Type of Transaction Distribution of L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Conditioned Graphs Conditioned Box Plots Conditioned Box Plots on ggplot2 ggplot(sales,aes(x=insp,y=))+geom_boxplot() + ggtitle('distribution of for the different transactions') ## Warning: Removed rows containing non-finite values (stat_boxplot). 1e+08 2e+08 3e+08 4e+08 ok unkn fraud Insp Distribution of for the different transactions L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

17 Other Plots Scatterplots Scatterplots in base graphics The Scatterplot is the natural graph for showing the relationship between two numerical variables dt <- filter(sales,prod=="p11") plot(dt$,dt$val, main='relationship between Value and ity for P1', xlab='',ylab='val') Relationship between Value and ity for P1 Val 2e+05 4e+05 6e L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Other Plots Scatterplots Scatterplots in ggplot2 ggplot(dt,aes(x=,y=val)) + geom_point() + ggtitle('relationship between Value and ity for P1') Relationship between Value and ity for P1 6e+05 4e+05 Val 2e+05 5e+04 1e+05 L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

18 Other Plots Scatterplots Conditioned Scatterplots using the ggplot2 package ggplot(dt,aes(x=,y=val)) + geom_point() + facet_wrap(~insp) ok unkn fraud 5e+04 1e+05 5e+04 1e+05 5e+04 1e+05 2e+05 4e+05 6e+05 Val L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Other Plots Scatterplot matrices Scatterplot matrices through package GGally These graphs try to present all pairwise scatterplots in a data set. They are unfeasible for data sets with many variables. library(ggally) ggpairs(sales,columns=3:4, diag=list(continuous="density", discrete="bar"),axislabels="show") Corr: Val Val 1e+08 2e+08 3e+08 4e+08 1e+06 2e+06 3e+06 4e e+06 2e+06 3e+06 4e+06 L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38

19 Other Plots Scatterplot matrices Scatterplot matrices involving nominal variables ggpairs(sales,columns=3:5,axislabels="show") Corr: Val Insp Val Insp 1e+082e+083e+084e+085e+08 1e+062e+063e+064e+06 ok unkn fraud e+06 2e+06 3e+06 4e+06 1e+05 2e+05 3e+05 1e+05 2e+05 3e+05 1e+05 2e+05 3e+05 L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38 Other Plots Parallel plots Parallel Plots Parallel plots are also interesting for visualizing a data set ggparcoord(filter(sales,prod=="p226"),columns=3:4,groupcolumn="insp") Val variable value Insp ok unkn fraud L.Torgo (FCUP - LIAAD / UP) Data Visualization Aug, / 38