Exploratory Data Analysis September 8, 2010

Transcription

1 Exploratory Data Analysis p. 1/2 Exploratory Data Analysis September 8, 2010

2 Exploratory Data Analysis p. 2/2 Scatter Plots plot(x,y) plot(y x) Note use of model formula Today: how to add lines/smoothed curves add/identify points add legends transformations

3 Exploratory Data Analysis p. 3/2 Old Faithful Eruptions > summary(geyser) Day interval duration Min. : 1.00 Min. :42.00 Min. : st Qu.: st Qu.: st Qu.:2.300 Median :16.00 Median :75.00 Median :4.000 Mean :12.30 Mean :71.01 Mean : rd Qu.: rd Qu.: rd Qu.:4.400 Max. :23.00 Max. :95.00 Max. :5.200 Day (of eruption) in August interval (waiting time until the next eruption in minutes) duration (duration of current eruption in minutes)

4 Exploratory Data Analysis p. 4/2 Scatter Plot Old Faithful Eruptions Waiting Time (mins) Duration (mins)

5 Exploratory Data Analysis p. 5/2 Adding a Smooth Trend The function lowess fits a one dimensional smooth trend to (x,y) pairs To construct a scatter plot with smooth trend 1. plot(duration, interval) 2. lines(lowess(duration, interval)) or 3. scatter.smooth(duration, interval)

6 Exploratory Data Analysis p. 6/2 Default Lowess Smooth Old Faithful Eruptions Waiting Time (mins) Duration (mins)

7 Exploratory Data Analysis p. 7/2 lowess The algorithm behind lowess and its cousin loess is quite complex, but the idea is to use robust local estimates of the curve A window is placed around a point x data points in the window are weighted so that points near x get the most weight using the points in the window and weights, use a robust weighted regression to predict at x the parameter f gives the proportion of points in the plot which influence the smooth at each value. Larger values give more smoothness. Default is 2/3.

8 Exploratory Data Analysis p. 8/2 Choice of f in lowess Old Faithful Eruptions Waiting Time (mins) f = 1 f = 2/3 f =.5 f = Duration (mins)

9 Exploratory Data Analysis p. 9/2 Legends legend(x=2,y=90, legend=c("f=1","f=2/3","f=1/2","f=1/10"), col=2:5, lty=c(2,1,3,4), lwd=rep(2,4)) x,y location for legend legend text for legend col colors used lty line types lwd line widths

10 Exploratory Data Analysis p. 10/2 Transformations For plots to be more effective, we may need to transform variables. (also may make normality assumptions more plausible) Common transformations log positive continuous measures, concentrations, size sqrt counts ˆ-1 Typically need the largest observation to be at least 10 times larger than the smallest case for transformations to be effective.

11 Exploratory Data Analysis p. 11/2 Log Transformations library(mass) data(animals) plot(brain body, data=animals) plot(brain body, log="xy",data=animals) Original Units Logarithmic Scale Brain Weight (g) African elephant Asian elephant Human Brachiosaurus Brain Weight (g) 5 e 01 5 e+01 5 e+03 1 e 01 1 e+03 Brachiosaurus Triceratops Dipliodocus Body Weight (kg) Body Weight (kg)

12 Exploratory Data Analysis p. 12/2 Identifying Points Draw plot, then use either identify(x,y, labels) locator(n) to label points to find coordinates of n locations left-click with mouse at locations (MAC click) to stop, right-click the mouse in the plot (MAC command-click) > identify(animals$body, Animals$brain, labels=rownames(animals)) Use text() to add other text or legends at locations

13 Exploratory Data Analysis p. 13/2 Multiple Plots per Page Use the mfrow option to control the number of plots in a page The command par(mfrow=c(1,2)) plot(brain body, data=animals) plot(brain body, log="xy",data=animals) creates a figure with 1 row of plots and 2 columns stays in effect until another mfrow command is sent See help(par) for more graphical control parameters

14 Exploratory Data Analysis p. 14/2 Powers Transformations Explore Box-Cox family of power transformations of y, x power(y, p) { y p 1 p (p 0) log(y) (p = 0) The ladder function of HH is built on the lattice package > ladder(brain body, data=animals, main="powers of Brain and Body Weight") See Adler Chapter 4 for installing/loading packages on different systems

15 Exploratory Data Analysis p. 15/2 Animals Example Powers Transformations of Brain and Body Weight brain^2 ~ body^ 1 brain^2 ~ body^ 0.5 brain^2 ~ body^0 brain^2 ~ body^0.5 brain^2 ~ body^1 brain^2 ~ body^2 3e+07 2e+07 1e e+00 3e+07 brain^1 ~ body^ 1 brain^1 ~ body^ 0.5 brain^1 ~ body^0 brain^1 ~ body^0.5 brain^1 ~ body^1 brain^1 ~ body^2 2e+07 1e e+00 3e+07 brain^0.5 ~ body^ 1 brain^0.5 ~ body^ 0.5 brain^0.5 ~ body^0 brain^0.5 ~ body^0.5 brain^0.5 ~ body^1 brain^0.5 ~ body^2 2e+07 1e brain 0e+00 3e+07 brain^0 ~ body^ 1 brain^0 ~ body^ 0.5 brain^0 ~ body^0 brain^0 ~ body^0.5 brain^0 ~ body^1 brain^0 ~ body^2 2e+07 1e e+00 3e+07 brain^ 0.5 ~ body^ 1brain^ 0.5 ~ body^ 0.5 brain^ 0.5 ~ body^0 brain^ 0.5 ~ body^0.5 brain^ 0.5 ~ body^1 brain^ 0.5 ~ body^2 2e+07 1e e+00 brain^ 1 ~ body^ 1 brain^ 1 ~ body^ 0.5 brain^ 1 ~ body^0 brain^ 1 ~ body^0.5 brain^ 1 ~ body^1 brain^ 1 ~ body^2 3e+07 2e+07 1e e+00 0e+00 4e+09 0e+00 4e+09 0e+00 4e+09 0e+00 4e+09 0e+00 4e+09 0e+00 4e+09 body

16 Exploratory Data Analysis p. 16/2 Scatterplot Matrices SPLOM plot(usair) or pairs(usair) pairs(usair, panel=panel.smooth) Add a smoother to each plot pairs(so2., panel=panel.smooth, data=usair) use a model formula pairs(so2 temp + mfgfirms + wind + raindays, panel=panel.smooth, data=usair) Hartigan s original version of a scatterplot matrix had histograms on the diagonal. We need to first define a function panel.hist for the diaginal panels See help(pairs)

17 Exploratory Data Analysis p. 17/2 Defining a function panel.hist = function(x, col="grey",...) { usr <- par("usr") on.exit(par(usr)) par(usr = c(usr[1:2], 0, 1.5) ) h <- hist(x, plot = FALSE) breaks <- h$breaks nb <- length(breaks) y <- h$counts; y <- y/max(y) rect(breaks[-nb],0,breaks[-1],y, col=col,...) }

18 Exploratory Data Analysis p. 18/2 SPLOM with histogram > pairs(so2 temp + mfgfirms + popn + wind + precip + raindays, data=usair, panel=panel.smooth, diag.panel=panel.hist) > pairs(log(so2) log(temp) + log(mfgfirms) + log(popn) + log(wind) + log(precip) + log(raindays), data=usair, panel=panel.smooth, diag.panel=panel.hist)

19 Exploratory Data Analysis p. 19/2 Scatterplot Matrix SO temp mfgfirms popn wind precip raindays

20 Exploratory Data Analysis p. 20/2 Scatterplot Matrix with transformation log(so2) log(temp) log(mfgfirms) log(popn) wind log(precip) log(raindays)

21 Exploratory Data Analysis p. 21/2 Ladder of Powers ladder(so2 temp, data=usair) ladder(so2 mfgfirms, data=usair) ladder(so2 popn, data=usair) ladder(so2 wind, data=usair) ladder(so2 precip, data=usair) ladder(so2 raindays, data=usair) Is there one trasformation for SO2 that "linearizes" relationships with other variables? Subjective choice of transformation