Regression III: Advanced Methods

Transcription

1 Lecture 3: Distributions Regression III: Advanced Methods William G. Jacoby Michigan State University

2 Goals of the lecture Examine data in graphical form Graphs for looking at univariate distributions Histograms Density estimation Quantile comparison plots Boxplots Plots exploring relationships Parallel boxplots Scatterplots (including scatterplot matrices and dynamic three-dimensional scatterplots) Bivariate & three dimensional density estimation Conditioning plots 2

3 Histograms Dissects the range of the data into bins of equal width along the horizontal axis Vertical axis represents the frequency counts (or percents, proportions) Bars represent the counts Fewer bins, smoother histogram, but less detail about the distribution Trade-off between smoothness and detail: We want to preserve as much detail as possible but we do not want the graph to be too rough (difficult to discern shape) Stem-and-leaf displays Alternative form of histogram that uses the numerical data themselves to form the bars of the histogram Data are broken into two parts: a stem and a leaf 3

4 Choosing the number of bins Simple rule of thumb for small datasets (approx. 100 or less) is: For larger samples, the car package for R implements Freedman and Diaconis (1981) recommended formula from the n.bins function: Frequency Histogram of income income 4

5 Nonparametric Density Estimation Although histograms can be very useful for examining the distribution of a variable, the diagram can differ dramatically depending on the number of bins employed This problem can be overcome (partially) using nonparametric density estimation Density estimation is an attempt to estimate the probability density function of a variable based on the sample, but less formally it can be thought of as a way of averaging and smoothing the histogram Since a density function encloses an area of 1, we first must rescale the histogram so that the total area under the smoothed line (the area within the bins) equals 1. In other words, we examine the proportion of cases at specific points in the histogram rather than the frequency counts 5

6 Kernel Density Estimation Essentially a sophisticated form of locally weighted averaging of the distribution Use a weight function (kernel) that ensures the enclosed area of the curve equals 1 Probability density functions (such as the standard normal density function) are good choices because they are smooth and symmetric The kernel density estimate is calculated as follows: 6

7 Selecting the Bandwidth Unlike histograms we no longer set the number of bins; instead we must select the bandwidth h. We can do this visually, but statistical theory provides some help: The population standard deviation σ is unknown so we replace it with an adaptive estimator of spread (The sample standard deviation S can be inflated if the underlying density isn t normal): Hinge spread is the inter-quartile range; is the hinge spread of the standard normal distribution. The formula for the bandwidth is then: 7

8 An example of Kernel Density Estimation If the underlying density distribution is substantially nonnormal, produces a window width 2h that is too wide (i.e., the line is too rough), but it is good as a starting point As the bandwidth increases, the density curve becomes smoother Ideally we want a smooth curve like the black line to the right (bw=1087) Density Histogram with Density Estimation income bw=1087 bw=400 8

9 R script for previous graph 9

10 Density Estimates with confidence envelopes The sm package for R allows you to plot variability bands that are a width of two standard errors These bands can be especially useful for assessing modality More details are in Bowman and Azzalini (1997: Chapter 2) Probability density function income 10

11 Density Estimates with a normal reference band The sm package also allows you to fit a normal reference band which indicates the likely position of the density estimate when the data are normal (blue area) Again, more details are in Bowman and Azzalini (1997: Chapter 2) Probability density function income 11

12 Quantile Comparison Plots (1) Quantile comparison plots are most useful for examining the tails of the distribution Unlike histograms and density functions, they do not require arbitrary bins and thus preserve the continuous nature of the data. They do so by comparing the sample distribution to a theoretical cumulative distribution function (CDF) We could substitute the empirical cumulative distribution function (ECDF), which gives the proportion of data that fall below each x value as x moves from left to right, for the CDF Because the EDCF is typically rough (a stair-step function that rises a height of 1/n at each observation), however, the quantile comparison plot does not construct it directly 12

13 Quantile Comparison Plots (2) Order the cases from lowest to highest X (1), X (2),,X (n) Calculate the cumulative proportion before each X (i): z i values that correspond to the cumulative probability P i are found from the inverse of the CDF: 13

14 Quantile Comparison Plots (3) Plot the z i values on the horizontal axis; the X (i) on the vertical axis If X is normal, then X (i) z i. In other words, the plot should be approximately linear Draw a line that connects the hinges (the quartiles) A 95% confidence envelope is constructed as follows: A positive skew is indicated by points above the line on both ends; A negative skew is indicated by points below the lines on both ends Heavy tailed distributions are indicated by points above the confidence envelope for high values and points below for low values. 14

15 Quantile Comparison Plots (4) QQ-plots are easily implemented using the car package. Here we see a positive skew LAWYERS since points lie above the OSTEOPATHS.CHIROPRACTORS line on both tails (a negative skew is indicated by points below the lines on both tails) As we can see this plot is useful for examining the tails of a distribution They tell us nothing about the mode, however income GENERAL.MANAGERS norm quantiles PHYSICIAN VETERINARIANS PILOTS 15

16 Boxplots Display the center, spread, and outliers of a distribution Vertical axis represents the range of the variable A box is drawn around the hinge spread A line is drawn at the median Outliers more than 1.5 hinge spreads past the hinges are marked individually Whiskers connect the box to the most extreme nonoutlying observation Income Boxplot of Income GENERAL.MANAGE PHYSICIANS LAWYERS OSTEOPATHS.CHIROPRACTORS VETERINARIANS 16

17 Side-by-side Boxplots: Helpful for comparing many distributions Boxplots of Income for different Occupation Types bc prof wc 17

18 Why use graphs to examine relationships? Anscombe s (1973) contrived data show the importance of using graphs in data analysis rather than simply looking at numerical outputs Four very different relationships with exactly the same correlation coefficient, standard deviation of the residuals, and coefficients and standard errors The linear regression line adequately summarizes only graph (a) Y Y (a) Accurate summary X (c) Drawn to outlier X Y Y (b) Distorts curvilinear rel X (d) "Chases" outlier X 18

19 Scatterplots One of the most used of all statistical graphs summarizes the relationship between two quantitative variables Including nonparametic smooths and linear regression lines can aid visualization of the trend A useful technique when one of the variables does not take on many different values, or when the sample is so large that the data are over-plotted and there are few empty spaces on the graph, is to jitter the data (i.e., add a random component to each value) We can jitter one or both of the variables in a scatterplot 19

20 Jittering scatterplots No jitter Conservative attitudes Conservative attitudes Age jittered Age

21 Identifying categorical variables Distinguishing between categories of a categorical control variable in scatterplots can help show important patterns we might otherwise miss Different slopes for nondemocracies and democracies suggests an interaction between democracy and income inequality in their effects on attitudes Attitudes towards inequality(mean) Democratic Non-Democratic Gini coefficient 21

22 Bivariate Density Estimates The kernel smoothing method used for histograms can be easily extended to the joint distribution of two random continuous variables The bivariate density function takes the following form: Where K is the kernel function and (h 1 and h 2 ) are the joint smoothing parameters For univariate densities, probabilities are associated with area under the density curve. For a bivariate density curve, probabilities are associated with volume under the density, where the total volume equals one 22

23 Types of Bivariate Density Plots Perspective plots: the joint distribution is shown in a 3D plot height is used to show level of density Imageplots: different intensities of colour or shading denote density levels Contour plots or slice plots: lines trace paths of constant levels of density (similar to the depiction of elevation in a geographical contour map) 23

24 R script for Bivariate Density Plots 24

25 Three Dimensional Density Estimates The three dimensional density estimate also extends simply from the bivariate case: Where K is the kernel function and (h 1, h 2, and h 3 ) are the joint smoothing parameter In these plots contours represent closed surfaces Like the other density estimates, these are helpful for assessing clustering of the data 25

26 Some examples of three dimensional density estimates prestige secpay income education gini gdp 26

27 Scatterplotmatrix Plots individual scatterplots for all possible bivariate relationships at one time Can be enhanced by adding density estimates for each variable on the diagonal Note: Only marginal relationships are depicted (i.e., no control for other variables) gini secpay gdp

28 Conditioning plots: An example from the CES jitter(lascale) Given : education Given : men age

29 Next Topic: Transformations for univariate and bivariate data 29