Ch3 E lement Elemen ar tary Descriptive Descriptiv Statistics

Transcription

1 Ch3 Elementary Descriptive Statistics

2 Section 3.1: Elementary Graphical Treatment of Data Before doing ANYTHING with data: Understand the question. An approximate answer to the exact question is always better than an exact answer to an approximate question. John Tukey. Know how the experiment was conducted.

3 The FIRST thing to do with the data is to PLOT THE DATA Plot all individual points. If there are connections between points, e.g. points are from same pairs (or sometimes separate blocks), show connections between related points.

4 Plotting data is an extremely important step. More often than not data I get when consulting have problems like incorrect data or attributes they didn t tell me about. Plotting helps reveal relationships and answers. Plotting is a very effective way to present results. A picture is worth a thousand words.

5 Example: 8 lb. test fishing line question: Which type(s) of line are strongest? Listing numerical data Trilene XL Trilene XT Stren It s hard to see what s happening without organizing the data.

6 A dot diagram XL XT Stren 11.8 ** 11.7 ** *** 11.6 ** *** 11.5 *** * 11.4 ** * 11.3 * * 11.2 ** 11.1 **** ** *

7 Two groups can be compared with back to back stem and leaf diagrams E.g. Stopping distances of bikes Treaded tire Smooth tire Or dot diagrams * ** * ** Treaded *** * * * Smooth

8 When there are associations between sets of data values, plot the data accordingly. E.g., Snowfall for duluth and White Bear Lake A not very good way to plot tthe data WB Lake Duluth 130 * 120 * 110 ** ** 100 *** * 90 ***** 80 ****** ****** 70 ** *** 60 ** ********** 50 **** *** 40 *** *** 30 * *** 20

9 Snowfall plot snow w_total year Duluth White Bear

10 A study of trace metals in South Indian River T=top water zinc concentration (mg/l) B=bottom water zinc (mg/l) Top Bottom

11 One of the first things to do when analyzing data is to PLOT the data Zinc Top Bottom This is not a useful way to plot the data. There is not a clear distinction between bottom water and top water zinc even though Bottom>Top at all 6 locations.

12 Abetterway Zinc Connect points in the same pair. Top Bottom

13 Abetterway Bottom=Top

14 This following plot would imply a natural ordering of sites from 1 to Zinc Top Bottom Site This would not be the best way to plot the data unless the sites 1 6 correspond to a natural ordering such as distance downstream of a factory.

15 Section 3.2: Quantiles and Related Graphical Tools Quantile: Roughly speaking, for a number p between 0 and 1, the p quantile of a distribution is a number such that a fraction p of the distribution lies to the left and a fraction 1 p of the distribution ib i lies to the right.

16 p quantile = 1O0*p th percentile Q(0.10) = 0.10 quantile = 10 th percentile Q(0.50) = 0.50 quantile = 50 th percentile = median Q(0.25) = quantile = 25 th percentile= first quartile Q(0.75) =0.75 quantile = 75 th percentile= third quartile

17 Boxplots are useful summaries, particularly when there are too many points for a dot plot. To make a boxplot, we need essentially 5 numbers.

18

19

20 Section Q Q Plots and Comparing Distributional Shapes Most of the statistical tools we will use in this class assume normal distributions (a bell shaped distribution for the population of possible values). In order to know if these are the right tools for a particular job, we need to be able to assess if the data appear to have come from a normal population.

21 With large amounts of data, one can draw a histogram of the measured values and see if it is bell shaped. A normal plot is a method for assessing normality that works well with big or small data sets. It gives a good visual check for normality.

22 Simulation: 100 observations, normal with mean=5, st dev=1 x< rnorm(100, mean=5, sd=1) qqnorm(x) 2 3 x Quantiles of Standard Normal

23 A normal plot is a plot of the data in a way such that data from normal populations will come out pretty much in a straight line. We plot the corresponding quantiles of a "standard d normal'' distribution ib i versus ordered d y values

24 In other words In order to plot the data and check for normality, we compare our observed data to what we would expect from a sample of standard normal data.

25 A standard normal distribution is a normal distribution with mean µ=0 standard d deviation i σ=1. 1 Any normal population can be thought of as a rescaled standard normal population. For example if Z is standard normal, then Z will have µ=100 and σ= 5. M lti l i ll l b 5 lti li th t d d d i ti b 5 Multiplying all values by 5 multiplies the standard deviation by 5. Adding 100 to every number adds 100 to the mean.

26 So if we plot ordered values from a normal population p against corresponding quantiles of a standard normal population, we expect to get a reasonably straight line, since any normal distribution is linearly related to the standard normal distribution.

27 With Excel normal quantile can be found with the NORMINV function. NORMDIST finds probabilities given a particular value. NORMINV is the inverse function finding a value with a given probability bilit of being less than that. t A cell is assigned for example the formula A cell is assigned for example the formula = NORMINV(C3, 0, 1) The 0, 1 indicates µ=0 and σ=1 o A standard normal quantile

28 The textbook plots the standard normal quantiles on the vertical axis and the ordered data points on the horizontal axis. Many software packages and other books plot the standard normal quantiles on the horizontal axis and the ordered data points on the vertical axis. Eith th l t h ld l k ``f i l '' t i ht if th Either way, the plot should look ``fairly'' straight if the data are from a normal distribution.

29 Here are ordered lifetimes of springs under 2 levels of stress. (page 379) Normal 950 stress 900 stress n i (i-0.5)/n Quantile Lifetime Lifetime Since n=10 for both sets the corresponding normal quantiles are the same for both sets.

30 To construct normal plots for these two data sets, we plot each ordered data set versus the standard normal quantiles from Excel ength Life-l Normal Quantiles 950 stress 900 stress Since both plots are fairly straight, these data are fairly normal.

31 Excel File of Lifetime of Springs Data Normal Ordered Ordered n i (i-0.5)/n Quantile E(Z) 900 stress 950 stress

32 Section 3.3: Numerical Summaries Measures of Location: The data are found spread around what value? Median = Q(O.50) = 50 th percentile. Sample mean = arithmetic mean = average x The mean is more affected by unusual values than the median. = n x i= 1 n i

33 Measures of Spread: R = Range = Biggest Smallest The size of the range can be affected by how many values we have. Many number will tend to have a larger range than fewer numbers. IQR = lnterquartile Range = Q(0.75) Q(0.25) Q q g Q( ) Q( ) Range that include half of the values.

34 Sample variance = s 2 = ( x x ) 2 i n 1 Essentially an average squared deviation from the mean. Sample standard deviation = s s 2 = = ( x x) 2 i n 1

35 Example: X 1 = 8 X 2 = 9 X 3 = x = = 7 3 ( ) + ( 9 7) + ( 4 7) 2 s = = s = 7 =

36 Statistics and Parameters A statistic is a numerical summary of the sample data. x = sample mean s 2 = sample variance

37 A parameter is a summary of an entire population or a theoretical distribution, for example a normal distribution. N µ = population mean σ 2 = population p variance x i= 1 µ = N 2 i= 1 σ = N i ( x µ ) 2 Average squared deviation from the mean. Second central moment. i N σ = population standard deviation σ = 2 σ

38 For a sample of size n, the sample variance is 1 s x x n 2 2 = ( i ) n 1 i= 1 Why divide by n 1? This makes s an 2 unbiased estimator of σ. Unbiased means on the average correct. 2

39 Suppose we have a large population of ball bearings with diameters µ=1cm and 2 σ = 0.02 σ = x Sample Mean If we knew µ we would find Fact min ( x i m) 2 = ( x i s n 2 2 ˆ ( xi µ ) σ = x) 2 i= So ( x x) ( µ ) and would be too small for σ 2. i x i 2 ( x i x) n Dividing by n 1 makes s 2 come out right (σ 2 )on average. 2 n

40 Notice that s 2 is undefined if n=1; we can't divide by zero. This makes sense. If we have only one number, that number tells us nothing about potential ti spread in the population.

41 Plotting summary statistics i over time is useful for issues such as quality control. Read section for general information.

42