1. Select[] is used to select items meeting a specified criterion.

Transcription

1 << ( ) Name: Before doing this assigment, complete the shape analysis of the calcaneum bone from the associated Tarsal Data document. Use the shape variables (PC scores) from that data set for this exercise Select[] is used to select items meeting a specified criterion. = { } (* *) [ # > ] { } 2. Length[] can be used to count items in a list. (* *) [ [ # > ]] (* *) [ [ # > ]] [ ] [ [ # > ]] [ ] // 3. RandomChoice[] randomly chooses items from a list with replacement (* *) [ [ ]] { } 4. RandomSample[] randomly samples from a list without replacement

2 2 111 Assignment 5 - Bootstrapping and Randomization with work from class.nb [ ] { } 5. Use RandomSample[] to randomize members of several groups (* *) = {{ } { } { } { } { } { } { }} (* *) [[ ]] = [ [[ ]]] {{ } { } { } { } { } { } { }} 6. Combining graphics with Show[] (* *) = [ [ [ ] ]]

3 Assignment 5 - Bootstrapping and Randomization with work from class.nb 1113 (* *) = [{ [{{ } { }}]}] (* *) [ ] Randomization techniques all involve generating a finite random distribution from your data and determining whether an observed statistic could plausibly have arisen from it. The p-value is the number of random observations that are equal to or fall beyond the observed value. Practice calculating p-values

4 4 111 Assignment 5 - Bootstrapping and Randomization with work from class.nb using data that are randomly generated from one of Mathematica s distribution functions. 1. Generate 10,000 random numbers using the NormalDistribution function. Save them in a variable and plot them as a histogram. 2. What are the mean, standard deviation, skewness, and kurtosis of the random numbers. Why are the mean and standard deviation different from the ones you specified? By how much are they different? What skewness and curtosis are expected from normally distributed numbers? 3. How many of the random numbers are greater than the mean? What proportion is that? What is the probability of a number being greater than the mean? What proportion are expected to be greater than the mean? 4. How many are less than 1.96 standard deviations from the mean? What proportion are expected to be less than that? 5. What proportion fall between plus and minus 1.96 standard deviations from the mean? 6. Are the results different if you repeat everything with a LogNormalDistribution instead of a NormalDistribution? Bootstrapping is used to estimate the uncertainty of a particular statistic like a sample mean. Uncertainty is usually measured as the standard error of the statistic or a confidence interval of particular size (e.g., 50% confidence interval, 95% confidence interval). Bootstrapping is not usually used to compare two or more statistics (e.g., it is not used to test for differences in mean between two samples). 1. Give a short definition of standard error. 2. Give a short definition of confidence interval. 3. Describe how bootstrapping can be used to calculate a standard error on a sample mean. 4. Describe how a standard error can be converted to 95% confidence interval. 5. Using the tarsal data, calculate the mean shape on PC 1 for each of the two digit categories (animals with four toes and those with five). 6. Use bootstrapping to calculate 1,000 random samples (with replacement) from each group and graph each set of 1,000 random samples as a histogram. 7. How do the mean values of the 1,000 random samples compare to the real means of each group? Why? 8. Calculate the standard error for the mean of each group. 9. Calculate a 95% confidence interval for the mean of each group. Describe in words what this confidence interval is Articulate the question. Decide what question you are asking and articulate it clearly. For example, you might want to know whether the faces of men and women are different in our shape analysis of class faces. If so, then the question might be does the average shape of women s faces differ from the average shape of men s faces?

5 Assignment 5 - Bootstrapping and Randomization with work from class.nb Identify the null hypothesis. A null hypothesis is the hypothesis you are trying to falsify. Randomization tests are amenable to questions where the null hypothesis is that there is no statistical difference between samples. The above question asks whether there is a difference in face shape, so the null hypothesis would be that there is no difference. 3. Reframe the question and null hypothesis in terms of population sampling and chance. The question of difference in male and female face shape implies that our male and female samples come from two different statistical populations, each with its own mean shape. The null hypothesis is that the male and female samples are drawn from a single population with a single mean shape. We want to know whether the observed means of the male and female faces are more different than expected in two samples drawn randomly from a single population. To test this with randomization, we first need a null population with a single mean and with both male and female faces, which we can obtain by grouping all our face data together we have such a population. 4. Calculate statistics and perform the randomization. Our question is about mean shapes. Calculate the difference (distance) between the means of males and females. This is the observed difference between the two groups. Our statistical question is whether this difference is larger than expected if the two groups were randomly drawn from the null population. Thus we want to draw a large number of samples, with sizes respectively equal to the original male and female samples, and calculate the difference between them. The fact that we use the same sample sizes incorporates sample imbalance into the test, and the fact that we are drawing from a larger population of real faces incorporates any skewness, kurtosis, or outliers into the results. Thus our test is robust to departures from normality, balanced sample size, and other hurdles of parametric test. 5. Calculate the p-value that the observed is different from the random distribution. To do this we need to know how the real difference between males and females compares to the distribution of differences between random samples. Exactly how we do this depends on our question. Here we are asking whether the real difference is larger, which makes this a onetailed test. We want to estimate the probability that a difference this large or larger could arise by chance, so we select the values from the random distribution that are greater than or equal to the observed difference. Divide that by the total number of random differences to obtain the frequency. That is the p-value. Note that some questions could be two-tailed: our question Calculate the means on all PCs for the arboreal and terrestrial groups. 2. Calculate the difference in the means. 3. Devise a randomization test to determine whether this difference is statistically significant (this is the non-parametric randomization equivalent of a multivariate analysis of variance MANOVA test) Describe the null hypothesis that is being tested Describe the randomization process Carry out the randomization 10,000 times and graph the resulting distances as a histogram. Plot a line on the histogram showing the observed distance between the groups Calculate the p-value for your test Describe in words what the results of your test mean.

6 6 111 Assignment 5 - Bootstrapping and Randomization with work from class.nb data = [ / / / / ] = [[ ]] { } = [[ ]] = [ [ [[ ]]] / ] = [ ] = # - /@ = [ ] { } = [ ] = = [[ ]] [ ] // [ [[ ]]] = [[{ }]] = [[{ }]] = [ ] = [ ] = [ = [ [[{ }]] ] = [ [[{ }]] ] = [ ] = [ ] {{ [ [[{ }]]]} { [ [[{ }]]]}} { }]

7 Assignment 5 - Bootstrapping and Randomization with work from class.nb 1117 [{ { [ ] [ [[ { }]]]} { [ ] [ [[{ }]]]} { [ ] [ [[ { }]]]} { [ ] [ [[{ }]]]}} ] ( - ) = [[{ }]]

8 8 111 Assignment 5 - Bootstrapping and Randomization with work from class.nb = [ ] = [[{ }]] = [[ ]] = [ ] = [ ] ( - ) [{{ [ ] [ [[ { }]]]} { [ ] [ [[{ }]]]} { [ ] [ [[ { }]]]} { [ ] [ [[{ }]]]}} ] = = [ ] = [[{ }]] = [[ ]] = [ ] = [ ( - ) { }

9 Assignment 5 - Bootstrapping and Randomization with work from class.nb 1119 [ [ ] [{ [{{ } { }}]}]] = [ [ # ]] //