Statistics, Data, and R. 2 Sample Means, Variances, Covariances, and Correlations

Transcription

1 Statistics, Data, and R 1 Populations and Samples A population refers to a group of animals that are part of the overall breeding structure in an industry. Examples are, Holstein dairy cattle in Canada that are on milk recording programs. Labrador retrievers in Ontario. Rainbow trout on the east coast of Canada. Racing pigeons of Quebec. Populations have parameters that describe the means and variances of traits that are observed on that population. The population mean for a trait is designated by the Greek letter mu, µ. The population standard deviation for a trait is designated by a Greek sigma, σ. Population parameters need to be estimated for use in genetic evaluation. These are estimated from samples of animals from the population. A sample is a subset of animals from the population. For example, the population of Holstein cows in Canada can be split into samples within each province. A sample might be cows in one herd. 2 Sample Means, Variances, Covariances, and Correlations Let y i be an observed trait value on an animal in the sample from the overall population, and let there be N such observations. The observation is composed of the population mean and a deviation (e i ) from that mean, i.e., y i = µ + e i. An unbiased estimator of the population mean is ˆµ = ( y i )/N, where is the summation symbol which means to add together the y i s. 1

2 Variance is an indicator of the range of possible values that y i could have. For example, if the minimum value of y i was 76 and the maximum value was 82, then the variance would be smaller than if the minimum was 25 and the maximum was 130. An estimator of the population variance is ˆσ 2 = ( yi 2 ( y i ) 2 /N)/(N 1), = (y i ˆµ) 2 /(N 1). Coefficient of Variation is a way to represent the degree of variation relative to the size of the mean, CV = 100%. ˆσˆµ Covariance is used to measure how two traits vary together. Let y i be one trait, like birthweight, and let w i be a different trait, like adjusted 200-d weight, both measured on the same animal. An estimator of the population covariance is ˆσ yw = ( y i w i ( y i )( w i )/N)/(N 1), = (y i ˆµ y )(w i ˆµ w )/(N 1). Covariances may be positive or negative. A positive covariance means that as one trait becomes larger in magnitude, so does the other trait. A negative covariance means that as one trait becomes larger the other trait becomes smaller. An easier way of looking at co-variation among traits is the correlation coefficient, ˆρ = ˆσ yw (ˆσ 2 y ˆσ 2 w).5. Correlation coefficients range between -1 and Normal Distribution Many quantitative traits of importance in livestock production follow the Normal Frequency Distribution. Every Normal distribution can be described entirely by its mean and variance as N(mean, variance). A Normal distribution with a mean of zero and a variance of one is known as the standard Normal distribution (N(0, 1)). A more general formulation is y i N(µ, σ 2 ), 2

3 where y i is the trait, µ is the mean of the population, and σ 2 is the variance of the observations. Table 1. Some commonly used values for the standard Normal distribution. z-values Percentage Point Confidence Interval Selection Intensity A few points to remember: z-value (z = (x i µ)/σ) is the trait value expressed as a difference from the population mean in standard deviation units. 3

4 Percentage point (p) gives the portion of the population above the given z-value. Confidence interval gives the portion of the distribution within z-value units of the mean, i.e. between z and +z on the horizontal axis. Selection intensity (i) is the average value (in standard deviation units and deviated from the mean) of the portion p of the population which lies above the z-value. The distribution is symmetric, with 50% above and 50% below the mean. Two-thirds (2/3) of the distribution, or about 67%, is within one standard deviation from the mean (i.e. between z-values of -1.0 and +1.0 on the standard Normal curve). About 95% of the distribution is within 2 standard deviations from the mean. These rules apply to any trait that follows a Normal distribution, by first standardizing the distribution by converting the observations for the trait (y i ) to z-values. For example, if y i N(µ, σ 2 ), then y i can be converted into a z-value as follows: z i = y i µ. σ The Normal distribution applies to the majority of traits recorded on livestock populations, but occasionally this is not the case. Examples of non-normality are as follows: 1. An animal either has a disease or does not have a disease (yes or no trait), which is a binomial distribution specified by p, a probability of having the disease. 2. Number of piglets born in a litter can be anywhere from 7 to 13 usually. The number born follows a Poisson distribution. 3. Calvings in cattle are categorized into 4 or 5 classes which range from Easy or Unassisted Calving, Assisted Calving, Difficult Calving, and Caesarian section. This is an example of a multinomial trait, i.e. more than two categories and probabilities associated with being in each. In many cases, traits are assumed to follow a normal distribution even if they do not, and the results are almost as good as using the more appropriate distribution. Distributions other than normal are often more complicated computationally. In practice, the first attempt should be to use the most appropriate distribution before making any simplification to a normal distribution. This course will only consider traits to follow a normal distribution. 4

5 4 Small Data Sets Most examples used in these notes to illustrate methods can be given in one table on less than a page of paper. In these cases, the student can enter the data into R manually in a few minutes. Below are data on 10 beef calves born at a research station within one week of each other. Beef calf data on birthweights (BW) and calving ease (CE). Calf Breed Sex CE BW(lbs) 1 AN M U 55 2 CH M E 68 3 HE M U 60 4 AN M U 52 5 CH F H 65 6 HE F E 64 7 CH F H 70 8 AN F E 61 9 HE F E CH M C 75 An easy way to enter the data is by columns of the table. calf = c(1:10) # makes a string of numbers 1 to 10 breed = c("an","ch","he","an","ch","he","ch","an", "HE","CH") sex = c("m","m","m","m","f","f","f","f","f","m") CE = c("u","e","u","u","h","e","h","e","e","c") BWT = c(55,68,60,52,65,64,70,61,63,75) Then the columns can be put into a data frame, as follows: beefdat = data.frame(calf,breed,sex,ce,bwt) beefdat # looks at the data, exactly like the table The data frame can be saved and used at other times. The saved file can not be viewed because it is stored in binary format 5

6 setwd(choose.dir()) save(beefdat,file="beef.rdata") # and retrieved later as load("beef.rdata") 4.1 Creating Design Matrices A desgin matrix is a matrix that relates levels of a factor to the observations. The observations, in this example, are the birthweights. The factors are breed, sex, and CE. The breed factor has 3 levels, namely AN, CH, and HE. The sex factor has 2 levels, M and F, and the CE factors has 4 levels, U, E, H, and C. A function to make a design matrix is as follows: desgn <- function(v) { if(is.numeric(v)) { va = v mrow = length(va) mcol = max(va) } if(is.character(v) { vf = factor(v) va = as.numeric(vf) mrow = length(va) mcol = length(levels(vf)) } X = matrix(data=c(0),nrow=mrow,ncol=mcol) for(i in 1:mrow) { ic = va[i] X[i,ic] = 1 } return(x) } # To use, then B = desgn(breed) S = desgn(sex) C = desgn(ce) 6

7 These matrices are B = 0 0, S = , C = Each row of a design matrix has one 1 and the remaining elements are zero. The location of the 1 indicates the level of the factor corresponding to that observation. If you summed together the elements of each row you will always get a vector of ones, or a J matrix with just one column. 4.2 The summary Function The data frame, beefdat, was created earlier. Now enter summary(beefdat) # Also try plot(beefdat) This gives information about each column of the data frame. If appropriate it gives the minimum and maximum value, median, and mean of numeric columns. For nonnumeric columns it gives the levels of that column and number of observations for each, or the total number of levels. This is useful to check if data have been entered correctly or if there are codes in that data that were not expected. The information may not be totally correct. For example, if missing birthweights were entered as 0, then R does not know that 0 means missing and assumes that 0 was a valid birthweight. The letters NA, signify a missing value, and these are skipped in the summary function. 7

8 4.3 Means and Variances The functions to calculate the means and variances are straightforward. Let y be a vector of the observations on a trait of interest. # The mean is mean(y) # The variance and standard deviation are var(y) sd(y) 4.4 Plotting The plot function is handy for obtaining a visual appreciation of the data. There are also the hist(), boxplot(), and qqnorm() functions that plot information. Use the?hist, for example, to find out more about a given function. This will usually show you all of the available options and examples of how the function is used. There are enough options and additional functions to make about any kind of graphical display that you like. 5 Large Data Sets Large, in these notes, means a data set with more than 30 observations. This could be real data that exists in a file on your computer. There are too many observations and variables to enter it manually into R. The data set can be up to 100,000 records, but R is not unlimited in space, and some functions may not be efficient with a large number of records. If the data set is larger than 100,000 records, then other programming approaches like FORTRAN or C++ should be considered, and computing should be on servers with multiprocessors and gigabytes of memory. For example, setting up a design matrix for a factor with a large data set may require too much memory. Techniques that do not require an explicit representation of the design matrix should be used. A chapter on analyses using this approach is given towards the end of the notes. To read a file of trotting horse data, as an example, into R, use 8

9 zz = file.choose() # allows you to browse for file # zz is the location or handle for the file trot = read.table(file = zz, header=false, col.names= c("race","horse","year","month","track","dist","time")) When a data frame is saved in R, it is written as a binary file, and the names of the columns form a header record in the file. Normally, data files do not have a header record, thus, header=false was indicated in the read.table() function, and the col.names had to be provided, otherwise R provides its own generic header names like V1, V2, Exploring the Data summary(trot) # as before with the small data sets dim(trot) # will indicate number of records and # number of columns in trot data frame horsef = length(factor(trot$horse)) # number of different horses # in the data set yearsf = length(factor(trot$year)) # number of different years yearsf = factor(trot$year) levels(yearsf) # list of years represented in data tapply(trot$times,trot$track,mean) # mean racing times by # track location 9