Applying multiple imputation on waterbird census data Comparing two imputation methods

Transcription

1 Applying multiple imputation on waterbird census data Comparing two imputation methods ISEC, Montpellier, 1 july 2014 Thierry Onkelinx, Koen Devos & Paul Quataert Thierry.Onkelinx@inbo.be

2 Contents 1 Introduction 2 Testing the Underhill index 3 An alternative way of imputing 4 Testing multiple imputation using INLA 5 Conclusions 2 / 22

3 1 Introduction 2 Testing the Underhill index 3 An alternative way of imputing 4 Testing multiple imputation using INLA 5 Conclusions 3 / 22

4 Waterbird census in Flanders (Belgium) Aim: monitor wintering birds Total number Average over months per winter Data collected by volunteers 1200 sites 23 winters 6 months per winter Missing data on average 26% (9% - 42%) Impution required / 22

5 Underhill-index Described by Underhill and Prys-Jones (1994) Algorithm 1 Replace all missings with starting value 2 Fit model to imputed dataset 3 Predict missing data 4 Replace imputed value with rounded prediction when prediction is larger 5 Re-iterate from 2. until imputations are stable Negative binomial GLM with global effects for winter, month and site 5 / 22

6 Underhill-index Described by Underhill and Prys-Jones (1994) Algorithm 1 Replace all missings with starting value 2 Fit model to imputed dataset 3 Predict missing data 4 Replace imputed value with rounded prediction when prediction is larger 5 Re-iterate from 2. until imputations are stable Negative binomial GLM with global effects for winter, month and site Potential problems Imputed values can never decrease: risk for bias Imputing with model predictions: risk for reduced standard errors 5 / 22

7 1 Introduction 2 Testing the Underhill index 3 An alternative way of imputing 4 Testing multiple imputation using INLA 5 Conclusions 6 / 22

8 Test setup Generate dataset (40 sites, 24 winters, 6 months) Remove 25% data completely at random Impute missing data Calculate total per winter and month over all sites Model totals Poisson regression with overdisperion Estimate per winter 7 / 22

9 Test setup Generate dataset (40 sites, 24 winters, 6 months) Remove 25% data completely at random Impute missing data Calculate total per winter and month over all sites Model totals Poisson regression with overdisperion Estimate per winter Relative bias: exp(β Underhill ) exp(β complete ) Relative SE: σ β Underhill σ βcomplete 7 / 22

10 Test setup Generate dataset (40 sites, 24 winters, 6 months) Remove 25% data completely at random Impute missing data Calculate total per winter and month over all sites Model totals Poisson regression with overdisperion Estimate per winter Relative bias: exp(β Underhill ) exp(β complete ) Relative SE: σ β Underhill σ βcomplete 1 Original Underhill-index, starting value = zero 2 Original Underhill-index, starting value = geometric mean 3 Altered Underhill-index, starting value = zero 4 Altered Underhill-index, starting value = geometric mean Altered index: replace imputation with rounded predictions 7 / 22

11 Data generating model Counts follow negative binomial distribution Fixed size Variable mean (defined on log-scale) Intercept Linear trend and random walk along winter Random intercept and random walk along winter per site Sine wave within a winter with variable phase among winters Gaussian noise at observation level Different dataset per simulation All datasets based on the same hyperparameters 8 / 22

12 Example of simulated dataset 150 True mean per site Time (winters) True mean per site Site 12 Site 13 Site 18 Site Site 30 Site 31 Site 34 Site Month 9 / Year

13 Evaluation of Underhill index Relative bias Algorithm Original Altered Starting value Mean Zero 80% 100% 120% 140% 160% Relative bias, 100% = complete dataset Relative SE Algorithm Original Altered Starting value Mean Zero 70% 80% 90% 100% Relative SE, 100% = complete dataset 10 / 22

14 1 Introduction 2 Testing the Underhill index 3 An alternative way of imputing 4 Testing multiple imputation using INLA 5 Conclusions 11 / 22

15 Requirements Choose model that doesn t require starting values We choose a negative binomial GLMM Winter and month as fixed effect (factors) Site as random intercept Fitted in R (R Core Team 2014) with INLA (Rue et al. 2009) 12 / 22

16 Requirements Choose model that doesn t require starting values We choose a negative binomial GLMM Winter and month as fixed effect (factors) Site as random intercept Fitted in R (R Core Team 2014) with INLA (Rue et al. 2009) Take the uncertainty of predictions into account Sample from negative binomial distribution Size Mean Sample from distribution of hyperparameter Sample from gaussian distribution Mean and SE of prediction on the link scale 12 / 22

17 1 Introduction 2 Testing the Underhill index 3 An alternative way of imputing 4 Testing multiple imputation using INLA 5 Conclusions 13 / 22

18 Test setup Same test datasets as for testing Underhill-index Fit INLA model to observed dataset Generate M sets of imputed values For each set m Calculate total per winter and month over all sites Model totals Save the regression parameters β i m and their SE σ i m 14 / 22

19 Test setup Same test datasets as for testing Underhill-index Fit INLA model to observed dataset Generate M sets of imputed values For each set m Calculate total per winter and month over all sites Model totals Save the regression parameters β i m and their SE σ i m Aggregate over all M sets (Rubin 1987) σ i = 1 M M m=1 β i = 1 M M m=1 σ 2 i m + M + 1 M β i m M m=1 (β i m β i ) 2 M 1 14 / 22

20 Evaluation of multiple imputation Relative bias Proportion of missing data 75% 50% 25% 5% 1% 60% 80% 100% 120% 140% Relative bias, 100% = complete dataset Relative SE Proportion of missing data 75% 50% 25% 5% 1% 90% 110% 130% Relative SE, 100% = complete dataset 15 / 22

21 Effect of design Relative bias Number of sites Winters % 100% 125% 150% Relative bias, 100% = complete dataset Relative SE Number of sites Winters % 100% 120% 140% 160% Relative SE, 100% = complete dataset 16 / 22

22 Effect of model for imputation Relative bias Model for imputation True mean Random walk per site Constant site 70% 80% 90% 100% 110% 120% 130% Relative bias, 100% = complete dataset Relative SE Model for imputation True mean Random walk per site Constant site 90% 100% 110% 120% 130% Relative SE, 100% = complete dataset 17 / 22

23 Real life examples Anser brachyrhynchus: 10% missing, 18 sites, 4 months Numenius arquata: 24% missing, 208 sites, 6 months Winter average 20,000 10, ,000 Anas platyrhynchos: 40% missing, 852 sites, 6 months 8,000 6,000 4,000 2, ,000 Haematopus ostralegus: 42% missing, 270 sites, 6 months 75,000 50,000 15,000 10,000 25, , Winter 18 / 22

24 1 Introduction 2 Testing the Underhill index 3 An alternative way of imputing 4 Testing multiple imputation using INLA 5 Conclusions 19 / 22

25 Underhill-index Must use zero as starting value Otherwise biased upward Underestimates standard errors Incorrect Type I errors! Too optimistic 20 / 22

26 Multiple imputation Unbiased estimates Increased standard errors Imputation = more uncertainty Implies lower power Sample size actual dataset < sample size complete dataset Increase of standard error depends on 1 Proportion of missing data 2 Size of dataset 3 Imputation model Marginal improvement with increased number of imputations 21 / 22

27 Questions? References R Core Team R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. Rubin, D.B Multiple Imputation for Nonresponse in Surveys. New York, NY: John Wiley & Sons, Ltd. Rue, Håvard, Sara Martino, Finn Lindgren, Daniel Simpson, and Andrea Riebler INLA: Functions Which Allow to Perform Full Bayesian Analysis of Latent Gaussian Models Using Integrated Nested Laplace Approximation. Underhill, L. G., and R. P. Prys-Jones Index Numbers for Waterbird Populations. I. Review and Methodology. Journal of Applied Ecology 31 (3): doi: / / 22