Generalized Additive Models

Transcription

1 :p Texts in Statistical Science Generalized Additive Models An Introduction with R Simon N. Wood

2 Contents Preface XV 1 Linear Models A simple linear model 2 Simple least squares estimation Sampling properties of~ So how old is the universe? Adding a distributional assumption 7 Testing hypotheses about (3 7 Confidence intervals Linear models in general The theory of linear models Least squares estimation of ( The distribution of i!j (~i - f3i) / Cr {1i ""' tn- p F-ratio results The influence matrix The residuals,, and fitted values, P, Results in terms of X The Gauss Markov Theorem: What's special about least squares? The geometry of linear modelling Least squares Fitting by orthogonal decompositions 20

3 viii Comparison of nested models Practical linear modelling Model fitting and model checking Model summary Model selection Another model selection example 31 A follow-up Confidence intervals Prediction Practical modelling with factors Identifiability Multiple factors 'Interactions' of factors Using factor variables in R General linear model specification in R Further linear modelling theory Constraints I: General linear constraints Constraints II: 'Contrasts' and factor variables Likelihood Non-independent data with variable variance AIC and Mallow's statistic Non-linear least squares Further reading Exercises 55 2 Generalized Linear Models The theory of GLMs The exponential family of distributions Fitting generalized linear models The IRLS objective is a quadratic approximation to the log-likelihood 66

4 p ix AIC forglms Large sample distribution of / Comparing models by hypothesis testing 69 Deviance 70 Model comparison with unknown and Pearson's statistic Canonical link functions Residuals 73 Pearson residuals 73 Deviance residuals Quasi-likelihood Geometry of GLMs The geometry of IRLS Geometry and IRLS convergence GLMs with R Binomial models and heart disease A Poisson regression epidemic model Log-linear models for categorical data Sole eggs in the Bristol channel Likelihood In variance Properties of the expected log-likelihood Consistency Large sample distribution of {J The generalized likelihood ratio test (GLRT) Derivation of 2.\ rv x; under Ho AIC in general Quasi-likelihood results Exercises 115

5 X 3 Introducing GAMs Introduction Univariate smooth functions Representing a smooth function: Regression splines 122 A very simple example: A polynomial basis 122 Another example: A cubic spline basis 124 Using the cubic spline basis Controlling the degree of smoothing with penalized regression splines Choosing the smoothing parameter,>.: Cross validation Additive models Penalized regression spline representation of an additive model Fitting additive models by penalized least squares Generalized additive models Summary Exercises Some GAM Theory Smoothing bases Why splines? 146 Natural cubic splines are smoothest interpolators 146 Cubic smoothing splines Cubic regression splines A cyclic cubic regression spline P-splines Thin plate regression splines 154 Thin plate splines 154 Thin plate regression splines 157 Properties of thin plate regression splines 158 Knot-based approximation Shrinkage smoothers 160

6 > Choosing the basis dimension Tensor product smooths Tensor product bases Tensor product penalties 4.2 Setting up GAMs as penalized GLMs Variable coefficient models 4.3 Justifying P-IRLS 4.4 Degrees of freedom and residual variance estimation Residual variance or scale parameter estimation 4.5 Smoothing parameter selection criteria Known scale parameter: UBRE Unknown scale parameter: Cross validation Problems with ordinary cross validation Generalized cross validation GCV/UBRE/AIC in the generalized case Approaches to GAM GCV /UBRE minimization 4.6 Numerical GCV/UBRE: Performance iteration Minimizing the GCV or UBRE score xi Stable and efficient evaluation of the scores and derivatives 183 The weighted constrained case Numerical GCV/UBRE optimization by outer iteration Differentiating the GCV/UBRE function 4.8 Distributional results Bayesian model, and posterior distribution of the parameters, for an additive model Structure of the prior Posterior distribution for a GAM Bayesian confidence intervals for non-linear functions of parameters P-values 4.9 Confidence interval erformance

7 xii Single smooths GAMs and their components Unconditional Bayesian confidence intervals Further GAM theory Comparing GAMs by hypothesis testing ANOVA decompositions and nesting The geometry of penalized regression The "natural" parameterization of a penalized smoother Other approaches to GAMs Backfitting GAMs Generalized smoothing splines Exercises GAMs in Practice: mgcv Cherry trees again Finer control of gam Smooths of several variables Parametric model terms Brain imaging example Preliminary modelling Would an additive structure be better? Isotropic or tensor product smooths? Detecting symmetry (with by variables) Comparing two surfaces Prediction with predict. gam 243 Prediction with lpmatrix Variances of non-linear functions of the fitted model Air pollution in Chicago example Mackerel egg survey example Model development Model predictions 260

8 xiii 5.5 Portuguese larks example Other packages Package gam Package gss Exercises Mixed Models and GAMMs Mixed models for balanced data A motivating example 277 The wrong approach: A fixed effects linear model 278 The right approach: A mixed effects model General principles A single random factor A model with two factors Discussion Linear mixed models in general Estimation of linear mixed models Directly maximizing a mixed model likelihood in R Inference with linear mixed models 295 Fixed effects 295 Inference about the random effects Predicting the random effects REML 298 The explicit form of the REML criterion A link with penalized regression The EM algorithm Linear mixed models in R Tree growth: An example using lme Several levels of nesting Generalized linear mixed models GLMMs with R 312

9 XIV 6.6 Generalized additive mixed models Smooths as mixed model components Inference with GAMMs GAMMswith R A GAMM for sole eggs The temperature in Cairo Exercises 325 A Some Matrix Algebra 331 A.1 Basic computat~onal efficiency 331 A.2 Covariance matrices 332 A.3 Differentiating a matrix inverse 332 A.4 Kronecker product 333 A.5 Orthogonal matrices and Householder matrices 333 A.6 QR decomposition 334 A.7 Choleski decomposition 334 A.8 Eigen-decomposition 335 A.9 Singular value decomposition 336 A.1 0 Pivoting 337 A.l1 Lanczos iteration 337 B Solutions to Exercises B.1 Chapter 1 B.2 Chapter 2 B.3 Chapter 3 B.4 Chapter 4 B.5 Chapter 5 B.6 Chapter 6 Bibliography Index