NONPARAMETRIC REGRESSION WIT MEASUREMENT ERROR: SOME RECENT PR David Ruppert Cornell University

Transcription

1 NONPARAMETRIC REGRESSION WIT MEASUREMENT ERROR: SOME RECENT PR David Ruppert Cornell University davidr (These transparencies, preprints, and references a link to Recent Talks and Recent Papers Work done jointly with Scott Berry, Berry Consultants R. J. Carroll, Texas A & M Jeff Maca, Novartis John Staudenmayer, U Mass

2 OUTLINE The problem nonparametric regression with m error Review of the currently available estimators New Bayesian spline approach (Berry, Carroll, 2002, JASA) Simulation results

3 THE PROBLEM OF MEASUREMENT ERR ILLUSTRATION 1.5 Gold Standard data true curve Spline Data with measureme y y x w

4 THE PROBLEM OF MEASUREMENT ERR ILLUSTRATION Gold Standard data true curve Spline Data with measureme y 0 y x w

5 THE PROBLEM OF MEASUREMENT ERR The regression model is where is only known to be smooth Observe and where var( ) = normally distributed Measurement error variance is estimated from! cate data. (Observe,.)

6 THE PROBLEM OF MEASUREMENT ERROR Measurement error occurs in a wide variety of pro Measuring nutrient intake Measuring airborne lead exposure Measuring blood pressure C"$# dating The effects of measurement error are: biased estimates of the regression curve increase in the perceived variability about the re

7 THE PROBLEM OF MEASUREMENT ERROR Other than the work of Fan and Truong (1993, A had been little done on nonparametric regression w ment error until Carroll, Maca, and Ruppert (1999, Biometrika) ( Berry, Carroll, Ruppert (2002, JASA) (BCR)

8 % REVIEW OF CURRENT ESTIMATORS Globally consistent nonparametric regression by d kernels (Fan and Truong, 1993, Annals) does not work so well % Fan & Truong show very poor asymptotic rat gence we have simulations showing poor finite-samp no methodology for bandwidth selection or infer

9 REVIEW OF CURRENT ESTIMATORS Standard measurement error method: SIMEX ' functional no assumptions on & very general can be applied to nearly any m error problem, parametric or nonparametric Structural Spline Regression splines for basic regression model Mixtures of normals for covariate density model Emphasis is on flexible parametric modeling, not ric modeling. (Little or no difference in practice.)

10 SIMEX The SIMEX method is due to Cook & Stefanski (1 The theory is in Carroll, et al. (1996, JASA) Also see Carroll, Ruppert, and Stefanski (1995, M Error in Nonlinear Models) SIMEX has been previously applied to parametric makes no assumptions about the true s. (Fun results in estimators which are approximately co consistent at least to order ( Here is the method, defined via a graph. ).

11 SIMEX, ILLUSTRATED Illustration of SIMEX SIMEX Estimate 0.8 Coefficient Naive Estimate * * + +, Measurement Error Variance

12 SIMEX CMR applied the SIMEX to nonparametric regress CMR have asymptotic theory in the local polynom (LPR) context. The estimators have the usual rates of converge They are approximately consistent, to order ( An asymptotic theory with rates seems very difficu but, simulations in CMR indicate that SIMEX/sp little better than SIMEX/kernel problem seems due to undersmoothing

13 SIMEX Staudenmayer (2000, Cornell PhD thesis) looked selection for SIMEX/LPR. With better bandwidth selection, SIMEX/LPR i with other methods.

14 9 STRUCTURAL MODELING The regression of on the observed is - / If we had a model 687 for and if we knew we could estimate 687 by minimizing over the ;: " < We need two things to make this work: convenient flexible form for convenient flexible distribution for. >=?1

15 H H D Model REGRESSION SPLINES D I " A 647 BC EJ A E:GF E: The key remaining issue: the joint distribution of ' CMR used a mixtures of normals for & and Gi to estimate the parameters. % This is an extension to measurement error Roeder & Wasserman (JASA, 1997).

16 FULLY BAYESIAN MODEL What s New? Answer: Fully Bayesian MCMC method in BCR Uses splines smoothing or penalized P-splines in this talk Structural are iid normal but seems robust to violations of normality

17 FULLY BAYESIAN MODEL Smoothing parameter is automatic Inference adjusts for the data-based smoothing p for measurement error Allow global confidence bands

18 FULLY BAYESIAN MODEL PARAMET Regression Model is a P-spline ;L iid K M Measurement Error Model N where N iid K ;L Structural Model AOPQ P iid K Parameters: 7 R SO P P

19 % P R R T Priors 7 is K FULLY BAYESIAN MODEL PARAMET 5L UT SV " ing parameter.] T ;Y Z[]\ Z^ is Gamma R is Inv-Gamma ;Y R ]\ is Inv-Gamma ;Y ]\ OP _=`PQSa P is K ;Y P ]\ P is Inv-Gamma Hyperparameters: Y R where is known. [W ]\ by ]\ by PQ]\ PQc=`PdSa BX R i P by Z all fixed at values making the priors noninformat E.g., a P e ).

20 % % & " Iterate through 7 R GIBBS SAMPLING P SO P ft S gs 9. All steps except one are easy, either gamma, inv or normal E.g., Here 7 ih 'kj l monqp rts other parameters u v/rtw x T V " xkh y m/z R x T V " is one of the other parameters Essentially we re fitting a spline to the impute observed s

21 % 7 Estimate of 7, call it { GIBBS SAMPLING, is x T V " x h averaged over T and.

22 < &. < R 9 GIBBS SAMPLING The exception to the sampling being quick and e Metropolis-Hastings step is needed for S " OPQ M P 7 } vi~>k < E: " 647 ƒ0 ih ' < < P A < OP>

23 7 7 W p BAYESIAN INFERENCE Let be the spline basis function evaluated on a i c' some interval, &. i c' is the curve on &.. { is the estimated curve. Let ˆ be the 4 < W MCMC sample quantile of 7 7 rš~ < { grid SD 7 Then, 7 { ˆ Œ cž SD 7 is a < i c' &. % simultaneous confidence band fo

24 7 7 W p ˆ BAYESIAN INFERENCE Let be derivatives of the spline basis function e i c' fine grid over &. i c' is the curve s derivative on &. { Let ˆ Then, is a < i i' on &. is the estimated derivative. be the 4 < W MCMC sample quantile of 7 7 rš~ < { grid SD 7 7 { Œ cž SD % simultaneous confidence band for 7

25 SIMULATIONS The six cases were considered. in each case. Case 1 The regression function is w A 1-12 S 1 wš E0. S e LC LCœ with, M,, OP P and Gold Standard data true curve Spline y x

26 Case 2 Same as Case 1 except.

27 Case 3 A modification of Case 1 above except that

28 1 J < Case 4 Case 1 of CMR so that 1[Ÿd -1, with OP L P L and. -12 et1d J 8, M 12 ž J L L,

29 Case 5 A modification of Case 4 of CMR so that with g, M LCo -12 e, S L i wš 12 ], OP Lª and

30 Case 6 The same as Case 1 above except that is chi-square(4) random variable. (Tests robustness tion of the structural assumptions.)

31 Mean Squared Bias «k Method Case 1 Case 2 Case 3 Case 4 Ca Naive ,108 3, Bayes Structural, 5 knots Structural, 15 knots Mean Squared Error «[ Method Case 1 Case 2 Case 3 Case 4 Ca Naive ,155 3, Bayes , Structural, 5 knots , Structural, 15 knots Results based on 200 Monte Carlo simulations for each was not included in the table it was not among the b

32 wš 8 is K R L L EXAMPLE SIMULATED for all ± 15 knot quadratic P-splines 2,000 iterations of Gibbs. First 667 deleted as bur

33 Gold Standard Data with measur y 0 y x wbar wbar mhat x x

34 < 10 beta(1) 0.5 yhat(1) 1.5 yhat(51) yhat(101) log(gamma) x(1) log(σ ) e sigma u mu x Results of Gibbs Sampling. Every twentieth ite Note: 4 ² ³ 4 LCœ < and. Also, log

35 < < < < <. { EXAMPLE SIMULATED What does the Bayes approach work so well? Here s tion: Bayes uses all possible information to estimate cially,. <. µ A µ A A ih other param. 0 rez v S0 ² ` ih. other param. 0 _rez v 0. { A 4 el C

36 DISCUSSION With the work of CMR and BCR we now have r ficient estimators for nonparametric regression w ment error. SIMEX (LPR and splines) in CMR (Flexible) Structural splines in CMR Fully Bayesian (hardcore structural) in BCR

37 DISCUSSION With BCR we have a methodology that automatically selects the amount of smoothing estimates the unknown s allows inference that takes account of the effects parameter selection and measurement error Most efficient methods appear to be structural, th may be competitive hardcore structural methods seem reasonably r