Abstract. 2.0 Some Fundamental Concepts of AIC. 1.0 Introduction

Transcription

1 Model Selecton n Lnear Mxed Effects Models Usng SAS PROC MIXED Long Ngo, Ischema Research, San Francsco ; Unversty of Calforna, Berkeley, CA Rchard Brand, Unversty of Calforna, Berkeley, CA Abstract Although there are dsadvantages assocated wth model buldng procedures such as backward, forward and stepwse procedures (e.g. multple testng, arbtrary sgnfcance level used n droppng or acqurng varables), many analysts use these procedures and are not aware that alternatve modelng selecton methods exst. Ths paper focuses on model selecton usng the Akake Informaton Crtera (AIC) n the case of lnear mxedeffects models. AIC s fundamental concepts are revewed and two examples are gven to demonstrate ts use through PROC MIXED. Master-level bostatstcans, epdemologsts, and others who are workng wth longtudnal data are encouraged to nvestgate AIC as the tool n modelng repeated measures data..0 Introducton Model selecton s one of the most frequently encountered problems n data analyss. In most observatonal epdemologcal studes, nvestgators frequently attempt to construct the most desrable statstcal model usng the popular methods of forward, backward, and stepwse regresson (4). Of course knowledge of the subject matter plays an mportant role n model selecton, but f based strctly on the data, model selecton s often carred out usng one of the automated procedures bult nto the software, of whch the most popular method s perhaps stepwse model selecton. These methods pose the problem of the arbtrary selecton of the sgnfcance level(s) n allowng a varable to enter nto or to be dropped from the model durng the selecton process (,4 ). There s also the problem of multple testng that comes wth fttng and refttng the model (,4). The ssue s made more complcated n the case of repeated or longtudnal data where selectng the best model means not only to select the best mean structure but also the most optmal varance-covarance structure (0,). Ths paper revews another model selecton method whch helps elmnate the problems assocated wth settng an arbtrary sgnfcance level requred n automated procedures such as stepwse. Usng a crteron lke AIC for selectng a model, bypasses the need to specfy a sgnfcance level n a model buldng process. The Akake Informaton Crtera (AIC) and ts prncples n model selecton wll be descrbed. The paper wll also show how one can set up a model selecton strategy usng AIC n lnear mxed-effects model framework. AIC and other related crtera (SBC, HQ, CAIC) are fully avalable n SAS PROC MIXED. Real data from two case studes wll be presented to show how model selecton usng AIC was used to acheve the desred objectve..0 Some Fundamental Concepts of AIC In order to understand the prncple behnd AIC, one needs to return to the defnton of the Kullback-Lebler nformaton (5,8) whch s consdered to be a measure of the dstance between two densty functons. In a model selecton problem, one would lke to select the model famly whch performs best. The dstance between the true model and the selected model can be represented by the Kullback-Lebler nformaton. If one assumes the true model s densty to be f (.) and the jont densty functon for the selected model to be g(., θ ), where θ s the estmated vector of d parameters by the maxmum lkelhood method, then the Kullback-Lebler dstance can be wrtten as K L f g f x = f x dx ( (.), (., )) log ( ) θ ( ) (*) gx (, θ) where x s the observed sample data of n ndependent observatons. In ths setup, the expectaton of (*) provdes the bass for model selecton, and the estmate of ths expectaton provdes a crteron for model selecton. The asymptotc approxmaton for the estmaton of ths expectaton s gven as (6,8) E f g f x f x dx g x ( (.), (., θ)) = + + K L ( )log( ( )) ( log( (, θ))) trace( ΣΩ )( ) where, log gxθ (, ) s the maxmzed log lkelhood functon based on the observed data, and θ s the maxmum lkelhood estmate of the parameter vector θ, and Σ = n and = Ω = n d n = n d log g( x, θ ) d log g( x, θ ), uv, =,,..., d, dθ u dθ v log g( x, θ ), uv, =,,..., d dθ udθ v

2 Notce that the term f ( x)log( f ( x)) dx s dependent on the true model whch s unknown; however, ths term s fxed when comparng between models. The part of ths estmated expectaton of the Kullback-Lebler dstance that s needed n the comparson process s log gx (, θ) + trace( ΣΩ ) () whch s computable for the parametrc model. When one can assume that the true model s contaned wthn the famly of models from whch the ftted model s obtaned then one can wrte () as log gx (, θ) + d (3). AIC s bascally twce the expresson of (3) so AIC = -loglkelhood + d (4). Thus AIC mposes a penalty of two unts per parameter n the model. Based on AIC, as the model selecton crteron, then among all possble models consdered, the one wth the smallest value of AIC s consdered to be the best model. Notce that f the true model f(.) s very dfferent (not contaned n the famly model that generates the selected model) from g(., θ ), then one should compute the trace term of ΣΩ drectly whch s more dffcult and computatonally expensve. 3.0 Model selecton usng AIC for lnear mxedeffects models Suppose there are n ndependent subjects wth m correlated measurements, =,,..., n. The margnal probablty densty of y s f( y b, Σ ( φ)) = exp( ( y x b y x b m ) Σ ( φ) ( ) ( π) Σ ( φ) The vector of both fxed and random parameters s θ =( φ, b ). So the maxmzed log lkelhood s then m log f ( y) = log π + log ( φ) ( y x b) ( φ) Σ Σ ( y x b) θ From here AIC can be computed easly wth d beng the sum of both the fxed and random effects parameters. The SAS system verson 6. gves ths value AIC for both the maxmum lkelhood and the restrcted maxmum lkelhood case (). One can also compute the trace term of ΣΩ gven n () snce the densty functon of the ftted model s known. There are two dfferent methods presented here for selectng the "best" or the "smallest AIC" model among all the models under consderaton. The frst method essentally dentfes all the possble mean functons and all the possble varance-covarance structures applcable to the queston of nterest. The number of possble models for consderaton ncludes all combnatons resulted from both the mean and varance-covarance structures. For each of these models, AIC s then computed and the model wth mnmum AIC s selected. The second method recommended by Wolfnger and Dggle (,,0) has the followng steps : Frst, usng the most complex mean structure under consderaton, select the best varance-covarance structure usng the restrcted maxmum lkelhood (REML). REML whch focuses on the covarance sde of the model should be used n place of the lkelhood. Let AIC_R denotes AIC derved from REML : AIC_R = -*(Restrcted lkelhood) + *(# covarance parameters). The varance-covarance structure wth the smallest AIC_R s selected. Then usng ths varance-covarance structure, go back and use AIC to select the best mean structure. In the followng example, both methods wll be llustrated. 4.0 Example The data are from 006 subjects who partcpated n an epdemology study on agng. Each subject was measured every mnute for 3 to 30 mnutes (TIME) from the start of a treadmll test. The response called VOKG, whch measures physcal ftness, s the amount of expended energy from exercse. The hgher the VOKG, the better the physcal performance. DURATION stands for the total tme (mnutes) the subject was able to exercse. AGE s n years. The analyss s done for each gender. Ths s a repeated measures study where the objectve s to construct the normogram shown n fgure. Table shows all the mean structures and varance-covarances of nterest. A total of 8 models were ftted, each has a AIC value. For each of 8 models consdered, the rounded AIC values are gven n table. Table : AIC values for all models consdered TIME DURA TION AGE TIME+ DURA TION TIME + AGE TIME + DURA TION+ AGE Smple CS AR()

3 The mnmum value of AIC=6384 from the model wth mean structure of TIME+DURATION+AGE and varance-covarance AR() ndcates the best model among the 8 consdered. The predcted value of the response VOKG was computed from ths model and then smoothed to allow the constructon of the normogram whch the physcan can use to classfy the patent nto the approprate percentle category. Fgure showed the normogram for the male group constructed by usng the predcted VOKG from the selected model. symmetry+eye for example, means that the R matrx n PROC Mxed specfcaton s of type compound symmetry and eye s the random effect. Fgure shows the AIC for all 5 models, and the lowest AIC (the best model), s from the model wth eye as the only fxed effect, and compound symmetry as the varance-covarance structure O r g n a l A I C Intercept Eye Vst Eye Vst Eye Vst Eye*V Mean Structure Cov Structure Smple CS CS+Eye CS+Vst CS+Eye+Vst Appendx shows the SAS macro program for automatng the selecton procedure for ths example. Also usng the second method, the same model was chosen as the best model. Usng method, the model wth the most complex mean structure among those consdered s the one wth TIME+DURATION+AGE. Ths model s ftted usng the three varance-covarance structures and the AIC_R s mnmum for AR(). Then usng AIC to reduce the mean structure. In ths case AIC chooses the mean structure TIME+DURATION+AGE so the model selected s exactly the same as the one chosen n method. 5.0 Example Data collected on 9 ndvduals, each had 3 vsts (one or two weeks apart, except for one subject wth one vst, and one other subject wth two vsts). For each vst, one measurement was made on each eye to assess the permeablty (Pdc) to sodum fluorescen of the corneal epthelum. The objectve s to obtan the varance component estmates from the best model. There are fve mean structures of nterest (ntercept, eye, vst, eye+vst, eye+vst+eye*vst) and fve varance-covarance structures of nterest (smple, compound symmetry, compound symmetry+eye, compound symmetry+vst, compound symmetry+eye+vst) resultng n 5 models. In ths descrpton of the canddate models, compound 6.0 Addtonal ssues The AIC s known to be nconsstent (,8,9). Other crtera such CAIC (), SBC (7), HQ (3) are consstent crtera whch are also avalable n SAS 6.. Model selecton usng formula () whch has the trace term may have better consstency property than the AIC but ths has not been mplemented. Usng smulaton, we are currently evaluatng the performance and consstency of the crteron usng the trace term formula (). Lnhart and Zuchn (7) also dscuss other dscrepancy measures whch emphasze on the fxed effects sde of the lnear mxedeffects model. In certan practcal stuatons, these may be preferable to the Kullback-Lebler based crtera. 7.0 Concluson Ths papers revews some fundamental concepts of the AIC and shows how model selecton for lnear mxed-effects models can be done usng AIC. Analysts are encouraged to utlze selecton crtera such as AIC and others whch are already mplemented n the SAS software. We presented 3

4 two dfferent procedures for selectng the best model. The procedure suggested by Dggle and Wolfnger of usng REML for varance-covarance structure selecton, and ML for mean structure selecton has a computatonal advantage. The number of possble models evaluated for ths procedure s equal to the sum of the mean structures and varance-covarance structures n consderaton. The method we proposed usng all combnatons of mean structures and varance-covarance structures of nterest s more computatonally expensve especally when the number of mean structures and/or varance-covarance structures s large. However, ths method provdes a more comprehensve examnaton of all models evaluated. 8.0 Reference. Bozdogan, H. (987). Model selecton and Akake s nformaton crtera (AIC) : the general theory and ts analytcal extensons. Psychometrka 5, Dggle, P; Lang, K; Zeger, S. (994). Analyss of Longtudnal Data, Oxford Press. 3. Hannan, E.J. and Qunn, A.G. (979). The determnaton of the order of an autoregresson. Journal of Royal Statstcal Socety, Vol. B 4, Hosmer, D.W. and Lemeshow, S. (989). Appled Logstc Regresson. New York : John Wley and Sons, Inc. 5. Kullback, Solomon (978). Informaton Theory and Statstcs, Massachusetts: Peter Smth. 6. Lnhart, H and Zuchn, W. (986). Model Selecton. New York : John Wley and Sons, Inc. 7. Schwarz, G. (978). Estmatng the dmenson of a model. Annual of Statstcs, Vol. 6, Shbata, Rte (989). From Data to Model. New York : Sprnger-Verlag, Shbata, Rte (986). Consstency of model selecton and parameter estmaton. Essays n Tme Seres and Alled Processes. Appled Probablty Trust, Wolfnger, R.D. (993). Covarance structure selecton n general mxed models. Communcatons n Statstcs, Smulaton and Computaton, Vol., Wolfnger, R.D. (996). Heterogeneous varancecovarance structures for repeated measures. Journal of Agrcultural, Bologcal, and Envronmental Statstcs, Vol., Number, SAS, SAS/STAT, SAS/GRAPH are regstered trademarks or trademarks of SAS Insttute Inc. n the USA and other countres. ndcates USA regstraton. 0.0 Authors Long Ngo Unversty of Calforna, Berkeley 40 Warren Hall Berkeley CA 9470 (50) lhn@oron.ref.org Rchard Brand, Ph.D. Unversty of Calforna, Berkeley Havland Hall Berkeley CA 9470 (50) brand@stat.berkeley.edu 9.0 Acknowledgements The authors would lke to thank Dr. Ira Tager for the exercse data n example, Dr. Mark Segal for hs advce, and Ischema Research for provdng fundng and support for ths paper to be presented at SUGI. 4

5 Appendx /********************************************* Program : ex5.sas Tme : :33:07 pm By : Long Ngo Input : e:\brand\pcerept\saslb\pcerept.sd Output : Purpose : Automatng the macro for model selecton for example mplementng adjustment for AIC and consstent AIC estmate. *********************************************/ optons ls=80 ps=60 pageno= mprnt;; lbname rjb e:\brand\pcerept\saslb ; ttle ex5.sas ; data a; set rjb.pcerept; format _all_; *strp all defned formats off the data; proc sort; by d; data rjb.acnfo; *establsh the fnal data set structure; length mean $5 randvar repttype $5 crtera $3 ac nobs nfxed ncov nsubj 5; %macro v(mean,repttype); * macro for handlng just fxed effects; proc mxed data=a method=ml; class d eye vst; model lnpce = &mean ; repeated / type=&repttype subject=d; ttle "Model wth Mean Structure = &mean "; ttle3 "Repeated Type = &repttype"; make fttng out=aout; make SolutonF out=aoutfx; make CovParms out=aoutcov; %f; %mend; %macro vr(mean,randvar,repttype); * macro for handlng both fxed and random effects; proc mxed data=a method=ml; class d eye vst; model lnpce = &mean ; random &randvar / subject=d; repeated / type=&repttype subject=d; ttle "Model wth Mean Structure = &mean "; ttle3 "Random Effects = &randvar"; ttle4 "Repeated Type = &repttype"; make fttng out=aout; make SolutonF out=aoutfx; make CovParms out=aoutcov; %f; %mend; %macro f; data acnfo; length mean $5 randvar repttype $5; set aout; mean="&mean"; randvar="&randvar"; repttype="&repttype"; where descr=: Akake ; ac=value; data nobs; set aout; where descr=: Observaton ; nobs=value; *obtan number of observatons n analyss; data tfx; set aoutfx (keep=est); f est ne 0; proc unvarate noprnt; var est; output out=nfxed n=nfxed; *get the number of fxed effect parameters; data tcov; set aoutcov (keep=est); proc unvarate noprnt; var est; output out=ncov n=ncov; *get the number of covarance parameters; *get the number of avalable subjects for each model; data nsubj; set rjb.pcerept (keep=d); proc sort; by d; data nsubj; set nsubj; by d; f frst.d; c=; proc unvarate noprnt; var c; output out=nsubj sum=nsubj; * construct the desred data set; data acnfo; merge acnfo nfxed nobs ncov nsubj; data rjb.acnfo; set rjb.acnfo acnfo; %mend; 5

6 *(mean repeated type for macro %v) and (mean random repeated type for %vr); %v(/ s,smple); %v(/ s,cs); %vr(/ s,eye,cs); %vr(/ s,vst,cs); %vr(/ s,eye vst, cs); %v(eye / s,smple); %v(eye / s,cs); %vr(eye / s,eye,cs); %vr(eye / s,vst,cs); %vr(eye / s,eye vst, cs); %v(vst / s,smple); %v(vst / s,cs); %vr(vst / s,eye,cs); %vr(vst / s,vst,cs); %vr(vst / s,eye vst, cs); %v(eye vst / s,smple); %v(eye vst / s,cs); %vr(eye vst / s,eye,cs); %vr(eye vst / s,vst,cs); %vr(eye vst / s,eye vst, cs); %v(eye vst eye*vst / s,smple); %v(eye vst eye*vst / s,cs); %vr(eye vst eye*vst / s,eye,cs); %vr(eye vst eye*vst / s,vst,cs); %vr(eye vst eye*vst / s,eye vst, cs); proc format; value meantype = Intercept = Eye 3= Vst 4= Eye Vst 5= Eye Vst Eye*Vst ; value covtype = Smple = CS 3= CS+EYE 4= CS+Vst 5= CS+Eye+Vst ; * set up the data for graphng; end; else f mean= eye / s then meantype=; else f mean= vst / s then meantype=3; else f mean= eye vst / s then meantype=4; else f mean= eye vst eye*vst / s then meantype=5; f randvar= None and repttype= smple then covtype=; else f randvar= None and repttype= cs then covtype=; else f randvar= eye and repttype= cs then covtype=3; else f randvar= vst and repttype= cs then covtype=4; else f randvar= eye vst and repttype= cs then covtype=5; f meantype= then ncov=ncov-; *adjust for the resd; logl = ac+ncov ; *Mxed defnes AIC = logl - number of cov parameters; *ths s to get back logl value; ac = -*logl + *(nfxed+ncov); *recompute orgnal def. of AIC; ac_con = -*logl + (nfxed+ncov)*(log(nsubj)+); *consstent AIC estmate; label meantype= Mean Structure covtype = Cov Structure ac = Orgnal AIC ac_con = Consstent AIC logl = Log Lkelhood nobs = Number of Obs n Analyss nsubj = Number of Subjects n Analyss nfxed = Number of Fxed Effects Parameters ncov = Number of COV parameters ; format meantype meantype. covtype covtype.; drop descr value mean; proc sort ; by meantype covtype; goptons devce = wnprtm; symbol c=black v=pont =j l= wdth=3; symbol c=black v=crcle =j l= wdth=3; symbol3 c=black v=plus =j l=3 wdth=3; symbol4 c=black v=star =j l=4 wdth=3; symbol5 c=black v=damond =j l=5 wdth=3; axs order = ( to 5) ; proc gplot; plot ac * meantype = covtype / haxs=axs; ttle h=0.75 Fgure : AIC Values Computed for 5 Models n Example ; data rjb.t; set rjb.acnfo; f randvar =: & then randvar= None ; f _n_= then delete; *drop the null case used for settng; f mean= / s then do; mean= Intercept ; meantype=; 6