EXST7034 Regression Techniques Geaghan Logistic regression Diagnostics Page 1

Transcription

1 Logstc regresson Dagnostcs Page 1 1 **********************************************************; 2 *** Logstc Regresson - Dsease outbreak example ***; 3 *** NKNW table 14.3 (Appendx C3) ***; 4 *** Study of a dsease outbreak from a mosquto born ***; 5 *** dsease wthn two sectors of a cty. ***; 6 **********************************************************; 7 8 dm'log;clear;output;clear'; 9 optons nodate nocenter nonumber ps=512 ls=132 nolabel; 10 ODS HTML style=mnmal rs=none body='c:\geaghan\current\exst7034\fall2005\sas\dseaseoutbreak01.html' ; NOTE: Wrtng HTML Body fle: C:\Geaghan\Current\EXST7034\Fall2005\SAS\DseaseOutbreak01.html TITLE1 ''; 13 data Dsease; nfle cards mssover; 14 nput case Age Status1 Status2 sector Dsease; 15 * Status classes are upper (0, 0), Mddle (1, 0) and Lower (0, 1); 16 status = 'Upper '; 17 f status1 eq 1 then status = 'Mddle'; 18 f status2 eq 1 then status = 'Lower'; 19 label case = 'case number' 20 age = 'Patents age' 21 status = 'Socoeconomc status upper, mddle and lower' 22 dsease = 'Dsease present = 1'; 23 Cards; NOTE: The data set WORK.DISEASE has 98 observatons and 7 varables. NOTE: DATA statement used (Total process tme): 0.02 seconds 0.02 seconds 122 ; proc logstc data=dsease DESCENDING alpha=0.05; 125 TITLE2 'Logstc regresson on Dsease data (wth Status1 and Status2)'; 126 model Dsease = Age Status1 Status2 Sector; 127 run; NOTE: PROC LOGISTIC s modelng the probablty that Dsease=1. NOTE: Convergence crteron (GCONV=1E-8) satsfed. NOTE: There were 98 observatons read from the data set WORK.DISEASE. NOTE: The PROCEDURE LOGISTIC prnted page 1. NOTE: PROCEDURE LOGISTIC used (Total process tme): 0.25 seconds 0.04 seconds proc logstc data=dsease DESCENDING alpha=0.05; 130 class status; 131 TITLE2 'Logstc regresson on Dsease data (wth CLASS Status - default)'; 132 model Dsease = Age Status Sector; 133 run; NOTE: PROC LOGISTIC s modelng the probablty that Dsease=1. NOTE: Convergence crteron (GCONV=1E-8) satsfed. NOTE: There were 98 observatons read from the data set WORK.DISEASE. NOTE: The PROCEDURE LOGISTIC prnted page 2. NOTE: PROCEDURE LOGISTIC used (Total process tme): 0.19 seconds 0.03 seconds proc logstc data=dsease DESCENDING alpha=0.05; 136 class status / param = glm; 137 TITLE2 'Logstc regresson on Dsease data (wth CLASS Status - GLM)'; 138 model Dsease = Age Status Sector / lackft RSQ plots; 139 output out=next1 PREDICTED=yhat Lower=lcl95 Upper=ucl95 dfbetas=_all_ 140 resdev=resdev dfdev=dfdev; 141 run; NOTE: PROC LOGISTIC s modelng the probablty that Dsease=1. NOTE: Convergence crteron (GCONV=1E-8) satsfed. NOTE: There were 98 observatons read from the data set WORK.DISEASE. NOTE: The data set WORK.NEXT1 has 98 observatons and 19 varables. NOTE: The PROCEDURE LOGISTIC prnted pages 3-4.

2 Logstc regresson Dagnostcs Page 2 NOTE: PROCEDURE LOGISTIC used (Total process tme): 0.23 seconds 0.10 seconds The three PROC LOGISTIC analyses dffer n only one regard, and all results are the same except the parameter estmates. There are 3 soco-economc levels and many dfferent ways of settng up the dummy varables. Several are explored below. Make sure you know what you are estmatng. The frst verson uses the status dummy varables provded wth the data set where two varables status1 and status2 have the followng values for the 3 soco-economc levels: Upper (0, 0), Mddle (1, 0) and Lower (0, 1). I created a class varable STATUS wth the three levels coded as Lower, Mddle, Upper. In the second PROC LOGISTIC the STATUS varable was placed n the class statement. In the thrd PROC LOGISTIC, for whch a full analyss s provded, the opton / param = glm; was requested on the CLASS statement. Logstc regresson on Dsease data (wth Status1 and Status2) The LOGISTIC Procedure Analyss of Maxmum Lkelhood Estmates Standard Wald Parameter DF Estmate Error Ch-Square Pr > ChSq Intercept Age Status Status sector Odds Rato Estmates Pont 95% Wald Effect EstmateConfdence Lmts Age Status Status sector Logstc regresson on Dsease data (wth CLASS Status - default) The LOGISTIC Procedure Class Level Informaton Desgn Class Value Varables status Lower 1 0 Mddle 0 1 Upper -1-1 Note the codng of the dummy varables, contrastng Lower and Mddle to Upper.

3 Logstc regresson Dagnostcs Page 3 Analyss of Maxmum Lkelhood Estmates Standard Wald Parameter DF Estmate Error Ch-Square Pr > ChSq Intercept <.0001 Age status Lower status Mddle sector Odds Rato Estmates Pont 95% Wald Effect EstmateConfdence Lmts Age status Lower vs Upper status Mddle vs Upper sector Logstc regresson on Dsease data (wth CLASS Status - GLM) The LOGISTIC Procedure Model Informaton Data Set WORK.DISEASE Response Varable Dsease Number of Response Levels 2 Model bnary logt Optmzaton Technque Fsher's scorng Number of Observatons Read 98 Number of Observatons Used 98 Response Profle Ordered Total Value Dsease Frequency Probablty modeled s Dsease=1. Class Level Informaton Class Value Desgn Varables status Lower Mddle Upper Note the GLM-lke codng of the dummy varables. Model Convergence Status Convergence crteron (GCONV=1E-8) satsfed. Model Ft Statstcs Intercept Intercept and Crteron Only Covarates AIC SC Log L R-Square Max-rescaled R-Square

4 Logstc regresson Dagnostcs Page 4 Model ft statstcs 1) Akake Informaton Crteron AIC = 2log( L) + 2 p where Log(L) s the log lkelhood and p s the number of parameters 2) Schwarz Crteron SC = 2log( L) + plog( f j ) 3) 2log L 2 [ Y log ( ˆ ) (1 )log (1 ˆ e π + Y e π ) ] n 1 Ths s analogous to the SSE n regresson and s gven n SAS as the 2 Log L. Two models (full and reduced) can be compared by calculatng the dfference n 2 Log L for both models. Ths dfference follows a ch square dstrbuton wth a d.f. equal to the dfference n d.f. for the two models. 2 n 4) Generalzed R 2 L(0) 1, where L(0) s the ntercept only model. L( θ ) Snce ths value reaches ts maxmum of less than 1 for dscrete models an adjustment has 2 R been proposed. Ths s called the Max-rescaled Rsquare n SAS. 2 R j max Testng Global Null Hypothess: BETA=0 Test Ch-Square DF Pr > ChSq Lkelhood Rato Score Wald All three tests are jont tests of the regresson coeffcents (β ) aganst 0. Lkelhood rato test Ths test compares the -2 Log L values for the model wth an ntercept only model (see Model ft statstcs above). The dfference ( ) follows a ch square dstrbuton wth degrees of freedom equal to p 1, the number of parameters not countng the ntercept. Ths s analogous to the SSE n regresson and s gven n SAS as the -2 Log L. Any two models (full and reduced) can be compared by calculatng the dfference n -2 Log L for both models. Ths dfference follows a ch square dstrbuton wth a d.f. equal to the dfference n d.f. for the two models. Ths test s analogous to the General Lnear Hypothess test for lnear models. Score statstcs These are based on vectors of the partal dervatves of the log lkelhood (wrt the parameter vector) and the matrx of second partal dervatves. It asymptotcally has a ch square dstrbuton. Wald statstcs are based on large sample statstcs and assume asymptotc normalty. Under the usual large sample condtons the logstc regresson maxmum lkelhood estmators are approxmately normally dstrbuted, have no bas and have varances and covarances that are functons of the partal second dervatve (Hessan) of the log lkelhood functon.

5 Logstc regresson Dagnostcs Page 5 Type 3 Analyss of Effects Wald Effect DF Ch-Square Pr > ChSq Age status sector Analyss of Maxmum Lkelhood Estmates Standard Wald Parameter DF Estmate Error Ch-Square Pr > ChSq Intercept Age status Lower status Mddle status Upper sector Wald : used to test ndvdual parameter estmates and to place confdence ntervals. It s based on a large sample assumpton of asymptotc normalty. Ch-square Test β / Var( β ) = [ β / Stderr( β )] 2 2 ( ˆ β 1.96 ˆ ) ( ˆ 1.96 ˆ ) Confdence nterval ( ) S β S β β P β Odds Rato Estmates Pont 95% Wald Effect Estmate Confdence Lmts Age status Lower vs Upper status Mddle vs Upper sector Assocaton of Predcted Probabltes and Observed Responses Percent Concordant 77.5 Somers' D Percent Dscordant 22.1 Gamma Percent Ted 0.3 Tau-a Pars 2077 c e e = 0.95 Assocaton of Predcted Probabltes and Observed Responses Observatons wth dfferent responses are pared and compared. If the one wth the lower observed response has a lower predcted value t s sad to be concordant. Otherwse they are dscordant or ted. SAS reports concordant, dscordant, tes and the number of pars examned. A number of other statstcs are based on the same nformaton of the number concordant (n c ) and the number dscordant (n d ). Where t s the total number of pars wth dfferent responses and N s the sum of observaton frequences n the data then the followng statstcs can be derved. Note that tes are gven by t n c n d. The statstc c s equal to (n c + 0.5(t n c n d ))/t. Somers D s equal to (n c n d )/t. The Goodman-Kruskal Gamma s (n c n d )/(n c +n d ) and Kendall s Tau-a s (n c n d )/(0.5N(N 1)/

6 Logstc regresson Dagnostcs Page 6 Partton for the Hosmer and Lemeshow Test Dsease = 1 Dsease = 0 Group Total Observed Expected Observed Expected Hosmer and Lemeshow Goodness-of-Ft Test Ch-Square DF Pr > ChSq Goodness-of-ft Most Goodness-of-ft tests (Pearson and devance) requre replcaton n subpopulatons. Ths s often a problem wth contnuous varables n the model. The Hosmer-Lemeshow Goodness-of-Ft Test can be used for sparser data. Ths test s only avalable for bnary models. In ths approach the data are sorted on the bass of ther response probablty (default), and dvded nto approxmately 10 groups (mnmum = 3). See SAS help for detals on the groupng. Once n groups a Ch square statstc s calculated. For each group we have S as the observed number of successes n the group, n as the total number of observatons n the group and π as the mean predcted probablty n each group (from the model). For the g groups the Ch square statstc s then calculated as: g 2 2 ( S nπ ) χ = n π (1 π ) = 1 The usual nterpretatons apply to ths lack of ft test. Small values of P would ndcate an nadequate model. Devance The devance n logstc regresson can be calculated as n [ ˆ π ˆ π ] DEV ( X, X,..., X ) = 2 Y log ( ) + (1 Y )log (1 ) 0 1 p 1 e e 1 Partal devances can be calculated for reduced models and the dfference n the two models (wth n p and n p+dff d.f.) should follow a Ch square dstrbuton wth dff d.f. Ths test of partal devances s also called the lkelhood rato test. Devance resduals Resdual analyss s not as smple wth Logstc Regresson as t was wth Lnear Regresson. Snce the dependent varable s 0 or 1 they are not normally dstrbuted and n fact the dstrbuton s not known. As a result resdual analyss n Logstc Regresson s done on Devance resduals. These are calculated as: dev =± { 2[ Y log ( ˆ π ) + (1 Y )log (1 ˆ π )]} e e where the sgn s + f Y ˆ ˆ π and the sgn s - f Y < π. Model devance the SS of these resduals wll sum to the model devance.

7 Logstc regresson Dagnostcs Page 7 Pearson resduals Gven the resdual e ˆ = Y π, the predcted probabltes ˆ π = Y e and r = the number of events n a gven observaton wth n trals, then the Pearson resdual s gven as: r ˆ nπ n ˆ π (1 ˆ π ) Confdence nterval dsplacement dagnostcs SAS provdes two measures of confdence nterval dsplacement as nfluence dagnostcs, C for ndvdual observatons and CBAR for overall change n the parameter estmates when ndvdual observatons are removed. 143 proc sort data=next1 nodupkey; by Age status; run; NOTE: There were 98 observatons read from the data set WORK.NEXT1. NOTE: 25 observatons wth duplcate key values were deleted. NOTE: The data set WORK.NEXT1 has 73 observatons and 19 varables. NOTE: PROCEDURE SORT used (Total process tme): 0.01 seconds 0.02 seconds 144 proc prnt data=next1; 145 TITLE2 'Lstng of one kept value for each by group from Logstc Reg'; 146 run; NOTE: There were 73 observatons read from the data set WORK.NEXT1. NOTE: The PROCEDURE PRINT prnted page 5. NOTE: PROCEDURE PRINT used (Total process tme): 0.11 seconds 0.08 seconds optons ps=56 ls=111; 149 proc sort data=dsease; by Age; run; NOTE: There were 98 observatons read from the data set WORK.DISEASE. NOTE: The data set WORK.DISEASE has 98 observatons and 7 varables. NOTE: PROCEDURE SORT used (Total process tme): 0.01 seconds 0.02 seconds 150 proc sort data=next1; by Age; run; NOTE: Input data set s already sorted, no sortng done. NOTE: PROCEDURE SORT used (Total process tme): 0.00 seconds 0.01 seconds 151 proc means data=dsease noprnt; by Age; var Dsease; 152 output out=next3 n=n mean=mean var=var; run; NOTE: There were 98 observatons read from the data set WORK.DISEASE. NOTE: The data set WORK.NEXT3 has 50 observatons and 6 varables. NOTE: PROCEDURE MEANS used (Total process tme): 0.01 seconds 0.02 seconds data three; set next1 next3; run; NOTE: There were 73 observatons read from the data set WORK.NEXT1. NOTE: There were 50 observatons read from the data set WORK.NEXT3. NOTE: The data set WORK.THREE has 123 observatons and 24 varables. NOTE: DATA statement used (Total process tme): 0.02 seconds 0.03 seconds 155 proc plot data=three; plot yhat*age='x' mean*age='o' / overlay; 156 TITLE2 'Plot of observed means (o) and predcted values (p)'; 157 run; ods html close; NOTE: There were 123 observatons read from the data set WORK.THREE. NOTE: The PROCEDURE PLOT prnted page 6. NOTE: PROCEDURE PLOT used (Total process tme): 0.09 seconds 0.03 seconds NOTE: SAS Insttute Inc., SAS Campus Drve, Cary, NC USA NOTE: The SAS System used: 6.59 seconds 0.82 seconds Influence and resdual dagnostcs are plotted below. There are apparently no generally accepted crtera for evaluaton of these statstcs n Logstc regresson. Your text suggests subjectve vsual assessment of an approprate graphc

8 Logstc regresson Dagnostcs Page 8 Logstc regresson on Dsease data The LOGISTIC Procedure - RESCHI * + P e a * * * r * * * * s * * * * o 1 + * * + n * * * * ** R * * * ** e s d u * ** * ** * **** * * * * * ****** ** a ** *** * ** ** * ** ** * * * * l * * * * * * * * * * * * -1 + * * * * + * * * * * RESDEV D 2 + * * + e v ** * * * * * * a * * * * * n * * * c 1 + * ** + e ** * * * * R e s d u * * ** * **** a * * ** * * * * ** * * * * ** l ** *** * ** ** * * ** * * * * * * * * * ** -1 + * * * * + * * * * * * * * * * H H a t D a ** * * * + g * * * * * o * * * * * n * * * a * * * * * ** * l * * * + * * * * * * * * * * * * * * * * * ** * * * * ** + * ** **** * * ** * * * * * * * * ** **** * * ****** **

9 Logstc regresson Dagnostcs Page 9 Logstc regresson on Dsease data The LOGISTIC Procedure DFBETA I n * + t e r c e p t * + D * * * * * f * * * B e t * * * * * * * a * * * * * * * * + * * * ** * ** ** * **** * * *** ****** ** ** * * * * ** * * * * * * * ** * ** *** * * *** ** ** * DFBETA * + ** * * * * * * ** * A * * * ** * * g * * ** * * ** * * * ** * ** ** * ********* * e ** * * * * ** * * * * * * ** * * * + * * * * * * D * * * * ** * f * * * * B * * * * e * * * t * * + a DFBETA s * * * * * t a t u s * * * **** L * * * o ** *** ** ** * w * ** * * * *** * * * e * * ** * * * * * * + r *** * * ** ** * * * **** * * * ****** ** * * * D * * * * f * * * * * * B * * * * e t * + a

10 Logstc regresson Dagnostcs Page 10 Logstc regresson on Dsease data The LOGISTIC Procedure DFBETA ** * s t a t * ** * u * ** ** * s * * + M ** ** d * *** * * * ** * * * * d l ** * * ** * * * **** * * * * ****** **** + e * * ** * * * * * D * * * * f B * + e * * * * t a * * * * * * * * * * * DFBETA * + s e * ** * * * c * * * * * * + t * * * ** * * o ** *** * * * * * ** ** * * * r * * ** ** * * ** * ** * ** **** * * * ****** ** D f B e t * * * a * ** * + ** * * * * * * * * + ** * * C C o n f d e n c e I n t e r v * + a * * * l * * * D s p l * * ** * * * * a c * * * * * * * e * * * * * * * * m * * * * ** * ** * * * e ** * * * ** ** * ** ** * * * n * ** ** * ** * **** * * * * * ****** ** + t ---- C

11 Logstc regresson Dagnostcs Page 11 Logstc regresson on Dsease data The LOGISTIC Procedure --- C CBAR o n f d e n c e I n t e r v a l ** * * * D * * * s p l * * * a * ** * * * * c * * * * * * e * * * * * * * * m * * * * * ** * ** * * e ** * * ** * ** * ** ** * * * n * ** ** * ** * **** ** * * * * ****** ** + t --- C B - DIFDEV D e l t a 3 + * * + D e v a 2 + * * + n * * * * c * * * * e * * * * * * * * * * * 1 + * * + * * * * * * * * * * * * ** * * * * * * * * ** * *** * * * ** * * * * * * * * * * **** * * * * ****** ** DIFCHISQ * + D e l t a C h S q u 4 + * + a r e * * * * 2 + * * + * * * * * * * * * * * * * * * * * * * * * * ** * * ** **** * * ** * ** * **** ***** * ** ** * * * * 0 + * ** * ** * **** * * * * * ****** **

12 Logstc regresson Dagnostcs Page 12 Lstng of one kept value for each by group from Logstc Reg D D F D F B F D B E B F E T E B T A T E A _ A D T _ s _ F A s t s B _ D t a t E I F a t a T n B t u t A S S D _ t E u s u _ t t s s L r e T s M s s d a a e s t E l u e r A L U e c t t c e a V y c c s c _ o d p c f O a A u u t a t E h l l d e A w d p t d b s g s s o s u L a 9 9 e p g e l e o e s e e 1 2 r e s _ t 5 5 v t e r e r r v Lower Lower Mddle Upper Lower Upper Lower Lower Mddle Upper Lower Lower Mddle Upper Mddle Upper Lower Mddle Mddle Upper Lower Upper Upper Lower Upper Mddle Lower Mddle Upper Lower Lower Mddle Mddle Lower Upper

13 Logstc regresson Dagnostcs Page Upper Upper Mddle Lower Mddle Upper Upper Upper Mddle Upper Lower Mddle Upper Lower Mddle Upper Mddle Upper Mddle Mddle Upper Mddle Lower Lower Upper Upper Lower Upper Mddle Lower Upper Lower Upper Lower Upper Mddle Upper Upper

14 Logstc regresson Dagnostcs Page proc plot data=three; plot yhat*age='x' mean*age='o' / overlay; 156 TITLE2 'Plot of observed means (o) and predcted values (p)'; 157 run; Plot of observed means (o) and predcted values (p) Plot of yhat*age. Symbol used s 'x'. Plot of mean*age. Symbol used s 'o'. yhat o o o o o o o oo o oo o o x x x x x x x x x x x o x x x x xx x x o o oo o o x x x x x x xxx x x x x o o x x x x x x x x x x x x o o x x x x x x o x x xx x x x x x x x x xxx x o xxxxxxx ooooo oo o o ooo o o o o o o o oo oo Age NOTE: 123 obs had mssng values. Logstc regresson how far can you take t? Interactons, polynomals, analyss of covarance. Yes! Of course, the logstc s already a curve, so polynomals wll alter the expected shape, ncludng gvng a quadratc shape to the log-odds dependent varable. Dtto for response surfaces; both of these are feasble wthn the context of logstc regresson. Other optons that work much lke PROC GLM or MIXED are optons for a soluton, NOINT and even STB, a standardzed regresson coeffcent. How about selecton crtera (forward, backward, stepwse)? Yes, they are there n PROC LOGISTIC. T here s also a best subset selecton opton, SELECTION=SCORE. Selecton s based on the score ch-square value. Also avalable are the optons usually assocated wth SAS stepwse applcatons, ncludng the INCLUDE, SLENTRY, SLSTAY, START, STOP and BEST optons. Instead of plottng R 2, Adjusted R 2, Mallow s C p or SSE p or other ads to model selecton, the LOGISTIC equvalent would be AIC p, SBC p, 2 log lkelhood and score statstcs.