Stat 401 B Lecture 26

Transcription

1 Stat B Lecture 6 Forward Selection The Forward selection rocedure looks to add variables to the model. Once added, those variables stay in the model even if they become insignificant at a later ste. Backward Selection The Backward selection rocedure looks to remove variables from the model. Once removed, those variables cannot reenter the model even if they would add significantly at a later ste. Mixed Selection A combination of the Forward and Backward selection rocedures. Starts out like Forward selection but looks to see if an added variable can be removed at a later ste.

2 Stat B Lecture 6 Mixed Set u Resonse: MDBH Stewise Regression Control Prob to Enter Prob to Leave Direction: Mixed SSE DFE MSE Adj AIC Lock Parameter Estimate ndf SS "F Ratio" "Prob>F" X X Ste History Ste Parameter Action "Sig Prob" Seq SS Stewise Regression Control Direction Mixed Prob to Enter controls what variables are added. Prob to Leave controls what variables are removed. Prob to Enter = Prob to Leave 5 The current estimates are exactly the same as with the Forward selection rocedure. Clicking on Ste will initiate the Mixed rocedure that starts like the Forward rocedure. 6

3 Stat B Lecture 6 Resonse: MDBH Stewise Regression Control Prob to Enter Prob to Leave Direction: Mixed SSE.59 DFE MSE Lock Parameter X X Ste History.76 Estimate ndf Adj.69 SS "F Ratio" AIC -.6 "Prob>F" Ste Parameter Action "Sig Prob" Seq SS Ste X is added to the model Predicted MDBH = *X R =.76 RMSE = MSE =.6958 =. 7 8 Ste By clicking on Ste you will invoke the Backward art of the Mixed rocedure. Because X is statistically significant and is the only variable in the model, clicking on Ste will not do anything. 9

4 Stat B Lecture 6 Ste Of the remaining variables not in the model X will add the largest sum of squares if added to the model. SS =. F Ratio = 8.9 Prob>F =. JMP Mixed Ste Because X will add the largest sum of squares and that addition is statistically significant, by clicking on Ste, JMP will add X to the model with X. Resonse: MDBH Stewise Regression Control Prob to Enter Prob to Leave Direction: Mixed SSE.5859 DFE MSE 7.59 Lock Parameter X X Ste History Ste Parameter X.86 Estimate Action ndf "Sig Prob".. Adj.779 SS Seq SS "F Ratio" AIC "Prob>F"

5 Stat B Lecture 6 Ste X is added to the model Predicted MDBH =. +.*X +.95*X R =.86 RMSE = MSE =.59 =. 7 Ste By clicking on Ste you will invoke the Backward art of the Mixed rocedure. Because X and X are statistically significant, clicking on Ste will not do anything. Ste Of the remaining variables not in the model X will add the largest sum of squares if added to the model. SS =.67 F Ratio = 7.78 Prob>F =. 5

6 Stat B Lecture 6 JMP Mixed Ste Because X will add the largest sum of squares and that addition is statistically significant, by clicking on Ste, JMP will add X to the model with X and X. 6 Resonse: MDBH Stewise Regression Control Prob to Enter Prob to Leave Direction: Mixed SSE.799 DFE MSE Lock Parameter X X Ste History Ste Parameter X X.867 Estimate Action ndf "Sig Prob"... Adj.8 SS Seq SS "F Ratio" AIC "Prob>F" Ste X is added to the model Predicted MDBH = *X.69*X +.67*X R =.867 RMSE = MSE =.8699 =. 96 8

7 Stat B Lecture 6 Ste By clicking on Ste you will invoke the Backward art of the Mixed rocedure. Note that variable X is no longer statistically significant and so it will be removed from the model when you click on Ste. 9 Resonse: MDBH Stewise Regression Control Prob to Enter Prob to Leave Direction: Mixed SSE.989 DFE MSE Lock Parameter X X Ste History Ste Parameter X X.8658 Estimate Action Removed ndf "Sig Prob" Adj.85 SS Seq SS "F Ratio" AIC "Prob>F" Ste X is removed from the model Predicted MDBH = *X.898*X R =.8658 RMSE = MSE =.8997 =. 86

8 Stat B Lecture 6 Ste Because X and X add significantly to the model they cannot be removed. Because X will not add significantly to the model it cannot be added. The Mixed rocedure stos. Resonse MDBH Summary of Fit Adj Root Mean Square Error Mean of Resonse Observations (or Sum Wgts) Analysis of Variance Source Model Error C. Total DF 7 9 Sum of Squares Parameter Estimates Term X X Effect Tests Source X X Estimate Narm DF Std Error e-5 Mean Square Sum of Squares t Ratio F Ratio 5.8 Prob > F <.* Prob> t <.* <.* <.* F Ratio Prob > F <.* <.* Finding the best model For this examle, the Forward selection rocedure did not find the best model. The Backward and Mixed selection rocedures came u with the best model.

9 Stat B Lecture 6 Finding the best model None of the automatic selection rocedures are guaranteed to find the best model. The only way to be sure, is to look at all ossible models. 5 For k exlanatory variables there k are ossible models. There are k -variable models. k There are -variable models. There are k -variable models. 6 When confronted with all ossible models, we often rely on summary statistics to describe features of the models. R adjr RMSE 7

10 Stat B Lecture 6 Another summary statistic used to assess the fit of a model is Mallows C. SSE C = MSE = k + Full ( n ) 8 The smaller C is the better the fit of the model. The full model will have C =. 9 JMP Fit Model Personality Stewise Red triangle ull down All Possible Models Right click on table Columns Check C

11 Stat B Lecture 6 Resonse: MDBH Model Number X,X,.867 X,X.8658 X,.86 X, X.5977 X RMSE RMSE.5... X,X X,X,. 5 6 = Number of Terms Lists all 7 models. -variable (full) model first. -variable models listed in order of the R value. -variable models listed in order of the R value. Model with X, X, X Highest R value. Model with X, X Lowest RMSE and lowest C. Which is best? Can t tell until you look at significance of the variables in the model.