Empirical Distributions of Parameter Estimates. in Binary Logistic Regression Using Bootstrap

Transcription

1 Int. Journal of Math. Analyss, Vol. 8, 4, no. 5, 7-7 HIKARI Ltd, Emprcal Dstrbutons of Parameter Estmates n Bnary Logstc Regresson Usng Bootstrap Anwar Ftranto* and Ng Me Cng Department of Mathematcs, Faculty of Scence and Insttute for Mathematcal Research Unverst Putra Malaysa, Malaysa *Correspondng author Department of Statstcs, Faculty of Mathematcs and Natural Scences Bogor Agrcultural Unversty, Indonesa Copyrght 4 Anwar Ftranto and Ng Me Cng. Ths s an open access artcle dstrbuted under the Creatve Commons Attrbuton Lcense, whch permts unrestrcted use, dstrbuton, and reproducton n any medum, provded the orgnal work s properly cted. Abstract Bootstrappng s a famous statstcal tool that nvolves resamplng procedure to select sample from a populaton. In ths study, we appled random- x bootstrap n bnary logstc regresson for publshed data set namely Umaru Impact data. We conducted bootstrap for the coeffcent by usng SAS (Statstcal Analyss System. We observe the dstrbuton of the estmated coeffcents wth dfferent sample szes. After conductng B= bootstrap replcatons, we found that the dstrbuton of parameters estmates s nearly normal. Keywords: Bnary logstc, Bootstrap, Parameter estmates Introducton Bootstrappng s a useful statstcal technque used for analyzes and obtan estmate coeffcent n regresson. We can use parametrc bootstrap when the dstrbuton form of the data s known, and we can conduct non parametrc bootstrap when the dstrbuton form of the data s unknown. We wll conduct non parametrc bootstrap for bnary logstc regresson coeffcents on an exstng data set where the dstrbutonal of the data s unknown. The bootstrap performance s

2 7 Anwar Ftranto and Ng Me Cng measured based on the bases and varances of the bootstrap estmates. We also try to nvestgate effect of sample sze on the dstrbuton of the parameter estmates. Logstc regresson s a popular and useful statstcal method n modelng categorcal dependent varable. Many researchers used t to analyze n all felds. Klenbaum and Kelvn, [] proposed that logstc regresson s a mathematcal modelng approach used to nvestgate the relatonshp between the ndependent varable and dchotomous dependent varable. Suppose we have a set of observaton wth the outcome s bnary, outcome s ether or. The correspondng model s wrtten as: Y = π ( x + ε, ( where π ( x = P( Y = The Y exp( β + β =. ( + exp( β + β s the outcome and ε are assumed to be ndependent dentcally dstrbuted wth zero mean and constant varance, σ. Because the ndependent varables x s not lnear n π ( x, therefore we need to transform the π x usng logt functon whch s wrtten as ( π ( x Logtπ ( x = ln ( π ( x exp( β + β + exp( β + β = ln( exp( β + β + exp( β + β = ln[exp( β + β ] = β + β By usng logt functon, t provdes much easer model to ft because t s n lnear form. In logstc regresson, we commonly use maxmum lkelhood to estmate the parameters whch are the best to ft the model. Czepel, [] stated that we are not able to use least square estmaton to produce the mnmum varance unbased estmators n logstc regresson. In bnary logstc regresson, our outcome s nvolvng value or only. Wth regard to bootstrap technques, Efron, [] ntroduced bootstrap approach to estmatng unknown parameters and makng nference about an unknown populaton. He also proposed hs dea about the bootstrap method can be used to estmate the parameters for multple regresson models. Mchael, [7] stated that bootstrap s a method that nvolves resamples procedure from a data. The basc

3 Emprcal dstrbutons of parameter estmates 7 dea of bootstrap s when we have a data wth unknown populaton s dstrbuton; we can replace t wth the known emprcal dstrbuton usng bootstrap. Batmanz et.al [] defned bootstrap as a computer-ntensve statstcal method where t treat data as a populaton and select samples from the populaton wth replacement. In ther research, they appled emprcal dstrbutons of the parameters to nvestgate the sgnfcant of the parameters n nonparametrc regresson and Conc Multvarate Adaptve Regresson Spnes (CMARS. They nvolve three dfferent bootstrappngs n ther research whch are random- x, fxed- x and Wld bootstrap n four data sets wth dfferent sample sze and scale. The results of ther study show that random- x method gves more precse and less complex model wth the medum sze and medum scale data. Sometmes, random- x bootstrap s also called as pars bootstrap or observaton bootstrap whch s frstly proposed by [4]. Let say we have a set of ' observaton Z = [ Y, xk,..., xnk ] where =,,,n and k s the number of varables. Then, we do the followng steps: ' ' ' Step : Select n observaton Z, Z,..., Z n as sample, Step : Ft the selected sample nto the model and calculate the estmates of parameters of the logstc regresson, Step : Repeated the above procedure B tmes to obtan the bootstrap estmates of parameters. Data and Method Data In ths artcle, we are usng the Umaru Impact data. Ths data s taken from [5]. The data consst of seven ndependent varables whch are age at enrollment, beck depresson score at admsson, IV drug use hstory at admsson, number of pror drug treatment, subject s race, treatment randomzaton assgnment, and treatment ste. The dependent varable s remaned drug free for months, whch s bnary wth means the partcpant remaned druggng free for months and for otherwse. Method We use PROC LOGISTIC n SAS to analyze ths data n logstc model. to obtan coeffcents of the logstc regresson, β where =,, K, 7 for the orgnal dataset. In bootstrappng stage, random- x bootstrap wll be employed. Frst, we bootstrap a sample sze n wth replacement by usng PROC SURVEYSELECT n SAS. Then we ft t nto logstc regresson model and obtan the each estmate of coeffcent, βˆ. We replcate the bootstrap B tmes. Then we compute the mean of each parameter estmates whch s denoted as

4 74 Anwar Ftranto and Ng Me Cng βˆ. Then, we observe the dstrbuton of the estmated coeffcent. We plot hstograms of the estmated coeffcents for each sample sze wth same scale, so that we can compare the locaton of the hstogram. In ths paper, bootstrap replcatons, B, s. The smulaton wll be conducted at four dfferent sample szes whch are 5, 5, 5 and 575. Sample szes 5 s referred as small sample, sample sze 5 s referred as medum sample, sample szes 5 s referred as large sample and sample sze 575 s exactly the number of the observaton n the orgnal data. Therefore, we can justfy the performance of bootstrap usng dfferent sample sze. Results and Dscusson Table : Maxmum Lkelhood Estmates for Orgnal Umaru Impact Data Parameter df Estmate Standard Error Wald Ch-Square p value Odd Rato Estmate 95% Wald Confdence Lmts Intercept x x x x x x x Table s the results that we have obtaned for the logstc regresson analyss of the data. From the result, we obtaned all the true value of parameters from our orgnal data regardless sgnfcance of the coeffcents. By applyng all coeffcents to the logstc regresson model, we obtan the followng predcted full model:.49x.x7 ˆ( π x =. + exp( x.4x +.7x +.8x4.9x5.49x.x 7 And the correspondng logt functon can be wrtten as follows: Logt π ( x = x.49x.x.4x x +.8x 4.9x 5 Varable x and x 4 have postve coeffcents whch mply that P( Y =

5 Emprcal dstrbutons of parameter estmates 75 ncreases as x and x4 ncrease. Meanwhle, varables x, x, x 5, x and x7 are havng negatve coeffcents to ndcate that P ( Y = wll ncrease f those varables at lower values and vce versa. After we obtaned the logstc regresson coeffcent, β where =,,,,7. We start our bootstrap and obtan the estmated coeffcent βˆ. Ths secton onwards present results of bootstrappng the logstc regresson coeffcents. We arbtrarly choose one of 8 parameters out of eght parameter estmates, namely ˆβ snce the other parameters wll have approxmately the same behavor. 45 n=5 8 n=5 4 5 Frequency n=5 n= Fgure : Hstogram of ˆβ when B= wth Dfferent Sample Sze We look for patterns of the dstrbuton of the estmated coeffcent when we ncrease the sample sze. We also concern about the pattern of the dstrbuton when B s gettng larger. Frstly, we look at the dstrbuton of ˆβ. The dstrbuton of ˆβ on wth dfferent sample szes s shown n the Fgure. We notced that when the sample sze s small and medum, n=5 and n=5, there s no clear pattern for the dstrbuton of the ˆβ. But when the sample sze 5 and 575, obvously we can observe that the dstrbuton of ˆβ s a bell shape and symmetrc. It s approxmate to the normal dstrbuton. For ˆβ coeffcent, we may conclude that as the sample ncreases, the dstrbuton of ˆβ s approxmate normal.

6 7 Anwar Ftranto and Ng Me Cng 4 Concluson In ths study, we used random- x bootstrap to estmate the coeffcent usng dfferent sample szes. After estmated the logstc regresson coeffcents, we nvestgated the dstrbuton of the estmated coeffcent. We found that when the sample sze becomes larger, the dstrbuton of the estmated coeffcent s approxmate to normal dstrbuton. References [] I. Batmanz, Yazıcı, C., Yerlkaya-Özkurt, F. Bootstrappng Conc Multvarate Adaptve Regresson Splnes (Bcmars. Mddle East Techncal Unversty: Turkey,. [] S. A. Czepel, Maxmum Lkelhood Estmaton of Logstc Regresson Models: Theory and Implementaton,, (accessed June. [] B. Efron, Bootstrap Methods: Another Look at the Jacknfe. The Annals of Statstcs. 7 (979, -. [4] D. A. Freedman, Bootstrappng regresson models. Annals of Statstcs. 9 (98, 8 8. [5] D. W. Hosmer and Lemeshow, S., Appled Logstc Regresson. Wley, NewYork,. [] D. G. Klenbaum, and Klen, M. Logstc Regresson: Statstcs for Bology and Health: Sprnger Scence Busness Meda,. [7] R. C. Mchael, Bootstrap Method, A Gude for Practtoners and Researchers: John Wley & Sons, Canada, 8. Receved: March, 4