Outlier Detection based on Robust Parameter Estimates

Transcription

1 Outler Detecton based on Robust Parameter Estmates Nor Azlda Aleng 1, Ny Ny Nang, Norzan Mohamed 3 and Kasyp Mokhtar 4 1,3 School of Informatcs and Appled Mathematcs, Unverst Malaysa Terengganu, 1030 Kuala Terengganu, Terengganu, Malaysa. Insttute for Communty (Health) Development (-CODE), Unverst Sultan Zanal Abdn, Gong Badak Campus, 1300 Kuala Terengganu, Malaysa. 4 School of Martme Busness and Management, Unverst Malaysa Terengganu, 1030 Kuala Terengganu, Terengganu, Malaysa. Orcd; Abstract Outlers can nfluence the analyss of data n varous dfferent ways. The outlers can lead to model msspecfcaton, ncorrect analyss results and can make all estmaton procedures meanngless. In regresson analyss, ordnary least square estmaton s most frequently used for estmaton of the parameters n the model. Unfortunately, ths estmator s senstve to outlers. Thus, n ths paper we proposed some statstcs for detecton of outlers based on robust estmaton, namely least trmmed squares (LTS). A smulaton study was performed to prove that the alternatve approach gves a better results than OLS estmaton to dentfy outlers. Keywords: Outlers, least trmmed squares (LTS) and robust regresson. INTRODUCTION The lnear regresson can be expressed n terms of matrces as y X ; where y s an N-dmensonal column vector, X s an N (K + 1) matrx and s an N- dmensonal column vector of error terms,.e. y1 y y 3 yn N 1 1 x 11 1 x 1 1 x x N1 N ( K 1) x1 K x K x 3K x NK 0 1 K ( K 1) 1 1 N N 1 In fttng multple lnear regesson model (1.1), the most wdely used ordnary least squares (OLS) to fnd the best estmates of. Unfortunately, n the presence of outlers, the OLS estmators are stll based. Outlers play an mportant role n regresson. An outlers (observatons) that s qute dfferent from most the other values or observatons n a data set. Observatons n a data set can be outlers n several dfferent ways. Accordng to Barnett and Lews (1994), an outler s an observaton that s nconsstent wth the rest of the data. Even one outler can effect the regresson model. There s an (1) evdence that the outlers can lead to model msspecfcaton, ncorrect analyss result and can make all estmaton procedures meanngless, (Rousseeuw and Leroy, 1987; Alma, 011; Zmmerman, 1994, 1995, 1998). Outlers n the response varable represent model falure. Outlers n the regressor varable values are extreme n X-space are called leverage ponts. Rousseeuw and Lorey (1987) defned that outlers n three types; 1) vertcal outlers, ) bad leverage pont, 3) good leverage pont. Vertcal outler s an observaton that has nfluence on the error term of y-dmenson (response) but s not nfluental n the space of x-dmenson (regressor). Good leverage pont s an observaton that are outlyng n the ndependent varables but s not a regresson outler. Ths pont does not affect the least square estmaton but t affects statstcal nference snce ths pont cut down the estmated standard errors. Good leverage ponts mprove the precson of the regresson coeffcents. Bad laverage pont s an observaton that s outlyng n ndependent varables and located far from the true regresson lne and reduce the precson of regresson coeffcents. Generally speakng, there are two technques for handlng the outlers. The frst s to use the some robust procedure whch resst ther nfluence n the statstcal analyss. The second s to remove outlers from the data set. Therefore, n ths study t s proposed some knd of robust procedure namely the least trmmed squares (LTS). Ths method s relable for dentfyng the regresson outlers both n smple and multvarates stuatons. The objectve of ths study was to detect the outlers and leverage ponts n the data set. The performance of the proposed method was dscussed extensvely by usng medcal data. MATERIALS AND METHODS Ths study focused on the blood pressure data. Systolc blood pressure (sbp) s a dependent varable and ndependent varables namely n Table 1. A total of 100 repondents were selected and dagnosed to have blood pressure problem based on WHO crtera. The explanaton 1349

2 of the varables s shown n Table 1 and the data were collected from Health Centre HUSM n Malaysa. Table 1: Explanaton of the Varables Code Varables Explanaton of the varables y SBP Systolc blood pressure x 1 AGE Age (year) x BMI Body mass ndex x TOTCHOLES Total cholesterol 3 (Mmol/L) x DIABETES Dabetes melltus; 0 = No, 4 1 = Yes x DBP Dastolc blood pressure 5 x HDL HDL cholesterol 6 x HEIGHT Heght (m) 7 x TRIG Trglycerdes 8 x WEIGHT Weght (kg) 9 As s well known, a large number of dagnostcs have been proposed to detect outlers. Practcally, the dagnostcs whch was based on the ordnary least squares estmates were not effcent and based when outlers exsted n the data. Thus, to remedy ths problem, least trmmed squres estmamtors (LTS) was proposed. Ths was an alternatve approach n dealng wth outlers n regresson analyss. The Least Trmmed Squares Estmators (LTS) A statstcal procedure s regarded as robust f t performs reasonably well even when the assumptons of the statstcal model are not true. If we assume our data follows standard lnear regresson, least squares estmates and test perform qute well, but they are not robust wth the presence of outler observaton(s) n the data set (Rousseeuw and Leroy, 1987). In ths case we proposed the popular robust technque s the called LTS estmator. LTS estmaton s a hgh breakdown pont. The breakdown pont s a measure for stablty of the estmator when the sample contans a large fracton of outlers (Hampel, 1975). LTS defned as: ( LTS) h r 1 argmn ( ) () where r(1)... r( n) are ordered squared resduals. Robust regresson s extremely useful n dentfyng outlers. LTS regresson s a relable data analytcal tool that may be used to dscover regresson outlers both n smple and multvarable condtons. In ths paper, three dagnosts are used to dentfy outlers whch are gven below. The robust dstance s defned as: T 1 RD( x ) [ x T( X )] C( X ) [ X T( X )] (3) where T( X ) and CX ( ) are the robust locatonand scatter matrx for thr multvarable. The Mahalanobs dstances s useful technque for detectng outlers s defned as 1 ( ). ( ) T MD x C x (4) Where C s the classcal sample covarance matrx. In classcal lnear regresson,the dagonal element of the hat matrx, T 1 h H X ( X X ) X T are used to dentfy leverage ponts. Rousseeuw and Van Zomeren (1990) defned the relatonshp between the h and MD h [(( MD ) / ( n 1)] [1/ n]. Rousseeuw and Lorey (1987) suggest usng h p / n and MD benchmarks for leverage and Mahalanobs dstances. The Cook s dstance s defned as, where p1;0.95 as 1 ˆ ˆ T CD ( p ) ( Y Y) ( Yˆ Yˆ ) (5) s estmator of the error varance, n r / n p. Cook suggests that CD be compared to a 1 central F dstrbuton wth p and n p degrees of freedom. Ths gves the cutoff values s very hgh. The conventonal cutoff value s 4 / n p. Generally, when the statstcs CD, h and MD are large, case may be an outler or nfluental case. Therefore the dagnostc s very mportant to dentfy the outlers and provdes resstant results n the presence of outlers. Below s the algorthm n SAS language for the multple lnear regresson and robust regresson. Ths algorthm usng SAS software whch s gven as follows: data BP; nput sbp age bm totcholes dabetes dbp hdl heght trg weght; datalnes; 13430

3

4 ; ods rtf fle='robdunc0.rtf' style=journal; ods graphcs on; /* frst we do multple lnear regresson */ proc reg data=bp; model sbp = age bm totcholes dabetes dbp hdl heght trg weght; /* then we do robust regresson, n ths case, LTS-estmaton */ proc robustreg data=bp method=lts; model sbp = age bm totcholes dabetes dbp hdl heght trg weght ; ods graphcs off; /* QQ plot and hstogram */ ods graphcs on; proc robustreg data=bp plots=(rdplot ddplot reshstogram resqqplot); model sbp = age bm totcholes dabetes dbp hdl heght trg weght; ods graphcs off; /* Cook s dstance plot*/ proc reg data=bp; plot (only label)=(rstudentbyleverage CooksD); ods graphcs off; /* then we do robust weghted regresson */ ods graphcs on; proc robustreg method=lts(h=33)fwls data=bp plots=all; model sbp = age bm totcholes dabetes dbp hdl heght trg weght /dagnostcs leverage; output out=robout r=resd sr=stdres; ods rtf close; RESULTS AND DISCUSSIONS In ths study, a set of real data whch s referred to blood pressure data s used to see how well the dagnostc statstcs wth robust estmator perform for the regresson model. The step s the blood pressure data was analyzed usng robust regresson method LTS wth dagnostcs tool such as Mahalanobs dstance, robust MCD dstance and standardzed robust resduals. Results of dagnostcs of outlers and leverage ponts are presented n Table. 1343

5 Obs Table : Robust dagnostcs based on least trmmed squares (LTS) Mahalanobs Dstance Robust MCD Dstance Leverage Standardzed Robust Resdual * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * Outler The model of blood pressure s: yˆ age dbp e heght weght (6) predct a trend. The results of Table show that the exstng of outlers n the blood pressure data. 13 observatons are consdered as outlers and observaton 93 has hgh leverage. Although, the set data presence of outlers, results reman robust. and R squared valus s (85.3%), ths ndcated the greater the ablty of that model to 13433

6 Outler and Leverage Dagnostcs for sbp Standardzed Robust Resdual Observatons Outlers Leverage Pts Res Cutoff Lev Cutoff Robust MCD Dstance Outler Leverage Outler and Leverage Fgure 1: Regresson dagnostc plot for systolc blood pressure. Based on Fgure 1, gves evdence of the presence of outlyng observatons because the pots fall behnd the band. Observatons (1, 0, 1, 3, 6, 43, 44, 49, 55, 57, 96, 97 and 100) are dentfed as outlers. We can see that, 0 observatons are dentfed as laverage ponts and observatons 1, 6 and 96 dentfed as outlers and leverage ponts at the same tme. CONCLUSION Least trmmed squres estmamtors (LTS) s an alternatve approach n dealng wth outlers n regresson analyss. The value R gave strong correlaton and relaton between varables, so t shown that strong good ft model. Robust verson of the dagnostcs detect all outlers n the data n one step. The results of the smulaton study agree well wth the real data. REFERENCES [1] Alma, O. G. (011). Comparson of robust regresson methods n lnear regresson. Int. [] Journal Contemp. Math. Scences, 6(9), [3] Barnett, V., & Lews, T. (1994). Outlers n statstcal data. New York: John Wley and Sons. [4] Rousseeuw, P. J. and Leory, A. (1987). Robust Regresson and Outler Detecton. Wley Seres n [5] Probablty and Statstcs. [6] Zmmerman, D. W. (1994). A note on the nfluence of outlers on parametrc and nonparametrc [7] tests. Journal of General Psychology. 11(4): [8] Zmmerman, D. W. (1995). Increasng the power of nonparametrc tests by detectng and [9] downweghtng outlers. Journal of Expermental Educaton, 64(1), [10] Zmmerman, D. W. (1998). Invaldaton of parametrc and nonparametrc statstcal tests by [11] concurrent volaton of two assumptons. Journal of Expermental Educaton. 67(1):