A New Graphical Multivariate Outlier Detection Technique Using Singular Value Decomposition

Transcription

1 Intenational Jounal of Engineeing Reseach & echnology (IJER) ISSN: Vol. Issue 6, August - 0 A New Gaphical Multivaiate Outlie Detection echnique Using Singula Value Decomposition Nishith Kuma, Mohammed Nasse Depatment of Statistics, Begum Roeya Univesity, Rangpu, Bangladesh; Depatment of Statistics, Rajshahi Univesity, Rajshahi, Bangladesh. Abstact It is well ecognized that even a high quality data set tends to contain a emaable pecent of unusual obsevations. hee ae vaious techniques to detect multivaiate outlies. But they ae highly depends on mathematics. We popose a new gaphical technique to detect multivaiate outlies which is easy to undestand without had mathematics, it can be applied to data fo both supevised and unsupevised leaning, it can be diectly applied to sepaate exteme outlies fom geneal outlies. Keywods: Outlies, Singula Value Decomposition, Pincipal Component Analysis.. Intoduction Outlies detection poblem is as old as statistics. Outlies pesent in both supevised and unsupevised leaning of multivaiate data set. Outlie can destoy ou analysis. So outlies detection should be the fist taget of a statistician o a eseache. In this pape we mainly poposed a method fo outlies detection in a multivaiate data set and apply this in seveal well nown data sets. In multivaiate statistics we detect outlies by Mahalalanobis Distance but it is not suitable fo outlie detection since it is dependent on nonobust mean and covaiance matix. o detect outlies, Rousseeuw and Leoy (987) poposed obust distances, which ae obustified vesions of the Mahalanobis distances RD i ( x i t n ) C n ( x with t, c ) obust estimates of location and scatte. ( n n Obsevations with RDi bigge than the citical value,0.975 can be consideed as potential outlies (Rousseeuw and Van Zomeen, 990). Regession analysis is one of the techniques of multivaiate statistics. By egession analysis we can analyze business, economics and social science data. hee ae some existing techniques to detect outlies by using Regession analysis. In egession analysis statisticians mainly follow two ways. (i) Afte fitting classical least squae lines they detect outlies in Y diection by standadized esiduals, studentized esiduals (Siantan; 96), deletion studentized esidual (Ellenbeg; 976) and outlies in X diection by high leveage values. (ii) he obust techniques that ae commonly used in the identification of multiple outlies ae least median of squaes (LMS) (Rousseeuw; 984), least timmed squaes (LS)(Rousseeuw; 984) and eweighted least squaes (RLS)(Rousseeuw and Leoy; 987) etc. In logistic egession outlies ae detected by genealized standadized peason esidual (GSPR) (Hadi and Simonoff, 993; Atinson, 994; Munie, 999 and Imon, 005).. Singula Value Decomposition he singula value decomposition (SVD) can be viewed as the extension of the eigenvalue i t n )

2 Intenational Jounal of Engineeing Reseach & echnology (IJER) ISSN: Vol. Issue 6, August - 0 decomposition fo the case of nonsquae matices. It shows that any eal matix can be diagonalized by using two othogonal matices. he eigen value decomposition, instead, wos only on squae matices and uses only one matix ( and its invese) to achieve diagonalization. If the matix is squae and symmetic, then the two othogonal matices of SVD become equal, and eigen value decomposition and SVD become one and same thing. Because the SVD is much moe geneal than the eigen value decomposition and intimately elated to the matix an and educed an least squae appoximations, it is a vey impotant and useful tool in matix theoy, statistics and signal analysis. It can be used as a data eduction technique. Singula value decomposition, specially its low an appoximation popety is an elegant pat of moden matix theoy. Afte its inception (936)[0] its two ways fascinating data eduction capacity emained unnoticed till the last quate of last centuy. Since then statisticians have been showing inceasing inteest to SVD fo pincipal component analysis (PCA), canonical coelation analysis (CCA) and cluste analysis. Pincipal component analysis (PCA), often pefomed by singula value decomposition (SVD), is a popula analysis method that has ecently been exploed as a method fo analyzing lage-scale expession data (Raychaudhui et al., 000; Alte et al., 000)[,]. Additionally SVD/PCA has been used to identify high-amplitude modes of fluctuations in macomolecula dynamics simulations (Gacia, 99; Romo et al., 995)[3,4], and identify stuctual intemediates in lysozyme folding using small-angle scatteing expeiments (Chen et al., 996)[5]. One of the challenges of bioinfomatics is to develop effective ways to analyze global gene expession data. A igoous appoach to gene expession analysis must involve an up-font chaacteization of the stuctue of the data. In addition to a boade utility in analysis methods, singula value decomposition (SVD) and pincipal component analysis (PCA) can be valuable tools in obtaining such a chaacteization. SVD and PCA ae common techniques fo analysis of multivaiate data, and gene expession data ae well suited to analysis using SVD/PCA. A single micoaay expeiment can geneate measuements fo thousands, o even tens of thousands of genes. Gene expession data ae cuently athe noisy, and SVD can detect and extact small signals fom noisy data. Since SVD can educe data in both ways columns (geneally indicates vaiables) and ows (geneally indicates cases), and is moe numeically stable, and moeove, PCA can be undetaen as a by poduct of SVD, in moden eseach it is being used moe fequently in place of classical PCA fo data compession ( Diamantaas and Kung,996;)[6], clusteing (Mutagh,00;)[7] and multivaiate outlies detection (Penny and Jolliffe,00;)[8]. 3. Low Ran Appoximation of SVD Low an appoximation (C. Ecat and G. Young, 936)[0] is an impotant popeties of SVD. It has an wondeful data eduction capacity with minimum ecovey eo. We can educe vaiables as well as obsevations by using SVD. If X is m n matix of an min(m,n). hen by singula value decomposition we can wite, X U V () whee U is the column othonomal matix whose columns ae the eigen vectos of XX, is the diagonal matix contain the singula values of X and V is the othogonal matix whose columns ae the eigen vectos of X X. Fom () we can wite uv X u v u v Suppose we appoximate X by X ~ whose an is < min(m,n). ~ X u v u v X ~ whee ode and U V U is m, u v is a diagonal matix of V is n. Now post multiply..

3 Intenational Jounal of Engineeing Reseach & echnology (IJER) ISSN: Vol. Issue 6, August - 0 V in both side we have, X ~ V U its fist column epesents the fist PC, second column epesents the second PC and so on. Hence we see that X is a m n matix but X ~ V is a m. Geneally n epesents no. of vaiables, so it educes data by minimizing no. of vaiables. outside the box. So in ou above gaph we can say that 6, 5, 30, 3, 34, 36, 0, 04 - ae unusual obsevations. Hubet, Rousseeuw and Banden(005)[9] declaed obsevations 5, 30, 3, 34, 36, 0- as outlies by using ROBPCA. 4. SVD Based Outlie Detection Method We develop a gaphical method of outlies detection using SVD. It is suitable fo both geneal multivaiate data and egession data. Fo this we constuct the scatte plots of fist two PC s, and fist PC and thid PC. We also mae a box in the scatte plot whose ange lies median( st PC) ± 3 mad( st PC) in the X-axis and median( nd PC/3 d PC) ± 3 mad( nd PC/3 d PC) in the Y-axis. Whee mad = median absolute deviation. he points that ae outside the box can be consideed as exteme outlies. he points outside one side of the box is temed as outlies. Along with the box we may constuct anothe smalle box bounded by.5/ MAD line. 5. Example In this section we will use ou method in some eal data sets. hese datasets ae well nown in multivaiate analysis and egession analysis. 5. Ca Data Ou fist example is the low-dimensional ca data set which is available in S-PLUS as the data fame cu.dimensions. Fo n= cas, p= chaacteistics wee measued such as the length, the width and the height of the ca. Using ou method we get the Figue-. Fom Figue-(a) we see that obsevations 5, 30, 3, 34, 36, 0, 04, 07, 08, 0 and ae outside the box by the two goups. Also fom Figue-(b) we see that the obsevations 6, 0 and 04- ae Figue. Scatte plot of ca data (a) scatte plot of fist two PC s and (b) scatte plot of fist and thid PC. 5.. Hawins-Badu-Kass (984) Data Hawins, Badu and ass (Hawins et al., 984)[0] constucted an atificial thee-pedicto data set containing 75 obsevations with 4 influential obsevations. Among them thee ae ten high leveage outlies (cases -0) and fo high leveage points (cases -4) -Imon (005)[]. If we apply ou method in this data then we get the Figue-. Fom Figue- we see that obsevations -4 ae outside ou box so obsevations -4 ae unusual obsevations. Also we see that thee clustes ae pesent in the data. Obsevations -0 mae st cluste, obsevations -4 mae second cluste and the est obsevations mae thid cluste in figue-(a). Index plot of standadized esiduals obtained fom LMS (Rousseeuw and Leoy, 987)[5] is out pefomed by ou plot. 3

4 Intenational Jounal of Engineeing Reseach & echnology (IJER) ISSN: Vol. Issue 6, August - 0 Figue. Scatte plot of Hawins, Badu and ass data (a) scatte plot of fist two PC s and (b) scatte plot of fist and thid PC Modified Bown Data We fist conside the data set given by Bown (980)[]. Hee the main objective was to see whethe an elevated level of acid phosphates (A.P.) in the blood seum togethe would be of value fo pedicting whethe o not postate cance patients also had lymph node involvement (L.N.I). Ryan (997)[3] pointed out that the oiginal data on the 53 patients which contains outlie (obsevation numbe 4). Imon and Hadi(005)[9] modified this data set by putting two moe outlies as cases 54 and 55. Also they showed that obsevations 4, 54 and 55 ae outlies by using genealized standadized Peason esidual (GSPR) (Hadi and Simonoff,993; Atinson, 994; Munie, 999; Imon, 005) [6,7,8,9]. Now we apply ou method in this data. Applying ou method we get the figue 3. Fom figue-3 we see that obsevations 4, 54, 55, 53 and 5 ae detected as outlies by ou method. Figue-3. Scatte plot of modified Bown data (a) scatte plot of fist two PC s and (b) scatte plot of fist and thid PC. 6. Advantages Ou Method Ou method has the following advantages ove othe competitive methods; It is easy to undestand without had mathematics. It can be applied fo both egession data and othe type of multivaiate data. It is diectly applied to sepaate exteme outlies fom geneal outlies. It can detect seveal clustes that othe outlies detection methods fail to pinpoint. It can single out not only outlying obsevations but also outlying vaiables. 7. Conclusion Fom ou above discussion we conclude that the poposed method is vey much helpful fo multivaiate outlie detection. By using this method we can also see the stuctue of multivaiate data gaphically. hough thee ae seveal existing methods fo detecting 4

5 Intenational Jounal of Engineeing Reseach & echnology (IJER) ISSN: Vol. Issue 6, August - 0 multivaiate outlies but SVD based technique is bette than those methods. 8. Refeence [] Rousseeuw P.J. and Leoy A. (987). Robust Regession and Outlie Detection. New Yo: Wiley. doi:0.00/ [] Rousseeuw P.J., Van Zomeen B.C. (990). Unmasing multivaiate outlies and leveage points. Jounal of the Ameican Statistical Association. Vol. 85(4), pp [3] Siantan K. S.(96). esting fo the single outlie in a egession model. Shanhya, vol-3, Seies A, [4] Ellenbeg J.H.(976). esting fo single outlie fom a geneal egession. Biometics, 3, doi:0.307/5975 [5] Rousseeuw P.J. (984). Least median of squaes egession. Jounal of the Ameican Statistical Association, 79, [6] Hadi A.S. and Simonoff J.S. (993). Pocedue fo the identification of multiple outlies in linea models. Jounal of the Ameican Statistical Association, 88, doi: 0.080/ [7] Atinson A.C. (994). Fast vey obust methods fo the detection of multiple outlies. Jounal of the Ameican Statistical Association 89; doi:0.080/ [8] Munie S. (999). Multiple outlie detection in logistic egession. Student 3, 7-6. [9] Imon A.H.M.R. and Hadi Ali S. (005). Identification of Multiple Outlies in Logistic Regession. Intenational Statistics Confeence on Statistics in the ecnological Age, Institute of Mathematical Sciences, Univesity of Malaya, Kualalumpu, Malaysia, Decembe 005. doi:0.080/ [0] Ecat C. and Young G. (936). he appoximation of one Matix by anothe of lowe Ran. Psychometia,,-8. doi: 0.007/BF [] Raychaudhui S., Stuat J.M. and Altman R.B. (000). Pincipal components analysis to summaize micoaay expeiments: application to spoulation time seies. Pac Symp Biocomput 000: [] Alte O., Bown P.O. and Botstein D. (000). Singula value decomposition fo genome- wide expession data pocessing and modeling. Poc Natl Acad Sci USA 000; 97: doi:0.073/pnas [3] Gacia A.E. (99). Lage-Amplitude Nonlinea Motions in Poteins. Phys Rev Lett 99; 68: doi: 0.03/PhysRevLett [4] Romo.D., Claage J.B., Soensen D.C. and Phillips G.N. J., (995). Automatic identification of discete substates in poteins: singula value decomposition analysis of time-aveaged cystallogaphic efinements. Poteins 995; :3-. doi:0.00/pot [5] Chen L., Hodgson K.O. and Doniach S. (996). A lysozyme folding intemediate evealed by solution X-ay scatteing. J Mol Biol 996; 6: doi:0.006/jmbi [6] Diamantaas K.I. and Kung S.Y. (996). Pincipal Components Neual Netwas: heoy And Applications, John wiley & sons, Inc. N.Y [7] Mutagh F.(00). Clusteing in High-dimensional Data Spaces. Classification, Clusteing and Data Analysis, eds. Jajuga,K., Soolowsi,A. and Boc,H., Spinge-Velag, Belin, pp [8] Penny K. I. and Jolliffe I.. (00). A compaison of multivaiate outlie detection methods fo clinical laboatoy safety data. Royal Statistical Society, pat 3, PP doi:0./ [9] Hubet M., Rousseeuw P.J. and Banden K.V. (005). ROBPCA: a New Appoach to Robust Pincipal Components Analysis. echnometics, 47, doi:0.98/

6 Intenational Jounal of Engineeing Reseach & echnology (IJER) ISSN: Vol. Issue 6, August - 0 [0] Hawins D. M., Badu D. and Kass G.V.(984),Location of seveal outlies in multiple egession data using elemental sets. echnometics, 0, doi:0.080/ [] Imon A. H. M. R. (005). Identifying multiple influential obsevations in linea Regession. Jounal of Applied Statistics 3, doi:0.080/ [] Bown B.W., J. (980). Pediction analysis fo binay data. in Biostatistics Caseboo, R.G. Mille, J., B. Efon, B. W. Bown, J., L.E. Moses (Eds.), New Yo: Wiley. [3] Ryan.P. (997). Moden Regession Methods, Wiley, New Yo. 6