DATA DIMENSIONALITY REDUCTION METHODS FOR ORDINAL DATA

Transcription

1 DATA DIMENIONALITY REDUCTION METHOD FOR ORDINAL DATA Matn Pokop Hana Řezanková Abstact Fom questonnae suvey we fequently get data, the values ae expessed n odnal (e.g. Lket) scale. The questonnae contans usually a lot of questons, so we get multdmensonal data matx. To smplfy calculatons wth the data t s useful to educe dmensonalty of the dataset. Fo odnal data we use dffeent o mpoved methods compaed to quanttatve data. Ths atcle ncludes the ovevew and compason of dmensonalty educton methods (e.g. pncpal component analyss, facto analyss, multdmensonal scalng, cluste analyss...). Fom these methods we get goups of smla vaables (latent classes), n some cases we can ceate ntepetaton of these new vaables. Key wods: odnal data, dmensonalty educton, latent class models, multdmensonal scalng, cluste analyss JEL Code: C3, C6, C8 Intoducton Am of ths theoetcal study s to descbe data dmensonalty educton methods especally fo odnal data. Ths knd of data we fequently get fom questonnae suveys. Thus we solve the methods to educe the numbe of vaables chaactezng ndvdual objects. Fom bg amount of vaables (questons) we make new latent vaables, whch ae ceated by goups of ognal vaables. Conventonal data dmensonalty educton methods usually assume quanttatve vaables, so we have to use modfed o dffeent statstcal methods. Thee exst seveal methods sometmes wth dffeent esults, so we compae the esults fom vaous methods. Applcaton of the methods n ths text wll be usually descbed n the softwae R. ome methods especally fo categocal data ae descbed n detal (latent class models). 523

2 1 Ovevew of methods Basc methods of the data dmensonalty educton ae pncpal component analyss PCA, facto analyss FA and multdmensonal scalng MD. Classcal FA methods assume lnea elatons among ognal vaables, new latent vaables ae contnuous and nomally dstbuted. Conventonal facto analyss s usually based on coelaton matx analyss, e.g. usng ank coellaton coeffcent. Fo moe detals see Hebák (2007). Common methods of latent vaables dentfcaton ae latent class models. Thee exst many methods and dffeent methods ae avalable n statstcal softwae packages, e.g. latent class cluste models LCC, dscete facto analyss models DFacto, latent tat analyss LTA, latent pofle analyss LPA, latent class egesson models LCR etc. Fo some of these methods n detal see obíšek and Řezanková (2011). 2 Pncpal component analyss ome methods ae based on multdmensonal space pojecton nto the space wth lowe dmenson. Basc method s pncpal component analyss. The am s to fnd eal dmenson of the data. To fnd eal dmensonalty ognal dataset X s tansfomed to the new coodnate system by an othogonal lnea tansfomaton. Let F (esp. G ) be the vecto of the ows coodnates (esp. columns) on the axs on ank s. These two vectos ae elated by the tanston fomula, e. g. n the case of PCA (equatons 1 and 2) thee ae F G 1 k x m G k, 1 k k x p G, k k k (1) (2) whee F denotes the coodnate of the ndvdual on the axs s, of the vaable k on the axs s, assocated to the vaable k, the egenvalue assocated wth the axs s, p the weght assocated to the ndvdual. 524 G denotes the coodnate m the weght k utable count of components goes fom the vaance, whch s explaned by the sum of the vaance of ognal vaables o fom sceeplot of egenvalues o fom the count of egenvalues, whch ae hghe than 1, f we use coelaton matx nstead of covaance matx. utablty and lmtaton of ths method conssts n the esult, when we get fom hgh count of vaables small count of components wth hgh popoton of explaned vaablty.

3 Hgh dependence of obseved vaables s also sutable, stong coelaton among ognal vaables and components too. Instead of conventonal pncpal component analyss fo quanttatve vaables t s possble to use categocal pncpal component analyss CATPCA, whch tansfoms categocal vaables nto quanttatve vaables and does not assume lnea elatons among vaables. Accodng to ebasten et al. (2008) although a PCA appled on categocal data would yeld esults compaable to those obtaned fom a Multple Coespondence Analyss (facto scoes and egenvalues ae lnealy elated), thee ae moe appopate technques to deal wth mxed data types, namely Multple Facto Analyss fo mxed data avalable n the FactoMneR R package. Multple Facto Analyss fom the same package s also an opton. 3 Multdmensonal scalng Othe method based on multdmensonal space pojecton nto the space wth lowe dmenson s multdmensonal scalng MD. ettng of axes (dmensons) s smla to PCA components settng. Multdmensonal scalng s moe geneal than facto analyss, because t s based on any elaton matx among vaables o ndvduals. Method MD s smla to cluste analyss, because t uses dstance matx of vaables o ndvduals pas. Ths dstance can be based on smlaty measue. mlaty of two vaables can be estmated by some of mutual symmetc smlaty measues. Basc smlaty measue of two quanttatve vaables s Peason coelaton coeffcent. To measue smlaty of odnal vaables t s possble to use e.g. peaman o Kendall ank coelaton coefcent o symmetc ommes coeffcent. Fo detals see e.g. Hendl (2006). Accodng to Holland (2008) nonmetc multdmensonal scalng (MD, also NMD and NM) s an odnaton technque, that dffes n seveal ways fom nealy all othe odnaton methods. In most odnaton methods, many axes ae calculated, but only a few ae vewed, owng to gaphcal lmtatons. In MD, a small numbe of axes ae explctly chosen po to the analyss and the data ae ftted to those dmensons; thee ae no hdden axes of vaaton. econd, most othe odnaton methods ae analytcal and theefoe esult n a sngle unque soluton to a set of data. In contast, MD s a numecal technque, that teatvely seeks a soluton and stops computaton when an acceptable soluton has been found, o t stops afte some pe-specfed numbe of attempts. As a esult, an MD odnaton s not a unque soluton and a subsequent MD analyss on the same set of data and followng the 525

4 same methodology wll lkely esult n a somewhat dffeent odnaton. Thd, MD s not an egenvalue-egenvecto technque lke pncpal components analyss o coespondence analyss, that odnates the data such that axs 1 explans the geatest amount of vaance, axs 2 explans the next geatest amount of vaance, and so on. As a esult, an MD odnaton can be otated, nveted, o centeed to any desed confguaton. Unlke othe odnaton methods, MD makes few assumptons about the natue of the data. Fo example, pncpal components analyss assumes lnea elatonshps and ecpocal aveagng assumes modal elatonshps. MD makes nethe of these assumptons, so s well suted fo a wde vaety of data. MD also allows the use of any dstance measue of the samples, unlke othe methods, whch specfy patcula measues, such as covaance o coelaton n PCA o the mpled ch-squaed measue n detended coespondence analyss. The method stats wth a matx of data consstng of n ows of samples and p columns of vaables, Fom ths symmetcal matx of all pawse dstances among samples s calculated wth an appopate dstance measue, such as Eucldean dstance, Manhattan dstance (cty block dstance), and Bay dstance. The MD odnaton wll be pefomed on ths dstance matx. Next, a desed numbe of m dmensons s chosen fo the odnaton. Dstances among samples n statng confguaton ae calculated, typcally wth a Eucldean metc. These dstances ae egessed aganst the ognal dstance matx and the pedcted odnaton dstances fo each pa of samples s calculated. A vaety of egesson methods can be used, ncludng lnea, polynomal, and non-paametc appoaches. In any case, the egesson s ftted by least-squaes. The goodness of ft of the egesson s measued based on the sum of squaed dffeences between odnaton-based dstances and the dstances pedcted by the egesson. Ths goodness of ft s called stess and can be calculated n seveal ways, e.g. fom equaton 3 wth one of the most common beng Kuskal s tess tess d h, h d h, dˆ 2 h h 2, (3) whee d h s the odnated dstance between samples h and, and dˆ s the dstance pedcted fom the egesson. Ths confguaton s then mpoved by movng the postons of samples n odnaton space by a small amount n the decton of steepest descent, the decton n whch stess changes most apdly. The odnaton dstance matx s ecalculated, the egesson pefomed agan and stess ecalculated, and ths ente pocedue of nudgng 526

5 samples and ecalculatng stess s epeated untl some small specfed toleance value s acheved o untl the pocedue conveges by falng to acheve any lowe values of stess, whch ndcates that a mnmum (pehaps local) has been found. A scee dagam (stess vesus numbe of dmensons) can then be plotted, on whch one can dentfy the pont beyond whch addtonal dmensons do not substantally lowe the stess value. A second cteon fo the appopate numbe of dmensons s the ntepetablty of the odnaton, that s, whethe the esults make sense. tess nceases both wth the numbe of samples and wth the numbe of vaables. R has two man MD functons avalable, somd, whch s pat of the MA lbay, and metamd, whch s pat of the vegan lbay. The metamd outne allows geate automaton of the odnaton pocess, so s usually the pefeed method. The metamd functon uses somd n ts calculatons as well as seveal helpe functons. The metamd outne also has the useful default behavo of followng the odnaton wth a otaton va pncpal components analyss such that MD axs 1 eflects the pncpal souce of vaaton and so on, as s chaactestc of egenvalue methods. 3 Ovevew of latent class and latent class egesson models Accodng to Lnze (2011) latent class analyss s a statstcal technque fo the analyss of multvaate categocal data. When obseved data take the fom of a sees of categocal esponses (as fo example, n publc opnon suveys, ndvdual-level votng data, studes of nte-ate elablty, o consume behavo and decson-makng), t s often ou nteest to nvestgate souces of confoundng between the obseved vaables, dentfy and chaacteze clustes of smla cases, and appoxmate the dstbuton of obsevatons acoss many vaables of nteest. Latent class models ae a useful tool fo accomplshng these goals. The latent class model seeks to statfy the coss-classffcaton table of obseved (manfest) vaables by an unobseved (latent) categocal vaable, that elmnates all confoundng between the manfest vaables. Responses to all of the manfest vaables ae assumed to be statstcally ndependent. The model, n effect, pobablstcally goups each obsevaton nto a latent class, whch n tun poduces expectatons about how that obsevaton wll espond on each manfest vaable. Although the model does not automatcally detemne the numbe of latent classes n a gven data set, t does offe a vaety of pasmony and goodness of ft statstcs, that the we may use n ode to make a theoetcally and empcally assessment. 527

6 Because the unobseved latent vaable s nomnal (membeshp of a class), the latent class model s actually a type of fnte mxtue model. The component dstbutons n the mxtue ae coss-classffcaton tables of equal dmenson to the obseved table of manfest vaables, and, followng the assumpton of condtonal ndependence, the fequency n each cell of each component table s smply the poduct of the espectve class-condtonal magnal fequences (the paametes estmated by the latent class model ae the popoton of obsevatons n each latent class, and the pobabltes of obsevng each esponse to each manfest vaable, condtonal on latent class). A weghted sum of these component tables foms an appoxmaton (o, densty estmate) of the dstbuton of cases acoss the cells of the obseved table. Obsevatons wth smla sets of esponses on the manfest vaables wll tend to cluste wthn the same latent classes. An extenson of ths basc model pemts the ncluson of covaates to pedct latent class membeshp. Wheeas n the basc model, evey obsevaton has the same pobablty of belongng to each latent class po to obsevng the esponses to the manfest vaables, n the moe geneal latent class egesson model, these po pobabltes vay by ndvdual as a functon of some set of ndependent vaables. polca s a softwae package fo the estmaton of latent class and latent class egesson models fo polytomous outcome vaables (vaables wth moe than two dstnct categoes), mplemented n the R The basc latent class model s a fnte mxtue model, n whch the component dstbutons ae assumed to be mult-way coss-classffcaton tables wth all vaables mutually ndependent. The latent class egesson model futhe enables us to estmate the effects of covaates on pedctng latent class membeshp. polca uses expectaton-maxmzaton and Newton-Raphson algothms to fnd maxmum lkelhood estmates of the model paametes. 4 Latent class models Accodng to Lnze (2011) the basc latent class model s a fnte mxtue model n whch the component dstbutons ae assumed to be mult-way coss-classfcaton tables wth all vaables mutually ndependent. uppose we obseve J polytomous categocal vaables (the manfest vaables), each of whch contans K j possble outcomes, fo ndvduals = 1,...,N. The manfest vaables may have dffeent numbes of outcomes, hence the ndexng by j. Denote as Y jk the obseved values of the J manfest vaables such that Y jk = 1 f espondent gves the k-th esponse to the j-th vaable, and Y jk = 0 othewse, whee j = 1,..., J and k = 1,...,K j. The latent class model appoxmates the obseved jont dstbuton of the manfest 528

7 vaables as the weghted sum of a fnte numbe R of consttuent coss-classfcaton tables. Let π denote the class-condtonal pobablty, that an obsevaton n class = 1,...,R poduces the k-th outcome on the j-th vaable. Wthn each class, fo each manfest vaable, K j theefoe k 1 1. Futhe denote as p the R mxng popotons that povde the weghts n the weghted sum of the component tables, wth p 1. The values of p ae also efeed to as the po pobabltes of latent class membeshp, as they epesent the uncondtonal pobablty that an ndvdual wll belong to each class befoe takng nto account the esponses Y jk povded on the manfest vaables. The pobablty that an ndvdual n class poduces a patcula set of J outcomes on the manfest vaables, assumng condtonal ndependence of the outcomes Y gven class membeshps, s the poduct f Y J K j ; jk Y. (4) j 1 k 1 The pobablty densty functon acoss all classes s the weghted sum P Y R J K j Y, p jk. (5) p 1 j 1 k 1 The paametes estmated by the latent class model ae p and π. Gven estmates pˆ and ˆ of p and π, espectvely, the posteo pobablty that each ndvdual belongs to each class, condtonal on the obseved values of the manfest vaables, can be calculated usng Bayes' fomula: whee 1,...R Pˆ Y pˆ f Y ; ˆ R pˆ f Y ; ˆ q1 q q wth any fnte mxtue model, the EM algothm s applcable because each ndvdual's class 529, (6). Recall that the ˆ ae estmates of outcome pobabltes condtonal on class. It s mpotant to eman awae that the numbe of ndependent paametes estmated by the latent class model nceases apdly wth R, J, and K j. Gven these values, the numbe of paametes s R K j R 1 j 1. If ths numbe exceeds ethe the total numbe of obsevatons, o one fewe than the total numbe of cells n the coss-classfcaton table of the manfest vaables, then the latent class model wll be undentfed. polca estmates the latent class model by maxmzng the log-lkelhood functon N R K j 1 1 j 1 k 1 J ln L ln p (7) wth espect to p and π, usng the expectaton-maxmzaton (EM) algothm. Ths loglkelhood functon s dentcal n fom to the standad fnte mxtue model log-lkelhood. As Y jk

8 membeshp s unknown and may be teated as mssng data. The EM algothm poceeds teatvely. Begn wth abtay ntal values of pˆ and ˆ, and label them the expectaton step, calculate the mssng class membeshp pobabltes usng Equaton 6, substtutng n pˆ and ˆ pˆ and ˆ. In. In the maxmzaton step, update the paamete estmates by ˆ Y wth maxmzng the log-lkelhood functon gven these posteo P N new 1 pˆ Pˆ Y (8) N 1 as the new po pobabltes and N Y Pˆ Y j new 1 ˆ (9) j N Pˆ Y 1 new as the new class-condtonal outcome pobabltes. In Equaton 9, ˆ s the vecto of length K j of class- condtonal outcome pobabltes fo the j-th manfest vaable; and Y j s the N K j matx of obseved outcomes Y jk on that vaable. The algothm epeats these steps, assgnng the new to the, untl the oveall log-lkelhood eaches a maxmum and ceases to ncement beyond some abtaly small value. j polca takes advantage of the teatve natue of the EM algothm to make t possble to estmate the latent class model even when some of the obsevatons on the manfest vaables ae mssng. Although polca does offe the opton to lstwse delete obsevatons wth mssng values befoe estmatng the model, t s not necessay to do so. Instead, when detemnng the poduct n Equaton 4 and the sum n the numeato of Equaton 9, polca smply excludes fom the calculaton any manfest vaables wth mssng obsevatons. The pos ae updated n Equaton 6 usng as many o as few manfest vaables as ae obseved fo each ndvdual. Dependng on the ntal values chosen fo complexty of the latent class model beng estmated, the EM algothm may only fnd a local maxmum of the log-lkelhood functon, athe than the desed global maxmum. Fo ths eason, t s always advsable to e-estmate a patcula model a couple of tmes when usng polca, n an attempt to fnd the global maxmze to be taken as the maxmum lkelhood soluton. One of the benefts of latent class analyss, n contast to othe statstcal technques fo clusteed data, s the vaety of tools avalable fo assessng model ft and detemnng an appopate numbe of latent classes R fo a gven data set. In some applcatons, the numbe 530 pˆ and ˆ, and the

9 of latent classes wll be selected fo pmaly theoetcal easons. In othe cases, howeve, the analyss may be of a moe exploatoy natue, wth the objectve beng to locate the best fttng o most pasmonous model. We may then begn by fttng a complete ndependence model wth R = 1, and then teatvely nceasng the numbe of latent classes by one untl a sutable ft has been acheved. Addng an addtonal class to a latent class model wll ncease the ft of the model, but at the sk of fttng to nose, and at the expense of estmatng a futhe K 1 1 model paametes. Pasmony ctea seek to stke a balance between j j ove- and unde-fttng the model to the data by penalzng the log-lkelhood by a functon of the numbe of paametes beng estmated. The two most wdely used pasmony measues ae the Bayesan nfomaton cteon, o BIC and Akake nfomaton cteon, o AIC. Pefeed models ae those that mnmze values of the BIC and/o AIC. Let Λ epesent the maxmum log-lkelhood of the model and Φ epesent the total numbe of estmated paametes. Then, AIC = -2Λ+2Φ and BIC = -2Λ+ Φ lnn. polca calculates these paametes automatcally when estmatng the latent class model. The BIC wll usually be moe appopate fo basc latent class models because of the elatve smplcty. Calculatng Peason's Χ 2 goodness of ft and lkelhood ato ch-squae (G 2 ) statstcs fo the obseved vesus pedcted cell counts s anothe method to help detemne how well a patcula model fts the data, fo moe detals see Lnze (2011). Lke the AIC and BIC, these statstcs ae outputted automatcally afte callng polca. 5 Optmal scalng by Gf methods The challenge wth categocal vaables s to fnd a sutable way to epesent dstances between vaable categoes and ndvduals n the factoal space. To ovecome ths poblem, we can look fo a non-lnea tansfomaton of each vaable, whethe t s nomnal, odnal, polynomal, o numecal wth optmal scalng. Ths s well explaned n Leeuw (2009), and an mplementaton s avalable n the coespondng R package homals. As extenson to ths tansfomaton, havng nonmetc vaables, we can use dmensonalty educton methods, e.g. nonlnea pncpal component analyss (NLPCA). The tem nonlnea petans to nonlnea tansfomatons of the obseved vaables. In Gf temnology, NLPCA can be defned as homogenety analyss wth estctons on the quantfcaton matx. 6 Fuzzy Clusteng 531

10 Accodng to Oksanen (2010) we have so fa woked wth classffcaton methods, whch mplctly assume, that thee ae dstnct classes. The eal stuaton s usually dffeent. If thee ae classes, they ae vague and have ntemedate and untypcal cases. Wth one wod, they ae fuzzy. Fuzzy classffcaton means, that each obsevaton has a cetan pobablty of belongng to a cetan class. In the csp case, t has pobablty 1 of belongng to a cetan class, and pobablty 0 of belongng to any othe class. In a fuzzy case, t has pobablty < 1 fo the best class, and pobabltes > 0 fo seveal othe classes. Fuzzy classffcaton s smla to K-means clusteng n fndng the optmal classffcaton fo a gven numbe of classes, but the poduced classffcaton s fuzzy: the esult s a pobablty pofle of class membeshp. The fuzzy clusteng s povded by functon fanny (equaton 10) n package cluste. Requested membeshp pobabltes we get fom the mnmalzaton of the functon n n 2 2 u u d k h jh j 1 j 1 J F n h1 2 2uh j 1 (10) whee the values of u ae membeshp pobabltes and values of d ae Eucldean dstances among the objects. It s dffcult to show the fuzzy esults gaphcally, but t s possble to use stas functon (wth many optonal paametes) to show the pobablty pofle, and t daws a convex hull of the csp classfcaton. The sze of the secto shows the pobablty of the class membeshp and n clea cases one of the segments s domnant. Concluson Ths study was amed to the ovevew of data dmensonalty educton method especally fo categocal (odnal) data. ome advantages and dffcultes of the methods wee pesented, n futue eseach these methods wll be aplled on eal dataset and compason of the esults wll be pefomed. Refeences 1. Hebák, Pet. Víceozměné statstcké metody 3. Paha: Infomatoum, Hendl, Jan. Přehled statstckých metod: analýza a metaanalýza dat. Paha: Potál, Holland, teven M. Non-metc multdmensonal scalng (MD). Athens: R foge, Le, ebasten, and Josse, Jule, and Husson, Fancos. FactoMneR: An R package fo multvaate analyss. Jounal of statstcal softwae 25 Mach 2008: sec

11 5. Leeuw, Jan, and Ma, Patck. Gf Methods fo Optmal calng n R: The Package homals. Jounal of statstcal softwae 31 Aug. 2009: sec Lnze, Dew, and Lews, Jeffey. polca: An R Package fo Polytomous Vaable Latent Class Analyss. Jounal of statstcal softwae 42 June 2011: sec Oksanen, Ja. Cluste Analyss: Tutoal wth R. Mendeley, obíšek, Lukáš, and Řezanková, Hana. ovnání metod po edukc dmenzonalty aplkovaných na odnální poměnné. Acta Oeconomca Pagensa : Contact Matn Pokop Vysoká škola ekonomcká v Paze nám. W. Chuchlla Paha 3 mas@post.cz Hana Řezanková Vysoká škola ekonomcká v Paze nám. W. Chuchlla Paha 3 hana.ezankova@vse.cz 533