Comparison of Different Distance Measures on Hierarchical Document Clustering in 2-Pass Retrieval

Transcription

1 Compariso of Differet Distace Measures o Hierarchical Documet Clusterig i 2-Pass Retrieval Azam Jalali Farhad Oroumchia Mahmoud Reza Hejazi Departmet of Computer ad Electrical Egieerig, Faculty of Egieerig, Uiversity of Tehra Ifo. Society group Ira Telecom. Research Ceter IT Departmet, Uiversity of Wollogog i Dubai Ifo. Society group Ira Telecom. Research Ceter Abstract: - Hierarchic documet clusterig has bee applied to search results (query-specific clusterig ) o the grouds of its potetial improved effectiveess compared both to that of static clusterig ad of covetioal iverted file search (IFS). I this paper we review ad compare the effects of seve differet measures of similarity amog documets i hierarchic query specific clusterig. We have coducted a umber of experimets usig OHSUMED documet collectio. The Experimets seems to idicate that the choice of similarity measure effects positively or egatively the quality of clusterig. Key-Words: - cluster-based search; Hierarchical clusterig; Distace measures Itroductio The cluster hypothesis states that relevat documets ted to be more similar to each other tha to o-relevat documets, ad therefore ted to appear i the same clusters [7] Documet clusterig has bee extesively ivestigated as a methodology for improvig documet search ad retrieval. I most of the previous research the strategy was to build a static clusterig of the etire collectio ad the match the query to the cluster ceteroids [].O the other had the behavior ad effectiveess of clusterig methods whe applied to the search results of a IR system (i.e. dyamic, or queryspecific clusterig) have ot bee extesively ivestigated[2].two broad types of query-specific clusterig that have bee maily used i IR: are cluster-based search ad cluster based browsig I cluster-based search, a sigle cluster is retrieved i respose to a query. The documets withi the retrieved cluster are ot raked i relatio to the query but rather the whole cluster is retrieved as a etity. Cluster represetatio refers to the formatio of cluster represetatives, or cetroids that attempt to summarize the cotets of a cluster for the purpose of retrievig the cluster. Icomig queries are matched agaist represetatives, ad the cluster whose represetatives are most similar to the query, is retrieved [2]. Three differet types of cluster-based searches have bee studied i IR: Topdow search, Bottom-up search ad optimal cluster search Cluster-based browsig paradigm clusters documets ito topically coheret groups, ad presets descriptive textual summaries to the user. The summaries cosist of topical terms that characterize each cluster geerally, ad a umber of typical titles that sample the cotets of the cluster. Iformed by the summaries, the user may select clusters, formig a sub-collectio, for iterative examiatio. The clusterig ad re-clusterig is doe o the fly, so that differet topics are see depedig o the sub-collectio clustered [4]. I order to cluster documets oe must first establish a pairwise measure of documet similarity. Numerous documet similarity measures have bee proposed, all of which treat each documet as a set of words, ofte with frequecy iformatio, ad measure the degree of word overlap betwee documets. This paper makes a case for the use of hierarchic query-specific clusterig i IR usig seve differet distace betwee documets.

2 We compare effectiveess of these distaces o hierarchic query specific clusterig. The rest of the paper is structured as follows: Sectio 2 details the documet clusterig algorithms; Sectio 3 provides the details of the experimet. Sectio 4 discusses the experimetal results. At the ed, sectio 5 outlies the coclusios ad the future directio of the work. 2 The Documet Clusterig Algorithms Two differet types of documet clusters ca be costructed. Oe is a flat partitio of the documets ito a collectio of subsets. The other is a hierarchical cluster, which ca be defied recursively as either a idividual documet or a partitio of the corpus ito sets, each of which is the hierarchically clustered. A hierarchical clusterig defies a tree, called a dedrogram, o the documets. The actual effectiveess of hierarchic clusterig ca be gauged by cluster based searches that retrieve the cluster that best matches the query [2] Four hierarchical methods were employed i the experimets: sigle lik, complete lik, group average, ad ceteroid method, which differ i how the distace betwee sub odes is defied i terms of their members. I pair wise sigle-likage clusterig, the distace betwee two odes is defied as the shortest distace amog the pair wise distaces betwee the members of the two odes. I pair wise maximum-likage clusterig, alteratively kow as pair wise complete likage clusterig, the distace betwee two odes is defied as the logest distace amog the pair wise distaces betwee the members of the two odes. I pairwise average-likage clusterig, the distace betwee two odes is defied as the average over all pairwise distaces betwee the elemets of the two odes. I pairwise cetroid-likage clusterig, the distace betwee two odes is defied as the distace betwee their cetroids. The cetroids are calculated by takig the mea over all the elemets i a cluster. As the distace from each ewly formed ode to existig odes ad items eed to be calculated at each step, the computig time of pairwise cetroid-likage clusterig may be sigificatly loger tha for the other hierarchical clusterig methods. The mai reaso behid the choice of these four methods is the fact that they have bee extesively used ad examied i the cotext of IR []. I order to cluster gee expressio data ito groups with similar gees or micro arrays, we should first defie what exactly we mea by similar. We used C clusterig library [], seve distace fuctios are available to measure similarity, or coversely, distace:. c Pearso correlatio The Pearso correlatio coefficiet is defied as x = i x yi y r i= σ x σ y I which x ad y are the sample meas respectively, ad σ σ x, y are the sample stadard deviatio of x ad y. The Pearso distace is the defied as d P = r. 2. a Absolute value of Pearso correlatio The distace is defied as usual as d a = - r. Where, r is the Pearso correlatio coefficiet. 3. u Ucetered Pearso correlatio (equivalet to the cosie of the agle betwee two data vectors.) The ucetered correlatio is defied as where r u = i= σ σ xi σ x () yi σ y ( ) 2 x = x i = i ( ) 2 y = y i = i This is the same expressio as for the regular Pearso correlatio coefficiet, except that the samples meas x ad y are set equal to zero. The distace correspodig to the ucetered correlatio coefficiet is defied as d u = r u. Where r u is the ucetered correlatio. 4. x Absolute ucetered Pearso correlatio (equivalet to the cosie of the smallest agle betwee two data vectors) The distace measure is defied usig the absolute value of the ucetered correlatio, d u =- r u ; where r u is the ucetered correlatio coefficiet. 5. s Spearma s rak correlatio The Spearma rak correlatio is a example of a o-parametric similarity measure. To calculate the ()

3 Spearma rak correlatio, each data value is replaced by their rak if the data i each vector is ordered by their value. The the Pearso correlatio betwee the two rak vectors istead of the data vectors is calculated. Spearma rak distace measure correspodig to the Spearma rak correlatio as d s = - r s ; where r s is the Spearma rak correlatio. 6. e Euclidea distace The Euclidea distace is the oly true metric amog the distace fuctios that are available i the C clusterig library, beig the oly distace fuctio satisfyig the triagle iequality. The Euclidea distace is defied as. d = i= ( x i y i ) 2 7. h Harmoically summed Euclidia distace The harmoically summed Euclidea distace is a variatio of the Euclidea distace, where the terms for the differet dimesios are summed iversely (similar to the harmoic mea): 2 d = = x i i y i The characters i frot of the distace measures are used i figures. 3 Experimetal details The mai objective of these experimets was to compare ad to examie effectiveess of differet distace betwee documet vectors i clusterig. The OHSUMED test collectio [9] was selected as test bed. Four hierarchic agglomerative methods were implemeted as the bechmark. Five differet values for top raked documets also were experimeted with i clusterig. 3. Documet Collectios ad Iitial Retrieval For the experimets, the OHSUMED medical abstracts collectio was selected as test collectio. It is a cliically orieted MEDLINE subset, cosistig of 348,566 refereces coverig all refereces from 27 medical jourals over a five-year period (987-99). I this study we used a subset of the OHSUMED[9] We selected all documets i 987, which icludes 54,7 documets ad 63 queries. The SMART documet retrieval system [2] was used i order to perform the iitial retrieval with atc.atc weightig scheme. The atc measure ormalizes the weights for documet legth, givig all documets a equal chace for retrieval. Formula shows atc weightig scheme of SMART retrieval system. To assig a idexig weight w ij that reflects the importace of each sigle-term Tj i a documet D i, differet factors should be cosidered, as follows: withi-documet term frequecy tf ij, which represets the first letter of the SMART label. collectio-wide term frequecy df j, which represets the secod letter of the SMART label. N I Formula, idf j = log ; F j where, N represets the umber of documets ad F j represets the documet frequecy of term T j. ormalizatio scheme, which represets the third letter of the SMART label. w ij = idf k = j [ idf tfij (.5 + ) 2 max_ tf k tfik ( max_ tf i i 2 )] () The default SMART stoplist ad stemmig were also used i idexig all the collectios ad queries. After the iitial retrieval, the top- raked documets were used to create the collectios that were clustered. Five differet values of were tested: =, =2, =3, =4, = Optimal cluster evaluatio I these experimets, the E effectiveess fuctio that was proposed by Jardie ad Va Rijsberge is used as optimally criterio [] [5] [5] []. The formula for the measure is give by: where P ad R correspod to the stadard defiitios for precisio ad recall (over the set of documets of a specific cluster), ad β is a parameter that reflects the relative importace attached to precisio ad recall. Three values of this parameter are usually used:,.5 ad 2, the first value attributig equal importace to precisio ad recall, the secod deemig precisio twice as importat as recall, ad the third treatig recall twice as importat as precisio. The E effectiveess measure ad these three values of the

4 parameter β are used i the experimets reported i this paper. The optimal cluster for ay give query is the cluster that yields the least E value for that query. Jardie ad Va Rijsberge amed this measure MK. It is used to measure optimal cluster effectiveess i our experimets. The mai advatage of optimal measures elimiates ay bias that may be itroduced from exteral sources. Exteral sources iclude the choice of a particular cluster-based search strategy that matches queries to clusters, ad the ability of a user durig a browsig sessio to choose the cluster, which is most relevat to his/her iformatio eed [2]. We have excluded all the queries that had fewer tha relevat documets i their top raked documets. Fourtee queries were left after the above process. For each of the remaiig queries we would ru the query i SMART system ad cluster the set with five differet values for top-raked documets (, 2, 3, 4, 5). 4 Experimetal Results I this sectio we aalyze the experimetal results ad discuss their implicatios. 4. Optimal Cluster Evaluatio Results Optimal cluster evaluatio results o the MK measure for SMART usig seve differet distace documet-clusterig methods are reported i the figure through 2. The figures preset a compariso of seve clusterig methods (Hierarchical with differet similarity) based o structures bag of words for 5 differet umber of documets (from to 5) ad three differet β values (.5,, 2). These figures compare MK values. The performace of the other 6 measures was similar ad it seems that all of them could be used for hierarchical clusterig i the secod stage of retrieval. The results that preseted i these figures may suggest that there is o sigificat degradatio of effectiveess for decreasig of with β =, β = 2. The effectiveess of applyig differet values of for β =. 5 (figures, 4, 7, ) appears to icrease as icreases, MK icreases too. Our results over all experimetal coditios idicated that the group average method was the most effective i terms of optimal cluster evaluatio. Maximum lik ad Cetroid methods were close to each other with ofte-egligible differeces i effectiveess, while sigle lik displayed the poorest effectiveess of the four. Va Rijsberge [7], ad also Seath ad Sokat [4], emphasized that the various associatio ad distace measures are mootoe with respect to each other. Cosequetly, a clusterig method that depeds oly o the rak orderig of the resemblace values would give similar results for all such measures. I cotrast to this claim, we perceived that the results of hierarchical documet clusterig usig seve distace fuctios were differet. Harmoic method produced results worse tha other methods. The results suggest that, with the exceptio of the smallest value of for the hierarchical methods, there is o sigificat icrease i effectiveess for larger umbers of top-raked documets. Cosequetly, if oe were to choose a uique value for, oe would also have to cosider practical issues. It may be advatageous from a efficiecy poit of view to cluster the top-2 or top-3 documets retured from a search rather tha, for example, the top-5 documets. Moreover, it ca be argued that if the resultig cluster structure was to be preseted to a user i a iteractive task eviromet, the a reduced documet space may be advatageous (e.g. allowig the user to easily ad quickly fid a few relevat documets which could start a relevace feedback iteratio or satisfy the user s iformatio eed). Table : Mea Relevat documets per query for differet best-raked documets Top - a Relevat documets per qu

5 .2.2 MK MK Fig. :Mkfor average-likage clusterig with β =.5 Fig. 4:Mkfor cetroid-likage clusterig with β =.5 MK MK Fig. 2: Mk for average-likage clusterig with β = Fig. 5: Mk for cetroid-likage clusterig with β = MK MK Fig. 3: Mkfor average-likage clusterig with β = 2 Fig. 6: Mkfor cetroid-likage clusterig with β = 2

6 .2.2 MK MK Fig. 7:Mkfor maximum-likage clusterig with β =.5 Fig.:Mkfor sigle-likage clusterig with β =.5 MK MK Fig. 8: Mk for maximum-likage clusterig with β = Fig.: Mk for sigle-likage clusterig with β = MK MK Fig.9: Mkfor maximum-likage clusterig with β = 2 Fig.2: Mkfor sigle-likage clusterig with β = 2

7 5 Future Work ad Coclusios We have perceived that the results of hierarchical documet clusterig deped o features values ad measure similarity betwee documets. The results produced by Harmoic Distace were worse tha the other measures. I future, more experimets will be coducted with other collectios ad differet methods. Aother directio would be to use this olie clusterig method ad use it as a method of presetatio i a iformatio retrieval system. I such a sceario, after clusterig the best represetatives of the clusters could be preseted to the user. I this case user ca use these cluster represetatives for choosig the browsig directio. Refereces: [] Aastasios Tombros, The effectiveess of query-based Hierarchic clusterig of documets for iformatio retrieval, Ph.D. Thesis, Departmet of Computig Sciece Faculty of Computig Sciece, Mathematics ad Statistics, Uiversity of Glasgov, 22. [2] Aastasios Tombros, Robert Villa, C.J. Va Rijsberge, The effectiveess of query-specific hierarchic clusterig i iformatio retrieval, Iformatio Processig ad Maagemet 38,pp ,22. [3] Croft, W. B. A model of cluster searchig based o classificatio, Iformatio Systems, 5, 89 95, 98. [4] Cuttig, D. R., Karger, D. R., Pederse, J. O., & Tukey, J. W. Scatter/Gather: A cluster-based approach to browsig large documet collectios, I Proceedigs of the 5th aual ACM SIGIR coferece, Copehage, Demark pp , 992. [5] Ellis, D., Furer-Hies, J., & Willett, P. Measurig the degree of similarity betwee objects i text retrieval systems, Perspectives i Iformatio Maagemet, 3(2), 28 49, 993. [6] El-Hamdouchi, A., & Willett, P. Techiques for the measuremet of clusterig tedecy i documet retrieval systems, Joural of Iformatio Sciece, 3, , 987. [7] Gordo, A.D. A review of hierarchical classificatio, Joural of the Royal Statistical Society, Series A, 5(2):9-37, 987. [8] Hearst, M. A., & Pederse, J. O. Reexamiig the Cluster Hypothesis: Scatter/Gather o retrieval results, I Proceedigs of the 9th Aual ACM SIGIR coferece, Zurich, Switzerlad pp , 996. [9] Hersh, W. R., Buckley, C., Leoe, T. J. ad Hickam, D. H. OHSUMED: A iteractive retrieval evaluatio ad ew large test collectio for research, I Proceedigs of the 7th Iteratioal ACM SIGIR Coferece o Research ad Developmet i Iformatio Retrieval (SIGIR 94), 92-2, 994. [] Jardie, N., & Va Rijsberge, C. J. The use of hierarchical clusterig i iformatio retrieval, Iformatio Storage ad Retrieval, 7, 27 24, 97. [] Michiel de Hoo, Seiya Imoto, Satoru Miyao, The C Clusterig Library, The uiversity of Tokyo, Istitute of Medical Sciece, Huma Geome Ceter, 23. [2] Salto, G. The SMART retrieval system experimets i automatic documet retrieval, Eglewood Cliffs, NJ: Pretice-Hall, 97. [3] Seath, P.H.A. ad Sokal, R.R.. Numerical taxoomy: the priciples ad practice of umerical classificatio. Sa Fracisco: W.H. Freema, 973. [4] Stefa Ruger, Susa Gauch, Feature Reductio for Documet Clusterig ad Classificatio, Techical Report DTR 2/8; Departmet of Computig, Imperial College; Lodo, Eglad, 2. [5] Va Rijsberge, C. J. Further experimets with hierarchic clusterig i documet retrieval, Iformatio Storage ad Retrieval,, 4, 974. [6] Va Rijsberge, C. J., & Croft, W. B. Documet clusterig: A evaluatio of some experimets with the Crafield 4 Collectio, Iformatio Processig & Maagemet,, 7 82, 975. [7] Va Rijsberge, C.J. Iformatio Retrieval. Lodo: Butterworths, 2d Editio, 979.