Web Page Clustering by Combining Dense Units

Transcription

1 Web Page Clusterg by Combg Dese Uts Morteza Haghr Chehregha, Hassa Abolhassa ad Mostafa Haghr Chehregha Departmet of CE, Sharf Uversty of Techology, Tehra, IRA {haghr, Departmet of ECE, Uversty of Tehra, Tehra, Ira, Abstract Oe of the most mportat approaches of extractg kowledge from the web s to cluster the web data. I ths paper a ovel method for clusterg the web pages s preseted whch at frst fds the dese uts usg K-Meas method ad the os these uts for costructg fal sutable clusters. The method also s exteded for herarchcal clusterg. The expermetal results show the hgh qualty of both flat ad herarchcal clusters. Idex Terms Web Mg, Web clusterg, herarchcal clusters, K-Meas. I. ITRODUCTIO Data clusterg s a mportat pre-processg task formato mg that ca provde useful results. Clusterg volves dvdg a set of data obects to o-specfed groups. The data obects wth each group should exhbt a large degree of smlarty whle the smlarty amog dfferet clusters should be mmzed. Clusterg takes a mportat role o the web; t ca be used for ehacg search results, ehacg web crawlg, ad orgazg ad presetg doma kowledge. Up to ow varous methods have bee troduced for clusterg the web, more of them use techques such as lk aalyss [], [,9], cotet mg [] ad combato of them [,]. I a overall categorzato, web data clusterg algorthms ca be dvded to two categores: Herarchal ad Parttog algorthms. Herarchcal algorthms cluster the web pages a mult level ad tree-lke structure, whle parttog methods cluster the web data a sgle ad flat level [,]. I geeral oly algorthms such as graph-based [,] ad tree-based [] methods ad methods that fd a set of cetrods (K-Meas) [], are appled for clusterg the web data []. Although herarchcal methods are ofte sad to have better results from qualty respectve, but usually they do ot provde the repetto ad the reassgmet of pages, whch may have bee poorly clustered the early steps []. Moreover, the tme complexty of the herarchcal methods s ofte quadratc [0]. Parttog methods dvde a collecto of documets to a set of groups, so as to maxmze a pre-defed ftess measure. It seems that the recet parttog clusterg methods are well suted for clusterg a large documet dataset due to ther relatvely low computatoal requremets [0]. The best kow parttog algorthm s K-Meas [] that a smple form selects K pots as cluster ceters ad assgs each data pot to the earest ceter. The updatg ad reassgg process ca be kept utl a covergece crtero s met. Ths algorthm ca be performed o a large data set almost lear tme complexty. K-Meas has some problems that make t approprate for may applcatos. I secto we wll expla some of ts drawbacks. I ths paper we propose a ew method whch at frst fds dese uts of the etre data set ad the combes them for creatg fal clusters. The combato process s doe ether flat or herarchcal modes ad so ca geerate flat or herarchcal clusters. Whle creatg dese uts, a ceter of gravty s determed for each of them. I the combato step, these ceters are used as represetatves of dese uts average dstace combato. The remag of the paper s orgazed as follows: I secto, some prmary deftos ad cocepts are troduced. Secto cotas a complete descrpto of the proposed method ad ts tme complexty aalyss. The expermetal results of the method are evaluated secto ad fally a cocluso s gve secto. II. PRIMARY DEIITIOS AD COCEPTS A. Documet Represetato ad Smlarty Measures I the most data mg algorthms, each data pot (such as a documet) s represeted as a V = { v,,, } vector v v v, where s the weght of dmeso vector space. About text documets, ths s kow as Vector Space Model [] that each weght v represets the weght of assocated term of the documet. The most commo term weghtg approach s the combato of Term requecy wth Iverse Documet requecy (T-ID) []. I ths approach the weght of term documet s defed as (). w = tf df = tf log ( / df) () That tf s the umber of occurreces of term the documet ; df s the total term frequecy data set ad s the umber of documets. The vector space model gves us a good opportuty for defg varous metrcs of smlarty betwee two documets. The most commo smlarty metrcs are Mkowsk dstaces [] ad cose measure []. Mkowsk dstaces

2 computes the dstace of documets m p ad m by () (for = t s coverted to Eucldea dstace). d / m D( mp, m) = m, p m, () = Cose measure s defed by (), where m t pm s the er product of two vectors. Both metrcs are wdely used text clusterg lteratures. However t seems that the cases whch the umber of dmesos of two vectors dffers cosderably, the cose s more useful. t mm p cos( mp, m) = m m () p B. Descrpto of K-Meas Algorthm K-Meas chooses K pots as cetrods ad assgs each web page to the earest oe. The the cetrods are updated. The assgg ad updatg process s kept utl a covergece crtero s met. Sce clusterg s a P-Complete problem, so ths algorthm (lke other algorthms) s a heurstc soluto that fds the earest local optmum of the search space. The local optmum ad therefore the qualty of the results are depedet to the tal selecto of the ceters. Dfferet steps of K- Meas are show gure.. Ital Step : Select K tal pots by radom;. Repeat for each data pot dp data set: fd the earest cetrod to dp; assg dp to the earest cetrod; for each cluster c cluster set: update the cetrod c;. utl (covergece s cofrmed) g.. The K-Meas algorthm. I the algorthm of gure, the crtero used for dstgushg the covergece ad so falzg the process s defed as (). Ths measure s kow as mmzg Sum of Squared Errors. K E( A, C) = ( a ) D ( c, x ) () = = I (), A s the assgmet matrx (shows whch web page has bee assged to whch cetrod) ad C = { c } s the set of K cetrods. Assgmet matrx cludes values of 0 ad ( crsp clusterg) ad s obtaed by assgg each data pot to the correspodg cetrod. I the algorthm of gure, updatg the cetrods s doe accordg to (). c = ( a ) x = a =, K () K-Meas has several drawbacks. The ma drawback s the sestvty of result clusters to the tal cetrods; so that t may covergece to the local optma [9] that s ot acceptable. I fact, K-meas s a optmzato soluto that fds the local optmum placed the vcty of the tal soluto. But the same tal cluster cetrods a dataset wll always geerate the same cluster results ad so repeatg the algorthm for achevg better results must be doe wth dfferet tal cetrods. III. PROPOSED ALGORITHM I ths secto, we troduce our algorthm ad aalyze ts tme complexty. The algorthm cludes two steps: costructg dese uts ad combg them for creatg fal clusters. The combato process ca be doe ether flat or herarchcal mode. I followg we descrbe each step wth eough detals. A. Costructg Dese Uts The algorthm at frst selects a costat value for cetrods deoted by C. It s a large value that s related to the umber of all data pots. If shows the umber of data pots, the C s selected as MPts. Where MPts shows the mmum requred data pots that must resde a small rego so that the rego ca be cosdered as dese. I the ext step, each data pot s assged to the cetrods ad after assgmet; the cetrods are updated accordg to (). The process s repeated utl the covergece crtero s met (smlar to typcal K-Meas). The covergece s met whe the posto of cetrods do ot chage more after. Therefore at the ed of ths step we wll have umerous sphercal dese uts whch each of them has a ceter pot represetg the dstrbuto of desty. The process has bee depcted gure.. Ital Step : Select MPts; Select C as MPts ;. Repeat for each data pot dp data set: assg dp to the earest ceter; for each ut u: update the ceter of u;. utl (covergece s cofrmed) g.. dg dese uts. B. lat combato of Dese Uts I the ext step we combe the dese uts for costructg fal clusters. Combg these dese uts a optmum way so that the fal clusters would be the best combato s a P-Complete problem. Ths s proofed Theorem. Theorem - dg the best combato of dese uts that yelds best clusters s a P-Complete problem. Proof- We kow that data clusterg s a P-Complete problem []. We ca costruct a hyper graph wth odes represetg the dese uts. Each ode has lks to two kds of odes: odes that correspod to the data

3 pots sde the dese ut, ad cotermous odes (w.r.t dstace betwee the ceters of dese uts). We kow that each hyper ode (dese ut) has a lmted umber of data pots. So the umber of hyper odes wll be tself from O(). Therefore combato of these uts ca be smulated as a clusterg problem ad so fdg the optmum soluto s a P-Complete problem. So, because of complexty cosderatos, we should take a heurstc method whch ca perform combato wth a reasoable tme complexty. or ths we propose two alteratves:. Defe a threshold ad every two dese uts whch have the smlarly more tha ths threshold, o together ad costruct a uque dese ut. So the resultat clusters wll be depedet o the order of traversg the dese uts. So the combato may lead to a local optmum whle for achevg better results several rus may be eeded.. Wth respect to the drawbacks of the frst approach, we ca follow a more heurstc method. We follow a process smlar to the average lkage herarchcal clusterg [0] but produce the clusters flat format. The method repeatedly combes smaller uts ad produces bgger oes. The crtero used for combg the smaller groups s average dstace betwee data sde two uts. The created clusters have hgh qualty, but fdg the average dstace betwee dfferet dese uts requres hgh tme complexty. Our process has a cosderable preferece for applyg ths method: I the step of costructg sphercal dese uts, the ceter of each ut s gaed whle assgg ad updatg process. Therefore agast of other lkage methods, here there s o eed for fdg average dstace betwee base clusters; comparg the dstace betwee uts ca be doe oly by comparg the dstace betwee ther ceters. Betwee these two methods, we select the secod oe. So after costructg the dese uts, we follow the process of gure. I the algorthm of gure, fdg ew ceter s doe accordg to ().. Italze Select K as the desred umber of clusters;. Repeat d the two earest uts; Jo them as a uque ut; d the ceter of ew ut;. utl (there are oly K uts) g.. The Combato Step. ew Ceter = C + C + + () That ad shows the umber of members of uts ad, respectvely. So, the ew ceter s weghted average of two ceters. The og process s kept utl the desred clusters are obtaed. C. Avalg Herarchcal Clusterg I ths secto we mprove the method for herarchcal clusterg. or ths each combato step, we must exame whether the combato yelds a compoud ut the same ut or a ew ut must be costructed hgher level. Usg a costat threshold as agglomerato level ca ot be useful. However small low level clusters should use smaller thresholds whle bg hgh level clusters should have greater thresholds. Sce the clusters are costructed from smaller oes to bgger oes ad we wat about the og thresholds to act ths maer, so we establsh a relatoshp betwee the thresholds ad the ot uts. We defe a measure ad try to optmze t durg the combato process. The measure s defed relato (). Max ( Clustered Ds ta ces of C ad C )? M () Ds tace betwee two ceters M s a costat value whch s determed by user. Therefore, accordg to the result of equato () we have two stuatos (we ame the left part of () as L):. If L M, the these two dese uts wll o wth each other ad wll costruct a uque ut the same level. The ceter of the ew ut s updated by the weghted averagg of relato (). Also, other parameters of the ew ut are calculated.. If L < M, the a ew ut wll be costructed a hgher level ad the two uts wll be ts chldre. The ew ut cotas combed uts wth ther updated parameters. Ths method gves us a heurstc way for costructg the herarchcal clusters. D. Aalyss of Tme Complexty of the Method K-Meas requres O() tme of complexty for fdg a lmted umber of cluster cetrods. But sce the umber of the cetrods s tself from O(), so the total tme complexty of the frst step wll be equal to O( ). Secod step (combato step) ca be mplemeted wth the tme complexty of O( ) whether flat or herarchcal mode. Ths process ca be doe as follows: at frst we cosder a array wth the sze of m- (m s the umber of dese uts). We store each cell of array the earest ceter to the ceter dexed by the array's d as well as the correspodg dstace. I each combato the shortest dstace s foud wth O(m). After fdg the shortest dstace, the array s updated for the ewly created ut whch ths ca be doe wth O(m) tme of complexty, too. Therefore the total tme complexty for ths part wll be from O(m ) that t s tself equal to O( ).

4 IV. EXPERIMETAL RESULTS or evaluato of proposed method we have used two data sets: Oe data set s collected from the set of REUTERS documets ad aother oe s collected from Poltcs area cotag web pages that are selected radomly about the some topcs of Poltcs doma. The propertes of the data sets are show Table. or stemmg we have used the Porter algorthm ad after calculatg the T/ID values of vectors, they were ormalzed. TABLE. Examed Data Sets. Data Set REUTERS Poltcs Sze 00 o.of Clusters 0 Selected MPts There are several methods for evaluato of clusterg results most two well kow methods amog them are etropy-based methods ([,0]) ad -measure []. Because of more geeralty of -measure for evaluato of web searches, we use ths measure for evaluato of our results. -measure combes two measures precso ad recall ad evaluates whether the clusterg ca remove the ose pages ad geerate clusters wth hgh qualty. If P ad R show Precso ad Recall respectvely, ths measure s defed by () where precso ad recall are obtaed by (9). I these formulas shows the sze of cluster, g shows the sze of class ad (,) shows the umber of pages of class cluster. ( P (, )* R (, )) gt (, ) =, = max{ (, )} () ( P (, ) + R (, ) P(, ) = (, )/, R(, ) = (, )/ g (9) TABLE - Comparso of methods for the frst data set. CETER ID K-Meas Proposed Method or more comparsos we have mplemeted the K- Meas ad have evaluated ts results (sce the results of the K-Meas are depedet o the tal cetrods, so we have ru the K-Meas fve tmes wth dfferet tal ceters ad have selected the best result). The evaluato results are show Tables ad. About the frst data set, the total -Measure s 0.0 for the proposed algorthm whle ts value s 0. for K-Meas method. The dfferece betwee these two values shows the qualty of the proposed algorthm. About the secod data set, the total -Measure value of the K-Meas ad the proposed method are 0.0 ad 0.9, respectvely. These values also show the effcecy of the proposed method. TABLE - Comparso of methods for the secod data set. CETERID 9 0 K-Meas Proposed Method I the ext step we exame the herarchcal clusterg. or ths we select M equal to 0. ad costruct the herarchy. The resultat herarchy s depcted gure. The umbers exstg each cluster depct ther umber of members ad we assg a uque ID for each cluster a left to rght BS order. g.. The obtaed herarchy. Evaluato methods such as -measure ad etropy are developed for evaluato of flat clusters (or lowest level of herarchcal clusters). So here we use a ehaced verso for herarchcal clusterg developed []. Accordg to [] the -measure values of dfferet clusters follow from a butto-up process: The precso of a paret cluster s calculated from the precso of ts chldre accordg to (0). I (0) P R shows the precso of the cluster members whch do ot belog to ay other clusters. Sce for computg the recall we must calculate the R R (s defed smlar wth P R ) by traversg all web pages, so a more smple way ca be doe by calculatg recall wthout cosderg pre-calculated values for sub-clusters []. k k k C. chldre PC = Pk + PR (0) k C. chldre However usg the adapted measure, the results are brought Table. We have appled the herarchcal method oly o the frst data set. I ths case the

5 obtaed total -Measure s 0.9 whch s acceptable for a herarchcal clusterg. TABLE - The results of herarchcal algorthm. CETER ID P V. COCLUSIO R I ths paper, a ew method for clusterg the web documets s troduced. I fact, the proposed method tres to overcome some drawbacks of the well kow K- Meas algorthm whch fds oly the earest local optmum. The method tres to mprove the local optmum toward global optmum. Ths method, at frst, fds dese uts by bombardg the data space wth too may cetrods. Therefore at the ed, we have dese uts wth a represetatve pot (ceter) for each of them. Havg these ceter pots, we ca apply average lkage clusterg algorthm o the dese uts ad costruct the fal clusters by og them. The method has bee exteded for herarchcal clusterg. or ths purpose a heurstc measure has bee defed ad optmzed each cluster. If a ew ot cluster does ot satsfy the measure, the cluster s created hgher level. The method has bee evaluated by well kow - Measure ad ts mproved verso for herarchcal clusters. Accordg to the evaluato results, both of the flat ad herarchcal clusters have sutable qualty. REERECES [] Cos K., Pedrycs W., Swarsk R., "Data Mg- Methods for Kowledge Dscovery", Kluwer Academc Publshers, 99. [] Evertt, B., "Cluster Aalyss", d Edto. Halsted Press, ew York, 90. [] Getoor, L., "Lk Mg: A ew Data Mg Challege", ACM SIGKDD Exploratos ewsletter, Vol., pp. 9, 00. [] Grra,., Crucau, M., Bouemaa,., "Usupervsed ad Sem-supervsed Clusterg: a Bref Survey", th ACM SIGMM teratoal workshop o Multmeda formato retreval, pp. 9-, 00. [] Haghr Chehregha, Morteza, Abolhassa, H., ad Haghr Chehregha, Mostafa, "Attag hgher qualty for Desty Based Algorthms", Web Reasog ad Rule Systems, LCS (Sprger), pp. 9-, 00. [] Hea, X., ad Zhaa, H., Dg, C.H.Q., Smo, H.D., "Web documet clusterg usg hyperlk structures", Computatoal Statstcs & Data Aalyss, Vol., pp.9-, 00. [] Hezger, M., "Hyperlk aalyss o the world wde web", Sxteeth ACM coferece o Hypertext ad hypermeda HYPERTEXT '0, pp., 00. [] Ja, A. K., Murty, M.., ad ly, P. J., "Data Clusterg: A Revew", ACM Computg Surveys (CSUR), Vol., Issue, pp., 999. [9] Kleberg, M., "Authortatve sources a hyperlked evromet", th Aual ACM-SIAM Symposum o Dscrete Algorthms, SODA, 99. [0] Koller, D., Saham, M., "Herarchcally classfyg documets usg very few words", th Iteratoal Coferece o Mache Learg, pp. 0, 99. [] Larse, B., ad Aoe, C., "ast ad effectve text mg usg lear-tme documet clusterg", SIGKDD 99, CA, pp., 999. [] Lu, B., Chag, K.C.-C., "Edtoral: Specal Issue o Web Cotet Mg", ACM SIGKDD Exploratos ewsletter, Vol., Issue, pp. -, 00. [] McQuee, J., "Some methods for classfcato ad aalyss of multvarate observatos", fth Berkeley Symposum o Mathematcal Statstcs ad Probablty, pp. -9, 9. [] evlle, J., Adler, M., Jese, D., "Clusterg Relatoal Data Usg Attrbute ad Lk Iformato", th Iteratoal Coferece o Artfcal Itellgece, 00. [] urme, M., Hokarata, A., Kärkkäe, T., "ExtMer: Combg Multple Rakg ad Clusterg Algorthms for Structured Documet Retreval", Sxteeth Workshop o Database ad Expert Systems Applcatos, pp. 0-00, 00. [] Qula, R.J., "C.: Programs for Mache Learg", Morga Kaufma, 99. [] Scheker, A., Last, A., Buke, H., Kadel, A., "A Comparso of Two ovel Algorthms for Clusterg Web Documets", d Iteratoal Workshop o Web Documet Aalyss, pp. -, 00. [] Salto, G., Buckley, C., "Term-weghtg approaches automatc text retreval". Iformato Processg ad Maagemet, (): pp. -, 9. [9] Selm, S.Z., Ismal, M.A. "K-meas type algorthms: A geeralzed covergece theorem ad characterzato of local optmalty", IEEE Tras. Patter Aal. Mach. Itell., pp., 9. [0] Stebach, M, Karyps, G., Kumar, V., "A Comparso of Documet Clusterg techques", KDD 000, Techcal report of Uversty of Mesota, 000. [] Wag, Y., ad Ktsuregawa, M., "Ehacg Cotet- Lk Coupled Web Page Clusterg ad It's Evaluato", DEWS' 00, 00. [] Wess, R., Vélez, B., ad Sheldo, M.A., "HyPursut: A Herarchcal etwork Search Ege that Explots Cotet-Lk Hypertext Clusterg", th ACM coferece o Hypertext, USA, pp. 0-9, 99. [] Yao, Z., ad Cho, B., "Bdrectoal Herarchcal Clusterg for Web Mg", IEEE/WIC Iteratoal Coferece o Web Itellgece WI, pp. 0-, 00. [] Zamr,O., Etzo, O., "Web documet clusterg: A feasblty demostrato", 9 th Iteratoal ACM SIGIR Coferece o Research ad Developmet of Iformato Retreval, pp.-, Australa, 99.