SUPPORT VECTOR CLUSTERING FOR WEB USAGE MINING

Transcription

1 1 SUPPOT VECTO CUSTEING FO WEB USAGE MINING WEI SHUNG CHUNG School of Computer Scence The Unversty of Oklhom Normn Oklhom E GUENWAD School of Computer Scence The Unversty of Oklhom Normn Oklhom THEODOE B. TAFAIS School of Industrl Engneerng The Unversty of Oklhom Normn Oklhom ABSTACT Ths pper pples the use of support vector clusterng SVC n the domn of web usge mnng. In ths method the dt ponts re trnsformed to hgh dmensonl spce clled the feture spce where support vectors re used to defne smllest sphere enclosng the dt. A soft-mrgn constnt s used to hndle outlers. The pper then performs eperments to compre SVC nd the K-Mens lgorthm usng web server log obtned from rel lfe eductonl web ste. The epermentl results show tht SVC provdes better user sesson clusters thn the K-Mens lgorthm n term of ntr-cluster sctter mesure but perform much worst n term of tme. SVC does not requre the number of clusters to be determned pror nd t cn hndle outlers nd ny shpe of clusters. INTODUCTION Web dt cn be used to understnd customers browsng behvors nd to gn strong compettve dvntge n e-busness. Web usge mnng nlyzes browsng ptterns from Web log dt n order to group users hvng smlr browsng ptterns. The dscovered knowledge s crtcl for e-commerce busnesses n order to derve better busness ntellgence. Web Usge Mnng nlyzes Web log usge ptterns to cluster users of smlr nterest. After the users clusters re dentfed new vstor wth lmted nvgton hstory cn be ctegorzed nto one of the clusters. By knowng the new user s predcted nterest mrketer cn send the rght promoton nformton to the new user. Ben-Hur et l. 1 presents novel method usng the support vector mchnes pproch for clusterng clled Support Vector Clusterng SVC. SVC does not hve bs of the number or the shpe of clusters nd t cn hndle nose or outlers. The obectve of ths pper s to present comprson study of SVC nd K- mens for Web usge dt mnng usng rel-lfe Web logs. We chose K-mens for comprson becuse t s wdely used lgorthm for clusterng. Shhb et l nd

2 Mobsher et l. used the technque to perform clusterng n Web usge mnng. Secton descrbes SVC nd K-mens. Secton 3 nd 4 present the comprson studes nd epermentl results respectvely. Secton 5 gves the conclusons nd future reserch. SUPPOT VECTO CUSTEING Support Vector Clusterng Ben-Hur et l. 1 s non-prmetrc clusterng lgorthm bsed on the support vector mchne pproch Vpnk A Gussn kernel functon s used to mp the dt ponts to hgh dmensonl spce clled the feture spce. The obectve s to serch for the mnml enclosng sphere. The mppng of the sphere bck to the dt spce gves rse to the formton of clusters. Incresng the wdth prmeter of the Gussn kernel wll ncrese the number of clusters. A soft mrgn constnt s used to hndle outlers by llowng the sphere n the feture spce not to enclose ll ponts. In the frst stge of the Support Vector Clusterng the sphere wth the mnml rdus whch encloses the dt ponts n feture spce s computed. In the second stge cluster ssgnment bsed on geometrc pproch s used. If pr of dt ponts belongs to dfferent clusters then ny lne tht connects them must pss beyond the mnml enclosng sphere n the feture spce. In other words there ests pont y on the lne wth the rdus lrger thn the rdus of the sphere n the feture spce sphere. Net we epln the bsc des of SVC formulton s descrbed n Ben- Hur et l. 1. et be dt set of N ponts wth { } 1 N X d I the nput spce. We serch for the smllest sphere enclosng the dt of rdus fter pplyng the nonlner trnsformton from the dt spce X to the feture spce. The optmzton problem cn be wrtten s follows: Mn s.t. 1.. N + The constrnts of the optmzton problem re s follows: + 1 where s the center of the sphere nd re slck vrbles tht re used to hndle outlers. The grngn s formulted s follows to solve the bove problem: where C

3 3 nd re grnge multplers. C s constnt nd s penlty term. Dfferenttng wth respect to nd leds to 1 3 C C 4 5 We cn then wrte the Krush-Kuhn-Tucker complementry condtons s follows: We turn the grngn nto the dul form by elmntng nd wth the substtuton of Eq. 3 4 nd 5 : W. 8 wth the constrnts C Mmzng the dul form s the SV problem tht we re solvng. As n the Support Vector Mchne method the dot products cn be replced by Mercer kernel K. In ths pper we use the Gussn kernel q e K where q s the wdth prmeter. The dul W cn be rewrtten fter replcng the dot products wth the Gussn kernel s follows: K K W 9 Ths problem s equvlent to the stndrd SV optmzton problem when the Gussn kernel s used. The dstnce squre of rdus of ech pont s feture spce mge from the center of the sphere cn be defned s: 1

4 4 Substtutng the center of the sphere wth equton 5 nd the kernel the bove cn be rewrtten s follows: 11 K K + K The sphere n the feture spce hs the rdus of s defned below. { s support vector} sphere 1 The verge of rdus of ll support vectors cn be used s the rdus of the sphere. The contour tht encloses the ponts n the dt spce s the set { } sphere Prmeters q nd C re used to control the shpe of the enclosng contours n the nput spce. These prmeters ffect the number of support vectors. For fed q s C s decresed the number of SVs decreses snce some of them turn nto bounded SVs nd the resultng shpes of the contours become smoother. Bounded SV s dt pont tht hs the rdus clculted usng Eq. 11 lrger thn sphere. After the support vectors re dentfed the net step s to lbel/defne the clusters. The clusters re defned s the connected components of the grph nduced by A. A s n dcency mtr where A between prs of ponts nd : A 1 f for ll y on the lne segment connectng nd y sphere 13 otherwse Ths lbelng method cn be used becuse the lne segment connectng ponts n dfferent components contn ponts outsde the sphere wheres the lne connectng close neghbors n the sme component les nsde the sphere. COMPAISON STUDIES The server logs re obtned from centrlzed eductonl lernng Web ste of OU Helth Scence Center Three sets of user sessons re studed. The frst eperment s run usng the frst 5 user sessons nd the second eperment s run usng the net 5 user sessons. The thrd eperment s run usng the user sessons. Snce K-Mens requres users to provde the K prmeter tht s the number of clusters n ntr-cluster sctter mesure s used to determne K. We mesure the ntr-cluster sctter for K strtng from to 13 nd choose the K tht gves the lest sctter. Intr-cluster sctter mesure s used to mesure the qulty of the clusterng results. The defnton of ntr-cluster sctter s defned s follows: J e C m 1 G where C s the number of clusters mens dt pont s n cluster m s the center of G defned s G

5 5 m 1 n for 1.. C. n n s the number of dt ponts n cluster. 1 Intr-cluster sctter mesures the compctness of the clusters. The smller the J e s the more compct the clusters re. Usng the heurstcs proposed n Cooley 1999 we defne unque user bsed on IP ddress. Bsed on Mobsher et. l.1996 the ndvdul log entres re grouped nto user sessons. A user sesson s sequence of ccesses by user. The durton of elpse tme between ny two consecutve ccesses n the sesson s lmted to specfed threshold 3 mnutes. The fetures used re the vld url pges of the ste. A unque number s ssgned to ech U { 1 }...m where m s the totl number of Us. To fcltte vrous dt mnng tsks the th user sesson s formulted s n m-dmensonl bnry ttrbute vector s wth the property s 1 f the user ccessed the th U durng the th sesson otherwse EXPEIMENTA ESUTS From the three epermentl studes we found tht SVC provdes better clusterng results thn K-Mens lgorthm. For the frst eperment K-Mens yelds the lest sctter when K 8. For SVC the ntr-cluster sctter mesure s zero when q s ncresed to 3 nd C 1 snce t gves the perfect prtton. SVC gves dentcl users sessons n ech cluster when q s 3. The clusterng results re better thn K-Mens even when q s lower thn 3. When q 1 the ntr-cluster sctter mesure s lower thn the best result of K- Mens by 53 %. In other words the clusters gven by SVC re 53% more compct thn tht of the K-Mens. When q SVC clusterng result s lmost 7% more compct thn K-Mens. Tble 1: The vlues of Intr-Cluster Sctter mesures for K-Mens nd SVC usng the frst dt set. Frst 5 Users Clusterng lgorthm Intr-Cluster Sctter K-Mens K SVC q3 C1 In the second eperment every dt pont becomes sngle cluster when q s ncresed from 1 to wth ntr-cluster sctter vlue of C hs to be decresed n order to further refne the clusterng soluton. When C s decresed from 1 SVC strts to detect outlers nd perform clusterng wthout consderng the outlers. Ths mproves the effectveness of the clusterng lgorthm. The ntr-cluster sctter for SVC n the second eperment s shown n Tble. The best result from SVC s when q 1 nd C.5. In ths cse there re 4 outlers detected. K-Mens gves the ntr-cluster sctter vlue of 5.14 wth

6 6 K 1. Clusters provded by SVC re bout 68% more compct thn tht of the K-Mens. Tble : The vlues of Intr-Cluster Sctter mesures for K-Mens nd SVC usng the second dt set. Second 5 Users Clusterng lgorthm Intr-Cluster Sctter K-Mens K SVC q1 C In the thrd eperment SVC gves the best ntr-cluster sctter of 6.55 when q 1.3 nd C 1 s shown n Tble 3 below. In ths cse there re clusters found. However only two clusters hve crdnlty lrger thn. K- Mens gves the best ntr-cluster sctter of when K 18. The clusters found usng SVC re bout 8% more compct thn tht of the K-Mens. Tble 3: The vlues of Intr-Cluster Sctter mesures for K-Mens nd SVC usng the thrd dt set. Users Clusterng lgorthm Intr-Cluster Sctter K-Mens K SVC q1.3 C We lso mesure the runnng tme of both lgorthms. In the frst eperment SVC requres 1 mnutes to cluster 5 user sessons compred to 5 seconds s requred by K-Mens. When the number of user sessons ncreses to SVC tkes bout 1 mnutes compred to bout 1 mnute s requred by K-Mens. Tble 4: The runnng tme of K-Mens nd SVC wth vryng number of users Number of Users unnng tme for K-Mens second unnng tme for SVC second CONCUSIONS AND FUTUE ESEACH The comprson shows tht SVC provdes better clusters n term of ntrcluster sctter especlly when the dt set s smll but perform much worst n term of tme when the number of users sessons ncrese. On verge SVC gves smller ntr-cluster sctter mesure compred to K-Mens s shown n the bove epermentl results. SVC provdes more compct clusters compred to K-Mens. The clusterng results cn lso be mproved when outlers re hndled n SVC. It hs been shown n Ben-Hur et l. 1 SVC cn hndle clusters wth rbtrry shpe nd hndle nosy dt. SVC s blty to hndle outlers mproves the clusterng results. However more effcent wy to determne the optml prmeters q nd C for SVC hs to be derved. In ddton sclble SVC s lso desred to hndle lrge dt sets.

7 7 EFEENCES Ben-Hur et l. 1 A. Ben-Hur D. Horn H.T. Segelmnn nd V.Vpnk. Support Vector Clusterng Journl of Mchne ernng eserch pp December 1 Cooley et l Cooley B. Mobsher nd J. Srvstv Dt preprton for Mnng World Wde Web Browsng Ptterns Knowledge nd Informton Systems Fu et l Y. Fu K. Sndhu nd M. Shh Clusterng of Web Users Bsed on Access Ptterns Interntonl Workshop on Web Usge Anlyss nd User Proflng WEBKDD'99 Sn Dego CA Shhb et l C. Shhb A.M. Shkesh J. Adb V. Shh Knowledge Dscovery from Users Web-Pge Nvgton In Proceedngs of the IEEE IDE97 Workshop Aprl Mobsher et l B. Mobsher N. Jn E-H. Hn nd J. Srvstv Web Mnng: Pttern dscovery from world wde web trnsctons Techncl eport 96-5 Unversty of Mnnesot Sept 1996 Vpnk 1995 Vldmr N. Vpnk. The Nture of Sttstcl ernng Theory. Sprnger N.Y Mobsher et l. B. Mobsher. Cooley nd J. Srvstv. Automtc personlzton bsed on Web usge mnng In Communctons of the ACM 43 8 Aug. Jn nd Dubes A. K. Jn nd. C. Dubes. Algorthms for clusterng Dt. Prentce Hll 1988.