Comparative Study of Universities Web Structure Mining

Transcription

1 Comprtive Study of Universities Web Structure Mining Z. Abdullh, A. R. Hmdn Abstrct This pper is ment to nlyze the rnking of University of Mlysi Terenggnu, UMT s website in the World Wide Web. There re only few reserches hve been done on compring the rnking of universities websites so this reserch will be ble to determine whether the existing UMT s website is serving its purpose which is to introduce UMT to the world. The rnking is bsed on hub nd uthority vlues which re ccordnce to the structure of the website. These vlues re computed using two webserching lgorithms, HITS nd SALSA. Three other universities websites re used s the benchmrks which re UM, Hrvrd nd Stnford. The result is clerly showing tht more work hs to be done on the existing UMT s website where importnt pges ccording to the benchmrks, do not exist in UMT s pges. The rnking of UMT s website will ct s guideline for the web-developer to develop more efficient website. I Keywords Algorithm, rnking, website, web structure mining. I. INTRODUCTION N the world we re living in tody, informtion technology plys very crucil role where ny informtion desired is vilble just t the fingertips. With the growth of World Wide Web (WWW) technology, cn be seen tht existing web dt covers n enormous spectrum of dt from the government dt, entertinment, commercil till the extend of reserch dt. However, the vst size of dt does not gurntee tht users will be ble to cquire wnted dt esily. In relity, there is n opinion [1] sying tht 99% dt on the WWW is useless to 99% of web browsers. In wy, by looking t the vrious number of websites, this does prove tht somewhere in the tngled mne of websites, exist some precious useful informtion but this informtion is impossible to be recognized by users solely by depending on intuition so this is where web mining comes into picture. Web mining is defined s converging reserch re from severl reserch communities, such s dtbse, informtion retrievl, mchine lerning nd nturl lnguge processing [2]-[5]. Web mining is the use of dt mining techniques to utomticlly dis-cover nd extrct informtion from Web documents nd services. The objects of web mining re web resources, which cn be web documents, web log files, structure of web sites nd structure of web documents themselves. Web mining cn be ctegorized into three res of interest bsed on which prt of the web to mine. These Z. Abdullh is with the University of Mlysi Terenggnu, 21030, Kul Terenggnu, Mlysi (corresponding uthor. phone: ; fx: ; e-mil: zilni@ umt.edu.my). A. R. Hmdn is with the University of Kebngsn Mlysi, Bngi, Selngor, Mlysi (e-mil: rh@ftsm.ukm.my). ctegories [3]-[5] re Web usge mining, Web content mining nd Web structure mining. This reserch focuses on Web structure mining which cn be regrded s the process of discovering structure informtion from the Web [3]-[5]. This type of mining cn be further divided into two kinds bsed on the kind of structurl dt used. The first type is pttern extrction using hyperlinks. A Hyperlink is structurl unit tht connects Web pge to different loction, either within the sme Web pge or to different Web pge. A hyperlink tht connects to different prt of the sme pge is clled n Intr-Document Hyperlink, nd hyperlink tht connects two different pges is clled n Inter-Document Hyperlink. The second type is the Document Structure where the content within web pge cn lso be orgnized in tree-structured formt, bsed on the vrious HTML nd XML tgs within the pge. This reserch will focus on the ppliction of Web structure mining on Universiti Mlysi Terenggnu s (UMT) website. The uthority nd hub vlue of the website will be determined using web-serching lgorithms which re HITS nd SALSA. For HITS, Kleinberg [6] distinguished between two types of Web pges which pertin to certin topic which re hub nd uthority. Hubs re primrily resource lists, linking to mny uthorities on the topic possibly without directly contining the uthorittive informtion. According to this model, hubs nd uthorities exhibit mutully reinforcing reltionship: good hubs point to mny good uthorities, nd good uthorities re pointed t by mny good hubs. In light of the mutully reinforcing reltionship, hubs nd uthorities should form communities, which cn be pictured s dense biprtite portions of the Web, where the hubs link densely to the uthorities. SALSA is stochstic pproch developed by Lempel nd Morn [7] in which the coupling between hubs nd uthorities is less tight. The intuition behind our pproch is the following: consider biprtite grph G, whose two prts correspond to hubs nd uthorities, where n edge between hub r nd uthority s mens tht there is n informtive link from r to s. Then, uthorities nd hubs pertining to the dominnt topic of the pges in G should be highly visible (rechble) from mny pges in G. Thus, we will ttempt to identify these pges by exmining certin rndom wlks in G, under the proviso tht such rndom wlks will tend to visit these highly visible pges more frequently thn other, less connected pges. This reserch is very importnt becuse most of the website nowdys hs been developed ll under the will nd ide of the web developer oneself without ny references to how other 2237

2 developers developing website of the sme ctegory with higher rnking hd done theirs. This is just the sme scenrio with UMT s website, where by compring the serch result on the WWW, UMT is rnked 85th while UM is rnked 5th on the serch result list. By looking t the resulted rnking, it is obvious tht UMT s website could do with this reserch to mend the rnking of the university s website thus chieving the notion of introducing more of UMT to the world. Up to this dte, reserches were done to compre the used lgorithms nd the mens to improve the lgorithms; none were done to compre the resulted reding of the lgorithms [8]-[12]. Another problem is tht existing reserches were done rndomly on dt extrcted from the WWW, none of the dt used were specificlly focusing on evluting the structure of ny university s website [8]-[15]. Aim of this reserch is to understnd the web structure mining lgorithms which re HITS nd SALSA to enble the development of such lgorithms using softwre MATLAB 7.0. The lgorithms re the tools to determine the hub nd uthority vlues of studied websites. The redings produced by the lgorithms will drive the reserch where they will be used to evlute the web structure on UMT s website through comprison with three websites which re Universiti Mly (UM), Hrvrd nd Stnford. The scope of this reserch is to use UMT s the reserch cse nd three other universities website (UM, Hrvrd nd Stnford) s benchmrks. As well to develop HITS nd SALSA s lgorithms using MATLAB 7.0. The remining prt of the pper is orgnized s follows. Sections II present the relted works. Section III elbortes the methodology. Section IV reports the result nd discussion. II. RELATED WORKS For HITS nd SALSA, the obvious comprison is the effectiveness of SALSA in coming up with more useful informtion for query to be compred to HITS [5],[8]. It ws shown tht in serching for single query, the Mutul Reinforcement pproch of HITS rnked mny irrelevnt pges s uthorities wheres SALSA which is less vulnerble to the Mutul Reinforcement Approch produced better finding. However, since the implementtion were bsed on query, this spect of the lgorithms will only be used s guidelines since this reserch will be bsed on exctly universities web pges. Severl reserches hve even introduced some modifiction to the existing lgorithms [10], [11], [14], [15] where in ll the reserch, the modified version ws proven to be producing more detiled results. The modified versions were intended to be implement in fvor of the originl lgorithms but still no such scenrios re seen in ny prt of the WWW since the originl lgorithms re still the only pplied version of the lgorithms, so for this reserch the modified version of both lgorithms re totlly not being tken into ccount. Appliction of HITS nd SALSA re pprent on the WWW, where exmples of websites tht use these pplictions re nd These websites ll provide sles nd online purchsing services only. It is yet to be found n ppliction of HITS nd SALSA on university s website nd tht is wht this reserch will try to initite. Some reserches were done using dt bsed on lrge unctegorized group of websites [1], [5], [8], [10], [13], [14] where the studied websites re rndomly selected from lrge web grph which ws induced by millions of web crwlers. This kind of reserches ber no result regrding the web structure of the websites where they only produce the redings of the hub nd uthority without explntion of wht elements those contribute to the high hub nd uthority vlues. To combt the lck of explntion on the responsible elements tht produce high hub nd uthority vlue, this reserch will present the explntion in form of ctegories of the universities web pges. This reserch is importnt becuse in the end, the result will show the structure on UMT s website nd the comprison between the rnkings of the websites in determining on how UMT s website cn be improve in term on incresing the hub nd the uthority vlue on the WWW so the website will serve the rel function it ws built for, to introduce UMT to the world. III. METHODOLOGY In developing the HITS (Fig. 1) nd SALSA (Fig. 2) lgorithms, softwre MATLAB 7.0 is the most pproprite to be used since it provides simultneously ppliction of progrmming nd mthemtics. It lso llows the user to conjure up imges of the results in ll intended forms mtrix, grph, imges, simultion nd etc. For this reserch, the lgorithms, the mpping of web structure mtrices of the universities websites, nd the resulted hub nd uthority vlues were executed using this softwre. Fig. 1 HITS Formul Fig. 1 is the formul of HITS. The hub nd uthority vlues re obtined by multipliction of the web structure mtrix, L 2238

3 with its djcency mtrix, LT in two different sequences. For hub it is LLT nd for uthority it is LTL. All hub nd uthority scores re checked with the eigenvector before obtining the finl vlue to ensure the finl sum is equl to one (1) which if not, it shows tht the web structure mtrix used s L is incomplete or contin error. h i, j i, j = = k k ( ih, k ),( jh, k ) () ( k ),( k, j ) h, i h (b) 1 1 G ( ) ( ) deg i deg k h 1 1 G ( ) ( ) deg i deg k Fig. 2 SALSA Formul () SALSA hub (b) SALSA uthority Fig. 2 is the formul of SALSA. Both equtions explin bout certin pge k points to both pges i nd j, hence pge j is rechble from pge i by two steps which re retrcting long the link from pge k till pge i nd then following the link from pge k till pge j. Hub vlue will be the vlue of nonzero entry of the mtrix (G) divided by the sum of entries h in row of G whilst the uthority vlue will be the vlue of nonzero element of G divided by sum of entries in column of G. The methodology used for this reserch is build of fivelevel steps. The steps re: A. Problem Definition The min problem of this reserch is to cquire the hub nd uthority vlues of the studied universities websites in order to determine which ctegory of pges tht contributes to the high hub nd uthority vlues. B. Dt Selection For this reserch dt ws collected from four websites where UMT s the studied website nd the other comprison websites, UM, Hrvrd nd Stnford. C. Preprocessing of Dt Fig. 3 Web Structure Mtrix of UMT s Website URL for ech pge of selected websites ws mnully extrcted from WWW. The rw dt then ws mnully trnsformed into binry mtrix form (0 represent no hyperlink nd 1 represent existing hyperlinks between mpped pges) (Fig. 3). The web structure mining lgorithms, HITS nd SALSA were developed using MATLAB. Fig. 4 Exmple of Hub nd Authority vlue of UMT s Website 2239

4 D. Appliction of Dt The complete mpped mtrices were executed using the two lgorithms, HITS nd SALSA. The redings of the lgorithms (Fig. 4) re in the form of mtrix (rw dt which re still not meningful until being processed). The redings re vlues of hub nd uthority for ech website. E. Processing of Dt The rw dt from the reding re turned into meningful dt by rnking the top 15 pges of ech website. Grphs re produced to show the differences between cse study nd the comprison websites in the top 15 pges rnking ccording to lgorithms. The comprisons re recorded s the result of the reserch. The frmework used for this reserch is s illustrted in Fig. 5. Fig. 5 Overview Model of the Frmework IV. RESULT AND DISCUSSION The experiments were crried out on Intel CoreTMi5-321M CPU t 2.50 GHz speed with 4GB RAM, running on Window 7 Home Premium. All lgorithms hve been developed using Jv s progrmming lnguge nd NetBens IDE 8.0 with JDK s pltform. Tble I nd Fig. 6 shows tht for UMT s website, in term of vlue, the highest rnked pges fll under the Generl ctegory (HITS uthority, ) whilst in term of highest number of pges, most pges in top 100 pges re under Acdemics ctegory (46 pges). Web pge Ctegory TABLE I TOP 100 RANKED PAGES OF UMT S WEBSITE No of Pges HITS (hub) Weight Vlue of Algorithms HITS SALSA SALSA (uthority) (hub) (uthority) Administrti on Generl Acdemics News & Events Fcilities Athletics Arts Others Totl Fig. 6 Top 100 Rnked Pges of U MT s Website Tble II nd Fig. 7 shows for UM s website, in term of vlue, the highest rnked pges fll under the Generl ctegory (HITS uthority, ) whilst in term of highest number of pges, most pges in top 100 pges re under Acdemics ctegory (35 pges). TABLE II TOP 100 RANKED PAGES OF UM S WEBSITE Weight Vlue of Algorithms Web pge No of Ctegory Pges HITS HITS SALSA SALSA (hub) (uthority) (hub) (uthority) Administrtion Generl Acdemics News & Events Fcilities Athletics Arts Alumni Others Totl Fig. 7 Top 100 Rnked Pges of UM s Website Tble III nd Fig. 8 shows tht in term of vlue, the highest rnked pges fll under the Generl ctegory (SALSA hub, ) whilst in term of highest number of pges, most pges in top 100 pges re under Administrtion ctegory (33 pges). 2240

5 TABLE III TOP 100 RANKED PAGES OF HARVARD S WEBSITE Weight Vlue of Algorithms Web pge No of Ctegory Pges HITS HITS SALSA SALSA (hub) (uthority) (hub) (uthority) Administrtion Generl Acdemics News &Events Fcilities Athletics Arts Alumni Others Totl Fig. 8 Top 100 Rnked Pges of Hrvrd s Website Tble IV nd Fig. 9 shows tht in term of vlue, the highest rnked pges fll under the Acdemics ctegory (SALSA hub nd uthority, both ) whilst in term of highest number of pges, most pges in top 100 pges re under Acdemics ctegory (35 pges). Tble V nd Fig. 10 re summry of the top 100 rnked pges in term of weight vlue ccordingly by lgorithms: 1. For UMT s website, both HITS nd SALSA for hubs nd uthority vlues ssigned pges under Generl ctegory with the highest weight. 2. For UM s website, HITS for hubs nd uthority vlues lso SALSA for hub vlue ssigned the highest weight to pges under Generl ctegory whilst SALSA for uthority vlue ssigned the highest weight to pges under Acdemics ctegory. 3. For Hrvrd s website, ll HITS nd SALSA vlues for hubs nd uthority re ssigned to pges under Generl ctegory. 4. For Stnford s website, ll vlues of HITS nd SALSA, both hubs nd uthority re ssigned to pges under Acdemics ctegory. TABLE IV TOP 100 RANKED PAGES OF STANFORD S WEBSITE Weight Vlue of Algorithms Web pge No of Ctegory Pges HITS HITS SALSA SALSA (hub) (uthority) (hub) (uthority) Administrtion Generl Acdemics News & Events Fcilities Athletics Arts Alumni Others Totl Fig. 9 Top 100 rnked pges of Stnford s Website Fig. 10 Highest Rnked Pges of Ech University TABLE V HIGHEST RANKED PAGES OF EACH UNIVERSITY HITS (hub) HITS (uthority) SALSA (hub) SALSA (uthority) University Vlue Ctegory Vlue Ctegory Vlue Ctegory Vlue Ctegory UMT UM Hvrd Stnford Ctegory; 2: Generl, 3: Acdemics 2241

6 Tble VI shows the pges under which ctegory contribute most to the top 100 rnked pges of ech university. For UMT it is Acdemics (46 pges), UM is lso Acdemics (35 pges), Hrvrd is Administrtion (33 pges) nd Stnford is Acdemics (35 pges). TABLE VI TOP 100 RANKED PAGES ACCORDING TO NO. OF PAGE Web pge Ctegory University No. of Pges Administrtion 1 UMT 18 UM 8 Hrvrd 33 Stnford 12 Generl 2 UMT 22 UM 31 Hrvrd 22 Stnford 27 Acdemics 3 UMT 46 UM 35 Hrvrd 24 Stnford 35 News & Events 4 UMT 8 UM 8 Hrvrd - Stnford 2 Fcilities 5 UMT - UM 13 Hrvrd 8 Stnford 4 Athletics 6 UMT - UM - Hrvrd - Stnford 6 Arts 7 UMT - UM 2 Hrvrd 13 Stnford 3 Alumni 8 UMT - UM - Hrvrd - Stnford 7 Others 9 UMT 6 UM 3 Hrvrd - Stnford 4 The mjor difference produced by the comprison ccording to Tble VI, is tht the three comprison websites hve listed pges under two other ctegory of pges in their top 100 rked pges which is lcking in UMT s website. The ctegories re: 1. Fcilities (UM: 13 pges, Hrvrd: 8 pges, Stnford: 4 pges) 2. Arts (UM: 2 pges, Hrvrd: 13 pges, Stnford: 3 pges) The comprison shows tht UMT s website still needs to be touch-up nd mong the mnners tht cn be implement re: First, websites tht contin informtion which cn be regrded s dditionl informtion such s Fcilities nd Arts need to be emphsis becuse it is cler tht these ctegories contribute quite portion of the hub nd uthority vlue s pprent in the three comprison websites. Up till the time of this reserch is conducted, UMT s website hs no specific pges under these two ctegories yet. Second, for the highest rnked pges ccording to weight vlues, even though UMT s website hs ssigned pges under Generl ctegory with the highest vlues which is the sme cse for UM s nd Hrvrd s website, it is pprent tht the vlues produced by UMT s website by fr re the lowest ( Exmple: HITS hub vlues; UMT=0.1297, UM=0.3616, Hrvrd=0.6667). This shows tht even though the emphsis hs lredy been given to the right ctegory, there re still elements missing (such s hyperlink to other meningful pges or meningful informtion on the specific website itself) which resulted in low weight vlues. Third, UMT s website cn lso crete direct link from pge home to other pges which cn increse the hub vlue like wht hs been done by Hrvrd nd Stnford. The min pges of ech ctegory is linked to every sub-pge in tht exct ctegory nd lso linked to sub-pges of other ctegories. Fourth, s in cse of Stnford s website, the weight vlues for ll ctegories re very consistent nd the website listed pges of ctegory Athletics (HITS: , SALSA: ) nd Alumni (ll HITS nd SALSA: ) under the top 100 rnked pges of its website. Even though the other two comprison website hve no pges under these two ctegories in their top 100 rnked pges, UMT s website cn still focus on these two ctegories since without doubt, Stnford is mong the best well-known university in the world nd they hve given emphsis on this. V. CONCLUSION So fr, there re no reserch hve been done focusing on web structure mining of universities websites in compring wht re the essentil pges to ensure the vlue of hub nd uthority re high. This reserch is done by focusing on four universities websites, UMT s the cse study nd three other websites, UM, Hrvrd nd Stnford s the comprison to identify the importnt pges university s website should emphsize on. Two web structure lgorithms, HITS nd SALSA hve been developed. The methodology used ws simple steps where the dt structure of ech website is mnully mpped into mtrix to enble the lgorithms to run through the mtrix nd produce the top-rnked pges of ech university. The result shown tht UMT s website is bsolutely lcking in two min ctegories of pges which ws emphsized in the comprison websites which re Fcilities nd Arts lso two minor ctegories which re Athletics nd Alumni. This finding will enble the UMT s website developer to improve the structure of the website nd crete the importnt non-existing pges s wht hs been done by the comprison universities. ACKNOWLEDGMENT This work is supported by the reserch grnt from Reserch 2242

7 Accelertion Center Excellence (RACE) of Universiti Kebngsn Mlysi. REFERENCES [1] A. Arsu, J. Cho, H. Grci-Molin, A. Pepcke, nd S. Rghvn, Serching the Web, ACM Trnsctions on Internet Technology (TOIT), 1 (1), pp. 2-43, [2] B.J. Jnsen, A. Spink, C. Blkely, nd S. Koshmn, Defining Session on Web Serch Engines, Journl of the Americn Society for Informtion Science nd Technology, 58(6), pp , [3] J. Srivstv, P. Desikn, nd V. Kumr, Web Mining- Accomplishments & Future Directions, University of Minnesot [4] J. Fürnkrnz, Web Mining, Dt Mining nd Knowledge Discovery Hndbook, pp , Springer-Verlg, [5] M. Eirinki, Web Mining: A Rodmp, Technicl Report, DB-NET 2004, t [6] J. Kleinberg, Authorittive Sources in Hyperlinked Environment, Proceeding of the 9th ACM SIAM Symposium on Discrete Algorithms, pp , [7] M. Ln, Algorithms nd Applictions of Preference Bsed Rnking for Informtion Retrievl, Ph.D Thesis, [8] M. Njork, Compring the Effectiveness of HITS nd SALSA, Proceeding of 16th ACM Conference on Informtion nd Knowledge Mngement (CIKM), [9] R. Lempel, nd S. Morn, Rnk-Stbility nd Rnk-Similrity of Link- Bsed Web Rnking Algorithms in Authority-Connected Grphs, Informtion Retrievl, pp , [10] Y Dun, J Wng, M Km, J Cnny, Privcy preserving link nlysis on dynmic weighted grph, Computtionl & Mthemticl Orgniztion Theory, 11 (2), , [11] Z. Chen, L. To, J. Wng, L. Wenyin, nd W. M, A Unified Frmework for Web Link Anlysis, Proc. 3rd Interntionl Conference on Web Informtion Systems Engineering (WISE2002), Singpore (regulr pper), pp , Dec [12] A. Borodin, G. O. Roberts, J. S. Rosenthl, nd P. Tsprs, Finding Authorities nd Hubs from Link Structures on the World WideWeb, Proceedings of the 10th Interntionl World Wide Web Conference, pp , [13] A.N. Lngville, nd C.D. Meyer, A Survey of Eigenvector Methods for Web Informtion Retrievl, Journl SIAM review, 47(1), , [14] A. Frht, T. LoFro, J.C. Miller, G. Re, L.A. Wrd, Authority rnkings from HITS, PgeRnk, nd SALSA: Existence, uniqueness, nd effect of initiliztion, SIAM Journl on Scientific Computing, 27 (4), , [15] J.C. Miller, G. Re, nd F. Schefer, Modifictions of Kleinberg s HITS Algorithm Using Mtrix Exponentition nd Web Log Records, Proceedings of the 24th nnul interntionl ACM SIGIR conference on Reserch nd development in informtion retrievl, pp ,