Using Markov Model and Popularity and Similarity-based Page Rank Algorithm for Web Page Access Prediction

Transcription

1 Iteratioal Coferece o Advaces i Egieerig ad Techology (ICAET'2014 March 29-30, 2014 Sigapore Usig Markov Model ad Popularity ad Similarity-based Page Rak Algorithm for Web Page Access Predictio Phyu Thwe Abstract With the rapid growth of the World Wide Web, the Iteret has become a huge repository of data ad serves as a importat platform for the dissemiatio of iformatio. The users accesses to Web sites are stored i Web server log file. These series ca be cosidered as a web access patter which is helpful to fid out the user behaviour. Through this behaviour iformatio, we ca fid out the accurate user ext request predictio that ca reduce the browsig time of web pages. I this paper, we proposed to use clusterig techiques to cluster the data sets from the results of the preprocessig phase. As a result, a more accurate Markov model is built based o each group rather tha the whole data sets. The accuracy of low order Markov model is ormally ot satisfactory. Therefore, we use popularity ad similarity-based page rak algorithm to make predictio whe the ambiguous results are foud. Keywords Page Rak Algorithm, Web Log Miig, Web Page Predictio, Web Usage Miig. I. INTRODUCTION HE study of web miig The study of Web miig Ttechiques to discover useful kowledge has become icreasigly importat because more orgaizatios rely o the Iteret to coduct daily busiess. However, with the magitude ad diversity of available iformatio from the Iteret, it is importat to locate the relevat iformatio to satisfy the requiremets of people with differet backgrouds. To assist Web surfers i browsig the Iteret more efficietly, oe of the topics that have attracted much attetio is modellig the Web user s browsig patters ad makig predictio. May researchers have applied data miig to extract user avigatio patters from Web logs. The discovered patters ca be used to predict which pages are likely to be clicked by a user give a sequece of pages that the user already visited. With the predictio, the server ca automatically sed the predicted pages to the cliet cache before the pages are actually requested to reduce the etwork latecy. Alteratively, the predicted pages ca be recommeded to the user by dyamically geeratig the liks to the pages o the curret browsig widow of the user to help the user fid relevat iformatio more efficietly. The techiques that have bee Phyu Thwe is with the Uiversity of Techology (Yataarpo Cyber City, Pyi Oo Lwi, Myamar ( pthwe19@gmail.com. used to fid such patters iclude associatio rule miig, sequetial patter miig ad clusterig. The rest of the paper is orgaized as follows: We overview some related work i Sectio 2. We preset the required theoretical backgroud cocerig the Markov Models ad the Page Rak Algorithm i Sectio 3. We provide the architecture of the proposed system i which PSPR ca be applied i Sectio 4. We prove that this PSPR algorithm ca be applied to ay web site s log data. Fially, we coclude with our system i Sectio 5. II. RELATED WORK I [4], users browsig behavior will be predicted at two levels to meet the ature of the avigatio. Oe is category stage ad the other is web page stage. I stage oe is to predict category. The uecessary categories ca be excluded. The scope of calculatio is massively reduced. A dyamic clusterig-based method is used to icrease a Markov model's accuracy i represetig a collectio of user web avigatio sessios [5]. The method makes use of the state cloig cocept to duplicate states i a way that separates i-liks whose correspodig secod-order probabilities diverge. I [6], a rough set clusterig is preseted to cluster web trasactios from web access logs ad Markov model is used for ext access predictio. Usig this approach, users ca effectively mie web log records to discover ad predict access patters. I [7], this paper ivestigates ito usig Markov chais to make lik predictio ad the trasitio matrix derived from Markov chais to acquire structural kowledge about Web sites. I this paper, a method for predictig the rakig positio of a Web page is proposed [8]. Assumig a set of successive past top-k rakigs, the evolutio of Web pages i terms of rakig tred sequeces used for Markov Models traiig is studied, which are i tur used to predict future rakigs. A hybrid probabilistic predictive model extedig the properties of Markov models by icorporatig lik aalysis methods is preseted i [9]. More specifically, the use of a Page Rak-style algorithm is proposed for assigig prior probabilities to the web pages based o their importace i the web site's graph. I [11] Usage Based Page Rak (UPR is itroduced. UPR is a variatio of the Page Rak algorithm, based o the visit 197

2 Iteratioal Coferece o Advaces i Egieerig ad Techology (ICAET'2014 March 29-30, 2014 Sigapore frequecy data obtaied from previous users' sessios. I [12] Access time-legth ad frequecy-based Page Rak (TFPR is itroduced ad is based o the legth of time spet o visitig a page ad the frequecy that a page was visited. I [13] Duratio based Page Rak (DPR ad Popularity based Page Rak (PPR is itroduced. Duratio Based Rak (DPR, which focuses o page duratio with size proportio. Popularity Based Page Rak (PPR rakig model, which focuses o both page duratio with size proportio ad frequecy value of page visits. III. BACKGROUND THEORY A. Markov Model The 1 st -order Markov models (Markov Chais provide a simple way to capture sequetial depedece [2, 3, 9], but they do ot take ito cosideratio the log-term memory aspects of web surfig behaviour sice they are based o the assumptio that the ext state to be visited is oly a fuctio of the curret oe. Higher-order Markov models are more accurate for predictig avigatioal paths. But, there exists a trade-off betwee improved coverage ad expoetial icrease i state space complexity as the order icreases. Moreover, such complex models ofte require iordiate amouts of traiig data, ad the icrease i the umber of states may eve have worse predictio accuracy ad ca sigificatly limit their applicability for applicatios requirig fast predictios, such as web persoalizatio. There have also bee proposed some mixture models that combie Markov models of differet orders. However, such models require much more resources i terms of preprocessig ad traiig. Therefore, it is evidet that the fial choice that should be made cocerig the kid of model that is to be used, depeds o the trade-off betwee the required predictio accuracy ad model's complexity/size. B. Page Rak Algorithm Page Rak is used to determie the importace of the page o the web. Surgey Bri ad Larry Page [1] proposed a rakig algorithm amed Page Rak (PR that uses the lik structure of the web to determie the importace of web pages. Accordig to this algorithm, if a page has importat liks to it, the its liks to other pages also become importat. Therefore, it takes back liks ito accout ad propagates the rakig through liks. I Page Rak, the rak score of a page is equally divided amog its outgoig liks ad that values of outgoig liks are i tur used to calculate the raks of pages poited by that page. Page Rak [11] is the most popular lik aalysis algorithm, used broadly for assigig umerical weightigs to web documets ad utilized from web search egies i order to rak the retrieved results. The algorithm models the behavior of a radom surfer, who either chooses a outgoig lik from the page he s curretly at, or jumps to a radom page after a few clicks.the Web is treated as a directed graph G = (V, E, where V is the set of vertices or odes, i.e., the set of all pages, ad E is the set of directed edges i the graph, i.e., hyperliks. I page rak calculatio, especially for larger systems, iterative calculatio method is used. I this method, the calculatio is implemeted with cycles. I the first cycle all rak values may be assiged to a costat value such as 1, ad with each iteratio of calculatio, the rak value become ormalized withi approximately 50 iteratios uder ε = IV. SYSTEM ARCHITECTURE The processig steps of the system have three mai phases. Preprocessig is performed i the first phase. The secod phase is clusterig web sessios usig K-meas clusterig. I the fial phase, Markov model is used to predict ext page access based o resultig web sessios. The popularity ad similarity-based page rak algorithm is used to decide the most relevat aswer if the ambiguous result is foud i Markov model predictio. The iput of the proposed system is a web log file. A web log is a file to which the web server writes iformatio each time a user requests a resource from that particular site. Fig. 1 Processig step of the proposed system A. Data Preprocessig The data preprocessig icludes three basic steps [10]. They are: Data Cleaig User Idetificatio Sessio Idetificatio 198

3 Iteratioal Coferece o Advaces i Egieerig ad Techology (ICAET'2014 March 29-30, 2014 Sigapore The first issue i the preprocessig phase is data cleaig. Web log data may eed to be cleaed from etries ivolvig pages that retured a error or graphics file accesses. I some cases such iformatio might be useful, but i others such data should be elimiated from a log file. User idetificatio meas idetifyig each user accessig Web site, whose goal is to mie every user s access characteristic. There are several ways to idetify idividual visitors. The most obvious solutio is to assume that each IP address (or each IP address/cliet aget pair idetifies a sigle visitor. The ext step is to perform sessio idetificatio, by dividig the click stream of each user ito sessios. The usual solutio i this case is to set a miimum timeout ad assume that cosecutive accesses withi it belog to the same sessio, or set a maximum timeout, where two cosecutive accesses that exceed it belog to differet sessios. B. Popularity ad Similarity-Based Page Rak Algorithm Popularity ad Similarity-based Page Rak (PSPR calculatio simply depeds o the duratio values of pages ad trasitios, the frequecy value of pages ad trasitios, their web page file size ad similarity of web page [15]. The popularity value of page rak was discussed i [13]. Popularity defies i two dimesios. They are page dimesio ad trasitio dimesio. For both dimesios, popularity defies i terms of time user speds o page, size of page ad visit frequecy of page. Page popularity is eeded for calculatig radom surfer jumpig behaviour of the user ad trasitio popularity is eeded for calculatig the ormal avigatig behaviour of the user. Similarity of web page is importat to predict ext page access because millio of users geerally access the similar web page i a particular Web site. The calculatio of the similarity is based o web page URL. The cotet of pages is ot cosidered ad this calculatio does ot eed for makig a tree structure of the Web site. For example, suppose /shuttle/missios/sts-73/missio-sts-73.html ad /shuttle/missios/sts-71/missio-sts-71.html are two requested pages i web log. These two URLs are stored i strig array by dividig "/" character. Ad the, we compute the legth of the two arrays ad give weight to the loger array: the last room of the array is give weight 1, the secod to the last room of the array is give weight 2, the third to give weight 3 ad so o ad so forth, util the first room of the array is give higher legth of the array. The similarity betwee two strigs is defied as the sum of the weight of those matchig substrigs divided by the sum of the total weights. This similarity measuremet icludes: (1 0 <= SURL <= 1, i.e. the similarity of ay pair of web pages is betwee 0.0 ad 1.0; (2 SURL = 0, whe the two web pages are totally differet; (3 SURL = 1, whe the two web pages are exactly same. PSPRi = ε w j PSPRj wj k max P I( P SURL j i SURLj k wi i + (1 ε w max / s WS j m m i / s m I the equatio 1, ε is a dampig factor ad usually ε = I(p i is the set that keeps the i-liks of that page. Out(p j is the set of pages that poit to p j. w is the umber of times pages j ad i appear cosecutively i all user sessios. d is the duratio of the trasactio ad s i is the size of the trasitio's result page. WS is the web sessio. SURL is the similarity of web page j to page i. w is the trasitio popularity w max / s j k m based o trasitio frequecy ad duratio. SURL is the similarity calculatio betwee SURL j k web pages. wi is the frequecy calculatio for page i. P wj j WS i is the average duratio calculatio for page i. maxm / sm The popularity of page is calculated based o page frequecy ad average duratio of page. By usig this equatio, we ca calculate the popularity ad similarity-based page rak (PSPR for every page. I order to make rak calculatios faster, we record required steps of our calculatios to database. The step values related to rak calculatios are, average duratio value of pages, average duratio values of trasitios, page size, frequecy value of pages, frequecy value of trasitios, the similarity value of pages. The result ca be used for ambiguous result foud i Markov model to make the correct decisio. V. EXPERIMENTAL EVALUATION This paper itroduces a method that itegrates clusterig, Markov model ad page rak algorithm i order to improve the Web page predictio accuracy. I this sectio, we preset experimetal results to evaluate the performace of our algorithm. Overall our experimet has verified the effectiveess of our proposed techiques i web page access predictio based o a particular website. For our experimets, we used NASA web server data sets. We obtaied the web logs i August, 1995 ad used the web logs from 01/Aug/1995 to 15/Aug/1995 as the traiig data (

4 Iteratioal Coferece o Advaces i Egieerig ad Techology (ICAET'2014 March 29-30, 2014 Sigapore set. For the first testig data set, the web logs from 16/Aug/1995 to 17/Aug/1995 are used. We filtered the records (such as *.jpg, *.gif, *.jpeg ad oly reserved the hits requestig web pages. Whe idetifyig user sessios, we set the sessio timeout to 30 miutes, with a miimum of 10 pageviews per sessio. After filterig out the web sessio data by preprocessig, the traiig data set cotaied records ad 5574 sessios, while the testig data set cotaied 1271 records ad 74 sessios. For the secod testig data set, we used the web logs from 16/Aug/1995 to 22/Aug/1995. After filterig out the web sessio data, the testig data set cotaied records ad 2482 sessios. I comparig the predictios with the real page visits, we use two similarity algorithms that are commoly preferred for fidig similarities of two sets. The first oe is called Osim [11, 12, 13] algorithm, which calculates the similarity of two sets without cosiderig the orderig of the elemets i the two sets betwee A ad B ad is defied as: A B OSim( A, B = (2 As the secod similarity metric we use Ksim similarity algorithm, which cocers Kedall Tau Distace [11, 12, 13] for measurig the similarity of ext page predictio set produced by traiig data set ad real page visit set o the test data. Kedall Tau Distace is the umber of pairwise icompatibility betwee two sets. ( u, v : r1 ', r2 ' havesameorderigof ( u, v, u v KSim( r1, r2 = (3 A B ( A B 1 Where, r 1 ' is a extesio of r 1, cotaiig all elemets icluded i r 2 but ot r 1 at the ed of the list (r 2 ' is defied aalogously. I our experimet setup, we make experimet with top-3 compariso that are measured by Ksim ad Osim methods. A. Experimetal Results The results of the experimet for the ext page predictio accuracy for popularity ad similarity-based page rakig algorithm uder Ksim ad Osim similarity metrics are give i Figure st order 2d order OSim KSim Fig. 2 Average OSim, KSim values for top-3 predictio outperforms 48.24% ad 59.25% for OSim ad KSim respectively, while PSPR based o 1st order Markov model outperforms 38.68% ad 55.92% respectively for the first testig data set. Therefore, we ca cofirm that popularity ad similarity-based page rak deped o 2d order Markov model ca improve the accuracy of Web page predictio st order 2d order OSim KSim Fig. 3 Average OSim, KSim values for top-3 predictio As depicted i Figure 3, PSPR based o 2d order Markov model outperforms PSPR based o 1st order Markov model sigificatly i all OSim ad KSim values. I the top-3 predictio, PSPR based o 2d order Markov model outperforms 48.89% ad 62.13% for OSim ad KSim respectively, while PSPR based o 1st order Markov model outperforms 42.25% ad 58.54% respectively for the secod testig data set. Therefore, we ca cofirm that popularity ad similarity-based page rak deped o 2d order Markov model ca improve the accuracy of Web page predictio. Web page access predictio ca be useful i may applicatios [14]. The improvemet for accuracy ca make a chage i the web advertisemet area. Usig web page access predictio, the right advertisemet will be added accordig to the users' browsig patters. Also, web page access predictio helps web admiistrators restructure the Web sites to improve site topology ad user persoalizatio as well as market segmetatio. Web page access predictio is also helpful for cachig the predicted page for faster access ad for improvig browsig ad avigatio orders. VI. CONCLUSION I this paper, we used Markov model ad popularity ad similarity-based page rak algorithm for web page access predictio. I our experimet, we observed that i both cases PSPR based o 2d order Markov Model are more tha promisig PSPR based o 1st order Markov Model i terms of accuracy (OSim ad KSim. Higher order Markov model result i better predictio accuracy sice they look at previous browsig history. But they are associated with higher state space complexity. As depicted i Figure 2, PSPR based o 2d order Markov model outperforms PSPR based o 1st order Markov model sigificatly i all OSim ad KSim values. I the top-3 predictio, PSPR based o 2d order markov model 200

5 Iteratioal Coferece o Advaces i Egieerig ad Techology (ICAET'2014 March 29-30, 2014 Sigapore REFERENCES [1] S. Bri, L. Page, The aatomy of a large-scale hypertextual Web search egie, Computer Networks, 30(1-7: , Proc. of WWW7 Coferece. [2] F.Khalil, J. Li ad H. Wag, Itegratig markov model with clusterig for predictig web page accesses. Proceedigs of the 13th Australasia World Wide Web Coferece (AusWeb 2007, Jue 30- July 4, Coffs Harbor, Australia, pp: [3] M. Deshpade ad G. Karypis. May Selective markov models for predictig web page accesses. ACM Tras. Iteret Techol., 4: [4] V.V.R.Maheswara Rao, Dr. V. Valli Kumari, "A Efficiet Hybrid Successive Markov Model for Predictig Web User Usage Behavior usig Web Usage Miig", Iteratioal Joural of Data Egieerig (IJDE Volume (1: Issue (5, pp [5] J. Borges, M. Levee, "A Dyamic Clusterig-Based Markov Model for Web Usage Miig", May 26, 2004 [6] S. Chimphlee, N. Salim, M. S. B. Ngadima, W. Chimphlee, S. Srioy, "Rough Sets Clusterig ad Markov model for Web Access Predictio", Proceedigs of the Postgraduate Aual Research Semiar 2006 [7] J. Zhu, "Usig Markov Chais for Structural Lik Predictio i Adaptive Web Sites" [8] M. Vazirgiais, D. Drosos, P. Seellart, A. Vlachou, "Web Page Rak Predictio with Markov Models", April 21-25, 2008 Beijig, Chia [9] M. Eiriaki, M. Vazirgiais, D. Kapogiais, Web Path Recommedatios based o Page Rakig ad Markov Models, WIDM 05, November 5, 2005, Breme, Germay [10] R. Khachaa, Dr. M. Puithavalli, Web Page Predictio for Web Persoalizatio:A Review, Global Joural of Computer Sciece ad Techology Volume 11 Issue 7 Versio 1.0 May 2011, ISSN: & Prit ISSN: [11] M. Eiriaki ad M. Vazirgiais. Nov Usage-based pagerak for web persoalizatio. I Data Miig, Fifth IEEE Iteratioal Coferece o, page 8 pp. [12] Y. Z. Guo, K. Ramamohaarao, ad L. Park. Nov Persoalized pagerak for web page predictio based o access time-legth ad frequecy. I Web Itelligece, IEEE/WIC/ACM Iteratioal Coferece o, pages [13] B. D. Guel, P. Sekul, Ivestigatig the Effect of Duratio, Page Size ad Frequecy o Next Page Recommedatio with Page Rak Algorithm, ACM. [14] F. Khalil, J. Li ad H. Wag, "Itegratig Recommedatio Models for Improved Web Page Predictio Accuracy", the Thirty-First Australasia Computer Sciece Coferece (ACSC2008, Wollogog, Australia. [15] P. Thwe, "Proposed Approach for Web Page Access Predictio Usig Popularity ad Similarity based Page Rak Algorithm", Iteratioal Joural of Sciece ad Techology Research, Volume 2 - Issue 3, March 2013 Editio [ ISSN ]