Coupling Web Usage Mining with Content Mining for Recommendation Systems

Transcription

1 Couping Web Usage Mining with Content Mining for Recommendation Systems Rehab Duwairi Department of Computer Science and Engineering Qatar Uniersity Qatar Ebraheem Ehaddad Department of Computer Science, Jordan Uni. of Sc. & Tech. Jordan Mohammed A-Towaiq, Department of Math. & Stat, Jordan Uni. of Sc. & Tech Jordan ABSTRACT In this paper, we introduce a noe approach for combining web usage mining and content mining. The approach is based on generating recommendations that tae into consideration users' naigation history as we as contents of pages isited. The integration of usage and content data can be performed at the offine phase or at the onine phase of the recommendation system. The former is referred to as pre-mining integration; whie the atter is caed post-mining integration. The current wor is concerned with pre-mining integration. The main rationaes of the combination are to improe the quaity of recommendations generated; oercome the new item probem and to improe the custering of users naigation patterns. In our experiments, the quaity of custering of user sessions is improed by 35% when compared with quaity of custers generated using usage patterns ony. Aso, the recommendations generated using the proposed approach are improed by 4% and 8% for the reca and precision respectiey and when compared with reca and precision obtained by using usage data ony. Keywords: recommender systems, coaboratie fitering, web usage mining, web content mining, and web personaization.. INTRODUCTION Web personaization is any action done by the web sites' administrator through computer based techniques to customize the web site for specific user or users. The personaization can be represented as recommended ins, products or targeted adertisements appearing to the users [8]. There are at east three approaches for buiding web personaization and recommendation systems, content-based, coaboratie fitering and web usage mining []. The content-based approach tries to accuratey anticipate the next user request, based on the content anaysis of the recenty isited pages by the user. These predictions are based on cacuating the simiarity of the current isited pages and the user's profie or mode. Pages that refect the user's interests and had neer been isited by the user can then be recommended [3]. The coaboratie fitering approach, by comparison, is one of the most popuar web personaization and recommendation technoogies and it uses peer opinions to predict the interests of others [9]. Here, instead of using the web pages content itsef, the web pages' ratings are used to generate recommendations. One of the we nown coaboratie fitering techniques is the K-Nearest-Neighbor (KNN). Gien the target user transaction, KNN compares this transaction with a transaction history of other users in order to find the K most simiar users. The simiarity can be based on ratings of items, access to simiar pages, or purchase of simiar items []. Neighborhood items are then used in the recommendation for the target user. Coaboratie fitering oercomes some content based imitations. The recommendations of items for users are based on user expicit ratings, instead of the items' content. The adantage of this approach, when compared with pure content based approaches is that it can capture pragmatic reationships among items based on their intended use []. Howeer, coaboratie fitering suffers from some probems such as: The cod start probem: initiay, there wi not be enough ratings and it competey denies any information that can be extracted from the content [7]. Web usage mining is concerned with the appication of data mining toos to web ogs in order to discoer users usage patterns; this heps in understanding and better sering the needs of web-based appications []. Web usage mining is diided into offine and onine components. The data preparation and the data mining techniques are appied at the offine component whereas the onine component inoes the recommendations to the actie users; these recommendations are based on the information extracted from the offine component. Hybrid approaches of content-based and coaboratie fitering, content-based and web usage mining are aso addressed. The combination is done to improe the oera system performance by combining the adantages of each technique. A noe approach is introduced in this paper. It is basicay based on a combination between the usage-mining of serer og fie and content mining for the corresponding web pages. The proposed approach consists of two components, namey, the onine and offine components. The offine component consists of three major tass, data preparation and preprocessing, generating simiarity matrixes and custering. The onine component consists of two tass, cassification and recommendation set generation. This paper is organized as foows: section proides an oeriew of the proposed framewor. Section 3 describes the dataset that was used. Section 4, on the other hand, describes the preprocessing and transformations that were appied to the dataset to mae it suitabe for the wor at hand. Section 5 describes the process of generating the session simiarity matrices (in usage, content, and usage-content spaces). Section 6, by comparison, describes the framewor that was used to custer users' sessions based on their simiarity. Section 7, describes the process of assigning a new session to the most simiar custer to that session. Section 8 describes the recommendation process where a set of pages is recommended to the user. Finay, section draws the concusions of this wor.

2 . COUPLING WEB USAGE AND CONTENT MINING 4. PREPROCESSING The dataset, which was used in this wor, is spit into training and testing subsets. The training sessions are the input to the offine component and the testing sessions are the input to the onine component. At the offine component, seera processing tass were appied on the training sessions to extract users' patterns. These patterns are used, by the onine component, to produce the recommendation set. The recommendation set is a group of web pages (URLs) that is predicted for the user to isit within her/his current session. The dataset preprocessing, generating simiarity matrixes and custering tass are performed at the offine component, whereas cassification and recommendation tass are performed at the onine component. The dataset reformatting and irreeant information eimination are done at the preprocessing tas. We buid three matrices, namey, content, usage and usage-content simiarity matrices. The content simiarity matrix hods the simiarity between eery two sessions in content, whereas the usage simiarity matrix hods the simiarity in usage. The usage-content simiarity matrix is buit by adding eery eement in the usage simiarity matrix to its corresponding eement in the content simiarity matrix. The usage-content simiarity matrix is used in the custering tas. After that, we custer the users sessions to custers using the -means custering agorithm. A new session is cassified to the most appropriate custer at the cassification tas which is the first tas of the onine component. The custer s mean is used by the recommendation tas to generate recommendations to the new actie session. 3. DATASET DESCRIPTION In our wor, we used msnbc.com anonymous web data [5]. The dataset describes users' isits to the msnbc website for the entire day of September, 8, 999. Each ine in the dataset corresponds to a user access session to the website. Page requests are at the ee of page category rather than URL ee. A the pages are cassified by a site administrator to seenteen topics which are "frontpage", "news", "tech", "oca", "opinion", "on-air", "misc", "weather", "heath", "iing", "business", "sports", "summary", "bbs" (buetin board serice), "trae", "msn-news", and "msn-sports". These topics are assigned the numbers from to 7 in the dataset, so corresponds to "frontpage", to "news", 3 to "tech" and so on. The number of URLs per category aries from to 5,. The dataset size is.9 MB and it contains 989,88 sessions. Tabe presents a sampe of the dataset hoding different sessions, the first session s represents one singe session containing two web page isits, both isits correspond to topic frontpage, whie session s contains 7 webpage isits, six of them correspond to topic frontpage and one of them corresponds to topic business. The space is used as a separator between any two consecutie web page isits in any session throughout the entire dataset. Tabe: A sampe of user sessions in the dataset Session Number S Sessions web isits S Irreeant and outier sessions are remoed from the dataset. Irreeant sessions are sessions of ength (i.e. consist of a isit to one page ony). Outier sessions are sessions which consist of isits to 3 or more pages (such sessions are usuay generated by automated programs) [4]. In tota, 365,435 irreeant sessions were eiminated and this eft 64,383 in the dataset. The tota number of outier sessions eiminated is 9,83 distinct sessions. This eaes 63,698 sessions from the row dataset. User sessions are of different engths and the same topics which are isited by seera users do not hae to agree on the order of their appearance. To faciitate the automatic extraction of sessions that hae common interests, eery session is represented as a ector of ength 7 (Reca 7 is the number of topics in the dataset). The first coumn represents topic ("frontpage"), and the second coumn represents topic "news" and so on. The Content-Matrix has the foowing definition: n C( i, j) if session i has isited topic j if session i did not isit topic j Where i is the session number, j is the topic number (from to 7) and n is the number of isits performed by session i to topic j. In usage preprocessing, we need to reformat the dataset in an appropriate format for capturing automatic usage behaior and comparing the site users based on their site naigation rather than their content interests. Anaogues to the content-space, we project each user session in a usage-space. With n web pages identified uniquey by their URLs, U {ur, ur ur (n-) }; and a set of m user sessions S {s, s s (m-) }, where m is the tota number of sessions and each s i S is a non-empty subset of U. The foowing formua iustrates the representation of any session s i in the usage-space. s i (ur, t), (ur, t),, (ur (n-), t) t is the time a user spent iewing a gien webpage. We used binary weights in our wor because the dataset doesn't inoe information a bout the page iew duration information []. This means that if a session refers to a page, then is paced in the corresponding ce in the ector; is paced otherwise. Unfortunatey, the dataset is recorded by the ee of webpage category (topics) instead of the ee of web page's URLs. To oercome this probem, we assume that the tota number of URLs for a the 7 site's topics, is, distinct URL and we wi refer to each web page by an id number instead of the page URL and to aoid dupicate ids we dedicate a unique range of ids for each topic. These ranges are proportiona to the distribution of eery topic in the dataset. 5. GENERATING SIMILARITY MATRIXES We generated three simiarity matrixes between a sessions, namey, content, usage and usage-content simiarity matrixes. We gae these three matrixes the foowing symbos C (m, m), U (m, m) and UC (m, m) respectiey, where m is the number of sessions in the dataset. We used the cosine simiarity measure to determine the simiarity between any two sessions. These matrixes were used in the next step to custer users' access sessions according to their simiarity in usage and content.

3 Generating Content Simiarity Matrix: This matrix is a two dimensiona matrix where eery ce contains a measure of the simiarity between two sessions, here the content simiarity between two sessions gies us an indicator to which ee the content of the two sessions are simiar. We appy the cosine simiarity measure between eery two sessions in the dataset using the content ectors representation. Equation presents the mathematica expression of the cosine simiarity between two ectors in the same space. sim (, ) i j i j i Where: i, j: are the ectors of the sessions number i and j and : is the ength of the ectors. As an exampe, eery coumn in the row C (, m) of the matrix records the simiarity between session number and sessions from to (m-). Generating Usage Simiarity Matrix: This matrix is a two dimensiona matrix where eery ce contains a measure of the simiarity between two sessions. The usage simiarity between two sessions gies us an indicator on the simiarity in browsing the website between two sessions. With the users' sessions represented in the usage-space, we now exacty what URLs a user isited in eery session. As much as two sessions isit the same web pages, the usage simiarity between them increases. Here aso we use the cosine simiarity as a simiarity measure but the ectors now in the usage-space. Generating Usage-Content Simiarity Matrix: The goa behind integrating usage and content simiarity matrixes is to hae a measure of the simiarity between users' sessions in both usage and pages' content. This matrix wi be the input for the custering tas where simiar sessions are grouped together in the same custer. We generate the usage-content simiarity matrix by adding eery ce of the usage simiarity matrix to the corresponding ce of the content simiarity matrix. Note that the simiarity aues may be greater than because in the Usage-Content Matrix, we are just adding corresponding simiarity aues from the Usage Matrix and the Content Matrix. j 6. CLUSTERING In this step, we group simiar sessions based on their webpage content and based on their website usage. We hae used the we nown -means custering agorithm. This agorithm has three main issues, namey, choosing initia custers' means, conergence criteria and determining the best aue of. We wi describe how we impemented them in the foowing subsections. Sessions custering resuts in a set of custers, C {c, c... c i }, in which each c i ( i ) is a subset of the set of user access sessions S, and is the number of custers. Choosing Initia Custers Means: To start the custering agorithm, we need initia means to be used in the first custering iteration. We used the Jaa random cass to generate a random number in the range from to the number of sessions to be custered. Then we use this number as an index to the seection of the initia mean session. We hae used this procedure to choose the initia means for the custers. () Conergence Criteria: After choosing the initia means, the Eucidian Distance is cacuated between the candidate sessions to be custered and the means of a custers. The candidate session is paced in the custer where the distance between this session and the custer mean is the smaest. This operation is performed on a users' sessions unti a of them are custered in custers. Custering a sessions is repeated seera times unti a conergence criterion is met. We assumed that conergence is achieed if the sessions stop shuffing between the custers. This criterion wors as foows: after each iteration, custers' means are recacuated. The means of iteration i are compared with the means of iteration (i -). The conergence criterion is met when a the means of iteration i are equa to their corresponding means of iteration (i -); otherwise, the custering continues for another iteration. Determining the Best Vaue of : Lie other partitioning custering agorithms, the number of custers is necessary to be specified in adance as an input for the -means custering agorithm. The best number of custers here means that partitioning of users sessions wi gie us the best groups of simiar users. Our approach is to appy the -means agorithm to the access sessions using aues arying from up to 5. For each, we cacuate custering quaity measure caed custer compactness (C mp ) and custer separation (S ep ) proposed in [6] to measure the quaity of the resuting custers. Equations and 4 gie the mathematica expressions of (C mp ) and (S ep ) respectiey. Equation 5 presents a measure that combines the two measures which is caed oera custer quaity (O cq ). Where is the number of custers generated on the data set D, ( Ci ) is the standard deiation of the custer c i, and (D) is the standard deiation of the data set D. Equation 3 presents the mathematica expression of the standard deiation for any set Q. N ( Q ) N i d i, ( mean ) Q Where d( i, (mean)q ) is the Eucidean distance between two ectors i and (mean)q, N is the number of members in Q, and (mean)q is the mean of Q. The smaer the C mp aue, the higher the aerage compactness in the output custers: S ep C mp ( ) i j j i i (, ) d ( mean) ci ( mean) cj exp where is the standard deiation of the data set D, c is the number of custers, exp() is the function that returns an exponentia aue, (mean)ci is the centroid of the custer c i, (mean)cj is the centroid of the custer c j, d() is the distance measure used by the custering agorithm, and, is the Eucidian Distance between the ( ) d ( mean ) ci ( mean ) cj ( c i ) ( D ) centroid of custer c i and the centroid of custer c j. Simiar to C mp, the smaer the S ep aue, the arger the oera dissimiarity among the output custers. (3) () (4)

4 O cq ( ) β C + ( β ) S, β (5) Where [,] mp ep β is the weight that baances the measures C mp and S ep. We used O cq (.5) to gie equa weights to the two measures. Therefore, the ower the O cq aue, the better quaity of the oera output custers. Naturay, the most satisfying quaity score indicates the best partition of access sessions, whie the corresponding suggests the best number of custers on the access sessions. 7. CLASSIFICATION Cassification is the first step in the onine component of the recommendation system. We use testing sessions seected randomy from the dataset to simuate actie user sessions. The testing sessions are diided into two parts, namey, cassification part and recommendation part. The first part is used to stimuate an actie user session and the second part is used to stimuate the future isits. The two parts are represented in the content-space. We cacuated the mean ector of eery custer and used this mean as a representatie of the custers' members. The mean ector of a custer is cacuated by counting the tota number of isits for eery topic, in the content-space, in a custer's sessions. With custer representaties, we compute the simiarity S im (s i, P ), where s i is the testing session to be cassified and P is the representatie of custer. The testing session is cassified to the custer with the highest S im. Here we use the cosine simiarity measure. Certainy, it's the we nown K Nearest Neighbors (KNN) cassification method, with K. So now eery testing session is cassified to the nearest custer based on web page s content and usage. The second part, recommendation part, represents the users' predicted future isits and it is used to eauate the recommendations of the system. 8. RECOMMENDATION The custers' means represent the patterns of their custers' members. Now, we need to choose from these patterns the recommendation set for the new actie user. In a specific custer mean, not a topics hae the same importance; some of the topics are isited more than others. We use a threshod to fiter out the east significant topics in eery custer mean. The threshod is a specific percentage of the tota number of pages isited in a the custers sessions. A % threshod means that any topic which was isited in ess than % of the tota number of custer's sessions wi be excuded from the custer's mean. To choose the best threshod aue, we tried aues from % up to 5%. 9. EXPERIMENTATION AND RESULT ANALYSIS Assessment Methodoogy: We use precision and reca to assess the prediction capabiities of our recommendation system. Our assessment strategy is based on comparing our recommendation system that coupes the usage and content to generate recommendations for an actie user to a recommendation system that uses ony usage data to generate recommendations. The comparison is carried out using the reca and precision parameters. The two systems are assessed using the same training and testing data in a case studies. out these sessions into 49,3 sessions (two thirds of data) for training and 4,566 sessions (one third of data) for testing. We too the training part starting from session number up to session number 49,3 and the testing part from session number 49,3 up to 63,697. In this paper, we are going to discuss the resuts for the first case study which consists of the first 3, sessions (from session number up to session 9,999) from the training data and the first, sessions from the testing sessions. Experiments: The number of custers () was determined experimentay, we hae tried different aues of from to 5. 5 was chosen as a maximum aue for due to the sma size of resuting custers which hods ess than % of the tota number of sessions using aues arger than 5. Figure shows the custers' quaity when the number of custers ranges from to 5 for both usage and content based recommendation and pure usage based recommendation. Reca that the measure for custer quaity used here is a minimization function. It is cear from the figure that the quaity of custers generated using usage and content data is better than the quaity of custers generated using content data ony. This impacts the quaity of the recommendations as we. Custers Quaity Choosing Best K Case Study Number of Custers Usage&Content Pure Usage Figure : Custer quaity for both usage & content and pure usage We experimented with seera aues for threshods to determine the interestingness of topics within a gien custer. These threshods range from % to 5%. n% means that a topic is considered uninteresting to a custer if ess than n% of that custer's sessions accessed a topic. The precision and reca aues are potted against seera aues of threshods as shown in figure. The aues of precision and reca are the aerage precision and reca for a testing sessions. We obsere that, with the increase of the threshod aue, the precision increases, whereas the reca decreases. This continues unti the two cures intersect at threshod6, then the cures amost sette. Figure 3 shows the precision aues for the recommender system that combines usage and content data and the recommender system that uses usage data ony. It is cear that combining usage and content data increases the precision. Figure 4 shows the reca aues for the two cases. Reca aues for the usage ony case are higher than the reca aues of usage and content case for threshod aues ess than. Howeer, this behaior is reersed for threshod aues greater than. Case Study: After a the preprocessing steps hae been performed, we hae 63,698 different sessions. We spit

5 Custers' Mean Threshod Case Study Threshod Vaue % Reca Precision REFERENCES [] M. Caypoo, A. Gohae, T. Miranda, P. Murnio, D. Netes, M. Sartin. Combining Content-Based and Coaboratie Fiters in an Onine Newspaper, Proceedings of ACM SIGIR Worshop on Recommender Systems, Aug [] H. Dai, B. and Mobasher, A road map to more effectie web personaization: Integrating domain nowedge with web usage mining. Proceedings of the Internationa Conference on Internet Computing; 3, pp ; Las Vegas, Neada. Figure : Choosing custers' mean threshod for case study.. CONCLUSION This paper has presented a noe approach for couping content and usage data in order to generate recommendations to users. The recommendations are in the form of suggesting web pages to the user to isit based on comparing that user's interests with simiar users who hae isited the website earier. The current user's interests are discoered by mining the content and usage properties of the few pages she/he has isited. The experiments that were conducted showed that couping the usage and content data when generating recommendations improes the quaity of custers and the quaity of recommended pages. The quaity of custers is computed using custer compactness and separations; both are minimizations functions. The quaity of the recommended web pages is determined using precision and reca. [3] B. D. Daison. Predicting web actions from htm content. Proceedings of the Thirteenth ACM Conference on Hypertext and Hypermedia (HT ), : pp ; Coege Par, MD. [4] L. Haibin, K. Vado: Combined mining of Web serer ogs and web contents for cassifying user naigation patterns and predicting users' future requests. Data & Knowedge Engineering, Vo. 6, No., 7, pp [5] S. Hettich, S. D. Bay. The UCI KDD Archie [ Irine, CA: Uniersity of Caifornia, Department of Information and Computer Science. [6] J. He, T. Ah-Hwee, L. T. Chew, Y. S. Sam. On quantitatie eauation of custering systems, Custering and Information Retriea, Kuwer Academic Pubishers, 3: pp [7] B. M. Kim, Q. Li, C. S. Par, S. Kim. A New Approach for Combining Content-based and Coaboratie Fiters, Journa of Inteigence Information Systems, Vo. 7, 6; pp Precision Precision Case Study Custers Mean Threshod Figure 3: Precision for case study. Reca Case Study U & C Pure [8] B. Mobasher, H. Dai, T. Luo, Y. Sun, J. Zhu, Integrating web usage and content mining for more effectie personaization. Proceedings of the First Internationa Conference on Eectronic Commerce and Web Technoogies, : pp ; Springer-Verag. [9] Q. Li, M. K. Byeong, An approach for combining contentbased and coaboratie fiters. Proceedings of the 6 th Internationa Worshop on Information Retriea with Asian Languages, 3; pp. 7-4 Sappro, Japan. [] J. Striastraa, R. Cooey, M. Deshpande, and P. N. Tan. Web usage mining: discoery and appications of usage patterns from web data. SIGKDD Exporations, Vo., ; pp Custers Mean Threshod Figure 4: Reca for case study. U & C Pure Usage