MODELING USER INTERESTS USING TOPIC MODEL

Transcription

1 Jornal of heoretcal and Appled Informaton echnology 0 th Febrary 203. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: MODELING USER INERESS USING OPIC MODEL QINJIAO MAO, BOQIN FENG, 2 SHANLIANG PAN he School of Electronc and Informaton Engneerng, X an Jaotong Unversty, 70049,Chna 2 College of Informaton Scence and Engneerng, Nngbo Unvertsy, 352,Chna E-mal: maoqnao@63.com, bqfeng@mal.xt.ed.cn, 2 panshanlang@nb.ed.cn ABSRAC In recommender systems, modelng ser nterest s a basc step to nderstand ser's personal featres. radtonal methods mostly st se the tems that the target sers navgated as ther nterests, whch makes the nherent nformaton nclear to the system and ths the recommendatons are not ntellgent enogh. In ths paper, we nvestgate the tlty of topc model called LDA for the task of modelng ser nterests whch are assmed as the latent varables behnd ser actvtes n the systems. By sng sch a probablstc model on the generaton of ser profles, the relatonshps amongst ser tem and nterest are constrcted. Each ser can be consdered as a generaton of the model. Based on ths ser nterest model, we propose three tem rankng methods for personalzed recommendaton. In order to get a better model for each algorthm, we se a smple atomatc parameter trnng way to select the model parameters. After careflly selectng the parameters, or methods can all receve encoragng reslts for recommender systems. Keywords: opc Model, User Interest, Recommender System, Collaboratve Flterng. INRODUCION When people are srfng on the nternet, one of the frst thngs they may do s to dentfy what knds of lnks or web pages they are nterested n, whch means they are wllng to clck or navgate. We beleve that there are sch thngs called ser nterests whch gde the ser actvtes. As n the real world, people all keep dstnct ser nterests whch gde them to seek personalzed nformaton dfferent from each other. A page they navgate may contan somethng related to ther nterests they want to get. Users keep the topc they follow n mnd and ths nformaton plays a role n the assessment of whether an tem s relevant to ther nterests. Snce sers on the web are sfferng from nformaton overload, catchng ser nterests s very helpfl to mprove ser experences n nowadays web. Recommender systems are becomng ncreasngly poplar thanks to ther tltes on provdng people wth recommendatons of tems they mght be nterested n and ths prchase or take a deep look at[]. he systems learn sers' preferences and recommend prodcts they are expected to fnd from the large scale of all avalable goods. herefore the obectve of the system s to provde people wth personalzed experence to match ther needs as the system s st desgned only for the target ser. In natral langage process, a topc model s a type of statstcal generatve model proposed frstly for analyzng latent abstract topcs n a collecton of docments. he most common model n se s called Latent Drchlet Allocaton(LDA) ntrodced by Ble[2], and t leads a new drecton n ths research. Afterwards, lots of smlar topc models are proposed to deal wth more complexty statons. A lmtaton of LDA s the nablty to model topc correlaton whch stems from the se of Drchlet dstrbton to model the varablty among the topc proportons. A correlated topc model was proposed where the topc proportons exhbt correlaton va the logstc normal dstrbton nstead of Drchlet dstrbton[3]. o captre topc evolvement n temporal data, how to ntegrate tmestamps nto topc models has been nvestgated. he dynamc topc model[4] smply dvde docments nto several sbsets accordng to ther tmestamps and bld topc modes for each sbset and transformatons between these models. he dynamc mxtre model[5] assmes that the mxtre of latent varables for all streams s dependent on the mxtre of the prevos tmestamp. hese models are all Markov chan-based models that pt the Markov assmptons on the topc states transtons. here are also models that do not assme the Markovan dependence over tme, for example, the topc over tme[6]. In ths model, tmestamps are drawn from the same beta 600

2 Jornal of heoretcal and Appled Informaton echnology 0 th Febrary 203. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: dstrbton for topcs. hogh all of these models take the temporal data nto consderaton, tmestamps are all connected to the docments, that s, one tmestamp for each docment snce they are models for text analytcs. In fact, ser nterests can be regarded as topcs to some extent. opcs can be sed to present ser nterests very well. A web ser proflng and clsterng framework based on LDA-based topc modelng wth an analogy to docment analyss n whch docments and words represent sers and ther actons was proposed by Hrosh Fmoto et al.[7] Whle most of the exstng works are focs on nformaton retrevng[8] or qeryng[9, 0], they can not easly be adapted drectly on some of the recommender systems. In ths paper, we present a ser nterests modelng method sng LDA wthot consderng the text of the tems snce lots of tems n the web lke moves or msc can not be descrbed well n texts. [8]Based on the nterests we nfer from ths model, we propose three knds of recommendaton methods called pre-lda, LDAknn, and LDA-tran. Sch a ser nterests modelng method s dfferent from the tradtonal contentbased nterest modelng n that they are defned as mxtre of tems. It can be sed qte exactly for recommender systems, and the tlty for personalzed recommendaton wll be dscssed n or paper. 2. USERS, IEMS AND INERESS In a recommender system, a ser s commonly descrbed by the tems they navgate or lke. Items can be all knds of web obects we may deal wth, for example, web pages, moves, songs or goods. he system keeps ser s actvtes on tems, and takes advantage of them to fgre ot the preference on the remans ones they have not vsted. A ser may keep several nterests n mnd, and each nterest has lots of tems related to t n the system. A ser sally only needs a tny fracton of the tems based on ther nterests on certan tmes nstead of the all tems nder that nterest all the tme. So, a good recommender system s to provde sers wth the rght tem at the rght tme. Exstng recommender systems are mostly amng at provdng sers only the rght tem wthot consderng the tme factor. Somehow ths hnders them from beng a good recommender system. herefore, nterests can be regarded as latent factors n recommender systems. Interests carred by the ser determne whch tems the ser wants to get. Unfortnately, t s hard for ser to descrbe or provde what ther nterests are. Even thogh a ser may tell what knd of thngs he/she nterested n, makng a recommendaton s stll dffclt. he cases of ths problem are manfold. Frst, the nterests descrbed by sers themselves are always arbtrary. herefore one nterest may be descrbed dfferently by dfferent sers, whle dfferent nterests may be descrbed as one thng. Second, even ser nterests are relable to get, recommender system need to cnstrct the relatonshp between tems and nterests, and then decde whch tems are ft for? ther nterests amongst the large scale of canddates. Ignorng the tme nformaton, sertem dataset has the same form as word-docment co occrrence matrx, wth each ser beng a docment, each tem beng a word, and a ser vstng on an tem beng a docment contans a word. 3. OPIC MODEL FOR USER INERESS In ths paper, we bld ser nterests sng topc model called Latent Drchlet Allocaton(LDA). Snce we are ntendng to model ser nterests, we wll nfy the concepts of topc and nterest and se nterest for both of them most of the tme. Under the context of recommender system, we map the obects 'ser, tem, nterest' we deal wth to the obects 'docment, word, topc' n the orgnal topc model respectvely. So, smlarly, we can defne the generatve process abot sers and ther tem profles. In the generatve process, a mltnomal dstrbton θ over nterests s randomly sampled from the Drchlet dstrbton wth parameter α for each ser, then an nterest z s sampled from the dstrbton θ, and an tem s generated randomly from the nterest dstrbton on tems wth parameterϕ whch s a sample of a Drchlet dstrbton wth paremeter β. he graphcal model representaton for the model s gven n Fgre, and a bref notaton abot the symbols sed n ths paper s smmarzed n able. here are lots of researches on extendng the LDA model to model more complcated statons. Whle n ths paper, we only dg nto the tlty of the topc model on recommendatons. 60

3 Jornal of heoretcal and Appled Informaton echnology 0 th Febrary 203. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: α β θ z φ I N M Fgre:Graphcal Representaton For LDA able:lst Of Notatons Used In hs Paper Symbol M N θ ϕ z m dz Descrpton Nmber of ser nterests Nmber of sers Nmber of nqe tems he mltnomal dstrbton of nterests to ser he mltnomal dstrbton of words to nterest z he nmber of tokens n docment d assgned to nterest z n he nmber of tokens of word w zw assgned to nterest z αβ, he parameters of the correspondng dstrbtons 4. RECOMMENDAION ON USER INERESS MODEL 4.. Rankng Items Based on User Profle In recommender systems, sers' actvtes are collected to constrct ser profles. Based on sers past navgated tems, we can nfer ther related nterest dstrbton. Personalzaton that takes advantage of the nterest dstrbton of the ser can be derved as follows. For a ser wth the nterest dstrbton of θ, whch s derved from the LDA-based model, we calclate the probablty of the tem I that the ser may choose as follow: PI ( ) = Pz ( = t ) PI ( z= t) = θ ϕ t t t= t= It follows the generaton process of the topc model on ser profle, and t makes a smple assmpton that the ser s nterests are depended on the past navgaton nformaton. hs the tems are ranked accordng to the probabltes and we call ths method "pre-lda" recommendaton Rankng Items Based on User Smlarty he rankng method we proposed n 4. wll sffer the problem of cold start. A new ser wth few tems known by the system wll make the topc dstrbton over ftted by the lttle nformaton abot the ser. he ser may start sng the system for tny tmes and has no clear or explct ntent except several clcks on some tems. he nformaton we have abot the target ser s nsffcent. So, t s dogmatc to smply take these tems as the ser's preference. So, we propose another way of tem rankng to do the recommendaton. We learn the dea from collaboratve ntellgence that we fnd smlar ser to the target ser based on the topc model and then make a recommendaton based on the smlar ser. he most smlar sers to the target ser are the ones who have the maxmm condtonal probablty of the tem set of the target ser, gven the canddate sers. Gven a target ser who has vsted the tems I, we calclate the probablty of generatng I nder the condton of the exstng known sers by PI ( ) = PI ( ) I I = P( z = t ) P( I z = t) I I t= = θ ϕ I I t t t= We select a ser set S, S = K for ser n whch the sers S have the largest PI ( ). hen we calclate the preference on nknown tems I for ser accordng to PI ( ) = PI ( ). S he probablty PI ( ) s calclated n the same way wth "pre-lda" n 4.. We call the recommendaton based on ths rankng method "LDA-KNN" Rankng Items Based on Item ranston Based on the topc model LDA we bld, there s addtonal nformaton we can get, whch we call 602

4 Jornal of heoretcal and Appled Informaton echnology 0 th Febrary 203. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: tem transton. ranston s one relatonshp between tems, ndcates the probablty that one tem wll be vsted after another. hoght the transton s calclated by the sage data, t nherts smlartes or assocatons between tems. ranstons from tem I to I 2 can be expressed by a condtonal dstrbton of accessng I 2 when havng the cle of I. In or generatve model, tems are generated randomly by the topc dstrbton. It seems lke each tem s generated ndependently and has no connecton to other ones. Whle n fact, we can get the assocatve relatonshps on tems accordng to the model we nfer. Assmng that each tem s only decded by one nterest, we can get the postorr dstrbton Pz ( = I) from the model. hen the condtonal dstrbton of I 2 nder I can be calclated by: PI ( I) = PI ( z= Pz ) ( = I) 2 2 = Here, the posteror dstrbton s calclated by: Pz ( ) = I = z PI (, z= ) PI (, z) Based on the tem relatonshps we get, we calclate the preference on nknown tems for ser by smmng p all the probabltes that can be transformed from the tems that the ser has ever vsted: PI ( ) = PI ( I) I I We call the recommendaton based on ths rankng method "LDA-tran" Parameter nng Method In each algorthm we proposed so far, the recommendaton reslts are senstve to the parameters αβ,, we se to estmate the model. It's hard to fnd global optmm parameters. We need to tne the parameters n order to maxmze the recommendaton precson S@K. We se a smple random searchng method called Atomatc Parameter ner (AP) to fnd the relatve better solton to the model. he basc dea s to randomly change one of these parameters, check f the reslt gets better nder the new parameter vale, and then decde f keep the vale or not. In detal, ths works as follows: Randomly select a nmber from {, 2, 3} and draw a new parameter vale for the th parameter. If the new parameter makes the reslt better than the old one, we assgn the new vale to the parameter. Loop ths process ntl the reslt s good enogh. When dealng wth LDA-knn, we add the parameter K nto the tnng method, whch makes selected from{,2,3,4}. 5. EXPERIMENS 5.. Evalaton Protocol In order to examne the effectveness of or models, we condcted an experment on the real world dataset call MoveLens. he data set was collected from the GropLens research ste. It contans 943 sers, 682 tems and a total of 00,000 ratng data. Each ser has no less than 20 ratngs. he data s randomly dvded nto a tranng set and a test set wth exactly 0 ratngs per ser n the test set. After the dvson, each ser has at least ten ratngs as hs profle for tranng. he experments were mplemented sng a modfed verson of the "Matlab opc Modelng oolbox.4", provded by Mark Steyvers and om Grffths Evalaton Metrcs For the performance measres, we se the metrc s@ k whch s defned as: Q s@ k= ( r (, I) k) Q Here, Q s the nmber of ser-tem pars we sed n the test set, r (, I ) s the rankng of tem I for ser and () s an ndcator fncton whch retrns when ts argment s tre and 0 otherwse. r (, I ) s correspondng the descent order of PI ( ). he metrc s@ k only consders the nmber of tems that retrned n the recommended lsts wthot consderng the order of the tems. For a more exhastve analyss, we employ another metrc called the mean recprocal rank ( MRR ): Q MRR = Q r (, I ) 603

5 Jornal of heoretcal and Appled Informaton echnology 0 th Febrary 203. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: We are hopng the tems n the test set wth smaller r (, I ) that can be recommended n the head of the lsts, sch that the bgger s@ k or MRR 's vale s, the better the algorthms performance Expermental Reslts In ths secton, we focs on how the evalaton measres evolve wth the parameters. Based on the two metrcs, or experments are done n ether hgher s@ k targeted or hgher MRR targeted. he two targeted APs receve reslts a lttle dfferent. In order to tne parameters to have a better reslt for recommendaton, we teratvely choosng a new vale for the randomly selected parameter to see f t mproves s@ k of the recommendaton or not to decde f parameter wll be kept or replaced by the new vale. By dong ths wth many dfferent ntal vales and keep the teraton ntl the metrc stop changng for a relatve long tme, we record the best reslts. And ths process s done smlar wth the obectve of hgher MRR. able 2: Correspondng Parameters And MRR S For Recommendaton Usng LDA Wth Hgher s@ k argeted. Method α β K MRR PreLDA LDAKNN LDAtran able 3: Correspondng Parameters And MRR S For Recommendaton Usng LDA Wth Hgher MRR argeted. Method α β K MRR PreLDA LDAKNN LDAtran Fgre 2 s@ k Of op-k Recommendaton Usng LDA Wth Hgher In Fgre, we present the reslts of recommendaton precson s@ k based on the topc models wth hgher s@ k targeted and the correspondng parameters and MRR s are lsted n able 2. As we can see that the performance of Pre-LDA s relatve worse that the other two methods, and LDA-tran s the best one of the three. By choosng parameters wth hgher MRR s, we get the correspondng parameters and MRR s lsted n able 3 and s@ k n Fgre 3. Fgre 3: s@ k Of op-k Recommendaton Usng LDA Wth Hgher MRR Besdes, we nvestgate the mpact of ser profle sze on the algorthms' precson. We dvded the sers by ther tem nmbers n tranng set nto [0,30), [30,60), [60,00), [0,200), [200,300),, [300,77] and mark the correspondng reslts as Profle-30, Profle-60, Profle-00, Profle-200, Profle-300, Profle>300. Fgre 4 shows the reslts of the three algorthms on dfferent sbsets. Form the reslts, t s nterestng to notce that all of the three algorthms perform better when the profles are relatve smaller, whch means that the tems we want to predct are decded by few tems and the precsons wll be declned f we take too mch tems as the cle to the specfc tems. So, n real 604

6 Jornal of heoretcal and Appled Informaton echnology 0 th Febrary 203. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: systems, settng a tme wndow to lmt the ser profle may be helpfl n makng the predcton. tranng data. We examne the model effectveness n recommender systems by three rankng methods on the model and dscss the reslts. Comparng to the prevos works, topc model sed n ths paper can bld the relatonshps on ser, tem and ser nterest whch are dfferent from the tradtonal way and qte helpfl to constrct algorthms to make recommendatons. In ths paper, we propose three knds of methods to rank tems for sers. he experments on the data of recommender system show the effectveness of these algorthms qaltatvely and qanttatvely. hogh or model seems qte effectve n recommendatons, or methods stll have some shortages. In or methods, we made an assmpton that ser nterest keep stll drng the whole perod, ths the model does not consder the temporal nformaton. As n real world, ser's personal preference n ther nterests also changes drng the tme, that s, the ser profles are changng too. hs problem may be the work we wll deal wth n the followng work. ACKNOWLEDGMENS: hs work s spported by the Natonal Natral Scence Fondaton of Chna (No.62028, No ), Nngbo Natral Scence Fondaton (No.202A60066), Zheang Provncal Natral Scence Fondaton (No.LY2F02020). REFRENCES: Fgre 4: s@ k Of Profle-30, Profle-60, Profle-00, Profle-200, Profle-300, Profle> CONCLUSION In ths paper, we vew ser nterests as latent varables hdden n the recommender system based on topc model. User nterests are represented by mltnomal dstrbtons on tems, and sers are modeled as mltnomal dstrbtons on nterests. Usng Gbbs samplng methods, ser nterests are nferred from a generatve model accordng the [] C. N. Zegler, G. Lasen and L. Schmdt- heme, "axonomy-drven comptaton of prodct recommendatons," n Proceedngs of the thrteenth ACM nternatonal conference on Informaton and knowledge management, 2004, pp [2] D. M. Ble, A. Y. Ng and M. I. Jordan, "Latent drchlet allocaton," he Jornal of Machne Learnng Research, vol. 3, pp , [3] D. M. Ble and J. D. Lafferty, "A correlated topc model of scence," he Annals of Appled Statstcs, vol., pp , [4] D. M. Ble and J. D. Lafferty, "Dynamc topc models," n Proceedngs of the 23rd nternatonal conference on Machne learnng,icml '06, New York, NY, USA, 2006, pp [5] X. We, J. Sn and X. Wang, "Dynamc mxtre models for mltple tme seres," n Proceedngs of the 20th nternatonal ont conference on 605

7 Jornal of heoretcal and Appled Informaton echnology 0 th Febrary 203. Vol. 48 No JAI & LLS. All rghts reserved. ISSN: E-ISSN: Artfcal ntellgence,ijcai'07, San Francsco, CA, USA, 2007, pp [6] X. Wang and A. McCallm, "opcs over tme: a non-markov contnos-tme model of topcal trends," n Proceedngs of the 2th ACM SIGKDD nternatonal conference on Knowledge dscovery and data mnng, New York, NY, USA, 2006, pp [7] H. Fmoto, M. Etoh, A. Knno, and Y. Aknaga, "Web ser proflng on proxy logs and ts evalaton n personalzaton," Web echnologes and Applcatons, pp , 20. [8] J. Y. Km, K. Collns-hompson, P. N. Bennett, and S.. Dmas, "Characterzng web content, ser nterests, and search behavor by readng level and topc," n WSDM '2, New York, NY, USA, 202, pp [9] L. L, G. X, Z. Yang, P. Dolog, Y. Zhang, and M. Ktsregawa, "An effcent approach to sggestng topcally related web qeres sng hdden topc model," World Wde Web, pp. -25, 202. [0] M. J. Carman, F. Crestan, M. Harvey, and M. Balle, "owards qery log based personalzaton sng topc models," n Proceedngs of the 9th ACM nternatonal conference on Informaton and knowledge management,cikm '0, New York, NY, USA, 200, pp