Collaborative Topic Regression with Multiple Graphs Factorization for Recommendation in Social Media

Transcription

1 Collaboratve Topc Regresson wth Multple Graphs Factorzaton for Recommendaton n Socal Meda Qng Zhang Key Laboratory of Computatonal Lngustcs (Pekng Unversty) Mnstry of Educaton, Chna zqcl@pku.edu.cn Houfeng Wang Key Laboratory of Computatonal Lngustcs (Pekng Unversty) Mnstry of Educaton, Chna wanghf@pku.edu.cn Abstract Wth a large amount of complex network data avalable from multple data sources, how to effectvely combne these avalable data wth exstng auxlary nformaton such as tem content nto the same recommendaton framework for more accurately modelng user preference s an nterestng and sgnfcant research topc for varous recommender systems. In ths paper, we propose a novel herarchcal Bayesan model to ntegrate multple socal network structures and content nformaton for tem recommendaton. The key dea s to formulate a jont optmzaton framework to learn latent user and tem representatons, wth smultaneously learned socal factors and latent topc varables. The man challenge s how to explot the shared nformaton among multple socal graphs n a probablstc framework. To tackle ths challenge, we ncorporate multple graphs probablstc factorzaton wth two alternatvely desgned combnaton strateges nto collaboratve topc regresson (CTR). Expermental results on real dataset demonstrate the effectveness of our approach. 1 Introducton Many real-lfe data have representatons n the form of multple vews (Lu et al., 013). For example, web pages usually consst of both text content and hyperlnk nformaton; mages on the web have relevant tags assocated wth ther content. In addton, t s also common that n real networks comprsng multple types of nodes are connected by multple types of lnks, formng heterogeneous nformaton networks (HIN) (Huang et al., 01). For example, n scentfc communty, varous types of lnks are formed for dfferent types of objects,.e., author wrtes paper, venue publshes paper, reader labels tag, and so on. Therefore, wth a large amount of complex network data avalable from multple data sources, how to effectvely combne ths knd of rch structure wth other auxlary nformaton such as content nformaton nto the same recommendaton framework s an nterestng and sgnfcant research topc for varous recommender systems. Ths paper ams to model multple socal graphs nto a prncpled herarchy Bayesan framework to mprove recommendng performance. The basc dea n ths paper s nspred by mult-vew learnng approach (Lu et al., 013),.e., leveragng the redundancy and consstency among dstnct vews (Kumar et al., 011) to strengthen the overall performance. We extend ths dea (Lu et al., 013) orgnally for clusterng problem to deal wth ratng scarcty problem when modelng user preference for recommendaton. Just as n general mult-vew learnng, each vew of objectve functon s assumed to be capable of correctly classfyng labeled examples separately. Then, they are smoothed wth respect to smlarty structures n all vews. Smlarly, n ths paper, we assume that our ndvdual vews of multple user socal relatons are smlar and complementary wth a shared latent structure. However, dfferent from mult-vew clusterng problem, our goal s to recover a sparse ratng matrx wth a large number of mssng user-tem pars rather than merely explotng cluster structure wth full task nformaton. Thus, the straghtforward mult-vew representaton of the objectve (ratng matrx) s Correspondng author Ths work s lcenced under a Creatve Commons Attrbuton 4.0 Internatonal Lcense. Page numbers and proceedngs footer are added by the organzers. Lcense detals: 33 Proceedngs of COLING 014, the 5th Internatonal Conference on Computatonal Lngustcs: Techncal Papers, pages 33 44, Dubln, Ireland, August

2 not avalable. Instead, we use sde nformaton (user socal graphs) to explot mult-vew learnng for mprovng collaboratve flterng (CF). As a result, collaboratve topc regresson (CTR) (Wang and Ble, 011) s employed as our basc learnng framework wth sde nformaton. Recently, CTR has ganed consderable attenton due to ts well-defned mathematcal framework and strong performance on user behavor predcton for varous real-world applcatons, such as document recommendaton (L et al., 013), tag recommendaton (Wang et al., 013), musc recommendaton (Purushotham et al., 01), celebrty recommendaton (Dng et al., 013) and vote predcton (Kang and Lerman, 013). However, all the extensons above merely focus on a sngle vew of user or tem relaton. In realty, a large amount of dverse socal graphs data are wdely exsted and partcularly valuable for mutually renforcng each other. Therefore, t should be well consdered. Takng ths nto consderaton, we extend CTR wth multple socal graphs factorzaton for recommender systems. The man challenge of ncorporatng multple relatons nto CTR s how to explot the shared nformaton among multple socal networks and how to further deal wth t to recover sparse ratng matrx n a probablstc framework. Prevous efforts, purely to address the frst ssue for clusterng problem, are usually to seek a weak consensus (Lu et al., 013) learnd from data jontly wth clusterng process. Intutvely, consensus can be seen as a latent cluster structure shared by dfferent vews. Thus, t means that learnng from dfferent vews should be softly regularzed towards a common latent structure. However, t s not easy to drectly formulate t n a probablstc framework, because weak consensus modelng can not be separated from a jont hgher task,.e., recoverng spare ratng matrx, n our case. To tackle ths challenge, we propose a novel herarchcal Bayesan model wth multple socal graphs factorzaton. We explot two ways of modelng shared nformaton for mult-vew based recommendaton. One s for heterogeneous network by drectly modelng dfferent vew specfc latent structures wth consensus for user representaton. The other s for homogeneous case, whch can be used as a transformed verson of heterogeneous relatons. In contrast wth the frst strategy, we model the latter usng a shared latent socal structure for all vews but wth dfferent user representatons. Thus, we can relax strong consensus assumpton n our heterogeneous case, through lnear combnaton of each sub-latent user wth mantaned sharng mechansm. The multple graphs factorzaton process n the proposed model can be seen as a regularzaton approach on each latent user for better uncoverng user-tem latent structures. Although, regularzaton technque for modelng multple heterogeneous networks s a hot research topc, n clusterng study from an algebra vew (Lu et al., 013; Kumar et al., 011), not much s known on usng t for collaboratve recommendaton problems n a more complex probablstc settng. The followng sectons wll dscuss those n detals and we use the terms network and graph nterchangeably throughout ths paper. Prelmnares In ths secton, we brefly revew collaboratve topc regresson (CTR) (Wang and Ble, 011), as the foundaton of our proposed model. Fgure 1 (left) shows the graphcal representaton of CTR, whch combnes the merts of tradtonal collaboratve flterng and probablstc topc modelng. Specfcally, the key mechansm of CTR s that usng topc vectors learned from LDA (Ble et al., 003) jontly controls the pror dstrbuton of latent tems n orgnal matrx factorzaton process of CF. The generatve process s descrbed as follows: For each user, draw user latent vector u N(0, λ 1 u I), multvarate Gauss dstrbuton wth zero mean. For each tem j, draw topc proportons θ j Drchlet(α), Drchlet dstrbuton. draw tem latent offset vector ɛ j N(0, λ 1 v I), and set the tem latent vector as v j = ɛ j + θ j. For each word w jn draw topc assgnment z jn Mult(θ), Multnomal dstrbuton. draw word w jn Mult(β zjn ), Multnomal dstrbuton. For each user-tem par (, j), 34

3 Fgure 1: CTR (left), heterogeneous CTR-MGF (mddle), homogeneous CTR-MGF (rght). draw the response r j N(u T v j, c 1 j ), unvarate Gauss dstrbuton, where c j s a confdence parameter for ratng r j, a > b. c j = a (hgher confdence), f r j = 1, and c j = b, f r j = 0. However, CTR does not take the complex socal network nformaton, whch s avalable and crucal n many real-world applcatons, nto consderaton. 3 CTR-MGF: Collaboratve Topc Regresson wth Multple Graphs Factorzaton In ths secton, we dscuss our proposed method, called CTR wth multple graphs factorzaton (CTR- MGF). Our model s a generalzed herarchcal Bayesan model whch jontly learns latent user, tem and multple latent socal factor spaces. Dfferent from prevous approaches, our method extends CTR to multple complex networks settng nstead of one partcular type of relaton for user or tem. Moreover, we consder two real general contexts n varous practcal applcatons. One s the context of heterogeneous networks. The other s the context of homogeneous networks. It s noted that, for relatve smplcty, n ths paper we only consder user orented complex network. The graphcal representaton of our models n Fgure 1 (mddle and rght) takes K = 3 networks as llustraton, whch can be arbtrary n our dervaton. It s also easy to see that Purushotham et al. (01) s a specal case of our proposed model, whch s not equpped wth graph sharng mechansm. 3.1 Model Notatons Each socal matrx Q corresponds to a socal network structure G = {V, E}, where users and ther socal relatons are represented as vertex set V and edge set E n network structure G, respectvely. The element q m n Q denotes the bnary relaton between user and graph specfc feature m n heterogeneous network or the relaton between two users and m n homogeneous network. 3. CTR-MGF for Heterogeneous Networks Heterogeneous network s formed by multple types of nodes beng connected by multple types of lnks. The key characterstc of heterogeneous network s that the szes of feature dmensons are dfferent among multple socal graphs. For example, n a socal musc sharng system such as LastFM, each user has multple heterogeneous relatons assocated wth the nterested musc,.e., user-artst, user-tag, and so on. Our model can handle all these relatons n the proposed framework, CTR-MGF. Specfcally, the generatve process of CTR-MGF for heterogeneous networks s lsted as follows: For each tem j, draw topc proportons θ j Drchlet(α), Drchlet dstrbuton. draw tem latent offset ɛ j N(0, λ 1 v I), multvarate Gauss dstrbuton and set the tem latent vector as v j = ɛ j + θ j. 35

4 For each word w jn draw topc assgnment z jn Mult(θ). draw word w jn Mult(β zjn ). For each heterogeneous socal graph k, For each socal graph specfc feature m draw graph factor-specfc latent feature vector s k m N(0, λ 1 s km I). For each user, draw the shared latent user vector among multple socal graphs u N(0, λ 1 u I). For each heterogeneous socal graph k For each socal graph specfc feature m draw graph specfc user heterogeneous relaton par qm k N(uT sk m, c 1 k,q m ). For each user-tem par (, j), draw the response r j N(u T v j, c 1 j ). In the above generatve process, the jont lkelhood of data,.e., R, Q,...,K, W, and the latent factors U, V, S,...,K under the full model s: p(r, U, V, S,...,K, Q,...,K, W, θ λ ) = p(r U, V ) p(w, θ β) p(u λ U ) p(v λ V ) K p(q k U, S k, λ Qk ) k K p(s k λ Sk ) k (1) For learnng the parameters, we develop an EM-style algorthm smlar to CTR. In our model, fndng the MAP s equvalent to maxmzng the followng log lkelhood obtaned by substtutng unvarate and multvarate Gaussan pdfs n Eq. 1: L = log( θ jz β zjn ) j n z λ U u T u λ V j λ Sk (s k m) T s k m m (v j θ j ) T (v j θ j ) j m c j (r j u T v j ) c k,qm (qm k u T s k m) () We employ coordnate ascent (CA) approach alternatvely optmzng latent factor varables u, v j, s,,...,k m and the smplex varables θ j as topc proportons. Specfcally, the followng update rules n CA are obtaned by settng the dervatve of L wth respect to u, v j, and sm,,...,k to zero. u = (λ U I + V T D c V + Sk T Dk q S k ) 1 (V T D c R + k Sk T Dk q Q k ) (3) k v j = (λ V I + U T D cj U) 1 (λ V θ j + U T D cj R j ) (4) s,,...,k m = (λ Sk I + U T D k q m U) 1 (U T D k q m Q k m) (5) where K s the total number of graphs. I s an dentty matrx of the same dmenson as that of latent space. U and V are matrces wth rows as latent users and latent tems, respectvely. S k s a matrx wth rows as socal factor-specfc latent feature vectors for graph k. R s a column vector wth values [r 1,..., r J ] T. Smlarly, R j = [r 1j,..., r Ij ] T. For graph k, Q k = [q1 k,..., qk M ]T and Q k m = [q1m k,..., qk Im ]T respectvely. Lkewse, Dq k, and Dq k m are smlarly defned wth dagonal elements c k,q. and c k,q.m, respectvely. D c s a dagonal matrx wth values dag(c 1,..., c J ). D cj = dag(c 1j,..., c Ij ). In addton, c j and c k,qm are also seen as the confdence parameters for r j and qm k, respectvely. The hgh confdence value a s set to the observed nteractve pars and the low confdence value b s set to the unobserved nteractve pars, where a > b > 0. 36

5 For our brevty, the remanng update rules for θ and β, can be obtaned usng the same way as descrbed n CTR (Wang and Ble, 011). Please see that for detals. It s worth notng that through our assumpton and the dervaton above, we have theoretcally proved that our modelng n ths case s equvalent to frst concatenatng features of dfferent vews together and then applyng Purushotham et al. (01) for recommendaton. 3.3 CTR-MGF for Homogeneous Networks In ths secton, we further extend the basc CTR to the context of homogeneous networks. In fact, any user specfc homogeneous networks can be obtaned through transformng correspondng heterogeneous networks. For example, n LastFM, we can construct two user-user homogeneous networks by computng the smlartes of user-tag and user-artst from orgnal heterogeneous networks. The goal of ths transformaton s to further explot weak consensus modelng scheme based on Secton 3.. Dfferent from the graph sharng mechansm presented n last secton, we relax the restrcton that all users have the same representaton. Specfcally, we assume each latent user has multple sub-graph specfc representatons. However, t s nontrval to model the relaxed assumpton drectly from orgnal perspectve. To acheve ths more weaker sharng mechansm, we are towards ts transformed perspectve,.e., sacrfcng heterogeneous characterstc, because we need to explot shared nformaton from latent graph specfc feature perspectve. Thus, we requre equal dmensons of dfferent graphs, whch motvates us to nvestgate the homogeneous case. The key dfferences between our model n ths secton and that n last secton are the strateges of latent user modelng and ts socal factor modelng. More specfcally, we model each latent user as a lnear combnaton of all sub-latent users assocated wth multple homogeneous networks. All these sub-latent homogeneous users are assocated wth a shared socal factor feature space. Thus, the shared nformaton among multple graphs can be exploted and t s more flexble to adjust the contrbuton of each sub-latent user to the fnal latent user representaton. The generatve process of CTR-MGF for homogeneous networks s lsted as follows: For each tem j, draw topc proportons θ j Drchlet(α), Drchlet dstrbuton. draw tem latent offset ɛ j N(0, λ 1 v I), multvarate Gauss dstrbuton and set the tem latent vector as v j = ɛ j + θ j. For each word w jn draw topc assgnment z jn Mult(θ). draw word w jn Mult(β zjn ). For each socal graph specfc feature m, regardng to all related homogeneous socal graphs draw a shared factor-specfc latent feature vector across multple graphs s m N(0, λ 1 s m I). For each user, For each homogeneous socal graph k draw a socal graph specfc latent user u k N(0, (λk u ) 1 I). For each socal graph specfc feature m draw graph specfc user homogeneous relaton par qm k N((uk )T s m, c 1 k,q m ). draw a fnal latent user u N( K T ku k, λ 1 u I). For each user-tem par (, j), draw the response r j N(u T v j, c 1 j ). In the above generatve process, the jont lkelhood of data,.e. R, Q,...,K and W, and the latent factors U, U,...,K, V and S under the full model s: p(r, U, V, S, U,...,K, Q,...,K, W, θ λ ) = p(r U, V ) p(w, θ β) K K ( p(q k U k, S, λ Qk )) p(s λ S ) ( p(u k λ Uk )) p(v λ V ) p(u λ U ) p(u U,...,K, λ C ) 37 (6)

6 Smlarly to last secton, we develop an EM-style algorthm to fnd the MAP solutons, whch s equvalent to maxmzng the followng log lkelhood by substtutng unvarate and multvarate Gaussan pdfs n Eq. 6: L = log( j n z j m θ jz β zjn ) λ Uk c k,qm (qm k (u k ) T s m ) λ V c j (r j u T v j ) λ C (u k ) T u k λ S s T ms m m (v j θ j ) T (v j θ j ) j (u T k u k ) T (u T k u k ) λ U u T u (7) We employ coordnate ascent (CA) approach as prevous secton alternatvely optmzng latent factor varables and smplex varables as topc proportons. Then we acqure the update rules by settng the dervatve of L wth respect to the followng varables to zero. u,,...,k = (λ Uk I + λ C T k I + ST D k q S) 1 (λ C u T k ( T p u p )λ CT k + S T Dq k Q k ) (8) p k u = (λ U I + λ C I + V T D c V ) 1 (V T D c R + λ C K k T k u k ) (9) v j = (λ V I + U T D cj U) 1 (U T D cj R j + λ V θ j ) (10) s m = (λ S I + Uk T Dk q m U k ) 1 ( Uk T Dk q m Q k m) (11) where K s the total number of graphs. I s an dentty matrx of the same dmenson as that of latent space. U and V are matrces wth rows as latent users and latent tems, respectvely. S s a matrx wth rows as the shared socal factor-specfc latent feature vectors for all graphs. T s the graph selecton weght, K T k = 1, T k >= 0. R = [r 1,..., r J ] T and R j = [r 1j,..., r Ij ] T. For graph k, Q k = [q1 k,..., qk M ]T, Q k m = [q1m k,..., qk Im ]T and U k s a matrx wth rows as the socal graph k specfc latent user vectors. Lkewse, Dq k, and Dq k m are smlarly defned wth dagonal elements c k,q. and c k,q.m, respectvely. D c = dag(c 1,..., c J ) and D cj = dag(c 1j,..., c Ij ). In addton, c j and c k,qm are also seen as the confdence parameters for r j and qm k, respectvely. The hgh confdence value a s set to the observed nteractve pars and the low confdence value b s set to the unobserved nteractve pars, where a > b > 0. For our brevty, the remanng update rules for θ and β, can be obtaned usng the same way as descrbed n CTR (Wang and Ble, 011). Please see that for detals. 3.4 Predcton Usng the learned parameters above, we can make n-matrx and out-of-matrx predctons defned n Wang and Ble (011). For n-matrx predcton, t refers to the case where those tems have been rated by at least one user n the system. To compute predcted ratng, we use r j (u ) T v j. (1) For out-of-matrx predcton, t refers to the case where those tems have never been rated by any user n the system. To compute predcted ratng, we use r j (u ) T θ j, (13) where the correspondng θ j s defned as topc proporton n Secton 3. and

7 3.5 Computatonal Issue To reduce computatonal costs when updatng u, v j and other varables wth smlar structure n update rule, we adopt the same strategy of matrx operaton shown n Hu et al. (008). Specfcally, drectly computng V T D c V and U T D cj U requres tme O(L J) and O(L I) for each user and tem, where J and I are the total number of tems and users respectvely, L s the dmenson of latent representaton space. Instead, we rewrte U T D cj U = U T (D cj bi)u + bu T U. Then, bu T U can be pre-computed and D cj bi has only I r non-zeros elements, where I r refers to the number of users who rated tem j and emprcally I r I. For other smlar structures,.e., V T D c V, S T D k q S, and so on, they are smlar. Therefore, we can sgnfcantly speed up computaton by ths sparsty property. 4 Experments 4.1 Data We evaluate our proposed method on real lfe dataset 1 from LastFm. LastFm s an onlne musc catalogue, powered by socal musc dscovery servce for personalzed recommendaton. Ths dataset (Cantador et al., 011) s challengng. Though t contans 9,834 pars of observed ratngs wth 189 users and 17,63 tems, the sparseness s qute low,.e., merely 0.783%, whch s much lower than that of the well-known Movelens dataset wth the sparseness 4.5%. On average, each user has 44.1 tems n the play lst, rangng from 0 to 50, and each tem appears n 4.95 users lbrares, rangng from 0 to 611. For each tem, the tag nformaton s used as bag-of-word representaton. After text processng, 11,946 dstnct words are remaned n the corpus. In addton, we further remove nosy users whch have no tems. We also construct two addtonal socal graphs for our experments. One s user-tag network extracted from user-tag-tem relatons n orgnal dataset. The other s user-user network through transformng the constructed user-tag network. The relaton n all graphs s bnary,.e., the avalable denoted as 1 and the unavalable denoted as 0. Table 1: Orgnal dataset descrpton Dataset users tems tags user-user relatons user-tags-tems user-tems relatons LastFm Metrcs Two metrcs for evaluatng the recommendaton performance are employed,.e., Recall and NDCG. Measure for plan relevance: Recall@k = relevance, (14) k where relevance denotes the total relevant papers n returned top-k result. Measure for rankng-based relevance: k =1 rel 1 log (1+) NDCG@k =, (15) IDCG where rel denotes the relevant degree whch s bnary n our task and IDCG s the optmal score computed usng the same form n numerator but wth optmal rankng known n advance. 4.3 Expermental Desgn In ths paper, we expect the proposed model Our-Homo n Secton 3. and Our-Heter n Secton 3.3 can jontly provde a general and systematc soluton to handlng the followng cases of usng multple graphs for recommendaton: Case 1:(Heterogeneous networks wth nose) Network data or the extracton process s usually mprecse or nosy n practce. Transform t nto homogeneous case and then use Our-Homo. 1 Data avalable at

8 Case :(Homogeneous networks) Our-Homo can be drectly employed as the tool for case 1. Case 3:(Heterogeneous networks wth hgh qualty) Our-Heter mght be drectly employed. It s not needed to be further transformed nto Homogeneous case. The detal experments n the followng sectons are presented to justfy the effectveness of our methods for the three cases above. 4.4 Experments for Case 1 and Case In ths secton, we manly focus on the most complex and common case 1 wth case n practce Baselnes We compare our proposed two models, the model n Secton 3. denoted as Our-Heter and the model n Secton 3.3 denoted as Our-Homo, wth some state-of-the-art algorthms. CTR: Ths method, descrbed n Wang and Ble (011), combnes both tem content nformaton and user-tem ratngs for CF. PMF: Ths method, descrbed n Salakhutdnov and Mnh (007), s a well-known matrx factorzaton method for CF, only usng nteractve ratng nformaton. SMF-1: Ths method, descrbed n Purushotham et al. (01), explots sngle user s socal network structure combned wth tem s content nformaton for CF. SMF-1 denotes usng our extracted user-tag relaton. SMF-: The same SMF method, descrbed n (Purushotham et al., 01). SMF- denotes usng orgnal user-user relaton. Our-Heter: Our model for heterogeneous networks, proposed n Secton 3., uses our extracted user-tag network and orgnal user-user network. Our-Homo: Our model for homogeneous networks, proposed n Secton 3.3, uses two homogeneous networks,.e., 1) the transformed user-user network through our extracted user-tag relaton, and ) orgnal user-user network Settngs For a far comparson, we use the smlar settngs as pror work n Purushotham et al. (01). Specfcally, to well judge the nfluence of multple socal network structures, we fx the effects of content nformaton to the same level that s optmal n SMF, λ v = 0.1. We randomly splt the dataset nto two parts, tranng (90%) and test datasets (10%), wth constrant that users n test dataset have more than half of the average number of rated tems,.e., 0. Ths expands the range of performance analyss for our evaluaton compared wth Purushotham et al. (01). The optmal parameters are obtaned on a small held-out dataset. For PMF, we set λ v = 100, λ u = For all CTR-based methods, we set a = 1, b = 0.01, λ v = 0.1. Specfcally, for CTR, we set λ u = For SMF-1 and SMF-, we set λ u = For Our-Homo, we set λ u = 0.01, λ u1 = λ u = λ s = 100, λ c = For Our-Heter, we set λ u = 0.01, λ s1 = λ s = 100. The remanng paramters are vared for experment analyss. It s noted that the task of out-of-matrx predcton s orgnally desgned for evaluatng tem content modelng n CTR rather than user socal graphs as n CTR-smf. Thus, we followed the same settng n baselne CTR-smf (Purushotham et al., 01), not consderng ths task Performance Comparson wth State-of-the-Art Methods Fgure shows the recall and NDCG results of all the methods when the number of latent factor s fxed to 00 (optmal for the baselnes). The proposed model Our-Homo consstently outperforms the baselnes and Our-Heter model under both recall and NDCG measures. Ths fndng demonstrates that (1) usng multple graphs for CTR s a necessary for mprovng recommendaton performance from both rankng and plan accuracy perspectves. () strong consensus for modelng shared nformaton undermnes the performance for multple graphs factorzaton as desgned n Our-Heter. (3) For heterogeneous case, we address that through smply transformng the heterogeneous network to homogeneous one and then use 40

9 Fgure : Our model comparson wth the state-of-the-art methods, for Recall and NDCG. Fgure 3: Performance comparson for dfferent latent factors top (50,100,00). Our-Homo. Ths s natural but the opposte s hard. Thus, our soluton Our-Homo for modelng weak consensus s effectve for both homogeneous and heterogeneous cases. In addton, we can see that CTR-smf (Purushotham et al., 01) s senstve to the qualty of graph (SMF-1 wth low qualty and SMF- wth hgh qualty as shown n Fgure ). In contrast, we can use the low qualty nosy graph (SMF-1) to mprove the overall performance by ths transformaton process. In fact, why Our-Heter does not perform well s manly due to the nosy graph-1. Transformaton can be seen as a denosng process Performance Analyss wth Dfferent Latent Factors Fgure 3 shows the results of the compared algorthms, wth dfferent number of latent factors for vared top recommended tem. It shows that K = 00 factors s optmal for all baselnes compared wth other choces of the number of latent factors. Ths justfes our fre choce of 00 latent factors reported n Fgure 3 (Other factor choces are omtted here due to page constrant, whch s not optmal for our baselnes) and suggests that the choce of latent factor number s crucal for all algorthms especally for PMF. In contrast, the proposed Our-Homo s more stable compared wth PMF and outperforms the other baselnes n an overall performance as reported n Fgure. 41

10 Fgure 4: Parameter analyss of graph selecton for our weak consensus modelng,.e., Our-Homo Impact of the Parameter for Graph Selecton Next, we examne how our algorthm Our-Homo s nfluenced by the graph-selecton weghts. In Fgure 4, the horzontal axs shows the graph proporton weght T 1 for graph-1 (user-tag network). In our case, two socal graphs are consdered. The weght for second graph s T = 1 T 1, whch s not shown n Fgure 4. Fgure 4 clearly proves the effectveness of our weak consensus modelng. Specfcally, we can see that 0.0 seemngly means that the frst graph s not selected due to the weght of the frst graph T 1 = 0.0 and that of second graph T = 1 (fully selected). However, due to our weak consensus modelng scheme, though the graph-1 weght s 0.0, t does not mean that the frst graph s removed. In fact, the effect of graph-1 s also actve through the shared varable s n Fgure 1 (rght). Ths further can be nvestgated from the Equaton 9. Apparently, the case of 0.0 weght for graph-1, u s only relevant to u 1, but from Equaton 8, we can see that u 1 s also nfluenced by u va the shared socal graph factors s. Thus, ths also explans why 0.0 weght for graph-1 s not equal to the result of SMF- as shown n Fgure, whch only uses graph-, orgnal user-user graph n SMF-. In addton, the valley n Fgure 4 mght be explaned that n the extreme cases (0.0 and 1.0), the denosng effect of weak consensus s slghtly strengthened because only one specfc graph (hgher qualty smf- or lower qualty smf-1 shown n Fg.) s drectly assocated wth fnal latent user combned wth shared varable. Therefore, the extreme case (1.0) s towards a relatve hgher performance. 4.5 Experments for Case 3 Though the proposed Our-Homo s more effectve than CTR and CTR-smf, t does not mean that the proposed another method Our-Heter s useless. In ths secton, we show how the case 3 wll be justfed Baselnes Our-Heter(N): Our model for heterogeneous networks, proposed n Secton 3., uses our modfed hgh qualty user-tag network as descrbed n Secton 4.5. and orgnal user-user socal network as SMF- n Secton SMF-1(N): The same SMF method wth sngle socal network, descrbed n Purushotham et al. (01). SMF-1 (N) denotes usng our modfed hgh qualty user-tag network as descrbed n Secton Our-Homo(O): The result of ths method s the same as that reported n Secton CTR(O): The result of ths method (Wang and Ble, 011) s the same as that reported n Secton Settngs We want to nvestgate whether Our-Heter wll outperform Our-Homo n the case where the heterogeneous networks wth less nose are avalable, compared wth prevous results n Fgure. The settngs 4

11 Fgure 5: Our model comparson wth the state-of-the-art methods (case 3), for Recall and NDCG. n case 3 are the same as that n Secton 4.4. except for the refned user-tag graph. In ths case, we construct a less nosy user-tag network by selectng top-10% tags accordng to tf df value. The optmal latent factor number s set to 100 for Our-Heter (N) through a small held-out dataset. The remanng parameters are kept as the same values n Secton For notatons n baselnes Secton, O denotes old settng and N denotes new settng wth updated user-tag network presented n ths secton Performance Comparson wth State-of-the-Art Methods Fgure 5 shows Our-Heter (N) can acheve mproved performance compared wth baselnes wthout transformaton, for the case where hgh qualty graphs are avalable. Specfcally, for recall measure, Our- Heter (N) produces the best result wth the ncreasng number of top recommended tems. In addton, we observe that modelng multple graphs s necessary to further mprove recommendng performance, whle multple hgh qualty heterogeneous graphs are avalable. For NDCG measure, Our-Heter (N) s comparable to our baselnes. Snce recall measure s only consdered for several reasons n prevous work (Wang and Ble, 011; Purushotham et al., 01), ND- CG s ntroduced as a plus compared wth prmarly focused recall. Therefore, Our-Heter (N) s also compettve n overall performance n case 3. In fact, as dscussed n Kang and Lerman (013), CTR-smf (Purushotham et al., 01) s not always superor to CTR (Wang and Ble, 011) and vce versa due to dfferent contexts. Lkewse, our model s under the mult-vew assumpton as dscussed n Secton 1 that should be checked n practce. 5 Conclusons In ths paper, we propose a general recommendaton framework wth multple data sources based on CTR. It s a prncpled herarchy Bayesan framework wth multple socal graphs factorzaton for recommender systems. In ths framework, two ways of consensus modelng are exploted. Specfcally, the proposed models Our-Homo and Our-Heter can jontly provde a general and systematc soluton to handlng three real cases of usng multple graphs wth tem content nformaton for recommendaton: case 1) Heterogeneous networks wth nose; case ) Homogeneous networks; case 3) Heterogeneous networks wth hgh qualty. Expermental results on real dataset demonstrate the effectveness of our approach. Whle ths framework s used for modelng multple user socal graphs, t can be easly extended to explotng other sde nformaton such as multple complex relatons for tems n varous applcatons. Acknowledgements We thank the anonymous revewers for ther valuable suggestons. Ths research was partly supported by Natonal Natural Scence Foundaton of Chna (No , ), Major Natonal Socal Scence Fund of Chna (No.1&ZD7), and Natonal Hgh Technology Research and Development Program of Chna (863 Program) (No.01AA011101). 43

12 References Davd M. Ble, Andrew Y. Ng, and Mchael I. Jordan Latent drchlet allocaton. Journal of Machne Learnng Research, 3: Iván Cantador, Peter Bruslovsky, and Tsv Kuflk nd workshop on nformaton heterogenety and fuson n recommender systems (hetrec 011). In Proceedngs of the 5th ACM Conference on Recommender Systems, RecSys 011, New York, NY, USA. ACM. Xuetao Dng, Xaomng Jn, Yuja L, and Langhao L Celebrty recommendaton wth collaboratve socal topc regresson. In Proceedngs of the Twenty-Thrd Internatonal Jont Conference on Artfcal Intellgence, IJCAI, pages AAAI Press. Yfan Hu, Yehuda Koren, and Chrs Volnsky Collaboratve flterng for mplct feedback datasets. In Eghth IEEE Internatonal Conference on Data Mnng, ICDM, pages IEEE. Hongzhao Huang, Arkatz Zubaga, Heng J, Hongbo Deng, Dong Wang, Heu Khac Le, Tarek F Abdelzaher, Jawe Han, Alce Leung, John Hancock, et al. 01. Tweet rankng based on heterogeneous networks. In Proceedngs of Internatonal Conference on Computatonal Lngustcs, COLING, pages Jeon-Hyung Kang and Krstna Lerman La-ctr: A lmted attenton collaboratve topc regresson for socal meda. In Proceedngs of AAAI Conference on Artfcal Intellgence, AAAI, pages Abhshek Kumar, Pyush Ra, and Hal Daumé III Co-regularzed mult-vew spectral clusterng. In Advances n Neural Informaton Processng Systems, NIPS, pages Yngmng L, Mng Yang, and Zhongfe Mark Zhang Scentfc artcles recommendaton. In Proceedngs of the nd ACM Internatonal Conference on Conference on Informaton & Knowledge Management, CIKM, pages ACM. Jalu Lu, Ch Wang, Jng Gao, and Jawe Han Mult-vew clusterng va jont nonnegatve matrx factorzaton. In Proceedngs of SIAM Data Mnng Conference, SDM. SIAM. Sanjay Purushotham, Yan Lu, and C-c J Kuo. 01. Collaboratve topc regresson wth socal matrx factorzaton for recommendaton systems. In Proceedngs of the 9th Internatonal Conference on Machne Learnng, ICML, pages Ruslan Salakhutdnov and Andry Mnh Probablstc matrx factorzaton. In Advances n Neural Informaton Processng Systems, NIPS, pages Chong Wang and Davd M Ble Collaboratve topc modelng for recommendng scentfc artcles. In Proceedngs of the 17th ACM SIGKDD Internatonal Conference on Knowledge Dscovery and Data Mnng, KDD, pages ACM. Hao Wang, Bny Chen, and Wu-Jun L Collaboratve topc regresson wth socal regularzaton for tag recommendaton. In Proceedngs of the Twenty-Thrd Internatonal Jont Conference on Artfcal Intellgence, IJCAI, pages AAAI Press. 44