Link Prediction in Heterogeneous Networks Based on Tensor Factorization

Transcription

1 Send Odes fo Repints to 36 The Open Cybenetics & Systemics Jounal, 204, 8, Open Access Link Pediction in Heteogeneous Netwoks Based on Tenso Factoization Piao Yong,2*, Li Xiaodong and Jiang He School of Softwae, Dalian Univesity of Technology, Dalian, Liaoning, 6620, P.R. China; 2 School of Electonic and Infomation Engineeing, Dalian Univesity of Technology, Dalian, 6023, P.R. China Abstact: Link Pediction, that is, pedicting the fomation of links o inteactions in a netwok in the futue, is an impotant task in netwok analysis. Link pediction povides useful insights fo othe applications, such as ecommendation system, disease-gene candidate detection and so on. Most link pediction methods assume that thee is only one single type in the netwok. Howeve, many eal-wold netwoks have heteogeneous inteactions. Link pediction in such netwoks is challenging since (a) the netwok has a complicated dependency stuctue; and (b) the links of diffeent types may cay diffeent kinds of semantic meanings, which is impotant to distinguish the fomation mechanisms of each link type. In this pape, we addess these challenges by poposing a geneal method based on tenso factoization fo link pediction in heteogeneous netwoks. Using a CANDECOMP/PARAFAC tenso factoization of the data, we illustate the usefulness of exploing the natual thee-dimensional stuctue of heteogeneous netwok. The expeiment on eal-wold heteogeneous netwok demonstates the effectiveness and efficiency of ou methodology. Keywods: Heteogeneous netwok, link pediction, tenso factoization.. INTRODUCTION In ecent yeas, complex netwok analysis has been eceiving inceasing attention by the scientific community due to the availability of massive netwok data fom divese domains. One fundamental task in netwok analysis is link pediction. Link pediction poblem [] can be boadly defined as: Given a snapshot of a social netwok at time t, we seek to accuately pedict the edges that will be added to the netwok duing the inteval fom time t to a given futue time t. It has been widely studied on homogeneous netwoks ecently. In eality, most netwoks ae intinsically multidimensional; thee might be multiple connections between any pai of nodes. Fo example, the YouTube netwok contains use, video, tag as nodes, with links fom types of follow-use, use-tag-video, and use-subscibe-video and so on. Those netwoks ae boadly defined as heteogeneous infomation netwoks [2] (Fig. ()). Thee ae two typical ways of handling the link pediction poblem in heteogeneous netwoks: teating all types of link equally; studying each type of link independently ignoing its coelation with othe link types [3]. Howeve, some infomation may be lost if diffeent elations ae not taken into account, which esults in the poblem: how to design an effective and geneal method fo link pediction in heteogeneous netwok. Heteogeneous netwoks can be oganized as a thidode tenso ( Node! Node! Link type ) o multi-dimensional *Addess coespondence to this autho at the School of Softwae. Dalian Univesity of Technology, Dalian, Liaoning, 6620, P.R. China; Tel: ; Fax: ; eic_piao@63.com 874-0X/4 Fig. (). Example of Heteogeneous Netwok. aay. In this pape, we descibe and evaluate one appoach based on tenso factoization to the link pediction task in heteogeneous netwoks. To summaize, the following woks have been done in this pape: ) We study the link pediction poblem in netwoks involving heteogeneous inteactions between actos. 2) We popose a method based on tenso factoization that can captue the coelation between diffeent types of links fo the link pediction poblem without loss of infomation. 3) Expeiments on eal data sets have demonstated the effectiveness of ou appoach compaed with taditional link pedictos. The est of the pape is oganized as follows. In Section 2, we biefly eview pevious wok elated to ou eseach. In Section 3, we popose a method based on tenso factoization fo link pediction in heteogeneous netwoks. In Section 4, 204 Bentham Open

2 Link Pediction in Heteogeneous Netwoks Based on Tenso Factoization The Open Cybenetics & Systemics Jounal, 204, Volume 8 37 we epot ou expeiments. We summaize this pape with ou conclusion in Section 5. Typically, we will use small lettes, like x, y z to epesent the node in a netwok and use the lette e to denote the edge. Fo a node x,!( x) epesents the set of neighbos of x. Tensos ae denoted by calligaphic uppe-case lettes ( A, B, ) matices by uppecase lettes (A, B,...), vectos by bold lowe case lettes (a, b,...), scalas by lowe case lettes (a, b,...). 2. RELATED WORK The liteatue on link pediction is vast; hee we will only give a vey bief inteview. The seminal wok of Liben- Nowell and Kleinbeg [] intoduces numeous unsupevised methods fo link pediction. That is by computing gaph based similaity scoes to pedict the link between two vetices, the highe scoe, the moe likely connected between two vetices. In addition to unsupevised appoaches, ecently many supevised methods have been aised [4-7] to extend the seminal wok. Those methods teat link pediction as a binay classification poblem by teating similaity metic as featues. The challenge fo those methods is featue enginee and it's difficult to extact good featues about node and edge in netwok as pivacy estiction. Howeve most of the appoaches intoduced above ae fo homogeneous netwok, few woks have been done fo link pediction in heteogeneous netwok. The link pediction poblem has also been studied in the context of statistical elational leaning [8, 9] and many poposed methods in those woks can handle any elational data sets. The goal of these methods is to pedict whethe the type of links between a pai of objects exists. The advantages of those methods ae that they ae vey geneal to data sets and can exploit the attibutes of the objects. Howeve those methods have a lot of paametes to tune and they ae not vey intuitive fo us. Pehaps the wok by Evim [0] is the closest to ou wok. Howeve, they focus on tempoal link pediction and model on homogeneous netwoks and have not consideed how diffeent elation types affect link fomation. Tenso factoizations have been widely applied to chemometices and ecently eseaches have applied tenso factoizations in web link analysis [], netwok taffic analyzation [2] and also in social netwoks fo analysis of chat oom [3] and communications [4] Moeove those woks based on tenso factoization have achieved good esults. Moe details of tenso factoizations and thei applications can efe this excellent eview pape [5]. 3. LINK PREDICTION BASED ON TENSORFAC- TORIZATION Since ou method is based on tenso factoization, we fist biefly eview this technique. In this pape, we pefom thee-dimensional analysis on the data, attempting to discove the latent factos that goven the associations among these multi-type objects. 3.. Peliminaies Tenso. A tenso is a multidimensional aay. Moe fomally, an N-way o Nth-ode tenso is an element of the tenso poduct of N vecto spaces, each of which has its own coodinate system. A thid tenso has thee indices. A fist ode tenso is a vecto, a second-ode tenso is a matix and tensos of ode thee o highe ae called higheode tensos. Unfolding. Unfolding also known as maticization o flattening, is the pocess of eodeing the elements of an N- way aay into a matix. The mode-d maticization of a tenso X " is denoted by X I! I2! L! IN n. Fo a thee-way X " tenso follows [2]: I! J! K, the mode-n unfolding ae defined as X ( i, p) = X ( i, j, k) () () whee p = j + ( k! )( J ) (2) X ( j, p) = X ( i, j, k) (3) (2) whee p = i + ( k! )( I) (4) X ( k, p) = X ( i, j, k) (5) (3) whee p = i + ( j! )( I ) (6) Konecte Poduct. he symbol! denotes the Konecte poduct of vectos, Fo example: x = a! b " x( k) = a( i) b( j) (7) whee k = j + ( i! ) J fo all " i " I," j " J (8) Khati-Rao Poduct. The symbol denotes the Khati-Rao poduct of two matices. Fo example, let I! K J! K A " and B " : A B = [ A(:,)! B(:) A(:,2)! B(:,2) L A(:, K)! B(:, K)] (9) X " I! I! L! I 2 N Rank-One Tensos. An N-way tenso is ank one if it can be witten as the oute poduct of N vectos, i.e. () (2) ( N ) X = a a L a (0) The symbol epesents the vecto oute poduct Tenso Rank. The ank of a tenso X, denoted ank ( X ), is defined as the smallest numbe of ank-one tensos that geneate X as thei sum 3.2. Data Repesentation A tenso is a multidimensional aay. If thee ae N nodes and K link types, then the data can be epesented as a theeway tenso of size N! N! K whee the (i, j, k) enty is nonzeo if node i is connected to node j by link type k. Fo the heteogeneous netwok example discussed at Sec., thee ae fou links types exist: fiendship connection, colleague connection, elative connection and the same name connection.

3 38 The Open Cybenetics & Systemics Jounal, 204, Volume 8 Yong et al Tenso Decomposition based Link Pediction Model We employ CANDECOMP/PARAFAC(CP) tenso decomposition [6] to captue the undelying pattens in the node-elationship-node tenso. The idea of the CP decomposition is factoizing a tenso into a sum of component ank one tensos, it is a highe-ode analog of the matix singula value decomposition (SVD). Given a thid-ode tenso I J K X " R!! we can wite it as: R # X "! a b c () = The intege denotes oute poduct of vectos, R is a positive I T J T and a! : a a =, b! : b b = and K T c! : c c =,! " is used to nomalize the columns of the matices A, B, C to length one. In contast to the solution matix of SVD, the columns of facto the matices A, B, C need not to be othonomal. Fig. (2) Illustates the decomposition. \illustates the decomposition. Fig. (2). CP Tenso Decomposition. The CP decomposition can geneate featue vectos fo the nodes in the gaph, which can be computed to get a similaity scoe that combines the multiple types of the gaph. Afte CP decomposition, we get thee facto matices: node matix A, elationship matix B, and node matix C. Link pediction can be computed accoding to the captued associations. We define a scoe matix S as follows: R S = "! a c = T (2) S Matix measues the latent elations between diffeent nodes in which S ij epesents the likelihood that node i will connect node j Computing the CP Decomposition In this pape, we use altenating least-squaes (ALS) with weighted-! -egulaization algoithm to fit the CP decomposition. Let X "!! be a thid-ode tenso. The goal is to I J K compute a CP decomposition with R components that best appoximates X, i.e., to find. R # min X " X with X =! a b c = [[!; A, B, C]] (3) X = The poblem of computing the CP decomposition that best appoximates X can be fomulated as a least squaes optimization poblem: min f ( A, B, C) =![[ A, B, C]] " ()! A( B C) T 2 X F X F (4) The ALS appoach fixes B and C to solve A by minimizing the objective function (the sum of squaed eos) the optimal A solution can be solved using the pseudo-invese of a Khati-Rao poduct: A X C B C C B B T T = () ( )(( ).*( )) (5) We can solve B and C using the same patten. The ALS algoithm fo CP decomposition is pesented in Algoithm : 4. EXPERIMENTS In this section, we show that ou poposed method can impove link pediction accuacy compaed with the methods that only use homogeneous object and link infomation. Fist we will eview some homogeneous methods as baseline. 4.. Baseline Methods All the methods can be viewed as computing a measue of poximity of similaity between nodes x and y. They assign a weight scoe( x, y ) to pais of nodes < x, y >, based on the netwok topology, and then poduce a anked list in deceasing ode of scoe( x, y ). Node Neighbohood Methods. Fo a node x, let!( x) denotes the set of neighbos of x. If two nodes neighbos have lage ovelap, they ae moe likely to fom a link in the futue.

4 Link Pediction in Heteogeneous Netwoks Based on Tenso Factoization The Open Cybenetics & Systemics Jounal, 204, Volume 8 39 Common neighbos. The moe common neighbos between x and y, the chance that x and y have a link between them inceases. The scoe defined as below: scoe( x, y) : =!( x) "!( y) (6) Newman [7] has computed this quantity in the context of collaboation netwoks to show that a coelation exists between the numbes of common neighbos of x and y at time t, and the pobability that they will collaboate in the futue. Jaccad's coefficient:!( x) "!( y) scoe( x, y) : = (7)! ( x) #! ( y) Adamic/Ada measue. Adamic and Ada [8] poposed this scoe as a metic of similaity between two web pages.! ( ( )) (8) z: featueshaed by x, y log fequency z Fo link pediction, Liben-Nowell and Kleinbeg [] efined this metic and assigned lage weight to common neighbos z of x and y which themselves have few neighbos!( z) The scoe defined as below: scoe( x, y) : = $ (9) log "( z) z!" ( x) #"( y) Pefeential attachment: The pemise of pefeential attachment is that nodes like to fom ties with popula nodes. A new edge has node x as its endpoint is popotional to!( x). The scoe defined as below: scoe( x, y) : =!( x)!( y) (20) Path Methods: katz measue Sums ove all possible paths between x and y, giving highe weights to shote paths. The scoe defined as below: " l scoe( x, y) = #! paths (2) l= ( l) x, y ( l) whee! > 0 and paths is the set of all length-l paths fom x to y. x, y SimRank is defined by the following ecusive pocedue: two nodes ae simila to the extent that thei neighbos. Specially, similaity( x, x ) = similaity( x, y) =! $ $ a"# ( x) b"# ( y) similaity( a, b) #( x) #( y) (22) whee! "[0,], fo link pediction scoe(x, y) = similaity (x, y) Data Sets UMLS. This data set contains data fom the Unified Medical Language System semantic wok gatheed by [9]. This consists of 35 entities and 54 elationships. The entities ae high-level concepts like 'Disease o Syndome', 'Diagnostic Pocedue', o 'Mammal'. The elation vebs illustate the elationship between those concepts Paamete Settings We use the top-k ecommendations metic [20] to evaluate link pediction esults. That is, we will pedict top k links fo each use. The pocedue is as below: fo each use, we andomly select one linked use and k- unlinked to fom the test set and the emaining linked use fom the taining set. We epeat this pocedue 60 times fo diffeent samplings. The goal is to find the position of the linked use in the ecommendation list. Thee ae k possible anks fo the linked use and the best esult is that no unlinked uses appea befoe it Dealing with High Computing Cost It's cost intensively to compute tenso factoization. So in this pape, we conducted ou expeiments on a Linux Ubuntu seve with fou Intel Xeon CPUs, each CPU has two coes, and each coe has 3.5 MB cache. We paallel [2] ou Matlab code, and this significantly impove the speed of the algoithm. We found that the ALS based tenso factoization pefoms bette Results and Discussions To demonstate the usefulness and effectiveness of ou model, we offe compaison expeiments on unsupevised link pediction methods. Accoding to [, 22]. Adamic/Ada measue and Katz measue pefoms well both in theoetical and pactical expeiments. So hee, we just compae these two measues with ou methods. Figs. (3 and 4) show the cumulative distibutions of anks fo the connected uses in the test set (k=8). 0% means that the connected use is at the fist place in the odeed list, while 00% means that it is in the last position. Compaing the esults, in Fig. (3) ou method povides bette pecision than unsupevised ones on the data sets. And in Fig. (4) ou method povides competitive effect to Adamic/Ada measue and both those two methods beat Katz measue. The eason fo that can be explained that ou model can captue the latent multi-elationships of data. 5. CONCLUSION In this pape, we study the poblem of link pediction in heteogeneous netwoks. Heteogeneous netwoks contain multi-elational links between entities and taditional matix based methods ae not expessive enough to model heteogeneous netwoks without infomation loss. We popose a new geneal and effective method based on tenso factoization to addess this poblem. Expeiments on the eal wold netwok show that by consideing multiple elations, the link pediction accuacy can be significantly impoved. In the futue, we will ty to scale ou model fo vey lage scale heteoge-

5 320 The Open Cybenetics & Systemics Jounal, 204, Volume 8 Yong et al. Fig. (3). Top-8 Link Pediction Pefomance fo UML Data (Result ). Fig. (4). Top-8 Link Pediction Pefomance o UML Data (Result 2). neous netwoks, such as utilizing the Hadoop platfom to speed up tenso factoization. CONFLICT OF INTEREST The authos confim that this aticle content has no conflict of inteest. ACKNOWLEDGEMENTS This wok was financially suppoted by the National Science Foundation of softwae bug waehouse pioity mining poject (No ). ABOUT THE AUTHORS Fist Autho Zhao Jing, Univesity lectue of Shenyang Univesity of Technology, Ph.D. The autho s majo is Fluid Machiney and Engineeing. 6 papes eceived by EI o SCI and 2 patents had been published. Second Autho Wang Jing, maste degee in engineeing, studying fo PhDs in Beijing Univesity of Chemical Technology. The autho s majo is Mechatonic Engineeing. Thid Autho Wang Shijie is a membe of the IEEE and the IEEE Compute Society.

6 Link Pediction in Heteogeneous Netwoks Based on Tenso Factoization The Open Cybenetics & Systemics Jounal, 204, Volume 8 32 REFERENCES [] D. Liben-Nowell and J. Kleinbeg. The link pediction poblem fo social netwoks. Ameican Society fo Infomation Science, vol.58 no. 7, pp , [2] J. Han. Mining heteogeneous infomation netwoks by exploing the powe of links. In Discovey Science, vol. 2, pp. 3-30, [3] D. Davis, R. Lichtenwalte, and N. Chawla. Multi-elational link pediction in heteogeneous infomation netwoks. In Advances in Social Netwoks Analysis and Mining, vol. 8, pp , 20. [4] L. Backstom and J. Leskovec. Supevised andom walks: pedicting and ecommending links in social netwoks. In Poceedings of the fouth ACM intenational confeence on Web seach and data mining, WSDM ', New Yok, 20, pp [5] A. Popescul and L. Unga. Statistical elational leaning fo link pediction. In IJCAI wokshop on leaning statistical models fom elational data, vol. 2003, pp , [6] M. Pujai and R. Kanawati. Supevised ank aggegation appoach fo link pediction in complex netwoks. In Poceedings of the 2st intenational confeence companion on Wold Wide Web, Hong Kong, 202, pp [7] N. Benchettaa, R. Kanawati, and C. Rouveiol. Supevised machine leaning applied to link pediction in bipatite social netwoks. In Advances in Social Netwoks Analysis and Mining, Chicago, 200, pp [8] M. A. Hasan, V. Chaoji, S. Salem, and M. Zaki. Link pediction using supevised leaning. In Poc. of SDM 06 wokshop on Link Analysis, Counteteoism and Secuity, New Yok, 2006, pp [9] Z. Xu, V. Tesp, S. Yu, and K. Yu. Nonpaametic elational leaning fo social netwok analysis. In KDD 2008 Wokshop on Social Netwok Mining and Analysis, Washington, 2008, pp [0] E. Aca, D. M. Dunlavy, and T. G. Kolda. Link pediction on evolving data using matix and tenso factoizations. In ICDMW'09: Poceedings of the 2009 IEEE Intenational Confeence on Data Mining Wokshops, New Yok, 2009, pp [] T. G. Kolda, B. W. Bade, and J. P. Kenny. Highe-ode web link analysis using multilinea algeba. In IEEE Intenational Confeence On Data Mining, 2005, Chicago, pp [2] J. Sun, D. Tao, and C. Faloutsos. Beyond steams and gaphs: dynamic tenso analysis. In Poceedings of the 2th ACM SIGKDD intenational confeence on Knowledge discovey and data mining, Beijing, 2006, pp [3] E. Aca, S. A. ampee. Collective sampling and analysis of high ode tensos fo chatoom communications. In ISI 2006: IEEE Intenational Confeence on Intelligence and Secuity Infomatics, Hong Kong, 2006, pp [4] B. Bade, M. Bey, and M. Bowne. Discussion tacking in enon using paafac. Suvey of Text Mining, vol. 2, pp , [5] T. G. Kolda and B. W. Bade. Tenso decompositions and applications. SIAM Review, vol.5, no. 3, pp , Sep [6] J. Caoll and J. Chang. Analysis of individual divegences in multidimensional scaling via an n-way genealization of "eckatyoung" decomposition. Psychometika, vol. 35, no. 3, pp , 970. [7] M. Newman. Clusteing and pefeential attachment in gowing netwoks. Physical Review E, vol.64, no. 2, pp , 200. [8] L. Adamic and E. Ada. Fiends and neighbos on the web. Social netwoks, vol. 25, no. 3, pp , [9] A. T. McCay. An uppe-level ontology fo the biomedical domain. Compaative and Functional Genomics, vol. 4, no., pp , [20] Y. Koen. Factoization meets the neighbohood: a multifaceted collaboative lteing model. In Poceeding of the 4th ACM SIGKDD intenational confeence on Knowledge discovey and data mining, Hong Kong, 2008, pp [2] J. Kepne. Paallel MATLAB fo multicoe and multinode computes, Society fo Industial Mathematics, vol.2, pp , [22] P. Saka, D. Chakabati, and A. Mooe. Theoetical justification of popula link pediction heuistics. In Poceedings of the Twenty- Second intenational joint confeence on Atificial Intelligence, New Yok, 20, pp Received: Septembe 22, 204 Revised: Novembe 30, 204 Accepted: Decembe 02, 204 Yong et al.; Licensee Bentham Open. This is an open access aticle licensed unde the tems of the Ceative Commons Attibution Non-Commecial License ( licenses/by-nc/3.0/) which pemits unesticted, non-commecial use, distibution and epoduction in any medium, povided the wok is popely cited.