Link prediction in graph construction for supervised and semi-supervised learning

Transcription

1 Link prediction in graph construction for supervised and semi-supervised learning Lilian Berton, Jorge Valverde-Rebaza and Alneu de Andrade Lopes Laboratory of Computational Intelligence (LABIC) University of São Paulo (USP) Brazil July 2015

2 Outline 1 Introduction 2 Proposal 3 Experiments 4 Conclusion Jorge Valverde-Rebaza Link prediction in graph construction 2 / 20

3 Outline 1 Introduction 2 Proposal 3 Experiments 4 Conclusion Jorge Valverde-Rebaza Link prediction in graph construction 3 / 20

4 Motivation Networks or graphs are a powerful relational representation that has been employed in different tasks of machine learning. Link prediction is an important scientific issue regarding network analysis that has attracted increasing attention in recent years. Many social, biological and information systems can be naturally described as networks, while some data are flat data. To apply graph-based methods to flat data is necessary to convert the data into a network, furthermore converting flat data to relational data can help to improve classification accuracy. Despite many methods for graph construction have been proposed, it is still an open problem. Jorge Valverde-Rebaza Link prediction in graph construction 4 / 20

5 Objective and hypothesis Propose a new method for graph construction using the link prediction intuition. If a network is very sparse, for example when a minimum spanning tree is applied, it misses structural information for the inference algorithms. If a network is very dense, for example when knn considering k > 10 is applied, the excess edges become noise in the graph. Considering a basic graph structure is possible add predicted edges, generating a new (balanced) graph structure. It can improves the quality of graphs leading to better classification accuracy in supervised and semi-supervised domains (SSL). Jorge Valverde-Rebaza Link prediction in graph construction 5 / 20

6 Graph Construction Many data sets are available in tabular flat format. It is necessary to convert the data into a network to be able to apply a graph-based algorithm. We apply k-nearest neighbor (knn), Mutual knn, Minimum/Maximum spanning tree (Min/MaxST) to generate an initial graph. (a) 3NN (b) M3NN (c) MinST (d) MaxST Figure: Graph construction methods. Jorge Valverde-Rebaza Link prediction in graph construction 6 / 20

7 Link Prediction (LP) Link prediction (LP) addresses the problem of predicting the existence of missing relations or new ones. Common Neighbors (c): s c v i,v j = Γ(v i ) Γ(v j ) Weighted CN (w): s w v i,v j = v k Γ(v i ) Γ(v j ) w(v i, v k ) + w(v k, v j ) Katz (k): s k v i,v j = l=1 βl paths l v i,v j = βa vi,v j + β 2 (A 2 ) vi,v j +... Figure: Link prediction process. Jorge Valverde-Rebaza Link prediction in graph construction 7 / 20

8 Outline 1 Introduction 2 Proposal 3 Experiments 4 Conclusion Jorge Valverde-Rebaza Link prediction in graph construction 8 / 20

9 Proposal To predict new links is assigned a score s vi v j for each pair of disconnected vertices v i and v j. All non-observed links are ranked according to their scores, and the links connecting more similar nodes are supposed to be of higher existence likelihoods. A percentage of the top ranked links can be considered. (a) Dataset (b) MinST (c) MinST+LP (Katz- 30%) Figure: LP construction steps. Jorge Valverde-Rebaza Link prediction in graph construction 9 / 20

10 Outline 1 Introduction 2 Proposal 3 Experiments 4 Conclusion Jorge Valverde-Rebaza Link prediction in graph construction 10 / 20

11 Datasets Table: Data sets descriptions for SSL classification Data set # Instances # Attributes # Classes g241c g241n Digit USPS COIL Table: Data sets descriptions for supervised classification Data set # Instances # Attributes # Classes Wine Ecoli Customers Cancer Blood Gaussians Gaussians Jorge Valverde-Rebaza Link prediction in graph construction 11 / 20

12 SSL experimental setup PCA was applied reducing the dimensions to 50 since in high-dimensional data the distance to the nearest neighbor approaches the distance to the farthest neighbor which degenerates the quality of the graph. 10 and 100 labeled vertices were randomly selected. We apply MinST, MaxST, knn and MkNN with 1 k 20, and the LP graphs (our proposal) considering the same methods combined with a LP measure: MinST+LP, MaxST+LP, knn+lp and MkNN+LP with 1 k 5. The weighted graph W uses the binary weighting approach. The algorithm used for the label inference task was the Local and Global Consistency (LGC). The average accuracy of 30 runs was used as evaluation. Jorge Valverde-Rebaza Link prediction in graph construction 12 / 20

13 Supervised experimental setup For Cancer dataset the instances with missing values were also removed. We apply MinST, MaxST, knn and MkNN with 1 k 20, and the LP graphs (our proposal) considering the same methods combined with a LP measure: MinST+LP, MaxST+LP, knn+lp and MkNN+LP with 1 k 3. The weighted graph W uses the opposite of Euclidean Distance. The relational algorithms used for the classification were: nobayes, nolb-lr-binary, nolb-lr-count, nolb-lr-mode, prn. The accuracy of 10-fold cross validation was used as evaluation. Jorge Valverde-Rebaza Link prediction in graph construction 13 / 20

14 Results CD knn+lp MkNN knn MinST+LP MaxST MinST MaxST+LP MkNN+LP Figure: Nemenyi post-hoc test for semi-supervised classification. CD knn+lp knn MkNN MinST+LP MaxST MkNN+LP MinST MaxST+LP Figure: Nemenyi post-hoc test for supervised classification. Jorge Valverde-Rebaza Link prediction in graph construction 14 / 20

15 Parameter analysis Figure: Distribution of parameters k and top percentage of links used for the graph construction methods in the supervised classification. Jorge Valverde-Rebaza Link prediction in graph construction 15 / 20

16 Average degree Average degree knn MkNN MST knn+lp Mk NN+LP MST+LP k or % of links * 10 Figure: Average degree for knn, MkNN, MST and LP versions: knn+lp, MkNN+LP, MSt+LP applied to Gaussians3 data set. LP versions use k = 3 and the common neighbors measure. Jorge Valverde-Rebaza Link prediction in graph construction 16 / 20

17 Outline 1 Introduction 2 Proposal 3 Experiments 4 Conclusion Jorge Valverde-Rebaza Link prediction in graph construction 17 / 20

18 Conclusions Link prediction (LP) has been used in many fields of science, as online social networks where links can be recommended as promising friendships. Here LP was used for graph construction: from an initial graph structure edges are predict generating a new balanced graph. The proposed graphs were evaluated in supervised and semi-supervised classification providing improvements in accuracy. The graphs are sparse and represent well the neighborhood of a point. In future work, other baseline methods could be tested as well other measures for LP. Our approach also could be applied in other domains of machine learning using graph-based methods. Jorge Valverde-Rebaza Link prediction in graph construction 18 / 20

19 References Berton, L. and Lopes, A. (2014). Graph construction based on labeled instances for semi-supervised learning. In Proceedings of 22nd ICPR, pages Liben-Nowell, D. and Kleinberg, J. (2007). The link-prediction problem for social networks. JASIST, 58(7): Lü, L. and Zhou, T. (2011). Link prediction in complex networks: A survey. Physica A: Statistical Mechanics and its Applications, 390(6): Macskassy, S. A. and Provost, F. J. (2007). Classification in networked data: A toolkit and a univariate case study. JMLR, 8: Rohban, M. H. and Rabiee, H. R. (2012). Supervised neighborhood graph construction for semi-supervised classification. Pattern Recognition, 45(4): Valverde-Rebaza, J. and Lopes, A. (2012). Link prediction in complex networks based on cluster information. In SBIA 12, pages Valverde-Rebaza, J. and Lopes, A. (2013). Exploiting behaviors of communities of Twitter users for link prediction. SNAM, 3(4): Jorge Valverde-Rebaza Link prediction in graph construction 19 / 20

20 Thank you Jorge Valverde-Rebaza