arxiv: v1 [cs.cr] 3 Jun 2017

Transcription

1 Spectrm-based deep neral networks for frad detection arxiv: v1 [cs.cr] 3 Jn 2017 ABSTRACT Shhan Yan Tongji University 4e66@tongji.ed.cn Jn Li University of Oregon lijn@cs.oregon.ed In this paper, we focs on frad detection on a signed graph with only a small set of labeled training data. We propose a novel framework that combines deep neral networks and spectral graph analysis. In particlar, we se the node projection (called as spectral coordinate) in the low dimensional spectral space of the graph s adjacency matrix as inpt of deep neral networks. Spectral coordinates in the spectral space captre the most sefl topology information of the network. De to the small dimension of spectral coordinates (compared with the dimension of the adjacency matrix derived from a graph), training deep neral networks becomes feasible. We develop and evalate two neral networks, deep atoencoder and convoltional neral network, in or frad detection framework. Experimental reslts on a real signed graph show that or spectrm based deep neral networks are effective in frad detection. KEYWORDS frad detection, spectrm, deep neral networks 1 INTRODUCTION Online social networks (OSNs) have become poplar social services for linking people together. Unfortnately, de to the openness of OSNs, fradsters can also easily register themselves, inject fake contents, or take fradlent activities, imposing severe secrity threads to OSNs and their legitimate participants. Many frad detection techniqes have been developed in recent years [1, 6, 9, 14], inclding content-based approaches and graph-based approaches. Different from content-based approaches that extract content featres, (i.e., text, URL), from ser activities on social networks [4], graph-based approaches identify frads based on network topologies. Often based on nspervised learning, the graph-based approaches consider frad as anomalies and extract varios graph featres associated with nodes, edges, ego-net, or commnities from the graph [2, 13]. In practice, a small set of labeled sers are often available and hence spervised learning based detection approaches cold be developed. In this paper, we introdce deep neral network models for detecting frads in signed graphs. Deep neral networks have achieved remarkable reslts in compter vision, natral langage processing, and speech recognition areas [7, 8, 12]. A deep neral network can learn different levels of representations on different layers of neral network [5]. However, one challenge of applying deep neral networks for frad detection is lack of sfficient labeled data. When deep neral networks with a high dimensional inpt Xintao W University of Arkansas xintaow@ark.ed Aidong L University of North Carolina at Charlotte aidong.l@ncc.ed have a large nmber of parameters, the deep neral networks need to be trained with a large training dataset [12]. Hence it is often infeasible to se the adjacency matrix of the nderlying graph as inpts of deep neral network models becase of the high dimension of the adjacency matrix and the small nmber of labeled sers. We propose a novel framework that combines spectral graph analysis with the deep neral networks. In particlar, we first project a graph to its spectral space formed by the principal eigenvectors of its adjacency matrix. The spectral space captres the main topological information of the graph. Each node is then mapped to a low dimensional point (called spectral coordinate) in the spectral space. We then se each node s spectral coordinate together with the aggregate information of its neighbor nodes spectral coordinates as the inpt of two deep neral network models, deep atoencoder and convoltional neral network. The advantages of or framework over past efforts are as follows. First, sing both spectral graph analysis and deep neral networks, we can avoid defining graph metrics (featres) to identify the difference between fradsters and reglar sers. Second, the low-dimensional spectral space contains the most sefl topology information of a graph. Comparing with the adjacency matrix, the dimension of spectral coordinates of nodes is mch lower. Ths, sing the node spectral coordinates as inpts to deep neral networks is sitable for real cases where the labeled sers are limited. Moreover, most of the existing works for frad detection focs on nsigned graphs in which there are only one type of links, while or framework covers signed networks. In order to captre both positive and negative edge information of a node in the signed graph, inpts of the two deep neral networks are composed by combining spectral coordinates of the node and its positive/negative-connected neighbors. 2 MODELS 2.1 Framework Given a signed ndirected graph G, each node in G indicates either a reglar ser or fradster. The signed graph G can be represented as a symmetric adjacency matrix A n n, where n is the nmber of nodes. In A n n, a ij = 1 (a ij = 1) indicates there is a positive (negative) edge between nodes i and j and a ij = 0 indicates no edge. A has n real eigenvales. Let λ i be the i-th largest eigenvales of A with eigenvector v i, λ 1 λ 2 λ n. The spectral decomposition of A is A = i λ i v i v T i (shown in Figre 1). There sally exist k leading eigenvales that are significantly greater than the rest ones for networks. The row vector α = (v 1,v 2,...,v k ) is

2 the spectral coordinate of node in the k-dimensional sbspace spanned by (v 1,..., v k ). Figre 1: Spectral decomposition of the adjacency matrix A In this work, we adopt deep atoencoder and convoltional neral network to identify fradsters. Instead of sing adjacency matrix, we adopt spectral coordinates to represent nodes as inpts to the two deep neral networks since spectral coordinates of nodes preserve the most sefl strctre information abot nodes. Meanwhile, given a node in a graph G, it has neighbors in s-steps. For example, when s = 1, the 1-step neighbors indicate the neighbors are one step away from the node. Spectral coordinate of s neighbors in s-steps represent broader topological information abot the node. For example, spectral coordinates of 1-step neighbors have been sccessflly sed to detect random link attacks from nsigned graphs [14]. Ths, we frther adopt the spectral coordinates of node neighbors in s-steps. In a signed graph, neighbors of node can be divided into 2 categories based on their edge types, i.e., neighbors connected by positive edges and neighbors connected by negative edges. We compte the mean vector of s-step neighbors spectral coordinates for each category, denoted as β s + where s indicates the s-step neighbors. Then, given a node, to captre a broad strctre information of, the final inpts of two deep neral networks combine the spectral coordinates of node and its two categories of neighbors in s-steps. We se a small part of labeled nodes to train the deep atoencoder and convoltional neral network. After training, the deep neral networks are able to identify fradsters of the rest nodes in the signed graph. 2.2 Using deep atoencoder (DAE) for frad detection and β s, DAE stacks mltiple basic atoencoder blocks hierarchically, which can captre mltiple levels of representations of the inpt data Figre 2: Architectre of DAE with spectral coordinates of node and its 1-step neighbors for frad detection 2 [3]. We adopt spectral coordinates of nodes in a signed graph as inpts to DAE. DAE can preserve the hidden knowledge abot the node from its spectral coordinate. Training DAE for frad detection contains two phases: the pre-training phase and training phase. In the pre-training phase, DAE trains the model in an nspervised manner. In the training phase, DAE trains the classifier and finetnes the whole model to predict the labels of nodes. Given the spectral coordinates of node and its two categories of neighbors in s-steps, the inpt x R (2s+1)k of DAE is defined as: x = [α β 1+ β 1... β s + β s ]. (1) Given a DAE, there are L encoders to compte the hidden representations z (1), z (2),..., z (L) of x layer by layer. The inpt of the (l + 1)-th encoder is the otpt of the l-th encoder. Specially, the inpt of the first encoder is x. Then, there are another L decoders to compte the reconstrcted inpt ˆx. The eqations of encoder and decoder are shown in Eqation 2 and 3, respectively. ẑ (L+l) z (l) = σ(w (l) z (l 1) + b (l) ), (2) = σ(w (L+l) ẑ (L+l 1) + b (L+l) ), (3) where W, b are the parameters of the encoder; σ is a nonlinear activation fnction. The objective fnction of pre-training is to make the reconstrcted inpt ˆx to be close to the original inpt x by minimizing the reconstrction sqared error, L(x, ˆx) = ˆx x 2. (4) After pretraining DAE, we stack L encoders layer by layer to generate the hidden representation z (L). z (L) captres the hidden information of x since it can be sed to reconstrct the inpt. Then, a softmax classifier is applied on top of the z (L) to predict the label of node. P(ŷ = c z (L) exp ( T c z (L) + b c ) ) = Cc =1 exp(t c z (L), (5) + b c ) where C is nmber of classes, c and b c are the parameters of softmax fnction for the c-th class. The parameters of softmax and deep atoencoder are trained and fine-tned by minimizing the cross entropy loss fnction, L = 1 N y i log(p(ŷ i )), (6) N i=1 where y i is the tre class of the i-th inpt, and N is the nmber of training data. Figre 2 shows the architectre of DAE with spectral coordinate of node and its 1-step neighbors. 2.3 Using convoltional neral network (CNN) for frad detection Convoltional neral networks are widely-sed in compter vision area[10]. A basic convoltional neral network is composed by a convoltion operation and a pooling operation. Given a node, the inpt of CNN X R (2s+1) k is represented as X = [α ; β 1+ ; β 1 ;... ; β s + ; β s ], (7)

3 Figre 3: Architectre of CNN with spectral coordinates of node and its 1-step neighbors for frad detection where ; indicates the vertical concatenation of two vectors. A convoltion operation which involves a filter W R m k is applied on a sb-matrix of X with m continos rows to generate a hidden featre: h j = σ((w X ) j:j+m 1 + b), (8) where σ indicates a nonlinear fnction; is a convoltion operation; b is a bias parameter; and m 2s + 1. Initially, the filter is partially connected to the inpt matrix X. Then, the filter slides throgh the whole inpt matrix X and generates a featre vector h = [h 1,,h 2s m+2 ]. After that, an average pooling operation, which calclates the average vale of the featre vector h, is sed to captre all discriminative featres of X. The average pooling operation is defined as z = mean(h). For example, when the filter size m is 1, the convoltion operation generates a featre vector h R 2s+1 which contains the hidden information of node and its all positive/negative-connected neighbors in s-steps. Then, an average pooling operation is applied on h to get the hidden featre z by calclating the average vale of h as the otpt of crrent filter. When the filter size m > 1, the model follows the similar procedre to generate hidden featres from X. Meanwhile, becase one hidden featre vector h is generated by one filter, the featre vector detects the same featre from different locations of X. To identify different aspects of featres from X, the model applies mltiple filters with different sizes of m to generate different featre vectors. After applying the pooling operation on each featre vector, the model encodes the inpt X to a representation vector z = [z 1, z 2,, z q ], where q is the nmber of featre vectors generated by q different filters. For each filter size m, CNN generates the same nmber of featre vectors. We then apply a softmax classifier on z to predict the node label P(ŷ = k z) by Eqation 5. The whole model is trained by minimizing the loss fnction shown in Eqation 6. The architectre of CNN with spectral coordinates of node and its 1-step neighbors is shown in Figre 3. 3 EXPERIMENTS To evalate the effectiveness and efficiency of or approach, we condct experiments on a signed graph for frad detection. 3.1 Experimental Setp Datasets. We condct or evalation on a signed network, WikiEditor, which is extracted from the UMD Wikipedia dataset[11]. The dataset is composed by vandals and benign sers who 3 edited the Wikipedia pages from Jan 2013 to Jly Different from benign sers, vandals edit articles in a deliberate attempt to damage Wikipedia. One edit may be reverted by bots or editors. Hence, each edit can belong to either revert or no-revert category. WikiEditor is bilt based on the co-edit relations. In particlar, a positive (negative) edge between sers i and j is added if the majority of their co-edits are from the same category (different categories). We remove those edits on meta pages (i.e., with titles containing User:, Talk:, User talk:, Wikipedia: ) becase the editings on those pages are not reverted by bot or administrators. We frther remove from or signed network those sers who do not have any co-edit relations with others. In WikiEditor, each ser is clearly labeled as either benign or vandal. Hence, we can evalate or models for frad detection on WikiEditor. Table 1: Statistics of WikiEditor # of Users (+,-) # of Links (+,-) WikiEditor (6086, 12906) (52139, 29177) Experimental settings. In or experiments, after projecting the signed graph to the spectral space, we first normalize the spectral coordinates of nodes. We then combine nodes and their 1-step neighbors spectral coordinates as inpts to deep neral networks. DAE contains two encoders. The dimensions of two encoders are 128 and 64. In CNN, the filter size m [1, 2, 3] and the nmber of filters q is 300. The training epochs of DAE and CNN models are 30 with early stopping. We randomly sample different percentages of nodes for training and se the rest of the nodes for testing. Since the labeled fradsters in real cases are sally small, we only adopt a small percentage of nodes as training data. We se the accracy to evalate the performance of different approaches for vandal detection. We report the mean vales of 10 different rns by sampling different training data. Baselines. We compare deep neral networks with two classical classifiers, k-nn and SVM. In k-nn, we set k = 3. In SVM, we adopt the RBF kernel and set the reglarization parameter c = 1. The inpts of k-nn and SVM are the same as DAE. Table 2: Accracy of vandal detection with varios sizes of training dataset Inpt A Algorithm Ratio of the training dataset 5% 10% 15% 20% k-nn 66.16% 68.82% 69.66% 74.00% SVM 67.81% 67.82% 67.88% 67.92% DAE 76.31% 78.55% 79.56% 80.59% CNN 76.70% 78.95% 80.09% 81.33% x SVM 71.60% 80.40% 80.82% 81.15% DAE 80.89% 81.13% 81.45% 81.92% k-nn 76.60% 77.38% 77.83% 78.19% CNN 80.57% 81.40% 82.02% 82.61%

4 3.2 Reslts We first compare each model with different sizes of training datasets sing spectral coordinates and the adjacency matrix. The defalt dimension of spectral coordinate k is 30. When sing the adjacency matrix A, the inpt of each algorithm is each row of A. In this scenario, CNN has only one size filter w 1 n. Deep neral networks vs. Baselines. Table 2 shows the accracy of vandal detection sing the adjacency matrix A and spectral coordinates x. We can observe that DAE and CNN models otperform the baselines significantly in all settings at the 10 8 level with a t-test. In the spectral space, the performance of SVM has a big jmp when the percentage of training data increasing from 5% to 10%. In the adjacency matrix space, the performances of SVM do not improve while increasing the size of training data. It indicates that the SVM cannot be well-trained with a small training dataset, especially when the inpt has high dimensions. The accracies of k-nn increase steadily while increasing the training data, bt the accracies of k-nn are also mch worse than or DAE and CNN. Spectrm vs. Adjacency matrix. In Table 2, we frther observe that sing the spectral coordinates as inpts (x ) achieves significantly better performance than sing the adjacency matrix, especially when the percentages of training data are 5% and 10%. Meanwhile, in the spectral space, when the percentage of training dataset is 5%, DAE performs better than CNN since DAE pre-trains the model first. The pre-training phase of DAE encodes the information of nodes into hidden layers, which make the classifier predict the node labels with small training data. On the contrary, when the percentage of training dataset is larger than 5%, CNN otperforms DAE in the spectral space. It indicates CNN has better performance with enogh training data. In the adjacency matrix space, the performances of DAE are worse than CNN with varios sizes of training data. This is becase DAE has mch more parameters than CNN when sing the adjacency matrix. DAE cannot be well-trained in a high-dimensional space with a small training dataset. Effect of the dimension of spectral coordinate k. Table 3 compares the deep neral networks with k-nn and SVM on varying the dimension of spectral coordinate k. In this experiment, we se 20% of nodes as training data and the rest of nodes as testing data. When inpts of algorithms are x, we can observe that DAE and CNN models otperform classical classifiers with varios dimensions of spectral coordinates. We can frther discover that DAE and CNN also achieve the most stable accracy with varios dimensions of spectral coordinates. However, the performance of SVM significantly drops while increasing the dimension of the spectral coordinate. This is becase both DAE and CNN learn the hidden representations of nodes from their spectral coordinates. The hidden representations of nodes are sefl for predicting the labels. Neighbor inclsion vs. Neighbor exclsion. In or experiment, the inpts of DAE and CNN combine spectral coordinates of nodes and their 1-step neighbors. We frther compare the performance of algorithms that adopt the node spectral coordinate with and withot combining the 1-step neighbors spectral coordinates as inpts in Table 3. We can observe that when sing the information of the 1- step neighbors spectral coordinates, the accracies of all algorithms achieve arond 1%-2% improvement. Therefore, combining the 4 Table 3: The accracy of vandal detection with varios dimensions of spectral coordinate k when 20% of nodes are sed as the training dataset. We frther compare algorithms sing node spectral coordinate with/withot combining neighbor spectral coordinates as inpts to algorithms. x Inpt Algorithm Dimension of spectral coordinate k SVM 81.00% 81.40% 81.15% 80.70% 76.28% DAE 81.86% 81.65% 81.92% 81.87% 81.61% k-nn 79.90% 78.85% 78.19% 78.33% 77.52% CNN 82.24% 82.26% 82.61% 82.42% 82.47% α SVM 79.79% 79.65% 80.19% 80.31% 77.94% DAE 80.47% 81.10% 81.17% 81.20% 81.40% k-nn 78.00% 77.49% 76.75% 76.55% 76.92% CNN 80.52% 81.22% 81.48% 81.44% 81.39% information of node neighbors can improve the performance of frad detection. Exection time. We also compare the exection time of deep neral networks sing spectral coordinates and the adjacency matrix. The DAE and CNN models are trained on a Nvidia Tesla K20 GPU. We observe that when the ratio of training data is 20%, the training time of each epoch for CNN and DAE models with adjacency matrix is 2 seconds. On the contrary, the training time of each epoch for CNN and DAE models with spectral coordinates is less than 1 second. Therefore, sing the spectral coordinates with low dimension is also more efficient than sing the adjacency matrix. 4 CONCLUSIONS We have presented a novel framework that applies deep neral networks on the spectral space of a signed graph to identify frads. In particlar, we first condct graph spectral projections on a signed graph to obtain node spectral coordinates. The node and its s-step neighbors spectral coordinates are combined together as inpts to the deep atoencoder and convoltional neral network models for frad detection. The experiment reslts show that both deep neral networks achieve promising reslts on frad detection. Or empirical evalation frther shows that combining the information of node neighbors can improve the effectiveness of deep neral networks on frad detection. ACKNOWLEDGMENTS The athors acknowledge the spport from the 973 Program of China (2014CB340404) and China Scholarship Concil to Shhan Yan and from National Science Fondation to Xintao W ( ), Jn Li ( ) and Aidong L ( ). This research was condcted while Shhan Yan visited University of Arkansas. REFERENCES [1] Leman Akogl and Christos Falotsos What is Strange in Large Networks? Graph-based Irreglarity and Frad Detection. (2012). [2] Leman Akogl, Hanghang Tong, and Danai Kotra Graph based anomaly detection and description: a srvey. DMKD 29, 3 (2015), [3] Pierre Baldi Atoencoders, Unspervised Learning, and Deep Architectres. In Workshop on Unspervised and Transfer Learning [4] FabrÃŋcio Benevento, Gabriel Magno, Tiago Rodriges, and VirgÃŋlio Almeida Detecting spammers on twitter. In CEAS.

5 [5] Y. Bengio, A. Corville, and P. Vincent Representation Learning: A Review and New Perspectives. TPAMI 35, 8 (2013), [6] Qiang Cao, Xiaowei Yang, Jieqi Y, and Christopher Palow Uncovering Large Grops of Active Malicios Acconts in Online Social Networks. In CCS. [7] Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton Speech Recognition with Deep Recrrent Neral Networks. In ICASSP. [8] Kaiming He, Xiangy Zhang, Shaoqing Ren, and Jian Sn Deep Residal Learning for Image Recognition. In CVPR. [9] Meng Jiang, Peng Ci, Alex Betel, Christos Falotsos, and Shiqiang Yang CatchSync: Catching Synchronized Behavior in Large Directed Graphs. In KDD. [10] Alex Krizhevsky, Ilya Stskever, and Geoffrey E. Hinton ImageNet Classification with Deep Convoltional Neral Networks. In NIPS. [11] Srijan Kmar, Francesca Spezzano, and V.S. Sbrahmanian VEWS: A Wikipedia Vandal Early Warning System. In KDD. [12] Yann LeCn, Yosha Bengio, and Geoffrey Hinton Deep learning. Natre 521, 7553 (2015), [13] Caleb C. Noble and Diane J. Cook Graph-based Anomaly Detection. In KDD. [14] X. Ying, X. W, and D. BarbarÃą Spectrm based frad detection in social networks. In ICDE