Data Mining in Bioinformatics Day 3: Graph Mining

Transcription

1 Graph Mining and Graph Kernels Data Mining in Bioinformatics Day 3: Graph Mining Karsten Borgwardt & Chloé-Agathe Azencott February 6 to February 17, 2012 Machine Learning and Computational Biology Research Group MPIs Tübingen August 24, 2008 ACM SIG KDD, Las Vegas From Borgwardt & Yan, Graph Mining & Graph Kernels, KDD tutorial, 2008 with permission from Xifeng Yan. Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 1

2 Graphs Graph Are Mining Everywhere and Graph Kernels Magwene et al. Genome Biology :R100 Co-expression Network Program Flow Social Network Chemical Compound Protein Structure Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 2

3 Mining Graph Patterns Graph Mining and Graph Kernels Graph Pattern Mining Frequent graph patterns Pattern summarization Optimal graph patterns Graph patterns with constraints Approximate graph patterns Graph Classification Pattern-based approach Decision tree Decision stumps Graph Compression Other important topics (graph model, laws, graph dynamics, social network analysis, visualization, summarization, graph clustering, link analysis, ) Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 3

4 Applications Graph Mining of and Graph Kernels Pattern Mining Mining biochemical structures Finding biological conserved subnetworks Finding functional modules Program control flow analysis Intrusion network analysis Mining communication networks Anomaly detection Mining XML structures Building blocks for graph classification, clustering, compression, comparison, correlation analysis, and indexing Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 4

5 Graph Pattern Mining Graph Mining and Graph Kernels Graph pattern mining (single graph setting) Graph classification (multiple graphs setting) Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 5

6 Graph Mining and Kernels Interesting Graph Patterns Interestingness measures / Objective functions Frequency: frequent graph patterns Discriminative: information gain, Fisher score Significance: G-test Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 6

7 Graphs Graph Mining and Graph Kernels A graph is an ordered pair (V, E) V is a set of vertices (nodes) E is a set of edges (links) e = (v1, v2) v1, v2 vertices in V undirected / directed / mixed loops / multigraphs / simple graphs unlabeled / labeled unweighted / weighted Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 7

8 Graph isomorphism Graph Mining and Graph Kernels G and H are isomorphic if there exist a bijection f between the vertices of G and the vertices of H such that any two vertices u and v are adjacent in H iff f(u) and f(v) are adjacent in G. f(1) = A f(2) = C f(3) = D f(4) = B f(5) = F f(6) = E Solving graph isomorphism is NP, and it s not known whether it is P or NPC. In practice, it can be solved efficiently for a number of classes of graphs [McKay1981]. The subgraph isomorphism problem is NP-hard. Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 8

9 Searching Graphs Graph Mining and Graph Kernels Breadth-First Search Explore all vertices at distance k before exploring vertices at distance (k+1). Depth-First Search Explore each branch til its end before exploring another branch. Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 9

10 Graph Mining and Graph Kernels Frequent Pattern Mining Frequent Pattern freq(g) is called the support of g θ is called the minimum support Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 10

11 Example: Graph Chemical Mining and Graph Kernels compounds (a) caffeine (b) theobromine (c) viagra Frequent subgraph: Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 11

12 Example: Graph Program Mining and Graph call Kernels graphs Min support = 2 Frequent subgraphs: Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 12

13 Algorithms Graph Mining and Graph Kernels Inductive Logic Programming (WARMR, King et al. 2001) Graphs are represented by Datalog facts Graph Based Approaches Apriori-based approach AGM/AcGM: Inokuchi et al. (PKDD 00) FSG: Kuramochi and Karypis (ICDM 01) PATH # : Vanetik and Gudes (ICDM 02, ICDM 04) FFSM: Huan et al. (ICDM 03) SPIN: Huan et al. (KDD 04) FTOSM: Hórvath et al. (KDD 06) Pattern growth approach Subdue: Holder et al. (KDD 94) MoFa: Borgelt and Berthold (ICDM 02) gspan: Yan and Han (ICDM 02) Gaston: Nijssen and Kok (KDD 04) CMTreeMiner: Chi et al. (TKDE 05) LEAP: Yan et al. (SIGMOD 08) Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 13

14 Apriori Property Graph Mining and Graph Kernels If a graph is frequent, all of its subgraphs are frequent. Therefore you can stop extending non-frequent patterns. k edges (k+1) edges heuristics Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 14

15 Cost Analysis Graph Mining and Graph Kernels number of candidates frequent infrequent (X) duplicate (X) data isomorphism checking Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 15

16 Design Principles Graph Mining and Graph Kernels Search Order breadth vs. depth complete vs. incomplete Support Calculation compute for each new pattern vs. remember all subgraph isomorphism tests 1. Generation of Candidate Patterns apriori vs. pattern growth 2. Discovery Order of Patterns DFS order path tree graph 3. Elimination of Duplicate Subgraphs passive vs. active Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 16

17 Generation Graph Mining of Candidate Graph Kernels Patterns size k size (k+1) size k size (k+1) size (k+2) G Q P G 1 G 2 G n G G 1 G 2 G 1 G 2 Apriori-Based Approach join 2 patterns of size k that share a pattern of size (k-1) Pattern-Growth Approach extend patterns of size k by 1 edge / vertex Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 17

18 Discovery Graph Mining Order and Graph Kernels Free Extension 6 edges 7 edges 22 new patterns Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 18

19 Discovery Graph Mining Order and Graph Right-Most Kernels Extension [Yan and Han ICDM 02] 6 edges start end right-most path depth-first search 7 edges 4 new patterns Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 19

20 Duplicates Elimination Graph Mining and Graph Kernels Existing patterns Newly discovered pattern g1, g2,, gm g Option 1 Check graph isomorphism of g with each pattern g1, g2,, gm (slow) Option 2 Transform each graph to a canonical label, create a hash value for this canonical label, and check if there is a match with g (faster) Option 3 Build a canonical order and generate graph patterns in that order (fastest) Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 20

21 Duplicates Graph Mining Elimination and Graph Kernels Run Time AIDS antiviral screen compound dataset from NCI/NIH [Wörlein et al. PKDD 05] Run time per pattern (msec) MoFA: option 1 + free extension gspan: option 1 + right-most extension FFSM: option 2 Gaston: option 3 Minimum support (in %) Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 21

22 Duplicates Graph Mining Elimination and Graph Kernels Memory Usage [Wörlein et al. PKDD 05] Memory usage (GB) MoFA: option 1 + free extension gspan: option 1 + right-most extension FFSM: option 2 Gaston: option 3 Minimum support (in %) Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 22

23 Graph Pattern Graph Mining and Explosion Graph Kernels Problem Conclusion: Many enumeration algorithms (AGM, FSG, gspan, Path- Join, MoFa, FFSM, SPIN, Gaston.) But: If a graph is frequent, all of its subgraphs are frequent (apriori property) An n-edge frequent graph may have 2 n subgraphs E.g.: AIDS antiviral screen dataset with 400+ compounds, support level 5% over1m frequent graph patterns Problem 1: How to interpret frequent patterns? Pattern summarization Problem 2: How to reduce the size of the pattern set? Closed and maximal graphs Problem 3: How to set the minimum support? Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 23

24 Pattern Graph Summarization Mining and Graph Kernels [Xin et al. KDD 06, Chen et al. CIKM 08] Too many patterns may not lead to more explicit knowledge It can confuse users as well as further discovery (e.g., clustering, classification, indexing, etc.) a small set of representative patterns that preserve most of the information Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 24

25 Pattern Graph Summarization Mining and Graph Kernels Pattern Distance distance patterns patterns data measure 1: pattern-based pattern containment pattern similarity measure 2: data-based data similarity Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 25

26 Closed Graph and Mining Maximal and Graph Kernels Patterns Closed Frequent Graph A frequent graph G is closed if there exists no supergraph of G that carries the same support as G If some of G s subgraphs have the same support, it is unnecessary to output these subgraphs (nonclosed graphs) Lossless compression: still ensures that the mining result is complete Maximal Frequent Graph A frequent graph G is maximal if there exists no supergraph of G that is frequent Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 26

27 Closed Graph and Mining Maximal and Graph Kernels Patterns Examples Data: Therefeore D is not closed is a subgraph of A, B, C but so is which has the same support (3) No supergraph of E is also a subgraph of all 3 graphs and therefore E is closed. is a subgraph of A, B and is also closed: none of its supergraphs has support 2 If θ = 70%, E is maximal: E is frequent None of its supergraphs is frequent Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 27

28 Closed Graph and Mining Maximal and Graph Kernels Patterns Sizes Number of patterns Minimum support Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 28

29 CloseGraph Mining (Yan and Graph and Kernels Han, KDD 03) [Yan and Han KDD 03] Pattern-Growth Approach k-edge G (k+1)-edge G 1 G 2 G n If: Under which condition can we stop searching supergraphs? (early termination) G and H are frequent G is a subgraph of H in any part of graphs in the dataset where G occurs, H also occurs, then we need not grow G, since none of G s supergraphs will be closed except those of H. Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 29

30 References Graph Mining & and Further Graph Kernels Reading B. McKay. Practical graph isomorphism. Congressus Numerantium, 30:45 87, M. Wörlein, T. Meinl, I. Fischer, and M. Philippsen, A quantitative comparison of the subgraph miners MoFa, gspan, FFSM, and Gaston, PKDD 2005 X. Yan and J. Han, gspan: graph-based substructure pattern mining, ICDM 2002 X. Yan and J. Han, CloseGraph: mining closed frequent graph patterns, KDD 2003 Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 30

31 More References Graph Mining and Graph Kernels C. Borgelt and M. R. Berthold, Mining molecular fragments: finding relevant substructures of molecules, ICDM 2002 C. Chen, C. X. Lin, X. Yan, and J. Han. On effective presentation of graph patterns: a structural representative approach, CIKM 2008 Y. Chi, Y. Xia, Y. Yang, and R. Muntz, Mining closed and maximal frequent subtrees from databases of labeled rooted trees, TKDE 2005 T. Horváth, J. Ramon, and S. Wrobel, Frequent subgraph mining in outerplanar graphs, KDD 2006 J. Huan, W. Wang, and J. Prins, Efficient mining of frequent subgraph in the presence of isomorphism, ICDM 2003 J. Huan, W. Wang, and J. Prins, and J. Yang, SPIN: Mining maximal frequent subgraphs from graph databases, KDD 2004 A. Inokuchi, T. Washio, and H. Motoda. An apriori-based algorithm for mining frequent substructures from graph data, PKDD 2000 R. King, A Srinivasan, and L Dehaspe, WARMR: a data mining tool for chemical data, J. Comput. Aided Mol. Des M. Kuramochi and G. Karypis. Frequent subgraph discovery, ICDM 2001 S. Nijssen and J. Kok, A quickstart in frequent structure mining can make a difference, KDD 2004 N. Vanetik, E. Gudes, and S. E. Shimony. Computing frequent graph patterns from semistructured data, ICDM 2002 D. Xin, H. Cheng, X. Yan, and J. Han, Extracting redundancy-aware top-k patterns, KDD 2006 X. Yan, H. Cheng, J. Han, and P. S. Yu, Mining significant graph patterns by leap search, SIGMOD 2008 Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 31

32 The end Graph Mining and Graph Kernels See you soon! Next: Feature Selection Karsten Borgwardt & Chloé-Agathe Azencott Data mining in Bioinformatics 32