Graph-based Learning. Larry Holder Computer Science and Engineering University of Texas at Arlington

Transcription

1 Graph-based Learning Larry Holder Computer Science and Engineering University of Texas at Arlingt 1

2 Graph-based Learning Multi-relatial data mining and learning SUBDUE graph-based relatial learner Discovery Clustering Graph grammar learning Supervised learning 2

3 Multi-Relatial Data Mining Looking for patterns involving multiple tables (relatis) in a relatial database Pers ID P1 P2 P3 Last Doe Doe Smith First Age John 30 Sally 29 Income Robert Married Pers1 P1 Pers2 RichCouple(X,Y) Pers(X,LastX,FirstX,AgeX,IncX) & Pers(Y,LastY,FirstY,AgeY,IncY) & Married(X,Y) & (IncX + IncY) > P3 P2 P7 3

4 Multi-Relatial Data Mining Approaches Transform to n-relatial problem First-order logic based Inductive Logic Programming (ILP) Graph based 4

5 Graph-based Data Mining Finding all subgraphs g within a set of graph transactis G such that freq( g) G > t where t is the minimum support 5

6 Graph-based Data Mining Systems Apriori-based Graph Mining (AGM) Inokuchi, Washio and Motoda,, 2003 Frequent Sub-Graph discovery (FSG) Kuramochi and Karypis,, 2001 Graph-based Substructure pattern mining (gspan) Yan and Han, 2002 Focus pruning and fast, code-based graph matching 6

7 Graph-based Relatial Learning Finding patterns in graph(s) Discovery Clustering Supervised learning Smith Robert Last First Married Pers Age Income Doe John Doe Sally Last First Married Pers Age Income Last First Pers Age Income

8 Graph-based Relatial Learning Graph-Based Inducti (GBI) Yoshida, Motoda and Indurkhya,, 1994 SUBstructure Discovery Using Examples (SUBDUE) Cook and Holder, 1994 Focus efficient subgraph generati and compressi-based heuristic search 8

9 SUBDUE Graph-based Discovery Graph representati Graph compressi and MDL Discovery algorithm Inexact graph match Background knowledge Parallel/distributed discovery 9

10 Graph Representati Input is a labeled (vertices and edges) directed graph A substructure is a cnected subgraph An instance of a substructure is an isomorphic subgraph of the input graph Input graph compressed by replacing instances with vertex representing substructure T1 S1 T2 S2 Input Database R1 T3 S3 C1 T4 S4 Substructure S1 (graph form) object object shape shape triangle square Compressed Database S1 S1 R1 S1 C1 S1 10

11 Graph Representati S 2 S 1 S 1 S 1 S 1 S 1 S 2 S 2 11

12 Graph Compressi and MDL Minimum Descripti Length (MDL) principle Best theory minimizes descripti length of theory and the data given theory Best substructure S minimizes descripti length of substructure definiti DL(S) and compressed graph DL(G S) min( DL( S) + S DL( G S)) 12

13 Discovery Algorithm 1. Create substructure for each unique vertex label triangle square rectangle triangle square circle triangle square triangle square Substructures: triangle (4), square (4), circle (1), rectangle (1) 13

14 Discovery Algorithm 2. Expand best substructures by an edge or edge+neighboring vertex triangle square rectangle triangle square circle triangle square triangle square Substructures: triangle square square rectangle circle rectangle rectangle triangle 14

15 Discovery Algorithm 3. Keep ly best beam-width substructures queue 4. Terminate when queue is empty or #discovered substructures > limit 5. Compress graph and repeat to generate hierarchical descripti 15

16 DNA Example 16

17 Sample SUBDUE Input sample.g: v 1 object v 2 object v 3 object v 4 object v 5 object v 6 object v 7 object v 8 object v 9 object v 10 object v 11 triangle v 12 triangle v 13 triangle v 14 triangle v 15 square v 16 square v 17 square v 18 square v 19 circle v 20 rectangle e 1 11 shape e 2 12 shape e 3 13 shape e 4 14 shape e 5 15 shape e 6 16 shape e 7 17 shape e 8 18 shape e 9 19 shape e shape e 1 5 e 2 6 e 3 7 e 4 8 e 5 10 e 9 10 e 10 2 e 10 3 e 10 4 T1 S1 T2 S2 R1 T3 S3 C1 T4 S4 17

18 Inexact Graph Match Some variatis may occur between instances Want to abstract over minor differences Difference = cost of transforming e graph to make it isomorphic to another Match if cost/size < threshold 18

19 Inexact Graph Match A a B 1 2 b B b A 3 4 a a b 5 B (1,3) 1 (1,4) 0 (1,5) 1 (1,λ) 1 (2,4) 7 (2,5) 6 (2,λ) 10 (2,3) 3 (2,5) 6 (2,λ) 9 (2,3) 7 (2,4) 7 (2,λ) 10 (2,3) 9 (2,4) 10 (2,5) 9 (2,λ) 11 Least-cost match is {(1,4), (2,3)} 19

20 Inexact Graph Match Vertices csidered by degree Polynomially cstrained Greedy after n k partial mappings csidered Suboptimal mappings rare for k>2 20

21 Background Knowledge User-defined substructures Two alternative uses Prime search queue Initial graph compressi Variant of discovery algorithm used to generate instances 21

22 Parallel/Distributed Discovery Divide graph into P partitis Distribute to P processors Each processor performs serial discovery local partiti Broadcast best substructures, evaluate other processors Master processor stores best global substructures 22

23 Graph-based Clustering Hierarchical, cceptual clustering Previous work defined classificati trees Inadequate in relatial domains Better hierarchical descripti: classificati lattice A cluster can have more than e parent A parent can be at any level (not ly e level above) Use iterative graph-based discovery 23

24 Clustering: DNA 24

25 Clustering: DNA C N C C DNA O O == P OH Coverage 61% C \ N C \ C O \ C / \ C C N C / \ O C C C \ O O O == P OH O CH 2 68% 71% 25

26 Learning Graph Grammars Graph grammar producti: S P S is a n-terminal P is a graph ctaining terminals and/or n- terminals S P 1 P 2 P n Recursive producti: S P S P P linked to S via a single edge Algorithm expential in number of linking edges 26

27 Example Graph Grammar S 2 a S 2 a b S 3 b S 3 S 3 c d e f 27

28 Graph Grammar Learning SUBDUE Extensis (SubdueGL( SubdueGL) Each iterati results in a graph grammar producti substructure Producti used to compress graph Replace instances of right-hand hand side with new vertex labeled with n-terminal left-hand side Iteratis ctinue until entire graph compressed to single n-terminal 28

29 SubdueGL Example Input graph Edge labels: t, s, next a a a a b c b d b e b f k x y x y r x y x y z q z q z q z q 29

30 SubdueGL Example First producti rule S 1 x y S 1 x y z q z q Input graph parsed by first producti a a a a b c b d b e b f k S 1 r S 1 30

31 SubdueGL Example Secd and third producti rules S 2 a S 2 a b S 3 b S 3 S 3 c d e f Input graph parsed by productis S 2 S 1 r S 1 k 31

32 Graph-Based Supervised Learning Input now a set of positive graphs and a set of negative graphs Input Hypothesis object object object shape shape triangle square 32

33 Graph-Based Supervised Learning Soluti 1 Find substructure compressing positive graphs, but not negative graphs Compress graphs and iterate until no further compressi Problem Compressing, instead of removing, partially- covered positive graphs leads to overly- specific hypotheses 33

34 Graph-Based Supervised Learning Soluti 2 Find substructure covering (i.e., subgraph of) positive graphs, but not negative graphs Remove covered positive graphs and iterate until all covered Substructure value = 1 - Error Error = # PosEgsNotCovered+ # NegEgsCovered # PosEgs+ # NegEgs 34

35 Supervised Learning: Cancer Chemical toxicity SUBDUE achieved 62% accuracy classifying carcinogenic vs. n-carcinogenic compounds six_ring p 6 cytogen_ca di227 in_group ctains compound in_group atom 7 element type charge c compound compound drosophila_slrl chromaberr p p positive ames ctains atom element type c 22 has_group amine charge -13 ashby_alert ashby_alert halide10 35

36 Applicati Domains Biochemical domains Protein data DNA data Toxicology (cancer) data Spatial-temporal temporal domains Earthquake data Aircraft Safety and Reporting System Web topology and search Social network analysis hyperlink web_page hyperlink home web_page hyperlink web_page 36

37 Summary Multi-relatial data mining and learning Graph-based relatial learning Discovery Clustering Graph grammar learning Supervised learning 37

38 Future Directis Efficient graph-based learning from incremental streaming data Supervised graphs All examples in e, cnected graph Graph-based anomaly detecti Improved scalability Graph and subgraph isomorphism 38

39 Further Informati Graph-based Data Mining SUBDUE Project ailab.uta.edu/subdue 39