Hypergraph Exploitation for Data Sciences

Transcription

1 Photos placed in horizontal position with even amount of white space between photos and header Hypergraph Exploitation for Data Sciences Photos placed in horizontal position with even amount of white space between photos and header Michael Wolf, Alicia Klinvex, and Daniel Dunlavy Graph Exploitation Symposium May, 216 Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy s National Nuclear Security Administration under contract DE-AC4-94AL85. SAND NO C.

2 Hypergraph Models for Data Science Explore usage of hypergraphs for modeling complex, multiway (non- pairwise) relational data Goal: Improve data analysis more accurate, faster Questions we are trying to address Why should we use hypergraphs? When are hypergraph models preferable over graph models? Are there qualitative differences? Are there computational differences? What hypergraph model should be used? How should we compute the solution to our hypergraph problems? Specific problem Data clustering: Determine groupings of data objects given sets of relationships between/amongst objects 2

3 What is a hypergraph? Graph Bob Hypergraph Bob Amy Carl Amy Carl Ed Dan Edges connect 2 vertices G(V, E) Ed Hyperedges connect 1 or more vertices Dan G(V, H) Generalization of graph Hyperedges represent multiway relationships between vertices Hyperedge set of 1 or more vertices Key feature: hyperedges can connect more than 2 vertices 3

4 What is a hypergraph? Hyperedges: s Bob Vertices: Users Amy 1 1 Bob 1 Carl 1 1 Dan 1 1 Ed 1 1 Relation data / hypergraph incidence matrix Amy Ed Carl Dan Convenient representation of relational data Each can be represented by hyperedge s/hyperedges consist of subsets of users Relational data is often stored as hypergraph incidence matrix E.g., D4M* tables *Kepner, Jeremy, et al. "Dynamic distributed dimensional data model (D4M) database and computation system." Acoustics, Speech and Signal Processing (ICASSP), 212 IEEE International Conference on. IEEE,

5 Hypergraph to Graph: Clique Expansion Vertices Hyperedges A 1 1 B 1 C 1 1 D 1 1 E 1 1 B C A A Graph Edges A B 1 Vertices C D E B C E g = X e h 2E h d(eh ) 2 E D E D hyperedge cardinality Graph obtained through clique expansion of hypergraph 5

6 Why hypergraphs? Hypergraph Graph Bob Bob Amy 1 1 Bob 1 Carl 1 1 Amy Carl Amy Carl Dan 1 1 Ed 1 1 Ed Dan Ed Dan Multiway relationships can be represented nonambiguously Were Carl, Dan, and Ed involved in same ? Typically graph models lose information Fix: multi- graphs + metadata, changes to algorithms 6

7 Why hypergraphs? Vertices Hypergraph incidence matrix Graph Incidence matrix E g = X e h 2E h Hypergraphs require significantly less storage space than graphs generated using clique expansion Hypergraph incidence matrices require fewer operations for matrix- vector multiplication Hypergraphs have computational advantages d(eh ) 2 7

8 Outline Introduction to Hypergraphs Spectral clustering Software implementation Spectral clustering results Conclusions 8

9 Motivating Problem: Clustering of Relational Data Determine groupings of data objects given sets of relationships amongst those objects Relationships may be represented as graph or hypergraph Focus: spectral clustering Compute the smallest eigenpairs of the graph or hypergraph Laplacian Normalized graph Laplacian: Hypergraph Laplacian (Zhou, et al., 26) Never explicitly form Laplacians L G = I D 1/2 vg (H GHG T D vg )D 1/2 vg L h = I D 1/2 vh H HD 1 eh HT HD 1/2 vh Laplacian operator defined by series of SpMV ops and vector addition Reason 1: Matrix- matrix products are expensive Reason 2: Dynamic graphs easier to change incidence matrix than Laplacian SpMV = Sparse matrix-dense vector multiplication 9

10 Spectral Clustering (hyper)graph incidence matrix Form (hyper)graph Laplacian: L Find eigenvectors of L: V k-means(v) to find clusters vertex cluster assignments Basic spectral clustering procedure of Ng, et al., 22 Compute the smallest eigenpairs of the normalized graph or hypergraph Laplacian L G = I D 1/2 vg (H GHG T D vg )D 1/2 vg L h = I D 1/2 vh H HD 1 eh HT HD 1/2 vh Perform k- means clustering on those eigenvectors (k- means++) Partition data points into clusters such that each data points belongs to cluster with nearest mean

11 Outline Introduction to Hypergraphs Spectral clustering Software implementation Spectral clustering results Conclusions 11

12 Trilinos for Data Analysis C++ Trilinos based software for data analytics problems High performance (target: billions of vertices) Early focus on spectral methods Hypergraphs and graphs Supports several eigensolvers available in Anasazi LOBPCG, TraceMin- Davidson, Riemannian Trust Region, Block Krylov- Schur Very fast convergence for Laplacians (especially hypergraphs) Trilinos/Anasazi supports modern HPC architectures MPI+X Where X = {multicore, GPUs, Xeon Phi } 12

13 Hypergraph Generator Randomly generated hypergraphs Stochastic block- like Can generate multiple random instances for each parameter set Parameterized Generator Clusters Vertices per cluster Vertices # of intra/inter- cluster hyperedges Intra/Inter- cluster hyperedge cardinalities Edges Incidence Matrix: Generate real-world inspired hypergraphs at different scales Generates ground truth 13

14 Outline Introduction to Hypergraphs Spectral clustering Software implementation Spectral clustering results Clustering Quality Runtimes Real Data Conclusions 14

15 Experimental Setup Experiments conducted on workstation with 128 GB of memory using 16 cores Runtime parameters 5 k- means trials per matrix Eigensolver: LOBPCG (available in Trilinos), Tolerance: 1e- 3 Number of computed eigenpairs: same as number of clusters Quality of our clustering measured using the Jaccard index T = true cluster assignments ground truth from generator P = predicted cluster assignments J(T,P)=1 means matches ground truth exactly 15

16 Data Sets P1 P2 P3 Hyperedge Cardinality Number of clusters 5 5 Vertices per cluster*,,,, Intra-cluster hyperedges* 4, 2, 2, 2, Inter-cluster hyperedges* 5, 2, 2, 2, Intra-cluster hyperedge cardinality* Inter-cluster hyperedge cardinality* Generate hypergraph incidence matrices 4 different sets of parameters Different levels of difficulties randomly generated hypergraphs for each parameter set * Designates mean value 16

17 Outline Introduction to Hypergraphs Spectral clustering Software implementation Spectral clustering results Clustering Quality Runtimes Real Data Conclusions 17

18 Clustering Quality: P1, P2, P3 1 Jaccard&Index graph hypergraph.5 P1 P2 P3 Hypergraph based clusters more similar to ground truth clusters than graph based clusters 18

19 Clustering Quality: Hyperedge Cardinality Jaccard index graph JI Jaccard index Hyperedge cardinality Hyperedge cardinality General trend: hypergraph based clusters more similar to ground truth clusters than graph based clusters 19

20 Outline Introduction to Hypergraphs Spectral clustering Software implementation Spectral clustering results Clustering Quality Runtimes Real Data Conclusions 2

21 Runtimes: P1, P2, P3 Runtime((s) P1 P2 P3 graph hypergraph Hypergraph models significantly more computationally efficient than graph models (up to 3x faster) 21

22 Runtimes: Hyperedge Cardinality Unexpected Trend 3 Runtime(Ratio within cluster Hyperedge cardinality 8 5 Runtime ratio = graph runtime / hypergraph runtime Hypergraph models up to 3x faster than graph models 22

23 Eigensolver Iterations: Hyperedge Cardinality Graph Model 5 Hypergraph Model graph LOBPCG iterations LOBPCG'Iterations cardinality within cluster Unexpected Trend Hyperedge cardinality LOBPCG'Iterations within cluster Hyperedge cardinality Hypergraph required fewer LOBPCG iterations than graph Better separation of eigenvalues in hypergraph Laplacian Hypergraph models converged faster (up to 6x) than graph models 23

24 Operator Apply Time: Hyperedge Cardinality 2 Operator(Apply Ratio within cluster Hyperedge cardinality Laplacian operator apply more efficient for hypergraph model than graph model (up to 17x faster) 24

25 Outline Introduction to Hypergraphs Spectral clustering Software implementation Spectral clustering results Runtimes Clustering Quality Real Data Conclusions 25

26 Real Data Hypergraph, 7 Clusters Ground Truth Hypergraph Clustering Hgraph EV mammals birds reptiles fish amphibians insects other invertebrates Hgraph EV mammals birds sea mammals fish reptiles/amphibians insects other invertebrates seal porpoise dolphin seasnake pitviper slowworm tuatara platypus tortoise sealion UCI ML Repository Zoo Data Set* Animals with 16 attributes (# legs, eggs, ) + category Hypergraph Clustering vs. ground truth for 7 clusters Jaccard index:.815 (Graph model.743) Merged reptiles/amphibians, new category: sea mammals * EV=eigenvector 26

27 Real Data Graph, 7 Clusters Ground Truth Graph Clustering Graph EV mammals birds reptiles fish amphibians insects other invertebrates Graph EV Graph Clustering vs. ground truth for 7 clusters Jaccard index:.743 Data harder for kmeans to separate Insects, birds, mammals, other invertebrates,? EV=eigenvector 27

28 Real Data Hypergraph, 3 Clusters Ground Truth Hypergraph Clustering Hgraph EV mam/rep/fis/amp birds invertebrates Hgraph EV mam/rep/fis/amp birds invertebrates UCI ML Repository Zoo Data Set* Merged into 3 clusters: mammals/reptiles/fish/amphibians, birds, invertebrates Hypergraph Clustering vs. ground truth for 3 clusters Jaccard index: 1. (Graph model.865) * EV=eigenvector 28

29 Outline Introduction to Hypergraphs Spectral clustering Software implementation Spectral clustering results Conclusions 29

30 Summary/Conclusions Explored hypergraphs for modeling relational data Focus: Spectral Clustering Hypergraphs have advantages over graphs Less ambiguous representation of relationships Computationally advantageous Software for Linear Algebra Based Data Analytics Built on Trilinos framework (support for modern HPC architectures) High performance (target: billions of vertices) Support for both hypergraph and graph models Compelling computational results Larger Jaccard indices for hypergraph over graph for several problems Graph models challenging to weight properly Dramatic reduction in runtime for hypergraph vs. graph HPC = High performance computing 3

31 Future Work Target larger problems Currently solving problems of O(k) vertices Eventual target: O(1 billion) vertices Performance improvements to software 2D partitioning important for scalability beyond O(k) processors Improved efficiency of parallel hypergraph generator Hypergraph vs. Graph for larger real data sets Extend beyond spectral clustering Theoretical hypergraph models 31

32 Acknowledgements/References Thanks Jon Berry, Rich Lehoucq Spectral Clustering Ng, Andrew Y., Michael I. Jordan, and Yair Weiss. "On spectral clustering: Analysis and an algorithm." Advances in neural information processing systems 2 (22): Hypergraph Laplacian Zhou, Dengyong, Jiayuan Huang, and Bernhard Schölkopf. "Learning with hypergraphs: Clustering, classification, and embedding." Advances in neural information processing systems