Searching complex graphs

Transcription

1

2 Searching complex graphs complex graph data Big volume: huge number of nodes/links Big variety: complex, heterogeneous schema Big velocity: e.g., frequently updated Noisy, ambiguous attributes and values Existing techniques: Pregel, Trinity, Big velocity Big volume Big data Big variety nosql graph databases (Gremin, Neo4j, Gigraph, infinitegraph) Graph searching/querying: a graph query/pattern Q, a data graph G, identify matches in G for Q Computationally efficient query models Identify hidden matches in noisy, heterogeneous network Distributed querying and partition management Galileo : Google Knowledge Graph Search (May, 2012) People who like things I like an example of Facebook graph search query (January, 2013) 2

3 Main challenges Computationally efficient query models Enabling graph search with semantic similarity (ontology-based search) Enhancing approximate subgraph querying (Nema, simulation-based graph pattern matching) Distributed graph searching: partition management (Sedge) Vision of a distributed, complex graph search engine Conclusion Searching Yinghui big, Wu complex ICDE 2013 graphs 3

4

5 Computational efficient graph querying Let s use subgraph isomorphism as an example shape G-ray find all [Tong the et. subgraphs al., KDD 07]: that approximate are isomorphic matches to by Q in preserving G (NP-hard): the query query languages shape and algorithms Dual-simulation [Shuai et.al, VLDB 12]: preserving edge-edge matching too restrictive and computationally expensive! approximate subgraph matching: topological and label similarity (B. Edit distance Gallagher. Matching structure and semantics: A survey on graph-based pattern TALE[Tian et. al., ICDE 08] matching. AAAI FS., 2006.) SIGMA [Mongiovi et. al., J. Bioinformatics & Computational Biology 10] SAPPER [Zhang et. al., VLDB 12]: two-hop neighborhood indexes Trust [Sambhoos et al., Information Fusion 10]: - approximate graph matching with number of edge misses and node label mismatch Developing P-Hom [W.Fan computational et.al., VLDB 10]: edge-path efficient matchingquery models Flexible to capture label/topological similarity in noisy graph Proximity+label similarity Can be efficiently computed SAGA [Tian et. al., Bioinformatics 06]: both structure and node label similarity NESS, Nema [Arijit et.al, SIGMOD 12, VLDB 13] Bounded simulation [W.Fan et.al, VLDB 10]: edge-path matching Provide balanced usability-expressiveness Can be easily maintained over dynamic data graphs 5

6 Example: a simple IMDB query Query Graph 1 Query Graph 2 Kate Winslet (Actor)? (Movie) Kate Winslet (Actor)? (Movie)? (Director) Bernard Hill (Actor)? (Director) Bernard Hill (Actor)? (Movie) Titanic (Movie) Multiple representation of a simple query Matches may look very different from queries (e.g., large edit distance) relax rigid matching constraints of subgraph isomorphism Kate Winslet (Actor) James Cameron (Director) Bernard Hill (Actor) Searching Yinghui Wu big ICDE graphs

7 NESS and Nema (SIGMOD 11, VLDB 13) Goal: find a matching function Φ with minimum matching cost C(Φ) Individual node matching cost : Sum of label similarity and neighbourhood (vector) similarity of v and Φ(v) v 1 Φ f(v 1,u 1 ) u 1 { u 2,0.5, u 3,0.5, u 4,0.25 } (2 hop with propagation factor 0.5 v 2 v 4 Q f(v 2,u 2 ) f(v 4,u 4 ) u 2 u 3 u 4 u 5 Iterative Inference algorithm: a Max-Sum Inference in Random Markov Field - Time complexity: O(n G. n Q + T. n Q. m Q. d Q ) G 7

8 Effectiveness & Efficiency BLINKS [SIGMOD 07] SAGA [Bioinfo 06] IsoRank [PNAS 08] gstore [VLDB 11] NeMa [VLDB 13] Precision Recall Time (top-1) 1.92 sec sec sec 0.92 sec 0.97 sec Precision IMDB - 3M, 11M; YAGO - 13M, 18M; DBPedia 5M, 20M IMDB YAGO DBpedia Accuracy Query graph contains 7 nodes, structural noise 30%, and label noise 50%. Top-k Match Finding Time (sec) IMDB YAGO Dbpedia Efficiency Top-1 Top-3 Top-5 8

9

10 Searching in noisy graphs (ICDE 13) Finding semantically related matches for Q in G Q: find suspect A using class I guns in a safehouse A: we found a match [who report to A] using class II guns [a type of class I guns] in a [relocated] safehouse Q safehouse not match!... G match! rifle relocate to... subclass... supervise report to Og Using ontology-information to Yinghui capture Wu ICDE semantically 2013 similar matches 10

11 Ontology-based searching Ontology graphs A travel ontology museum Royal Gallery (RG) attraction Restaurants park Disneyland guided tours Holiday tour (HT) tourists Leisure center Culture tour (CT) waterfront riverside Holiday Cafe Royal Place (RP) Holiday Plazza (HP) Ontology-based subgraph querying: given a data graph G, a query graph Q and an ontology graph Og, identify best matches that are semantically close to Q 11

12 Querying framework A querying framework based on filtering-and-verification (1) offline ontology indexing: develop ontology views of G as an ontology index, by summarizing G using Og (2) online ontology-based filtering-and-verification A query evaluation framework (comparing with query enumeration): index construction in (O( G log G ) time) filtering in O( Q I time) Og ontology index Q query view G Q 1 Q 2 Q 4 Q 5 Q 3 Q(G) equivalence! Q(G) Q verification 12

13 Experimental results: effectiveness James Cameron from CrossDomain: G: 1.07M nodes, 3.86M edges Og: 1.44M nodes, 5.3M edges James Cameron Q 1 Cannes Festival Aliens Ghosts of the Abyss Aliens of the Deep Walt Disney Pictures Walt Disney Pictures Q 2 Flamingo from Flickr: G: 1.3M nodes, 6.42M edges Og: 3.64M entities (DBPedia) Flamingo Picture Picture Miami San Diego Pink San Diego Pink Seaworld Florida 13 Ontology matching identifies much Yinghui Wu more ICDE 2013 meaningful hidden matches

14 Experimental results: efficiency 30% of the running time of traditional subgraph querying algorithm, e.g., VF2 Effective even with a single concept graph Scale well with data size Ontology index can be efficiently updated upon changes to data graphs Ontology matching outperforms Yinghui traditional Wu ICDE 2013 graph querying in efficiency 14

15

16 Graph partition and management(sigmod 12) Sedge: partition strategy Complementary partitioning repartition the graph with region constraint On-demand partitioning: envelop grouping for cross queries Two-level partition management v1 v2 v3 v5 C1 C2 v4 C3 P3 P1 P2 Run independently 16

17 Graph partition management Sedge: system architecture 17

18 Sedge: performance evaluation SP 2 Bench (DBLP-based) Significant cross query reduction (complete removal of cross queries for several cases) Complementary and on demand partition is effective effectively improve efficiency for cross queries 18

19

20 Dealing with decentralized world Real-life graphs are distributed Our vision: a dispatching-local evaluation-assembling framework Querying distributed graphs? High communication cost; inter-source communication is expensive/not allowed Local evaluation+assembling 20 Distributed querying over Yinghui independent Wu ICDE 2013 graph sources 20

21 Distributed graph querying framework (rejected, still trying) Requester site Sc Q and a set of graph Sources F1,, Fn Q(Fi) Q(Fi) distributing at Sc: source filtering and query decomposition sources Q(G) Q... local evaluation: generate partial results Q(Fi) requester Sc Q(Fi) Assembling at Sc dispatching + local evaluation + assembling 11

22 Distributed graph querying: real-life life queries wiki Dbpedia Flickr 22

23 Searching complex graph: a big graph issue computationally efficient query models partition strategy & management/ distributed querying compression/summarization view-based querying Big volume incremental/dynami c graph querying and maintenance Spatial-temporal /stream graph querying Big velocity Searching big graph data Big variety semantic searching e.g., ontology-based indexing and querying usability-expressive power: query suggestion/transformatio n/rewriting/refinement knowledge inference A great source of research topics and promising search tools 23

24 resources All of our software and data will be announced in this link: Ness and Nema: source code Sedge: project homepage (docs, source code and dataset) Ontology-based subgraph matching Thank you! Acknowledgement: Information Network Science CTA Our group: Xifeng Yan, Shengqi, Arijit 24