Query-Based Outlier Detection in Heterogeneous Information Networks

Transcription

1 Query-Based Outlier Detection in Heterogeneous Information Networks Jonathan Kuck 1, Honglei Zhuang 1, Xifeng Yan 2, Hasan Cam 3, Jiawei Han 1 1 Uniersity of Illinois at Urbana-Champaign 2 Uniersity of California at Santa Barbara 3 US Army Research Lab

2 Heterogeneous information networks are networks composed of multi-typed, interconnected ertices and links Outlier detection aims to find ertices that deiate significantly from other ertices Howeer, outliers in such a heterogeneous network could be defined in many different ways E.g. Finding outlier in authors of this paper Jonathan Honglei Xifeng Hasan Jiawei

3 Heterogeneous information networks are networks composed of multi-typed, interconnected ertices and links Outlier detection aims to find ertices that deiate significantly from other ertices Howeer, outliers in such a heterogeneous network could be defined in many different ways E.g. Finding outlier in authors of this paper Jonathan Honglei Xifeng Hasan Jiawei

4 Heterogeneous information networks are networks composed of multi-typed, interconnected ertices and links Outlier detection aims to find ertices that deiate significantly from other ertices Howeer, outliers in such a heterogeneous network could be defined in many different ways E.g. Finding outlier in authors of this paper Jonathan Honglei Xifeng Hasan Jiawei As users hae their own intuition about which kind of outliers they are interested in Allow users to specify queries for outlier detection

5 Research Challenges How do users interact with the system to specify their queries? How do we define a general outlierness measure for different queries? How to efficiently find outliers for different queries?

6 Outline Basic Concepts and Notations Outlier Query Language NetOut: Outlierness Measure Processing Outlier Queries Experimental Results Summary

7 Basic Concepts and Notations Heterogeneous Information Network An information network with multiple types of ertices G V, E,, T where V is the set of ertices; E is the set of edges; T is the set of types, and : V T assigns each ertex a type. Network schema An instantiated network

8 Basic Concepts and Notations Meta-Path An ordered sequence of ertex types, denoted as E.g. P Author,Paper,Venue Meta-Path Instantiation Use P i, j to denote paths between two ertices with the types that align with the gien meta-path E.g. Zoe, KDD is shown below in red solid lines P P P, i j Network schema An instantiated network

9 Basic Concepts and Notations Neighborhood Based on a meta-path P, defined as E.g. N P Zoe KDD, ICDE Neighbor Vector, N P i j P i j Based on a meta-path P, the neighbor ector P i describes how many paths there are from i to each of its neighbors E.g. Zoe KDD:3, ICDE:2 P Network schema An instantiated network

10 Formalization of Outlier Queries Gien a heterogeneous information network A query can be formalized as where Q S, S, Pw, c S V is a set of (same-typed) candidate ertices c Specifying from where outliers are chosen from S V is a set of (same-typed) reference ertices r Specifying a sample of normal ertices Optional. By default it equals to the candidate set Pw, define a weighted set of meta-paths Specifying how two ertices are compared r

11 Outlier Query Language General Formulation FIND OUTLIERS FROM author{ C. Faloutsos }.paper.author COMPARE TO enue { KDD }.paper.author JUDGED BY author.paper.enue TOP 10;

12 Outlier Query Language General Formulation FIND OUTLIERS FROM author{ C. Faloutsos }.paper.author Specifying the candidate set Sc NPC. Faloutsos P Author, Paper, Author COMPARE TO enue { KDD }.paper.author JUDGED BY author.paper.enue TOP 10;

13 Outlier Query Language General Formulation FIND OUTLIERS Specifying the candidate set FROM author{ C. Faloutsos }.paper.author COMPARE TO enue { KDD }.paper.author Specifying the reference set JUDGED BY author.paper.enue TOP 10; Sr NPKDD P Venue, Paper, Author

14 Outlier Query Language General Formulation FIND OUTLIERS Specifying the candidate set FROM author{ C. Faloutsos }.paper.author COMPARE TO enue { KDD }.paper.author JUDGED BY author.paper.enue Compare ertices by neighbor ector where TOP 10; P P Author, Paper, Venue i Specifying the reference set

15 Outlier Query Language General Formulation FIND OUTLIERS Specifying the candidate set FROM author{ C. Faloutsos }.paper.author COMPARE TO enue { KDD }.paper.author JUDGED BY author.paper.enue TOP 10; Only return top 10 outliers Specifying how ertices are compared Specifying the reference set

16 Outlier Query Language (cont ) Allow complicated judging standards E.g. multiple weighted neighbor ectors JUDGED BY author.paper.enue: 0.6, author.paper.author: 0.4 Allow operations on sets E.g. select by conditions FROM enue { KDD }.paper.author AS A WHERE COUNT(A.paper)> 10

17 NetOut: A General Network-Based Outlierness Measure P Gien a meta-path (in JUDGED BY clause) Define the connectiity between two ertices, 1, i j PP i j The more paths between them, the more similar they are Define relatie connectiity Compare the connectiity to self-connectiity, i j 1 i, PP j 1 i, PP i Use self-connectiity as a expected connectiity to measure whether two ertices are unexpectedly connected or unexpectedly disconnected Outlier Measure: NetOut, i i j S j r

18 Comparing NetOut to Other Measures A toy example. Neighbor Vector VLDB KDD STOC SIGGRAPH Reference Author(s) Sarah Rob Lucy *Joe *Emma Outlier measure comparison. The lower alue, the more likely to be an outlier. NetOut PathSim [1] CosSim Sarah Rob Lucy Joe Emma Joe is not necessarily an interesting outlier, as those papers might simply be noise Emma is obiously an outlier and should be assigned a lower alue [1] Sun, Yizhou, et al. "PathSim: Meta path-based top-k similarity search in heterogeneous information networks." VLDB 2011.

19 Comparison on Real Data Set Apply on network constructed from DBLP data set Find outliers among Christos Faloutsos coauthors, in terms of their publishing enues authors with ery few paper; uninteresting outliers

20 Query Processing The calculation of NetOut can be written as 2 2, ;,,, 1, j r j r j r i j i S i i i j S i i i j S i Q

21 Query Processing The calculation of NetOut can be written as Time complexity: 2 2, ;,,, 1, j r j r j r i j i S i i i j S i i i j S i Q O( S r ), only calculated once O( S c ), calculate oer all the candidate ertices c r O S S

22 Pre-Materialization of Meta-Path We obsere Calculation of outlierness only has time complexity of O S S Retrieing i for each ertex is much more time consuming Pre-materialization c Pre-store the materialization of all the length-2 meta-paths r

23 Selectie Pre-Materialization Storing materialization of all length-2 metapaths can be space-consuming Selectie Pre-materialization From a gien set of training queries, take ertices that frequently appear in the candidate sets (e.g. those appear in more than 10% of queries) Pre-materialize all the length-2 meta-paths for these frequently appearing ertices

24 Experimental Setup Data Set DBLP Data Set 2,244,018 publications, 1,274,360 authors Construct the network according to the schema aboe Publishing enue information is included Terms are extracted from publication titles Query Sets Design three different types of queries to find different kinds of outliers Generate 10,000 random queries for each type of queries as a query set

25 Case Study With different queries, the outlier measure is able to capture different outliers accordingly E.g. Finding outliers in Christos Faloutsos coauthors Outlier results by comparing publishing enues Outlier results by comparing coauthor communities

26 Efficiency Study Comparing strategies Baseline: No pre-materialization Pre-Materialization (PM): All length-2 meta-path instantiations are pre-computed and indexed Selectie Pre-Materialization (SPM): Only a subset of instantiations with relatie frequency larger than 0.01 are indexed

27 Parameter Studies for SPM Different thresholds for selectie pre-materialization

28 Summary Propose a framework for query-based outlier detection in heterogeneous information networks Formalize the definition of a query and proide a query language for users to interact with the system Design a general outlierness measure NetOut which can effectiely find interesting outliers Present an efficient implementation to process such queries

29 Thank you Introduction Outlier Query Language NetOut Measure Query Processing Experimental Results

30