E6885 Network Science Lecture 11: Knowledge Graphs

Transcription

1 E 6885 Topics in Signal Processing -- Network Science E6885 Network Science Lecture 11: Knowledge Graphs Ching-Yung Lin, Dept. of Electrical Engineering, Columbia University November 25th, 2013

2 Course Structure 2 Class Date Lecture Topics Covered 09/09/13 1 Overview of Network Science 09/16/13 2 Network Representation and Feature Extraction 09/23/13 3 Network Paritioning, Clustering and Visualization 09/30/13 4 Network Analysis Use Case 10/07/13 5 Network Sampling, Estimation, and Modeling 10/14/13 6 Network Topology Inference 10/21/13 7 Network Information Flow 10/28/13 8 Dynamic & Probabilistic Networks and Graph Database 11/11/13 9 Final Project Proposal Presentation 11/18/13 10 Graph Databases II 11/25/13 Knowledge Graphs E688511Network Science Lecture 11: Knowledge Graphs

3 Relational Term-Suggestion Q. What keywords should I put in the search box to get the information I really want?

4 Multi-partite Network Analytics Term Suggestion and Query Expansion Documentbased Influenced by test collection characteristics Log-based Ontologybased Query log, failure for rare queries WordNet Click log, biased in favor of top ranks Limited semantic relatedness Not publicly available Difficult to update Wikipedia Simple concept links only Multi-partite network analytics Network community -based Extracting human factor Incorporate expertise

5 Document-based Influenced by test collection characteristics No consideration of key terms that are highly sem antically related but do not frequently co-occur. Influenced apple juice apple tree apple store apple TV Kim, M. AND Choi, K. A Comparison of collocation-based similarity measures in query expansion. Information Processing and Management 35 (1999),

6 Multi-partite Network Analytics Term Suggestion and Query Expansion Documentbased Influenced by test collection characteristics Log-based Ontologybased Query log, failure for rare queries WordNet Click log, biased in favor of top ranks Limited semantic relatedness Not publicly available Difficult to update Wikipedia Simple concept links only Multi-partite network analytics Network community -based Extracting human factor Incorporate expertise

7 Log-based Cluster queries with similar clicked URLs Identifying the mapping between queries and clicked URLs Pet food Dog food BAEZA-YATES, R., AND TIBERI, A Extracting Semantic Relations from Query Logs. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2007),

8 Multi-partite Network Analytics Term Suggestion and Query Expansion Documentbased Influenced by test collection characteristics Log-based Ontologybased Query log, failure for rare queries WordNet Click log, biased in favor of top ranks Limited semantic relatedness Not publicly available Difficult to update Wikipedia Simple concept links only Multi-partite network analytics Network community -based Extracting human factor Incorporate expertise

9 WordNet as Ontology Manuallyconstructed system based on individual words benef it will be limited System is not easily updated Pedersen, T, Patwardhan, S and Michelizzi, J. "WordNet::Similarity Measuring the Relatedness of Concepts" 2004 In Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI2004) pp

10 Wikipedia as Ontology

11 Wikipedia as Ontology Wikipedia is a web-based free encyclopedia that anyone can edit. The English Wikipedia edition 2.4 million articles 1 billion words. Wikipedia relies on the power of collective intelligence by peer-reviewed approaches rather than the authority of indivi dual. high quality, almost noise free.

12 Previous Approaches Merely as an online dictionary and utilize it onl y as a structured knowledge database Using associated hyperlinks MILNE, D., WITTEN, I. H., AND NICHOLS, D A KnowledgeBased Search Engine Powered by Wikipedia. In Proceedings of the 16th ACM Conference on Information and Knowledge Management (CIKM 2007),

13 Multi-partite Network Analytics Term Suggestion and Query Expansion Documentbased Influenced by test collection characteristics Log-based Ontologybased Query log, failure for rare queries WordNet Click log, biased in favor of top ranks Limited semantic relatedness Not publicly available W 2.0 Difficult to update Wikipedia Simple concept links only Multi-partite network analytics Network community -based Extracting human factor Incorporate expertise

14 Multi-partite Network Analytics Term Suggestion and Query Expansion Documentbased Influenced by test collection characteristics Log-based Ontologybased Query log, failure for rare queries WordNet Click log, biased in favor of top ranks Limited semantic relatedness Not publicly available Difficult to update Wikipedia Simple concept links only Our Challenge Multi-partite network analytics Crawling is resourceintensive Human factor modeling Semantic relatedness difficult to evaluate

15 Wikipedia as Ontology 6/3/12 15

16 Query Data Sampling Semantic Relatedness Weighting Ontology Relative Importance Ranking Contributor Expertise Analysis Optimization Visualization Interface Evaluation Interface

17 Key Term T C T L C C T L C T Layer by layer L L C C:contributors C C T:Terms L:Categories C

18 Query Data Sampling Semantic Relatedness Weighting Ontology Relative Importance Ranking Contributor Expertise Analysis Optimization Visualization Interface Evaluation Interface

19

20 Query Data Sampling Semantic Relatedness Weighting Ontology Relative Importance Ranking Contributor Expertise Analysis Optimization Visualization Interface Evaluation Interface

21

22 Contributor to categories Expertise Contributor to contributor Expertise inference Term to categories Term to Term Contributor Expertise factor

23 Query Data Sampling Semantic Relatedness Weighting Ontology Relative Importance Ranking Contributor Expertise Analysis Optimization Visualization Interface Evaluation Interface

24 High Semantic Relatedness Term Suggestion from Our System

25 Word-completion Term Suggestion

26 Experiment I P@1 P@5 S@5 S@20 MRR Simple link Contributor Expertise Performance Comparison for Different Relationship Levels. Using BibSonomy Dataset

27 Experiment II Accuracy on different categories Wordnet Bag of words Our algorithm Literature 62.0% ± 5% 62.7% ± 4% 76.8% ± 6% Natural science 60.7% ± 4% 65.6% ± 6% 73.3% ± 3% Sociology 72.1% ± 5% 62.9% ± 5% 72.5% ± 7% Business 60.4% ± 6% 58.5% ± 8% 67.1% ± 7% Law 52.2% ± 9% 50.4% ± 8% 66.3% ± 6% Engineering 54.0% ± 6% 68.3% ± 5% 66.2% ± 4% Electrical & Computer Eng. Life Science 77.0% ± 4% 68.0% ± 3% 82.3% ± 3% 73.1% ± 6% 70.9% ± 6% 81.4% ± 7% Agriculture 72.6% ± 5% 65.1% ± 6% 72.3% ± 5% Medical 63.0% ± 8% 65.6% ± 7% 61.6% ± 8% ODP-based precision evaluation results increase 12.5% in average

28 Precision Comparison With Paraphrase Detection System Synonyms Zhao et al. Our approach Hyponymy Antonyms Paraphrase % of the suggested terms are reported as related, i.e., synonyms (22%), hyponyms (37%) or antonyms (23%)

29 References Jyh-Ren Shieh, Ching-Yung Lin, Shun-Xuan Wang, Ja-Ling Wu, Relational Term-Suggestion Graphs Incorporating Multi-Partite Concept and Expertise Networks, ACM Transactions on Intelligent Systems and Technology (2012). Jyh-Ren Shieh, Ching-Yung Lin, Shun-Xuan Wang, Ja-Ling Wu, Building Multi-Modal Relational Graphs for Multimedia Retrieval, International Journal of Multimedia Data Engineering and Management (IJMDEM): pp (2011). Best paper award nomination. Jyh-Ren Shieh, Yung-Huan Hsieh, Yang-Ting Yeh, Tse-Chung Su, Ching-Yung Lin, Ja-Ling Wu, Building term suggestion relational graphs from collective intelligence, World Wide Web Conference (WWW 2009) pp (2009). Jyh-Ren Shieh, Yang-Ting Yeh, Chih-Hung Lin, Ching-Yung Lin and Ja-Ling Wu, Using Semantic Graphs for Image Search, IEEE International Conference on Multimedia & Expo (ICME 2008), pp (2008). 29 E6885 Network Science Lecture 11: Knowledge Graphs

30 Part-based Object Detection by Learning Random Attributed Graphs Ref: DQ Zhang and SF Chang, Detecting image near-duplicate by stochastic attributeed relational graph matching with learning, ACM MM E6885 Network Science Lecture 11: Knowledge Graphs

31 Problem 1 : Object Detection and Part Identification a. Does the input image contain the specified object? b. Where are the object parts? E6885 Network Science Lecture 11: Knowledge Graphs

32 Problem 2 : Learning Part-based Object Model Automatically learn the structure and parameters Minimum supervision : no object location and part location E6885 Network Science Lecture 11: Knowledge Graphs

33 Prior Work on Part-based Object Detection Model with Hand-built structure Model without spatial structure Model with learned structure and part statistics Pictorial structure, [Felzenszwalb & Huttenlocher 98 ] AdaBoost, Constellation Model, Elastic Bunch Graph, [Viola & Jones, 01 ] [Wiskott et. al 97 ] [Burl, Weber, Fergus, Perona, Caltech, Oxford ] MRF model, [Li 94 ] This new model : Graph-based representation; Can handle multi-view object detection E6885 Network Science Lecture 11: Knowledge Graphs

34 Part-based Representation of Visual Scene Visual scenes are considered as the composition of the parts with certain spatial/attribute relations, modeled as Attributed Relational Graph (ARG) ARG IND Detection as Computing ARG similarity Part?? == Part relation Attributed Relational Graph (ARG) ARG Similarity E6885 Network Science Lecture 11: Knowledge Graphs

35 ARG based on Interest Point Detection Region-based representation had very bad performance! Interest point detector: SUSAN (Smallest Univalue Segment Assimilating Nucleus) corner detector Local features at vertexes Spatial location, Color, Gabor filter coefficients Part relational features at edges Spatial coordinate difference E6885 Network Science Lecture 11: Knowledge Graphs

36 Stochastic Framework for ARG Similarity Vertex Correspondence Ys ARG s Attribute Transformation ARG t Yt Stochastic Process that Transforms ARG s to ARG t ARG similarity is the likelihood or likelihood ratio of the stochastic process that transforms source ARG to target ARG H: Hypotheses: H = 1, Graph t is similar to Graph s H = 0, Graph t is not similar to Graph s E6885 Network Science Lecture 11: Knowledge Graphs

37 Non-linear Scene Transformation Vertex Correspondence Scene changes: object movement, occlusion etc. Camera changes: view point change, panning etc Photometric changes: Lighting etc. Digitization changes: Resolution, gray scale etc. Ys Graph s Model Occlusion of objects Addition of objects Attribute Transformation Graph t Yt Model Object appearance change, Object move, Photometric change E6885 Network Science Lecture 11: Knowledge Graphs

38 Generative Model of the Stochastic Transformation Process Graph s x H: Hypothesis X {x11, x12,..., x32 } H H=1 : two graphs are similar Graph t 2 Graph S 2 3 x32 Product Graph Graph t X : Correspondence Matrix E6885 Network Science Lecture 11: Knowledge Graphs

39 Transformation Likelihood Transformation Likelihood x11 1 Prior MRF for constraints 1 Graph t 2 Graph S 2 3 x32 11,12 ( x11, x12 ) 0 Conditional density for attribute transformation x Graph t 2 Graph S 2 3 x32 Transformation likelihood is: 11, 22 ( x11, x22 ) 1 E6885 Network Science Lecture 11: Knowledge Graphs

40 Learning to Match ARGs Feature point level learning: Label every feature point pairs Vertex-level annotation Image level learning: Label duplicate pairs and non-duplicate pairs Use Variational Expectation-Maximization (E-M) Positive Samples Negative Samples E6885 Network Science Lecture 11: Knowledge Graphs

41 Experiments and Results Data set Images are picked up from TREC-VID 2003 video frames (partly based on TDT2 topic detection ground truth) 150 duplicate pairs, 300 non-duplicate images Learning Training set: 30 duplicate pairs, 60 nonduplicate images Feature point level learning 5 duplicate pairs, 10 non-duplicate images Image level learning 25 duplicate pairs, 50 non-duplicate images Feature point level learning Initial parameters Image-level learning Final parameters E6885 Network Science Lecture 11: Knowledge Graphs

42 Compare with other similarity measures (CH) HSV color histogram (LED) Local Edge Descriptor (AFDIP) Average feature distance of interest points (GRAPH) ARG matching with learning (GRAPH-M) ARG matching with manual parameter adjustment Precision Recall E6885 Network Science Lecture 11: Knowledge Graphs

43 Summary of Part-based ARG Visual Modeling Algorithm Statistical part-based similarity measure performs much better than global color histogram and gridbased edge map Learning-based ARG matching not only save human cost,but also may give better performance E6885 Network Science Lecture 11: Knowledge Graphs

44 Questions? 44 E6885 Network Science Lecture 11: Knowledge Graphs