KNOWLEDGE GRAPH CONSTRUCTION

Transcription

1 KNOWLEDGE GRAPH CONSTRUCTION Jay Pujara Karlsruhe Institute of Technology 7/7/2015

2 Can Computers Create Knowledge? Internet Massive source of publicly available information Knowledge

3 Computers + Knowledge =

4 Motivating Problem: New Opportunities Internet Extraction Knowledge Graph (KG) Massive source of publicly available information Cutting-edge IE methods Structured representation of entities, their labels and the relationships between them

5 Motivating Problem: Real Challenges Internet Extraction Knowledge Graph Difficult! Noisy! Contains many errors and inconsistencies

6 Knowledge Graphs in the wild

7 Overview Problem: Build a Knowledge Graph from millions of noisy extractions Approach: Knowledge Graph Identification reasons jointly over all facts in the knowledge graph Method: Use probabilistic soft logic to easily specify models and efficiently optimize them Results: State-of-the-art performance on real-world datasets producing knowledge graphs with millions of facts

8 NELL: The Never-Ending Language Learner Large-scale IE project (Carlson et al., 2010) Lifelong learning: aims to read the web Ontology of known labels and relations Knowledge base contains millions of facts

9 Examples of NELL errors

10 Entity co-reference errors Kyrgyzstan has many variants: Kyrgystan Kyrgistan Kyrghyzstan Kyrgzstan Kyrgyz Republic

11 Missing and spurious labels Kyrgyzstan is labeled a bird and a country

12 Missing and spurious relations Kyrgyzstan s location is ambiguous Kazakhstan, Russia and US are included in possible locations

13 Violations of ontological knowledge Equivalence of co-referent entities (sameas) SameAs(Kyrgyzstan, Kyrgyz Republic) Mutual exclusion (disjointwith) of labels MUT(bird, country) Selectional preferences (domain/range) of relations RNG(countryLocation, continent) Enforcing these constraints require jointly considering multiple extractions

14 KNOWLEDGE GRAPH IDENTIFICATION

15 Motivating Problem (revised) (noisy) Extraction Graph Knowledge Graph Internet Large-scale IE Joint Reasoning =

16 Knowledge Graph Identification Problem: Knowledge Graph Extraction Graph Performs graph identification: entity resolution collective classification link prediction Knowledge Graph Identification Solution: Knowledge Graph Identification (KGI) Enforces ontological constraints Incorporates multiple uncertain sources =

17 Illustration of KGI: Extractions Uncertain Extractions:.5: Lbl(Kyrgyzstan, bird).7: Lbl(Kyrgyzstan, country).9: Lbl(Kyrgyz Republic, country).8: Rel(Kyrgyz Republic, Bishkek, hascapital)

18 Illustration of KGI: Extraction Graph Uncertain Extractions:.5: Lbl(Kyrgyzstan, bird).7: Lbl(Kyrgyzstan, country).9: Lbl(Kyrgyz Republic, country).8: Rel(Kyrgyz Republic, Bishkek, hascapital) Extraction Graph Kyrgyzstan Lbl bird country Kyrgyz Republic Rel(hasCapital) Bishkek

19 Illustration of KGI: Ontology + ER Uncertain Extractions:.5: Lbl(Kyrgyzstan, bird).7: Lbl(Kyrgyzstan, country).9: Lbl(Kyrgyz Republic, country).8: Rel(Kyrgyz Republic, Bishkek, hascapital) Ontology: Dom(hasCapital, country) Mut(country, bird) Entity Resolution: SameEnt(Kyrgyz Republic, Kyrgyzstan) (Annotated) Extraction Graph SameEnt Kyrgyzstan Lbl country bird Kyrgyz Republic Dom Rel(hasCapital) Bishkek

20 Illustration of KGI Uncertain Extractions:.5: Lbl(Kyrgyzstan, bird).7: Lbl(Kyrgyzstan, country).9: Lbl(Kyrgyz Republic, country).8: Rel(Kyrgyz Republic, Bishkek, hascapital) Ontology: Dom(hasCapital, country) Mut(country, bird) Entity Resolution: SameEnt(Kyrgyz Republic, Kyrgyzstan) (Annotated) Extraction Graph SameEnt Kyrgyzstan Lbl country bird Kyrgyz Republic Dom Rel(hasCapital) Bishkek After Knowledge Graph Identification Kyrgyzstan Lbl country Kyrgyz Republic Rel(hasCapital) Bishkek

21 (Pujara et al., ISWC13) Viewing KGI as a probabilistic graphical model Lbl(Kyrgyzstan, bird) Rel(hasCapital, Kyrgyzstan, Bishkek) Lbl(Kyrgyzstan, country) Lbl(Kyrgyz Republic, country) Lbl(Kyrgyz Republic, bird) Rel(hasCapital, Kyrgyz Republic, Bishkek)

22 (Pujara et al., ISWC13) Background: Probabilistic Soft Logic (PSL) (Broecheler et al., UAI10; Kimming et al., NIPS-ProbProg12) Templating language for hinge-loss MRFs, very scalable! Model specified as a collection of logical formulas SameEnt(E 1,E 2 ) ^ Lbl(E 1,L) ) Lbl(E 2,L) Uses soft-logic formulation Truth values of atoms relaxed to [0,1] interval Truth values of formulas derived from Lukasiewicz t-norm

23 (Pujara et al., ISWC13) Background: PSL Rules to Distributions Rules are grounded by substituting literals into formulas w EL : SameEnt(Kyrgyzstan, Kyrygyz Republic) ^ Lbl(Kyrgyzstan, country) ) Lbl(Kyrygyz Republic, country) Each ground rule has a weighted distance to satisfaction derived from the formula s truth value P(G E) = 1 Z exp $ % r R w r ϕ (G)& r ' The PSL program can be interpreted as a joint probability distribution over all variables in knowledge graph, conditioned on the extractions

24 (Pujara et al., ISWC13) Background: Finding the best knowledge graph MPE inference solves max G P(G) to find the best KG In PSL, inference solved by convex optimization Efficient: running time empirically scales with O( R ) (Bach et al., NIPS12)

25 (Pujara et al., ISWC13) PSL Rules for KGI Model

26 (Pujara et al., ISWC13) PSL Rules: Uncertain Extractions Weight for source T (relations) Predicate representing uncertain relation extraction from extractor T Relation in Knowledge Graph w CR-T : CandRel T (E 1,E 2,R) w CL-T : CandLbl T (E,L) ) Rel(E 1,E 2,R) ) Lbl(E,L) Weight for source T (labels) Predicate representing uncertain label extraction from extractor T Label in Knowledge Graph

27 (Pujara et al., ISWC13) PSL Rules: Entity Resolution SameEnt predicate captures confidence that entities are co-referent Rules require co-referent entities to have the same labels and relations Creates an equivalence class of co-referent entities

28 (Pujara et al., ISWC13) PSL Rules: Ontology Inverse: w O : Inv(R, S) ^ Rel(E 1,E 2,R) ) Rel(E 2,E 1,S) Selectional Preference: w O : Dom(R, L) ^ Rel(E 1,E 2,R) ) Lbl(E 1,L) w O : Rng(R, L) ^ Rel(E 1,E 2,R) ) Lbl(E 2,L) Subsumption: w O : Sub(L, P ) ^ Lbl(E,L) ) Lbl(E,P) w O : RSub(R, S) ^ Rel(E 1,E 2,R) ) Rel(E 1,E 2,S) Mutual Exclusion: w O : Mut(L 1,L 2 ) ^ Lbl(E,L 1 ) ) Lbl(E,L 2 ) w O : RMut(R, S) ^ Rel(E 1,E 2,R) ) Rel(E 1,E 2,S) Adapted from Jiang et al., ICDM 2012

29 [ 1] CandLbl struct (Kyrgyzstan, bird) ) Lbl(Kyrgyzstan, bird) φ 1 φ φ [ 2] CandRel pat (Kyrgyz Rep., Asia, locatedin) ) Rel(Kyrgyz Rep., Asia, locatedin) Lbl(Kyrgyzstan, bird) Lbl(Kyrgyz Rep., bird) [ 3] SameEnt(Kyrgyz Rep., Kyrgyzstan) ^ Lbl(Kyrgyz Rep., country) ) Lbl(Kyrgyzstan, country) φ 5 φ [ 4] Dom(locatedIn, country) ^ Rel(Kyrgyz Rep., Asia, locatedin) ) Lbl(Kyrgyz Rep., country) Lbl(Kyrgyzstan, country) Lbl(Kyrgyz Rep., country) [ 5] Mut(country, bird) ^ Lbl(Kyrgyzstan, country) ) Lbl(Kyrgyzstan, bird) φ φ 2 φ 3 φ 4 Rel(Kyrgyz Rep., Asia, locatedin) φ

30 Probability Distribution over KGs P(G E) = 1 Z exp $ % r R w r ϕ (G)& r ' CandLbl T (kyrgyzstan, bird) ) Lbl(kyrgyzstan, bird) Mut(bird, country) ^ Lbl(kyrgyzstan, bird) ) Lbl(kyrgyzstan, country) SameEnt(kyrgz republic, kyrgyzstan) ^ Lbl(kyrgz republic, country) ) Lbl(kyrgyzstan, country)

31 EVALUATION

32 (Pujara et al., ISWC13) Two Evaluation Datasets Description LinkedBrainz Community-supplied data about musical artists, labels, and creative works NELL Real-world IE system extracting general facts from the WWW Noise Realistic synthetic noise Imperfect extractors and ambiguous web pages Candidate Facts Unique Labels and Relations Ontological Constraints 810K 1.3M K

33 (Pujara et al., ISWC13) LinkedBrainz Open source communitydriven structured database of music metadata Uses proprietary schema to represent data Built on popular ontologies such as FOAF and FRBR Widely used for music data (e.g. BBC Music Site) LinkedBrainz project provides an RDF mapping from MusicBrainz data to Music Ontology using the D2RQ tool

34 (Pujara et al., ISWC13) LinkedBrainz dataset for KGI mo:release mo:record mo:record mo:track mo:track mo:published_as mo:signal mo:label mo:label foaf:maker mo:musicalartist inverseof subclassof subclassof foaf:made mo:solomusicartist mo:musicgroup Mapping to FRBR/FOAF ontology DOM RNG INV SUB RSUB MUT rdfs:domain rdfs:range owl:inverseof rdfs:subclassof rdfs:subpropertyof owl:disjointwith

35 (Pujara et al., ISWC13) LinkedBrainz experiments Comparisons: Baseline PSL-EROnly PSL-OntOnly PSL-KGI model Use noisy truth values as fact scores Only apply rules for Entity Resolution Only apply rules for Ontological reasoning Apply Knowledge Graph Identification AUC Precision Recall F1 at.5 Max F1 Baseline PSL-EROnly PSL-OntOnly PSL-KGI

36 (Pujara et al., ISWC13) NELL Evaluation: two settings Target Set: restrict to a subset of KG (Jiang, ICDM12) Complete: Infer full knowledge graph?? Closed-world model Uses a target set: subset of KG Derived from 2-hop neighborhood Excludes trivially satisfied variables Open-world model All possible entities, relations, labels Inference assigns truth value to each variable

37 (Pujara et al., ISWC13) NELL experiments: Target Set Task: Compute truth values of a target set derived from the evaluation data Comparisons: Baseline Average confidences of extractors for each fact in the NELL candidates NELL Evaluate NELL s promotions (on the full knowledge graph) MLN Method of (Jiang, ICDM12) estimates marginal probabilities with MC-SAT PSL-KGI Apply full Knowledge Graph Identification model Running Time: Inference completes in 10 seconds, values for 25K facts AUC F1 Baseline NELL MLN (Jiang, 12) PSL-KGI

38 (Pujara et al., ISWC13) NELL experiments: Complete knowledge graph Task: Compute a full knowledge graph from uncertain extractions Comparisons: NELL NELL s strategy: ensure ontological consistency with existing KB PSL-KGI Apply full Knowledge Graph Identification model Running Time: Inference completes in 130 minutes, producing 4.3M facts AUC Precision Recall F1 NELL PSL-KGI

39 Knowledge Graph Identification. Pujara, Miao, Getoor, Cohen Conclusion Knowledge Graph Identification is a powerful technique for producing knowledge graphs from noisy IE system output Using PSL we are able to enforce global ontological constraints and capture uncertainty in our model Unlike previous work, our approach infers complete knowledge graphs for datasets with millions of extractions Code available on GitHub: Questions?

40 Problem: Incremental Updates to KG? How do we add new extractions to the Knowledge Graph?

41 Naïve Approach: Full KGI over extractions

42 Improving the naïve approach Intuition: Much of previous KG does not change Online collective inference: Selectively update the MAP state Bound the regret of partial updates Efficiently determine which variables to infer

43 Key Idea: fix some variables, infer others

44 Key Collaborators

45 Approximation: KGI over subset of graph

46 Theory: Bounding Inference Regret Regret = kfull inference partial updatek

47 Theory: Bounding Inference Regret Regret = kfull inference partial updatek R n (x, y S ; ẇ), 1 n kh(x; ẇ) h(x, y S; ẇ)k 1

48 Theory: Bounding Inference Regret R n (x, y S ; ẇ), 1 n kh(x; ẇ) h(x, y S; ẇ)k 1 R n (x, y S ; ẇ) apple O s Bkwk 2 n w p ky S ŷ S k 1!