Hybrid Acquisition of Temporal Scopes for RDF Data

Transcription

1 Hybrid Acquisition of Temporal Scopes for RDF Data Anisa Rula 1, Matteo Palmonari 1, Axel-Cyrille Ngonga Ngomo 2, Daniel Gerber 2, Jens Lehmann 2, and Lorenz Bühmann 2 1. University of Milano-Bicocca, SITI Lab 2. Universität Leipzig, Institut für Informatik, AKSW

2 Anisa Rula 2 Outline 1. Introduction & Motivation 2. Approach Overview 3. Details of the Approach 4. Experimental Evaluation 5. Conclusions

3 Anisa Rula 3 Temporal Scoping of RDF triples

4 Temporal Scoping of RDF triples Some facts are always valid while other facts are valid for a certain time interval (volatile facts) Anisa Rula 3

5 Anisa Rula 3 Temporal Scoping of RDF triples Some facts are always valid while other facts are valid for a certain time interval (volatile facts) Volatile facts are represented by triples whose validity is defined by a time interval i.e. the temporal scope

6 Anisa Rula 3 Temporal Scoping of RDF triples Some facts are always valid while other facts are valid for a certain time interval (volatile facts) Volatile facts are represented by triples whose validity is defined by a time interval i.e. the temporal scope team A.C. Milan Alexandre Pato team S.C. Corinthians

7 Anisa Rula 3 Temporal Scoping of RDF triples Some facts are always valid while other facts are valid for a certain time interval (volatile facts) Volatile facts are represented by triples whose validity is defined by a time interval i.e. the temporal scope Temporal scopes, represented by time intervals team A.C. Milan Alexandre Pato team S.C. Corinthians

8 Anisa Rula 3 Temporal Scoping of RDF triples Some facts are always valid while other facts are valid for a certain time interval (volatile facts) Volatile facts are represented by triples whose validity is defined by a time interval i.e. the temporal scope Temporally annotated RDF triples Temporal scopes, represented by time intervals team A.C. Milan Alexandre Pato team S.C. Corinthians

9 Motivation & Challenges Motivation Anisa Rula 4

10 Motivation & Challenges Motivation World changes: relations represented in RDF triples may be valid only for a specific time interval [Gutierrez et al.,2005] o E.g. <Alexandre_Pato, team, A.C._Milan> [2007,2013] Anisa Rula 4

11 Motivation & Challenges Motivation World changes: relations represented in RDF triples may be valid only for a specific time interval [Gutierrez et al.,2005] o E.g. <Alexandre_Pato, team, A.C._Milan> [2007,2013] Many applications have to use temporally annotated RDF triples o E.g. Temporal Query Answering, Question Answering over KBs, Temporal Reasoning, Timelines Anisa Rula 4

12 Anisa Rula 4 Motivation & Challenges Motivation World changes: relations represented in RDF triples may be valid only for a specific time interval [Gutierrez et al.,2005] Temporally annotated RDF triples are largely unavailable or incomplete in the LOD o E.g. <Alexandre_Pato, team, A.C._Milan> [2007,2013] Many applications have to use temporally annotated RDF triples o E.g. Temporal Query Answering, Question Answering over KBs, Temporal (Rula et al., 2012) Reasoning, Timelines

13 Anisa Rula 4 Motivation & Challenges Motivation World changes: relations represented in RDF triples may be valid only for a specific time interval [Gutierrez et al.,2005] Temporally annotated RDF triples are largely unavailable or incomplete in the LOD o E.g. <Alexandre_Pato, team, A.C._Milan> [2007,2013] Many applications have to use temporally annotated RDF triples o E.g. Temporal Query Answering, Question Answering over KBs, Temporal (Rula et al., 2012) Reasoning, Timelines Challenges

14 Anisa Rula 4 Motivation & Challenges Motivation World changes: relations represented in RDF triples may be valid only for a specific time interval [Gutierrez et al.,2005] Temporally annotated RDF triples are largely unavailable or incomplete in the LOD o E.g. <Alexandre_Pato, team, A.C._Milan> [2007,2013] Many applications have to use temporally annotated RDF triples o E.g. Temporal Query Answering, Question Answering over KBs, Temporal (Rula et al., 2012) Reasoning, Timelines Challenges Low availability and quality of temporal information in RDF data

15 Anisa Rula 4 Motivation & Challenges Motivation World changes: relations represented in RDF triples may be valid only for a specific time interval [Gutierrez et al.,2005] Temporally annotated RDF triples are largely unavailable or incomplete in the LOD o E.g. <Alexandre_Pato, team, A.C._Milan> [2007,2013] Many applications have to use temporally annotated RDF triples o E.g. Temporal Query Answering, Question Answering over KBs, Temporal (Rula et al., 2012) Reasoning, Timelines Challenges Low availability and quality of temporal information in RDF data NLP challenges for web-scale temporal information extraction (scalability, availability of corpus, conflicting information) [Derczynsk et al., 2013, Ling et al., 2010]

16 Approach Overview: Use the Web as Source of Evidence Anisa Rula Anisa Rula 5 Use evidence from the Web for temporal scoping of RDF triples team A.C. Milan Alexandre Pato team S.C. Corinthians

17 Approach Overview: Use the Web as Source of Evidence Anisa Rula Anisa Rula 5 Use evidence from the Web for temporal scoping of RDF triples team A.C. Milan Alexandre Pato team S.C. Corinthians Source of evidence Web of Data - RDF (61.9 Billion) World Wide Web (1.8 Billion)

18 Approach Overview: Use the Web as Source of Evidence Anisa Rula Anisa Rula 5 Use evidence from the Web for temporal scoping of RDF triples team A.C. Milan Alexandre Pato team S.C. Corinthians Source of evidence Web of Data - RDF (61.9 Billion) World Wide Web (1.8 Billion)

19 Approach Overview: Use the Web as Source of Evidence Anisa Rula Anisa Rula 5 Use evidence from the Web for temporal scoping of RDF triples team A.C. Milan Alexandre Pato team S.C. Corinthians Source of evidence Web of Data - RDF (61.9 Billion) World Wide Web (1.8 Billion) Temporally annotated RDF triples Alexandre Pato team team A.C. Milan S.C. Corinthians

20 Approach Overview: Hybrid Acquisition of Time Scopes Anisa Rula 6 Temporal Information Extraction Mapping facts to time intervals Temporally annotated RDF triples t 1 occ 1 t 2 occ 2 Web of Documents t 3 occ 3 t 4 occ 4 Matching Selection <s,p,o> fact <s,p,o>[x 1,y 1 ],,[x n,y n ] Reasoning Set of disconnected time intervals Web of Data

21 Anisa Rula 7 Temporal Information Extraction - Web Documents DeFacto [Lehmann & al. 2012] Retrieves a set of webpages that confirm the given RDF triple The RDF triple issued to the search engine is verbalized by using natural language patterns Temporal Extension for DeFacto (TempDeFacto) Apply Named Entity Tagger to extract the entities of type Date class Observe the occurrences of the labels of the subject and object in less than 20 tokens Analyze the context window of n characters before and after subjectobject occurrences in order to retrieve the time points Return a distribution vector of date and their number of occurrences

22 Anisa Rula 8 Temporal Information Extraction - Web Documents <Alexandre_Pato,team, A.C._Milan>

23 Anisa Rula 8 Temporal Information Extraction - Web Documents <Alexandre_Pato,team, A.C._Milan>

24 Anisa Rula 8 Temporal Information Extraction - Web Documents <Alexandre_Pato,team, A.C._Milan> Occurrences of the labels of the subject and object Alexandre Pato played for A.C. Milan Pato s striker Milan CR7 Mi

25 Anisa Rula 8 Temporal Information Extraction - Web Documents <Alexandre_Pato,team, A.C._Milan> Occurrences of the labels of the subject and object Alexandre Pato played for A.C. Milan Pato s striker Milan CR7 Mi Context window of n characters before and after subject-object occurrences Pato played for A.C. Milan from 2007 to A.C. Milan s top striker Pato left in In 2013 Pato visited Milan for a short holiday.

26 Anisa Rula 8 Temporal Information Extraction - Web Documents <Alexandre_Pato,team, A.C._Milan> Occurrences of the labels of the subject and object Alexandre Pato played for A.C. Milan Pato s striker Milan CR7 Mi Context window of n characters before and after subject-object occurrences Pato played for A.C. Milan from 2007 to A.C. Milan s top striker Pato left in In 2013 Pato visited Milan for a short holiday. Named Entity Tagger

27 Anisa Rula 8 Temporal Information Extraction - Web Documents <Alexandre_Pato,team, A.C._Milan> Occurrences of the labels of the subject and object Alexandre Pato played for A.C. Milan Pato s striker Milan CR7 Mi Context window of n characters before and after subject-object occurrences Pato played for A.C. Milan from 2007 to A.C. Milan s top striker Pato left in In 2013 Pato visited Milan for a short holiday. Named Entity Tagger DeFacto Vector (dfv)

28 Anisa Rula 9 Temporal Information Extraction - Web of Data Content negotiation <Alexandre_Pato> RDF document d Alexandre_Pato The set of time intervals for a given triple with starting and ending time points defined with the set of relevant time points Regular expressions T Alexandre_Pato Relevant Time Points = {1989, 2000, 2006, 2007, 2008, 2013}

29 Anisa Rula 9 Temporal Information Extraction - Web of Data Content negotiation <Alexandre_Pato> RDF document d Alexandre_Pato The set of time intervals for a given triple with starting and ending time points defined with the set of relevant time points Relevant Interval Matrix (RIM) Regular expressions null null null null null null 0 null null null null null 0 0 null null null null null null null null null null T Alexandre_Pato Relevant Time Points = {1989, 2000, 2006, 2007, 2008, 2013} rrrrrr ttii tt jj RRRRRR ee wwwwwww ii, jj > 0 ffffff ii jj rrrrrr ttii tt jj = nnnnnnnn ffffff ii > jj rrrrrr ttii tt jj = 0

30 Anisa Rula 10 Mapping Facts to Time Intervals - Matching 1. Matching temporal distribution (dfv) against the relevant time interval matrix dfv RIM null null null null null null RDF data null null null null null null null null null null null null null null null Matching Selection Reasoning

31 Anisa Rula 10 Mapping Facts to Time Intervals - Matching 1. Matching temporal distribution (dfv) against the relevant time interval matrix dfv null null null null null null RDF data RIM null null null null null null null null null null null null null null null Matching Selection Reasoning Significance Matrix (SM)

32 Anisa Rula 10 Mapping Facts to Time Intervals - Matching 1. Matching temporal distribution (dfv) against the relevant time interval matrix dfv null null null null null null RDF data RIM null null null null null null null null null null null null null null null Matching Selection Reasoning Significance Matrix (SM) ssss 2007:2008 = = 6.5

33 Anisa Rula 11 Mapping Facts to Time Intervals - Selection 2. Mapping Selection: Matching top-k function: selects the k intervals that have highest scores in the SM neighbor-x: selects a set of intervals whose significance score is close to the maximum significance score in the SM matrix, up to a certain threshold x neighbor-k-x: selects the top-k intervals in the neighborhood of the interval with higher significance score Selection Reasoning SM

34 Anisa Rula 11 Mapping Facts to Time Intervals - Selection 2. Mapping Selection: Matching top-k function: selects the k intervals that have highest scores in the SM neighbor-x: selects a set of intervals whose significance score is close to the maximum significance score in the SM matrix, up to a certain threshold x neighbor-k-x: selects the top-k intervals in the neighborhood of the interval with higher significance score Selection Reasoning SM top-k, kk = 3 [2006,2013][2007, 2013][2008, 2013]

35 Anisa Rula 11 Mapping Facts to Time Intervals - Selection 2. Mapping Selection: Matching top-k function: selects the k intervals that have highest scores in the SM neighbor-x: selects a set of intervals whose significance score is close to the maximum significance score in the SM matrix, up to a certain threshold x neighbor-k-x: selects the top-k intervals in the neighborhood of the interval with higher significance score Selection Reasoning 1989 SM top-k, kk = 3 [2006,2013][2007, 2013][2008, 2013] neighbor, xx = 23 [2007,2008][2006,2013][2007, 2013][2008, 2013]

36 Anisa Rula 11 Mapping Facts to Time Intervals - Selection 2. Mapping Selection: Matching top-k function: selects the k intervals that have highest scores in the SM neighbor-x: selects a set of intervals whose significance score is close to the maximum significance score in the SM matrix, up to a certain threshold x neighbor-k-x: selects the top-k intervals in the neighborhood of the interval with higher significance score Selection Reasoning SM top-k, kk = 3 [2006,2013][2007, 2013][2008, 2013] neighbor, xx = 23 [2007,2008][2006,2013][2007, 2013][2008, 2013] neighbor-k-x, kk = 2, xx = 23 [2007, 2013][2008, 2013]

37 Anisa Rula 11 Mapping Facts to Time Intervals - Selection 2. Mapping Selection: Matching top-k function: selects the k intervals that have highest scores in the SM neighbor-x: selects a set of intervals whose significance score is close to the maximum significance score in the SM matrix, up to a certain threshold x neighbor-k-x: selects the top-k intervals in the neighborhood of the interval with higher significance score Selection Reasoning SM top-k, kk = 3 [2006,2013][2007, 2013][2008, 2013] neighbor, xx = 23 [2007,2008][2006,2013][2007, 2013][2008, 2013] neighbor-k-x, kk = 2, xx = 23 [2007, 2013][2008, 2013]

38 Anisa Rula 12 Mapping Facts to Time Intervals - Reasoning 3. Interval merging via reasoning based on Allen s algebra relation Matching Selection Reasoning

39 Anisa Rula 12 Mapping Facts to Time Intervals - Reasoning 3. Interval merging via reasoning based on Allen s algebra relation Matching Selection Reasoning [2007, 2013][2008, 2013] [ ] <Alexander_Pato,playsFor, A.C._Milan>

40 Anisa Rula 13 Experimental Setup - Dataset Dataset: 2500 DBpedia triples with semantic equivalent triples in Freebase and Yago2 Dataset # facts Domain Property Equivalent Property Freebase Yago2 DBpedia 1000 Sport team team playsfor DBpedia 1000 Politicians office government_positions_held holdspoliticalposition DBpedia 500 Celebrities spouse spouse ismarriedto Gold standard: triples annotated with temporal scopes in Yago2 manually curated to correct missing or wrong values

41 Experimental Setup - Evaluation Measures The evaluation measures capture the degree of overlap between the retrieved intervals and the intervals in the gold standard R Ref Precision (for a triple): number of time points in the temporal scope that fall into the time interval in the gold standard Recall (for a triple): number of time points in the gold standard that are covered by the temporal scope F1 measure (for a triple): the harmonic mean of precision and recall Anisa Rula 14

42 Anisa Rula 14 Experimental Setup - Evaluation Measures The evaluation measures capture the degree of overlap between the retrieved intervals and the intervals in the gold standard R Ref F1= F1= F1= Precision (for a triple): number of time points in the temporal scope that fall into the time interval in the gold standard Recall (for a triple): number of time points in the gold standard that are covered by the temporal scope F1 measure (for a triple): the harmonic mean of precision and recall

43 Anisa Rula 14 Experimental Setup - Evaluation Measures The evaluation measures capture the degree of overlap between the retrieved intervals and the intervals in the gold standard R Ref F1= F1= F1= Precision (for a triple): number of time points in the temporal scope that fall into the time interval in the gold standard Recall (for a triple): number of time points in the gold standard that are covered by the temporal scope F1 measure (for a triple): the harmonic mean of precision and recall Macro-averaged F1 (avgf-1): aggregated measure for a set of triples

44 Experimental Results - Accuracy of Best Configurations for all Properties Different sources for the creation of the RIM Setup different configurations in the selection and reasoning steps: o E.g. config top-3 refers to selection function top-3 and reasoning = yes Temp prop DBpedia Freebase TemporalDeFacto Config #facts avgf1 Config #facts avgf1 Config #facts avgf1 playsfor top-1 loc top-1 loc top holdspolitica lposition neigh neigh top ismarriedto neigh neigh top Good quality of the approach with an avgf1 of up to 70% Using evidence from RDF documents the performance can be significantly improved (significantly better results for two properties and negligibly worst results for one property) Anisa Rula 15

45 Anisa Rula 16 Experimental Results - Accuracy with vs. without Reasoning for all Properties The best configurations for the three properties Temp prop Source Configuration With reasoning Without reasoning #fact avgf1 #fact avgf1 playsfor TempDeFacto top holdspoliticalposition DBpedia neigh ismarriedto DBpedia neigh The best results are obtained when reasoning is enabled

46 Conclusions & Future Work Anisa Rula 17

47 Conclusions & Future Work Summary Temporal extension of the DeFacto framework Modeling a space of relevant time intervals given an RDF triple Mapping volatile facts to time intervals based on a three-phase algorithm Unsupervised method Anisa Rula 17

48 Conclusions & Future Work Summary Temporal extension of the DeFacto framework Modeling a space of relevant time intervals given an RDF triple Mapping volatile facts to time intervals based on a three-phase algorithm Unsupervised method Future work Anisa Rula 17

49 Conclusions & Future Work Summary Temporal extension of the DeFacto framework Modeling a space of relevant time intervals given an RDF triple Mapping volatile facts to time intervals based on a three-phase algorithm Unsupervised method Future work Determine when to add or not to add the temporal scope based on the confidence of the acquisition process Collect additional relevant time points to improve the overall results Show the effectiveness of acquired temporal scopes in temporal query answering Anisa Rula 17

50 Anisa Rula 18 Thank you for your attention Question? #eswc2014rula

51 References [Rula&2012] Anisa Rula, Matteo Palmonari, Andreas Harth, Steffen Stadtmüller, Andrea Maurino: On the Diversity and Availability of Temporal Information in Linked Open Data. International Semantic Web Conference (1) 2012: [Gutiérrez&2005] C. Gutierrez, C. A. Hurtado, and A. A. Vaisman. Temporal RDF. In The 2 nd ESWC, pages , 2005 [Lehmann&2012] Jens Lehmann, Daniel Gerber, Mohamed Morsey, Axel-Cyrille Ngonga Ngomo: DeFacto - Deep Fact Validation. International Semantic Web Conference (1) 2012: [Ling&2010] X. Ling and D. S. Weld. Temporal information extraction. In 25th AAAI, [Derczynsk&2013] L. Derczynski and R. Gaizauskas. Information retrieval for temporal bounding. In 4th ICTIR, pages 29:129 29:130. ACM, Anisa Rula 19

52 Approach Overview: Time Interval Representation and Relevant Interval Matrix When does <Alexander_Pato,playsFor, A.C._Milan>? All possible time intervals from all possible time points Triangular Matrix Vector of time points now now Relevant time intervals from a set of relevant time points Intuition: use evidence from the Web to reduce the set of considered time intervals and to identify the most significance time intervalanisa Rula 20

53 Experimental Results - Accuracy with Different Selection Functions Dataset: DBpedia and property:<holdspoliticalposition> 1 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 precision recall F1 top-k, k = 1 0,686 0,654 0,67 top-k, k = 2 0,515 0,865 0,645 top-k, k = 3 0,426 0,924 0,583 neighbor x = 10 0,689 0,709 0,699 For higher k in top-k selection, recall increases while precision decreases Best precision-recall trade-off with neighbor-x, x=10 Anisa Rula 21