Using BabelNet in Bridging the Gap Between Natural Language Queries and Linked Data Concepts

Transcription

1 Using BabelNet in Bridging the Gap Between Natural Language Queries and Linked Data Concepts Khadija Elbedweihy, Stuart N. Wrigley, Fabio Ciravegna and and Ziqi Zhang OAK Research Group, Department of Computer Science, University of Sheffield, UK

2 Outline MoOvaOon and Problem Statement Natural Language Query Approach Approach Steps EvaluaOon Results and Discussion

3 MoOvaOon SemanOc Search Wikipedia states that Seman&c Search: seeks to improve search accuracy by understanding searcher intent and the contextual meaning of terms as they appear in the searchable dataspace, whether on the Web or within a closed system, to generate more relevant results SemanOc search evaluaoons reported user preference for free natural language as a query approach (simple, fast & flexible) as opposed to controlled or view- based inputs.

4 Problem Statement Complete freedom increases difficulty of matching query terms with the underlying data and ontologies. Word sense disambiguaoon (WSD) is core to the soluoon. QuesOon: How tall is...? : property height tall is polysemous, should be first disambiguated: great in verocal dimension; tall people; tall buildings, etc. too improbable to admit of belief; a tall story, Another difficulty: Named EnOty (NE) recognioon and disambiguaoon.

5 Approach Free- NL semanoc search approach, matching user query terms with the underlying ontology using: 1) An extended- Lesk WSD approach. 2) A NE recogniser. 3) A set of advanced string similarity algorithms and ontology- based heurisocs to match disambiguated query terms to ontology concepts and properoes.

6 Extended- Lesk WSD approach WordNet is predominant, however its granularity is a problem for achieving high performance in WSD. BabelNet is a very large mulolingual ontology with wide- coverage obtained from both WordNet and Wikipedia. For disambiguaoon, bags are extended with senses glosses and different lexical and semanoc relaoons. Include synonyms, hyponyms, hypernyms, abribute, see also and similar to relaoons.

7 Extended- Lesk WSD approach InformaOon added from a Wikipedia page (W), mapped to a WordNet synset includes: 1. labels; page Play (theatre) à add play and theatre 2. set of pages redirecong to W; Playlet redirects to Play 3. set of pages linked from W; links in the page Play (theatre) include literature, comedy, etc. Synonyms of synset S, associated with Wikipedia page W: WordNet synonyms of S in addioon to lemmas of wikipedia informaoon of W".

8 Extended- Lesk WSD approach Feature P R F 1 Baseline Synonyms Syn + hypo Syn + gloss examples (WN) Syn + gloss examples (Wiki) Syn + gloss examples (WN + Wiki) Syn + hyper Syn + semrel Syn + hypo + gloss(wn) Syn + hypo + gloss(wn) + hyper Syn + hypo + gloss(wn) + hyper + semrel Syn+hypo+gloss(WN)+hyper+semRel+relGlosses Sentences with less than seven words: f- measure of 81.34%

9 Approach Steps 1. RecogniOon and disambiguaoon of Named EnOOes. 2. Parsing and DisambiguaOon of the NL query. 3. Matching query terms with ontology concepts and properoes. 4. GeneraOon of candidate triples. 5. IntegraOon of triples and generaoon of SPARQL queries.

10 1.RecogniOon and disambiguaoon of Named EnOOes Named enooes recognised using AlchemyAPI. AlchemyAPI had the best recognioon performance in NERD evaluaoon of SOA NE recognizers. AlchemyAPI exhibits poor disambiguaoon performance Each NE is disambiguated using our BabelNet- based WSD approach.

11 1.RecogniOon and disambiguaoon of Named EnOOes Example: In which country does the Nile start? Matches of Nile in BabelNet include: hbp://dbpedia.org/resource/nile (singer) hbp://dbpedia.org/resource/nile (TV series) hbp://dbpedia.org/resource/nile (band) hbp://dbpedia.org/resource/nile Match selected (Nile: river): overlapping terms between sense and query (geography, area, culture, cononent) more than other senses.

12 2.Parsing and DisambiguaOon of the NL query Stanford Parser used to gather lemmas and POS tags. Proper nouns idenofied by the parser and not recognized by AlchemyAPI are disambiguated and added to the recognized enooes. Example: In which country does the Nile start? The algorithm does not miss the enoty Nile, although it was not recognized by AlchemyAPI.

13 2.Parsing and DisambiguaOon of the NL query Example: Which socware has been developed by organiza&ons founded in California? Output: Word Lemma POS posilon sorware sorware NP 1 developed develop VBN 2 organizaoons organize NNS 3 founded find VBN 4 California California NP 5 Equivalent output generated using keywords or phrases.

14 3.Matching Query Terms with Ontology Concepts & ProperOes Noun phrases, nouns and adjecoves are matched with concepts and properoes. Verbs are matched only with properoes. Candidate ontology matches ordered using: Jaro- Winkler and Double Metaphone string similarity algorithms. Jaro- Winkler threshold to accept a match is set to 0.791, shown in literature to be the best threshold value.

15 3.Matching Query Terms with Ontology Concepts & ProperOes Matching process uses the following in order: 1. query term (e.g., created) 2. lemma (e.g., create) 3. derivaoonally related forms (creator) If no matches, disambiguate query term and use expansion terms in order: 1. synonyms 2. hyponyms 3. hypernyms 4. semanoc relaoons (e.g., height as an abribute for tall)

16 4. GeneraOon of Candidate Query Triples Structure of the ontology (taxonomy of classes and domain and range of properoes) used to link matched concepts and proper>es and recognized en>>es to generate query triples. Three- Terms Rule Each three consecuove terms matched with set of templates. E.g., Which television shows were created by Walt Disney? Template (concept- property- instance) generates triples:?television_show <dbo:creator> <res:walt_disney>?television_show <dbp:creator> <res:walt_disney>?television_show <dbo:creativedirector> <res:walt_disney>

17 Three- Terms Rule Examples of templates used in three- terms rule: concept- property- instance airports located in California actors born in Germany instance- property- instance Was Natalie Portman born in the United States? property- concept- instance birthdays of actors of television show Charmed

18 Two- Terms Rule Two- Terms Rule, used when: 1) There is fewer than three derived terms 2) No match between query terms and three- term template 3) Matched template did not generate candidate triples E.g., In which films directed by Garry Marshall was Julia Roberts starring? <Garry Marshall, Julia Roberts, starring> : matched to a three- terms template but does not generate triples.

19 Two- Terms Rule Two- Terms Rule QuesOon: what is the area code of Berlin? Template (property- instance) generates the triples: <res:berlin> <dbp:areacode>?area_code <res:berlin> <dbo:areacode>?area_code

20 ComparaOves Compara>ves Scenarios: 1) ComparaOve used with a numeric datatype property: e.g., companies with more than 500,000 employees?company <dbp:numemployees>?employee?company <dbp:numberofemployees>?employee?company a <dbocompany> FILTER (?employee > )

21 ComparaOves 2) ComparaOve is used with a concept: e.g., places with more than 2 caves Generate the same triples for places with caves:?place a < a < Add the aggregate restricoon: GROUP BY?place HAVING (COUNT(?cave)>2).

22 ComparaOves 3) ComparaOve is used with an object property e.g., countries with more than 2 official languages Similarly, generate the same triples for country and official language and add the restricoon: GROUP BY?country HAVING (COUNT(?official_language) > 2) 4) Generic ComparaOves e.g., Which mountains are higher than the Nanga Parbat?

23 Generic ComparaOves Difficulty: idenofy the property referred to by the compara>ve term. 1) Select best relaoon according to query context. IdenOfy all numeric datatype properoes associated with the concept mountain, include: lats, longd, prominence, firstascent, elevaoon, longm, 2) Disambiguate synsets of all properoes and use WSD approach to idenofy the most related synset to the query. property elevaoon is correctly selected

24 5. IntegraOon of Triples and GeneraOon of SPARQL Queries Generated triples integrated to produce SPARQL query. Query term posioons used to order the generated triples. Triples originaong from the same query term are executed in order unol an answer is found. Duplicates are removed while merging the triples. SELECT and WHERE clauses added in addioon to any aggregate restricoons or soluoon modifiers.

25 EvaluaOon Test data from 2nd Open Challenge at QALD- 2. Results produced by QALD- 2 evaluaoon tool. Very promising results: 76% of quesoons answered correct. Approach Answered Correct Precision Recall F 1 BELA QAKiS Alexandria SenseAware SemSeK MHE

26 Discussion Design choices affected by priority for precision or recall: 1. Query Relaxa>on e.g., Give me all actors starring in Last Ac&on Hero RestricOng results to actors harms recall Not all enooes in LD are typed, let alone correctly typed Query relaxaoon favors recall but affects precision e.g. How many films did Leonardo DiCaprio star in? Return TV series rather than only films such as res:parenthood (1990 TV series). Decision: favor precision; keep restricoon when specified.

27 2. Best or All Matches Discussion e.g., sorware by organizaoons founded in California ProperOes matched: foundaoon and foundaoonplace Using only best match (foundaoon ) does not generate all results à affects recall. Using all properoes (may not be relevant to the query) would harm precision. Decision: use all matches; with high value for the similarity threshold; perform checks against the ontology structure to assure relevant matches are only used.

28 3. Query Expansion Discussion Can be useful for recall, when the query term is not sufficient to return all answers. Example: use website and homepage if any of them used in a query and both have matches in the ontology. Quality of expansion terms influenced by WSD approach; wrong sense idenoficaoon will lead to noisy list of terms. Decision: perform query expansion only when no matches found in the ontology for a term; or no results generated using the idenofied matches.

29 QuesOons QuesOons?!

30 AddiOonal Slides AddiOonal Slides!