A Keyword-based Structured Query Language

Transcription

1 Expressive and Flexible Access to Web-Extracted Data : A Keyword-based Structured Query Language Department of Computer Science and Engineering Indian Institute of Technology Delhi 22th September 2011

2 Outline 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

3 Data Sets Unstructured data Data can be of any type. Do not follow any rules, sequence. e.g. text,video data. Structured data Data is organised. (entity) Similar entities are grouped together. (class,relation)

4 Information Overload Problem Generally, structured data sets have schema items numbering in the millions. Problem??

5 Information Overload Problem Generally, structured data sets have schema items numbering in the millions. Problem?? Writing structured queries is a tedious work due to the huge available information. This is Information Overload Problem. Solutions??

6 Outline 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

7 Information Need Find all people of German nationality who have won a Nobel award.

8 Information Need Find all people of German nationality who have won a Nobel award. Query q(x):- GERMAN PEOPLE(x), haswonprize(x,y), NOBEL PRIZE(y).

9 Information Need Find all people of German nationality who have won a Nobel award. Query q(x):- GERMAN PEOPLE(x), haswonprize(x,y), NOBEL PRIZE(y). Highly expressive. Very less flexible.

10 Outline 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

11 Keyword Query german has won nobel award.

12 Keyword Query german has won nobel award. Highly flexible. Less expressive.

13 Keyword Query german has won nobel award. Highly flexible. Less expressive. Our Goal Retain the expressive structure of structured queries. Also, incorporate the flexibility of keyword queries.

14 Outline 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

15 New Query german, has won(nobel award). This approach is flexible and is structured as well. Here, keywords will be replaced by candidate schema items.

16 New Query german, has won(nobel award). This approach is flexible and is structured as well. Here, keywords will be replaced by candidate schema items. Problem The number of possible queries is exponential in the size of the query.

17 An Example Have to use disambiguation steps before query evaluation to avoid exponential blow-up.

18 Outline Entity Search Semantic Search and Knowledge Base Preliminaries 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

19 Entity Search Entity Search Semantic Search and Knowledge Base Preliminaries Entity Search Entities form the most basic data-level construct. e.g. Los Angeles, Albert Einstein, etc. Reduces ambiguity in search. e.g. entity Max Planck is different from entity Max Planck Institute or the Max Planck Medal.

20 Outline Entity Search Semantic Search and Knowledge Base Preliminaries 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

21 Entity Search Semantic Search and Knowledge Base Preliminaries Semantic Search and Knowledge Base Semantic Search Entities and relations form the base level data constructs. Semantics defined by KBs are taken into consideration. e.g. search for GUITERIST would return documents about B B King, Jimi Hendrix.

22 Entity Search Semantic Search and Knowledge Base Preliminaries Semantic Search and Knowledge Base Semantic Search Entities and relations form the base level data constructs. Semantics defined by KBs are taken into consideration. e.g. search for GUITERIST would return documents about B B King, Jimi Hendrix. Knowledge Base A general term for a domain ontology and instance data. NOBEL LAUREATE is-a PERSON that has a haswonprize relation to some NOBEL PRIZE. Albert Einstein is-a PHYSICIST.

23 Outline Entity Search Semantic Search and Knowledge Base Preliminaries 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

24 Preliminaries Entity Search Semantic Search and Knowledge Base Preliminaries Entities denote constant terms such as people, places etc. Primitive concepts are types or classes to which entities may belong, e.g. SCIENTIST, COUNTRY. Relations are binary relationships that can exist between entities and/or primitive concepts, e.g. bornin. A Concept is any number of entities or primitive concepts joined by logical connectives such as conjunctions and relations. e.g. (SCIENTIST, haswonprize(nobel Prize in Physics)).

25 Outline 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

26

27 Structured Keyword Query Document Query DQ ::= Q (Q 1 ), (Q 2 ),..., (Q n ) where each Q i is a structured keyword query. Structured Keyword Query Q ::= k k(q) Q 1, Q 2,..., Q n where k is a keyword phrase. e.g. german,scientists, have won( nobel award ).

28 Disambiguation Graph Let Q be a structured keyword query. Then a disambiguation graph G = (V, E), and P a set of disjoint partitions of the nodes in V, are given by the following. V = k Q M(k). P = k Q {M(k)} E = edges(q).

29 Disambiguation Graph... Edge Generating Function edges(k) = {}. edges(k(q)) = ( n 1 M(k),n 2 root(q) (n 1, n 2 )) edges(q) edges(q 1, Q 2 ) = ( n 1 root(q 1 ),n 2 root(q 2 ) (n 1, n 2 )) edges(q 1 ) edges(q 2 )

30 Disambiguation Graph Example root(k) = M(k) root(k(q)) = M(k) root(q 1, Q 2 ) = root(q 1 ) root(q 2 ).

31 Semantic Similarity Semantic Similarity Sim Lin (A, B) = 2.log(P(LCS(A,B))) log(p(a))+log(p(b)) Sim Jaccard (A, B) = A B A B

32 Let C be a general concept expressed using the primitive concepts, relations, and entities in the knowledge base KB. Then the support of C by KB is given by, support(c, KB) = 0 if C is primitive. support(c, KB) = semanticsim(c 1, C 2,..., C n )+ i support(c i, KB), otherwise. where each C i is the i-th concept expression in a conjunction or relation occurring in C.

33 Knowledge Base Support... A Concept C GERMAN PEOPLE, SCIENTIST, haswonprize(nobel PRIZE), livesin(country, hasofficiallanguage(english Language)) Let C 1, C 2, C 3 be defined as follows: C 1 = haswonprize(nobel PRIZE) C 2 = livesin(country, C 3 ) C 3 = hasofficiallanguage(english Language) So, C= GERMAN PEOPLE, SCIENTIST, C 1, C 2.

34 Support Of A Concept support(c, KB) semanticsim(german PEOPLE, SCIENTIST, C 1, C 2 )+ semanticsim(haswonprize, NOBEL PRIZE)+ semanticsim(livesin, COUNTRY, C 3 )+ semanticsim(hasofficiallanguage, English Language).

35 Syntactic Similarity and Score Aggregation Syntactic Similarity syntactic matching of the concept label (or one of its synonyms) to the keyword. We use edit distance or q-gram distance. Score Aggregation Semantic and syntactic similarities are aggregated. It should be monotonic.

36 Concept Score Concept Score Let Q be a structured keyword query, C a concept, a binary aggregation function, and KB a knowledge base. Then, score(c, Q, KB) = support(c, KB) syntaxsim(c, Q). We define a new weight function as, w(v) = syntaxsim(label(v), k), where v M(k) w((v 1, v 2 )) = support((item(v 1 ), item(v 2 )), KB)

37 Approximate Score Approximate Score Let Q be a structured keyword query, KB a knowledge base, a binary aggregation function, and G a subgraph of the disambiguation graph of Q representing a concept C. Then, score(g, Q, KB) = ( (v 1,v 2 ) E w((v 1, v 2 )) v V w(v))

38 Outline 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

39 Disambiguation Problem Disambiguation Problem Find the top-k maximally scoring subgraphs corresponding to concept interpretations of the original query. Implement a rank-join algorithm assuming, scoring function is monotonic, sorted access to vertices and edges is possible.

40 Partial Interpretation Partial Interpretation No subgraph can be found that spans all partitions in a disambiguation graph for some query. Then, we build partial interpretations. We retrieve the documents and favor those which contain additional keywords. Rather, we can use document index to do a lookup of the keywords and then boosting documents from entity search, that occurs in this set.

41 Ranking Ranking The entities could be ranked by their relationships to the query in the KB, or based on statistics over the document corpus. Alternatively, we may have a document list as the result, ranked by the relevance of the corresponding entities to the document. We may also rank groups of documents per entity.

42 Outline Disambiguation Task Entity Search Task Document Retrieval Task 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

43 Disambiguation Task Disambiguation Task Entity Search Task Document Retrieval Task

44 Outline Disambiguation Task Entity Search Task Document Retrieval Task 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

45 Entity Search Task Disambiguation Task Entity Search Task Document Retrieval Task

46 Outline Disambiguation Task Entity Search Task Document Retrieval Task 1 2 Entity Search Semantic Search and Knowledge Base Preliminaries 3 4 Disambiguation Task Entity Search Task Document Retrieval Task

47 Document Retrieval Task Disambiguation Task Entity Search Task Document Retrieval Task

48 System Performance Disambiguation Task Entity Search Task Document Retrieval Task

49 Disambiguation Task Entity Search Task Document Retrieval Task Thank You