Towards Efficient and Effective Semantic Table Interpretation Ziqi Zhang

Transcription

1 Towards Efficient and Effective Semantic Table Interpretation Ziqi Zhang Department of Computer Science, University of Sheffield

2 Outline Define semantic table interpretation State-of-the-art and motivation The method TableMiner Evaluation

3 Semantic Table Interpretation Input Ontology Relational table Goals/Tasks Label columns by concepts Link cells to named entities Connect columns by relations Thing Artist Work Location Actor/ Actress Name Film Country 1 Tom Hanks Philadelphia USA 2 Jamie Foxx Ray USA 3 Kate Winslet The Reader UK 99 Charlize Theron Rel:performIn Film Rel:performIn < > Monster Table of Best Actor/Actress Country South Africa Ent:USA Ent:UK

4 Semantic Table Interpretation Input Ontology Relational table Goals/Tasks Label columns by concepts Link cells to named entities Connect columns by relations Column classification/ header disambiguation Cell disambiguation Relation interpretation

5 Motivation and State-of-the-art 154 mil. relational tables on the Web and growing [Cafarella2008] Classic Information Extraction methods do not work [Limaye2010, Lu2013] They cannot model the complex interdependence among table components

6 Motivation and State-of-the-art SoA semantic table interpretation methods, e.g. [Limaye2010, Venetis2011, Mulwad2013] Limitation 1 Inference is exhaustive, but unnecessary Name Film Country 1 Tom Hanks Philadelphia USA 2 Jamie Foxx Ray USA 3 Kate Winslet The Reader UK 99 Charlize Theron < > Monster Table of Best Actor/Actress South Africa Goal: Assign a concept to this column Hint: Content in the column gives useful clues How much do we need for inference (99 rows in this example)? - Human: SOME (learn by example) - SoA: ALL

7 Motivation and State-of-the-art SoA semantic table interpretation methods, e.g. [Limaye2010, Venetis2011, Mulwad2013] Limitation 2 Contextual features for inference SoA: features only from within the table Context outside the table also makes hint for interpretation. E.g., the words in the paragraph are often found in descriptions of actors Table of Best Actor/Actress

8 TableMiner

9 TableMiner Two tasks: Column classification Cell disambiguation Non-exhaustive inference in a bootstrapping pattern phase 1 inference with partial content phase 2 propagation and update Contextual features both inside and outside tables

10 TableMiner Phase 1 I-Inf Incremental inference with stopping (I-Inf) T j a column; C j candidate concepts for the column; E i,j candidate entities for a cell

11 TableMiner Phase 1 I-Inf Incremental inference with stopping (I-Inf) T j a column; C j candidate concepts for the column; E i,j candidate entities for a cell Itr.1 E i,j = {<e 1,s 1 >, <e 2,s 2 >, }. (until stop)

12 TableMiner Phase 1 I-Inf Incremental inference with stopping (I-Inf) T j a column; C j candidate concepts for the column; E i,j candidate entities for a cell C j = {<c 1,s 1 >, <c 2,s 2 >} concepts = {<c 1,s 1 >, <c 2,s 2 >, } Itr.1 E i,j = {<e 1,s 1 >, <e 2,s 2 >, }. (until stop)

13 TableMiner Phase 1 I-Inf Incremental inference with stopping (I-Inf) T j a column; C j candidate concepts for the column; E i,j candidate entities for a cell H(C j ) H(prevC j ) <t? C j = {<c 1,s 1 >, <c 2,s 2 >} Yes stop No next itr. concepts = {<c 1,s 1 >, <c 2,s 2 >, } Itr.1 E i,j = {<e 1,s 1 >, <e 2,s 2 >, }. (until stop)

14 TableMiner Phase 1 I-Inf Incremental inference with stopping (I-Inf) T j a column; C j candidate concepts for the column; E i,j candidate entities for a cell H(C j ) H(prevC j ) <t? C j = {<c 1,s 1 >, <c 2,s 2 >, <c 3,s 3 >} Yes stop No next itr. concepts = {<c 1,s 1 >, <c 3,s 3 >, } Itr.2 E i,j = {<e 1,s 1 >, <e 2,s 2 >, }. (until stop)

15 TableMiner Phase 1 I-Inf Incremental inference with stopping (I-Inf) T j a column; C j candidate concepts for the column; E i,j candidate entities for a cell H(C j ) H(prevC j ) <t? C j = {<c 1,s 1 >, <c 2,s 2 >, <c 3,s 3 >,. <c 11,s 11 >} Yes stop No next itr. concepts = {<c 11,s 11 >} Itr.3 E i,j =. {<e 1,s 1 >, <e 2,s 2 >, } (until stop)

16 TableMiner Phase 1 I-Inf To compute scores of candidate named entities (e.g. <e 1,s 1 >) and concepts (e.g., <c 1,s 1 >) Candidate NE Build a feature vector of a candidate using the ontology Build a feature vector of the cell/column header using its context Compute vector similarity Candidate concept: same principle, but also depends on score of contributing NEs

17 TableMiner Phase 2 Propagate, Update When I-Inf stops Select the highest scoring candidate concept c + to label the column Propagate: use c + as constrain to disambiguate remaining cells candidate NEs not belonging to c + are discarded Update: Re-compute c + after all cells are disambiguated If the new c + is different, revise disambiguation across the entire column with it as new constraint Repeat until no change Use as constraint to disambiguate cells C j = {<c 1,s 1 >, <c 2,s 2 >, <c 3,s 3 >,. <c 11,s 11 >} c + Rank and select

18 Evaluation

19 TableMiner Evaluation Data Freebase as reference ontology/background knowledge Limaye Web tables from Limaye2010 originally annotated with Wikipedia Cells are automatically mapped to Freebase some are unmapped Columns are manually annotated IMDB 7,354 cast tables of films mapped to Freebase

20 TableMiner Evaluation Baselines (both uses exhaustive inference) B first - cell disambiguation: choose the top ranked NE candidate in the Freebase search result - column classification: each disambiguated cell casts a vote to the set of concepts the NEs belong to, and the majority wins B sim - cell disambiguation: string similarity + feature vector similarity (in-table context only) - column classification: the majority vote method as above + string similarity

21 TableMiner Evaluation Results Cell disambiguation Manual validation of 932 cell annotations in Limaye112 not covered by the above results (i.e., unmapped cells) If only consider those cells where at least one system predicts correctly

22 TableMiner Evaluation Results Column classification best only a column is labelled correctly only if the concept is suitable for the data in the column and is specific enough best or ok a column is labelled correctly if the concept is suitable for the data in the column, though not very specific (E.g., Film Actors may be the best, while Artist or Person is OK, but Engineer is incorrect)

23 TableMiner Evaluation Results Efficiency TableMiner is efficient because Column classification: processes partial content from a column (avg. 57% Limaye112, 43% IMDB) Cell disambiguation: constrained by column classification, resulting in smaller NE candidate space (avg. 32% reduction Limaye32, 24% IMDB) Fewer candidates => less time spent on retrieval and feature space creation (typically >90% of CPU in the pipeline, Limaye2010)

24 TableMiner Conclusion TableMiner take-home messages How can it be more effective? Use both context within and outside tables as features for inference Message 1 How can it be more efficient? Perform inference with partial data and follow the bootstrapping pattern of learning Message 2

25 References [Cafarella2008] Cafarella, M.J., Halevy, A., Wang, D.Z., Wu, E., Zhang, Y. 2008: Webtables: exploring the power of tables on the web. Proceedings of VLDB Endowment 1(1), [Limaye2010] Limaye, G., Sarawagi, S., Chakrabarti, S. 2010: Annotating and searching web tables using entities, types and relationships. Proceedings of the VLDB Endowment 3(1-2), [Lu2013] Lu, C., Bing, L., Lam, W., Chan, K., Gu, Y. 2013: Web entity detection for semi-structured text data records with unlabeled data. International Journal of Computational Linguistics and Applications [Mulwad2013] Mulwad, V., Finin, T., Joshi, A. 2013: Semantic message passing for generating linked data from tables. In: International Semantic Web Conference (1). pp Lecture Notes in Computer Science, Springer [Venetis2011] Venetis, P., Halevy, A., Madhavan, J., Pas ca, M., Shen,W.,Wu, F., Miao, G.,Wu, C. 2011: Recovering semantics of tables on the web. Proceedings of VLDB Endowment 4(9),

26 Thank you