Semantic Retrieval of the TIB AV-Portal. Dr. Sven Strobel IATUL 2015 July 9, 2015; Hannover

Transcription

1 Semantic Retrieval of the TIB AV-Portal Dr. Sven Strobel IATUL 2015 July 9, 2015; Hannover

2 Semantic Retrieval of the TIB AV-Portal Contents 1. TIB AV-Portal 2. Automatic Video Analysis 3. Named-Entity Recognition 4. Metadata and Retrieval 5. Semantic Retrieval 2

3 1. TIB AV-Portal Profile Free web-based portal for scientific videos from the realms of science & technology Automatic video analysis: scene, text, speech and image recognition Development Competence Centre for Non-Textual Materials at TIB in cooperation with Hasso Plattner Institute ; Launch: April 2014 av.getinfo.de Target Group Teachers, students, researchers 3

4 1. TIB AV-Portal Focus of the collection Videos from the fields of engineering, architecture, chemistry, informatics, mathematics and physics (TIB core subjects) Recordings of lectures and conferences, experiments, interviews, animations, simulations etc. Content 2900 videos / 1900 film credits with external links (June 2015) Most videos under open access av.getinfo.de 4

5 Semantic Retrieval of the TIB AV-Portal Contents 1. TIB AV-Portal 2. Automatic Video Analysis 3. Named-Entity Recognition 4. Metadata and Retrieval 5. Semantic Retrieval 5

6 2. Automatic Video Analysis DOI assignment Permanent linking / citability Time-related video segments Full-text search in the OCR transcript Full-text search in the speech transcript Search for image motifs Linking textual metadata (OCR / speech transcripts) with GND ontology Named entity recognition

7 Semantic Retrieval of the TIB AV-Portal Contents 1. TIB AV-Portal 2. Automatic Video Analysis 3. Named-Entity Recognition 4. Metadata and Retrieval 5. Semantic Retrieval 7

8 3. Named-Entity Recognition Definition Named-Entity Recognition Linking automatically extracted textual metadata with terms of a knowledge base Textual Metadata Knowledge Base OCR transcript OCR transcript OCR transcript OCR transcript Speech Speech transcript Speech transcript Speech transcript transcript GND subject headings GND subject sections for the 6 TIB core subjects 8

9 Textual Metadata Speech transcript 9

10 Knowledge Base Video segments indexed by GND subject headings 10

11 Algorithm of Named-Entity Recognition context disambiguate GND: Thermodynamik Figure is based on slide 37 from Steinmetz, N.; Sack, H.: Cross-Lingual Semantic Mapping of Authority Files. Presentation held at Semantic Web in Libraries Hamburg (2013). 11

12 Benefits of Named-Entity Recognition Fine-grained descriptions of the video segments enable pinpoint segment-based searches within the video content. Linking textual metadata with the GND ontology enables a semantic search. 12

13 Semantic Retrieval of the TIB AV-Portal Contents 1. TIB AV-Portal 2. Automatic Video Analysis 3. Named-Entity Recognition 4. Metadata and Retrieval 5. Semantic Retrieval 13

14 4. Metadata and Retrieval Taxonomic Schema Metadata Manual metadata Coarse grained Highly reliable Automatic metadata Fine grained Less reliable Search for classical metadata (title, author...) OCR transcript Keyword based full text search in the written content of the video Speech transcript Keyword based full text search in the spoken content of the video Automatic indexing 14

15 4. Metadata and Retrieval Taxonomic Schema

16 Semantic Retrieval of the TIB AV-Portal Contents 1. TIB AV-Portal 2. Automatic Video Analysis 3. Named-Entity Recognition 4. Metadata and Retrieval 5. Semantic Retrieval 16

17 5. Semantic Retrieval Semantic search is based on the TIB AV-Portal knowledge base. This knowledge base includes among other things: GND subject headings plus synonyms English translations of the GND subject headings from DBpedia, LCSH, MACS and WTI Thesaurus 17

18 Textual Query When the user enters a search term, all available synonyms and English (or German) translations from the TIB AV-Portal knowledge base are automatically included in the query. 18

19 Textual Query Example: Wärmelehre Thermodynamik (speech transcript) Thermodynamics (speech transcript) Thermodynamik (GND term) Thermodynamics (manual metadata) 19

20 Semantic faceted search Refine search results Facets: Subject Language Author & contributors Publisher Licence Year of Publication Person Organization Image motif 20

21 Semantic faceted search Search index Facet terms are terms from GND. Search index stores: URI of the GND term ID of the video Position, which was assigned to that term No keyword-based search but rather an entity -search Search returns videos that contain the selected faceted term and highlights the corresponding video segments 21

22 Semantic faceted search Example of a search result 22

23 Semantic faceted search Benefits By clicking on a facet, that term, synonyms of that term and translations of that term are included in the query. Improving Recall GND facet terms are disambiguated. Improving Precision 23

24 Thank you for your attention!