TIB AV-Portal. Margret Plank 19th of January 2015 TACC Meeting

Transcription

1 TIB AV-Portal Margret Plank 19th of January 2015 TACC Meeting

2 German National Library of Science and Technology (TIB) German National Library of Science and Technology for all areas of engineering as well as architecture, chemistry, computer science, mathematics and physics Financed by German Federal Government and all Federal States TIB portal for science and technology offers more than 160 million data sets from professional databases, publishers services and library catalogues Access to full texts, research data, audiovisual media and 3D models DOI registration agency for research data since 2005 Source: TIB 2

3 Competence Centre for non-textual materials The significance of non-textual materials like Videos and 3D-Objects in research and teaching is steadily increasing. The goal of the competence centre is to improve access to and the use of nontextual material. 3

4 Competence Centre for nontextual materials TIB AV-Portal Scope Mediaspecific portal for sientific videos Automated video analysis with scene, speech, text and image recognition crosslingual retrieval english / german Persistent identifiers (DOI) and Media fragment Identifiers (MFID) Target Group Researchers, university staff, students Content Research & teaching films covering topics from science and technology 2500 videos / 900 hours ( Jan. 2015) Most of it under open access licences 4

5 What is meant by scientific videos? Recordings of conferences and other scientific lectures, talks and discussions Recordings of experiments from the area of research and development Microscopic images, images taken using special cameras and techniques Documentation and presentation of research work and results Presentations of 3D models (such as architectural models) on film Recordings of university seminars and lectures elearning material such as online tutorials, MOOCs, teaching experiments and experiments for private study Presentations of scientific organisations Documentaries, reports and portraits 5

6 Scientific Videos Scientific videos are a valuable source - to illustrate and share knowledge concerning findings, methodologies or procedures within the scientific community - to make e.g. lab experiments transparent and reproducible - visualize temporal components by e.g. using zooming or stretching techniques etc. User needs High quality content Free access and usage of videos Advanced findability on a permanent basis Citation of videos and video segments Links to videos and video segments Interlinkage with further research data 6

7 Video Portals Can YouTube manage this? NO You cant find what you are looking for if it s not in the title You cant find a specific video without the URL Unclear how long the content will be there You can only cite the whole video not a segment You can only link to the title not to a segment 7

8 Video Portals Can other video portals manage this? To some extend. 8

9 TIB Services for audiovisual media Access Citability Permanence Long term preservation Searchability Curation Cataloging Licencing quality content Generate recognition for the creators 9

10 TIB AV-Portal 10

11 1. Automatic Video Analysis of the TIB AV-Portal 1. Spoken Language Skorupka, Sascha: Experiment der Woche, Visual Concepts 3. Text Overlays 4. Structural Information (Video Segments) 11

12 2. Automatic Indexing of Videos on the Basis of the GND Text Analysis: OCR analysis of text overlays (e.g. on slides) OCR transcript Audio Analysis: speech to text audio transcript Named Entity Recognition - Reference Vocabulary: subject headings of the Gemeinsame Normdatei (GND = Integrated Authority File) - Analyzed Text Content: OCR transcript and audio transcript 12

13 Automatic Indexing on the Segment Level The individual segments of the video are automatically indexed with GND subject headings. Pinpoint segment based searches within the videos Skorupka, Sascha: Experiment der Woche,

14 Problem: English Labels are Lacking in the GND The GND contains only very few English labels regarding the subject headings used in the AV Portal knowledge base: subject headings from the field of science and technology. 1 For the automatic indexing of English-language videos, an English indexing vocabulary is lacking No segment-based queries within English-language videos using subject headings 1 In 2013, the results of the MACS project had not yet been published in the GND dump. 14

15 3. Mapping the GND Entities onto the DBpedia and other Authority Files Solving the Problem Gaining English labels for the GND entities of the AV Portal knowledge base ( subject headings) by mapping the GND entities onto the DBpedia and other authority files (LCSH, MACS results, and the WTI Thesaurus) DBpedia The DBpedia contains structured information from the Wikipedia (info boxes, tables, web links). This information is made available as Linked Data. 15

16 Mapping the GND Entities onto the DBpedia 2 context labels Advent Vorweihnachtszeit Adventszeit Vorweihnachtsfest find DBpedia candidate(s) dbpde:advent dbpde:the_advent dbpde:advent_cornwall dbpde:advent_creek dbpde:advent_computers dbpde:advent_(band) dbpde:advent_(publisher) dbpde:advent_records dbpde:advent_international disambiguate dbpde:advent language link dbp:advent 2 Figure: Steinmetz, N. / Sack, H.: Cross-Lingual Semantic Mapping of Authority Files. SWIB 2013, Hamburg. Foliennr

17 4. Results English Labels GND Subject Headings German DBpedia English DBpedia % % GND subject headings could be translated into English by means of the GND/DBpedia mapping Overall result of all mapping strategies ( DBpedia; LCSH; MACS; WTI Thesaurus): (55%) GND subject headings got (at least) one English label 17

18 Quality of the Results of the Mapping Algorithm GND German DBpedia Ground Truth (Evaluation Data) 180 manual mappings of GND entities of the AV portal knowledge base to the corresponding German DBpedia entities Recall The algorithm mapped 90% of the GND entities of the Ground Truth. Precision 92% of the GND entities were mapped correctly. 18

19 5. Additional Benefit of the GND/DBpedia Mapping Context Information Problem GND contains only a small amount of context information. Often, a reliable disambiguation of GND entities in the process of the Named Entity Recognition is not possible. 19

20 Disambiguation of GND Entities Audio Transcript Die Winde arbeitet nach dem Prinzip des Wellrads, kombiniert mit einem Hebel... Context Information Luftströmung Meteorologie Klimatologie Hochatmosphäre Magnetosphäre Context Information Hebezeug Hebel Wellrad Fahrzeugbau Fördertechnik Wind <wind> Winde <windlass> Winde is indexed

21 Mapping the GND Entities onto the DBpedia Solving the Problem Extracting context information from the DBpedia for the GND entities of the AV Portal knowledge base Improving the disambiguation of GND entities Context Information from the DBpedia WikiLinks (Internal Wikipedia Links) of the DBpedia entities Wikipedia articles to which the DBpedia entities refer 21

22 DBpedia 22

23 Results Context Information GND Subject Headings German DBpedia English DBpedia % % For GND subject headings, additional context information could be extracted from the German DBpedia (WikiLinks, Wikipedia articles) Improving the disambiguation of the German-language NER For GND subject headings, additional context information could be extracted from the English DBpedia (WikiLinks, Wikipedia articles) Improving the disambiguation of the English-language NER 23

24 TIB AV-Portal: Demonstration 24

25 TIB AV-Portal Next steps.. Improve search options Make results from videoanalysis editable Linked open data service > via SPARQL endpoint Enrich TIB AV-Portal by linked open data Deal with video as add-ons and with video abstracts Provide video metrics and statistics 25

26 Thank you for your attention!