UNCOVERING THE HIDDEN TREASURES: ENABLING VIDEO DISCOVERY AND PRESERVATION FOR FUTURE GENERATIONS

Transcription

1 UNCOVERING THE HIDDEN TREASURES: ENABLING VIDEO DISCOVERY AND PRESERVATION FOR FUTURE GENERATIONS Niels Dam, PhD VP Product Management, Digital Services

2 An early example of using video to document research.. 2

3 Why did we at ProQuest start looking at Video? 3

4 Starting point: Video is flourishing in particular for education purposes research use emerging AUTHOR CREATED / PRIMARY SOURCE E D U C A T I O N R E S E A R C H PRODUCED / PROMOTIONAL 4

5 Starting point: Long history of making text based content digital accessible & discoverable

6 Starting point: State of Video Discoverability Total items in discovery services Video items in discovery services (0.06%)

7 What did we learn when we started looking at video discovery? 7

8 Current Status of Video Metadata Which of these descriptions most closely fits the current status of the metadata and cataloging of your on-campus video material? 13% 19% 33% 45% Very rudimentary and incomplete Mixed - some good and some incomplete Good - fairly current and complete Very good - ongoing Source: ProQuest Survey 8

9 The Hypothesis digitization transcription metadata indexing editorial tools player streaming Create flexible full service offering utilizing automated transcription to create indexing data that would promote the discoverability and usability of institutionally created video. 9

10 Pilot Project Creating the Road Map 550 items broad spectrum of content wide timeframe of creation 10

11 Was the Hypothesis Valid? Ingestion and automated handling Conversion to Digital Automated Transcription Metadata Creation Output of formatted metadata and XML Hosting and streaming 11

12 Automated transcription: Results Accuracy of Automated Transcription 50-75% 0-20% 20-50% 75-85% 85-95% 12

13 Automated transcription & Metadata development Value of metadata based on transcription accuracy Below 50%, spotty metadata Between 50% - 75%, usable metadata Above 75%, very useful metadata 13

14 Automated transcription: Great Expectations when it was good, it was vary good 94% So executives so make decisions for all sorts of reasons shareholder value certainly in the top of the list. But their people there are humans and as a result they have the same rains that everyone else has more or less which means that we are driven by emotional influences. And as a result sometimes we might they might do things that. Maybe not necessarily the right thing to do is certainly not consistent with shareholder value there's lots of examples of that. Both in corporate America and financial services and actually in everyday life as well. 14

15 The Findings Subjects: Classification Codes: Companies: People: Keywords: Bank holding companies Financial services Bank acquisitions & mergers Economic crisis 8110 Commercial banking services 8100 Financial services industry 2330 Acquisitions & mergers 1110 Economic conditions & forecasts Merril Lynch Bank of America Lehman Brothers Bear Sterns Fleet Bank Dick Fuld John Thain Ken Lewis Hugh McConnell Wall Street, Toxic assets, Hedge fund 15

16 The Findings but when it was bad. Silent Films Singing Japanese 16

17 Automated transcription: Other problems Recording Quality West Chester University: LUVIMKing Speaker Clarity University of Minnesota: Packaging People for Automotive Safety,

18 Automated transcription: Currency bias University of Maryland: Puccini's Tosca (lyrics circa 1900) Dartmouth College: Margaret Mead (circa 1977) 18

19 19 Automated transcription: Caption display Caption is jarring Random sentence structure Odd punctuation and capitalization

20 Manual transcription: Results % + 20

21 21 Manual transcription: Metadata improvement People Companies Entities subjects

22 22 Manual transcription: Captioning Speaker identification Proper sentence structure and punctuation Appropriate capitalization

23 What is our approach to: Uncovering the hidden treasures: Enabling video discovery and preservation for future generations? 23

24 Entity extraction elaborate meta data Search and click Time stamped manual transcript synced with video Automatic recommendation of other relevant resources 24

25 What is needed to enable video discovery digitization transcription metadata editorial tools indexing player streaming Working with USC Digital Repository to provide Preservation offering 1/20/

26 Long term digital preservation demands expertise Video and audio digital preservation needs expertise: 40 PB infrastructure Geo redundancy Secure environment Expert automatic monitoring Expert restoration services The newer the format the faster the degradation. Film Video Harddrive ~ 50 years ~ 20 years ~ 5 years 26