Audio/Video Collection Migration, Storage and Preservation

Transcription

1 Audio/Video Collection Migration, Storage and Preservation PASIG Madrid, Spain July 5, 2010 Brian Campanotti CTO Front Porch Digital

2 Video Tape Preservation The Reality Video tape and film archives are in a constant state of decay and present different challenges than document archives Every year valuable content (monetary, cultural and historical) is being lost to deterioration Access to legacy material requires human intervention and is naturally plagued with difficulties in locating content Legacy video tape machines are becoming harder to find and maintain adding to the risk of access loss Resource and labour costs have become a barrier to large scale migration (<5% of content is being migrated annually)

3 Video Tape Preservation The Goal Define a straight forward workflow to alleviate bottlenecks and minimize the ingest operator skill set Allow machines to manage the migration process efficiently and in an unattended fashion in many parallel formats Rely heavily on software based tools to identify problematic content Automatically collect and mine for valuable metadata at each step in the process Automatically manage the long term storage of this file based content via automated policies including replication

4 Effective Content Migration Video/Film Masters Lossless MXF Wrapped JPEG2000 Preserva2on Package DIVArchive CSM Windows Media MXF Wrapped IMX50 MPEG2 MPEG4 AVC QT Wrapped DV25/50 Flash Video High Resolu2on Front Porch Digital Inc. All rights reserved. Solo Migra2on Engine Frame Based Content Metadata Proxy Resolu2on Data Tape

5 What is Content Storage Management Content Storage Management (CSM) is an object based, content aware intelligent storage management solution not to be confused with HSM (SAM-QFS, etc) CSM systems interface to any type of device which produces or consumes encoded content (images, audio, video, etc.) and bridges the gap to any type/amount of nearline, archive and offline storage CSM solutions provide content aware features such as transcoding, timecode based partial restore, file based quality assurance, usage analytics, storage metrics gathering and reporting, etc. CSM solutions allow connection to various user systems to fundamentally link dynamic metadata with the content repository

6 CSM Storage Pyramid CSM 6

7 Storage Format Choices Hard Disk Storage Flash Storage Optical Storage Data Tape Storage Pros Fast transfer speeds Random access Random access Fast transfer speeds Good storage density Persistent storage No contact read/write Storage density Random access media Low energy costs Acquisition format Rugged media Fast mechanical times Acquisition format Emerging holographic Cost per TB Multiple R/W streams Fast access Low replication costs Extremely portable Rugged Expansion costs Low replication costs Cons Tied to host chassis Limited read/write Low storage density Sequential data access Not portable Cost per TB Low transfer speeds Head and tape wear Cost per TB Proprietary formats Questionable shelf life Slower access Cost of ownership Low transfer speeds Portability Single R/W stream Highly mechanical Low storage density Single R/W stream Expansion costs High replication costs 7

8 Current Storage Technologies Type Open Or Prop Media Capacity (GB) Media 50Mbps (Hours) Drive Cost Street Price ($/TB) Drive Speed (MB/s) DVD RAM Optical Open $ $80 3 BLU RAY Optical Open 50 2 $$ $ HOLOGRAPHIC Optical Prop $$$ $ FLASH Flash Open $2k 10k 45 CLOUD Many - Infinite - - $3k/Year Slow 9940B Tape Prop $$$ $ T10000B Tape Prop $$$$$ $ SAIT Tape Prop $$$ $ SAIT2 Tape Prop $$$$ $ LTO4 Tape Open $$ $ LTO5 Tape Open $$$ $

9 Storage Performance at 100Mbps 9

10 Media Longevity Media shelf life is not a concern as migration to new storage media/technology requires no human intervention Migration is performed as a background task by the CSM solution Migrated content is a byte-for-byte duplicate of the original with NO degradation including full checksums! Additional software based checks and balances can be added to the migration process for additional protection Recent trends show a 50% increase in storage performance and 100% increase in storage density per generation

11 Content Protection and Disaster Recovery Most video tape and film archives in the world consist of one single copy of each asset! The Universal Studios fire in June 2008 destroyed nearly 50,000 valuable archive assets, some irreplaceable Once in the file-based domain, identical duplicate copies of all assets can be easily made by the CSM and moved offsite for the simple cost of the media itself Geographically distributed and federated CSM based digital repositories can provide true content protection and an extremely high SLA

12 Real World Success United Nations International Criminal Tribunal on Rwanda (Tanzania) Migration, preservation, storage and dissemination of 100,000 hours of evidence footage, court proceedings and witness testimonials Library of Congress Audio/Video Archives (Culpepper, VA) Migration and preservation of national heritage content International Olympic Committee (Lausanne, Switzerland) Storage and preservation of historical Olympic content European Parliament (Brussels, Belgium) Migration, storage, preservation and accessibility for all EP proceedings Shoah Foundation (University of Southern California) Migration and preservation of first person testimonies from Holocaust survivors and witnesses Major League Baseball (Long Island, NY) Preservation and accessibility to more than 100 years of baseball history

13 Audio/Video Collection Migration, Storage and Preservation Thank You!