DRS Update. HL Digital Preservation Services & Library Technology Services Created 2/2017, Updated 4/2017

Size: px
Start display at page:

Download "DRS Update. HL Digital Preservation Services & Library Technology Services Created 2/2017, Updated 4/2017"

Transcription

1 Update HL Digital Preservation Services & Library Technology Services Created 2/2017, Updated 4/2017 1

2 AGENDA DRS DRS DRS Architecture DRS DRS DRS Work 2

3 COLLABORATIVELY MANAGED DRS Business Owner Digital Preservation Services Key Responsibilities: Preservation & usage policies, strategies Manage & communicate about service Represent users & content s preservation needs Define high-level enhancement roadmaps Preservation plans Preservation outreach, consulting, guidelines DRS Technology Owner Library Technology Services Key Responsibilities: Technology & security policies, strategies Manage hardware, software, development Bug fixes & enhancements System monitoring & scaling Refine roadmaps based on resources System testing & documentation User support & training on systems 3

4 WHAT IS THE DRS? Harvard-maintained service for digital collections & content for: preservation keep the content safe keep the information usable long-term on modern platforms delivery to users A service - not storage or a tool Includes preservation & IT staff actively monitoring the content and systems Includes documented policies, practices & preservation plans Uses technology & systems but these change over time 4

5 KEY POLICIES What can be deposited into the DRS Who can deposit to the DRS Obligations of collection managers Responsibilities of DRS staff Retention policies Discovery & access policies Delivery services Preservation services Review began in March 2017 DRS Policy Guide: 5

6 DRS STORAGE FEES $6.00 Fee per GB per Year $5.00 $4.00 $3.00 $2.00 $1.00 $

7 KEY STRATEGIES Format guidance (preferred & accepted for deposit) Deposit tools with automatic technical characterization validated against documented content models Constant bit integrity checking of multiple distributed copies Regular storage refreshes Format migrations Expert networks 7

8 HIGHLIGHTS In production since 2000 Integrated with Library infrastructure & services from over 50 Harvard units Supports curatorial management Rich set of metadata Accepts any format but provides incentives to deposit in recommended preservable formats Supports HRCI content Proven track record Experience with several metadata migrations, maintenance of persistent names, several storage refreshes, planning for format migrations this year Never lost a file! Spawned well-used open-source tools (JHOVE, FITS) 8

9 ARCHITECTURAL LAYERS Discovery (outside the DRS but links maintained between the DRS and discovery systems) Delivery Applications and APIs for users & external systems Utility services (authentication, name resolution, etc.) Ingest Deposit tools and services Automated ingest of batches Management Infrastructure for metadata and content management and reporting Storage Replicated, geographically-distributed content and metadata Automated tools for monitoring content integrity 9

10 WHAT S IN THE DRS? Quantity: 64.7 million files, 212 TB per copy as of 4/2017 Image 58% Many formats Images, audio, text, digitized books, web sites, documents, biomedical image stacks, and video But primarily digitized images and text Text 41% Format distribution 4/2017 Container 1% Other 0% 10

11 WHO OWNS THE CONTENT? DRS by Owner TB Year Art Museum Business Design Divinity Education FAS Government HU Archives Law Medical Other Radcliffe Does not include content from the Harvard-Google project 11

12 GROWING USAGE 12

13 IMLS DRS Audit (1 res. until 6/16) Planning for Next-Gen DRS Storage Arcadia Easier DRS Deposits Project (1 term position through11/16) Arcadia Long-Term Preservation Project (2 term positions through 11/16) LARGE PARALLEL PROJECTS Migration to Next-Gen DRS Storage DP/DRS and Media Pres. Roadmap DRS2 Metadata Migration (1 term position through 6/17) Planning for Format Migrations Audio Format Migration Ongoing Enhancements, Upgrades and Project Support (2+ developers + systems and user support + management) 6/ /2016 4/2017 We are here 6/

14 IMLS DRS Audit Prepared for eventual certification as a trustworthy repository; indicated areas to improve that have since been addressed Planning for Next-Gen DRS Storage Migration to Next-Gen DRS Storage OUTCOMES Refreshed storage with greater geographic distribution; less preservation risk Arcadia Easier DRS Deposits Project Arcadia Long-Term Preservation Project Requirements ready for a BatchBuilder replacement; DRS discussion group and DRS community platform established Video fully supported for preservation and basic delivery; requirements ready for the other most requested formats DP/DRS and Media Pres. Roadmap DRS2 Metadata Migration All DRS content described consistently to support preservation planning and achieves Level 3B PREMIS conformance (highest level) Planning for Format Migrations Ongoing Enhancements, Upgrades and Project Support Audio Format Migration All DRS audio accessible to users on modern platforms; have framework for format migrations 6/ /2016 4/2017 We are here 6/

15 IMLS DRS Audit (1 res. until 6/16) Planning for Next-Gen DRS Storage UPDATE ON DRS2 METADATA MIGRATION Migration to Next-Gen DRS Storage Arcadia Easier DRS Deposits Project (1 term position through11/16) Arcadia Long-Term Preservation Project (2 term positions through 11/16) DRS2 Metadata Migration (1 term position through 6/17) Planning for Format Migrations DP/DRS and Media Pres. Roadmap Audio Format Migration Ongoing Enhancements, Upgrades and Project Support (2+ developers + systems and user support + management) 6/ /2016 4/2017 We are here 6/

16 DRS2 METADATA MIGRATION PROJECT Last piece of the DRS2 Project (move to the nextgeneration DRS) Previously completed: Transition all infrastructure to standards-based object model and metadata schemas New modern management tools and API-based services layer Support for more formats Metadata migration Re-describe all the content at the object and file-level Result: more accurate and detailed metadata to support curatorial management and preservation planning 16

17 MIGRATION SEQUENCED BY CONTENT TYPE Status: 2/ Color Profiles - DONE 2. Text Methodologies - DONE 3. PDF Documents - DONE 4. Target Images - DONE 5. Still Images & Page-turned Documents - ALMOST DONE 6. Google books - SPECS READY 7. Audio - SPECS READY 8. Opaque Containers - SPECS READY 9. Web Harvests - SPECS READY 10. Biomedical Images - SPECS READY 17

18 MIGRATION SEQUENCED BY CONTENT TYPE Status: 4/ Color Profiles - DONE 2. Text Methodologies - DONE 3. PDF Documents - DONE 4. Target Images - DONE 5. Still Images & Page-turned Documents - ALMOST DONE 6. Google books - DONE 7. Audio - IN PROGRESS 8. Opaque Containers - SPECS READY 9. Web Harvests - SPECS READY 10. Biomedical Images - SPECS READY 18

19 OVERALL MIGRATION STATUS (2/2017) Files migrated 82% Left to migrate 18% 16% = Still image & Page-turned objects 2% = Google books 0.1% = Audio 0.1% = Web harvests 0.04% = Opaque containers 0.01% = Biomedical images 19

20 OVERALL MIGRATION STATUS (4/2017) Files migrated 99% Left to migrate 1% 0.6% = Still image & Page-turned objects 0.1% = Audio 0.1% = Web harvests 0.04% = Opaque containers 0.01% = Biomedical images 20

21 STATUS OF METADATA MIGRATION FOR STILL IMAGES & PAGE-TURNED DOCUMENTS (4/2017) Live chart at: 21

22 OVERALL STATUS OF METADATA MIGRATION (2/2017) Live chart at: 22

23 OVERALL STATUS OF METADATA MIGRATION (4/2017) Live chart at: 23

24 DATA CLEANUP Fixed metadata errors for thousands of objects Report is in progress including: Overall findings Recommendations for preventing future metadata errors Individual sections for each of the 55 DRS content-owning units To share this experience a paper will be written to present at a conference 24

25 IMLS DRS Audit (1 res. until 6/16) Planning for Next-Gen DRS Storage UPDATE ON AUDIO FORMAT MIGRATION Migration to Next-Gen DRS Storage Arcadia Easier DRS Deposits Project (1 term position through11/16) Arcadia Long-Term Preservation Project (2 term positions through 11/16) DRS2 Metadata Migration (1 term position through 6/17) Planning for Format Migrations DP/DRS and Media Pres. Roadmap Audio Format Migration Ongoing Enhancements, Upgrades and Project Support (2+ developers + systems and user support + management) 6/ /2016 4/2017 We are here 6/

26 AUDIO FORMAT MIGRATION STEPS Status: 4/ Migrate audio metadata (Spring 2017) 1. SMIL playlists - DONE 2. Audio objects - IN TESTING 2. Prepare for format migration (Spring-Summer 2017) 1. Format migration spec - IN PROCESS 2. Format migration tools - IN PROCESS 3. Update DRS ingest tools - NOT STARTED 4. Test migration - NOT STARTED 3. Run format migration (late Summer 2017) 1. SMIL playlists - NOT STARTED 2. RealAudio- NOT STARTED 26

27 IMLS DRS Audit (1 res. until 6/16) Planning for Next-Gen DRS Storage UPDATE ON LONG-TERM PRESERVATION PROJECT Migration to Next-Gen DRS Storage Arcadia Easier DRS Deposits Project (1 term position through11/16) Arcadia Long-Term Preservation Project (2 term positions through 11/16) DRS2 Metadata Migration (1 term position through 6/17) Planning for Format Migrations DP/DRS and Media Pres. Roadmap Audio Format Migration Ongoing Enhancements, Upgrades and Project Support (2+ developers + systems and user support + management) 6/ /2016 4/2017 We are here 6/

28 LONG-TERM PRESERVATION PROJECT 3-year project enabled by Arcadia (ended Nov. 30, 2016) Goal to add support to the DRS for formats mostrequested by curators: video word processing CAD (2D and 3D) disk images RAW camera images (image sequences for scanned film) New fast-tracking process working with consultants to help with the analysis Work carries on through the digital preservation / DRS and media roadmaps 28

29 FORMAT SUPPORT BY PROJECT END (11/30/2016) Datasets 6% Other Still Images 6% Other OCR Text 1% Presentations 1% Databases 5% Software 5% Vector Graphics 6% Spreadsheets 8% Newspapers 1% DNG 6% GIS 1% Articles 1% Disk Images 8% Video 19% Word Processing 14% CAD 11% Chart shows format distribution of 60 requests made from by curators to add support for new formats in the DRS Analysis Complete, Development Complete Analysis Complete, Development Partially Complete Analysis Complete, Development to Start When Resources are Available Work to be Scheduled in the 29

30 VIDEO DELIVERY 30

31 COMPLIANCE WITH THE NDSA LEVELS OF PRESERVATION* 1. Before the DRS2 project: * Storage & Geographic Location File Fixity and Data Integrity Information Security Metadata Level One (Lowest) Level Two Level Three Level Four (Highest) File Formats = DRS compliance 31

32 COMPLIANCE WITH THE NDSA LEVELS OF PRESERVATION 2. After the DRS2 project: Storage & Geographic Location File Fixity and Data Integrity Information Security Metadata Level One (Lowest) Level Two Level Three Level Four (Highest) File Formats = DRS compliance 32

33 COMPLIANCE WITH THE NDSA LEVELS OF PRESERVATION 3. After the latest DRS storage upgrade and the audio format migration: Storage & Geographic Location File Fixity and Data Integrity Information Security Metadata Level One (Lowest) Level Two Level Three Level Four (Highest) File Formats = DRS compliance 33

34 FUTURE WORK Depends on Library priorities Currently on horizon: Complete projects & enhancements in-flight (audio migration, CJK support, scaling, easier deposits, ) Full support through delivery for disk images, RAW camera images, , CAD files, image sequences Delivery service improvements Expose more DRS documentation, roadmap DRS certification Additional DRS deposit streams from Harvard & external infrastructure (e.g. Dataverse, Archive-It, Kaltura) Evaluate options and start planning for next-gen DRS 34

35 RESOURCES DRS has become foundational HL infrastructure Average IT maintenance requires 4-5 FTE Major enhancements require additional staff Relative priority Description 1 Audio format migration & delivery improvements FY18 2 Easier deposits FY18- FY19 3 Support for disk image content model FY19 4 Make DRS content easier to embed in Canvas, etc. FY19 5 Video delivery enhancements FY20 6 Depositor efficiencies FY20 Timing given current level of staffing Slide adapted from Library IT Planning for FY18, Feb. 10, With existing staff we estimate this could be accomplished in FY18 With 1 additional developer we estimate this could be accomplished in FY18

36 DISCUSSION 36

Building on to the Digital Preservation Foundation at Harvard Library. Andrea Goethals ABCD-Library Meeting June 27, 2016

Building on to the Digital Preservation Foundation at Harvard Library. Andrea Goethals ABCD-Library Meeting June 27, 2016 Building on to the Digital Preservation Foundation at Harvard Library Andrea Goethals ABCD-Library Meeting June 27, 2016 What do we already have? What do we still need? Where I ll focus DIGITAL PRESERVATION

More information

Protecting Future Access Now Models for Preserving Locally Created Content

Protecting Future Access Now Models for Preserving Locally Created Content Protecting Future Access Now Models for Preserving Locally Created Content By Amy Kirchhoff Archive Service Product Manager, Portico, ITHAKA Amigos Online Conference Digital Preservation: What s Now, What

More information

Preserving Digital Content at Scale

Preserving Digital Content at Scale Preserving Digital Content at Scale CNI Albuquerque, NM April 4, 2017 Carolyn Caizzi Head of Repository and Digital Curation Northwestern University Libraries State of the Northwestern Libraries Local

More information

Agenda. Bibliography

Agenda. Bibliography Humor 2 1 Agenda 3 Trusted Digital Repositories (TDR) definition Open Archival Information System (OAIS) its relevance to TDRs Requirements for a TDR Trustworthy Repositories Audit & Certification: Criteria

More information

DRS Policy Guide. Management of DRS operations is the responsibility of staff in Library Technology Services (LTS).

DRS Policy Guide. Management of DRS operations is the responsibility of staff in Library Technology Services (LTS). Harvard University Library Office for Information Systems DRS Policy Guide This Guide defines the policies associated with the Harvard Library Digital Repository Service (DRS) and is intended for Harvard

More information

Digital Preservation at NARA

Digital Preservation at NARA Digital Preservation at NARA Policy, Records, Technology Leslie Johnston Director of Digital Preservation US National Archives and Records Administration (NARA) ARMA, April 18, 2018 Policy Managing Government

More information

DRS 2 Glossary. access flag An object access flag records the least restrictive access flag recorded for one of the object s files: ο ο

DRS 2 Glossary. access flag An object access flag records the least restrictive access flag recorded for one of the object s files: ο ο Harvard University Information Technology Library Technology Services DRS 2 Glossary access flag An object access flag records the least restrictive access flag recorded for one of the object s files:

More information

GEOSS Data Management Principles: Importance and Implementation

GEOSS Data Management Principles: Importance and Implementation GEOSS Data Management Principles: Importance and Implementation Alex de Sherbinin / Associate Director / CIESIN, Columbia University Gregory Giuliani / Lecturer / University of Geneva Joan Maso / Researcher

More information

PRESERVING DIGITAL OBJECTS

PRESERVING DIGITAL OBJECTS MODULE 12 PRESERVING DIGITAL OBJECTS Erin O Meara and Kate Stratton Preserving Digital Objects 51 Case Study 2: University of North Carolina Chapel Hill By Jill Sexton, former Head of Digital Research

More information

Digital The Harold B. Lee Library

Digital The Harold B. Lee Library Digital Preservation @ The Harold B. Lee Library CIMA 23 May 2013 How we got here? 1. Understanding Digital Preservation 2. Search for Content 3. Maintain Optical Disc Storage 4. In House Preservation

More information

An overview of the OAIS and Representation Information

An overview of the OAIS and Representation Information An overview of the OAIS and Representation Information JORUM, DCC and JISC Forum Long-term Curation and Preservation of Learning Objects February 9 th 2006 University of Glasgow Manjula Patel UKOLN and

More information

Mercè Crosas, Ph.D. Chief Data Science and Technology Officer Institute for Quantitative Social Science (IQSS) Harvard

Mercè Crosas, Ph.D. Chief Data Science and Technology Officer Institute for Quantitative Social Science (IQSS) Harvard Mercè Crosas, Ph.D. Chief Data Science and Technology Officer Institute for Quantitative Social Science (IQSS) Harvard University @mercecrosas mercecrosas.com Open Research Cloud, May 11, 2017 Best Practices

More information

Applying Archival Science to Digital Curation: Advocacy for the Archivist s Role in Implementing and Managing Trusted Digital Repositories

Applying Archival Science to Digital Curation: Advocacy for the Archivist s Role in Implementing and Managing Trusted Digital Repositories Purdue University Purdue e-pubs Libraries Faculty and Staff Presentations Purdue Libraries 2015 Applying Archival Science to Digital Curation: Advocacy for the Archivist s Role in Implementing and Managing

More information

Data Curation Handbook Steps

Data Curation Handbook Steps Data Curation Handbook Steps By Lisa R. Johnston Preliminary Step 0: Establish Your Data Curation Service: Repository data curation services should be sustained through appropriate staffing and business

More information

University of British Columbia Library. Persistent Digital Collections Implementation Plan. Final project report Summary version

University of British Columbia Library. Persistent Digital Collections Implementation Plan. Final project report Summary version University of British Columbia Library Persistent Digital Collections Implementation Plan Final project report Summary version May 16, 2012 Prepared by 1. Introduction In 2011 Artefactual Systems Inc.

More information

BUILDING A NEW DIGITAL LIBRARY FOR THE NATIONAL LIBRARY OF AUSTRALIA

BUILDING A NEW DIGITAL LIBRARY FOR THE NATIONAL LIBRARY OF AUSTRALIA BUILDING A NEW DIGITAL LIBRARY FOR THE NATIONAL LIBRARY OF AUSTRALIA Strategic Directions 2012-2014 However, the growth of our digital collections is outpacing our capacity to manage, preserve and deliver

More information

Working with Islandora

Working with Islandora Working with Islandora Erin Tripp, discoverygarden erin@discoverygarden.ca @eeohalloran April 21, 2015 Jasna, Slovakia Presentation Agenda Introductions Islandora Software Islandora Community Islandora

More information

Digital Preservation DMFUG 2017

Digital Preservation DMFUG 2017 Digital Preservation DMFUG 2017 1 The need, the goal, a tutorial In 2000, the University of California, Berkeley estimated that 93% of the world's yearly intellectual output is produced in digital form

More information

DIGITAL STEWARDSHIP SUPPLEMENTARY INFORMATION FORM

DIGITAL STEWARDSHIP SUPPLEMENTARY INFORMATION FORM OMB No. 3137 0071, Exp. Date: 09/30/2015 DIGITAL STEWARDSHIP SUPPLEMENTARY INFORMATION FORM Introduction: IMLS is committed to expanding public access to IMLS-funded research, data and other digital products:

More information

Data management Backgrounds and steps to implementation; A pragmatic approach.

Data management Backgrounds and steps to implementation; A pragmatic approach. Data management Backgrounds and steps to implementation; A pragmatic approach. Research and data management through the years Find the differences 2 Research and data management through the years Find

More information

Working with a Preservation Software Vendor - The Kentucky Experience Glen McAninch

Working with a Preservation Software Vendor - The Kentucky Experience Glen McAninch Working with a Preservation Software Vendor - The Kentucky Experience Glen McAninch Kentucky Department for Libraries and Archives November 2014 Best Practices Exchange Montgomery, Alabama Who We Are Kentucky

More information

DRI: Preservation Planning Case Study Getting Started in Digital Preservation Digital Preservation Coalition November 2013 Dublin, Ireland

DRI: Preservation Planning Case Study Getting Started in Digital Preservation Digital Preservation Coalition November 2013 Dublin, Ireland DRI: Preservation Planning Case Study Getting Started in Digital Preservation Digital Preservation Coalition November 2013 Dublin, Ireland Dr Aileen O Carroll Policy Manager Digital Repository of Ireland

More information

Lessons Learned. Implementing Rosetta in the Harold B. Lee Library

Lessons Learned. Implementing Rosetta in the Harold B. Lee Library Lessons Learned Implementing Rosetta in the Harold B. Lee Library Provide Long Term Digital Access 1. To preserve BYU digital items: Digitized images, audio, video, Electronic articles, university records,

More information

The Data Curation Profiles Toolkit: Interview Worksheet

The Data Curation Profiles Toolkit: Interview Worksheet Purdue University Purdue e-pubs Data Curation Profiles Toolkit 11-29-2010 The Data Curation Profiles Toolkit: Interview Worksheet Jake Carlson Purdue University, jakecar@umich.edu Follow this and additional

More information

Writing a Data Management Plan A guide for the perplexed

Writing a Data Management Plan A guide for the perplexed March 29, 2012 Writing a Data Management Plan A guide for the perplexed Agenda Rationale and Motivations for Data Management Plans Data and data structures Metadata and provenance Provisions for privacy,

More information

Conducting a Self-Assessment of a Long-Term Archive for Interdisciplinary Scientific Data as a Trustworthy Digital Repository

Conducting a Self-Assessment of a Long-Term Archive for Interdisciplinary Scientific Data as a Trustworthy Digital Repository Conducting a Self-Assessment of a Long-Term Archive for Interdisciplinary Scientific Data as a Trustworthy Digital Repository Robert R. Downs and Robert S. Chen Center for International Earth Science Information

More information

ebooks Preservation at Scholars Portal Kate Davis & Grant Hurley Scholars Portal, Ontario Council of University Libraries

ebooks Preservation at Scholars Portal Kate Davis & Grant Hurley Scholars Portal, Ontario Council of University Libraries ebooks Preservation at Scholars Portal Kate Davis & Grant Hurley Scholars Portal, Ontario Council of University Libraries The Charlotte Initiative Open Conference March 10, 2017 Outline OCUL and Scholars

More information

NDSA Web Archiving Survey

NDSA Web Archiving Survey NDSA Web Archiving Survey Introduction In 2011 and 2013, the National Digital Stewardship Alliance (NDSA) conducted surveys of U.S. organizations currently or prospectively engaged in web archiving to

More information

archiving with the IBM CommonStore solution

archiving with the IBM CommonStore solution IBM Software Group E-mail archiving with the IBM CommonStore solution Comprehensive flexible reliable Borut Obran Genis d.o.o. 2006 IBM Corporation Agenda Overview Mailbox management Discovery Compliance

More information

Copyright 2008, Paul Conway.

Copyright 2008, Paul Conway. Unless otherwise noted, the content of this course material is licensed under a Creative Commons Attribution - Non-Commercial - Share Alike 3.0 License.. http://creativecommons.org/licenses/by-nc-sa/3.0/

More information

Woodson Research Center Digital Preservation Policy

Woodson Research Center Digital Preservation Policy Primary Policy Aims Risk Assessment Needs Statement Project/Purpose Statement Goals and Objectives Projects to Undertake in 2016 Organizational Commitments Financial Commitments Personnel Preservation

More information

ORACLE SERVICES FOR APPLICATION MIGRATIONS TO ORACLE HARDWARE INFRASTRUCTURES

ORACLE SERVICES FOR APPLICATION MIGRATIONS TO ORACLE HARDWARE INFRASTRUCTURES ORACLE SERVICES FOR APPLICATION MIGRATIONS TO ORACLE HARDWARE INFRASTRUCTURES SERVICE, SUPPORT AND EXPERT GUIDANCE FOR THE MIGRATION AND IMPLEMENTATION OF YOUR ORACLE APPLICATIONS ON ORACLE INFRASTRUCTURE

More information

ISO Self-Assessment at the British Library. Caylin Smith Repository

ISO Self-Assessment at the British Library. Caylin Smith Repository ISO 16363 Self-Assessment at the British Library Caylin Smith Repository Manager caylin.smith@bl.uk @caylinssmith Outline Digital Preservation at the British Library The Library s Digital Collections Achieving

More information

Managing Image Metadata

Managing Image Metadata Managing Image Metadata Building Workflows for Effective Discovery Morag Boyd, The Ohio State University Libraries Worth a Thousand Words: A Global Perspective on Image Description, Discovery, and Access

More information

JISC WORK PACKAGE: (Project Plan Appendix B, Version 2 )

JISC WORK PACKAGE: (Project Plan Appendix B, Version 2 ) Date: 22/10/2008 JISC WORK PACKAGE: (Project Plan Appendix B, Version 2 ) WORKPACKAGES Month 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 1: Project Management 2: Institutional Repository

More information

DCH-RP Trust-Building Report

DCH-RP Trust-Building Report DCH-RP Trust-Building Report Raivo Ruusalepp Estonian Ministry of Culture DCH-RP and EUDAT workshop Stockholm, June 3 rd, 2014 Topics Trust in a digital repository Trust in a distributed digital repository

More information

Digital Preservation and The Digital Repository Infrastructure

Digital Preservation and The Digital Repository Infrastructure Marymount University 5/12/2016 Digital Preservation and The Digital Repository Infrastructure Adam Retter adam@evolvedbinary.com @adamretter Adam Retter Consultant Scala / Java Concurrency and Databases

More information

Digital Preservation Standards Using ISO for assessment

Digital Preservation Standards Using ISO for assessment Digital Preservation Standards Using ISO 16363 for assessment Preservation Administrators Interest Group, American Library Association, June 25, 2016 Amy Rudersdorf Senior Consultant, AVPreserve amy@avpreserve.com

More information

Richard Marciano Alexandra Chassanoff David Pcolar Bing Zhu Chien-Yi Hu. March 24, 2010

Richard Marciano Alexandra Chassanoff David Pcolar Bing Zhu Chien-Yi Hu. March 24, 2010 Richard Marciano Alexandra Chassanoff David Pcolar Bing Zhu Chien-Yi Hu March 24, 2010 What is the feasibility of repository interoperability at the policy level? Can a preservation environment be assembled

More information

Emory Libraries Digital Collections Steering Committee Policy Suite

Emory Libraries Digital Collections Steering Committee Policy Suite Emory Libraries Digital Collections Steering Committee Policy Suite Last Revised: March, 2018 Digital Collections Development Policy 2 Digital Preservation Policy 5 Digital Object Retention Policy 8 Third-Party

More information

Ensuring Proper Storage for Earth Science Data: The USGS Process to Certify Trusted Digital Repositories

Ensuring Proper Storage for Earth Science Data: The USGS Process to Certify Trusted Digital Repositories Ensuring Proper Storage for Earth Science Data: The USGS Process to Certify Trusted Digital Repositories U.S. Department of the Interior U.S. Geological Survey John Faundeen, Clara Brown, Keith Kirk PECORA

More information

GETTING STARTED WITH DIGITAL COMMONWEALTH

GETTING STARTED WITH DIGITAL COMMONWEALTH GETTING STARTED WITH DIGITAL COMMONWEALTH Digital Commonwealth (www.digitalcommonwealth.org) is a Web portal and fee-based repository service for online cultural heritage materials held by Massachusetts

More information

Data Curation Profile Plant Genetics / Corn Breeding

Data Curation Profile Plant Genetics / Corn Breeding Profile Author Author s Institution Contact Researcher(s) Interviewed Researcher s Institution Katherine Chiang Cornell University Library ksc3@cornell.edu Withheld Cornell University Date of Creation

More information

The OAIS Reference Model: current implementations

The OAIS Reference Model: current implementations The OAIS Reference Model: current implementations Michael Day, UKOLN, University of Bath m.day@ukoln.ac.uk Chinese-European Workshop on Digital Preservation, Beijing, China, 14-16 July 2004 Presentation

More information

Digital Preservation: How to Plan

Digital Preservation: How to Plan Digital Preservation: How to Plan Preservation Planning with Plato Christoph Becker Vienna University of Technology http://www.ifs.tuwien.ac.at/~becker Sofia, September 2009 Outline Why preservation planning?

More information

RADAR. Establishing a generic Research Data Repository: RESEARCH DATA REPOSITORY. Dr. Angelina Kraft

RADAR. Establishing a generic Research Data Repository: RESEARCH DATA REPOSITORY. Dr. Angelina Kraft RESEARCH DATA REPOSITORY http://www.radar-projekt.org http://www.radar-service.eu Establishing a generic Research Data Repository: RADAR Digital Infrastructures for Research 2016 Conference 28 th - 30

More information

PASIG Directions & Issues

PASIG Directions & Issues PASIG Directions & Issues Richard Boulderstone Director estrategy & Programmes April 2011 PASIG June 2007: Concluding comments The British Library has a substantial long term vision for electronic resources

More information

7.3. In t r o d u c t i o n to m e t a d a t a

7.3. In t r o d u c t i o n to m e t a d a t a 7. Standards for Data Documentation 7.1. Introducing standards for data documentation Data documentation has several functions. It records where images, text, and other forms of digital data are from,

More information

Developing a Framework for File Format Migrations. ipres 2015 Chapel Hill, NC 3 November 2015 Joey Heinen and Andrea Goethals

Developing a Framework for File Format Migrations. ipres 2015 Chapel Hill, NC 3 November 2015 Joey Heinen and Andrea Goethals Developing a Framework for File Format Migrations ipres 2015 Chapel Hill, NC 3 November 2015 Joey Heinen and Andrea Goethals Outline 1. Project Drivers 2. Migration Framework 3. Key Lessons Learned 4.

More information

CONTENTdm & The Digital Collection Gateway New Looks for Discovery and Delivery

CONTENTdm & The Digital Collection Gateway New Looks for Discovery and Delivery CONTENTdm & The Digital Collection Gateway New Looks for Discovery and Delivery EVERY CONNECTION has a starting point. OCLC EMEA Regional Council Meeting Deutsche Nationalbibliothek Frankfurt 2 nd March

More information

MarkLogic. A Modern Data Platform To Support Your Critical Path COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.

MarkLogic. A Modern Data Platform To Support Your Critical Path COPYRIGHT 2016 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. MarkLogic A Modern Data Platform To Support Your Critical Path SLIDE: 2 Inception Pre- Post- Distribution Archive Taxonomies Semantics Technical Descriptive Customers Usage SLIDE: 4 Inception Pre- Post-

More information

Archives in a Networked Information Society: The Problem of Sustainability in the Digital Information Environment

Archives in a Networked Information Society: The Problem of Sustainability in the Digital Information Environment Archives in a Networked Information Society: The Problem of Sustainability in the Digital Information Environment Shigeo Sugimoto Research Center for Knowledge Communities Graduate School of Library, Information

More information

DRI: Dr Aileen O Carroll Policy Manager Digital Repository of Ireland Royal Irish Academy

DRI: Dr Aileen O Carroll Policy Manager Digital Repository of Ireland Royal Irish Academy DRI: Dr Aileen O Carroll Policy Manager Digital Repository of Ireland Royal Irish Academy Dr Kathryn Cassidy Software Engineer Digital Repository of Ireland Trinity College Dublin Development of a Preservation

More information

Mass Digitisation Enabling Access, Use and Reuse

Mass Digitisation Enabling Access, Use and Reuse Mass Digitisation Enabling Access, Use and Reuse National Digitisation Centre, Mikkeli, National Library of Finland Triangelipäivät 30.10.2008 Tiina Ison, Senior Analyst, Project Manager Organisation of

More information

Helping Journals to Upgrade Data Publications for Reusable Research

Helping Journals to Upgrade Data Publications for Reusable Research Helping Journals to Upgrade Data Publications for Reusable Research Sonia Barbosa (Project Manager) Eleni Castro (Project Coordinator) Ins9tute for Quan9ta9ve Social Science (IQSS) Harvard University @thedataorg

More information

Long-term digital preservation of UNSWorks

Long-term digital preservation of UNSWorks Long-term digital preservation of UNSWorks UNSW Library Arif Shaon, Maude Frances CAUL Community Days 2014 UNSW Australia The University of New South Wales at a Glance: https://www.unsw.edu.au/sites/default/files/documents/unsw4009_miniguide_2012_aw2_v2.pdf

More information

User Stories : Digital Archiving of UNHCR EDRMS Content. Prepared for UNHCR Open Preservation Foundation, May 2017 Version 0.5

User Stories : Digital Archiving of UNHCR EDRMS Content. Prepared for UNHCR Open Preservation Foundation, May 2017 Version 0.5 User Stories : Digital Archiving of UNHCR EDRMS Content Prepared for UNHCR Open Preservation Foundation, May 2017 Version 0.5 Introduction This document presents the user stories that describe key interactions

More information

Importance of cultural heritage:

Importance of cultural heritage: Cultural heritage: Consists of tangible and intangible, natural and cultural, movable and immovable assets inherited from the past. Extremely valuable for the present and the future of communities. Access,

More information

DEVELOPING, ENABLING, AND SUPPORTING DATA AND REPOSITORY CERTIFICATION

DEVELOPING, ENABLING, AND SUPPORTING DATA AND REPOSITORY CERTIFICATION DEVELOPING, ENABLING, AND SUPPORTING DATA AND REPOSITORY CERTIFICATION Plato Smith, Ph.D., Data Management Librarian DataONE Member Node Special Topics Discussion June 8, 2017, 2pm - 2:30 pm ASSESSING

More information

The e-depot in practice. Barbara Sierman Digital Preservation Officer Madrid,

The e-depot in practice. Barbara Sierman Digital Preservation Officer Madrid, Barbara Sierman Digital Preservation Officer Madrid, 16-03-2006 e-depot in practice Short introduction of the e-depot 4 Cases with different aspects Characteristics of the supplier Specialities, problems

More information

Electronic Records Archives: Philadelphia Federal Executive Board

Electronic Records Archives: Philadelphia Federal Executive Board Electronic Records Archives: Philadelphia Federal Executive Board L. Reynolds Cahoon Assistant Archivist for HR and IT and Chief Information Officer 18 March 2004 Agenda (The Mission) Electronic Records

More information

Strategy for long term preservation of material collected for the Netarchive by the Royal Library and the State and University Library 2014

Strategy for long term preservation of material collected for the Netarchive by the Royal Library and the State and University Library 2014 Strategy for long term preservation of material collected for the Netarchive by the Royal Library and the State and University Library 2014 Introduction This document presents a strategy for long term

More information

Utilizing Digital Library Infrastructure to Build Modern Research Collections

Utilizing Digital Library Infrastructure to Build Modern Research Collections Utilizing Digital Library Infrastructure to Build Modern Research Collections Mark Phillips Wisconsin Association of Academic Librarians What is a digital library? 2 After 15+ years surely we've figured

More information

MAPPING STANDARDS! FOR RICHER ASSESSMENTS. Bertram Lyons AVPreserve Digital Preservation 2014 Washington, DC

MAPPING STANDARDS! FOR RICHER ASSESSMENTS. Bertram Lyons AVPreserve Digital Preservation 2014 Washington, DC MAPPING STANDARDS! FOR RICHER ASSESSMENTS Bertram Lyons AVPreserve Digital Preservation 2014 Washington, DC NDSA Levels of Digital Preservation! Matrix (Version 1) ISO 16363:2012! Audit & Certification

More information

Update on Dataverse Dryad-Dataverse Community Meeting. Mercè Crosas, Elizabeth Quigley & Eleni Castro. Data Science > IQSS > Harvard University

Update on Dataverse Dryad-Dataverse Community Meeting. Mercè Crosas, Elizabeth Quigley & Eleni Castro. Data Science > IQSS > Harvard University Update on Dataverse Image credit: David Bygott (CC-BY-NC-SA) 2014 Dryad-Dataverse Community Meeting Mercè Crosas, Elizabeth Quigley & Eleni Castro Data Science > IQSS > Harvard University Introduction

More information

Next-Generation Melvyl Pilot supported by WorldCat Local: The Future of Searching

Next-Generation Melvyl Pilot supported by WorldCat Local: The Future of Searching Next-Generation Melvyl Pilot supported by WorldCat Local: The Future of Searching The UNIVERSITY of CALIFORNIA LIBRARIES Call To Action The University of California Bibliographic Services Task Force Report,

More information

Draft Digital Preservation Policy for IGNCA. Dr. Aditya Tripathi Banaras Hindu University Varanasi

Draft Digital Preservation Policy for IGNCA. Dr. Aditya Tripathi Banaras Hindu University Varanasi Draft Digital Preservation Policy for IGNCA Dr. Aditya Tripathi Banaras Hindu University Varanasi aditya@bhu.ac.in adityatripathi@hotmail.com Digital Preservation Born Digital Object Regardless of U S

More information

Certification. F. Genova (thanks to I. Dillo and Hervé L Hours)

Certification. F. Genova (thanks to I. Dillo and Hervé L Hours) Certification F. Genova (thanks to I. Dillo and Hervé L Hours) Perhaps the biggest challenge in sharing data is trust: how do you create a system robust enough for scientists to trust that, if they share,

More information

What do you do when your file formats become obsolete? Lydia T. Motyka Florida Center for Library Automation USETDA 2011

What do you do when your file formats become obsolete? Lydia T. Motyka Florida Center for Library Automation USETDA 2011 What do you do when your file formats become obsolete? Lydia T. Motyka Florida Center for Library Automation USETDA 2011 The FCLA, the FDA, and DAITSS FDA: a service of the Florida Center for Library Automation

More information

Assessing the FAIRness of Datasets in Trustworthy Digital Repositories: a 5 star scale

Assessing the FAIRness of Datasets in Trustworthy Digital Repositories: a 5 star scale Assessing the FAIRness of Datasets in Trustworthy Digital Repositories: a 5 star scale Peter Doorn, Director DANS Ingrid Dillo, Deputy Director DANS 2nd DPHEP Collaboration Workshop CERN, Geneva, 13 March

More information

EarthCube and Cyberinfrastructure for the Earth Sciences: Lessons and Perspective from OpenTopography

EarthCube and Cyberinfrastructure for the Earth Sciences: Lessons and Perspective from OpenTopography EarthCube and Cyberinfrastructure for the Earth Sciences: Lessons and Perspective from OpenTopography Christopher Crosby, San Diego Supercomputer Center J Ramon Arrowsmith, Arizona State University Chaitan

More information

Data Curation Profile Human Genomics

Data Curation Profile Human Genomics Data Curation Profile Human Genomics Profile Author Profile Author Institution Name Contact J. Carlson N. Brown Purdue University J. Carlson, jrcarlso@purdue.edu Date of Creation October 27, 2009 Date

More information

What steps to take. when AV is yet to become a priority for your organisation

What steps to take. when AV is yet to become a priority for your organisation What steps to take when AV is yet to become a priority for your organisation Somaya Langley Digital Preservation Specialist Policy and Planning (Polonsky Fellow), Cambridge University Library Digital Preservation

More information

Data Archiving and Networked Services. Valentijn Gilissen, MA

Data Archiving and Networked Services. Valentijn Gilissen, MA Data Archiving and Networked Services Dutch archaeological data depositing, processing, archiving and accessing at DANS a repository with ten years of history, setting its sails to the future Valentijn

More information

Building for the Future

Building for the Future Building for the Future The National Digital Newspaper Program Deborah Thomas US Library of Congress DigCCurr 2007 Chapel Hill, NC April 19, 2007 1 What is NDNP? Provide access to historic newspapers Select

More information

Improving a Trustworthy Data Repository with ISO 16363

Improving a Trustworthy Data Repository with ISO 16363 Improving a Trustworthy Data Repository with ISO 16363 Robert R. Downs 1 1 rdowns@ciesin.columbia.edu NASA Socioeconomic Data and Applications Center (SEDAC) Center for International Earth Science Information

More information

What is Islandora? Islandora is an open source digital repository that preserves, manages, and showcases your institution s unique material.

What is Islandora? Islandora is an open source digital repository that preserves, manages, and showcases your institution s unique material. What is Islandora? Islandora is an open source digital repository that preserves, manages, and showcases your institution s unique material. 1 P age What is Islandora? Islandora is an open source digital

More information

Deposit-Only Service Needs Report (last edited 9/8/2016, Tricia Patterson)

Deposit-Only Service Needs Report (last edited 9/8/2016, Tricia Patterson) Deposit-Only Service Needs Report (last edited 9/8/2016, Tricia Patterson) Introduction In November of 2015, the Easier DRS Depositing project was initiated to simplify and streamline the process of Harvard

More information

Preservation and Access of Digital Audiovisual Assets at the Guggenheim

Preservation and Access of Digital Audiovisual Assets at the Guggenheim Preservation and Access of Digital Audiovisual Assets at the Guggenheim Summary The Solomon R. Guggenheim Museum holds a variety of highly valuable born-digital and digitized audiovisual assets, including

More information

UC Irvine LAUC-I and Library Staff Research

UC Irvine LAUC-I and Library Staff Research UC Irvine LAUC-I and Library Staff Research Title Research Data Management: Local UCI Outreach to Faculty Permalink https://escholarship.org/uc/item/18f3v1j7 Author Tsang, Daniel C Publication Date 2013-02-25

More information

NSF Data Management Plan Template Duke University Libraries Data and GIS Services

NSF Data Management Plan Template Duke University Libraries Data and GIS Services NSF Data Management Plan Template Duke University Libraries Data and GIS Services NSF Data Management Plan Requirement Overview The Data Management Plan (DMP) should be a supplementary document of no more

More information

UNT Libraries TRAC Audit Checklist

UNT Libraries TRAC Audit Checklist UNT Libraries TRAC Audit Checklist Date: October 2015 Version: 1.0 Contributors: Mark Phillips Assistant Dean for Digital Libraries Daniel Alemneh Supervisor, Digital Curation Unit Ana Krahmer Supervisor,

More information

Introduction to Digital Preservation. Danielle Mericle University of Oregon

Introduction to Digital Preservation. Danielle Mericle University of Oregon Introduction to Digital Preservation Danielle Mericle dmericle@uoregon.edu University of Oregon What is Digital Preservation? the series of management policies and activities necessary to ensure the enduring

More information

Research Data Edinburgh: MANTRA & Edinburgh DataShare. Stuart Macdonald EDINA & Data Library University of Edinburgh

Research Data Edinburgh: MANTRA & Edinburgh DataShare. Stuart Macdonald EDINA & Data Library University of Edinburgh Research Data Services @ Edinburgh: MANTRA & Edinburgh DataShare Stuart Macdonald EDINA & Data Library University of Edinburgh NFAIS Open Data Seminar, 16 June 2016 Context EDINA and Data Library are a

More information

Technology Special Interest Group Thursday, December 4, Tony Hanson Webmaster Technology Special Interest Group Leader

Technology Special Interest Group Thursday, December 4, Tony Hanson Webmaster Technology Special Interest Group Leader Technology Special Interest Group Thursday, December 4, 2014 Tony Hanson Webmaster Technology Special Interest Group Leader 2014 2015 Schedule Sept: Backing Up Your Data Oct: Archiving 1: Organizing Your

More information

RADAR A Repository for Long Tail Data

RADAR A Repository for Long Tail Data RADAR A Repository for Long Tail Data Angelina Kraft, Janna Neumann German National Library of Science and Technology TIB 36th Annual IATUL Conference Hannover, July 6 th, 2015 funded by IN A NUTSHELL

More information

ProQuest Dissertations and Theses Overview. Austin McLean and Marlene Coles CGS Summer Workshop, July 2017

ProQuest Dissertations and Theses Overview. Austin McLean and Marlene Coles CGS Summer Workshop, July 2017 ProQuest Dissertations and Theses Overview Austin McLean and Marlene Coles CGS Summer Workshop, July 2017 Agenda Dissertations and ProQuest Short form video Pilot Project 2 A mission that aligns with universities

More information

Connect America Fund Webinar: Filing with the HUBB Portal. High Cost Program December 13, 2017

Connect America Fund Webinar: Filing with the HUBB Portal. High Cost Program December 13, 2017 Connect America Fund Webinar: Filing with the HUBB Portal High Cost Program December 13, 2017 1 Housekeeping Use the Audio section of your control panel to select an audio source Turn on your computer

More information

UVic Libraries digital preservation framework Digital Preservation Working Group 29 March 2017

UVic Libraries digital preservation framework Digital Preservation Working Group 29 March 2017 UVic Libraries digital preservation framework Digital Preservation Working Group 29 March 2017 Purpose This document formalizes the University of Victoria Libraries continuing commitment to the long-term

More information

Assigns a persistent identifier that will always point to the object and/or its metadata.

Assigns a persistent identifier that will always point to the object and/or its metadata. Preservation Policy Mountain Scholar is committed to providing long-term access to the digital works it contains. Adhering to best practices, Mountain Scholar and CSU Libraries staff use digital preservation

More information

C4: Library, Archives, Museum, and more: connecting for improved heritage services in the City of Burnaby

C4: Library, Archives, Museum, and more: connecting for improved heritage services in the City of Burnaby C4: Library, Archives, Museum, and more: connecting for improved heritage services in the City of Burnaby Kathy Bryce, Partner, Andornot Consulting Inc. and Arilea Sill, IAP Administrator, Government of

More information

Putting Open Access into Practice

Putting Open Access into Practice Putting Open Access into Practice Dr. Nancy Pontika Connecting Repositories (CORE) Knowledge Media Institute Open University Twitter: @oacore VTT, Espoo (Finland) 11-12 May 2015 This work is licensed under

More information

Data Virtualization Implementation Methodology and Best Practices

Data Virtualization Implementation Methodology and Best Practices White Paper Data Virtualization Implementation Methodology and Best Practices INTRODUCTION Cisco s proven Data Virtualization Implementation Methodology and Best Practices is compiled from our successful

More information

The What, Why, Who and How of Where: Building a Portal for Geospatial Data. Alan Darnell Director, Scholars Portal

The What, Why, Who and How of Where: Building a Portal for Geospatial Data. Alan Darnell Director, Scholars Portal The What, Why, Who and How of Where: Building a Portal for Geospatial Data Alan Darnell Director, Scholars Portal What? Scholars GeoPortal Beta release Fall 2011 Production release March 2012 OLITA Award

More information

Understanding my data and getting value from it

Understanding my data and getting value from it Understanding my data and getting value from it Creating Value With GDPR: Practical Steps 20 th February 2017 Gregory Campbell Governance, Regulatory and Legal Consultant, IBM Analytics gcampbell@uk.ibm.com

More information

Indiana University Research Technology and the Research Data Alliance

Indiana University Research Technology and the Research Data Alliance Indiana University Research Technology and the Research Data Alliance Rob Quick Manager High Throughput Computing Operations Officer - OSG and SWAMP Board Member - RDA Organizational Assembly RDA Mission

More information

Robin Dale RLG

Robin Dale RLG Robin Dale RLG Robin.Dale@notes.rlg.org Diversity of applications (commercial, home-grown, operational, etc.) in the organization, structure and encoding of documents and data Complexity varies greatly

More information

Certification Efforts at Nestor Working Group and cooperation with Certification Efforts at RLG/OCLC to become an international ISO standard

Certification Efforts at Nestor Working Group and cooperation with Certification Efforts at RLG/OCLC to become an international ISO standard Certification Efforts at Nestor Working Group and cooperation with Certification Efforts at RLG/OCLC to become an international ISO standard Paper presented at the ipres 2007 in Beijing by Christian Keitel,

More information

Introduction to. Digital Curation Workshop. March 14, 2013 SFU Wosk Centre for Dialogue Vancouver, BC

Introduction to. Digital Curation Workshop. March 14, 2013 SFU Wosk Centre for Dialogue Vancouver, BC Introduction to Digital Curation Workshop March 14, 2013 SFU Wosk Centre for Dialogue Vancouver, BC What is Archivematica? digital preservation/curation system designed to maintain standards-based, longterm

More information

CA Test Data Manager Key Scenarios

CA Test Data Manager Key Scenarios WHITE PAPER APRIL 2016 CA Test Data Manager Key Scenarios Generate and secure all the data needed for rigorous testing, and provision it to highly distributed teams on demand. Muhammad Arif Application

More information