MetaMorphosisLitReviewJISC

Transcription

1 MetaMorphosisLitReviewJISC Paper/Publication Summary Preserv final report, March 2007 Main points of project: To investigate the potential of long term preservation in Institutional Repositories (IRs) Preservation metadata key in flow of data Worked with TNA to produce PRONOM ROAR (link through OAI harvesting service resulting in format profiles for repositories listed in ROAR list of open access repositories). Used PREMIS to assess needs of IRs Project resulted in a range of tools/plugins for Eprints Outputs are very much to do with retrospectively considering file format, but the report stresses that restricting file format at upload stage (if upload is the onus of the depositor), a request for source version should be made too (I think this means that if the depositor has migrated the file in order to upload, they should upload the original also, even though it will not be viewable) Creation of preservation metadata should not be the responsibility of the depositor, however, an acknowledgement of rights to migrate to facilitate access is required Planning not action is key Identifies IRs as having good potential for long term preservation of digital objects Useful/relevant to our project? Yes and no. It s not a long read, but it is a bit confusing due to the varied objectives of the project Further reading AIMS: An Inter- Institutional Model for Stewardship, January 2012 (University of Virginia, University of Stanford, University of Hull, Yale University) Points we could draw upon: production of a diagram showing how we feel our system/model might look. This will probably change, but could be a good activity importance of consulting with stakeholders An inter-institutional framework for stewarding born digital content Writers see best practice as impossible at this stage, and stress good practice as being key acknowledge that there is no single solution Framework identifies the Functions of stewardship as Collection Development Accessioning Arrangement and Description Discovery and Access Each of these functions is then addressed by the following sections Definition Preface Keys to Success Objectives (with outcomes outlined along with decision points and tasks) Yes. Very useful document particularly on formulating plans for ingest (Collection Development and Accession stages) Paradigm AIMS doc does not really touch on cataloguing Karen s Directory Printer (karenware.com) e-records.chrisprom.com/page_id=175 The sections set out points that must be taken into consideration, along with other points to consider. Collection Development Includes useful section on what to consider/include on a Born Digital Collecting Policy (should supplement general collecting policy); need for a Digital Preservation Strategy; Legal Agreements. A lot of emphasis is on talking to the depositor, and sample questionnaires are provided in the appendix. Accessioning Includes recommendations on processes and documentation to be put in place for stages identified by writers. Things to think about: Writers stressed need to have a team approach to the whole process (including formulation of policy). This is ok at the moment, as that is what we do anyway, but the document shows that in a lot of cases the curator is quite often not working hand in hand with the archivists who will be handed the digital objects later on. This needs to be addressed in the review of documentation Interesting points on potential for multiple structures/hierarchy arrangements when dealing with digital objects Recommends risk assessment approach to opening/closing of objects for which content is not known Making documents available online constitutes publication (in the UK) what are the implications of this? Need to inform depositor/need to change rules of citation/copying

2 JISC funded Version - LSE Versions Toolkit - Developed by LSE as part of the JISC FAIR funding stream. The project aimed to create a toolkit to help with the management of versions of academic research outputs. Toolkit relates more to the management of research outputs from a librarian s point of view for submission to the institutional repository. Designed to help academics in all areas of management of their research outputs but does include some interesting tips. Surveyed economist researchers. Interesting stuff from their survey: 59% of articles go through >4 versions and co-works can end up with as many as 60 versions 36% of respondents kept all versions 54% kept the main versions Only 8% kept the last version only Ways of organising by project / date - i.e. this is what people do Lots of comments re needing help re version control and naming conventions Provides good information on naming conventions that would help us when training or working with academics who want to deposit electronic archives with us. But more suitable for published documents N/A Helpful tips for naming conventions included in the toolkit: Add the date to the first page of the document Use sensible title e.g. draft, submitted version, accepted version, published version, updated version Versions versus revisions i.e. version = major changes; revision = smaller changes Keep milestone versions only Milestone versions are defined as: Books / journal articles - draft, submitted version, accepted version, publishers proof, published version Presentations final version only Conference paper final version; revision for publication Working paper submitted version, published version Choose one location as main storage space and make sure devices sync to it / you update the main location area on a regular basis. Decide on a fileplan that works for you and stick to it! JISC Developing an Institutional repository for Leeds Met Uni Early project that covers all areas of development of institutional repository e.g. audience development, choosing software, metadata, development of interface, ensuring repository content is findable on the web etc. Leeds Met didn t use Eprints and although future plans were to open up repository content to wider range of resources at this point focused on published outputs N/A

3 JISC Cairo Project Joint project between John Rylands, Bodleian and Wellcome Libraries. Aimed to bring together existing tools for ingest in a documented, automated, integrated workflow, to produce repository-independent metadata packages, in the form of METS files, that could provide the basis for long term life cycle management. Aimed to develop a user-friendly ingest tool for both accession and metadata creation that could be used by organisations using repository software and regular filestores. At the time of writing the final report it was however only at the alpha development stage. Looked to address four main areas: 1. Fragmentation of tools 2. Metadata requirements 3. Preservation of complex archive collections 4. Reduce cost of ingest Yes has the potential to be worthwhile testing perhaps for accession stages and metadata extraction. However when the final report was written tool was only in alpha development stage. Project extended work of Paradigm and Digital Curation in Action projects Project was extended by futurearch project Cairo project site has info on the tools developed Outputs: 1. Completed a survey of current ingest tools. For each (53 in all), the following information was recorded: source URL, formats covered, technology base, operating system, dependencies, tool category, license, description, output method, maturity and other notes. Results published as a Tools Survey (2007) 2. Use cases i.e. case studies of how archivists etc could use the tools published on project website 3. Content models i.e. metadata packages based on standards METS, PREMIS, EAD, DublinCore, MODS. Developed into xml schemas. User documentation was not completed as part of the project. Outputs: 1. Cairo architecture for ingest. Open source and available via the project website. Potential for it to be developed. Uses Java and Eclipse and is easy to download. 2. The Cairo tool can be used to create metadata along the lines for some of the content models. At the time the final report was written not all models were available as the project team was waiting for SOAPI project to develop its tool. 3. Users work their way through a form interface Oxford were going to be taking the work forward with user testing and the creation of user documentation. At the point of writing the final report the tool was not so far developed that it could be implemented on a general basis at organisations Need to look at project website to see what development followed. Or the FutureArch project links to right Final report is particularly user friendly particularly as it s designed to create tools for archivists etc who don t have the technical knowledge. Paradigm Workbook for management of personal digital archives (in progress) REMAP Like AIMS project takes you through the main areas of collection management: 1. Collection development proposes a number of options 2. Accession information on how to go about acquisition (helpful with good links that we should have a look at) 3. Agreements includes a checklist (quite helpful) 4. Appraisal and disposal (this section was disappointing and either provided no information e.g. secure disposal or information on a level that we already know e.g. appraisal thinking about content, context and structure) 5. Administrative and preservation metadata (largest section) 6. Arranging and cataloguing hybrid collections contains several pages giving an overview of current standards i.e. ISAD(G), ISAAR(CPF) etc Builds on the REPROMAN project developed at Hull Uni. This was a project to develop a webbased tool that interacts with the Fedora repository allowing users to interact with the repository in various ways. Relates specifically to using repository software for records management Ideas that could be relevant for us: Creation of alerts using Calendar software e.g. reports on the content of the repository; events e.g. you need to do something / this just happened; dates e.g. notification of destruction dates etc. they also created responses e.g. snooze, take default action They used to create the alerts for the period of the project but the long-term intention was to develop RSS feeds. The project led to the Hydra project to develop a plug-in for Fedora Commons Yes useful reference tool with links to other resources. Some of the information is very basic and a little obvious e.g. appraisal and the agreements sections. Probably not at this stage Very technical and specific to using Fedora. Some of the ideas (listed left) would be interesting to follow up with EPrints Paradigm final report

4 INCREASE The main aim of the project was to increase the content of the repository by creating systems for bulk-import of metadata and items into an EPrints repository. It is quite an early project which reported in Methodology: collected information on current practice by website survey (to find databases that they could bulk import from), interviews and online questionnaire Bulk import was time consuming due to amount of time required to modify EPrints plug ins and staff time to improve metadata Encouraged self deposit by: making things simple (e.g. allowing depositors to just use their institutional log-in as authentication - Repositories commonly use LDAP or Shibboleth for this purpose), constant reminders and creation of data to prove impact of self-deposit Used local administrators to create basic records in Eprints and to be a point of contact for academics. i.e. the academics sent their outputs to the administrator worked well. More about None pulling metadata from other databases across the 3 universities that had the joint repository. There is a lot about the methodology which is quite specific to the organisations. Used and adapted Eprints plug-ins. There are many see Bulk imports questioned whether they were a good idea. Created further information here Born Digital Guidance for Donors, Dealers, and Archival Repositories M Forstrum, N Kuhl, S Thomas, J Leighton John, M Barnard, G Redwine, Kate Donovan, E Farr, W Hansen, S Shaw General overview a very basic look at the process of transfer of born-digital (and supposedly hybrid) collections to a repository. Would be useful to refer a really interested donor to parts of it. Intro Document aims to provide recommendations for the safekeeping or wellbeing of born digital material transferred from depositors to repositories. Body of report surveys main issues related to such acquisitions and is intended for a broad range of interest levels and expertise. Initial collection review Considerations and approaches that inform collection review. Useful points from this chapter Take into account the potential for the storage vehicle of digital records to be damaged by mould, water, dust etc. Other useful points Consider differences in copyright law when making material available online world wide a disclaimer on site allowing access should cover this Should we create a disc image (file or files that contains an exact copy of the content and structure of a hard drive or other storage media)? If so this needs to be included in deposit/donation agreement. section 3.4. However, this section seems o assume the original structure will not be used, rather the documents pulled out from it to make them available Statement on allowing files to be decrypted in deposit agreement? Advise on packaging for transit of digital media People might send us obsolete storage we need to say what will happen to them Too basic to be of much use to archivists; potentially useful for an education tool for depositors. Appendix A is a kind of checklist for depositors and repositories and could be adapted to make a useful document to give to depositors Appendix B also worth a look for making lists of the related costs and potential costs Carcanet Preservation Project May 2012, University of Manchester Library Appendix C checklists for donors and dealers, and for repositories could be adapted for use Practical project funded under the Digital Preservation Strand 12/11 (same as Digital Directorate). Main aim of the project was looking at the practical steps that need to be taken to preserve (rather than make accessible) s. Took snapshots of accounts of two members of staff at external organisation. Ended up with 6.pst files c170k s. Tested available tools for characterisation, metadata extraction, migration to formats for preservation. Also looked at the workflows for dealing with acquisition and preservation of s (with a little appraisal) One drawback is that it deals with Microsoft Outlook and s created as part of business activities. Although the workflows would essentially be the same different issues arise when dealing with personal accounts see below Yes has good information on what toolkits are reliable (although one of their conclusions was that they did not have enough time to test a full range of tools). Also has a good workflow for dealing with s. Refers to the INSPECT project significant properties for s. Also a good number of tools are recommended, in particular Kernal pst Reporter for extraction of descriptive metadata from pst files and Aid4Mail for converting to xml. DPC have a Technology Watch Report on preserving s (notes below)

5 Digital Preservation Good in-depth introduction into preserving s. Coalition Preserving by Has a good literature review and a huge bibliography. Also directs reader to the main texts to read Christopher Prom re technical aspects, legal considerations, organisational constraints of preserving s. December 2011 Also considers issue of preservation for s more widely. E.g. dealing with s created by proprietary systems (e.g. Microsoft) to accounts in the Cloud A little idealistic re best practice e.g. says can only develop an management and preservation policy once we have a full understanding of the technical and legal issues. [Events don t always happen in this way] P16-17 records management and what organisations do. E.g delete at a certain date, delete when person leaves organisation etc Management and preservation of s has largely been left to the user. User management is largely ad hoc and a mass of unsorted s. The author states that preservation methods should support the preferences that people have expressed for personal digital archive applications and the fact that people often prefer to manage their resources by scattering copies among multiple locations It s a good introduction to the subject of preserving s. It also tends to go over some issues in little detail and poses as many questions as it answers. The big issue of making them accessible is a particular example of this. Has a very good bibliography, including a top4, which we should read in the future There is a lot of information on the general standards of s. From transmission onwards. Xml is generally the accepted format for preservation and an xml format the Account Schema has been developed. However there is no way to make this format accessible (query, render, display) so recommends that messages also preserved as EML (esp those with attachments as it supports them being written as separate files) Sections that would be really helpful are limited in scope. E.g. how to make s accessible once they have been preserved. Although it states that xml is the best format for long term preservation but that it doesn t support access it rattles through a potential method of the processes required to make them accessible (using other tools) in one paragraph. Has a list of good recommendations for organisations and individuals which will be helpful for when we write our digital preservation procedures. Offers NO guidance on what to do about sensitive information in s or redaction. Rethinking Personal digital archiving Parts 1 and 2 by Catherine C Marshall D-Lib Magazine March / April 2008 Vol 14 Number 3/4 Includes a glossary which might be helpful Part 1 Looks at the idea of benign neglect of digital preservation and ideas around how people approach DP (at home) (and that it tends to be a mix and match approach) e.g. stick it in the cloud (esp ) & social media (e.g. Flickr), deal with it later, keep everything, the next generation will sort it out, move complete contents of old pc to new, transfer to external drives. Considers the drawbacks of this and why these solutions do not constitute DP. Describes issues re duplication and poor filing structures Part 2 Evaluation of a digital collection requires a heuristic assessment. Covers: What should we keep? Where should we put it How should we maintain it But doesn t really answer the questions just talks about current practice by record creators (in the home) Source, actions and disposition all important when assessing archive collections Archiving as a side effect rather than a strategy Talks a lot about the management of photographs including effects of using social media to store. It would potentially be of help in building our knowledge on training depositors and potential depositors, internal academics on how to manage their digital collections but it focuses more on how families can look after their digital collections. Although it did have some useful information and this was incorporated in to FAQ s and guidance for individuals. Bibliography includes further articles by author on personal archives inc by academics that would be worthwhile reading About records created as part of writing a paper Managing information over a lifetime Marshall.pdf Long-term fate of digital belongings An easy read so if we get anyone who is really keen on their own DP we could recommend this for initial reading.

6 Maureen Pennock Curating s section from the DCC Digital Curation Manual, 2006 Split into the following sections: Curating s - Standards (and non-standards) governing s - Authenticity and integrity including e.g.s of where s have been used in court cases (inc Tiffs being dismissed as an inappropriate format for preserving s) - How do you capture s for Rec Mgt purposes - Legislation (this section is very good) considers DPA (principles 4, 5 & 7 are the main ones); Regulation of Investigatory Powers Act (defines extent to which orgs. Can monitor and access communications inc ); Human Rights Act (as an EU law it overrides UK law i.e. the RIP Act); IPR legislation - Risk management inc a list of the main risks of failing to properly manage s Roles and responsibilities this mainly relates to the roles of the creating organisation - Need to DP / RMgt content in policy (not just re use of ) - repositories little published information on how well they work in terms of long-term preservation - presents the question of how you provide access. Preservation: - 2 forms of structure are important to preserve the structure of the individual (with the header and main body) and a series of communications ( thread) - considers the main potential formats for preservation (TIFF, pdf, RFC 2822 and xml) Xml best - Tiff not accepted as a good enough format re authenticity and integrity of records. - RFC 2822 not good as the attachment is encoded - XML file processed and rendered using XSLT (good for migration / is reliant on fact computers will continue to read Unicode characters but generally accepted that they will p26-7) Practical steps (for organisations) This is a really good introduction to management especially from a Records Management point of view. Like a lot of the literature it does seem to be written with a non-specialist (senior mgt) audience in mind, but it would be a good reference tool to pick out the main points. For the technical side of I would recommend reading Digital Preservation Coalition Preserving by Christopher Prom December 2011 (see above) rather than this but it does provide a good basic introduction JISC have guidance on the RIP legislation (regulation of investory powers) we will need to look at this if as an organisation the IOE ever moves towards capturing s for Rec Mgt in a systematic way also has a good bibliography The section on legislation is very good Would also help with the FAQs for organisations wanting to deposit archives You made it, you take care of it. Data management as personal information management Kathleen Fear, International Journal of Digital Curation A review of the way academics manage their data as part of research projects. Conducted survey and interviews USA study Argues that data management is a form of personal information management (PIM) def/n p60 Interesting points - Although researchers didn t own it they expressed a strong feeling of control over it - Most respondents couldn t estimate how much data they had but less than 1GB was a common answer - Many didn t know the lifespan of the data (1/5); more than a 1/3 said ten years or more - Funding applications don t include data management strategies and if they are requested for applications academics just include a generic statement (i.e. they don t think about it) social science and humanities particularly bad at this - All had problems sharing data 49% did it through some form of external media - Sharing data many felt that writing a paper was enough sharing of data, and many don t consider future re-use, but they don t discard it either - There is no cleaning up of data at the end of the project it just remains where it is and staff who understand it tend to leave - It s the research assistants, PhD students etc who really understand the management of the data and they are the ones that leave - Terminology in surveys e.g. what do you do with your data was much more productive than how do you manage your data. Many don t consider themselves as managing data information created en route to a publication better way of looking at it - They want training in how to manage data but it has to be packaged as something that aims to support researchers in performing activities that they feel they do not have time for, not to change work practices they already have It read a little bit like a published dissertation e.g. intro, methodology, lit review, findings, conclusion. Most of it was fairly standard: e.g defining data, importance of DP, researchers goals are not aligned with those of repository developers Useful for internal records and data management training Bibliography has a few articles that might help if we have to consider this in more depth

7 Technology Watch Report. The Open Archival Information System Reference Model: Introductory Guide Trusted Digital Repositories: attributes and responsibilities RLG OCLC report A slimmed down explanation of the OAIS model. Introduces the packages etc. Establishes the attributes requires of a digital repository for cultural organisations. Includes useful recommendation boxes summarising what they think is really important. Focuses very much on how to build an OAIS compliant repository Definitely a good intro into the concepts Large, but a requirement!

8