NEDLIB LB5648 Mapping Functionality of Off-line Archiving and Provision Systems to OAIS

Size: px

Start display at page:

Download "NEDLIB LB5648 Mapping Functionality of Off-line Archiving and Provision Systems to OAIS"

Egbert West
5 years ago
Views:

1 NEDLIB LB5648 Mapping Functionality of Off-line Archiving and Provision Systems to OAIS Name of Client: Distribution List: Author: Authorised by: European Commission NEDLIB project Berkemeyer, Jörg DDB Date of Issue: 19 January, 1999 Issue: 0.1 Reference: Total Number of Pages: 10 Contact Details for DDB: GEN-232 Die Deutsche Bibliothek Adickesallee 1 D Frankfurt a.m. Germany Tel.: (+49) Fax: (+49) berkemeyer@dbf.ddb.de

2 NEDLIB LB 5648 TABLE OF CONTENTS TABLE OF CONTENTS... i 1. Introduction and Purpose Types of Media Diskettes CD-ROM Others OAIS and Off-line Media OAIS Environment vs. Environment of an Off-line System Representation Information Information Packages vs. Digital Objects OAIS Responsibilities and Off-line Systems at national libraries Negotiation and Submission of Media by Publishers Control for Long-term preservation and availability Workflow Selection Acquisition and registration Verification Record access conditions Description Content-indexing Storage-handling Retrieval Delivery Access Monitoring Functional Model Ingest Archival Storage Data Management Administration Access Conclusions Literature...8 i

3 1. INTRODUCTION AND PURPOSE This document describes the OAIS model in view of characteristics and requirements of off-line media. Parts of it could serve as a supplement to the part on off-line tools/mmb in deliverable D2.2.1 Tools and Modules for Capture and Installation of Electronic Publications. It refers to the functional entities and central concepts of the OAIS model. 2. TYPES OF MEDIA OAIS model claims to be valid for all kind of objects physical objects and digital objects. Off-line media carry both characteristics as they are physical objects in form of CD-ROM, diskette, booklet, jewel case etc. as well as digital objects represented by data. Some of the physical objects can be transferred through digitisation (e.g. scanning the label and booklet). Although some standards emerged, as far as setup, operating systems, applications for development, multimedia etc. is concerned, the spectrum of media types is still heterogeneous. New technologies (hard- and software) are appearing constantly, requiring new strategies from deposit libraries. 2.1 Diskettes As most PCs were equipped with diskette drives, this was the first choice as medium for publishing. The publication types were: proprietary software, databases, texts, numeric data, and learning software. 8, 5 ¼ and 3 ½ disks were on the market, nowadays 3 ½ disks are the standard. The older 8 and 5 ½ formats were housed in a vinyl jacket, whereas the 3 ½ format has a plastic shell. Data is stored on a magnetizable material and the read/write head of the drive is in physical contact with the surface of the data carrier. Synonyms are flexible disk and floppy disk. 2.2 CD-ROM Whereas at the beginning of electronic publishing diskettes were dominating the market, nowadays CD-ROMs (Compact Disk Read Only Memory) are the main publishing medium. The high storage capacity makes it possible to publish multimedia contents. Nearly all PCs are equipped with CD-drives, that serve for software distribution and use, multimedia applications and games. 2.3 Others New types of media and combinations of off-line and online media are appearing on the market. DVD, with a total number of devices sold in the U.S. of 1.5 Mio. (Jan 99), is one important new standard. However, most of the published DVD are videos. Portable electronic books 1 and publications for handheld devices are niche markets. 3. OAIS AND OFF-LINE MEDIA OAIS archives are organisations (people and systems) that are responsible for preservation of and access to information. National libraries and national archives qualify for this definition, as they have the legal obligation to preserve the documentary heritage of their countries. Most of the national libraries and legal authorities have defined, that off-line publications belong to this 1 See the homepages of the dominating producers of portable electronic books at: and 1

4 documentary heritage and have therefore to be preserved. To fulfil this obligation, systems have to be implemented to administer, preserve and present electronic publications. The OAIS model addresses a full range of archival information preservation functions ingest, archival storage, data management, access, and dissemination. Another part of the model includes the migration of digital information to new media and formats. Beside these archive functions the model describes internal and external interfaces. The OAIS model should also be interesting for those parties producing the information. In terms of off-line publications and national libraries the major group are publishers of CD-ROMs, DVDs and Diskettes. Concepts for preservation of digital information are more complex than in case of traditional media (books). The OAIS model demands a closer co-operation of archives (national library) and producer (publisher). Another central concept of OAIS are Information Packages that hold content, preservation information and packaging information. In case of off-line media this means keeping the original medium or an image of the medium respectively together with additional information. This information may come from the publisher, in most cases however it will come from traditional bibliographic services, like subject indexing, bibliographic record, etc. Main purpose of off-line provisioning systems is to administer these media. Other services may be offered as well, like access to the WWW, charging, user management etc. This does not exclude the conformance to OAIS, as conformant models include stand-alone archives and larger systems with additional services OAIS Environment vs. Environment of an Off-line System A system for provision of off-line media at a national library or archive is interlinked with systems and groups of the environment. The simple model describes links to Producer, Consumer and Management. Publisher Off-line System User Policies Figure 1: Environmental model of an OAIS mapped to the situation of a national library responsible for deposit of off-line media The major group providing off-line media to a library are publishers and CD-ROM producers respectively. These organisations resemble those involved in the traditional model for books, as publishers often are producing CD-ROM titles as well. Supplying chains are similar to traditional workflow at libraries and the guidelines for deposit of off-line/traditional media depend on national 2 See: Consultative Committee for Space Data Systems: Reference Model for an Open Archival Information System (OAIS), White Book W

5 law and agreements between library and publisher. With digital publication it is easier for individuals to publish documents or multimedia titles surpassing the traditional publisher and changing the traditional publication chain. Although this problem is less relevant for off-line publications where traditional publishers are dominating the market, technical achievements could change the situation in the future. Deposit libraries and archives will have to integrate these actors in their workflow 3, who are less aware of deposit requirements and who are less well organised (e.g. publishers association). Preservation is one aspect of the responsibilities of national library, access is the other. Especially the scientific community should be able to access off-line titles like traditional media. The term consumer, which is used by the OAIS model, is in library context replaced by reader and user respectively. A smaller group of users are library staff, who need access to several off-line publications for their daily duties. The term management, used by the model, is not meant in sense of daily operations and management of the archival systems. Management is the role played by those individuals, organisations, and bodies who set overall OAIS policy in a broader policy domain. In terms of national libraries this could be legislative organisations setting deposit law or board of directors defining deposit guidelines for off-line media. 3.2 Representation Information OAIS model demands a clear definition of information. Information is defined as data, that can be interpreted by a user. OAIS document gives the example of a CD-ROM containing ASCII text (data) along with a definition of ASCII and description of format, contents etc. (representation information). To present and preserve off-line publications a thorough understanding of hard- and software is necessary. This information has to be collected and preserved as representation information. Some of these representation information can be generalised for off-line publications, e.g. ISO9660 industry standard definitions, substrate, reflective material, reading mechanism of a CD-ROM. Other representative information can be described for groups of publications, e.g. publisher, software for development. A lot of information however has to be gathered individually, e.g. system requirements, proprietary software, access restrictions. An off-line archiving system has to ensure, that representation information will be understandable in the future. Data Object interpreted using its Representation Information yields Information Object CD-ROM Diskettes Images of CDs/Disks Metadata Software Hardware... 3 See chapter 4. 3

6 3.3 Information Packages vs. Digital Objects To describe processes of submission and dissemination of information the OAIS model introduces the concept of Information Packages (IP) in a variety of forms. Information Packages are preserved by an OAIS conformant archive, complemented by Information Packages for submission, archival and dissemination, representing the three processes of an archive. Electronic publications of an off-line system come in several formats and on different media. In order to provide standard policies and methods for administration, an off-line system should use a uniform object format. For example the MMB system is ready to make use of the most common formats and media (CD-ROM, diskettes, file formats), other objects need to be transformed (labels, special file formats, manuals). These files are stored in system controlled directories called Digital Objects, combining all objects of a publication and independent of the original media. Digital Objects make use of a container principle, similar to the Information Package concept of OAIS. They contain files, images, installation scripts, configurations, CDs, disk images, and user workspaces. 3.4 OAIS Responsibilities and Off-line Systems at National Libraries The OAIS document defines several mandatory responsibilities that an organisation must fulfil in order to be OAIS conformant: negotiation of information submission with producers, control of information to ensure Long-term preservation, determine user communities, independent availability, define documented policies and procedures, presenting the information to the designated community Negotiation and Submission of Media by Publishers Most national libraries and archives included off-line publications into the deposit legislation. As most publications come on CD-ROM and diskettes, respectively, some de facto standards are established. Other information on technical issues, installation, hard- and software requirements come with the installation description of most off-line titles and need not be provided by the publisher in an additional process Control for Long-term Preservation and Availability Most off-line publications come on reliable media that can easily be transferred to new media of the same kind or other media. The OAIS model defines three relevant issues to be addressed by the archive: copyright, authority to modify representation information, and agreements with external organisations. Copyright aspects with digital publications do not differ from traditional publications at libraries like books, magazines and newspapers. Deposit law allows in general to store the publications and make them available locally. Some national depositories are allowed to modify representation information of publications as part of their obligation to preserve the documents. Establishing agreements with other organisations to use services in preservation are no obstacle to national depositories. Parts of the preservation process can be sub-contracted, like digitisation, or delegated to other libraries. The user community is defined by national deposit laws and includes in most cases the general public. This public should be able to access the off-line publications without being involved in system administration, installation and registration processes. The long-term understandability of contents is rather an intellectual than a technical problem. Most off-line publications address a general public or interest groups and are well-documented, assuring that the contents will be understandable in future. Problems of emulation, migration and long-term 4

7 availability of off-line publications have not been solved yet. The OAIS model describes this dilemma, but does not offer a solution too WORKFLOW The workflow for electronic publications at a deposit library includes following steps: Selection Selection of relevant deposit material is based on legal deposit and therefore varies from country to country. NEDLIB project decided not to define specific selection criteria, as they may vary from country to country. However most national Archives and national libraries are collecting off-line media, more precisely CD-ROMs and diskettes. Other formats, like DVD, will be relevant in the future and have to be taken into account. Hybrid media, consisting of combined online and off-line content, are appearing on the market. They demand new archiving and access strategies, as well as technological solutions. 4.2 Acquisition and Registration The process of acquisition and registration resembles the traditional processes for books, magazines and newspapers. Off-line media are sent from the publisher to the deposit library. Sometimes diskettes and CD-ROMs come in combination with books or magazines, resulting in two separate acquisition processes or a combined one. Publishers may be asked to provide additional information like metadata, system requirements etc. 4.3 Verification Verification is described as a control procedure, checking integrity of medium, file format and document. For this purpose an off-line publication may be installed, virus checked, or checked in general by commercial software tools. This includes: Checking the integrity of the original data carrier (e.g. scratches on CD-ROM, diskette damaged). A test installation on current systems, if the publication uses individual installation procedures. If not, the contents of the publication should be checked with common viewers. Removing the publication from the system by use of de-installation processes of the system or the application respectively. Although a virus check should have been done by the publisher, another check at the library is necessary to protect the archive systems from threads and damages. Results of these processes could lead to metadata, recorded as representation information in the OAIS model. 4 See page See: van der Werf (1999a), page

8 4.4 Record Access Conditions Most of the copyright, deposit and license issues are regulated under national deposit law. License and authorisation processes can be tested, documented and, if necessary, negotiated with the publisher. 4.5 Description For cataloguing and title description the library staff needs to install and use the electronic publication. Some information comes with the accompanying material (handbook, label, booklet), other information can be provided by the publisher. 4.6 Content-indexing Automatic indexing of off-line media is difficult because of the heterogeneous file formats and application types as well as the varying contents of the publications. 4.7 Storage-handling Off-line publications require new and special processes for storage, as media types (e.g disks, CD- ROMs) are different to traditional media like paper. Simple storage in the archives of the deposit library resemble workflow processes of books, although the lifetime of the new media is shorter than that of paper, requiring different archiving conditions (humidity, light, shelves etc.). Storage also includes archiving of additional material like booklets, jewel cases, handbooks etc. Storage in the OAIS model proposes a transformation of the contents (SIP) to an AIP (Archival Information Package). The model leaves the concrete realisation of processes and methods to the deposit system. 4.8 Retrieval The original publication or the image of a off-line publication must be retrievable from the storage system. In OAIS model this requires another transformation and mapping to DIPs (Dissemination Information Packages). 4.9 Delivery This stage includes transformation and re-packaging of contents of the off-line publication to make them available to the user. Functions are in some cases limited by the original application. The offline system must be capable to handle several forms of delivery, including different processes for printing, copying, sound and visual reproduction. Conditions for delivery had to be identified in previous stages of the workflow Access Access includes several processes in user/system interaction. Access to off-line media can be provided locally at a single workstation, locally at a networked workstation, and at a remote workstation. Access to off-line publications may be limited by copyright and license agreements. The user is supported by Finding Aids that make use of descriptive information provided by the offline system. 6

9 4.11 Monitoring The use of off-line media as well as the submission and processing of off-line media needs to be monitored by the system. This may include: Access and usage statistics Monitoring of authentication and verification processes. Information on transformation and delivery processes. 5. FUNCTIONAL MODEL The OAIS models describes five functional entities Ingest, Archival Storage, Data Management, Administration, and Access. An off-line system can fulfil some parts of the model, whereas other functions have to be addressed by other parts of the complete demonstrator developed in Nedlib context Ingest The Ingest entity encloses traditional library activities like receiving the publications from the publisher, cataloguing, classification, as well as system specific activities like recording the installation, quality assurance, and transformation to digital objects (generating an AIP). The later functions have been described in the previous workflow section. Ingest is mainly addressed by workpackage 2 of NEDLIB Archival Storage Storage, maintenance and retrieval services and functions are provided by this entity of an OAIS conformant archive. Similar services are found at most off-line systems, although they may not be part of a single entity. E.g. within the MMB system these tasks rely on several operating system, database and administrative components Data Management The OAIS model integrates all data management functions into one entity (Data Management). A real-world solution will separate these functions, as they are provided by different software systems and managed by different divisions of the library organisation Administration The Administration entity of the OAIS model is heterogeneous, too, including softwareindependent duties like negotiations with publishers, defining policies and standards, customer support as well as technical requirements, like monitoring the system and configuration management. The administrative part of an off-line system has to fulfil some of these duties and supports library staff to provide services. Standard and policy definition as well as negotiation with publishers are system-independent, but could be influenced by requirements and capabilities of the off-line system Access Access functions of an off-line system include search function, description of media, location and availability of information stored. Users can request and receive off-line publications with an OAIS conformant system. The generating of HTML-pages presenting information and installation functions to the user could be compared to the Dissemination Information Package function of the 7

10 OAIS model. Users receive information and access services after submitting a request without being involved in complicated search and administrative functions. 6. CONCLUSIONS Main types of publishing, national libraries have to deal with, are CD-ROM and diskettes. New data carriers like DVD can easily be integrated in the workflow, however, mixed formats (off-line publications with online access) require new steps in the workflow. The OAIS model seems to be relevant for off-line publications, as the workflow, the concept of information packages, and the functional model could be mapped to the processes for submission, cataloguing, storage and access of off-line publications. The concept of digital objects is represented by the original media (CD-ROM, diskettes) and images of these media respectively, complemented by descriptive information. A container for these Information Package (IP) could be a file structure or a relational database. 7. LITERATURE Reich, L. and Sawyer, D. (1998): Reference Model for an Open Archival Information System (OAIS) Recommendation Concerning Space Data Systems Standards. Houston. Riehm, U. et al (1992): Elektronisches Publizieren Eine kritische Bestandsaufnahme. Berlin. Van der Werf, T. (1999): Functional Model of an Deposit System for Electronic Publications (internal paper). Den Haag. Van der Werf, T. (1999): Data Model for a Deposit System for Electronic Publications (internal paper). Den Haag. 8

The OAIS Reference Model: current implementations

The OAIS Reference Model: current implementations Michael Day, UKOLN, University of Bath m.day@ukoln.ac.uk Chinese-European Workshop on Digital Preservation, Beijing, China, 14-16 July 2004 Presentation