OpenDLib: a Digital Library Service System

Transcription

1 OpenDLib: a Digital Library Service System Leonardo Candela Istituto di Scienza e Tecnologie dell Informazione A. Faedo (ISTI-CNR) Via G. Moruzzi, Pisa (Italy) Phone number: Fax number: leonardo.candela@isti.cnr.it Donatella Castelli Istituto di Scienza e Tecnologie dell Informazione A. Faedo (ISTI-CNR) Via G. Moruzzi, Pisa (Italy) Phone number: Fax number: donatella.castelli@isti.cnr.it Pasquale Pagano Istituto di Scienza e Tecnologie dell Informazione A. Faedo (ISTI-CNR) Via G. Moruzzi, Pisa (Italy) Phone number: Fax number: pasquale.pagano@isti.cnr.it Manuele Simi Istituto di Scienza e Tecnologie dell Informazione A. Faedo (ISTI-CNR) Via G. Moruzzi, Pisa (Italy) Phone number: Fax number: manuele.simi@isti.cnr.it

2 OpenDLib: a Digital Library Service System Leonardo Candela, Donatella Castelli, Pasquale Pagano and Manuele Simi Istituto di Scienza e Tecnologie dell Informazione Italian National Research Council Pisa, Italy Abstract This chapter introduces OpenDLib, a Digital Library Service System developed at ISTI-CNR for easing the creation and management of Digital Libraries. It discusses the motivations underlying the development of such a system and describes it by presenting (i) the characteristics of the huge kind of content it is capable to manage, (ii) the set of functions it natively provides its Digital Libraries with, (iii) the powerful and flexibility of its component-oriented architectural paradigm as a key feature for addressing different application scenarios, and (iv) the technologies the system development rely on. The authors hope that understanding the OpenDLib foundational principles will not only inform stakeholders and decision makers of the features implemented by this existing system, but also assist researchers and application developers in the understanding of the issues, and their possible solutions, that arises when building Digital Library systems aiming at serve such a broad class of application scenarios. INTRODUCTION Digital Libraries (DLs) are complex systems whose lifetime span approximately across the last fifteen years. Along this period a lot of focused and from-scratch developed systems have been realized to serve the needs of specific communities. These systems are tightly bounded to the characteristics of the single scenario they have been conceived for and results hard to be maintained along the time. In addition to them, a few general-purpose Digital Library Systems, i.e., software systems designed to be easily used for realizing Digital Libraries suitable for certain application contexts, have been conceived. Such second class of systems systematizes the techniques and the software needed to implement Digital Libraries with certain characteristics resulting in a product that communities can use to build their Digital Libraries. OpenDLib is an example of such a second class of Digital Library System with an architecture explicitly designed to support plug-and-play expansions. The design and development of OpenDLib was initiated in 2000 as a response to a pressing request for software that could enable different user communities to create their own DLs. We decided to design a general-purpose software that could be customized to meet the needs of the different application scenarios. At that time we called this software a Digital Library Service System, to stress that it is a system that manages digital library services and makes them publicly available (Castelli & Pagano, 20003). The role of OpenDLib is analogous to the role of a database management system for a database,

3 i.e., it supports the creation and maintenance of distributed DLs. A DL can be created by instantiating OpenDLib appropriately and then either loading or harvesting the content to be managed. Our initial aim was to design a software tool that could provide a number of core DL functions on general content and that could easily be expandable. A DL is a very expensive resource, therefore it must be maintained over time even when new technologies that enable new functions are developed or when new kinds of usages are proposed. To satisfy this dynamic scenario, the DL must grow over time along several dimensions, e.g., services, metadata formats supported, hosting servers, user communities, etc. OpenDLib was designed to support this powerful notion of evolution. BACKGROUND AND REQUIREMENTS FOR A DIGITAL LIBRARY SYSTEM The systems developed in the digital library area until now are mainly dedicated to support digital repositories for satisfying the needs of single Institutions. Among such systems, usually called digital repository systems, Fedora (Lagoze, Payette, Shin & Wilper, 2005), DSpace (Tansley, Bass and Smith, 2003), and EPrints (Millington and Nixon, 2007) represent the most popular and adopted ones. Fedora is a Repository System specifically designed for storing and managing complex objects. It is implemented as a set of web services that provide full programmatic management of digital objects as well and search and access to multiple representations of such objects (Payette and Thornton, 2002). Fedora is particularly well-suited to work in a broad web service framework and act as the foundation layer for a variety of multi-tiered systems, service-oriented architectures, and end-user applications. DSpace is an open source digital repository software system for research Institutions (Tansley, Bass, Stuve, Branschofsky, Chudnov, McClellan & Smith, 2003). It enables organizations to capture and describe digital material using a submission workflow module, or a variety of programmatic ingest options; to distribute an organization s digital assets over the web through a search and retrieval system; and to preserve digital assets over the long term. EPrints, now in its version 3, is among the first free repository software for building OAI-compliant repositories and probably one of the most diffused (On December 2007 there exist 240 known archives running this software). This new version is a major leap forward in functionality, aiming at simplifying the tasks of the various players (depositors, researchers, managers) involved in running and maintaining a Repository. Digital repository systems, however, only provide core functionalities, such as search, retrieval, and access to information objects, which are not able to meet application specific requirements as those digital libraries are usually requested to satisfy. For example, during its lifetime a DL may have to support additional functionalities for satisfy the needs of new organizations that join the DL offering their specific information content and their computers to host the system; or it may have to manage policies that regulate the access to content and services. OpenDLib (OpenDLib; Castelli & Pagano, 2002; Castelli & Pagano, 2003) has been conceived to meet these specific DL requirements.

4 O P E N D L I B F E A T U R E S OpenDLib was created as a software toolkit for setting up digital libraries capable to satisfy multifaceted needs expressed by heterogeneous user communities. Because of this goal, it must be capable to support a powerful and flexible content model, a set of functions for appropriately dealing with such potentially rich content, and an architectural paradigm making it capable to be exploited in application scenarios having different deployment requirements. As regards the content, OpenDLib can handle a wide variety of information object types. This capability originates from the fact that it exploits a powerful and flexible Document Model named DoMDL (Candela, Castelli, Pagano & Simi, 2005). DoMDL has been designed to represent structured, multilingual and multimedia objects in a way that can be customized according to which content has to be handled. For example, it can be used to describe a lecture as the composition of the teacher presentation together with the slides and the summary of the talk transcript. Moreover the object may be disseminated as the MPEG3 format of the video or the SMIL object synchronizing its parts. In order to represent objects with completely different structures, DoMDL distinguishes four main aspects of document modelling and, using terms and definitions very similar to those coined in the IFLA FRBR model (IFLA, 1998), it represents these aspects through the following entities: Document, Edition, View, and Manifestation. The Document entity, representing the object as a distinct intellectual creation, captures the more general aspect of it. The Edition models an instance of the document entity along the time dimension. The preliminary version of a paper and the version published in the proceedings, are examples of successive editions of the same object. The View is the way through which an edition is perceived. A view excludes physical aspects that are not related to how a document is to be perceived. For example, the original edition of the proceedings of a workshop might be disseminated under two different views: (a) a structured textual view containing a Preface created by the conference chairs, and the list of thematic sessions containing the accepted papers, and (b) a presentation view, containing the list of the slides used in the presentations. The Manifestation models the physical formats by which an object is disseminated, e.g., the MPEG file containing the video recording of a lecture given at a certain summer school, or the AVI file of the same video. These entities are semantically connected by means of a set of possibly multiple relationships, i.e. there may be multiple editions of the same document, multiple views of the same edition and multiple manifestations of the same view. Each of the entities of the model has a set of attributes that specify the rights on the modelled document aspects. This enables, for example, to model possibly different rights on different editions, different access policies on different views or on different parts of the same view, and so on. The information objects of an OpenDLib digital library are organised into a set of virtual collections, each characterised by its own access policy. Authorised people can define new collections dynamically, i.e., by specifying membership criteria. In the same digital library, for example, it is possible to maintain a collection of grey literature accessible to all users and a collection of historical images accessible only to

5 a specific group of researchers. Each collection is automatically updated whenever a new object matching the membership criteria is published in the digital library. From the functional point of view, the basic release of OpenDLib provides services to support the submission, description, indexing, search, browsing, retrieval, access, preservation and visualization of information objects. These objects can be submitted as files in a chosen format or as URLs to objects stored elsewhere. They can be described using one or more metadata formats. The OpenDLib search service offers different search options: free text or fielded (where fields can be selected from a variety of known metadata formats); single or cross-language; with or without relevance feedback. Information objects retrieved can be navigated over all their editions, versions, structures, metadata and formats. All the above services can be customised according to several dimensions such as, for example, metadata formats, controlled vocabularies, and browsable fields. OpenDLib is also equipped with other digital library specific services, such as the ones providing the enforcement of access policies on information objects and the management of user-shelves able to maintain information objects versions, resultsets, session results, and other information. In addition, a number of administration functions are also given to support preservation of objects, objects reviewing process, and handling of users and user group profiles. From the architectural point of view, OpenDLib adopts a component-based approach. This application development approach is the most suitable to realise the OpenDLib goal because: (i) the system is assembled from discrete executable components which are developed and deployed somewhat independently of one another, potentially by different players; (ii) the system may be upgraded with smaller increments, i.e. by upgrading only some of the constituent components. In particular, this aspect is one of the key points to reach interoperability, as upgrading the appropriate constituents of a system makes it able to interact with other systems; (iii) components may be shared by systems; this creates opportunities for reuse that heavily contributes to lowering the development and maintenance costs and the time to market; and (iv) though not strictly related to their being component-based, component-based systems tend to be distributed. OpenDLib implements this approach by realising a Service-oriented Architecture so it consists of an open federation of services that can be distributed and replicated on different hosting servers. By exploiting this feature, OpenDLib supports three kinds of dynamic expansions: (i) new classes of services can easily be added to the federation; (ii) new instances of a replicated or distributed service can be deployed on either an existing or a new hosting server; (iii) the configurations of the services can be modified so that they can handle new object types, new metadata formats and support new usages. Thus the architectural configuration, chosen when the digital library is set up, can be changed later to satisfy new emerging needs. For instance, a replication of an Index instance can be created to reduce workloads when the number of search requests exceeds an established threshold, whereas an Index instance, able to serve queries in a language not previously supported can be added to satisfy the needs of a new community of users. All the above dynamic updates in the configuration of the federation can be done on the fly, i.e. without switching off the digital library. This architectural design provides a great flexibility in the management of the digital library, so that it can be managed and hosted either by a single or by a

6 multitude of organisations. For example, an organisation can decide to maintain an instance of the repository service, in order to have local control over its own documents, but to share all the other services with other institutions. The OpenDLib architecture has been designed to be highly interoperable with other digital libraries. In particular, an OpenDLib library can act both as an OAI-PMH data provider and as an OAI-PMH service provider (Lagoze and Van de Sompel, 2001). This implies that the metadata maintained by an OpenDLib digital library can be open to other digital libraries and, vice-versa, the OpenDLib services can access the metadata published by any other OAI-PMH compliant digital library. From an implementation point of view, OpenDLib is an HTTP (web) server-based software system, requiring only open source technologies and running on several of the major operating systems, such as Solaris, Linux, SCO, Digital Unix, Unix V, and AIX. OpenDLib is distributed as an Apache Server Module that contains all the modules belonging to the OpenDLib kernel. At activation time, it loads the set of self-installing modules representing DL Web Services in accordance with the configuration instructions. DL service modules are either function libraries, or object libraries used to define a class and its methods. The system adopts XML as a standard for internal (service to service) and external exchange of messages. An HTTP-based communication protocol, named OpenDLib Protocol (OLP), has been designed in order to regulate the communication among the services. The OLP messages can be sent as REST or SOAP requests while a set of APIs have been defined to facilitate its exploitation in other existing systems. C O N C L U S I O N S OpenDLib is now an operational system that has been used for building a number of DLs. Each of these DLs has its own specific distributed architectural configuration that reflects the needs of the application scenario where it operates. Thanks to its expandability OpenDLib proved to support the creation of very different DLs by dynamically and progressively aggregating resources provided by many disperse organizations. Recently OpenDLib has been extended. Its new version, OpenDLibG (Candela, Castelli, Pagano, & Simi, 2006) makes it able to exploit the storage and processing capabilities offered by a Grid Infrastructure. Thanks to this extension an OpenDLib digital library is now able to handle a much wider class of information objects, namely objects requiring huge storage capabilities like raw data and particular types of images, videos, and 3D objects. Further, it is also able to process on-demand such data to generate new information as the result of computational intensive elaborations that OpenDLib automatically distributes to exploit the available computational resources. This extension has confirmed the goodness of the system, its openness and the power and flexibility of the DoMDL.

7 R E F E R E N C E S Candela, L., Castelli, D., Pagano, P. and Simi, M. (2005). From Heterogeneous Information Spaces to Virtual Documents. In E.A. Fox, E.J. Neuhold, P. Premsmit, and V. Wuwongse (Eds.), Lecture Notes in Computer Science: Digital Libraries. Implementing Strategies and Sharing Experiences, 8th International Conference on Asian Digital Libraries, ICADL 2005 (pp ). Berlin: Springer- Verlag. Candela, L., Castelli, D., Pagano, P. and Simi, M. (2006). OpenDLibG: Extending OpenDLib by exploiting a glite Grid Infrastructure. Lecture Notes in Computer Science: Proceeding of the 10th European Conference on Research and Advanced Technology for Digital Libraries, ECDL Berlin: Springer-Verlag. Castelli, D. and Pagano, P. (2002). OpenDLib: A Digital Library Service System. In M. Agosti and C. Thanos (Eds.), Lecture Notes in Computer Science: Research and Advanced Technology for Digital Libraries, Sixth European Conference ECDL 2002 (pp ). Berlin: Springer-Verlag. Castelli, D. and Pagano, P. (2003). A System for Building Expandable Digital Libraries. In ACM/IEEE 2003 Joint Conference on Digital Libraries JCDL 2003 (pp ). Berlin: Springer- Verlag. EPrints for Digital Repositories (Accessed December 2007) IFLA Study Group on the Functional Requirements for Bibliographic Records. (1998). Functional Requirements for Bibliographic Records: Final Report. Muenche: Saur, K.G. ( Lagoze, C., Payette, S., Shin, E., and Wilper, C. Fedora: An Architecture for Complex Objects and their Relationships. International Journal on Digital Libraries, 2005, 6, Lagoze, C. and Van de Sompel, H. (2001) The open archives initiative: building a lowbarrier interoperability framework. In Proceedings of the 1st ACM/IEEE-CS Joint Conference on Digital Libraries (Roanoke, Virginia, United States). JCDL '01. ACM, New York, NY Lynch, C.A. (2003). Institutional Repositories: Essential Infrastructure for Scholarship in the Digital Age. ARL, 226, 1 7. Millington, P. and Nixon, W.J. EPrints 3 Pre-Launch Briefing. Ariadne 50 (2007) OpenDLib. A Digital Library Service System (Accessed December 2007). Payette, S. and Thornton, S. (2002). The Mellon Fedora Project: Digital Library Architecture Meets XML and Web Services. In M. Agosti and C. Thanos (Eds.), Lecture Notes in Computer Science: Research and Advanced Technology for Digital Libraries, Sixth European Conference ECDL 2002 (pp ). Berlin: Springer- Verlag. Tansley, R., Bass, M. and Smith, M. DSpace as an Open Archival Information System: Current Status and Future Directions. Research and Advanced Technology for Digital

8 Libraries, 7th European Conference, ECDL 2003, Trondheim, Norway, August 17-22, 2003, Proceedings, Springer-Verlag, 2003, Tansley, R., Bass, M., Stuve, D., Branschofsky, M., Chudnov, D., McClellan, G., and Smith, M. (2003). The DSpace Institutional Digital Repository System: current functionality. Proceedings of the third ACM/IEEE-CS joint conference on digital libraries (87 97). IEEE Computer Society. Terms and Definitions Digital Library Service System: A Digital Library System based on a Service Oriented Architecture providing the functionality required by a Digital Library in terms of distributed services. Digital Library System: A software system that is based on a (potentially distributed) architecture and provides all functionality that is required by a particular Digital Library. Users interact with a Digital Library through the corresponding Digital Library System. Document Model: The abstract representation of the physical or semantic structure of a given set of documents. Institutional Repository: A set of services that a research institution offers to the members of its community for the management and dissemination of digital materials created by the institution and its community members. It is most essentially an organizational commitment to the stewardship of these digital materials, including long-term preservation where appropriate, as well as organization and access or distribution. [C. A. Lynch] Service Oriented Architecture: A style of information systems architecture that enables the creation of applications that are built by combining loosely couplet and interoperable services. Grid Infrastructure: The underlying basic framework for Grid computing. Grid computing: computing model that provides the ability to perform higher throughput computing by taking advantage of many networked computers to model a virtual computer architecture that is able to distribute process execution across a parallel infrastructure.

9 Biographical Sketch Leonardo Candela Dr. Leonardo Candela is a researcher at Networked Multimedia Information Systems (NMIS) Laboratory of the Institute of Information Science and Technologies - Italian National Research Council (ISTI - CNR). He graduated in Computer Science in 2001 at University of Pisa and completed a Ph.D. in Information Engineering in 2006 at University of Pisa. In 2001 he joined the NMIS Laboratory and was involved in the CYCLADES ( and Open Archives Forum ( projects. He is currently involved in the DILIGENT ( and DRIVER ( projects and he is an active member of the DELOS ( working group on the Digital Library Reference Model ( He is member of the OAI-ORE ( Liaison Group. His research interests include Digital Library [Management] Systems and Architectures, Digital Libraries Models, Distributed Information Retrieval, and Grid Computing. Donatella Castelli Dr. Donatella Castelli is a Senior Researcher working at the Information Science and Technologies - Italian National Research Council (ISTI - CNR) since She graduated in Computer Science at the Department of Computer Science of the University of Pisa and there she was employed as researcher for two years before joining ISTI-CNR Networked Multimedia Information Systems. Since 1996 she has scientifically coordinated several European and National funded projects on digital libraries acquiring considerable experience in this domain. She is currently leading the activity of the DELOS Network of Excellence on Digital Libraries dedicated to the definition of a Reference Model for digital libraries. Her current research interests include digital library architectures and infrastructures, information object modeling and interoperability. Pasquale Pagano Dr. Pasquale Pagano is a Senior Research Associate of the Networked Multimedia Information Systems Laboratory of the Institute of Information Science and Technologies - Italian National Research Council (ISTI - CNR). He received his M.Sc. in Information Systems Technologies from the Department of Computer Science of the University of Pisa (1998), and his Ph.D. degree in Information Engineering from the Department of Information Engineering: Electronics, Information Theory, Telecommunications of the same university (2006). He has a strong background in digital library distributed architectures. He has participated in the design of the most relevant DL systems developed by CNR - ISTI. Currently Pasquale is the technical support manager of the "DILIGENT: A testbed DIgital Library Infrastructure on Grid ENabled Technology" project. He is also leading the development and deployment of the OpenDLib ( digital library service system in different application areas (libraries, archives, commercial publishing organizations). His current research interests include digital libraries services and architectures. Manuele Simi Manuele Simi is a Software Engineer and Developer. He received his B.Sc in Computer Science from the University of Pisa in He has been employed for several IT companies

10 as Software Engineer and Developer and coordinator of small teams. In 2002, he joined the Information Science and Technologies - Italian National Research Council (ISTI - CNR). His professional experience covers the design and implementation of reusable components, libraries, web services, GUIs and Web applications for different target platforms. He has worked extensively with Perl, C/C++ and Java and he is experienced in UML modelling. He has been involved in small projects as well as in very large projects with a significant experience in designing and implementing Service Oriented Architectures. At ISTI-CNR, he has worked for the Scholnet and DILIGENT projects and he was one of the main implementers of the OpenDLib ( digital library service system. He has published a number of articles in some Italian magazines dedicated to the software development field.