How to Build a Digital Library

Transcription

1

2 How to Build a Digital Library

3 The Morgan Kaufmann Series in Multimedia Information and Systems Series Editor, Edward A. Fox, Virginia Polytechnic University How to Build a Digital Library Ian H. Witten and David Bainbridge Digital Watermarking Ingemar J. Cox, Matthew L. Miller, and Jeffrey A. Bloom Readings in Multimedia Computing and Networking Edited by Kevin Jeffay and HongJiang Zhang Introduction to Data Compression, Second Edition Khalid Sayood Multimedia Servers: Applications, Environments, and Design Dinkar Sitaram and Asit Dan Managing Gigabytes: Compressing and Indexing Documents and Images, Second Edition Ian H. Witten, Alistair Moffat, and Timothy C. Bell Digital Compression for Multimedia: Principles and Standards Jerry D. Gibson, Toby Berger, Tom Lookabaugh, Dave Lindbergh, and Richard L. Baker Practical Digital Libraries: Books, Bytes, and Bucks Michael Lesk Readings in Information Retrieval Edited by Karen Sparck Jones and Peter Willett

4 Documents are the digitalibrary s building blocks. It is time to step down from our high-level discussion of digitalibraries whathey are, how they are organized, and whathey look like to nity-grity details of how to representhe documents theycontain. To do a thorough job we wil have to descend even further and look athe representation of the characters that make up textual documents and the fonts in which those characters are portrayed. For audio, images and video we examine the interplay between signal quantization, sampling rate and internal redundancy that underlies multimedia representations.documents are the digitalibrary s building blocks. It is time to step down from our high-level discussion of dig Documents are the digitalibrary s building blocks. It is time to step down from our high-level discussion of digitalibraries whathey are, how they are organized, and whathey look like to nity-grity details of how to representhe documents they contain. To do a thorough job we wil have to descend even further and look athe representation of the characters that make up textual documents and the fonts in which those characters are portrayed. For audio, images and video we examine the interplay between signal quantization, sampling rate and internal redundancy that underlies multimedia representations.documents are the digitalibrary s building blocks. It is time to step down from our high-level discussion of dig Documents are the digitalibrary s building blocks. It is time to step down from our high-level discussion of digitalibraries whathey are, how they are organized, and whathey look like to nity-grity details of how How to Build a to representhe documents they contain. To do a thorough job we wil have to descend even further and look athe representation of the characters that make up textual documents and the fonts in which those characters are portrayed. For audio, images and video we examine the interplay between signal quantization, sampling rate and internal redundancy that underlies multimedia representations.documents are the digitalibrary s building blocks. It is time step down from our high-level discussion of dig Documents are the digitalibrary s building blocks. It is time to step down from our high-level discussion of digitalibraries whathey are, how they are organized, and whathey look like to nity-grity details of how to representhe documents they contain. To do a thorough job we wil have descend even further and look athe representation of the characters that make up textual documents and the fontsin which those characters are portrayed. For audio, images and video we examine the interplay between signal quantization, sampling rate and internal redundancy that underlies multimedia representations.documents are the digitalibrary s building blocks. It is time to step down from our high-level discussion of dig Documents are the digitalibrary s building blocks. It is time to step down from our high-level discussion of digitalibraries what they are, how they are organized, and what they look like to nity-grity details of how to represent the documents they contain. To do a thorough job we wil have to descend even further and look at the representation of the characters that make up textual documents and the fonts in which those Digital Library Ian H. Witten Computer Science Department University of Waikato David Bainbridge Computer Science Department University of Waikato

5 Publishing Director Assistant Publishing Services Manager Senior Developmental Editor Editorial Assistant Project Management Cover Design Text Design Composition Copyeditor Proofreader Indexer Printer Diane D. Cerra Edward Wade Marilyn Uffner Alan Mona Buehler Yonie Overton Frances Baca Design Mark Ong, Side by Side Studios Susan Riley, Side by Side Studios Carol Leyba Ken DellaPenta Steve Rath The Maple-Vail Book Manufacturing Group Designations used by companies to distinguish their products are often claimed as trademarks or registered trademarks. In all instances in which Morgan Kaufmann Publishers is aware of a claim, the product names appear in initial capital or all capital letters. Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration. Morgan Kaufmann Publishers An imprint of Elsevier Science 340 Pine Street, Sixth Floor San Francisco, CA by Elsevier Science (USA) All rights reserved. Printed in the United States of America No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means electronic, mechanical, photocopying, or otherwise without the prior written permission of the publisher. Library of Congress Control Number: ISBN: This book is printed on acid-free paper.

6 Chapter 1 Orientation... 1 Example One: Supporting human development... 1 Example Two: Pushing on the frontiers of science... 2 Example Three: Preserving a traditional culture... 3 Example Four: Exploring popular music... 4 The scope of digital libraries... 5 Libraries and digital libraries... 5 The changing face of libraries... 8 In the beginning The information explosion The Alexandrian principle Early technodreams The library catalog The changing nature of books Digital libraries in developing countries Disseminating humanitarian information Disaster relief Preserving indigenous culture Locally produced information The technological infrastructure The Greenstone software The pen is mighty: Wield it wisely Copyright Collecting from the Web Illegal and harmful material Cultural sensitivity Notes and sources Chapter 2 Preliminaries Sources of material Ideology Converting an existing library Building a new collection Virtual libraries Bibliographic organization Objectives of a bibliographic system Bibliographic entities Modes of access Digitizing documents Scanning Optical character recognition Interactive OCR Page handling Planning an image digitization project... 68

7 Inside an OCR shop An example project Notes and sources Chapter 3 Presentation Presenting documents Hierarchically structured documents Plain, unstructured text documents Page images Page images and extracted text Audio and photographic images Video Music Foreign languages Presenting metadata Searching Types of query Case-folding and stemming Phrase searching Different query interfaces Browsing Browsing alphabetical lists Ordering lists of words in Chinese Browsing by date Hierarchical classification structures Phrase browsing A phrase browsing interface Key phrases Browsing using extracted metadata Acronyms Language identification Notes and sources Collections Metadata Searching Browsing Chapter 4 Documents Representing characters Unicode The Unicode character set Composite and combining characters Unicode character encodings Hindi and related scripts Using Unicode in a digital library

8 Representing documents Plain text Indexing Word segmentation Page description languages: PostScript and PDF PostScript Fonts Text extraction Using PostScript in a digital library Portable Document Format: PDF PDF and PostScript Word-processor documents Rich Text Format Native Word formats LaTeX format Representing images Lossless image compression: GIF and PNG Lossy image compression: JPEG Progressive refinement Representing audio and video Multimedia compression: MPEG MPEG video MPEG audio Mixing media Other multimedia formats Using multimedia in a digital library Notes and sources Chapter 5 Markup and metadata Hypertext markup language: HTML Basic HTML Using HTML in a digital library Extensible markup language: XML Development of markup and stylesheet languages The XML metalanguage Parsing XML Using XML in a digital library Presenting marked-up documents Cascading style sheets: CSS Extensible stylesheet language: XSL Bibliographic metadata MARC Dublin Core BibTeX Refer Metadata for images and multimedia

9 Image metadata: TIFF Multimedia metadata: MPEG Extracting metadata Extracting document metadata Generic entity extraction Bibliographic references Language identification Acronym extraction Key-phrase extraction Phrase hierarchies Notes and sources Chapter 6 Construction Why Greenstone? What it does How to use it Using the Collector Creating a new collection Working with existing collections Document formats Building collections manually: A walkthrough Getting started Making a framework for the collection Importing the documents Building the indexes Installing the collection Importing and building Files and directories Object identifiers Plug-ins The import process The build process Greenstone archive documents Document metadata Inside the documents Collection configuration file Default configuration file Subcollections and supercollections Getting the most out of your documents Plug-ins Classifiers Format statements Building collections graphically Notes and sources

10 Chapter 7 Delivery Processes and protocols Processes The null protocol implementation The CORBA protocol implementation Preliminaries The macro language The collection information database Responding to user requests Performing a search Retrieving a document Browsing a hierarchical classifier Generating the home page Using the protocol Actions Operational aspects Configuring the receptionist Configuring the site Notes and sources Chapter 8 Interoperability More markup Names Links Types Resource description Collection-level metadata Document exchange Open ebook Query languages Common command language XML Query Protocols Z Supporting the Z39.50 protocol The Open Archives Initiative Supporting the OAI protocol Research protocols Dienst Simple digital library interoperability protocol Translating between protocols Discussion Notes and sources