PASIG Directions & Issues

Transcription

1 PASIG Directions & Issues Richard Boulderstone Director estrategy & Programmes April 2011

2 PASIG June 2007: Concluding comments The British Library has a substantial long term vision for electronic resources We believe that we have the correct processes & design principles to build the capability we need We have made good initial progress although not without setbacks We have designed a highly scalable, fault tolerant, redundant, self-validating architecture We are seeking partners to collaborate in the more complex areas of the architecture, such as Digital Preservation, Digital Rights Management, Resource Discovery, Trusted Repositories, Metadata Strategies, Dataset Management and Preservation, 2

3 Roadmap for introducing IT capabilities to support Library s business goals (BL IT Strategy )

4 Web archive toolset Rights Enforcement Delivery channels Customer facing services Discovery Metadata management Acquisition, Description Fulfilment Preservation Organisational management Reading rooms Physical content Physical access rules (ABRS) ILS (Aleph) PCMS (Aleph) NSB PC ASB (WMS & BL gateway) Mobile devices & App stores Research environments (RIC, MBS) Customer self service (CMI) Inventory mgmt (Aleph, DLS) Articles (ETOC/ CAS) ecommerce platform Resource Discovery (Primo, Themed Collections) Themed Collections Digital content Identifier assignment & resolution (Aleph, IAMS, DLS, ARKs, Resolver service) IAMS (IAMS) Payments (Oracle, Netbanx, EPOS) B2B ordering Content creation (digitisation, publishing, etc) Finance (Oracle) Usage data Marketing, events & exhibitions Order fulfilment (ABRS, DSS IRMDS, DLS) Non-Aleph material (Excel, Access) MI & BI, Records mgmt HR applications (Northgate, etc) Team collaboration Digital library storage Customer Management (CMI) DIPS Web site Multimedia search IT service management (Helpdesk, SR) Data sharing Technical metadata mgmt (DLS) Digital ingest (DLS) Planets toolset Digital rights Enterprise storage Estates security risk Enterprise IT (Office, Exchange, etc) The definition of a long term target IT architecture provides strategic direction for the evolution of our systems infrastructure. However, delivery of the architecture is achieved through business-led programmes. Therefore, investment priorities will be determined by business need. In this context, the execution and delivery of our business-led programmes will determine the development of the IT architecture over the period of the strategy. Focus on continuous improvement Requires extension to achieve aspirations Requires major investment to achieve aspirations

5 Digital Library System Principles Security Layered Security Provable Authenticity Single Point of Failure Preservation Intolerant of bit loss Never be down to your last copy Preserve Metadata Self checking, self healing Implementation Vendor Independence Commodity Hardware Purchased as needed Scalable Highly Scalable Extensible Software Transactional and re-entrant 5 5

6 DLS Network 4 Nodes (Complete Copies) British Library, St Pancras British Library, Boston Spa National Library of Wales National Library of Scotland NLS Node Access Gateway Additional Access Points BSp Bodleian Library, Oxford Cambridge University Library TCD JANET Trinity College Library, Dublin NLW Ox Ca StP 6 6

7 DLS Architecture DSE 1 Web Access SQL Gateway Site Firewall Site Firewall Gateway Web Access MI DB MI Aggregator SQL SQL Management Site Temp ROT Store Upload Management Site Temp ROT Store Upload LDL Access DB JHOVE Source Working Area Library Management Domain Site DB Domain Site DB SQL Born Brightsolid WAP Newspapers ejournals ESTAR Mailroom DB1 BizTalk 1 Mailroom 1 Boston Spa JANET National Library of Wales DSE 2 Web Access Gateway Site Firewall Site Firewall Gateway Web Access SQL SQL Management Site Temp ROT Store Upload Management Site Temp ROT Store Upload LDL Access DB JHOVE LeadTools Source Working Area Library Management Domain Site DB Domain Site DB SQL JISC 1 JISC 2 SQL JISC Catalogue DB Sound Archive BizTalk 2 Mailroom 2 Mailroom DB2 St Pancras s s National Library of Scotland 7

8 Replication Ingest into BSP BSP notifies all sites All sites update holdings BSP SCO LON BSP SCO LON NLS and LON request object from BSP BSP replicates to NLS NLS WAL BSP WAL Ingest NLS notifies all sites Key All sites update holdings Notify BSP replicates to LON LON notifies all sites Request Data All sites update holdings NLW requests object from LON LON replicates to NLW BSP SCO LON BSP SCO LON NLW notifies all sites WAL WAL All sites update holdings NLW LON 8 8

9 Cryptographic Services Every object signed as part of Ingest Digital Provenance Detect corruption Detect tamper Inter-site Trust Sites have no trust model between them Signature verified on receipt Every object checked on a regular basis 9 9

10 Digital Library System - Ingest Store now contains: 1.25 Million Items 140 Terabytes of Content A Terabyte is 1,000,000 Megabytes or 10^12 bytes 10

11 Object Size Distribution 11 11

12 Issues - Content Content ejournals Huge Range of Formats (PDF Still Dominant) Large Number of Publishers (A Few Large, Many Small) Supplementary Information Updates Open Access Images (Digitisation) Format Standards (Now using JPEG 2000) Resolution Debate (How Many Megabytes / Page?) Viewer Web Archives Legal Deposit Legislation Domain Crawl Size Automated vs. Manual Selection Many Others Datasets (Selection, Curation, Size, etc) 12

13 Issues Metadata & Preservation Metadata Use Cases (Access & Preservation) Persistent Identifiers (Datacite Service) Level of Granularity Synchronisation Across Stores Standards (METS, MODS, etc) Schema vs. RDF Access vs. Preservation Design Considerations External References Preservation Proprietary File Types Dynamic Content Warc Files Image Errors 13

14 Issues Access & Infrastructure Access API Definitions to Support Discovery & Navigation Of Rights Controlled Content Authentication Rights Discovery Metadata Image, Video, Audio, Document, Edit Viewer for Range of Content Types Infrastructure Buy / Build / Open Source Modularity of Components Scalability & Security are never solved 14

15 Digitisation: Post Processing Errors 15

16 PASIG April June 2011: 2007: Concluding comments The British Library has a substantial long term vision for electronic resources We believe that we have the correct processes & design principles to build the capability we need We have made good initial progress although not without setbacks We have designed a highly scalable, fault tolerant, redundant, self-validating architecture We are seeking partners to collaborate in the more complex areas of the architecture, such as Digital Preservation, Digital Rights Management, Resource Discovery, Trusted Repositories, Metadata Strategies, Dataset Management and Preservation, 16