Optical Layout Recognition (OLR)

Size: px

Start display at page:

Download "Optical Layout Recognition (OLR)"

Griffin Todd
5 years ago
Views:

1 Optical Layout Recognition (OLR) From unstructured to structured newspaper data Claus Gravenhorst, CCS Content Conversion Specialists GmbH ENP information day, Paris, November 27, 2014

2 Agenda About CCS General OLR-workflow for mass digitization Layout and structure analysis ENP OLR workflow Quality assurance Output METS/ALTO package Use of structural data Access and presentation

3 About CCS CCS Content Conversion Specialists GmbH (Hamburg), as technical project partner, will provide its expertise and docworks technology to set up and operate a mass digitization workflow for creating high quality structured content from 2 million scanned newspaper pages provided by 5 library partners Page volume: BNF=1.000 k, NLE=500 k, SUB HH=480 k, NLF=90 k, SBB=10 k The distributed OLR workflow enables the contribution of project partners (content providers) to the integrated quality assurance process CCS is also contributing to the specification of the ENMAP metadata model

4 General workflow for mass digitization Check in Check out Document UID Barcode Item Tracking Re-Scan QA+Correcti QA+Correcti + on on Correction Manual QA in-house near-shore off-shore multiple locations Scanner Robot- Book- Document- Microfilm- Scanning Z Metadata Image Metadata Conversion Imaging Layout Analysis OCR ISR Reject Condition Final Output Automated QA Database Repository Delivery QA random Manual QA in-house near-shore

Layout and structure analysis Layout analysis based on bottom up approach General rule system enables recognition of words, text lines, text blocks,

issue) - content page (a page that consists of content/text only) - illustration page (a page that has at least one illustration) - advertisement page

5 Layout and structure analysis Layout analysis based on bottom up approach General rule system enables recognition of words, text lines, text blocks, columns and classification of text blocks, illustrations, advertisements, tables and the following page types: - title page (the title page of an issue) - content page (a page that consists of content/text only) - illustration page (a page that has at least one illustration) - advertisement page (a page that contains adverts only) Structure analysis through classification of headlines and grouping of zones into articles (incl. article continuation)

6 ENP OLR workflow Conversion without scanning Digital Image Metadata Delivery Digital Object Return Material location Conversion facility Inspection / Automatic QA Reject Conversion MD Recording Doc Delivery

7 Possible conversion scenarios A) Conversion at library (on-site) B) Conversion off-shore at CCS data center, final QA at the library via internet transfer (remote QA solution) C) Conversion off-shore at CCS, final QA at the library by backup shipment

8 Scenario B Remote QA at library RQA QA Library Storage INPUT HDD HDD HDD dw Share POOL Internet Master OUTPUT METS ALTO Offshore CCS Storage dw Share POOL IN OUT

9 Quality CCS Automated markup and basic manual correction: - Headlines, illustrations, tables, captions, advertisements, etc. - Article segmentation and grouping of zones into articles (incl. Content Provider (Library) Recommended: - Zoning: correct classification of blocks as text or illustration - Article segmentation: correct identification of headlines/text blocks/captions - Grouping: correct grouping of blocks (text, illustration) to articles - Metadata: correct title, issue date and issue number Optional: - Page types: correct page types - Page numbers: correct page sequence - OCR: perform text correction of specific zones (e.g. headlines, captions)

10 Output METS/ALTO package METS/ALTO metadata schemas to describe the structured digital ouput object A newspaper issue processed in docworks is converted into one METS XML file. It reflects the whole physical and logical structure, manages all links to the image files and the related ALTO XML files. ALTO is based on a standardized page description schema and contains all information of a page (print space, margins, coordinates, OCR results). Benefits of structural markup: - better browsing and more precise text search - better access and display on tablet and mobile devices - automated article classification and clustering through data/text mining and linguistic technologies - user engagement for manual online text correction, article classification, annotation, building personal collections, etc. - sharing articles via social media platforms like Facebook, Twitter, etc. METS = Metadada Encoding and Transmission Standard ALTO = Analyzed Layout and Text Object

11 Access and Presentation (I) Sample presentation system (Veridian) Browse by date, title Text search Article hit list Word highlighting

12 Access and Presentation (II) Issue Table of contents

13 Access and Presentation (III) Text & image view User text correction Article clipping Print article Distribute via and social media platforms

14 Thank you for your attention!

TEI, METS and ALTO, why we need all of them. Günter Mühlberger University of Innsbruck Digitisation and Digital Preservation

TEI, METS and ALTO, why we need all of them Günter Mühlberger University of Innsbruck Digitisation and Digital Preservation Agenda Introduction Problem statement Proposed solution Starting point Mass digitisation