Natural Language Inspired Approach for Handwritten Text Line Detection in Legacy Documents

Transcription

1 Natural Language Inspired Approach for Handwritten Text Line Detection in Legacy Documents Vicente Bosch Alejandro Hector Toselli Enrique Vidal Pattern Recognition and Human Language Technologies group Universidad Politécnica de Valencia

2 Presentation Outline Introduction 1 Statistical Framework 2 Modelling 4 System Architecture 7 Experimental Setup and Results 12 Conclusions & Future Work 17 1

3 Introduction Document layout analysis (DLA) is an important task for any Document Retrieval Information System. Text line detection (TLD) is a DLA task that yields the physical locations. of text lines given an input page image; required for systems that require text line images as input. TLD in handwritten legacy documents is a difficult task due to: Handwritten text issues: variable inter-line spacing, overlapping and touching strokes of adjacent lines, etc Legacy document issues: smears, background variations, uneven illumination, humidity spots and bleed-through marks Current state of the art-methods are mostly based on heuristic approaches and require parameter tunning. We present a formal approach for text line detection in handwritten text based in machine-learning techniques. 2

4 Statistical Framework (1) We assume that the input image contains one or more paragraphs of single-column parallel text with no images or diagram figures which we mathematically represent as a sequence of observations o = o 1, o 2,..., o m. TLD problem can be formulated as the problem of finding the most likely text lines sequence, ĥ = h 1, h 2,..., h n, for a given input sequence o; which we can decompose using Bayes rule: ĥ = arg max h P (h o) = arg max h P (o h) P (h) 3

5 Statistical Framework (2) As the actual physical location (and not just the best label sequence) is required we rewrite the formula to obtain this: ĥ = arg max h P (o, b h) P (h) Which we can approximate by the dominant term, max b P (o, b h): (ˆb, b ĥ) arg max b,h P (h) P (o, b h) 4

6 Modelling (1) 5

7 Modelling (2) Two levels of modelling: Morphological: That model the page vertical regions, usually approximated by HMMs Syntactical: a Language Model (LM) that restrics how those vertical regions are composed to form an actual page, modelled as a stochastic finite state grammar (SFSG) Vertical region types: Blank Space (BS), Normal Line (NL), Inter Line (IL) and Non-Text Line LM allows us to force restricctions: NL+IL, NT+IL, BS, etc. Both modelling levels represented by finite-state automaton, can be integrated into a single global model on which our problem can be easily solved. 6

8 Modelling (3) HMM Sample: SFSG Sample: 7

9 System Overview Page Images Page layout corpus Preprocessing Cleaned Page Images Feature Extraction HMM Training LM Training Training Decoding Feature Vectors Off-line line HMMs LM Model Type label and Region position coordinates 8

10 Natural Language Inspired Approach for Handwritten Text Line Detection in Legacy Documents Valencia April 18, 2012 Preprocessing

11 Feature Extraction b0 b1 b2 b3 b4 b5 b6 b7 bf-2 bf-1 bf X1 XL h1 h2 h3 h4 h5 h6 h7 hf-2 hf-1 hf D 10

12 Feauture Extraction

13 Feauture Extraction 12

14 Corpus Description (1) Experiments are carried out with a corpus compiled from a XIX century Spanish manuscript identified as Cristo-Salvador (CS). Kindly provided by the Biblioteca Valenciana Digital (BiVaLDi). Small document composed of 53 colour images of text pages, scanned at 300 dpi and written by a single writer. We employ the so-called book partition: Training set is formed by the first 33 page images. Test set contains the 20 remaining pages. 13

15 Corpus Description (2) 14

16 Corpus Description (3) Number of: Training Test Total Pages Normal-text lines (NL) Blank Lines (BL) Non-text Lines (NT) Inter Lines (IL) Each page was annotated with a succession of reference labels (NL, NT, BL and IL). Vertical regions were delimited by executing standard methods for text line detection based on vertical projection profiles and manually verified/corrected. Labelling of the different regions was performed manually by an operator. 15

17 Evaluation Measures Quality of the text line detection was measured using the line error rate (LER). LER is performed by comparing the sequences of automatically obtained region labels with the corresponding reference label sequences. The LER is computed in the same way as the well known WER. 16

18 Experiments and Results Training and decoding parameters were empirically tuned. Three model languages where tested: Prior, Conditional and Line-number constrained LM LER(%) Prior 0.86 Conditional 0.70 LN-Constrained

19 Conclusions & Future Work We have presented a new approach for text line detection by using a statistical framework similar to that already employed in many topics of NLP. It avoids the traditional heuristics approaches usually adopted for this task. The proposed approach not only has up to par detection accuracy,with current state of the art solutions, but also yields baselines of better quality (visually closer to the actual line). Extend this approach not only to detect, but also to classify line-region types in order to determine for example titles, short lines, beginning and and end of paragraphs, etc. It is envisioned that the proposed stochastic framework serves as a cornerstone to implementing interactive approaches to line detection similar to those used for handwritten text transcription used in Multimodal interactive transcription systems. 18

20 Questions 19