HTR Part II: Handwritten Text Recognition

Transcription

1 HTR Part II: Handwritten Text Recognition Preprocessing and Feature Extraction for Off-Line Continuous HTR Alejandro H. Toselli & PRHLT-Group Departamento de Sistemas Informáticos y Computación Universidad Politécnica de Valencia May 2nd A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 1 / 50 Outline Introduction: HTR Generalities Motivation Use of conventional OCR systems for Cursive HTR HTR Related Definitions Criteria to Classify HTR Systems HTR Approach based on ASR Technology HTR System Overview Preprocessing at Page Image Level Noise Reduction and Background Removal Skew Correction of Page Image Text Line Images Detection and Extraction Preprocessing at Text Line Image Level Correction/Normalization of Style Attributes Slope and Slant Corrections Size Normalization Feature Extraction Examples of Features Extraction used in HTR Features Extraction used by FKI Group Features Extraction used by RWTH-i6 Group Dimensionality Reduction A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 2 / 50

2 Introduction: HTR Generalities Motivation Motivation 1 Still large part of the currently handled information can be found in handwritten documents:... personal letters handwritten notes survey forms historical manuscripts 2 HTR has become an important issue for many industrial applications: Recognition of legal amounts handwritten in bank checks Signature verification Reading of postal addresses... A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 3 / 50 Introduction: HTR Generalities Use of conventional OCR systems for Cursive HTR Use of conventional OCRs for Cursive HTR OCR: characters are converted from machine-print/handwriting images to ASCII text. Types: For recognizing machine-printed words/characters. For recognizing handwritten words/characters. Word recognition is based on the previous detection and segmentation of their corresponding characters (successfully applied in machine-printed words). Characters/strokes segmentation can be a quite difficult task in cursive handwritten text. Fail to recognize cursive handwritten text: variety of handwriting styles, diversity of stroke shapes and positions, characters touching (or overlapping) each other, etc. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 4 / 50

3 Introduction: HTR Generalities Use of conventional OCR systems for Cursive HTR Use of conventional OCRs for Cursive HTR: Examples Easy Difficult A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 5 / 50 Introduction: HTR Generalities HTR Related Definitions HTR Related Definitions Handwritten Text Recognition: is the task of transforming language represented in its spatial form of handwritten graphical marks into its symbolic representation (ASCII). Text Interpretation: is the task to determine the meaning of a recognized text. Additionally: Writer Identification: is the task of determining the author of a text handwriting from a set of writers. Signature Verification: examination of a signature to determine whether the handwriting is genuine; i.e. whether this belongs to the person who signed it. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 6 / 50

4 Introduction: HTR Generalities Criteria to Classify HTR Systems Criteria to Classify HTR Systems 01 ON LINE Point Sequence Representation (digital pen, tablet, etc.) OFF LINE Bitmap (Image) Representation (Camara, scanner, video, etc.) Other criteria: With/Without Explicit Segmentation Writer Dependent/Independent Open/Closed Vocabulary A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 7 / 50 Introduction: HTR Generalities HTR Approach based on ASR Technology HTR Approach based on ASR Technology Off-Line Handwritten Text Recognition: Very difficult task, which still remains as an open research topic. Share many common points with Automatic Speech Recognition (ASR). ASR-Based Technology Off-Line Handwritten Text Recognition Technology is based on the current and already consolidated ASR Technology. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 8 / 50

5 Introduction: HTR Generalities HTR Approach based on ASR Technology Main Features of the ASR Technology The Current ASR Technology Based on continuous Hidden Markov Models (HMMs). No explicit segmentation of the input speech signal is required. Modeling of Different Perception Levels: Morphological, Lexical and Syntactical. Recognition/Decoding is carried out by searching within a space structured by the above-mentioned perception levels. Use of Stochastic Finite State Automatons (SFSA) to build such strutured search space: HMMs, Regular Language Models. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 9 / 50 HTR System Overview A generic Pattern Recognition System Usually, PR systems share a general structure of four building blocks: 1 Data Acquisition and Preprocessing 2 Feature Extraction Process and Selection (data representation) 3 Training Process 4 Recognition/Classification A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 10 / 50

6 HTR System Overview HTR System Overview Handwritten Text Images Transcriptions PREPROCESSING Normalized Text Lines FEATURE EXTRACTION LANGUAGE MODELING Features Vectors Sequence Lexical and Language Models TRAINING Test Set HMM Models RECOGNITION Words Sequence A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 11 / 50 Preprocessing at Page Image Level Preprocessing at Page Image Level Preprocessing steps commonly applied to handwritten text pages images: Noise Reduction: filtering out noise, which can be due to an intrinsic feature of the original image and/or the digitization process. Background Removal: aiming at removing everything that is not handwritten text. Binarization: transforming (mapping) scaled grey level images into a black-white images. Skew Correction: detection and correction of the page angle with respect to the horizontal direction. Text Lines Extraction: detection and extraction of text lines from page image. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 12 / 50

7 Preprocessing at Page Image Level Noise Reduction and Background Removal Noise Reduction and Background Removal Issues; Stains, smears, ink bleed through. Uneven illumination. contrast variation. Noisy backgroung. etc. Solution: Methods based mainly on (combination of) different thresholding approaches, connected components and image preprocessing filters. Popular thresholding methods: N. Otsu, A threshold selection method from grey level histogram, IEEE Trans. Syst. Man Cybern., vol. 9 no. 1, 1979, pp Global thresholding method. W. Niblack, An Introduction to Digital Image Processing, pp , Prentice Hall, Local thresholding method. J. Sauvola, M. Pietikäinen, Adaptive document image binarization, Pattern Recognition, Volume 33, Issue 2, February 2000, Pages , ISSN Local thresholding method.... A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 13 / 50 Preprocessing at Page Image Level Noise Reduction and Background Removal Noise Reduction and Background Removal:Example A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 14 / 50

8 Preprocessing at Page Image Level Skew Correction of Page Image Skew Correction of Page Image Issues: Introduced during the image scanning process. Defined as the angle betwwen the X-axis and the scanned text lines of the page. Solution: rotation transform when the skew angle is detected. Typical methods based on: Multiresolution, Eigenvalues, Connected components Hough transform Horizontal projection profiles Statistical mixture model (EM algorithm) etc. B. Gatos, N. Papamarkos, and C. Chamzas. Skew detection and text line position determination in digitized documents. Pattern Recognition, 30(9): , A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 15 / 50 Preprocessing at Page Image Level Skew Correction of Page Image Skew Correction of Page Image: Example A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 16 / 50

9 Preprocessing at Page Image Level Text Line Images Detection and Extraction Text Line Images Detection and Extraction There are two different issues with respect to the text lines in a given page image: Detection and Extraction. Typical methods for text line detection are based on: Multiresolution, Eigenvalues Hough transform Horizontal projection profiles ( + RLSA ) Statistical approaches: HMMs etc. Typical methods for text line detection are based on: Connected components Dynamic programming, Distance Maps (cost functions) etc. Laurence Likforman-Sulem, Abderrazak Zahour, and Bruno Taconet Text Line Segmentation of Historical Documents: A Survey. Int. J. Doc. Anal. Recognit. 9, 2 (April 2007), A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 17 / 50 Preprocessing at Page Image Level Text Line Images Detection and Extraction Text Line Images Detection and Extraction: Example A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 18 / 50

10 Preprocessing at Text Line Image Level Preprocessing at Text Line Image Level Style attributes of handwritten text to be corrected/normalized: Slope Angle: angle of the handwritten text line with respect to the horizontal direction. Slant Angle: angle of the handwritten text strokes with respect to the vertical direction. Height of the handwritten text strokes: which can vary according to the task and writer. Of particular interest is the relationship among the sizes of ascender letters (e.g.: b, l, t), descender letter (e.g.: p, q, j) and normal letters (e.g.: a, c, u). Characters Width: as the text height, characters width can vary according to the task and writer. Stroke Thickness: the use of different writing instruments can lead to a variable stroke type and thickness. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 19 / 50 Preprocessing at Text Line Image Level Correction/Normalization of Style Attributes Correction/Normalization of Style Attributes: Overview a Original Image b.1 b.2 b.3 Slope Correction b.4 b.5 c Slant Correction d Size Normalization A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 20 / 50

11 Preprocessing at Text Line Image Level Slope and Slant Corrections Slope and Slant Correction Rotation/shift transform when the slope/slant angle is detected. Methods based on: Dynamic Programming: Bertolami, R.; Uchida, S.; Zimmermann, M.; Bunke, H., Non-Uniform Slant Correction for Handwritten Text Line Recognition, Document Analysis and Recognition, ICDAR Ninth International Conference on, vol.1, no., pp.18,22, Sept Gabor filters: Das Gupta, J.; Chanda, B., Novel methods for slope and slant correction of off-line handwritten text word, Emerging Applications of Information Technology (EAIT), 2012 Third International Conference on, vol., no., pp.295,298, Nov Dec Machine learning techniques (MLP): España-Boquera, S.; Castro-Bleda, M.J.; Gorbe-Moya, J.; Zamora-Martinez, F., Improving Offline Handwritten Text Recognition with Hybrid HMM/ANN Models, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol.33, no.4, pp.767,779, April Edges detectors (Sobel filters) RLSA, projection profiles, linear regression (more or less heuristics) A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 21 / 50 Preprocessing at Text Line Image Level Slope and Slant Corrections Slope Correction: RLSA and Linear Regression Approach based on: Run-length Smearing Algorithm and word slope angles computed by linear regression technique. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 22 / 50

12 Preprocessing at Text Line Image Level Slope and Slant Corrections Slope Correction: Horizontal Projection Profile (HPP) HPP α (i) = cols j=1 I α(i, j) F(HPP α )=std(hpp α )= rows (HPP α HPP α (i)) 2 i=1 rows ˆα = arg max std(hpp α ) α [45:135] A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 23 / 50 Preprocessing at Text Line Image Level Slope and Slant Corrections Slant Correction: Vertical Projection Profile (VPP) (1) Shear transformation is applied on the image I for a wide range of angles α [45 : 135]. I α is a resulting image sheared by the angle α. For each sheared image the corresponding vertical projection profile is obtained: VPP α (j) = rows i=1 I α(i, j) The slant angle ˆα is that which maximizes: ˆα = arg max F(VPP α ) α [45:135] where F can be one of the following objective functions: Standard Deviation: F(VPPα ) = cols Profile Length: F(VPPα ) = cols 1 j=1 (VPP α VPP α (j)) 2 j=1 cols IDIAP: F(VPPα ) = j:c α (j)=1 VPP α(j) 2 : C α (j) = 1 + (VPP α (j) VPP α (j + 1)) 2 VPP α (j) max(i I(i,j)=1) min(i I(i,j)=1) A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 24 / 50

13 Preprocessing at Text Line Image Level Slope and Slant Corrections Slant Correction: Vertical Projection Profile (VPP) (2) Angle = -40 Angle = -20 Angle = 0 Angle = 20 A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 25 / 50 Preprocessing at Text Line Image Level Slope and Slant Corrections Slant Correction: Vertical Proj. Profile (VPP) (3) When the standard deviation of vertical projection profile histogram std (VPP α ), presents a multimodal distribution: std (VPPα) In these cases, a slant angle ᾱ is computed as the average value of the angles α whose std (VPP α ) ˆα(1 ρ): α std (VPP α ) ᾱ = α:α ˆα(1 ρ) α:α ˆα(1 ρ) α ρ [0, 1] std (VPP α ) A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 26 / 50

14 Preprocessing at Text Line Image Level Size Normalization Size Normalization Encompasses two different steps: text region detection and normalization. Into a text line image, there different areas are envisioned: Ascender Area Main Body Area Descender Area Text line mormalization approaches based on: Conventional techniques: U.-V. Marti and H. Bunke. Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition system. International Journal of Pattern Recognition and Artificial Intelligence, 15:6590, Machine learning techniques (MLP): España-Boquera, S.; Castro-Bleda, M.J.; Gorbe-Moya, J.; Zamora-Martinez, F., Improving Offline Handwritten Text Recognition with Hybrid HMM/ANN Models, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol.33, no.4, pp.767,779, April A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 27 / 50 Preprocessing at Text Line Image Level Size Normalization Size Normalization: Based on RLSA and Linear Regression 1 Orig. Image 2 RLSA 3 Upper-Border 4 Lower-Border 5 Linear-Rgrs. 6 Resizing A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 28 / 50

15 Feature Extraction Feature Extraction (FE): Generalities Establish a new representation space of input signal (image). FE should: be compact. minimize the intra-class variability. maximize the inter-class variability. be adequate to the kind of recognizer/classifier employed. be easy to obtained at low computational cost. The FE quality is indirectly evaluated through assessing the recognizer/classifier performance. There are several formal methods described in the literature for FE selection, although they are not widely used. One desirable FE property would be that the original signal could be rebuilt again (exactly or approximately) by using this FE. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 29 / 50 Feature Extraction Feature Extraction: Taxonomy (1) Global or Local Features. Extracted from statistical distribution of points: Advant.: simple, low computational cost and dimensionality. Desadv.: can not permit to rebuild the original signal. E.g.: vertical/horizontal projection profile, mean, standard deviation. Extracted from global transformations or series expansions: Advant.: allow to rebuild (totally or approximately) the original signal. Desadv.: generally involve a high computational cost. E.g.: Fourier Transforms, Wavelets Discrete Transforms, 2D Gabor Filters, Geometric Moments. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 30 / 50

16 Feature Extraction Feature Extraction: Taxonomy (2) Topological and geometrical features extraction: Advant.: fairly robust across different handwriting styles and distortions. Disadv.: complex implementations and computation times. E.g.: contour chains, geometric property measures and representations, primitives to build topological structures and graphs to define relations among them. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 31 / 50 Feature Extraction Feature Extraction: HMMs Feature Extraction suitable for being used with HMMs: Local feature extraction. Sequence of feature vectors of fixed dimensionality. Feature vectors extracted following the natural writing direction (from left to right). Based on geometric features. Allow to rebuild approximately the original (preprocessed) image. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 32 / 50

17 Feature Extraction Examples of Features Extraction used in HTR Comparative among PRHLT s, FKI s and RWTH s FEs PRHLT FKI RWHT-i6 (old) Based on Bazzi&Schwartz FKI RWTH Type Geometric Geometric Geometric Prev. Preproc. yes yes no Extracted from Local Window Column of Pixels Column of Pixels Vector dimens PCA(7 height img ) Size-Norm. Imag. no yes (height img ) yes (height img ) #Vectors/Image width img width img width img Issam Bazzi, Richard Schwartz y John Makhoul. An Omnifont Open-Vocabulary OCR System for English and Arabic. IEEE Trans. Pattern Analysis and Machine Intelligence. 21(6), (June 1999). Marti, U. and Bunke, H Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition systems. In Hidden Markov Models: Applications in Computer Vision World Scientific Series In Machine Perception And Artificial Intelligence Series, vol. 45. World Scientific Publishing Co., River Edge, NJ, P. Dreuw, D. Rybach, C. Gollan and H. Ney. Writer Adaptive Training and Writing Variant Model Refinement for Offline Arabic Handwriting Recognition. In 10th Int. Conf. on Document Analysis and Recognition (July 2009). A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 33 / 50 Feature Extraction Examples of Features Extraction used in HTR PRHLT FE Method: Sample Window N M Original Image Grid Sample Window A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 34 / 50

18 Feature Extraction Examples of Features Extraction used in HTR PRHLT FE Method: Grey Level Feature n m Stroke segment enclosed by the sampling window Sub-sampling points defined inside the sampling window and spread out over the stroke segment smoothed by the Gaussian filter The image intensity Î(i, j), smoothed by a Gaussian filter, is: [ Î(i, j) = I(i, j) exp 1 ( j (n/2) 2 2 (n/4) 2 + i )] (m/2)2 (m/4) 2 A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 35 / 50 Feature Extraction Examples of Features Extraction used in HTR PRHLT FE Method: Derivative Features n m Average number of pixels per column (ANPC): m I(i, j) i=1 g j = m Linear approximation: mse 0 mse(a, b) = n j=1 w j (g j (a j +b)) ANPR Gaussian filter: w j = exp ( ) 1 (j n/2) 2 2 (n/4) 2 ANPC Restriction: mse(a, b) a = 0 and mse(a, b) b = Derivative: a = ( n ) ( w n j g j w j j j=1 j=1 ( n ) w 2 ( n j j j=1 ) ( n j=1 w j j=1 w j ) ( n ) ( n j=1 w j j 2 ) j=1 w j g j j ) A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 36 / 50

19 Feature Extraction Examples of Features Extraction used in HTR PRHLT FE Method: Visual Representation (1) Features Vectors Sequence {}}{ Grey Level Horizontal Derivative Vertical Derivative A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 37 / 50 Feature Extraction Examples of Features Extraction used in HTR PRHLT FE Method: Visual Representation (2) ORIGINAL IMAGE NORMALIZED GREY LEVEL HORIZONTAL DERIVATIVE VERTICAL DERIVATE K = 16 3 K = 20 3 A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 38 / 50

20 Feature Extraction Features Extraction used by FKI Group FKI FE Method FE using one-column-width sliding window, moving along the image from left to right at one-column-step. Nine geometrical quantities are computed on each position: c 1 (j): Number of black pixels in the window. c 2 (j): Center of gravity of the window. c 3 (j): Second order moment of the window. c 4 (j): Position of the upper contour in the window. c 5 (j): Position of the lower contour in the window. c 6 (j): Orientation of the upper contour in the window. c 7 (j): Orientation of the lower contour in the window. c 8 (j): Number of black-white transitions in vertical direction. c 9 (j): Number of black pixels between the upper and lower contours. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 39 / 50 Feature Extraction Features Extraction used by FKI Group FKI FE Method: Mathematical Formulation Expressed in mathematical terms: blk: black pixel wht: white pixel m c 1 (j) = I(i, j) i=1 c 2 (j) = 1 m m i I(i, j) i=1 c 3 (j) = 1 m m 2 i 2 I(i, j) i=1 c 4 (j) = min(i I(i, j) = blk) c 5 (j) = max(i I(i, j) = blk) c 6 (j) = c 4(j + 1) c 4 (j 1) 2 c 7 (j) = c 5(j + 1) c 5 (j 1) 2 c 8 (j) = NT blk wht (I(i, j)) c 9 (j) = I(i, j) c 4 (j)<j<c 5 (j) A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 40 / 50

21 Feature Extraction Features Extraction used by FKI Group FKI FE Method: Vissual example j 1 j j+1 i= i=m=12 c 1 (j) = 5 c 2 (j) = = 7, 4 c 3 (j) = 4, , , , , c 4 (j) = 3 c 5 (j) = 11 c 6 (j) = 4 2 = c 7 (j) = = 0, 5 2 c 8 (j) = 3 c 9 (j) = 3 = 7, 44 A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 41 / 50 Feature Extraction Features Extraction used by RWTH-i6 Group RWTH-i6 FE Method (OLD) Using one-column-width sliding window, moving along the image from left to right at one-column-step: 1 Column of image-pixels X j at every column-step: j = 1,..., W img. 2 Augmented with their spatial derivatives: = X j X j 1. Thereby, features on each point j: Features(j) = (X j j ). 3 To incorporate spatial context, 7 consecutive columns-features in a sliding window are concatenated: (X j 3 j 3 )(X j 2 j 2 )... (X j+3 j+3 ). 4 Finally, PCA transformation matrix is applied to reduce the high features dimensionality. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 42 / 50

22 Feature Extraction Features Extraction used by RWTH-i6 Group RWTH-i6 FE Method (New) Employ a normaization scheme of handwritten test line image attributes based on image moments. Kozielski, M.; Forster, J.; Ney, H., Moment-Based Image Normalization for Handwritten Text Recognition, Frontiers in Handwriting Recognition (ICFHR), 2012 International Conference on, pp.256,261, Sept The algorithm operates directly on grey-scale images Image gradient and zero-th order moments to globally normalize the stroke thickness of a pattern. The image is segmented into slices using a sliding window, where size and shift of the sliding window are estimated using moments. Local variability in size and position is modelled independently in separate slices using second-order moments. The final feature vector is subject to PCA transformation and its number of components is reduced to 30. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 43 / 50 Feature Extraction Dimensionality Reduction Dimensionality Reduction for real-value FE Vectors Objective: Aiming at reducing the recognition time without significant loss of accuracy. Applied on real-value (feature) vectors. Filter out noise. Discard redundant features. Among the most employed dimensionality reduction methods, we have: PCA: Principal Analysis Components LDA: Linear Discriminant Analysis A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 44 / 50

23 Feature Extraction Dimensionality Reduction PCA: Principal Analysis Components Aim at: reducing the data dimensionality (of sequence feature vectors). characterizing the main trends exhibited by data set. Method: based on covariance analysis between components (or factors). Characteristic: it does not require any previous knowledge about the class label of each of vector (unsupervised method). Formalization: Orthogonal linear transformation which projects the data onto a new coordinate system, such that the greatest variance by any projection of the data comes to lie on the first coordinate (called the first principal component), the second greatest variance on the second coordinate, and so on. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 45 / 50 Feature Extraction Dimensionality Reduction PCA: Example (1) Two-class labeled points onto 2D-real-space: µ A = (25 25) ( ) 32 6 σ A = 6 2 µ B = ( ) ( ) 32 6 σ B = 6 2 A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 46 / 50

24 Feature Extraction Dimensionality Reduction PCA: Example (2) Two-class labeled points onto 2D-real-space: µ A = (25 25) ( ) 32 6 σ A = 6 2 µ B = ( ) ( ) 32 6 σ B = 6 2 A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 47 / 50 Feature Extraction Dimensionality Reduction LDA: Linear Discriminant Analysis Objective: to project dataset on a space of small dimension such that its class information is maximally preserved. Method: based on covariance analysis between-class and within-class of components (or factors). Characteristic: requires a previous knowledge about the class label of each vector (supervised method). Formalization: as PCA, LDA involves an orthogonal linear transformation which projects the input data onto a new coordinates system, such the projected data have maximal between-class-variance and minimal within-class-variance. A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 48 / 50

25 Feature Extraction Dimensionality Reduction LDA: Example of the PCA case Two-class labeled points onto 2D-real-space: PCA Projection µ A = (25 25) ( ) 32 6 σ A = 6 2 µ B = ( ) ( ) 32 6 σ B = 6 2 A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 49 / 50 Feature Extraction Dimensionality Reduction LDA: Example of the LDA case Two-class labeled points onto 2D-real-space: LDA projection µ A = (25 25) ( ) 32 6 σ A = 6 2 µ B = ( ) ( ) 32 6 σ B = 6 2 A. H. Toselli & PRHLT-Group (DSIC - UPV) Preproc. and FE for Off-Line HTR 50 / 50