Feature Extraction and Selection for Handwriting Identification: A review

Transcription

1 Feature Extraction and Selection for Handwriting Identification: A review ABSTRACT Handwriting is a skill that is personal to individual [28]. The relation of character, shape and the style of writing are visually different from one to another. Handwriting identification is a process to identify or verify the authorship of a handwriting document. Asserting authorship identity based on handwritten text requires three steps: Data acquisition and preprocessing, Feature extraction, and Classification. In the first step, the handwriting images is preprocessed and normalized to perform handwriting identification correctly. The second step involves extracting relevant and informative features. The third step implements the classification decision. Handwriting features are writer s characteristics to individuality. There are two different approaches to obtaining features: feature extraction and feature selection. In feature extraction the features that may have discriminating power were extracted, while in feature selection, a subset of the original set of features is selected. This paper concentrated on the state of art of feature extraction and selection approaches based on character, word, text line, and paragraph. Despite continuous effort, handwriting identification remains a challenging issue, due to different approaches use different varieties of features, having different. Therefore, our study will focus on handwriting identification based on feature selection to simplify features extracting task, optimize classification system complexity, reduce running time and improve the classification accuracy. Keywords Features extraction, Features selection, Handwriting identification 1. INTRODUCTION Handwriting is a skill that is personal to individual [28]. The relation of character, shape and the styles of writing are visually different from one to another, see Figure 1[10]. Handwriting identification is a process to identify or verify the writer of a handwriting document. Writer identification is the process of identifying the author from a group of writers. On the other hand writer verification is the task of determining whether two handwriting samples were written by the same or by different writers. Asserting authorship identity based on handwritten text requires three traditional steps: Data acquisition and preprocessing, Feature extraction, and Classification, Figure 2. First the handwritten text is preprocessed and normalized to perform handwriting identification correctly. Second involves obtaining a relevant and informative set of features. The features are based on the philosophy that feature sets can be designed to extract certain types of information from the image [4]. Figure 1. Samples of Handwriting style of four words by three writers, each of whom wrote the same word three times. Third involves making the classification decision. There are several classifiers available, such as Nearest Neighbor (NN), Neural Networks, Decision Trees and Support Vector Machines (SVMs). The performance of a classifier relies on the quality of the features. 375

2 Figure 2. Block diagram of writer authentication process The main issue in handwriting identification is how to get the features that reflect the varieties of handwriting. Many approaches have been proposed to extract features from handwritten characters, words, text line, and paragraph levels. A good set of features should represent characteristics that are particular for one class and be as invariant as possible to changes within this class. This paper is organized as: section 2 describes the handwriting feature extraction methods for character, word, text line, and paragraph. In section 3, handwriting subset features selection issues are presented. Section 4 concludes for literatures and analysis done or future expects for handwriting identification. 2. Features Extraction Handwriting features are writer s characteristics to individuality. Features that are shared by many writers are called inter-class characteristics. Individual features are those that are unique to a writer. Feature extraction is crucially important step in the construction of any pattern classification and it aims at the extraction of the relevant information that characterizes each class. The Handwriting extracted features can be classified as: geometric, directional, global, and local features. Such features as are variably used from one task to another (identification and verification). In this section of the study, we present the feature extraction methods for handwriting identification. 2.1 Character level Features Based on manual examination by the experts, handwriting features can be categorized as elements of style and execution. Huber and Headrick [1] have identified 21 elements of such features: Elements of Style(Arrangement, Class of Allograph, Connections, Allograph Construction, Dimensions, Slant or Slope, Spacing), Elements of Execution(Abbreviations, Commencements and Terminations, Diacritics and Punctuation,, Line Continuity, Line Quality, Pen Control, Writing Movement) and, Attributes of All Writing Habits (Consistency or Natural Variation, Persistency, Lateral Expansion, Word Proportions). Computerized handwriting identification consist of two type of handwriting features macro and micro [2, 3]. Macro features are gray-scale based (entropy, threshold, no. of black pixels), contour based (external and internal contours), slope-based (horizontal, positive, vertical and negative), stroke-width, slant and height. Micro-features are gradient-based features consist of 512 bits corresponding to gradient (192 bits), structural (192 bits), and concavity (128 bits), see Figure 3[4]. The gradient features capture the stroke flow orientation and its variations using the frequency of the gradient directions. The structural features representing the coarser shape of the character capture the presence of corners, diagonal lines, and vertical and horizontal lines in the gradient image. The concavity features capture the major topological and geometrical features including direction of bays, presence of holes, and large vertical and horizontal strokes. In [2, 3] micro features (GSC) are found to be discriminating for the writers. Furthermore, they found G, b, N, I, K, J, W, D, h, f are the 10 most discriminating characters. More information about GSC can be found in [4]. Figure 3. GSC features maps Size and shape of the characters have large amount of individuality information associated [1-3, 36]. Visually, handwriting text has different appearances and physical structures that reflect the differences of the handwriting. Leedham et al. [5] have extracted 11 globule- features (macrofeatures) for the authorship identification of handwritten documents containing digits. The features are: aspect ratio, number of end points, number of junctions, shape size and number of loops, width and height distributions, slant, shape, average curvature, and gradient features. The Hamming distance is used as classifier after these features were appropriately binarized. Figure 4. An example of loop shape, junction point and end branches Distribution of directions of the strokes has ability to characterize the writer. Direction based features have lots of individuality information associated with them (slant, curvature, regularity). Wang et al. [7] have analyzed the strength of the directional features for Chinese character images. First, directional element features (DEFs) are extracted. Then Component Analysis (PCA) and Linear Discriminate Analysis 376

3 (LDA) are used to reduce the dimensionality of the feature vector. Euclidean distance classifier is used to assign writer authorship. Pervouchine et al. [6] also present a method based on 25 structural features extracted from the skeleton of word th. Three categories of features were identified: indispensable, partially relevant and irrelevant. A neural network was used as a classifier and a genetic algorithm was used to search for optimal feature sets. The height of the character is found to be relevant (more discriminating) while width and pseudo-pressure information was not found as relevant(less discriminating). Except these set of features, which are extracted from character image (binary or grey level), researchers have also designed algorithm for features extraction for online handwriting. Online handwriting has extra temporal information about the sequence and the structure of the strokes in the character and thus possess more individuality information about the writer [38]. Nakamura and Kidode[8] have analyzed the individuality of the characters (Kanji). They extracted a set of feature in terms of character shape and writing behavior. The features directly correspond to the conventional features such as terminations, beginning. Pen pressure and pen inclinations were found to be more discriminating than other features. Figure 5. An example of some geometric features extracted from grapheme th [6] 2.2 Word level features The GSC feature extractor was further extended to adapt to handwritten word images for purposes of studying handwriting individuality [10]. For a word image, the corresponding GSC features contain 384-bit gradient features, 384-bit structural features, and 256-bit concavity features, leading to a binary vector of 1024 dimensions. Tomai et al. [11] have used four different types of feature extraction methods: Gradient, Structural and Concavity (GSC), Word Model Recognizer (WMR), Shape Curvature (SC), and Shape Context (SCON). WMR extracts the features that capture the distribution of the directions in the each segment of the word. SC extracts the features that represent the character curvature information. SCON extracts the features that measure the similarity between character contour shapes. Another set of features which has also been extracted is morphological features. Morphological features capture information about geometrical and structural properties of the words. Zois and Anastassopoulos [12] proposed a method for writer identification and verification using the features obtained from the morphological transformation of a thinned single word images. After image thresholding and curve thinning, the horizontal projection profiles are resampled, divided into 10 segments, and processed morphologically to obtain 20- dimensional feature vectors. Classification is performed using Bayesian classifier and multilayer perceptron (ANN). 2.3 Line level Features Hertel and Bunke [13] proposed a system based on text lines for writer identification. 100 features are extracted using the height of the three main writing zones, slant and character width, the distances between connected components, the blobs enclosed inside ink loops, the upper/lower contours, and the thinned trace processed using dilation operations. The features are subsequently used in a k-nearest-neighbor classifier that compares the feature vector extracted from a given input text to a number of prototype vectors coming from writers with known identity. Good results were reported when using all 100 features to determine the identity of a handwritten text line. Figure 6: Handwriting writing zones 2.4 Texture Analysis Image features are computed also at the document levels. Said et al. [14] treat the writer identification task as a texture analysis problem. They propose a text-independent approach using textural features derived from the grey-level co-occurrence matrix and Gabor filter techniques. Weighted approach Euclidean distance and Gabor features achieved 96 percent writer identification accuracy. Cha and Srihari [15, 16] take two pages of handwritten text as input and determine if they have been produced by the same writer. The features used to characterize a page of text include 377

4 writing slant and skew, character height, stroke width, frequency of loops and blobs, and others. In [17], this approach is extended to Chinese handwriting. Bulacu et al. [18] propose a texture-level approach using edgebased probability distribution functions PDFs as features for text-independent writer identification task. Edge-hinge distribution introduced as a new feature. The key idea behind this feature is to consider two edg-fragments in the neighborhood of a pixel and compute the joint probability distribution of the orientations of the two fragments. Bensefia et al. [19-20] use graphemes (sub or supra-allographic fragments) to address the writer verification task based on text blocks as well as on handwritten words. Graphemes generated by a handwriting segmentation method to encode the individual characteristics of handwriting independent of the text content. It is shown that each handwriting can be characterized by a set of invariant features, called the writer s invariants. Fractal features Srihari [3] 2001 Grey scale features P Offline Bensefia [18] 2002 Directional features P Offline Zhang [2] 2003 GSC features C Offline Leedham [5] 2003 Structural features C Offline Wang [7] 2003 Directional features C Offline Zhang [10] 2003 GSC W Offline Nakamura [8] 2005 Pen-input features C On-line Pervouchine [ 6] 2006 Structural features C Offline Bulacu [42] 2007 Directional features P Offline Neils [51] 2008 Allograph prototype Offline C matching Discrete character Chan [50] 2008 prototype C On-line distribution Tan [52] 2009 Continuous character prototype distribution C On-line (C, W, L, and P respectively means Character, Word, Line, Paragraph) Figure 7. Samples of invariant handwriting From the information retrieval perspective [22, 23] have proved that handwritten documents can be analyzed for their textual and graphical content, i.e. handwriting applications fall into the problem of information retrieval. Such application, one can seek for example to retrieve the documents from the database that contain certain calligraphy corresponding to specific writers. Other possible applications concern the detection of the various handwritings present in a document, or the dating of the documents compared to the chronology of the work of the author. Schomaker and Bulacu [24] introduce approach of connectedcomponent contours (CO3) for upper-case handwriting. This method is extended for off-line writer identification in [25], using the contours of fragmented connected components (FCO3) in mixed-style handwritten samples of limited size. Fractal features are useful to characterize certain handwriting styles. Seropian et al [26, 27] propose fractal analysis of handwriting. From a handwritten text line, a set of invariant features are extracted and used as a reference base in order to analyze an unknown writing by measuring the similarity. Table 1. Extracted Feature for handwriting identification. Author Year Feature Feature Domain Description Level Said [40] 1998 Gabor filters P Offline Zois [12] 2000 Morphological features W Offline Marti [ 39] 2001 Connected Components Enclose regions, Lower and upper profiles, L Offline As it has been seen from section 2, features extraction is crucially important step in the construction of any handwriting classification system. However, all the classification techniques have an underlying common dependence on the a priori selected features for their success or failure. 3. Feature Selection There are two different approaches to obtaining a subset of features: feature extraction and feature selection. In feature extraction the features that may have discriminating power were extracted, while in feature selection, a subset of the original set of features is selected. 3.1 Feature relevance The main idea of features selection is to select a subset of input variables by cutout features with weakly or no predictive information while maintaining or performing classification accuracy. John et al. [34] analyzed feature relevance as: strong and weak relevance. Strong relevance means that a feature cannot be removed from the feature set without loss of classification accuracy. Weak relevance means that a feature can sometimes contribute to classification accuracy. 3.2 Feature Selection Problem Selecting the most meaningful features is a crucial step in the process of classification problems especially in handwriting identification because: (1) it is necessary to find all possible feature subsets that can be formed from the initial set and which result in time consuming, (2) every feature is meaningful for at least some of discriminations, and (3) variations within intraclass and between inter-class is not too much high. Anyway, it has been observed that beyond a certain point, the inclusion of additional features leads to a worse rather than better performance [35]. 3.3 Feature Selection Approaches Research works in the feature subset selection have been based on two main models: the filter model and the wrapper model. In the first model, subset selection is performed as a pre-processing 378

5 module before the learning algorithm. While in the wrapper model, subset selection is performed as a cover around the learning algorithm [34]. Search for the optimal number of features may be done in one of the following ways as identified by (Weiss, 1990) cite in [53]: Single, independent features: Every feature is judged independently of its merit towards classification. Each feature that passes a test for statistical significance is used. Best stepwise feature selection: The best feature is chosen. The process is repeated to include additional features. However, backtracking is not done to discard an already picked feature. Stepwise feature elimination: All features are chosen at first. Then we start eliminating the worst of the set one by one. The process may be continued until the classification performance deteriorates. Best combined stepwise feature selection and elimination: This is a combination of the previous two methods. The best features are chosen one by one however a feature may be eliminated also, if the performance falls. Optimal branch-and-bound: This method uses feature elimination and backtracking. It is heuristically possible to limit the search by using a criterion function. In recent years, several techniques have been proposed for feature selection: principal component analysis (PCA), selforganizing feature maps, sequential forward search/sequential backward search (SFS/SBS), inter intra class distance radios (ICDRs), neural networks (NNs) and genetic algorithms (GAs)[33]. 4. Conclusion and Discussion In this paper, we discussed the problem of handwriting features from different aspects such as extraction methods, subset selection techniques, and classification algorithms. Handwriting features based character, word, line, and paragraph are introduced. Despite continuous effort, handwriting identification remains a challenging issue, due to different approaches use different varieties of features, having different accuracies and goals as well. Table 1 has shown us the previous studies of handwriting identification were totally focused on feature extraction methods. However selecting the most relevant features will offer a potential improvement by: Simplifying features extracting task, optimizing classification system complexity, reducing running time and improving the classification accuracy. Therefore, our study will focus on feature selection based handwriting identification. 5. REFERENCES [1] R. Huber and A. Headrick, Handwriting Identification: Facts and Fundamentals, Boca Roton, CRC Press, [2] Zhang B., S. Srihari, and S. Lee, Individuality of handwritten characters, in International Conference on Document Analysis and Recognition, (Edinburgh, Scotland), pp , August [3] S. Srihari, S. Cha, H. Arora, and S. Lee, Individuality of handwriting: a validation study, in International Conference on Document Analysis and Recognition, pp , [4] J. Favata and G. Srikantan, A multiple feature/resolution approach to handprinted digit and character recognition, International Journal of Imaging Systems and Technology, vol. 7, pp , [5] G. Leedham and S. Chachra, Writer identification using innovative binarised features of handwritten numerals, in International Conference on Document Analysis and Recognition, [6] V. Pervouchine and G. Leedham, Extraction and analysis of document examiner features from vector skeletons of grapheme th, in Document Analysis Systems, pp , [7] X.Wang,. Ding, H. Liu, Writer identification using directional element features and linear transform, in: Proceedings of the 7th International Conference on Document Analysis and Recognition, 2003, pp [8] Y. Nakamura and M. Kidode, Individuality analysis of online kanji handwriting, ininternational Conference on Document Analysis and Recognition, [9] K. Steinke, Recognition of writer by handwriting images, Pattern Recognition, vol. 14, no. 1-6, pp , [10] B. Zhang and S. N. Srihari, Analysis of handwriting individuality using word features, in International Conference on Document Analysis and Recognition, (Edinburgh, Scotland), pp , August [11] C Tomai I., B. Zhang and S. N. Srihari, Discriminatory power of handwritten words for writer recognition, in International Conference on Pattern Recognition, vol. 2, (Cambridge, UK), pp , [12] E. Zois and V. Anastassopoulos, Morphological waveform coding for writer identification, Pattern Recognition, vol. 33, pp , March [13] C. Hertel and H. Bunke, A set of novel features for writer identification, in International Conference Audio and Video-Based Biometric Person Authentication, pp , [52] [14] H. Said, G. Peake, T. Tan, and K. Baker, Writer Identification from Non-Uniformly Skewed Handwriting Images, Proc. Ninth British Machine Vision Conf., pp , [15] S.-H. Cha and S. Srihari. Writer identification: statistical analysis and dichotomizer,, Springer LNCS 1876, pp , [16] S.-H. Cha and S. Srihari. Multiple feature integration for writer verification, In Schomaker, L.R.B., Vuurpijl, L.G. (eds.): Proc. 7th Int. Workshop Frontiers in Handwriting Recognition, pp , [17] Y. Zhu, T. Tan, Y. Wang, Biometric personal identification based on handwriting, in: Proceedings of the 15th International Conference on Pattern Recognition, 2000, pp [18] M. Bulacu, L. Schomaker, and L. Vuurpijl, Writer Identification Using Edge-Based Directional Features, 379

6 Proc. Seventh Int l Conf. Document Analysis and Recognition, pp , Aug [19] A. Bensefia, A. Nosary, T. Paquet, and L. Heutte, Writer Identification by Writer s Invariants, Proc. Eighth Int l Workshop Frontiers in Handwriting Recognition, pp , Aug [20] A. Bensefia, T. Paquet, L. Heutte, Handwriting analysis for writer verification, in: Proceedings of the 9th International Workshop on Frontiers in Handwriting Recognition, 2004, pp [21] A. Bensefia, T. Paquet, and L. Heutte, A Writer Identification and Verification System, Pattern Recognition Letters, vol. 26, no. 10, pp , Oct [22] A. Bensefia, T. Paquet, L. Heutte, Information retrieval based writer identification, in: Proceedings of the 7th International Conference on Document Analysis and Recognition, 2003, pp [23] A. Bensefia, Thierry Paquet, Laurent Heutte, Handwritten Document Analysis for Automatic Writer Recognition, Electronic Letters on Computer Vision and Image Analysis 5(2):72-86, 2005 [24] L. Schomaker and M. Bulacu, Automatic Writer Identification Using Connected-Component Contours and Edge-Based Features of Uppercase Western Script, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 6, pp , June [25] L. Schomaker, M. Bulacu, and K. Franke, Automatic Writer Identification Using Fragmented Connected- Component Contours, Proc. Ninth Int l Workshop Frontiers in Handwriting Recognition (IWFHR), pp , Oct [26] [22] A. Seropian, M. Grimaldi, N. Vincent, Writer identification based on the fractal construction of a reference base, in: Proceedings of the 7th International Conference on Document Analysis and Recognition, 2003, pp [27] [23] A. Seropian, N. Vincent, Writer authentication and fractal compression, in: Proceedings of the 8th International Workshop on Frontiers in Handwriting Recognition, 2002, pp [28] C. Vielhauer and J. Dittmann, Biometrics for User Authentication, Encyclopedia of Multimedia, B. Furth, Ed. Berlin, Germany. [29] Oliveira, L. S., Benahmed, N., Sabourin, R., Bortolozzi, F., Suen, C. Y., Feature Subset Selection Using Genetic Algorithms for Handwritten Digit Recognition Proc. XIV Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 01), P.362, [30] Jain, A.K.; Duin, R.P.W.; Jianchang Mao. Statistical pattern recognition: a review., Pattern Analysis and Machine Intelligence, Jan IEEE [31] Scheidat Tobias, Franziska Wolf, Claus Vielhauer; Analyzing Handwriting Biometrics in Metadata Context ;Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6072, 60720H, 2006 SPIE-IS&T X/06/$15 [32] Stephen M. Watt, Xiaofang Xie, Recognition of Large Sets of Handwritten Mathematical Symbols, IEEE International Conference on Document Analysis and Recognition (ICDAR), pp [33] Impedovo, D.; Pirlo, G.; Automatic Signature Verification: The State of the Art ; Systems, Man, and Cybernetics, Part C: Applications and Reviews, Sept IEEE, Page(s): ,Volume 38, Issue 5 [34] G.H. John, R. Kohavi, K. Pfleger, Irrelevant features and the subset selection problem, in: Proceedings of 11th International Conference on Machine Learning (ML94), Rutgers University, New Brunswick, NJ, 1994, pp [35] Oliveira, L.S.: A Methodology for Feature Selection Using Multi-objective Genetic Algorithms for Handwritten Digit String, International Journal of Pattern Recognition and Artificial intelligence 17 (6) (2003) [36] S. Srihari, S. Cha, H. Arora, and S. Lee, Individuality of Handwriting, J. Forensic Sciences, vol. 47, no. 4, pp. 1-17, July [37] Marius Lucian Bulacu ; Statistical Pattern Recognition for Automatic Writer Identification and Verification, 15 march 2007, theses, University of Groningen, Netherlands. [38] R. jean Plamondon and S. N. Srihari, On-line and off-line handwriting recognition:a comprehensive survey, IEEE Transactions on Pattern Analysis and Machine intelligence, vol. 22, pp , Sep [39] U.-V. Marti, R. Messerli, and H. Bunke, Writer Identification Using Text Line Based Features, Proc. Sixth Int l Conf. Document Analysis and Recognition (ICDAR), pp , Sept [40] H. Said, G. Peake, T. Tan, and K. Baker, Writer Identification from Non-Uniformly Skewed Handwriting Images, Proc. Ninth British Machine Vision Conf., pp , [41] A. K. Jain, P. Flynn, and A. Ross, Handbook of Biometric s. NewYork: Springer-Verlag, 2007 [42] Bulacu, M.; Schomaker, L.; Text-Independent Writer Identification and Verification Using Textural and Allographic Features ; Pattern Analysis and Machine Intelligence, April 2007 IEEE JNL Page(s): Volume 29 [43] A. Jain, L. Hong, and S. Pankanti. Biometric identification. Commun. ACM, 43(2):90 98, [44] Schomaker, L.; Advances in Writer Identification and Verification ; Document Analysis and Recognition, Sept IEEE CNF Page(s): , Volume 2 [45] Kudo M, Sklansky J., Comparison of Algorithms that Select Features for Pattern Classifiers Pattern Recognition, Vol.33, pp.25 41, 2000 [46] Jain, A.K.; Duin, R.P.W.; Jianchang Mao. Statistical pattern recognition: a review., Pattern Analysis and Machine Intelligence, Jan IEEE [47] Vielhauer, Claus Ralf Steinmetz; Handwriting: Feature Correlation Analysis for Biometric Hashes, EURASIP 380

7 Journal on Applied Signal Processing, January 2004, Pages: , Volume 2004, Issue 1 [48] Soryani M., and N. Rafat; Application of Genetic Algorithms to Feature Subset Selection in a Farsi OCR, proceedings of world academy of science, engineering and technology volume 18 december 2006 issn [49] Kamilah Azah Muda & Siti Mariyam Shamsuddin; Generalized Global Shape Features for Individuality of Handwriting, Postgraduate Annual Research Seminar 2007 (3-4 July 20 [50] [9] S.K. Chan, C. Viard-Gaudin, Y.H. Tay, Online writer identification using character prototypes distributions, in: Proceedings of SPIE The International Society for Optical Engineering, [51] R. Niels, F. Gootjen, L. Vuurpijl, Writer identification through Information retrieval: the allograph weight vector, in: International Conference on Frontiers in Handwriting Recognition, 2008, pp [52] Tan,G.X., Automatic writer identification framework for online handwritten document susing character prototypes, Pattern Recognition(2009), Elsevier Ltd. [53] Jameel Akhtar, Perspective of Pattern Recognition in Handwriting Character Recognition, PHD thesis, Tulane University,