Font Type Extraction and Character Prototyping Using Gabor Filters

Transcription

1 Font Type Extraction and Character Prototyping Using Gabor Filters Bénédicte Allier, Hubert Emptoz Laboratoire de Reconnaissance des Formes et Vision (RFV) INSA de Lyon 0 a. A. Einstein, 6961 Villeurbanne cedex, FRANCE {allier ; emptoz}@rf.insa-lyon.fr Abstract In this paper, we present an automatic method for character prototyping and font type characterization in machine-printed document images at a character leel. To do so, we use a generic textural approach, which considers text as a texture, instead of working at a pixel leel like most of the methods proposed so far. In this way, Gabor filtering seems to be an appropriate tool for texture characterization, since its design has been inspired by the human isual system. The objectie of the paper is then to erify this hypothesis by applying our method on a corpus composed of what we call typographically rich and recurrent machine-printed document images. 1. Introduction The aim of our work is to study a general approach for the characterization of character shapes and fonts, in other words we want to group together, on the one hand, letters written in a same font, without explicitly recognizing the font types; and, on a second hand, same character shapes gien a font type, without explicitly recognizing the shapes. The knowing of such information is useful in many applications such as logical labeling, OCR performances improement [1, ], character prototype extraction... Many methods hae yet been proposed in the literature to recognize font styles in document images; among them: Zramdini [1] proposes an Optical Font Recognition (OFR) system based on the study of digital typography, Chaudhuri and Garain [3] detect italics and bold at a character leel by analyzing character widths in a few gien directions, Wong et al. [4] use geometrical and statistical properties of text blocks at different leels (e.g. total amount of black pixels composing the block, block height, eccentricity ), Eglin et al. [5] use statistical features such as compactness, entropy... calculated on text blocks to coarsely characterize font families, Duffy [6] uses shape redundancies based on the calculation of symmetrical differences to group together characters written in a same font, and Doermann et al. [7], presenting a more global method at a word leel, use morphological mathematics and the orientation of surrounding blocks for bold and italics detection. In fact, most of the proposed methods work on segmented document images at a pixel leel, thus suffering from eery kind of noise images may present. To aoid this, we decided to pay attention to more global methods classically used in natural images processing, considering machine-printed text images as textured images. Gabor filtering is a good method to analyze texture since it has been designed to simulate the human isual system [8], and it may be worth testing at a character leel as this has neer been done so far (Gabor filters hae only been used in a few ery particular document images applications [9-1], and among them Zhu et al. proed that this tool was worth testing at a text zone leel to differentiate fonts [1]). This paper is then organized as follows. In the next section we will introduce the well-known multi-channel Gabor filtering method. Section 3, justifies that character prototyping can be seen as a textural problem. Results for font characterization are presented and discussed in section 4. Finally, section 5 presents a brief summary of the paper and concluding remarks on our future work.. Texture feature extraction Multi-channel filtering techniques enable the extraction of texture characteristics locally in frequency and in orientation, in other words texture features calculations (in frequency and orientation) can be done for each pixel oer a region of interest surrounding it. This method is particularly interesting since it has been inspired from the human ision system that decomposes the retinal image into a great number of filtered images, each one containing intensity ariations oer a small range of frequency and orientation [8]. The idea of the Gabor approach is then to design a multi-channel filtering particularly selectie in frequency and orientation in order to fully characterize texture. Each filter is then applied to the original textured image, and a further analysis enables the creation of a simple characterizing feature ector (based on statistical calculations). Proceedings of the Seenth International Conference on Document Analysis and Recognition (ICDAR 003) /03 $ IEEE

2 .1. Gabor filters theory A D Gabor function h is a sinusoid plane wae modulated by a Gaussian enelope and oriented with angle θ from the x-axis. The mathematical formulation, in the spatial domain, for fundamental frequency u 0 along the x-axis (i.e. θ =0 ), is: y u x y 4 σ σ sinusoidal ( x, y ) = exp 1 x cos( π x ) h 0 1 function Gaussian enelope where σ x (resp. σ y ) is the Gaussian standard deiation along the x-axis (resp. the y-axis). Filters with orientation θ (θ 0 ) are obtained by rotating the latter equation. Selectiity of the filter bank in frequency and orientation is clearer in the frequency domain, that is why we apply the Fourier transform to equation (1), yielding: (, ) = TF( h( x, y) ) H u ( u ) 0 exp 1 u σ u σ = A. ( ) 1 u 0 exp u σ u σ where σ u = 1/πσ x, σ = 1/πσ y and A = πσ x σ y. Thus, in the frequency domain, the signal is represented by two Gaussians along the x-axis, centered in u 0 and in u 0, as shown on Figure 1: (a) (b) (1) () Figure 1. Gabor filters in a window, represented in spatial and frequency domains for u 0 = 1/8 ; θ = 0 (a) and for u 0 = 1/8 ; θ = 45 (b) Note that, due to the Fourier transform, filter bandwidths increase with u 0 resulting in less selectie filters. Moreoer, it can be shown that the Gabor filters nearly uniformly coer the spatial-frequency domain, and since there is little oerlapping between filters, they form a nearly orthogonal basis resulting in a powerful texture characterizing tool [13] (cf. Figure ). Figure. Frequency representation of a bank of 4 Gabor filters (6 frequency ranges, 4 orientations), the origin (u;)=(0;0) is at the center of the image In our experiments, we use a bank of 4 Gabor filters, as presented in [14], defined in a window, with S=4 radial frequencies (u 0 = 0.05, 0.1, 0., 0.4) and K=6 alues of orientation θ (θ=0, π/6, π /3, π/, π/3, 5π/6), since it seems to enable the characterization of a wide range of textures. Also, we work on sets of N input images composed of prototype images on the one hand, and of test images (that hae to be classified) on the other hand. Our method can be diided into 3 main steps: (a) Gabor filters are applied to each input image independently, resulting in 4 filtered images per input image (b) statistical features are calculated on each filtered image, producing as many feature ectors that enable the calculus of a distance measure between prototype and test images (this method will be explained in the next sub-section) (c) distances are carried to a M-dimension space on which a Bayesian classifier will be processed.. Feature extraction As we said before, our aim is to classify character images thanks to a distance measure calculated on feature ectors extracted from each filtered image. For that, feature ectors we use for any input image are composed of statistical obserations on its 4 related filtered images. Statistical obserations characterize intensity repartition using first and second moments i.e. mean and standard deiation. These moments are notated µ and σ, and are gien, on an image I of dimensions N w N w, by: N N w w I( i ; j ) N w N w [ I( i ; j ) µ ] i= 1 j = 1 µ = and σ = () N w N w i= 1 j = 1 The distance measure is defined on these features according to [14]. That is, for two response images i and j µ ;σ µ ;σ, and distance d ij k oer of features ( i i ) and ( j ) the k th filter, the global distance d ij is gien by: j u Proceedings of the Seenth International Conference on Document Analysis and Recognition (ICDAR 003) /03 $ IEEE

3 d ij = d ( i; j) = µ i µ j σ i σ j (3) norm ( µ ) norm ( σ ) 4 4 k d = ij k = 1 k = 1 where norm(µ) (resp. norm(σ)) are the standard deiations of µ (resp. σ) oer the entire set of input images, used to normalize the two features. 3. Character classification The image set we are testing our method on, is composed of machine-printed documents considered as what we call typographically rich and recurrent documents. In other words, they are strongly structured documents presenting only a few well distinguished recurrent font types regularly disposed from one page to another (this is typically the case of dictionaries). More precisely, our corpus can be diided into 4 subsets of homogeneous layouts, presenting in aerage 5 font types each. In our approach, the text is considered as a juxtaposition of small textured regions at a character leel. Thus, different characters gien a font type and a same character written in different font types produce different textures. Looking carefully at a few characters, e.g. a and a or o and z, obiously reeals that they can t gie the same texture features according to the Gabor analysis presented in section, since they don t hae the same orientations nor the same apparent frequencies. This is what we want to show now, trying to separate them into different classes Experimentation We are now working on a set of binary characters all written in the same font, embedded in images, and coming from a preceding physical segmentation step (we won t gie more details on this pre-processing phase since it is not our purpose here). The character images (tests and prototypes) are randomly chosen among binarized document images of the corpus. Thus, we get 14 different patterns (the most frequently used: a, c, d, e, i, l, m, n, o, p, r, s, t, u ), resulting in 560 character images. Feature ectors, and then distances are calculated between test images and prototype images, and carried on a 14-dimension space for classification purposes. The classification is processed using the AutoClass software inoling Bayesian classification theory, presented in [15]. It aims at automatically determining the maximally probable number of classes, using the classical finite mixture distribution as fundamental model. 3.. Results The classification algorithm is processed 500 times on the same dataset, since the search includes a random component (AutoClass recommends at least 50 trials), and we obtain 17 classes without a priori knowledge. Examining the results, we notice that the classifier has managed to separate well ery particular patterns, storing them into independent classes. This is the case of characters e, r, s, t, i, a, o, l, m and c, except a few isolated mistakes. Two other classes are composed of mixed characters namely n and u on the one hand, and d and p on the other hand. These are not explicitly classification errors since Gabor filtering has been designed to be orientation and translation inariant. Thus, as n and u are a same pattern rotated with angle 180, Gabor analysis logically characterizes them the same way, and so does it for d and p. Other classes are rejection classes, containing on the most 5 test images each. Finally, the error rate is ealuated to 3.6%, that is, we get a 96.4% recognition rate, which is really satisfying. These results proe that Gabor filtering is useful at a character leel for characterization, thus enabling character classification (or prototype extraction) but the problem is that the separation of same rotated patterns has not been soled yet. Plotting the distances between test images and prototype images for characters d and p confirms that the discrimination is not possible using only Gabor filtering (cf. Figure 3). Our idea is then to complete our characterization process using a ery simple non time-consuming distance measure. Figure 3. Distance between test images d (crosses) and p (circles) and prototype images: d on x-axis, p on y-axis For that we use a pattern matching method, inspired from [16], based on the calculus of a symmetrical difference between a reference pattern and a test pattern. The principle of our method is to count the amount of black pixels in both images, the origin of the images taken in the upper left corner of the images and matching processed oer a small window surrounding the origin (to preent from possible pattern shifting in the image due to Proceedings of the Seenth International Conference on Document Analysis and Recognition (ICDAR 003) /03 $ IEEE

4 the digitization or the segmentation phases) Figure 4. A score is calculated by counting the number of black pixels in the reference image finding correspondence in the test image N RT, plus the number of black pixels in the test image finding correspondence in the reference image N TR and rationalize it by the total amount of black pixels in both images N B, that is, score = ( N RT N TR ) / N B. A threshold thresh is empirically chosen equal to 90%, so that when score > thresh, characters are considered as similar, else they are different. (-1;-1) (0;0) (0;0) samples in a set of 8 document images (with 4 different layouts) results in 11 font types oer 370 character images (cf. Figure 6). As before, we use AutoClass algorithm, in a 11-dimension space (a) Figure 4. (a) reference image (b) test image (c) matching in a window surrounding the reference image origin (0;0): best matching (-1;-1), score=(88)/(109)=84.% With this calculus, we get a 95% recognition rate for characters u and n oer 79 samples, and a 100% recognition rate for characters d and p oer 4 samples. These results are encouraging since they can be obtained using a ery simple algorithm. Notice that a few errors still exist between characters u and n : they are due to degradations on the character shapes making them differ from only a few pixels (cf. Figure 5). Figure 5. Examples of classification errors with simple pattern matching method Finally, these results proe that Gabor filtering is useful at a character leel for characterization, thus enabling character classification, and een prototype extraction (using the same pattern matching method as presented before). Our aim now is to carry on studying this tool to see if, as we guess, it is still powerful for font type characterization. 4. Font type classification 4.1. Experimentation (b) The method used in this experiment is the same as before: the first step is to create a character image set of different font types gien a pattern. The character we chose in this experiment is a since it is widely used in French, and since it is present een in poorly used font types in our documents. Randomly choosing character (c) Figure 6. Character prototypes randomly chosen for font extraction with corresponding class numbers 4.. Results The classification algorithm is again processed 500 times on the same dataset. We obtain 13 classes without a priori knowledge. Among them 11 contain single font types (een though some font types look ery similar but a small scaling factor), and rejection classes (all in all 8 samples). One of the rejection classes is worth studying, since it is not so strange: it groups together 3 characters of classes #1, #4 and #9 which are ery bold-faced fonts of same sizes, indeed not so dissimilar. Note that this is the only confusion between classes #1 and #9 which are ery similar but a small difference in the upper tail of the character. Finally, the error rate is ealuated to less than 3.0%, that is we get more than 97.0% recognition rate, which is really interesting considering the low representatieness of some classes (4 classes are less than 10 sample images), as well as the little difference that exists between some other classes. Experimentations hae also been made on character s oer 350 test images, resulting in a 96.85% recognition rate. Actually, Gabor filtering seems to be a good tool for character font type extraction, een if further experiments should be done to enforce our diagnostic. 5. Discussion and further work We hae presented an interesting approach for font type classification and character prototyping based on a global statistical approach. In our experiments, characters are considered as textures, and we analyze them using Gabor filtering. We hae shown, on a few examples, that a bank of 4 Gabor filters (4 frequencies and 6 orientations) was enough to discriminate 11 fonts and 14 Proceedings of the Seenth International Conference on Document Analysis and Recognition (ICDAR 003) /03 $ IEEE

5 characters shapes. As Gabor filters were designed to be neither translation nor rotation sensitie, we had to complete our method incorporating a simple shape matching algorithm to distinguish between similar characters (from Gabor point of iew) such as u and n or d and p. We achiee a 96.8% aerage recognition rate for both experiment types. Our method doesn t allow recognition of font attributes separately yet (e.g. font type or name (Courier, Heletica,...), font size ), this should be done using further analysis as proposed in [17]. Neertheless, results are globally encouraging since they allow grouping of binarized randomly chosen characters, sometimes suffering from strong artifacts that algorithms at a pixel leel (like the pattern matching solution we presented in section 3) would hae considered as different (a complete analysis of a few interesting shape redundancy methods for character prototyping are presented in [18]). To enforce our work, more experiments should be done on character shapes: this will be part of our future work. It would also be interesting to study if the dimensionality of the feature space can be lowered, this would mean that a few prototypes would be enough to classify a wide range of character shapes. Finally, we should focus on the possibility of lowering the number of characterizing Gabor filters, using the method proposed in [13], to aoid useless computational burdens. This work is being processed at this ery moment. 6. References [1] A. ZRAMDINI "Study of Optical Font Recognition Based on Global Typographical Features", thesis of the Thesis of the Uniersity of Fribourg, Fribourg (Switzerland), 1995, 170p. [] B. B. CHAUDHURI and U. GARAIN, "Automatic Detection of Italic, Bold and All-Capital words in Document Images" Proc. of 14th International Conference on Pattern Recognition - ICPR, Brisbane (Australia), 1998, pp [3] B. B. CHAUDHURI and U. GARAIN, "Extraction of type style-based meta-information from imaged documents", International Journal on Document Analysis and Recognition - IJDAR, ol. 3, no. 3, 001, pp [4] K. Y. WONG, R. G. CASEY, and F. M. WAHL, "Document Analysis System", IBM Journal of Research and Deelopment, ol. 6, no. 6, 198, pp [5] V. EGLIN, S. BRES, and H. EMPTOZ, "Statistical characterization and classification of printed text in a multiscale context" Proc. of nd International Workshop on Statistical Techniques in Pattern Recognition - SPR 98, Sydney (NSW), (Australia), 1998, pp [6] L. DUFFY "Recherche d'information logique dans les documents à typographie riche et récurrente. Application aux sommaires", thesis of the Institut National des Sciences Appliquées - INSA de Lyon, Lyon (France), 1997, 160p. [7] D. S. DOERMANN, E. RIVLIN, and A. ROSENFELD, "The function of documents", International Journal of Computer Vision - IJCV, 1998, pp [8] M. PORAT and Y. Y. ZEEVI, "The generalized Gabor scheme of image representation in biological and machine ision", IEEE Transactions on Pattern Analyis and Machine Intelligence, ol. 10, no. 4, 1988, pp [9] V. WU, R. MANMATHA, and E. M. RISEMAN, "Finding Text in Images" Proc. of Second ACM International Conference on Digital Libraries, Philadelphia, PA (USA), 1997, pp.3-6. [10] A. K. JAIN and S. K. BHATTACHARJEE, "Address Block Location on Enelopes Using Gabor Filters", Pattern Recognition, ol. 5, no. 1, 199, pp [11] A. K. JAIN, S. K. BHATTACHARJEE, and Y. CHEN, "On texture in document images" Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Champaign, Illinois (USA), 199, pp [1] Y. ZHU, T. TAN, and Y. WANG, "Font Recognition Based On Global Texture Analysis", IEEE Transactions on Pattern Analysis and Machine Intelligence, ol. 3, no. 10, 001, pp [13] A. K. JAIN and F. FARROKHNIA, "Unsuperised Texture Segmentation Using Gabor Filters", Pattern Recognition, ol. 4, no. 1, 1991, pp [14] B. S. MANJUNATH and W. Y. MA, "Texture Features for Browsing and Retrieal of Image Data", IEEE Transactions on Pattern Analysis and Machine Intelligence, ol. 18, no. 8, 1996, pp [15] P. CHEESEMAN and J. STUTZ, "Bayesian classification (Autoclass): Theory and results", Adances In Knowledge Discoery And Data Mining, 1996, pp [16] Y. CHENEVOY "Reconnaissance structurelle de documents imprimés : études et réalisations", thesis of the Thèse de l'inpl, 199, xxp. [17] A. ZRAMDINI "Study of Optical Font Recognition Based on Global Typographical Features", thesis of the IIUF-Uniersité de Fribourg, Fribourg (Suisse), 1995, 170p. [18] B. ALLIER and N. BOURSIER, "Reconstruction de caractères dégradés: étude pour la détermination d'un caractère "idéal"" Report of Institut National des Sciences Appliquées - INSA, Lyon RR00-XX, décembre 00, 49p. Proceedings of the Seenth International Conference on Document Analysis and Recognition (ICDAR 003) /03 $ IEEE