Input sensitive thresholding for ancient Hebrew manuscript

Transcription

1 Pattern Recognition Letters 26 (2005) Input sensitive thresholding for ancient Hebrew manuscript Itay Bar-Yosef * Department of Computer Science, Ben Gurion University, Beer-Sheva 84105, Israel Received 21 April 2004; received in revised form 4 July 2004 Available online 7 December 2004 Abstract In this paper, we describe an input sensitive thresholding algorithm for ancient Hebrew calligraphy documents. Usually, historical document images are of poor quality since the documents have degraded over time due to storage conditions. However, the distribution of noise in one document is not uniform and the characters quality may vary. We develop tools to identify noisy characters and apply more sophisticated tools to process them. First, we use a global thresholding method to obtain an initial binary image. This suffices for noise free characters. Then we evaluate the document characters and invoke an accurate local method only on the noisy characters. Results show that our method detects a very high percent of the noisy characters, and that the local method achieves very accurate results. Ó 2004 Elsevier B.V. All rights reserved. Keywords: Binarization; Old manuscripts; Character segmentation 1. Introduction In general, historical document images are of poor quality because the documents have degraded over time due to storage conditions, and to the quality of the written parchment. As a result, the foreground and background are difficult to separate. The problem is particularly difficult because many documents have varying contrast, smudges, variable background intensity and presence of seeping ink from the other side of the document (see Fig. 1). * Tel.: address: itaybar@cs.bgu.ac.il Several thresholding approaches have been reported in the literature on binarization of text documents with noisy background. In (Don, 1995), presented a noise attribute thresholding method for document image binarization. The method utilized noise attribute features, based on a simple noise model, to overcome the difficulty that some objects do not form prominent peaks in the histogram. Negishi et al. (1999) presented a thresholding algorithm to extract the character bodies from a noisy background. They deal with extremely dirty and considerably large images, where the gray level of the characters overlaps with the background. Liu and Srihari (1997) developed a thresholding algorithm based on texture features /$ - see front matter Ó 2004 Elsevier B.V. All rights reserved. doi: /j.patrec

2 I. Bar-Yosef / Pattern Recognition Letters 26 (2005) Fig. 1. Example of an old Hebrew manuscript. Their proposed algorithm utilized two fundamental attributes of document images: (i) the characters normally occupy separable gray level range in the gray-scale histogram, and (ii) that the text images contain highly structured-stroke units. Tan et al. (2000) established a method for removing interfering strokes from double-sided handwritten documents based on the observation that the edges of the sipping strokes from the reverse side are not as sharp as those on the front side, they adopt an edge detection approach to suppress unwanted background patterns. In (Solihin and Leedham, 1999), the authors proposed a new class of histogram based global thresholding techniques called Integral Ratio (QIR, NIR). The proposed class of techniques was based on a two stage thresholding approach requiring each pixel of a handwritten image to be placed into one of three classes: foreground, background, and a fuzzy area between them. In (Leedham et al., 2002), a comparison between several global methods found that the QIR method performed the best on degraded document images. In our approach, we use the QIR method as a first global phase (see Section 2). The methods described above do not suffice for our data, since it has wide variation in quality even within one document. Although some of the characters are well separated from the background, the sipping ink, the stains and the gray level overlap between other characters and the mentioned noise, make the thresholding task very difficult. Our algorithm is an improvement of the multi-stage binarization method presented in (Bar-Yosef et al., 2004), where first a global method is used and the produced connected components are refined using a local adaptive method. In our data, a substantial amount of characters is in good condition, and some are badly degraded. This led us to develop an input sensitive system, where a global thresholding is applied first, and the quality of the produced components are evaluated by an evaluation procedure that decides which components should be refined. We observe that it is easy to detect the seed body of each character. This leave a fuzzy area around the seed on which we apply our local refinement procedure. Our paper is organized as follows: the global phase is presented in Section 2, the quality evaluation procedure is introduced in Section 3, Section 4 presents the adaptive local method and Section 5 presents experimental results. 2. Global thresholding An important feature of the documents in our research is that the extremely large size of the documents makes the amount of smudge and ink stains very small compared with the relatively good characters, where this noise cannot be noticed in the imageõs histogram. As a consequence, almost any histogram based method which does not tend to over-threshold the documents, will provide reasonable preliminary results. Still, not all the characters are segmented well. Our task is therefore to efficiently detect the characters that should undergo a refining procedure. In Section 3 we describe the discrimination between good and bad characters. Another feature of the image quality is that the character edges are not sharp and blend with the background. This creates a range of gray level values, placed near the histogramõs valley. A substantial part of this range is detected as foreground in global methods, which usually choose a threshold in the histogramõs valley (see Fig. 2). In our case, this often results in merged characters as can be seen in Fig. 3(b). The following step is taken to overcome this problem.

3 1170 I. Bar-Yosef / Pattern Recognition Letters 26 (2005) x Fig. 2. Bi-modal histogram of the image in Fig. 1. f B ¼ 1 if f < T C 0 otherwise Experiments show that the average number of merged characters after this refinement phase is reduced significantly, especially with under-thresholded images (see Fig. 3). In the following step, we ignore small blobs and letter punctuation (in Hebrew these are the vowels which are mostly positioned under the text lines) using a simple line extraction scheme. After this stage, f B Õs connected components are composed of foreground objects, few of them connected to some noise. Denote by CC ={CC 1,CC 2,...,CC N } the set of connected components in f B. 3. Quality evaluation Fig. 3. (a) Original, (b) after global thresholding and (c) after refinement. Let f be a gray scale image of an old Hebrew document. Select a global threshold T f using any global histogram based method, and obtain a binary image f B. f B ¼ 1 if f < T f 0 otherwise Let C B be the set of pixels belonging to the contours of the components in f B. Denote T C as the mean gray scale value of C B. Use T C as a threshold value on f, to refine f B. As we have observed, in every document there exists a set of characters that are segmented correctly by the previous step, denoted by CC seg,a set of characters with some noise attached to each, denoted by CC noise and a third set of merged characters due to a significant amount of noise in that characters area, denoted by CC merge. The purpose of this stage is to distinguish between the three sets: The first task is to obtain the members of CC merge. In every language or script, there are some structural rules which define basic features for a character. The written Hebraic text, used in our experiments, is called squared writing as most of the characters have approximately equal average width and height. Let H CC be the median height of the bounding boxes of the connected components of the predefined set CC, and let W i be the width of the bounding box of CC i. The set CC merge is defined by CC merge ={CC i jw i P H CC*a}, where a is an empirical constant. We show below how to distinguish between CC seg and CC noise. For each connected component, we create a reference image, which contains the low intensity pixels of the character. In a good character we expect only the boundary not to be included in the reference image, where in noisy character the noise is also brighter than the character intensity, thus will not be in the reference image as well. We compute R i, the reference image of CC i,

4 I. Bar-Yosef / Pattern Recognition Letters 26 (2005) Adaptive local method Fig. 4. (a) A character with some noise, (c) a character without noise (orig.) and (b,d) the character minus its reference image (diff.). i = {1,...,N}. Let D i be the distance image produced by applying the Distance Transform on R i within the bounding box of CC i. The distances of the edge pixels of CC i provide information about the quality of the character CC i as follows: uniform distances of the character edges according to D i imply a good character, while non-uniform distances of the edge pixels of CC i, from R i, imply a character with patches of noise. Fig. 4 shows two characters and the difference between the character and its reference image. The quality evaluation procedure is therefore: For each CC i, i =1,...,N: Let m i be the mean gray scale value of pixels belonging to CC i. Let R i be the reference image of CC i using m i as a local threshold: R i ¼ 1 if CC i 6 m i 0 otherwise Let Gstd be the global standard deviation of the distances created by the distance transform for all CC i, i =1,...,N (only the edge pixels). Let Lstd i be CC i Õs standard deviation of the distances created by applying the distance transform on R i. Every component CC i, with Lstd i 6 Gstd is classified as a well segmented character, and added to the set CC seg, the rest are classified as CC noise members. The members of CC noise and CC merge (defined above) will be processed using the adaptive local method presented in the following section. Even when a character is noisy or faded, the seed body of the character is easy to detect. This fact led us to adopt a region growing scheme in which we first detect the seed image of the characters, and then apply a growing process that expands the character to its final form. Since the binarization of the foreground objects is affected only by their local environment, we process information within the bounding box of each connected component of CC noise, CC merge. In the following two steps, we collect the foreground pixels by first finding a seed set of such pixels and then growing the set according to local neighborhood data Creating the seed image The first step in this stage is to derive for each component its seed image. If the component is in CC noise, then its seed image is R i, described in the previous section. Else, the component is in CC merge, then we use a pixel clustering algorithm based on the K-means algorithm (McQueen, 1967). Two clusters are considered, a foreground cluster and a background cluster, according to the gray level. After calculating the foreground and background clusters, we use the average gray level of the foreground cluster as a threshold to generate the seed image consisting of the pixels darker than the threshold The growing process The growing process is an iterative process, in which during each iteration a set of candidate pixels is observed. Each pixel from this set is tested whether it can join the foreground or not. The process is terminated when no pixel is added to the foreground. The algorithm goes as follows. Repeat until the foreground set does not change: Find all candidate pixels. The candidate pixels are background pixels which are 8-connected to the growing foreground. For each candidate pixel p, consider its 7 7 neighborhood: let M f be the average gray scale value of the foreground pixels in this window,

5 1172 I. Bar-Yosef / Pattern Recognition Letters 26 (2005) Figs. 6 and 7 show an input image and the produced binary image. 5. Experimental results and discussion Fig. 5. (a) A character with dark noise around it (original). (b) As can be seen, the dark ink is incorrectly considered as part of the character (binary). M b be the average gray scale value of the background pixels in this window. Assign p to the class whose average is closest to the gray level of p (according to M f and M b ). Fig. 6. Input image. The performance of our approach has been evaluated on a set of 30 Hebrew calligraphy manuscripts from 14th to 16th century 1 similar to the document in Fig. 1. The document size varies from to , where the over-sized documents were split into several images of roughly size, each containing an average of 200 characters, of approximately size. The results were evaluated by visual inspection, where in each document the percentage of correctly segmented characters was calculated (a correct character is a character which is not under or over-thresholded). In addition, we summarize the performance of the quality evaluation procedure (see Section 3): The average percentage of correctly segmented characters is 94%, where in a substantial set of the documents the average percentage reached up to 98%. Most of the problems occurred in documents where the character strokes had almost disappeared, and the accuracy of the adaptive local method was too sensitive to handle the rapid intensity changes. In documents which contained very dark ink stains connected to any character, additional structural rules are needed in order to obtain a noise free character (see Fig. 5 for an example). The set CC seg contained mostly very well segmented characters, and few characters with small patches of noise. 90% of the set CC noise are noisy characters, and the remaining 10% are good characters which were wrongly classified as a CC noise member. This means that from the non-merged characters that were refined using the refinement procedure, 10% did not need to be refined. Fig. 7. Binarized image. 1 The documents are taken from the archive of the School of Library, Archive and Information Studies, The Hebrew University.

6 I. Bar-Yosef / Pattern Recognition Letters 26 (2005) On average, in each document 25% of the components belong to CC merge, 15% belong to CC noise and 60% belong to CC seg. This means that on average, 40% of each documentõs characters, need to be refined. The results presented here show that for any document, a substantial set of its characters are thresholded correctly in the first global stage, and the remaining noisy characters are well identified by the evaluation procedure and are thresholded accurately by the local refinement method. Our algorithm was compared to the multi-stage binarization algorithm presented in (Bar-Yosef et al., 2004), in which first the image was thresholded using the a global method, and then a similar refining method was invoked on each connected component to produce the final accurate binary image. We compared only the results of the set CC seg for each document, which in (Bar-Yosef et al., 2004) was processed with the refining method: In cases where the characters had brighter regions or faded strokes due to intensity degradation, the algorithm presented in this paper performed better since the refining method was too accurate to handle the rapid intensity changes, while the CC seg members were correctly segmented. The computational complexity is strongly reduced since the refining method is very time consuming. 6. Future work We are now working on a generalization of the approach presented in this paper in order to make it robust and adequate to handle any type of text document, regardless of its condition and not restricted to any alphabet. Acknowledgment I would like to thank my advisers Prof. Klara Kedem and Prof. Itshak Dinstein for helping me with this paper. References Bar-Yosef, I., Kedem, K., Dinstien, I., Beit-Arie, M., Angle, E., Classification of Hebrew calligraphic handwrit ing styles: preliminary results. In: Proc. Internat. Workshop on Document Image Analysis for Libraries. pp Don, H.S., A noise attribute thresholding method for document image binarization. In: Proc. 3rd Internat. Conf. on Document Analysis and Recognition, Canada, pp Leedham, G., Varma, S., Patanakar, A., Govindaraju, V., Separating text and background in degraded document images a comparison of global thresholding techniques for multi-stage thresholding. In: Proc. Eighth Internat. Workshop on Frontiers in Handwriting Recognition. pp (thresholding Algorithms for Text/Background Seg). Liu, Y., Srihari, S.N., Document image binarization based on texture features. IEEE Trans. PAMI 19 (5), McQueen, J.B., Some methods of classification and analysis of multivariate observationsproc. 5th Berkeley Symposium in Mathematics, Statistics and Probability, vol. 1. University of California, Berkeley, USA, pp Negishi, H., Kato, J., Hase, H., Watanabe, T., Character extraction from noisy background for an automatic reference system. In: Proc. 5th Internat. Conf. Document Analysis and Recognition, India, pp Solihin, Y., Leedham, C.G., Integral ratio: A new class of global thresholding techniques for handwritten images. IEEE Trans. PAMI 21 (8), , pp Tan, C.L., Cao, R., Shen, P., Wang, Q., Chee, J., Chang, J., Removal of interfering strokes in double-sided document images. In: IEEE Workshop on the Application of Computer Vision, WACV2000, California, 4 6 December, pp