Efficient Off-Line Cursive Handwriting Word Recognition

Transcription

1 Effcent Off-Lne Cursve Handwrtng Word Recognton B. Gatos, I. Pratkaks, A.L. Kesds, S.J. Perantons To cte ths verson: B. Gatos, I. Pratkaks, A.L. Kesds, S.J. Perantons. Effcent Off-Lne Cursve Handwrtng Word Recognton. Gu Lorette. Tenth Internatonal Workshop on Fronters n Handwrtng Recognton, Oct 6, La Baule (France), Suvsoft, 6. <nra-143> HAL Id: nra Submtted on 6 Oct 6 HAL s a mult-dscplnar open access archve for the depost and dssemnaton of scentfc research documents, whether the are publshed or not. The documents ma come from teachng and research nsttutons n France or abroad, or from publc or prvate research centers. L archve ouverte plurdscplnare HAL, est destnée au dépôt et à la dffuson de documents scentfques de nveau recherche, publés ou non, émanant des établssements d ensegnement et de recherche franças ou étrangers, des laboratores publcs ou prvés.

2 Effcent Off-Lne Cursve Handwrtng Word Recognton B. Gatos, I. Pratkaks, A.L. Kesds, S.J. Perantons Computatonal Intellgence Laborator, Insttute of Informatcs and Telecommuncatons, Natonal Center for Scentfc Research Demokrtos, GR Aga Paraskev, Athens, Greece {bgat,pratka,akesds,sper}@t.demokrtos.gr Abstract In ths paper, we present an off-lne cursve word handwrtng recognton methodolog. Ths s based on a novel combnaton of two dfferent modes of word mage normalzaton and robust hbrd feature etracton. Word mage normalzaton s performed b usng as a reference pont the geometrc center of the word mage as well as b placng the baselne of the word n the center of a rectangular bo. Addtonall, mage pre-processng s performed n order to correct word skew, word slant as well as to normalze the stroke thckness. At a net step, two tpes of features are combned n a hbrd fashon. The frst one, dvdes the word mage nto a set of zones and calculates the denst of the character pels n each zone. In the second tpe of features, we calculate the area that s formed from the projectons of the upper and lower profle of the word. The performance of the proposed methodolog s demonstrated after testng wth the reference IAM cursve handwrtng database. Kewords: Handwrtng word recognton, Hbrd feature etracton 1. Introducton Off-lne cursve handwrtng recognton has acheved a great attenton for man ears due to ts mportant contrbuton n the dgtal lbrares evoluton. In the lterature, two general approaches can be dentfed: the segmentaton approach and the global or segmentaton-free approach. The segmentaton approach requres that each word has to be segmented nto characters whle the global approach entals the recognton of the whole word. In the segmentaton approach, the crucal step s to splt a scanned btmap mage of a document nto ndvdual characters [6]. A segmentaton-free approach s followed n [1][3][8][11][13][14][15][19] where lne and word segmentaton s used for creatng an nde based on word matchng. In [15], a dscusson on dfferent approaches to word matchng s gven. In [1], Ulam s dstance s used for mage matchng b dentfng the smallest number of mutatons between two strngs. In [3], a two-dmensonal mage s converted nto a onedmensonal strng. The method descrbes how to etract nformaton from the strngs and compute the dstance between them resultng n smlar matches. In the segmentaton-free approach of [19], word matchng s based on the vertcal bar patterns. Each word s represented as a seres of vertcal bars that s used for the matchng process. Word mage matchng s also appled n [13] usng the weghted Hausdorff dstance. Before applng the matchng process usng the Hausdorff dstance a normalzaton scheme s used for each word. Word matchng s also performed n [11] where global and local features based on profle sgnatures and morphologcal cavtes are used for each word characterzaton. In [18] a votng sstem s used for fuson of multple handwrtten word recognton technques based on ranks and confdence values. In ths work, we present an off-lne handwrtng word recognton sstem that s based on a novel combnaton of two dfferent modes of word mage normalzaton and robust hbrd feature etracton. The remanng of the paper s organsed as follows. In Secton, the pre-processng step s detaled whle n Secton 3 a novel robust hbrd feature etracton s presented. Epermental results are dscussed n Secton 4 and, fnall, conclusons are drawn n Secton 5.. Pre-processng.1. Word Image Normalzaton At the word mage normalzaton step we frst remove the skew and then resze the word n order to ft n a rectangular bo whle preservng ts aspect rato. The eact postonng of the word n the rectangular bo can be acheved b () usng as a reference pont the geometrc center of the word mage or b () placng the baselne of the word n the center of the rectangular bo. Both word skew and baselne detecton s accomplshed usng the followng methodolog based on horzontal projectons: Let m(,) be the word mage arra havng 1s for foreground and s for background pels, and be the wdth and the heght of the word mage, respectvel. We frst calculate the left and the rght horzontal word projectons LP and RP (see Fgure 1) as follows:

3 LP ( ) m(, ), RP( ) m(, ) (1) Then, we calculate the global ma of LP and RP projectons for L and R. At a net step, we calculate values L1, L and R1, R whch correspond to the nearest values from both sdes of L and R havng LP()<.LP( L ) and RP()<.RP( R ): L1 L R1 R :( LP() <.LP(L) & ( )), [,L] :( LP() <.LP(L) & mn( )), [L, ] :( RP() <.RP(R ) & ( )), [,R ] :( RP() <.RP(R ) & mn( )), [R, ] (a) () R1 + R L1 L θ tan ( ) (3) As shown n Fgure 1b, after word skew correcton, L1 R1 and L R and therefore the baselne s accuratel detected n the L1 - L lmts... Slant Correcton The word slant s chosen as the slant whch gves the mnmum entrop of a vertcal projecton hstogram [3]. The vertcal projecton hstogram s calculated b countng the number of foreground pels n each column of the bnar mage. The dstrbuton s then normalsed to have an area equals to 1. The basc dea can be demonstrated usng a vertcal lne as an eample. When the lne s slanted at an angle, t wll have a low flat dstrbuton. When the lne s uprght, the dstrbuton wll be tall and narrow, whch wll result n a lower entrop measure than for the low flat dstrbuton of the slanted lne. The vertcal projecton hstogram s calculated for a range of slant correcton angles a, where the angle ranges n ± 45. The correcton angle a s measured relatve to the normal. The word slant, a m, s gven as : a m H mn H (4) a ± R N 1 p log p (5) where N s the number of bns n the vertcal projecton hstogram that equals to and p s the probablt of the foreground pel appearng n bn. The word s then corrected b a m usng : tan( a m ) (6) (7) An eample of slant corrected word mage s shown n Fgure b. Fgure 1. Skew correcton of the word mages. (a) The orgnal word mage and the left/rght horzontal word projectons; The word mage wth corrected skew and the horzontal projectons that help to accuratel defne the word baselne. Due to the word skew, the dstrbutons of the left and the rght horzontal word projectons LP and RP ehbt a vertcal offset. The word skew s gven b the followng formula:.3. Stroke Thckness Normalzaton For the stroke thckness normalzaton process, we use an teratve skeletonzaton method presented n [1]. Ths method s smpl an etenson of Zhang and Suen s method []. The skeleton obtaned s not trul 8- connected, snce some non-juncton pels have more than two neghbors, makng the skeleton useless for algorthms that requre ths constrant. Therefore, some pels have to be removed. The skeleton s nspected, and each pel s tested usng a lookup table. The result s a true 8-connected skeleton where onl juncton pels have more than two 8-neghbors (see Fgure c). Fnall, we normalze the stroke thckness b applng a dlaton operator (see Fgure d).

4 (a) where, s() ( ZH) ZH Z s () ZH Z V H, e() ( ZH + 1) ZH ZH, e () ( + 1) ZH ZV (c) In the case of features based on word (upper/lower) profle projectons, the word mage s dvded nto two sectons separated b the horzontal lne t whch passes through the center of mass of the word mage ( t, t ) (see Eq. 9). (d) Fgure. (a) Bnarzed mage; Slant correcton; (c) Skeletonzaton; (d) skeleton dlaton. m(, ) t m(, ) (9) 3. Hbrd feature scheme For the word matchng, feature etracton from the word mages s requred. Several features and methods have been proposed based on strokes, contour analss, zones, projectons etc. [1][][4][17]. In our approach, we emplo two tpes of features n a hbrd fashon. The frst one, whch s based on [], dvdes the word mage nto a set of zones and calculates the denst of the character pels n each zone. The second tpe of features s based on the work n [17], where we calculate the area that s formed from the projectons of the upper and lower profle of the word. (a) Fgure 3. Feature etracton of a word mage based on zones. (a) The normalzed word mage; Features based on zones. Darker squares ndcate hgher denst of character pels. In the case of features based on zones, the mage s dvded nto horzontal and vertcal zones. In each zone, we calculate the denst of the character pels (see Fgure 3). Let Z H and Z V be the total number of zones formed n both horzontal and vertcal drecton. Then, features based on zones f z (), Z H Z V -1 are calculated as follows: z ( ) ( ) e e f ( ) m(, ) (8) s ( ) s ( ) (a) (c) Fgure 4. Feature etracton of a word mage based on word profle projectons. (a) The normalzed word mage; Upper and lower word profles; (c) The etracted features. Darker squares ndcate hgher denst of zone pels. Upper/lower word profles (Eq. (1),(11)) are computed b consderng, for each mage column, the dstance between the horzontal lne t and the closest character pel to the upper/lower boundar of the word mage (see Fgure 4): where where up ( ), t t t, f m(, ) :( m(, ) 1& mn( )), [, t ], else ( ) lo t, f m(, ) t :( m(, ) 1& ()), [t,, t ], else (1) (11) Let P V be the total number of blocks formed n each produced zone (upper, lower). For each block, we

5 calculate the area of the upper/lower word profles denoted as n the followng: e ( ) p up _ ar ( ) up ( ) s ( ) e ( ) p lo _ ar ( ) lo ( ) s ( ) f (1) f (13) where, s() ( PV), e() ( PV 1) PV P + V PV PV and.. P V -1. Fgure 4 llustrates the features etracted from a word mage usng projectons of word profles. The overall calculaton of the proposed hbrd feature vector s gven n Eq. 14. The correspondng feature vector length equals to Z H Z V +P V. e() e() z f () m(, ),... ZHZV s() s() e( ZHZV) p f () f up_ ar () up ( ), ZZ H V... ZZ H V + PV (14) s( ZHZV) e( ZHZV+ PV) p f lo_ ar ( ) lo( ), ZHZV + PV... ZHZV + PV s( ZHZV+ PV) 4. Epermental Results For our eperments, we have used the IAM handwrtng database v3. [1] that s publcl avalable and has been used b several research groups meanwhle [16]. The orgnal database conssts of 1153 solated and labeled words. For a meanngful epermentaton we have used 697 words whch have been correctl segmented as well as each of them havng man nstances. We have splt the used dataset nto a tranng set of 3171 words and a testng set of 3799 words. As t has alread been descrbed n Sectons and 3 we have used a normalzaton step followed b a feature etracton step. Durng ths, the sze of the normalzed word mages used s 3 and 3. In the case of features based on zones, the word mage s dvded nto three (Z H 3) horzontal and thrt (Z V 3) vertcal zones formng a total of nnet (9) blocks wth sze 11 (see Fgure 3). Therefore, the total number of features s nnet (9). In the case of features based on word (upper/lower) profle projectons we keep the same sze of the normalzed mage, whle the mage s dvded nto thrt (3) vertcal zones ( P V 3 ) (see Fgure 4). Consequentl, the total number of features equals to st (6). Combnaton of features based on zones and features based on word profle projectons led to the hbrd feature etracton model (Eq. 14) that uses a total of one hundred and fft (15) features. Moreover, we have tested a combnaton of two dfferent modes of normalzaton (baselne and geometrc center adjustment) precedng the hbrd feature etracton scheme. In ths case the etracted features are doubled ( ). For the partcular classfcaton problem, the classfcaton step was performed usng two well-known classfcaton algorthms, namel the Mnmum Dstance Classfer (MDC) [7] and the Support Vector Machnes (SVM) [5]. Formall, the support vector machnes (SVM) requre the soluton of an optmsaton problem, gven a tranng set of nstance-label pars (, ), 1,,m, n where R and {1, 1} m. The optmsaton problem s defned as follows : mn m 1 T ωω+ C ξ 1 subject to T ( ω φ( ) + b) 1 ξ ξ ω, b, ξ (15) Accordng to ths, tranng vectors are mapped nto a hgher dmensonal space b the functon φ. Then, SVM fnds a lnear separatng hperplane wth the mal margn n ths hgher dmensonal space. For ths search, there are a few parameters that pla a crtcal role at the classfcaton performance. Frstl, the parameter C n Eq. 15, that apples a penalt at the error term. Secondl, the so-called kernel functon denoted as: T K(, j) φ( ) φ( j). In our case, SVM was used n conjuncton wth the Radal Bass Functon (RBF) kernel, a popular, generalpurpose et powerful kernel, denoted as: K(, ) ep( γ ) (16) j j Furthermore, a grd search was performed n order to fnd the optmal values for both the varance parameter (γ) of the RBF kernel and the cost parameter (C) of SVM (see Eq. 15). Table 1 depcts the (%) recognton rate acheved after combnng dfferent normalzaton and preprocessng modes as well as usng ether sngle features or the hbrd feature etracton scheme. We can draw several conclusons. Frst, n all cases the use of the hbrd model outperforms the use of a sngle feature ether based on zones or based on projectons. Second, the skew and slant correcton, as well as the stroke thckness normalzaton pre-processng stages mprove the performance of the classfcaton sstem. Fnall, the best performance s acheved b usng the SVM classfer n the case of the combnaton of two dfferent modes of normalzaton precedng the hbrd feature etracton scheme. The correspondng recognton rate equals to 87,68% and can be consdered one of the hghest performances among the state-of-the-art approaches for offlne cursve handwrtng word recognton. Smlar efforts that have been tested aganst the IAM database have acheved a classfcaton accurac up to 8.76% [9].

6 Table 1. Epermental results PRE-PROCESSING NORMALIZATION FEATURE EXTRACTION Baselne Zones Projectons Eperment Skew correcton Slant correcton Stroke Thnkness normalsaton Geom. Center MDC CLASSIFIER ,54% 76,18% ,36% 69,49% ,6% 8,71% ,% 8,97% ,1% 84,3% ,49% 84,3% 7-8,% 85,39% 8 8,34% 87,68% SVM 5. Conclusons Ths paper proposes an off-lne cursve word handwrtng recognton methodolog that s based on a novel combnaton of two dfferent modes of word mage normalzaton and robust hbrd feature etracton. After a valdaton of the proposed approach wth the reference IAM database we have acheved a performance whch one of the hghest among the state-of-the-art. Our future research wll focus on eplotng new features as well as fuson methods to further mprove the current performance. References [1] D. Bhat, "An evolutonar measure for mage matchng", Proceedngs of the 14 th Internatonal Conference on Pattern Recognton, ICPR 98, volume I, 1998, pp [] M. Bokser, "Omndocument technologes", Proceedngs of the IEEE, 8(7), 199, pp [3] R. Buse, ZQ Lu, "A structural and relatonal approach to handwrtten word recognton", IEEE Transactons on Sstems, Man, and Cbernetcs. 7, 1997, pp [4] S-H Cha, Y-C Shn, S. N. Srhar, "Appromate stroke sequence strng matchng algorthm for character recognton and analss", Proceedngs of the 5 th Internatonal Conference on Document Analss and Recognton (ICDAR 99), 1999, pp [5] C. Cortes, V. Vapnk, "Support-vector network", Machne Learnng, 1995, pp [6] B. Gatos, N. Papamarkos, C. Chamzas, "A bnar tree based OCR technque for machne prnted characters", Engneerng Applcatons of Artfcal Intellgence, 1(4), 1997, pp [7] R.C. Gonzalez, R.E. Woods, Dgtal Image Processng, Addson Wesle, [8] D. Gullevc, C. Y. Suen, "HMM word recognton engne", Fourth Internatonal Conference on Document Analss and Recognton (ICDAR 97), 1997, pp [9] S. Günter, H. Bunke, "An Evaluaton of Ensemble Methods n Handwrtten Word Recognton Based on Feature Selecton", Internatonal Conference on Pattern recognton (ICPR'4), 4, [1] IAM handwrtng database v3., [11] P. Keaton, H. Greenspan, R. Goodman, "Keword spottng for cursve document retreval", Workshop on Document Image Analss (DIA'97), 1997, pp [1] H. J. Lee, B. Chen, "Recognton of Handwrtten Chnese Characters va Short Lne Segments", Pattern Recognton 5 (5), 199, pp [13] Y. Lu, C. Tan, H. Wehua, L. Fan, "An approach to word mage matchng based on weghted Hausdorff dstance", Sth Internatonal Conference on Document Analss and Recognton (ICDAR 1), 1, pp [14] S. Madhvanath, V. Govndaraju, "Local reference lnes for handwrtten word recognton", Pattern Recognton, 3, 1999, pp 1-8. [15] A. Marcolno, V. Ramos, M. Ármalo, J. C. Pnto, "Lne and Word matchng n old documents", Proceedngs of the 5 th IberoAmercan Smpsum on Pattern Recognton (SIARP ),, pp [16] U. Mart, H. Bunke, "The IAM-database: an Englsh sentence database for off-lne handwrtng recognton", Internatonal Journal of Document Analss and Recognton, 5,, pp [17] T. M. Rath, R. Manmatha, "Features for word spottng n hstorcal documents", Proceedngs of the Seventh Internatonal Conference on Document Analss and Recognton (ICDAR'3), 3, pp 18-. [18] B. Verma, P. Gader, W. Chen, "Fuson of multple handwrtten word recognton technques", Pattern Recognton Letters () 9, 1, pp [19] H. Wehua, C. L. Tan, S. Y. Sung, Y. Xu, "Word shape recognton for mage-based document retreval", IEEE Internatonal Conference on Image Processng (ICIP'1), 1, pp [] M. Zhang, C. Suen, Dgtal Image Processng, nd edton, 1987, pp