Spiral Recognition Methodology and Its Application for Recognition of Chinese Bank Checks

Transcription

1 Spial Recognition Methodology and Its Application fo Recognition of Chinese Bank Checks Hanshen Tang 1, Emmanuel Augustin 2, Ching Y. Suen 1, Olivie Baet 2, Mohamed Cheiet 3 1 Cente fo Patten Recognition and Machine Intelligence, Concodia Univesity, 1455 de Maisonneuve Blvd. West, Montéal, Québec H3G 1M8, Canada {hstang, suen}@cs.concodia.ca 2 Atificial Intelligence and Image Analysis (A2iA), 40 bis, ue Fabet, Pais, Fance {ea, ob}@a2ia.com 3 École de Technologie Supéieue, 1100 ue Note-dame Ouest, Montéal, Québec H3C 1K3, Canada Mohamed.Cheiet@etsmtl.ca Abstact. This pape pesents the spial ecognition methodology with its application in unconstained handwitten Chinese legal amount ecognition in a pactical envionment of a CheckReade. This pape fist descibes the failed application of neual netwok - hidden Makov model hybid ecognize on Chinese bank check legal amount ecognition, and explains the easons fo the failue: the neual netwok - hidden Makov model hybid ecognize can not handle the complexity in the taining fo Chinese legal amounts. Then a spial ecognition methodology is pesented. This methodology enables the system to incease its ecognition powe (both the ecognition ate and the numbe of ecognized chaactes) duing the taining iteations. Some expeiments wee done to show that the spial ecognition methodology has a high pefomance in the ecognition of unconstained handwitten Chinese legal amounts. The ecognition ate at the chaacte level is 93.5%, and the ecognition ate at the legal amount level is 60%.Combined with the ecognition of coutesy amount, the oveall eo ate is less than 1%. Keywods: spial ecognition methodology, offline handwiting ecognition, Asian chaacte ecognition 1. Intoduction Recognition of unconstained handwitten Chinese chaactes has been a challenging topic fo a while. Automatic pocessing of legal amounts handwitten on Chinese bank checks is one of the most pactical and pomising applications. A lot of eseach has been done [1-3]. Because of the lage size of the chaacte set, vaiety in fonts and witing styles, the complex stuctues of chaactes, and the ambiguity in gamma, the poblem emains vey difficult. Thee is not yet a viable ecognition poduct in the industy fo the ecognition of unconstained handwitten Chinese scipt. Neual netwok - hidden Makov model (NN-HMM) hybid ecognize has a good pefomance in the ecognition of legal amounts handwitten in Latin languages. The hybid ecognize can handle the uncetainty in segmentation of the handwitten cusive scipt vey well. A2iA CheckReade is based on it, and makes vey good esult [4]. Howeve, the application of NN-HMM hybid ecognize in the ecognition of legal amounts handwitten in Chinese was not successful, due to the complexity of the taining fo unconstained handwitten Chinese chaactes. Moe details ae pesented in Section 2. In ode to solve the poblem, we intoduced a spial ecognition methodology, as illustated in Figue 1. In Phase 1, the system geneates annotations of chaacte samples. In Phase 2, the annotation esults ae analyzed (semi-automatically) and manually coected (if necessay). In Phase 3, a neual netwok chaacte ecognize is tained with the annotation esults. Finally in Phase 4, the tained neual netwok chaacte ecognize pocesses the input data, and then passes the esults to a hidden Makov model legal amount ecognize fo the final ecognition esults. As the pocess iteates, the numbe of ecognized classes is inceasing, while the numbe of annotation chaacte samples is also inceasing. Details and its application ae pesented in Section 3.

2 II. Annotation & Analysis I. Annotation Numbe of classes III. Taining IV. Recognition & Analysis Fig. 1: Spial ecognition methodology. Fig. 2: HMM pocessing of the Fench wod et. [5] 2. Neual Netwok Hidden Makov Model Hybid Recognize A neual netwok chaacte ecognize handles well the vaiety of shapes, fonts, and handwiting styles, as long as enough data is fed to the netwok duing the taining phase. Hidden Makov model wod ecognize can handle the uncetainty in segmentation of the cusive handwitten scipt. Figue 2 illustates how the HMM wod ecognize pocesses the Fench wod et. Fig. 4: Taining and testing of a NN-HMM hybid ecognize. Fig. 3: NN-HMM hybid ecognize. [5] Thus NN-HMM hybid ecognize has a good pefomance in the ecognition of legal amounts handwitten in Latin languages. Figue 3 illustates one such system: the NN-HMM hybid ecognize used in A2iA Check- Reade [4]. Figue 4 shows vitually the idea of the taining and ecognition of the system, although the system is not actually implemented in the exact way. The neual netwok ecognize shown hee is a pobabilistic neual netwok ecognize. It poduces the chaacte ecognition esults as chaacte candidate lists with confidence values. The hidden Makov model ecognize takes the candidate lists and poduces the wod ecognition esults as wod candidate lists with confidence values. In the taining phase, the tue sequences of wods ae passed to the fowad-backwad pocess of the HMM wod ecognize. Then, the expected chaacte esults can be computed. They ae then used to conduct the back popagation of the NN chaacte ecognize. Eventually, the system can achieve ecognition ates vaying fom 65% to 85% (depending on counties and quality of the bank check images) with the eo ates as low as 0.1% [4]. Table 1: Chinese chaacte set fo legal amounts.

3 Howeve, when we tied to build an application of the NN-HMM hybid ecognize on Chinese chaactes, the expeimental esults wee not satisfactoy. The ecognition ate at the legal amount level was aound 27%. The taining of the ecognize did not convege. We analyzed the poblem, and found the following easons: 1) Chinese chaactes have moe complex stuctues. Some diffeent vesions of chaactes have the same meanings. Many chaactes can be futhe split into adicals, while thee ae some common adicals in diffeent chaactes. Table 1 shows the chaacte set fo Chinese legal amounts. Figue 5 illustates that some of the chaactes can be futhe split into adicals. Moeove, some adicals themselves ae chaactes. These make accuate segmentation at chaacte level vey difficult, and thus significantly inceases the complexity of the candidate lists that the HMM ecognize need to pocess. amount. This makes the fowad-backwad pocess duing the taining phase of the HMM ecognize vey difficult to stabilize. 4) The NN-HMM hybid ecognize is tained based on the maximum mutual infomation estimation. The following disciminant taining citeion is applied: log R ( Λ) = log O λ ) log O mw) mw) w whee P O λ ) is the pobability of a tue path, P ( ( O mw) mw ) and is the pobability of all the othe possible paths. The taining pocess intends to maximize log R ( Λ). Howeve, if thee ae too many possible paths, w log P ( O mw) mw) becomes excessively lage, and eventually stops pope taining. Unfotunately, this is exactly the case we had. Since the NN-HMM hybid ecognize could not be tained popely, we intoduced the spial ecognition methodology. 3. Spial Recognition Methodology Fig. 5 Chinese adicals that fom chaactes. The chaactes ae on the left of the aows. The adicals ae on the ight of the aows. Some of the adicals themselves ae also chaactes. Figue 6 shows a gaph of segmentation paths fo a Chinese legal amount. The complexity of the gaph is elatively low. One can easily find many moe complicated cases. Fig. 6: Gaph of segmentation paths fo a Chinese legal amount. The coect segmentation path is bolded. 2) Fo legal amounts, each Chinese chaacte itself can fom a wod, which makes it difficult to build a wod dictionay. Afte chaacte ecognition, the system diectly poceeds to amount level ecognition with the HMM ecognize. Usually, a legal amount consists of a long chaacte sequence. 3) The gamma fo Chinese legal amounts is ambiguous. Fo a given numeic amount, thee can be moe than tens of possible chaacte sequences in Chinese legal The idea of the spial ecognition is incemental: tain and develop the system step by step. As shown in Figue 1, the spial ecognition methodology has 4 phases: annotation, annotation analysis, taining, and ecognition. Each phase is dependent on its pevious phase. Konai et al. intoduced an iteative method to educe human involvement to pepae taining data [6]. They fist manually built a bootstap set, and then tied to enlage the set with some techniques. Ou methodology is moe obust and moe automatic. Gamma and contextual infomation is used to aid the automatic pocess. Although some manual analysis duing the pocess may be equied, the human involvement is futhe educed. We built a gamma-contextual tool fo Chinese legal amount. The gamma tool can accomplish two tasks: One is to check whethe a sequence of hypothetical chaacte / adical obsevations is a valid legal amount; if yes, the equivalent numeic amount is etuned. The othe one is to geneate all possible sequences of chaactes / adicals with a given numeic amount. Duing the ecognition phase, the tool can help the ecognize to bypass some of the syntactically o contextually impossible sequences of hypothetical chaacte / adical obsevations, which esult fom ove-segmentation and unde-segmentation, etc. It may also be used to combine edundant sequences of hypothetical chaacte / adical obsevations due to the pesentations of adicals (efe to Figue 5). Duing the taining phase, the tool can help to diect the ecognition system to coect and stable states. It may also be used to extend the chaacte level ecognize to a adical level ecognize (efe to Figue 5).

4 3.1 Initial iteation In the fist iteation of the pocess, the annotation tool has not much pio knowledge. With the segmentation infomation (including segmentation points, estimated chaacte width, and inte-chaacte spaces, etc.) and gamma and contextual infomation (possible chaacte sequences geneated fo a given numeic amount of the bank check), the automatic annotation tool can poduce annotated chaacte samples fo neat and simple bank checks. All the checks with noises and uncetainty in segmentation ae ejected. Figue 7 shows a snapshot of the annotation esult. 3.2 Following Iteations In the second and following iteations of the pocess, annotation has gatheed moe pio knowledge. With the segmentation infomation, gamma and contextual infomation, and ecognition infomation, the automatic annotation tool can poduce moe annotated chaacte samples fo moe bank checks. Figue 9 shows a segmentation-gamma gaph fo the legal amount on a bank check. With the gaph, annotated chaacte samples can be geneated of bette quality. The annotation analysis, taining, and ecognition phase is simila to those of the initial iteation. Analysis shows that the numbe of annotated samples, the quality of the annotation, and the ecognition powe ae all inceasing duing the iteations. Figues 10 and 11 show the ecognition esult of a bank check handwitten in Chinese. Fig. 7: Annotation esult. The uppe pat is the image of the legal amount on a neat bank check. The lowe pat is the annotated chaacte samples. Then an annotation analysis tool is applied, as shown in Figue 8. In this stage, the annotation accuacy is not vey high. But the qualities of the annotations fo some key chaactes, such as dolla, only, ten, and hunded ae high enough to tain the neual netwok chaacte ecognize. The aveage annotation eo ate fo these chaactes is aound 4%. If the annotation quality is not good enough, manual coection may be necessay. Afte annotation analysis and manual coection (if necessay), the annotated chaacte samples ae fed to the neual netwok chaacte ecognize. Then the ecognition phase is the same as the nomal NN-HMM hybid ecognize. Fig. 9: Segmentation-gamma gaph fo a legal amount image. The numbes in the vetices ae tansition states. The edges ae labeled with the numbe associated with the chaacte object indices. This diected gaph show all the possible segmentation fo a give legal amount. Fig. 10: Input bank check image. Fig. 8: Annotation analysis tool. The uppe pat of the window shows the annotated chaactes. The lowe pat of the window shows fom which oiginal legal amount a selected chaacte is.

5 of annotated chaacte samples inceased fom 221k to 315k fo the taining set, and inceased fom 72k to 79k fo the test set. Figues 14 and 15 show the confusion matices afte the second and the eighth iteations, espectively. It s obvious that afte seveal iteations, the confusion among diffeent chaactes deceased. Fo key chaactes, such as dolla, only, ten, and hunded, the changes wee elatively small (aveage 7% incease). Howeve, fo the othe chaactes, the ecognition ates incease apidly (aveage 15% incease). Fig. 11: Recognition esult fo the image in Figue 10. The ecognition esult option 0 is , which is the coect numeic value. 4. Expeimental esults and analysis Afte 8 iteations of the pocess on the database of a taining set of 47.8 thousand eal bank checks, and a test set of 12 thousand, the aveage annotation eo ate educes to 3%. The ecognition ate fo annotated samples is 90.66% fo the top candidate. The eal ecognition at the chaacte level is estimated by Annotation Recognition Rate Real Recognition Rate = 1 Annotation Eo Rate. Thus at the chaacte level, the estimated eal ecognition ate is 93.5% fo the top candidate, while the amount level ecognition ate is 60% fo the top candidate, and 76% fo the top 4 candidates. Combined with the ecognition of the coutesy amount, the ecognition ate is 85%, with an eo ate of 1%. Figue 12 shows the chaacte level ecognition ate inceased fom 79.4% to 93.5% duing the 8 iteations. Figue 13 shows that the numbe Fig. 12: Recognition ate at chaacte level. Fig. 13: Numbe of annotated chaacte samples Fig. 14: Confusion matix afte 2 nd iteation. Classes 10, 11, 12, 15, and 18 ae ten, hunded, thousand, dolla and only, espectively. Class 14 is "hunded billion, which we have neve seen in a eal bank check. Consequently, its ecognition ate is always 0.

6 Fig. 15: Confusion matix afte 8 th iteation. Classes 10, 11, 12, 15, and 18 ae ten, hunded, thousand, dolla and only, espectively. The eason that the incease of the ecognition ates of key chaactes was not significant is the following: Key chaactes wee bette annotated in the initial iteation, and consequently, bette tained. Howeve, fo the othe classes, the powe of the spial ecognition methodology is clealy illustated. Paticulaly, the ecognition ate of Class 19 (fo unde-segmented obsevations) inceased 34.1%. This means that the system was well tained towads tue chaactes against segmentation eos, and thus the oveall legal amount ecognition ate was inceased. 5. Conclusion Neual netwok hidden Makov model hybid ecognize is poweful. Howeve, it can not handle excessively complicated cases in the taining phase. The spial ecognition methodology is intoduced to solve the poblem. Moe contextual infomation can be extacted fom the bank check images. The NN-HMM hybid ecognize is tained step-by-step, and eventually it achieves a bette ecognition pefomance. The ecognition ate is 93.5% at the chaacte level, and 60% at the amount level. The automatic annotation stategy looks quite pomising, which enables the system to be tained with moe samples fo each chaacte, as well as with moe vaiety of chaactes, if a pope gamma and vocabulay ae given. The system can be futhe extended to ecognize Chinese adicals (as mentioned in Section 3), which is a pomising appoach fo geneal-pupose handwitten Chinese chaacte ecognition. Acknowledgement The authos would like to expess thei thanks to le Fonds Québécois de la echeche su la natue et les technologies fo the funding to the student eseache, and thanks to Pofesso G. Stamon fo his help fo the initiation and the poceeding of the collaboation poject. Refeences [1] J. W. Tai, A syntactic-semantic appoach fo descibing Chinese chaactes, Compute Pocessing of Chinese and Oiental Languages, Vol. 1, No.3, Chinese Language Compute Society, San Fancisco, May 1984, pp [2] H. Chen, L. Gaeth, Y. Wu, and R. Zitseman, Segmentation and ecognition of continuous handwiting Chinese text, Advances in Oiental Document Analysis and Recognition Techniques, Wold Scientific, Singapoe, 1998, pp [3] M. L. Yu, P. C. K. Kwok, C. H. Leung, and K. W. Tse, Recognition of Chinese bank cheque amounts, Intenational Jounal on Document Analysis and Recognition, Vol. 3, Issue 4, Spinge, Belin, 2001, pp [4] N. Goski, V. Anisimov, E, Augustin, O. Baet, and S. Maximov, Industial bank check pocessing: the A2iA Check- Reade, Intenational Jounal on Document Analysis and Recognition, Vol. 3, Spinge, Belin, 2001, pp [5] Y. H. Tay, P.-M. Lallican, M. Khalid, S, Kne, and C. Viad-Gaudin, An Analytical Handwitten Wod Recognition System with Wod-level Disciminant Taining, Poceeding of ICDAR 2001, Seattle, 2001, pp [6] A. Konai, K. Mohiuddin, and S. Connell, Recognition of cusive witing on pesonal checks, Poceeding of the Fifth Intenational Wokshop on Fonties in Handwiting Recognition (IWFHR-5), Colcheste, 1996, pp