Incremental MQDF Learning for Writer Adaptive Handwriting Recognition 1

Transcription

1 200 2th Internatonal Conference on Fronters n Handwrtng Recognton Incremental MQDF Learnng for Wrter Adaptve Handwrtng Recognton Ka Dng, Lanwen Jn * School of Electronc and Informaton Engneerng, South Chna Unversty of echnology, Guangzhou, P.R.Chna * E-Mal: lanwen.jn@gmal.com Abstract Wrter adaptaton has been proved to be an effectve approach to mprove the recognton performance of the wrter-ndependent recognzer for a partcular wrter. In ths paper, we propose a wrter adaptve handwrtng recognton approach by ncremental learnng the Modfed Quadratc Dscrmnant Functon (MQDF classfer. We derved the soluton of Incremental MQDF (IMQDF and then present a Dscrmnatve IMQDF (DIMQDF by dervng the soluton of IMQDF n the updated dscrmnatve feature space. Based on IMQDF or DIMQDF, the wrter adaptaton s fnally performed by updatng the MQDF recognzer adaptvely. he expermental results for recognzng handwrtng Chnese characters ndcate that the proposed IMQDF and DIQMDF approaches can reduce as much as 52.7% and 45.38% error rate respectvely on the wrter-dependent dataset whle only have less than 0.8% accuracy loss on the wrter-ndependent dataset. In other words, the proposed IMQDF and DIMQDF based wrter adaptaton approaches can sgnfcantly ncrease the recognton accuracy on wrter-dependent dataset whle only have lmted negatve nfluence for general wrter.. Introducton Wth the emergence of Personal Dgtal Assstants (PDA and of ablet PCs usng pen based nterfaces, the handwrtng recognton accuracy s the key factor n determnng the acceptablty of a handwrtng recognton system and the whole applcaton n whch t s mplemented. Motvated by these, many researchers devoted themselves to the feld of handwrtng character recognton and acheved great progress durng the past 40 years [-3]. However, recent researches [4-6] have shown that the recognton accuracy usng state-of-the-art technques cannot satsfy user s expectatons [7]. he tranng set of a wrter-ndependent recognzer s typcally composed of large sets of data produced by dfferent wrters to acheve a good performance for general wrter, but t s not optmal wth respect to any partcular wrter. A straghtforward dea to ncrease the recognton accuracy for a specfc user s adaptng the wrter-ndependent recognzer to the specfc wrtng style, whch s also known as wrter adaptaton [8-2]. Many the prevous works ether desgn for the small scale (small classes, small vocabulary recognton problem [8-0] or focus on feature transformaton module and gnore the classfer desgn module durng the wrter adaptaton process [-2]. In ths paper, we propose a wrter adaptaton approach for large scale onlne handwrtng Chnese character recognton by adaptvely updatng the Modfed Quadratc Dscrmnant Functon (MQDF recognzer, whch has been successful n achevng state-of-the-art performance for handwrtng character recognton [3-4]. However, the assumpton used n typcal mplementatons of the tradtonal MQDF, that a complete dataset for tranng s gven n advance, conflcts wth the prncples of the wrter adaptaton. o solve ths problem, we have derved a soluton of the MQDF model usng ncremental learnng (henceforth referred to as the IMQDF, whch enables the MQDF technque to be appled to the wrter adaptaton for the frst tme. On the other hand, n practcal, the raw feature s usually transformed to a dscrmnatve feature space to reduce the feature dmenson as well as mprove the recognton accuracy. In ths stuaton, we can drectly apply the IMQDF approach by assumng the dscrmnatve feature space s preserved durng the ncremental learnng process. hs work s supported n part by the research fundng of NSFC (no. U , , GDSP (no , 2007B , 2008A , 2009B /0 $ IEEE DOI 0.09/ICFHR

2 However, ths assumpton s not reasonable and may cause recognton accuracy decrease due to the dscrmnatve feature space s actually changed when the ncremental samples are provded. o solve ths problem, we proposed a Dscrmnatve IMQDF (DIMQDF to adaptvely learn the MQDF model n the updated dscrmnatve feature space. Based on these, the wrter adaptaton s carred out by adaptve updatng MQDF recognzer to learn the specfc wrter s wrtng styles. Expermental results on the wrter-dependent dataset verfy that both of the IMQDF and DIMQDF approaches are very effectve to mprove the recognton accuracy for a partcular wrter, and DIMQDF acheves better performance than IMQDF approach. Moreover, expermental results on the ndependent-dataset show that both of the proposed wrter adaptaton approaches have lmted accuracy loss for the general wrter. he remander of ths paper s organzed as follows: Secton2 frst revews the MQDF approach and then derves the soluton of IMQDF and DIMQDF. he WIMQDF/WDIMQDF approach s proposed n Secton3. And the proposed wrter adaptaton method s presented n Secton4. Secton5 gves expermental results and Secton6 concludes ths paper. 2. Incremental learnng of MQDF 2. MQDF classfer Based on a Bayesan decson rule that classfes the nput pattern to the class as the maxmum a posteror (MAP probablty of all the classes, the quadratc dscrmnant functon (QDF s obtaned assumng a multvarate Gaussan densty and an equal pror probablty for each class. he MQDF proposed by Kmura et al. [3] ncorporates a modfcaton to the QDF through a K-L transform and smoothng the mnor egenvalues to mprove the computaton effcency and classfcaton performance. he dscrmnant functon of a QDF classfer can be represented as [4]: g0 ( x, ω = ( x x ( x x + log Σ, =,, M ( where M s the number of classes and ω represents the th class, and x denotes the mean vector of class ω, D s the dmenson of x. By K-L transform, the covarance matrx can be dagonalzed as: Σ = ΦΛΦ (2 where Λ = dag[ λ,, λd] wth λ, j =,, D, j beng the egenvalues (ordered n decreasng order of Σ, and Φ = [ φ,, φ D ] wth φ, j j =,, D, beng the ordered egenvectors, Φ s orthonormal (untary such that Φ Φ = I. Accordng to (2, the QDF can be rewrtten n the form of egenvectors and egenvalues: g0 ( x, ω = [ Φ ( x x] Λ Φ ( x x + log Λ D D (3 [ ( ] 2 = ϕj x x + log λj λ j= j j= By replacng the mnor egenvalues wth a constantδ, the MQDF classfer can be expressed as: 2 K δ 2 g( x, ω = x x ( [ ϕj( x x] δ j= λj K + log λ + ( D Klogδ j= j (4 where K denotes the number of domnant egenvectors. Snce the tranng of the QDF classfer always underestmate the patterns egenvalues by lmted sample set, the mnor egenvalues become some knd of unstable noses and affect the classfer s robustness. By smoothng them n the MQDF classfer, not only the classfcaton performance s mproved, but also the computaton tme and storage for the parameters are saved. 2.2 Incremental learnng of MQDF (IMQDF Suppose X and Y are two sets of observatons n the feature space, where X s the gven observaton set wth N samples X= {x } (=,, N n M classes, and Y s a set of new observatons wth L ncremental samples Y = {y } j (j=,,l n P classes. It s worthwhle notng that some of classes n Y may be not avalable n X, n other words, some new classes may be ntroduced. hus, the mxed observaton set Z=X Y = {z k } (,,L+N has L+N samples n C classes, where C M and C P. Wthout loss of generalty, we assume that n of the N orgnal samples and l of the L ncremental samples belong to class C (=,, C, therefore, n the updated observaton set, the sample number belongng to the C class s s =n +l. Let x, Σ x, y, Σ represent the th class s mean vector y and covarance matrx of the gven observaton set and the new observaton set respectvely. hen we have n n k = l l k = x = x k (5 y = y k (6 560

3 Σ = x x x x = x x x x (7 x y n n ( k ( k k k n l l ( ( k k k k l Σ = y y y y = y y y y (8 Accordng to the above defntons, each class s mean vector z and covarance matrx Σ z of the mxed dataset can be updated as follows: s n l n ( x + l y = j = k + j = s n + l j= n + l s z z kzk z z s n l n ( x+ l y n ( l x+ y x k xk yk yk n+ l n+ l n+ l n+ l n n l l ( xk xk x xk ( yk yk y y n k k + l n = n+ l l = nl ( yy 2 x x x y y x ( n+ l n l nl ( y ( x y 2 x y x n+ l n+ l ( n+ l z z x y (9 Σ = = + = = Σ + Σ + (0 After updatng the mean vector and covarance matrx of each class, the covarance matrx of each class s dagonalzed by K-L transformaton to obtan the domnant egenvalues and egenvectors. Fnally the MQDF classfer can be updated accordng to Eq. ( Dscrmnatve IMQDF (DIMQDF In practcal, before usng MQDF, a dscrmnatve transform approach, such as Lnear Dscrmnatve Analyss (LDA [5], Prncpal Component Analyss (PCA[4] and General ensor Dscrmnatve Analyss (GDA[6], s usually employed to transform the raw feature to a dscrmnatve feature space to reduce the feature dmenson as well as mprove the recognton performance. After then, the MQDF classfer s performed n the dscrmnatve feature space. Due to the proposed IMQDF only can be used n the preserved feature space, therefore, when the dscrmnatve transform approach s used, the IMQDF approach can t be drectly employed wthout the assumpton that the dscrmnatve feature space s preserved durng the ncremental learnng process, but ths assumpton may cause performance loss. o solve ths problem, we proposed a Dscrmnatve IMQDF (DIMQDF to mprove performance of IMQDF by adaptvely learnng the MQDF classfer n the updated dscrmnatve feature space. Snce the LDA, also known as Fsher Dscrmnatve Analyss (FDA, s wdely used approach n the character recognton applcatons [,3,4,,2,4], n ths secton, we focus on how to ncremental learnng MQDF classfer n the LDA feature space. Suppose x, Σ x, y, Σ represents the th class s y mean vector and covarance matrx of the gven and new observaton sets respectvely n the raw feature space. And W orglda denotes the LDA transform matrx, whch s traned by the gven observaton data. When the ncremental samples are provded, the LDA transformaton matrx s updated as W nclda accordng to the ILDA approach [2]. After then, each class s mean vector y and nclda covarance matrx Σ y nclda of the ncremental data can be computed n the updated dscrmnatve feature space accordng to: y = W y ( -nclda nclda l y nclda ( Wnclda k -nclda ( Wnclda k -nclda Σ = y y y y (2 Smlar wth above steps, each class s mean vector x and covarance matrx Σ nclda x nclda of the gven data can be computed n the new dscrmnatve feature space as: x-nclda = W x (3 nclda l x nclda ( Wnclda xk Wnclda x( Wnclda xk Wnclda x l Wnclda ( xk x( xk x Wnclda Wnclda xwnclda Σ = = = Σ (4 Snce all the class s mean vector and the covarance matrx of the gven data and ncremental data are obtaned, each class s updated mean vector and covarance matrx of mxed dataset can be computed by nsertng Eq. (~(4 to Eq. (9 and Eq. (0 respectvely. After obtanng the domnant egenvalues and egenvectors of each class s covarance matrx by K-L transformaton, the MQDF classfer can be updated n the updated dscrmnatve feature space accordng to Eq. (4. 4. Wrter adaptaton Fgure depcts a flow dagram of the proposed WDIMQDF/DIMQDF based wrter adaptaton approach. he ntal MQDF modelng s traned usng the full wrter-ndependent tranng set. he preprocessng and feature extracton are performed frstly. After then, the feature space s transformed to a dscrmnatve feature space to reduce the feature dmenson and mprove the recognton accuracy by LDA. hereafter, to generate the MQDF recognzer for the wrter-ndependent dataset, the adaptaton 56

4 Wrter-ndependent ranng set Preprocessng & Feature Extracton LDA ransform Estmate each class s mean and covarance MQDF recognzer LDA {W orglda} ILDA {W nclda} IMQDF modelng Re-estmate each class s mean and covarance n updated dscrmnatve space Wrter-ndependent ranng set LDA ransform Estmate each class s mean and covarance Estmate each class s mean and covarance Smoothng of mnor egenvalues Updated MQDF recognzer Fgure An overvew of wrter adaptaton process (dashed lnes show the flow of events for wrter adaptaton approach tres to fnd a statstcal model and estmates the model s parameters to ft the feature dstrbuton of each class. When addtonal samples for a partcular wrter are provded, the LDA feature space s updated frst, and then the parameters of each class s feature he evaluaton of the proposed wrter adaptaton approach was conducted usng two subsets of SCU- COUCH handwrtng dataset [8], GB subset (henceforth referred to as CouchGB dataset and Word8888 subset. he GB subset contans 68 dstrbuton are re-estmated n the updated wrters samples of 3755 categores of Chnese dscrmnatve feature space to learn the wrtng styles of the partcular wrter adaptvely. Fnally, the updated MQDF recognzer s generated to mprove the recognton accuracy for the partcular wrter. characters, whereas the Word8888 subset conssts of 30 wrters samples of 8888 categores of most frequently used handwrtten words. Fgure2 shows some samples of the character 我 from the GB In the wrter-ndependent recognzer tranng subset and the words contans character 我 n process, the feature of the ndependent tranng dataset Word8888 subset. It s worthwhle notcng that many s frst extracted usng an 8-drectonal feature dfferent words contan same character. For example, extracton method [7]. And then, the parameters of all of the words n fgure2 (b contan the character each class s dstrbuton are estmated to generate the 我. hs ndcates that data collected n ths way, n wrter-ndependent MQDF recognzer accordng to the whch the same character s collected for many tmes methods descrbed n Secton 2.. n accordance wth the context of the word corpus, he accuracy of a recognton system for a provdes us wth partcularly realstc wrterndependent ncremental handwrtten samples. partcular wrter can be mproved by adjustng the recognzer s model to match the dstrbuton of the partcular wrter s handwrtng more closely, rather than usng the more generalzed models traned for a large number of wrters. herefore, our wrter adaptaton approach begns by adaptve updatng the new LDA feature space usng the ncremental samples. After transformng the gven samples dstrbuton parameters nto updated dscrmnatve feature space, we also compute the dstrbuton models of the wrterdependent dataset n the update dscrmnatve feature space. In ths way, the dstrbuton models of both the (a Character 我 (b Words contan 我 wrter-ndependent and wrter-dependent datasets are Fgure2: Samples from the SCU-COUCH database obtaned n the updated dscrmnatve feature space. o buld a general purpose classfer, we randomly hereafter, the dstrbuton models of the mxed selected 34 sets of data from the CouchGB to buld dataset can be generated accordng to Eq. (9 and (0. a wrter-ndependent baselne classfer, wth the Fnally, the updated MQDF recognzer s developed remanng 34 sets used to test the performance of the accordng to Eq. (4 usng the dstrbuton models of baselne classfer, as well as to evaluate the nfluence the mxed dataset. after the adaptaton. In order to generate partcularly 5. Experments and Analyss 5. Data preparaton and expermental setup realstc wrter-dependent ncremental handwrtten samples, we frst manually segmented all of the handwrtten words samples nto separate characters, whch results n 2078 categores of 9595 solated Chnese characters, to form a new dataset, the 562

5 Fgure 4 Performance of dfferent wrter adaptaton approaches for each partcular wrter IncCouchDB dataset. For each partcular wrter s handwrtten samples from the IncCouchDB dataset, we randomly selected 50% of each category s data for adaptve learnng the IMQDF model and then usng the remanng 50% to test the wrter adapton performance. It s worth notng that the two datasets don t share any common wrters. 5.2 Baselne Performance on CouchGB and IncCouchDB before wrter adaptaton After the classfer was traned by 34 sets of CouchGB data, ts performance was evaluated on both the testng sets of CouchGB dataset and IncCouchDB dataset respectvely. able shows the average recognton rate of the baselne MQDF classfer on these two datasets. he results demonstrate that, snce many of the wrtng styles of IncCouchDB are unseen n the tranng dataset, the recognton accuracy on IncCouchDB testng datasets s much lower than whch on CouchGB testng dataset. In other words, the recognton accuracy for the specfc wrter should be further mproved. able Baselne performance on two datasets op op5 op0 CouchGB 96.03% 99.30% 99.49% IncCouchDB 88.56% 95.93% 96.82% 5.3 Performance comparson wth dfferent wrter adaptaton approaches In ths experment, we examned the performance of the varous wrter adaptaton approaches based on ILDA [2] and the proposed IMQDF and DMQDF approaches on the wrter-dependent IncCouchDB dataset. It worth notng that snce the MQDF classfer s not used n ths experment n prevous method [2], we mplement an ILDA plus MQDF based wrter adaptaton approach (we refer t as ILDAMQDF, where the LDA transform matrx s frst updated usng the new ncremental data accordng to ILDA, and then the MQDF classfer model s transformed to the updated LDA feature space wthout employng the IMQDF approach. From the results gven n table 2, t can be seen that the ILDA and ILDAMQDF based wrter adaptaton can only reduce less than 26% error rate and acheve less than 9.0% recognton rate for the partcular wrter, whle the proposed IMQDF and DIMQDF based wrter adaptaton approaches can reduce above 45% error rate and acheve above 93.75% recognton rate. In other words, they can mprove the recognton accuracy from 88.56% to 93.75% and 94.59% respectvely. On the other hand, t can be also found that the DIMQDF sgnfcantly outperform IMQDF, ths ndcates that by ncrementally learnng the MQDF model n the updated dscrmnatve feature space, the performance of the wrter adaptaton can be further mproved. Generally, the expermental results demonstrate that the proposed IMQDF based wrter adaptaton approach s very effectve to mprove the recognton accuracy for the specfc wrter, and by adaptve learnng the MQDF recognzer n the updated dscrmnatve feature space, the performance of the wrter adaptaton can be further mproved. able2 Recognton performance of varous wrter adaptaton approaches Approach Intal Adapted Error rate reducton ILDA[2] 82.64% 87.08% 25.62% ILDAMQDF 88.56% 9.0% 2.39% IMQDF 88.56% 93.75% 45.38% DIMQDF 88.56% 94.59% 52.7% o examne the nfluence of wrter adaptaton approaches for total 30 dfferent wrters nvolved, fgure 4 shows the error rate reducton of these four adaptaton approaches for each partcular wrter. From the result gven n fgure 4, we can see that the proposed wrter adaptaton approaches sgnfcantly outperforms ILDA and ILDAMQDF approach for every partcular wrter. In addton to evaluatng the performance mprovement of the IMQDF and DIMQDF based wrter adaptaton approaches on the wrter-dependent dataset; we also examned the mpact of the proposed wrter adaptaton approaches for the general purpose 563

6 wrter, snce we don t expect the adaptaton to the partcular wrter s handwrtng style s at the cost of losng too much generalty for other wrter styles. he expermental result demonstrates that the accuracy loss on wrter-ndependent CouchGB dataset are only 0.02%, 0.8%, for IMQDF and DIMQDF based wrter adaptaton approaches respectvely. It ndcates that whle the IMQDF and DIMQDF approaches can sgnfcantly mprove the accuracy for dfferent specfc wrters, they have very lmted negatve nfluence to general wrter. 6. Concluson In ths paper, we proposed a new wrter adapton method to mprove the recognton of handwrtten Chnese character by ncremental learnng of MQDF model. We proposed the soluton of IMQDF for the frst tme and then presented the DIMQDF approach. Based on IMQDF or DIMQDF, the wrter adaptaton was fnally carred out by updatng the MQDF classfer model adaptvely. Expermental results demonstrated that both of the proposed IMQDF and DIMQDF based wrter adaptaton approaches can sgnfcantly ncrease the recognton accuracy for a specfc wrter, whle at the same tme, havng lmted negatve mpact for the general wrter. References [] C. Lu, S. Jaeger, M. Nakagawa, Onlne Recognton of Chnese Characters: he State-of-the-Art, IEEE rans. PAMI, 26(2: 98-23, [2] R. Plamondon and S.N. Srhar, On-lne and off-lne handwrtng recognton: A comprehensve survey, IEEE rans. PAMI, 22(: , [3]. Long, L. Jn, A Novel Orentaton Free Method for Onlne Unconstraned Cursve Handwrtten Chnese Word Recognton, ICPR 2008: [4] K. Dng, G. Deng, L. Jn, An Investgaton of Imagnary Stroke echnque for Cursve Onlne Handwrtng Chnese Character Recognton, ICDAR 2009:53-535, [5] D. Wang, C. Lu, et al., CASIA-OLHWDB: A Database of Onlne Handwrtten Chnese Characters, ICDAR2009: [6] C.Lu, X.Zhou, Onlne Japanese character recognton usng trajectory-based normalzaton and drecton feature extracton, IWFHR2006: , [7] R.A. Cole, J. Maran et al., Survey of the state of the art n human language technology, Cteseer, 997. [8] S.D.Connel, and A.K. Jan, Wrter Adaptaton of Onlne Handwrtng Models, IEEE rans. PAMI., 24(3: pp , [9] J.J. LaVola, and R.C.Zeleznk, A Practcal Approach for Wrter-Dependent Symbol Recognton Usng a Wrter- Independent Symbol Recognzer, IEEE rans. PAMI., 29(: , [0] Vuor, V. and eknllnen Korkeakoulu, Adaptve methods for on-lne recognton of solated handwrtten characters, Helsnk Unversty of echnology, 2002 [] Z. Huang, K. Dng, L. Jn, Wrter Adaptve Onlne Handwrtng Recognton Usng Incremental Lnear Dscrmnant Analyss, ICDAR2009: [2] L. Jn, K. Dng, Z. Huang, Incremental Learnng of LDA model for Chnese Wrter Adaptaton, accepted by NeuroComputng 200. [3] F.Kmura, K.akashna et al, Modfed quadratc dscrmnant functons and the applcaton to Chnese character recognton, IEEE rans. PAMI., 9: 49-53, 987. [4]. Long and L. Jn, Buldng Compact MQDF Classfer for Large Character Set Recognton by Subspace Dstrbuton Sharng, Pattern Recognton, 4(9:pp , [5]R.O. Duda, P.E. Hart and D.G. Stork, Pattern Classfcaton, 200 [6] D. ao, X. L, et al., General ensor Dscrmnant Analyss and Gabor Features for Gat Recognton, IEEE rans. on PAMI, 9(0 ( [7] Z. Ba and Q. Huo, A Study On the Use of 8- Drectonal Features For Onlne Handwrtten Chnese Character Recognton, ICDAR 2005: [8] SCU-COUCH dataset, Webste: 564