202 International Conference on Frontier in Handwriting Recognition Building a Compact On-line MRF Recognizer for Large Character Set uing Structured Dictionary Repreentation and Vector Quantization Technique Bilan Zhu and Maaki Nakagawa Department of Computer and Information Science, Tokyo Univerity of Agriculture and Technology, Tokyo 84-8588, Japan E-mail {zhubilan, nakagawa}@cc.tuat.ac.jp Abtract Thi paper decribe a method for building a compact on-line Markov random field (MRF) recognizer for large handwritten Japanee character et uing tructured dictionary repreentation and vector quantization (VQ) technique. The method plit character pattern into radical, whoe model by MRF are hared by different character uch that a character model i contructed from the contituent radical model. Many ditinct radical are hared by many character with the reult that the torage pace of model dictionary can be aved. Moreover, in order to further compre the parameter, we employ VQ technique to cluter parameter et of the mean vector and covariance matrixe for MRF unary feature and binary feature a well a the tranition probabilitie of each tate into group. By haring a common parameter et for each group, the dictionary of the MRF recognizer can be greatly compreed without recognition accuracy lo.. Introduction Due to the development and proliferation of penbaed or touch-baed input device uch a tablet terminal, mart phone, electronic whiteboard and digital pen (e.g., Anoto pen), on-line handwritten character recognition i reviving even larger attention than before. Although character claifier with high recognition accuracy have been reported [-5], the demand for peeding up recognition and mall memory conumption i very high for portable device a well a dek-top application for which handwriting recognition i incorporated a one of module. Performance of thee relatively mall device require a handwriting recognition ytem a mall a poible and recognition peed a fat a poible while maintaining high accuracy. Even for a dektop PC with relatively high device performance, recognition peed within 0.5 econd per page and memory conumption within 0 MB are required in actual application. Therefore, we need to refine the recognition cheme to reduce the ytem memory conumption. Chinee, Japanee or Korean have thouand of different categorie and their large character et i problematic not only in recognition rate but alo in recognition peed and memory conumption. We have propoed a robut on-line handwritten Japanee character recognition method uing a MRF model [4, 5]. Although it realize high recognition performance, however, the MRF recognizer require about 20MB memory conumption due to the larger character et of 7,097 character categorie within which included are 2,965 Kanji character of the JIS level- tandard and 3,390 Kanji character of the level-2 tandard a well a kana, Englih alphabet and o on. After combining the MRF recognizer with an off-line recognizer by the modified quadratic dicriminant function (MQDF) and linguitic context proceing at the tring recognition tep, the recognition ytem would require more larger memory conumption becaue of the large memory ize for the MQDF dictionary and the linguitic context dictionary. Therefore, we need to compre the memory ize a greatly a poible while maintaining high accuracy. In egmentation-free HMM-baed Englih word recognition, each letter i modeled a a HMM, word HMM are contructed by concatenating letter HMM during recognition [6-8]. We can regard each radical of Japanee character a an Englih letter and each Japanee character a an Englih word, then we can 978-0-7695-4774-9/2 $26.00 202 IEEE DOI 0.09/ICFHR.202.88 55
apply the Englih word recognition method to recognize Japanee character. However, the radical in Japanee character are different from the Englih letter largely, where the radical with the ame type from different Japanee character may have different ize, vertical-horizontal ratio and poition with the reult that they have very different covariance matrixe among MRF model. It reult in low recognition performance if we merge the radical MRF model with different covariance matrixe. We preented an effective tructured templatebaed on-line recognizer [9]. However, it did not need to conider the covariance matrixe of feature point, o that there wa no problem uch a MRF recognizer. Nakai et al. [0-2] preented a tructured HMMbaed on-line Japanee recognizer, where ubtroke HMM were ued to contruct the character model by concatenating them. However, only few character categorie (06 categorie) were ued o that it did not caue the problem that the radical with the ame type from different Japanee character have very different covariance matrixe. Moreover, only haring the common parameter of ubtroke limit the compreion effectivene. Long et al. [3] contructed a compact off-line MQDF recognizer by VQ technique. They plit each two adjacent parameter element of feature vector a a et, and clutered the parameter et into group, and then by haring a common parameter et for each group, the dictionary of the recognizer wa greatly compreed. In our on-line MRF recognizer, there are many imilar unary feature and binary feature among character or radical. Therefore, we can plit the parameter of each mean vector, thoe of each covariance matrix for unary feature and binary feature, a well a thoe of the tranition probabilitie of each tate a a et, and cluter the parameter et into group, with each group haring a common parameter et. It can reult in more effective compreion. In thi paper, we preent a method for building a compact on-line MRF recognizer for large handwritten Japanee character et, where the radical model are hared by different character, and a character model i contructed from the contituent radical MRF model. We invetigate how to normalize the ize and poition of radical with the ame type from different character. Moreover, we employ VQ technique to cluter the parameter et of the mean vector and the covariance matrixe a well a the tate tranition probabilitie for MRF into group. By haring a common parameter et for each group, the dictionary of the MRF recognizer can be greatly compreed without recognition accuracy lo. The ret of thi paper i organized a follow: Section 2 give an overview of our online handwritten character recognition method. Section 3 preent the contruction of our tructured MRF dictionary. Section 4 decribe the dictionary compreion by VQ. Section 5 preent the experimental reult and Section 6 draw our concluion. 2. Recognition ytem overview We linearly normalize an input pattern by converting the pen-tip trace pattern to a tandard ize, preerving the horizontal-vertical ratio. After normalization, we extract feature point uing the method developed by Ramner [4]. Firt, the tart and end point of every troke are picked up a feature point. Then, the mot ditant point from the traight line between adjacent feature point i elected a a feature point if the ditance to the traight line i greater than a threhold value. Thi election i done recurively until no more feature point are elected. Thi feature point extracting proce i hown in Fig. (a). The extracted feature point repreent the tructure of a pattern. They are effective and more efficient to proce compared with proceing all the pen-tip point, a wa done in previou tudie [6-8]. We et feature point from an input pattern a ite S={, 2, 3,, I } and tate of a C a label L={l, l 2, l 3,,l J }. The method recognize the input pattern by aigning label to the ite to make the matching between the input pattern and each C uch a F={ = l, 2 = l, 3 = l 3,, 9 = l 8, 0 = l 8 }, a hown in Fig. (b). The notation F i a configuration and denote a mapping from S to L. We employ the coordinate of feature point a unary feature and the difference in coordinate between the neighboring feature point a binary feature. We then ue a MRF model to match the feature point with the tate of each character cla and obtain a imilarity for each one. We then elect the each character cla with the larget imilarity a the recognition reult [4, 5]. The neighborhood ytem of MRF model i et according to the ucceive adjacent feature point in writing order. We define a linear-chain MRF for each C, a hown in Fig. (c), where each label ha a tate and each tate ha three tranition. Therefore, the energy function i a follow: 56
E( O, F C) = E( O F, C) + E( F C). = I [ log P( O l log P( O l ], l log P( l l i i i i i i i i i= () (a) (b) where l i the label of a C aigned to i, i unary feature vector extracted from ite i, and O i the i O i j i the binary feature vector extracted from the combination of i and j. The maller the energy function in () become, the larger the imilarity between the input pattern and a C. Each C ha a linear-chain MRF, and the ytem ue the Viterbi earch to match feature point of the input pattern with tate for the MRF model of each C and to find the matching path with the mallet energy in () for each C. The unary feature vector O i comprie X and Y coordinate of i. The binary feature vector O ha i i two element (dx: X coordinate of i - X coordinate of i-, dy: Y coordinate of i - Y coordinate of i- ). Gauian function are ued to etimate P( O l, and i P( O l, l. P( l l i i ii i i i i etimated a follow: (c) Figure. Feature point extraction and labeling and linear-chain MRF. P( l i P( l Number of tranition from l to l i i l = i. (2) Number of ite aigned l l 0 = Number of Number of aigned l i (a) To train the MRF of each C, we firt initialize the feature point of an arbitrary character pattern among the training pattern of the C a tate of the MRF, et each unary feature vector of each feature point a the mean of the Gauian function for each ingle-tate, and et each binary feature vector between two adjacent feature point a the mean of the Gauian function for each pair-tate, and initialize the variance of thoe Gauian function and the tate tranition probabilitie a. Then we ue the Viterbi algorithm or the Baum-Welch algorithm to train the parameter of the MRF (the mean and variance of Gauian function and the tate tranition probabilitie). We repeat the training until the optimal parameter are obtained. 3. Contruction of tructured MRF dictionary Figure 2. Interface to conruct tructured MRF dictionary. (b) We firt create a MRF recognizer at the character level. From each character MRF model, we can obtain the mean vector of unary feature that repreent the mean coordinate of the feature point of training 57
pattern. Fig. 2 (a) and (b) how the mean coordinate of two character [ ] and [ ]. Each mall red rectangle how a feature point. All the troke are concatenated into a troke for each character to achieve troke-number independence. We created an interface to contruct tructured MRF dictionary a hown in Fig. 2. By a moue right click at a feature point, we can cut the character pattern at the feature point and, by a moue left click at a feature point we can concatenate the two radical beide the feature point. We can plit each character model into everal part by thee operation. Then, for each part we can regiter it a a new radical to the radical dictionary by cutting it and normalizing it to the radical dictionary ize. We can alo earch imilar radical from the regitered radical dictionary then elect a imilar radical index to regiter it. Fig. 2 how the plit character model for character [ ] and [ ]. After all character MRF model are plit and are regitered, we match all training pattern with character MRF model, and then we can cut each training pattern into radical at the place where character MRF model are plit. From the bounding boxe of radical of training pattern, we can obtain the mean bounding box a hown in Fig. 2. A hown in Fig. 2, we can ee that the radical with the ame type uch a [ ] from different character may have different ize and poition. We need to normalize the ize and poition of radical pattern. We can normalize them by the following two method: Method : Normalize each radical pattern according to the bounding box of the radical pattern. Method 2: Normalize each radical pattern according to the mean bounding box of the radical of training pattern In the method, there i a problem to extract radical pattern from character pattern. Thi i very difficult for Japanee character. Without robut extraction method, we need to hypothetically egment radical at every feature point, which reult in very larger proceing time. To olve the problem, we normalize radical pattern by the mean bounding boxe of radical appearing in n the ame character cla a hown in Fig.3, where the orange pattern how the input radical pattern. We cut out radical pattern, and then normalize them by the mean bounding box, then ue the normalized radical pattern to train the radical MRF model. When recognizing, we et the radical MRF model into the character model according to the mean bounding boxe by normalizing the mean vector and the covariance matrixe with the mean bounding box to contruct the character model. Each radical MRF may come from variou radical pattern in many character categorie that have very different ize and poition. We conider that different ize will bring different variance, which may reult in low recognition performance if we merge the radical MRF model with too different variance. Therefore, we propoe a method that ue the k-mean method to cluter the ize of each radical into group and plit the radical into multiple clae. We optimize the number of the group. 3. Dictionary compreion by VQ We propoe a dictionary compreion method inpired by the contruction method for a compact offline MQDF recognizer uing VQ technique [3]. In our on-line MRF recognizer, there are many imilar unary feature and binary feature among character model or radical model. We ue a linear-chain MRF for MRF model, where each label ha a tate and each tate ha three tranition. A hown in Fig. 4, each mean vector ha two parameter element, and each covariance matrix ha four parameter element for both of unary feature and binary feature. Each tate Figure 3. Radical pattern normalization. 58
TABLE I. STATISTICS OF CHARACTER PATTERN DATABASES. Nakayoi_t Kuchibue_d #writer 63 20 Total,962 0,403 #character /each writer Kanji/Kana/ Symbol/alpha #character categorie /each writer #average category character 5,643/5,068/ 5,799/3,723/ numeral,085/66 86/65 Total 4,438 3,356 Kanji/Kana/ Symbol/alpha numeral 4058/69 49/62 2976/69/ 46/62 Total 2.3 3.6 Kanji/Kana/ Symbol/alpha numeral.4/22.0 5.5/.0.9/30.0/ 7.4/2.7 We firt compared the performance of three model: the original MRF without tructured dictionary repreentation, the tructured MRF (Str-MRF), and the tructured MRF which plit each radical into multiple clae by clutering the ize for each radical with a k-mean method (Str-MRF2). Table 2 how the reult where C r i the character recognition rate, peed i the average time to recognize a character. ha three tranition probabilitie o that it ha three parameter element. We conider the parameter of each mean vector, thoe of each covariance matrix and thoe of three tranition probabilitie of each tate a a et. Then we cluter the parameter et into group, with each group haring a common parameter et that i the canter of the group. Therefore, we only need to tore the indexe of the group and the center parameter of the group, o that it can realize more effective compreion. 4. Experiment Figure 4. Clutering parameter by VQ. We evaluate the compact recognizer for 2,965 Kanji character of the JIS level- tandard. We trained the character recognizer by uing an on-line Japanee handwriting databae called Nakayoi [5]. On the other hand, the performance tet wa made on an on-line Japanee handwriting databae called Kuchibue [5]. Table how the detail of the databae. Each character cla (character category) ha a different number of ample pattern, and kana and ymbol have more pattern (ee Table ). To maintain balance, we elected 00 pattern at random from each character cla of the Kuchibue databae and ued the ame number of ample pattern for each character cla to evaluate the performance. The experiment were implemented on an Intel(R) Xeon(R) CPU W5590 @ 3.36 GHz 3.36 GHz (2 proceer) with 2 GB memory. TABLE II. COMPARISON OF STRUCTED MODELS Method Original Performance MRF Str-MRF Str-MRF2 Cr (%) 97.45 95.20 96.62 Tet memory 8MB 690KB 2.9MB peed 78.8u 78.9u 78.7u From thee reult, we can ee that Str-MRF largely degrade the character recognition accuracy, although it compree the memory pace largely. Str- MRF2 yield better recognition accuracy than Str- MRF, and it alo compree the memory pace from the original MRF. A for the peed, no lo ha been oberved. Then, we compared the performance of the dictionary compreion method by VQ. We can cluter the parameter et into 255 group, and ave each group index at byte data (VQ). We can alo cluter the parameter et into the optimal number of group, and ave each group index at 2 byte data (VQ2). Moreover, after contructing the tructured MRF, we can apply the VQ method to the tructured dictionary to further compre the dictionary. For original MRF, we ue 4 byte data to tore each parameter of the tate tranition probabilitie and the covariance matrixe, and ue 2 byte data to tore each parameter of the mean vector. Table 3 how the reult. 59
Method Performance Tet TABLE III. Original MRF dictionary COMPARISON OF VQ S VQ VQ2 VQ2+ Str- MRF2 Cr (%) 97.45 96.50 97.55 96.72 memory 8MB 975KB.72MB 750KB peed 78.8u 96.6u 96.6u 96.6u From thee reult, we can ee that VQ2 achieve better character recognition accuracy, although it conume lightly more memory pace compared to VQ. Combining the tr-mrf2 with VQ2 yield better recognition accuracy and maller memory pace than VQ. A for the peed, little lo ha been oberved from VQ. 5. Concluion Thi paper preented a method for building a compact on-line MRF recognizer for large handwritten Japanee character et uing tructured dictionary repreentation and VQ technique. The radical model were hared by different character, and a character model wa contructed from the contituent radical MRF model. Moreover, we employed VQ technique to further compre the memory pace without recognition accuracy lo. Reference [] B. Zhu, X.-D. Zhou, C.-L. Liu and M. Nakagawa: A robut model for on-line handwritten Japanee text recognition, Int l J. Document Analyi and Recognition (IJDAR), 3(2), pp.2-3, 200. [2] H. Oda, B. Zhu, J. Tokuno, M. Onuma: A. Kitadai and M. Nakagawa: A compact on-line and off-line combined recognizer. Proc. 0th IWFHR, pp. 33-38, 2006. [3] C.-L. Liu and X.-D. Zhou: Online Japanee character recognition uing trajectory-baed normalization and direction feature extraction, Proc. 0th IWFHR, pp.27-222, 2006. [4] B. Zhu and M. Nakagawa: A MRF model with parameter optimization by CRF for on-line recognition of handwritten Japanee character, Proc. Document Recognition and Retrieval XVIII (DRR) that i part of IS&T/SPIE Electronic Imaging, San Joe, USA, 20. [5] B. Zhu and M. Nakagawa: On-line handwritten Japanee character recognition uing a MRF model with Parameter Optimization by CRF, Proc. th ICADR, pp. 603-607, 20. [6] S. Gunter and H. Bunke: HMM-baed handwritten word recognition: on the optimization of the number of tate, training Iteration and Gauian component, Pattern Recognition, 37, pp. 2069-2079, 2004. [7] M. Liwicki and H. Bunke: HMM-baed on-line recognition of handwritten whiteboard note, Proc. 0th IWFHR, pp. 595-599, 2006. [8] S. Jaeger, S. Manke and J. Reichert: Online handwriting recognition: the Npen++ recognizer, Int l J. Document Analyi and Recognition, Vol. 3, pp. 69-80, 200. [9] M. Nakagawa and L.V. Tu: Structural learning of character pattern for on-line recognition of handwritten Japanee character, Advance in Structural and Syntactic Pattern Recognition, P. Perner, P. Wang, and A. Roenfeld, ed., pp. 80-88, Springer-Verlag, 996. [0] M. Nakai, N. Akira, H. Shimodaira and S. Sagayama: Subtroke approach to HMM-baed on-line kanji handwriting recognition, Prof. 6th ICDAR, pp. 49-495, 200. [] M. Nakai, T. Sudo, H. Shimodaira and S. Sagayama: Pen preure feature for writer-independent on-line handwriting recognition baed on ubtroke HMM, Proc. 6th ICPR, 3, pp. 220-223, 2002. [2] M. Nakai, H. Shimodaira and S. Sagayama: Generation of hierarchical dictionary for troke-order free kanji handwriting recognition baed on ubtroke HMM, Proc. 7th ICDAR, pp. 54-58, 2003. [3] T. Long and L. Jin: Building compact MQDF claifier for large character et recognition by ubpace ditribution haring, Pattern Recognition, 4, pp. 296-2925, 2008. [4] U. Ramer: An iterative procedure for the polygonal approximation of plan cloed curve, Computer Graphic and Image Proceing, (3), pp. 244-256, 972. [5] M. Nakagawa and K. Matumoto: Collection of on-line handwritten Japanee character pattern databae and their analyi, Int l J. Document Analyi and Recognition (IJDAR), 7(), pp.69-8, 2004. 60