Recognition of Tifinagh Characters Using Self Organizing Map And Fuzzy K-Nearest Neighbor

Transcription

1 Global Journal of Computer Scence and Technology Volume Issue 5 Verson.0 September 0 Type: Double Blnd Peer Revewed Internatonal Research Journal Publsher: Global Journals Inc. (USA) Onlne ISSN: & Prnt ISSN: Recognton of Tfnagh Characters Usng Self Organzng Map And Fuzzy K-Nearest Neghbor By S. Gounane, M. Fakr, B. Boukhalene FST SMS Unversty Ben Mellal, Morocco Abstract - In ths paper we present a comparson between SOM (Self-Organzaton Map) neural network and Fuzzy K-nearest Neghbor algorthms and ther applcaton to handwrtng Tfnagh character recognton. The Box approach proposed n [] s used for features extracton. Expermentaton s carred out on a lmted database of nearly 00 samples. The results showed that Fuzzy K-Nearest Neghbor had a very good performance. Keywords : Tfnagh characters, Fuzzy k-nearest Neghbor, Self Organzng Map, Fuzzy k-means, Features extracton. GJCST Classfcaton : I..3, F.. Recognton of Tfnagh Characters Usng Self Organzng Map And Fuzzy K-Nearest Neghbor Strctly as per the complance and regulatons of: 0. S. Gounane, M. Fakr, B. Boukhalene.Ths s a research/revew paper, dstrbuted under the terms of the Creatve Commons Attrbuton-Noncommercal 3.0 Unported Lcense permttng all non commercal use, dstrbuton, and reproducton n any medum, provded the orgnal work s properly cted.

2 Recognton of Tfnagh Characters Usng Self Organzng Map And Fuzzy K-Nearest Neghbor Abstract - In ths paper we present a comparson between SOM (Self-Organzaton Map) neural network and Fuzzy K- nearest Neghbor algorthms and ther applcaton to handwrtng Tfnagh character recognton. The Box approach proposed n [] s used for features extracton. Expermentaton s carred out on a lmted database of nearly 00 samples. The results showed that Fuzzy K-Nearest Neghbor had a very good performance. Keywords : Tfnagh characters, Fuzzy k - Nearest Neg - hbor, Self Organzng Map, Fuzzy k- means, Features Features extracton. I. INTRODUCTION T he recognton of characters from scanned mages of documents has been a problem that has receved much attenton n the felds of mage processng, pattern recognton and artfcal ntellgence. For many years, fuzzy logc and Artfcal Neural Networks have been used n a wde range of problem domans: process control (where Fuzzy Controllers have been very popular), management and decson makng, operatons research, economcs and pattern recognton and classfcaton. Ths paper presents an applcaton of both SOM neural network and fuzzy k-nearest Neghbor n recognton of handwrtten Tfnagh characters. Ths paper s organzed as follows: n secton [] a features extracton method, whch s an essental step pror to pattern recognton, s descrbed. Secton [] descrbes the archtecture and the learnng mechansm of the SOM neural networks. Secton [3] presents the k-nearest Neghbor algorthm. Secton [4] gves results of the applcaton of both SOM and Fuzzy k-nn on Tfnagh handwrtng character recognton. II. FEATURES EXTRACTION Preprocessng technques lke thnnng, slant correcton and smoothenng are appled. For extractng the features, the Box approach proposed n [] s used here. Ths approach requres the spatal dvson of the Author α : Informaton Processng & Telecommuncaton Team, Dep. Of Computer Scences FST SMS Unversty Ben Mellal, Morocco. E-mal : Gounane.sad@gmal.com Author Ω : Informaton Processng & Telecommuncaton Team, Dep. Of Computer Scences FST SMS Unversty Ben Mellal, Morocco. E-mal : fakfad@yahoo.fr Author β : Informaton Processng & Telecommuncaton Team FP SMS Unversty Ben Mellal, Morocco. E-mal : boukhalene@yahoo.fr S. Gounane α, M. Fakr Ω, B. Boukhalene β character mage. The maor advantage of ths approach stems from ts robustness to varaton, ease of mplementaton and hgh recognton rate. Each character mage s dvded nto L H boxes so that the portons of character wll be n some of these boxes. There could be boxes that are empty (Fgure (a)). The choce of number of boxes s arrved at by expermentaton. Elements of a Normalzed Vector that descrbe the character s obtaned by dvdng the number of all black pxels present n ths box by ther total number for each box, (Fgure (b)). One can easly see that ths characterzaton s nvarant of the character mage dmensons. Hence an mage of whatever sze gets transformed nto a vector of L H predetermned dmensons. III. Fgure : Features extracton SELF ORGANIZING MAP (SOM) The Self-Organzng Maps, abbrevated SOM, was developed by Professor Teuvo Kohonen n the early 980s. Self-organzng maps are a specal class of artfcal neural network, because those are based on compettve learnng and the learnng tself s unsupervsed. Also SOM s consdered as a specal case n data-mnng, t can be appled to both clusterng and proectng the data onto a lower dmensonal dsplay at the same tme. In a SOM network, there s an nput layer and an output layer whch s usually desgned as twodmensonal arrangement of neurons that maps n dmensonal nput to two dmensonal (Fgure ). It s bascally a compettve network wth the characterstc of self-organzaton provdng a topology-preservng mappng from the nput space to the clusters. September 0 9 Global Journal of Computer Scence and Technology Volume XI Issue XV Verson I 0 Global Journals Inc. (US)

3 Recognton of Tfnagh Characters Usng Self Organzng Map And Fuzzy K-Nearest Neghbor September 0 30 Global Journal of Computer Scence and Technology Volume XI Issue XV Verson I The algorthm proceeds frst by ntalzng the synaptc weghts n the map for the neurons. Ths can be done by assgnng them small values pcked from a random number generator; n so dong no pror order s mposed on the feature map. Once the map has been properly ntalzed, there are three essental processes nvolved n the formaton of the self-organzng map: Competton, cooperaton and synaptc adaptaton. Fgure : SOM archtecture a) Competton Let X = [ x,..., x n ] be the nput pattern and W = [ w,..., w n ] the synaptc weght vector of each neuron n the map. To fnd the best match of the nput vector X wth the synaptc weght vectors W, compare the nner products W T. X for =,,,l (l s the number of output neurons) and select the largest denoted (X ). Maxmzng W T. X s equvalent to mnmzng W X [] then one can have: ( W X ) ( X ) = arg mn () The neuron number (X ) s called the wnnng neuron. b) Cooperaton In neurobology, a neuron that s frng tends to excte neurons n ts mmedate neghborhood more than those farther away. The same wth output neurons n the SOM, a topologcal neghborhood around the wnnng neuron s made and t decay smoothly wth lateral dstance. Another unque feature of the SOM algorthm s that the sze of the topologcal neghborhood shrnks wth tme. Let h, ( k) denote the topologcal neghborhood at tme k, centered on the wnnng neuron, and denote a typcal neuron of a set of excted (cooperatng) neurons around wnnng neuron. On can assume that the topologcal neghborhood h, s unmodal functon of the lateral dstance d,, such that t satsfes: s symmetrc and attans ts maxmum value at. h, the wnnng neuron for whch d, =0.. The ampltude of h, decreases monotoncally wth ncreasng d,, decayng to zero for d,. A typcal choce of h, h, s the Gaussan functon ( k) = e d, σ ( k ) k τ = σ 0e () σ ( k ) (3) Where τ s the tme constant of the algorthm. c) Synaptc adaptaton By defnton, for the network to be selforganzng (and unsupervsed), the synaptc weght vector W of neuron n the network s requred to change n relaton to the nput vector X. Ths change s expressed by the equaton as follows: W ( W ( )) (, ( X ) k k + ) = W ( k) + α ( k) h ( k) X (4) Where α(k) s the learnng-rate. The exact form of learnng-rate s not mportant. It can be lnear, exponental or nversely proportonal. However t should be tme varyng. In partcular, t should start at an ntal value α 0, and then decrease gradually wth ncreasng tme n. Ths requrement can be satsfed by usng an exponental learnng-rate, as shown by: τ α k ( ) = α 0e τ k (5) Where s another tme constant of the SOM algorthm. Fgure 3 : example of topologcal neghbourhoods (Gaussan) IV. FUZZY K-NEAREST NEIGHBOR Fuzzy knn s part of supervsed learnng that has been used n many applcatons n the feld of data mnng, statstcal pattern recognton, mage processng 0 Global Journals Inc. (US)

4 Recognton of Tfnagh Characters Usng Self Organzng Map And Fuzzy K-Nearest Neghbor and many others. Some successful applcatons are ncludng recognton of handwrtng and satellte mage. Fuzzy K nearest neghbor algorthm s very smple. It works based on an unsupervsed clusterng algorthm lke fuzzy k-means algorthm to determne prototypes representng clusters. Then the mnmum dstance from the query nstance to these prototypes s used to determne the K-nearest neghbors. After and basng on membershp functon we take the neghbor wth the maxmum membershp to be the predcton of the query nstance. a) Fuzzy k-means The Fuzzy k-means clusterng algorthm s an mprovement of the k-means algorthms. K-Means are the smplest methods of clusterng data. The fuzzy K- means algorthm uses a set of N unlabeled feature vectors and classfes them nto k classes, where k s gven by the user. From these N feature vectors, k of them are randomly selected as ntal seeds. The feature vectors are assgned to the closest seeds dependng on ts dstance from t and on a gven membershp functon. The mean of features belongng to a class s taken as the new center. The features are reassgned; ths process s repeated untl convergence. In a fuzzy clusterng, a pattern vector can belong to all clusters wth dfferent degrees gven by a membershp functon [fgure 4]. One can prove that such a functon exsts [5], and for each cluster C t s defned as follows: ( X, c ) / d m f ( X ) = N m R, m (6) m / d ( X, c ) = Where c s the center of the class C, and The parameter m s defned by the user to adust the fuzzness of the clusterng. The hard clusterng case s obtaned by takng m =. Fgure 4 : Membershp functon b) Fuzzy K-means algorthm. Choose randomly the k prototypes c.. Compute the degree of membershp of all feature vectors X n all clusters C : = f ( X ). 3. Compute new cluster prototypes as follows: c N ( ) = = N ( ) = 4. Iterate back and force between () and (3) untl the membershps or cluster centers for successve teraton dffer by more than some prescrbed value ε. c) Fuzzy K - Nearest Neghbor The Fuzzy k-nn classfers consst on proxmty measures. They are deally suted for modelng the non parametrc dstrbuton on handwrtten word recognton data. For a gven character X, the fuzzy classfer computes the membershp of X n dfferent classes C, C,... C N. The character X s allocated to the class for whch the membershp functon yelds the maxmum value. By a learnng process (Fuzzy k-mean, k-mean, LVQ...) we assgn a number of prototypes P for each classc. After generatng the k-nearest Neghbor prototypes P for a character (by dstance smlarty), the degree of membershp of the vector X to the class C : (X ), can be calculated as follow: k = m x m ( X, P ) ( X, P ) m (7) / d = ( X ) = N (8) / d m Where s the membershp of the prototype P to the class C. To compute ths value tow methods are proposed: Usng a crsp assgnment of P to C : P C = (9) 0 P C Or by usng the k-nn to determne k-nearest neghbor of P, then the number of neghbor belongng to the same class as P denoted n s used to compute as follows: n / k = = (0) 0.49n / k V. RESULTS A ava applcaton was developed n order to test and compare those two algorthms. To extract features each, character was dvded nto boxes, the nput layer of the SOM neural network was made of 63 neurons and 35 for the output layer (number of character to recognze), for the fuzzy k-nearest neghbor the fuzzness factor m s equal to and number of neghbors k s equal to 3. September 0 3 Global Journal of Computer Scence and Technology Volume XI Issue XV Verson I 0 Global Journals Inc. (US)

5 Recognton of Tfnagh Characters Usng Self Organzng Map And Fuzzy K-Nearest Neghbor September 0 3 Global Journal of Computer Scence and Technology Volume XI Issue XV Verson I As there s no standard database avalable for handwrtten Tfnagh characters, a database was created from samples of dfferent wrtng styles wth dfferent szes. The database also ncludes some complex samples that are even hard to recognze by careful nspecton Fgure 5: Samples used for tranng (fgure 5). Ths database contans 00 samples (about 5 samples per character) dvded nto two dsont sets, one for tranng both SOM and Fuzzy k-nn, and another for testng these two algorthms. The features extracton method used here s scalng nvarant, that's why there s always a recognton problem between Tfnagh character o and o, where o s are mstaken as o s and vce versa. Some other recognton problems are wth and, o and o but not persstent as the frst one. Experments shows that the best results are gven by the Fuzzy k-nn where n many cases t success to recognze handwrtten characters that SOM algorthm fals to (fgure 6). 0 Global Journals Inc. (US) Fgure 5 : Samples used for tranng Fgure 6 : handwrtten character. SOM algorthm (left) fals ( ) but not Fuzzy k-nn(rght) ( ) VI. CONCLUSION In ths paper two clusterng algorthms are presented, Self Organzng Map neural network and the Fuzzy k-nearest Neghbor. We appled these two algorthms to Tfnagh character recognton. The box approach was used to extract features from the character mage. Fuzzy k-nn was more robust and fast than the SOM. REFERENCES REFERENCES REFERENCIAS. M.Hanmandlu, A.V.Nath, A.C.Mshra, & V.K.Madasu. (007). Fuzzy Model Based Recognton of Handwrtten Hnd Numerals usng bacteral foragng. 6th IEEE/ACIS ICIS.. Haykn, & Smon. (999). Neural Networks: a comprehensve fondaton. New ersey: Prentce-hall nc. 3. A.Mngot, S., & Lma, J. O. (005). Comparng SOM neural network wth Fuzzy c-means, K-means and tradtonal herarchcal clusterng algorthms. European Journal of Operatonal Research. 4. D.Slezak, B.Kang-Junzhong, T.Km, & G.Kuroda. Segnal processng, Image processng and Pattern recognaton. New York: Sprnger. 5. J.Sagau. (n.d.). Logque flou en classfcaton. Technques de l'ingéneur, H3638.

6 Recognton of Tfnagh Characters Usng Self Organzng Map And Fuzzy K-Nearest Neghbor 6. (J.C.), & BEZDEK. (98). Pattern recognton wth fuzzy obectve functon algorthms. Plenum Press. 7. A.Tellaeche, Burgos-Artzzu, X. P., G.Paares, & A.Rbero. (007). On Combnng Support Vector Machnes and Fuzzy K-Means n Vson-based Precson Agrculture. World Academy of Scence, Engneerng and Technology. 8. F.Zheng. (n.d.). K-Means-based fuzzy classfer. 9. Nasser, S., Alkhald, R., & Vert, G. (n.d.). A Modfed Fuzzy K-means Clusterng usng Expectaton Maxmzaton. 0. Nassery, P., & Faez, K. (n.d.). Sgnature pattern recognton usng psudo Zernke moments and fuzzy logc classfer.. O.Matan, R.K.Kang, C.E.Stenard, & B.Boser. (990). Handwrtten character recognton usng neural network archtectures. 4th USPS advanced Technology Conference, Washngton D.C, S.Nasser, R.Alkhald, & G.Vert. (n.d.). A Modfed Fuzzy K-means clusterng usng expectaton Maxmzaton. 3. S.Theodords, & Koutroumbas, K. (009). Pattern recognaton (fort edton). Elsever. 4. V.Ganapathy, & Lew, K. L. (008). Handwrtten Character Recognton Usng Multscale Neural. September 0 33 Global Journal of Computer Scence and Technology Volume XI Issue XV Verson I 0 Global Journals Inc. (US)

7 Recognton of Tfnagh Characters Usng Self Organzng Map And Fuzzy K-Nearest Neghbor September 0 34 Global Journal of Computer Scence and Technology Volume XI Issue XV Verson I Ths page s ntentonally left blank 0 Global Journals Inc. (US)