On Supportng Identfcaton n a Hand-Based Bometrc Framework Pe-Fang Guo 1, Prabr Bhattacharya 2, and Nawwaf Kharma 1 1 Electrcal & Computer Engneerng, Concorda Unversty, 1455 de Masonneuve Blvd., Montreal, QC H3G 1M8, Canada {pf_guo,kharma}ece.concorda.ca 2 Computer Scence Department, Unversty of Cncnnat, 814 Rhodes Hall, Cncnnat, OH 45221-0030, USA bhattapr@ucmal.uc.edu Abstract. Research on hand features has drawn consderable attenton to the bometrc-based dentfcaton feld n past decades. In ths paper, the technque of the feature generaton s carred out by ntegratng genetc programmng and the expectaton maxmzaton algorthm wth the ftness of the mean square error measure (GP-EM-MSE) n order to mprove the overall performance of a hand-based bometrc system. The GP program trees of the approach are utlzed to fnd optmal generated feature representatons n a nonlnear fashon; derved from EM, the learnng task results n the smple k- means problem that reveals better convergence propertes. As a subsequent refnement of the dentfcaton, GP-EM-MSE exhbts an mproved capablty whch acheves a recognton rate of 96% accuracy by usng the generated features, better than the performance obtaned by the selected prmtve features. Keywords: Feature generaton, bometrc dentfcaton, genetc programmng, the expectaton maxmzaton algorthm, mean square error, classfcaton. 1 Introducton Bometrcs-based dentfcaton s a verfcaton approach usng bologcal features n each ndvdual. Hand features have been wdely used n desgnng a bometrc dentfcaton system and the challenge has been establshed [1], [2]. In pattern recognton, the analytcal selecton of features and the automatc generaton of features provde two dstnct approaches, because they rely on dfferent sources of nformaton [3]. It may be useful to explore new bometrc dentfcaton systems that combne both methods of the feature selecton and generaton. In ths study, buldng upon our prevous work of the feature selecton on hand mages, we present an automatc method of the feature generaton, usng genetc programmng and the expectaton maxmzaton algorthms wth the mean square error measure (GP-EM-MSE) for a hand-based bometrc dentfcaton system. The benefts from the combned system of the feature selecton and generaton are as follows: A. Elmoataz et al. (Eds.): ICISP 2010, LNCS 6134, pp. 210 217, 2010. Sprnger-Verlag Berln Hedelberg 2010
On Supportng Identfcaton n a Hand-Based Bometrc Framework 211 to ncrease the probablty that the errors of the ndvdual feature selecton may be compensated for by the correct results of automatc feature generaton. to rase the contrbuton that the overall performance of the classfcaton would make toward a hgher recognton rate by the mprovement of the qualty of feature representatons. The organzaton of the paper s as follows: Secton 2 descrbes brefly our prevous work on the feature selecton for hand mages. In Secton 3, GP and the k-means problem va EM are descrbed as the bass of the platform, ncludng the evaluaton phase. Secton 4 presents dentfcaton results, and concluson and future work are gven n Secton 5. 2 Prevous Work on the Feature Selecton for Hand Images Prevous work on hand mages can be categorzed as the unsupervsed feature selecton for clusterng hand mages. The man tool for accomplshng ths was a genetc algorthm (GA). The method, named cooperatve coevolutonary clusterng algorthm (CCCA), was desgned to search for a proper number (wthout pror knowledge of t) of clusters of hand mages, and smultaneously to acheve the feature selecton. CCCA was mplemented usng 100 hand mages as the test set; see a sample of a hand mage n Fg.1. The results showed that the dmensonalty of the clusterng space was reduced from 84 orgnal features to 41 selected features (11 geometrc and 30 statstcal features), wth 4 clusters produced. At the end, the output clusters were labeled wth the number of nput mage patterns per class, assgned to each cluster. The detals of the CCCA method appear n our prevous work [4], [5]. Fg. 1. A sample of a hand mage 3 The GP-EM-MSE Bometrc Identfcaton System The pre-requste for the technque of the feature generaton s the preparaton of prmtve feature sets. Usng our prevous method n [4] on a dataset of 200 hand mages, we succeeded n clusterng the 200 mages nto 4 categores, wth a total
212 P.-F. Guo, P. Bhattacharya, and N. Kharma number of 41 features selected as the prmtve feature set for the current study of the feature generaton. Fg.2 presents the method of the feature generaton by ntegratng GP and EM wth the MSE ftness ndcator (GP-EM-MSE). In the approach, GP program trees can be vewed as sequences of applcatons of functons (replaced wth mathematc operators) to arguments (replaced wth 41 prmtve features), whch satsfy requrements for ratonal expressons of the generated features n a straghtforward way; va EM, the learnng task results n a smple hypothess of the k-means problem. In the next subsectons, we provde descrptons of each component of the bometrcbased GP-EM-MSE system, ncludng the steps of the computaton. Fg. 2. The GP-EM-MSE hand-based bometrc dentfcaton system 3.1 The GP Applcaton GP mantans a populaton of ndvduals by program trees. In each teraton, t produces a new generaton of ndvduals usng reproducton, crossover, and mutaton. The ftness of a gven ndvdual program n the populaton s typcally determned by executng the program on a set of the tranng data. The teraton of GP s llustrated n Fg. 3; the detals of the GP process can be found n [6]. Fg. 3. The dagram of the evolutonary GP teraton The GP program trees n the approach The choces of termnal arguments and functons provde partcular representatons for descrbng GP program trees n the problem doman. In ths study, we choose the followng mathematcal operators:
On Supportng Identfcaton n a Hand-Based Bometrc Framework 213 {+, -,,, square root, sne, cosne, tan, exponental, absolute, square, negatve}, whch consttute the GP functon set; termnal arguments receve 41 prmtve features selected by the method [4]. As shown n Fg.4, an example of the ratonal expresson, h2 X h5 tg(h5 + h8), can be expressed as the GP program tree, On the tree, the sgn of tg s a mathematcal operator that takes one termnal argument, and the sgns of +, - and X are operators that need two arguments. The argument, hd, d = 2, 5, 8, s replaced wth the (d+1)th prmtve feature. - X tg h2 h5 + h5 h8 Fg. 4. A GP program tree for the ratonal expresson of h2 X h5 tg(h5 + h8) The ftness functon, MSE The MSE measure s well known n the functon approxmaton and learnng system theores when the number of classes s assumed to be a pror known. In ths supervsed problem of the feature generaton, we employ the MSE measure as the ftness whch s gven by [7]: the MSE ftness = D( c k m j= 1 = 1 j, r j ), (1) where D s the Eucldean dstance between the nstance r and ts mean center c j. 3.2 Dervatve of the k-means Problem va EM Gven the observed data R = { r }, dvded nto the k known classes; the hdden varables Z = {< z 1,, z j,, z k >} n ths case ndcate the ndex of the jth mxture component that generates r. Wth the approxmaton, the data R can be descrbed n the mxture of the k Gaussans. The purpose s to ft the Gaussan densty mxture to the data R by optmzng the Gaussans mxture parameters, whch nclude the mxng proporton, the mean values, and the covarance matrx. In practce, the most common use s to assume the equal mxng proporton (1/k) wth a unvarate case for all the k categores [8], from whch t smplfes the task to a hypothess of the k-means problem [9]. Applyng the maxmum lkelhood estmator, the k-means problem s broken down nto two separate steps: E step E z j r n k r (2)
214 P.-F. Guo, P. Bhattacharya, and N. Kharma M step m m z j z j (3) where μ j s the mean value of the jth Gaussan, and E[z j ] s the probablty that r s generated by the jth Gaussan dstrbuton. Further detals can be found n [9], [10], regardng the dervatve of the k-means problem va EM. In the mplementaton, the data R are replaced wth the generated data produced by GP-EM-MSE. The EM steps, expressed n Eqs. (2) and (3), are executed on the data R, and consttute a jont loop wth the GP teraton. 3.2 The GP-EM-MSE Recursve Computaton The steps of the GP-EM-MSE computaton are summarzed n Table 1. Table 1. The GP-EM-MSE recursve computaton Note: begn from each populaton of the generated features, P_feature p. FOR ndex p = 1 TO 32 populatons DO FOR ndex = 1 TO 100 mages DO Perform the data transformaton based on the varaton of the GP program trees. END Implement the E-step, expressed n Eq. (2), to estmate the expected values, E[z j ]. Implement the M-step, expressed n Eq. (3), to revse the values of the k-means. Replace wth the new revsed values of the k-means. Evaluate each of the pupulatons by the MSE ftness, expressed n Eq. (1). Vary and generated new populatons by applyng the reproducaton, crossover and mutaton. END Save the best generated features. 3.3 The Evaluaton Phase Usng the Classfers MDC and KNN The task of the dentfcaton s to classfy samples nto categores so that samples n the same categores are as smlar as possble and samples n dfferent categores are as dssmlar as possble. Mathematcally, the problem of classfcaton can be formulated n terms of a predetermned measure [11]. For the mnmum dstance classfer (MDC), the dstance s defned as an ndex of smlarty so that the mnmum dstance s dentcal to the maxmum smlarty; for the K nearest neghbors (KNN), each mage pattern s classfed to the frequent class among ts neghbors based on a smlarty measure. In ths study, the Eucldean dstance measure s employed for both MDC and KNN n the evaluaton. 4 Identfcaton Experments In the mplementaton, we dvded the 200 mages nto the tranng and testng sets, each wth 100 mages. Usng the 41 prmtve features as the nput, the GP-EM-MSE tranng s evolved wth the maxmum tree depth of 5 and the populaton sze of 32.
On Supportng Identfcaton n a Hand-Based Bometrc Framework 215 4.1 Feature Generated Results We ended the GP-EM-MSE tranng after runnng 400 teratons. The total tme for computaton was about three mnutes on a Pentum 4 at 1.60 GHz. The results for the two features, P_feature 1 and P_feature 2, produced by GP-EM-MSE are as follows: P _ feature 1 = cos{sn[ h0 h39 tg( h22 h21)] tg[ h32 h0 + h39 h27] + sn[( h30 h7) 2 e h2 h19 2 ] ( h12 h31 + h14 h26) } and P _ feature 2 = cos cos[ ( h 32 + h25) ( h9 + h28)], h25 h9 h8 h10 + cos( ) + + cos( h35 h20) h23 h8 h22 sn( h25 + h32) h8 where hd s the (d+1)th feature of 41 prmtve features, and the convergence results are presented n Fg. 5; n the sequel, the resultng generated features wll be employed to dentfy hand mages. 1 ftness 0.75 0.5 C_feature 1 C_feature 2 0.25 0 0 50 100 150 200 250 300 350 400 teratons Fg. 5. Convergence results for the features produced by GP-EM-MSE 4.2 Identfcaton Results The goal of the dentfcaton s to classfy hand mages nto a known number of 4 categores or classes accordng to the decson space of the generated features. Wth the classfer MDC, Table 2 shows the confuson matrx of the dentfcaton performance for four classes on the test set, usng two features, P_feature 1 and P_feature 2, produced by GP-EM-MSE. Each row of the table represents the dentfcaton performance for a gven category, whle each column represents a percentage of the number of samples n an actual class. It can be observed, from Table 2, that the dfference of the related dentfcaton accuracy among the classes s small, rangng from 95.00 % to 96.96 %.
216 P.-F. Guo, P. Bhattacharya, and N. Kharma Table 2. Identfcaton accuracy (%) usng two features, P_feature 1 and P_feature 2, produced by GP-EM-MSE wth MDC categores class I class II class III class IV class I (33 samples) 96.96 3.04 0 0 class II (25 samples) 0 96.00 0 4.00 class III (22 samples) 4.55 0 95.45 0 class IV (20 samples) 0 5.00 0 95.00 4.3 Comparson Results The classfers MDC and KNN are utlzed n order to assess the capablty of dfferent feature sets over dfferent classfcaton systems. In terms of the dentfcaton accuracy, Table 3 shows the comparson results between the features produced by GP-EM-MSE wth MDC and 41 prmtve features selected by the method [4] wth KNN n whch the value of K s tested by the nput of 41 prmtve features (K = 41). Table 3. The comparson between the features produced by GP-EM-MSE / MDC and 41 prmtve features selected by [4] / KNN, n terms of the dentfcaton accuracy (%) feature types / classfer no. of features dentfcaton accuracy (%) the prmtve features selected by [4] / KNN 41 93.0 P_feature 1 / MDC 1 92.0 P_feature 1, P_feature 2 / MDC 2 96.0 It can be seen from Table 3 that the dentfcaton accuracy usng one feature, P_feature 1, s slghtly lower than when usng 41 prmtve features. However, the combnaton of two features, P_feature 1 and P_feature 2, produced by GP-EM-MSE acheves the best dentfcaton performance wth an accuracy rate at 96.0%. 5 Concluson and Future Work As an extenson of the method of CCCA for the feature selecton n [4], the purpose of GP-EM-MSE n ths study s to fnd the mproved feature representatons n an optmal control envronment n order to further mnmze dentfcaton errors for the hand-based bometrc system. GP-EM-MSE has succeeded n producng correct features to dentfy hand mages wth the mproved performance. The prevous work of the CCCA method was evolved wthout supervson n such a way that the selected feature sets operated globally to cluster hand mages. Consequently, the current study of GP-EM-MSE for the feature generaton s a supervsed learnng algorthm, va the results of CCCA. In the case of some not-sosmple problems, the GP-EM-MSE method offers a customzed general purpose research platform by jontly optmzng feature representatons, whch demonstrates that such a combnaton of methods of the feature selecton and generaton leads to a hgher recognton rate.
On Supportng Identfcaton n a Hand-Based Bometrc Framework 217 Our future work wll be dedcated to examnng the robustness of the system n dffcult pattern dentfcaton problems and large databases of samples. Acknowledgments. Ths work s supported by grants from the NSERC. References 1. Zheng, G., Wang, C.-J., Boult, T.E.: Applcaton of projectve nvarants n hand geometry bometrcs. IEEE Trans. Informaton Forenscs and Securty 2, 758 768 (2007) 2. Zhang, D., Kong, W., You, J.: Onlne palmprnt dentfcaton. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1041 1050 (2003) 3. Rzk, M.M., Zmuda, M.A., Tamburno, L.A.: Evolvng pattern recognton systems. IEEE Trans. Evol. Comput. 6, 594 609 (2002) 4. Kharma, N., Suen, C.Y., Guo, P.-F.: Palmprnts: a cooperatve co-evolutonary algorthm for clusterng hand mages. Int. J. Image and Graphcs 5, 595 616 (2005) 5. Guo, P.-F.: Palmprnts: a cooperatve co-evolutonary clusterng algorthm for hand-based bometrc dentfcaton. M.A.Sc. thess, Concorda Unversty, Montreal, Canada (2003) 6. Koza, J.R.: Genetc Programmng II: Automatc Dscovery of Reusable Programs. MIT Pr., Cambrdge (1994) 7. Lee, C.-Y.: Effcent automatc engneerng desgn synthess va evolutonary exploraton. Ph.D thess, Calforna Insttute of Technology, Pasadena, Calforna (2002) 8. Raymer, M.L., Doom, T.E., Kuhn, L.A., Punch, W.F.: Knowledge dscovery n medcal and bologcal datasets usng a hybrd Bayes classfer/evolutonary algorthm. IEEE Trans. Syst., Man, Cybern., Part B 33, 802 813 (2003) 9. Bshop, C.M.: Pattern Recognton and Machne Learnng. Sprnger, New York (2006) 10. Mtchell, T.M.: Machne Learnng. McGraw-Hll, New York (1997) 11. Fukunaga, K.: Introducton to Statstcal Pattern Recognton, 2nd edn. Academc, Boston (1990)