A Genetic Programming-PCA Hybrid Face Recognition Algorithm

Transcription

1 Journal of Sgnal and Informaton Processng, 20, 2, do:0.4236/jsp Publshed Onlne August 20 ( A Genetc Programmng-PCA Hybrd Face Recognton Algorthm Behzad Bozorgtabar, Gholam Al Reza Rad School of Electrcal Engneerng, Iran Unversty of Scence and Technology, Tehran, Iran. Emal: b_bozorgtabar@elec.ust.ac.r, Reza@ust.ac.r Receved June th, 20; revsed July 24 th, 20; accepted August 5 th, 20. ABSTRACT Increasng demand for a fast and relable face recognton technology has oblged researchers to try and examne dfferent pattern recognton schemes. But untl now, Genetc Programmng (GP), acclamed pattern recognton, data mnng and relaton dscovery methodology, has been neglected n face recognton lterature. Ths paper tres to apply GP to face recognton. Frst Prncpal Component Analyss (PCA) s used to extract features, and then GP s used to classfy mage groups. To further mprove the results, a leveragng method s also utlzed. It s shown that although GP mght not be effcent n ts solated form, a leveraged GP can offer results comparable to other Face recognton solutons. Keywords: Face Recognton, Prncpal Component Analyss, Genetc Programmng, Leveragng Algorthm. Introducton Face recognton has become one of the most actve research areas of pattern recognton snce the early 990s. In the past 20 years, sgnfcant advances have been made n desgn of successful classfer for face recognton []. However the dversty of the face patterns makes t dffcult to create robust recognton systems and the complexty of the algorthms makes them hard to mplement. Prncpal components analyss (PCA) method [2], whch s the base of well-known face recognton algorthm, Egenfaces [3,4], s an appearance-based technque used wdely for the feature extracton and has recorded a great performance n face recognton. PCA based approaches typcally nclude two phases: tranng and classfcaton. In the tranng phase, an egenspace s establshed from the tranng samples usng PCA and the tranng face mages are mapped to the egenspace for classfcaton. In the classfcaton phase, an nput face s projected to the same egenspace and classfed by an approprate classfer, such as Support Vector Machnes (SVMs) or Neural Networks [5]. Genetc programmng s an evolutonary algorthm methodology nspred by bologcal evoluton [6]. Evolutonary algorthms create a populaton of abstract representatons of canddate solutons, whch s evolved usng bology nspred operators such as selecton, cross-over and mutaton towards better solutons. In recent years, Genetc Programmng and other evolutonary algorthms has been used n classfcaton and pattern recognton problems [7,8], although to the authors knowledge, Genetc Programmng has never been used n Face Recognton Doman. In many applcatons, Genetc programmng yelds smplfed symbolcal representaton of the underlyng system t tres to model. Ths leads to effcent checkng of a new sample [9]. On the other hand the complexty and the tme needed to fnd such representaton dscourages ts use n many applcatons. Leveragng algorthms are a group of determnstc algorthms where a set of weak learners are used to create a strong learner [0]. Whle t s not algorthmcally constraned, most leveragng algorthms teratvely employ weak learners based on a dstrbuton and combne them wth weghtng to form a fnal strong learner. In ths paper, Genetc Programmng s utlzed to classfy face mages. As mages are usually large, PCA s used to extract mage features and thus reduce data dmenson. The Genetc Programmng s then appled to the extracted features. Usng a tranng group, Genetc Programmng dscovers possble relatonshp between the extracted features, whch s n turn used to classfy new mages. To mprove results, a leveragng scheme s ntroduced, whch employs Genetc Programmng as a weak learner, and combne results of several Genetc

2 A Genetc Programmng-PCA Hybrd Face Recognton Algorthm 7 Programmng classfcatons as a sngle strong classfer. The rest of paper s organzed as follows: n Sectons 2 and 3, PCA and Genetc Programmng are ntroduced respectvely. Secton 4 presents the ntroduced algorthm, where Genetc Programmng s used wth and wthout leveragng. In Secton 5, smulatons are done on a selected face database and results are compared to prevous studes. 2. Feature Extracton Prncpal Component Analyss Let there be R face mages n the tranng set, where each mage s a 2-dmensonal array of sze m n of ntensty values. The mage can be converted nto a vector of D (where D = m n) pxels. The rows of pxels of the mage are placed one after another to form the vector. If the tranng set of R mages s defned by, 2, R, then the covarance matrx s defned as: R T R I T where,, DR 2 R R and R x R () (2) s the mean mage of the tranng set. Also the dmenson of the covarance matrx s D D. The egenvalues and egenvectors are then calculated from the covarance matrx. Let,, DR Q Q Q2 Q r R (generally, r < R) be the r normalzed egenvectors correspondng to r largest egenvalues. Each of the r egenvectors s called an Egenface. Now, each of the face mages of the tranng set s projected nto the Egenface space to obtan ts cor- DR respondng Egenface based feature Z R, whch s defned as: Z QTY,.2,, R (3) where Y s the mean-subtracted mage of 3. Genetc Programmng []. Genetc programmng s a methodology nspred by bologcal evoluton to fnd equatons, computer programs, analog crcuts or n general any sutable structure for a predefned problem [9]. Genetc programmng s general mechansms are almost dentcal to genetc algorthms, as genetc programmng s consdered ether a specalzed form of genetc algorthms or an expanson of t [6]. Genetc programmng s usually mplemented smlar to the followng algorthm: ) Create ntal populaton. Indvdual solutons (call- ed chromosomes) are usually generated randomly. 2) Evaluate the ftness of each ndvdual n the populaton. 3) Select best-rankng ndvduals to reproduce. 4) Breed new generaton through crossover and/or mutaton (genetc operatons) and gve brth to offsprng. 5) Repeat from step 2 untl a termnaton condton s reached (tme lmt or suffcent ftness acheved). Fgure llustrates the general Genetc programmng algorthm. In GP ndvdual populaton members (chromosomes) are not fxed length lnear character strngs that encode possble soluton to the problem lke n GA, but they are programs that, when executed, provde soluton to the problem. These programs are expressed n GP as parse trees of varyng szes and shapes, what makes these methods flexble n ther applcaton to the wde range of problems. The dfference n chromosomes representaton s the man and almost the only dfference between ths method and GA. The overall Darwnan dea of survval stays the same, but there are changes n mutaton and crossover operators and n ftness functon calculaton. Each node of tree can be functon, operator, varable or constant number. Trees can be evaluated n a recursve manner, n whch each node s operator or functon s executed up on the results of ts chldren s evaluaton. Tree structure can easly represent a mathematcal equaton or a Turng complete program. NO Create Intal Populaton Evaluate Ftness of Each Indvdual Apply Selecton Crossover /Mutaton Termnaton Crtera Reached? END YES Fgure. Genetc programmng s flowchart.

3 72 A Genetc Programmng-PCA Hybrd Face Recognton Algorthm 4. Classfcaton Algorthm where f j,m s result of nth teraton on the jth group, n s the teraton number from total N repettons, and err j,k s 4.. Usng Genetc Programmng sum of total errors for all mages n the tranng group. To classfy a gven dataset, t s usually enough to fnd a To determne a new mage s class, all values acqured way for dfferentatng classes. Usng genetc programmng, ths translates to fndng a functon whch outputs a est C s nomnated as the new canddate class. It should from (6) are compared. The class whch yelds the great- unque value for each dfferent class: be noted that a threshold could be defned, as f the results of all classfers for an mage yeld lower than a 0 C0 certan value, the mage s certanly msclassfed. C f (4) 5. Smulaton and Results n C Ths s proven to be dffcult. As a result, genetc programmng s used to fnd a functon per class that can dscrmnate only a certan class from others: f C 0 C Ths method creates N dfferent functons for a total of N classes. Test mages are tested one by one aganst the functons, and the frst functon to return a non-zero value s used to determne the mage s class Leveragng Algorthm Leveragng s a method of usng multple results to mprove detecton. A leveragng algorthm employs multple weak classfers to create a strong classfer. The followng leveragng algorthm s used n ths paper: Instead of usng all tranng mages as nput, the whole group s parttoned to k dfferent groups. Fgure 2 shows sample face mages whch are parttoned to three varous clusters. Detector functon f,j s then obtaned as a functon whch can detect class from other classes n group j. To further mprove the results of classfcaton, algorthm above could be repeated N tmes. For a gven mage, the followng equaton creates the results of classfcaton: C f jn, p jn, errj, n N n The algorthms were mplemented n Python and then were tested on the AT&T face mage database [2]. The AT&T database conssts of 40 groups, each contanng ten 2 92 gray scale mages of a sngle subject. Each subject s mages dffer n lghtng, facal expresson and detals (smlng/frownng, glasses/no glasses, etc.). (5) Two set of mages were created from the AT&T database; For the Fve-to-Fve dataset, fve random mages of each group were selected for tranng whle the others were used for testng. For the Leave-One-Out set, 9 mages were used for tranng and the remanng mage was kept for valdaton. Frst Genetc Programmng was tested wthout leveragng. To evolve the populaton, an Evolutonary Strategy (ES) of + λ wth λ = 4 was chosen. Mutaton rate was set to 5 percent. The selected functon set was {+,,, <, >, MIN, MA, AND, OR, NOT, CNST} where Boolean operators frst compare ther operands wth 0 and CNST returns a random constant floatng pont number n range of [ 0, 0]. Inputs were chosen from all avalable PCA results. To lmt algorthm tme and prevent bloat, each chromosome s depth was lmted to 25 and a maxmum of teratons for each evoluton was mantaned. To test the leveraged algorthm, algorthm was executed wth the same parameters. Also the number of teratons was set to N = 8, whle the set was dvded to k = 8 dfferent groups. (6) Table shows a few of dscovered relatonshp functons for a set of pctures. It could be seen that the generated formulas are often smple whle only dependng on a parttonng f p f p f2 p f3 p Fgure 2. Parttonng sample AT&T face database.

4 A Genetc Programmng-PCA Hybrd Face Recognton Algorthm 73 Table. Examples of Acqured Relatonshp Functons for Detectng Image Group. PCA[n] Is the Nth Value on PCA Vector. No Functon (PCA[8]- MA(PCA[9], PCA[7]))> PCA[2] 2 AND((PCA[2]< PCA[3]), MIN(PCA[], )) 3 (PCA[0] NOT(PCA[5])) 4 (PCA[] (PCA[2]> (PCA[2]-PCA[8]))) few components and as a result have a relatvely low computatonal overhead. Results are brought n Table 2, where they are compared to EgenFace [3] and SVM [3] clusterng methods. It s observed that Genetc Programmng wthout leveragng has the worst results. On the other hand, Leveraged Genetc programmng beats other methods n Fve-to- Fve. In leave-one-out the results are repeated for Genetc Programmng, although ths tme Leveraged Genetc Programmng fell %2.5 (one mage n total of 40 mages) short of SVM. Whle t could be nferred that the Genetc Programmng s usable as a feature detector, t s beleved that PCA s lmted n reducng data dmenson [4]. Further research s requred to nvestgate Genetc Programmng s results wth 2D PCA and Multlnear PCA. To further nvestgate Genetc Programmng s performance, number of parttoned class groups was changed and the results were brought n Table 3. It was observed that the further parttonng of the mages ncreases recognton error, whle decreasng k mght mandates ncrease n tme spent for Genetc Programmng s evoluton. 6. Conclusons Genetc programmng s a general purpose search algorthm that can be utlzed n classfcaton problems. In ths paper, Genetc programmng was exploted to classfy face mages. The results showed that Genetc Programmng alone s not sutable, as requred tme and computatonal overhead surpasses that of other methods, and also ts recognton rato s usually lower. Table 2. Comparson of Dfferent Algorthms Recognton Rate. Method Fve-to-Fve Leave One Out Egenface 87.0% 85.0% SVM 9.0% 95.0% GP 63.5% 67.5% Leveraged GP 9.5% 92.5% Table 3. Effect of Number of Parttons n Leveraged Genetc Programmng on Recognton Rate. Number of Parttons Recognton Rate % 4 9.5% 5 9.5% 8 9.0% % To mprove results, a leveragng algorthm was appled to Genetc Programmng. The leveraged Genetc Programmng showed a good recognton rate, comparable to or n some cases even better than that of other methods. REFERENCES [] S. Lu, Y. Tan and D. L, New research Advances of Facal Expresson Recognton, Internatonal Conference on Machne Learnng and Cybernetcs, Baodng, Vol. 2, July 2009, pp [2] I. T. Jollffe, Prncpal Component Analyss, Sprnger- Verlag New York, Inc., [3] M. Turk and A. Pentland, Egenfaces for Recognton, Journal of Cogntve Neuroscence, Vol. 3, No., 99, pp [4] A. Pentland, B. Moghaddam and T. Starner, Vew-Based and Modular Egenspaces for Face Recognton, Proceedngs CVPR 94, 994 IEEE Computer Socety Conference on, Seattle, July 994, pp [5] A. Eleyan and H. Demrel, PCA and LDA Based Face Recognton Usng Feedforward Neural Network Classfer, Lecture Notes n Computer Scence, Vol. 405, 2006, pp do:0.007/848035_28 [6] J. R. Koza, Genetc Programmng: On the Programmng of Computer by Means of Natural Selecton, MIT Press: Cambrdge, 992. [7] S. uesong and Y. Zhou, Gray Intensty Images Processng for PD Pattern Recognton Based on Genetc Programmng, Internatonal Jont Conference on Artfcal Intellgence JCAI 09, Hakou, 2009, pp [8] A. Teredesa and V. Govndaraju, Issues n Evolvng GP Based Classfers for a Pattern Recognton Task, Proceedngs of the 2004 IEEE Congress on Evolutonary Computaton, June 2004, pp [9] J. R. Koza, M. A. Keane, M. J. Streeter, W. Mydlowec, J. Yu and G. Lanza, Genetc Programmng IV: Routne Human-Compettve Machne Intellgence, Kluwer Academc Publshers, Norwell, [0] N. Krause and Y. Snger, Leveragng the Margn More Carefully, Proceedngs of the Twenty-Frst Internatonal Conference on Machne Learnng, Banff, 2004, p. 63.

5 74 A Genetc Programmng-PCA Hybrd Face Recognton Algorthm [] J. K. Sng, S. Thakur, D. K. Basu and M. Naspur, Drect Kernel PCA wth RBF Neural Networks for Face Recognton, IEEE TENCON Regon 0 Conference, Hyderabad, 2008, pp. -6. [2] AT&T, The Database of Faces, e.html. [3] Y. Q. Pan and Y. Lu, Face Recognton Usng Kernel PCA and Hybrd Flexble Neural Tree, Proceedngs of the 2007 Internatonal Conference on Wavelet Analyss and Pattern Recognton, Bejng, 2-4 November 2007, pp [4] J. Wang, Y. Chen and M. Adjouad, A Comparatve Study of Multlnear Prncpal Component Analyss for Face Recognton, 37th IEEE Appled Image Pattern Recognton Workshop, 2008, pp. -6. do:0.09/aipr