Face Recognition Method Based on Within-class Clustering SVM

Transcription

1 Face Recognton Method Based on Wthn-class Clusterng SVM Yan Wu, Xao Yao and Yng Xa Department of Computer Scence and Engneerng Tong Unversty Shangha, Chna Abstract - A face recognton method based on Wthn-class Clusterng SVM (CCSVM) s presented n ths paper n order to decrease the negatve effect caused by nosy tranng samples n the recognton process. Based on the dscontnuty of fnte samples dstrbuton n the hgh dmenson space and the exstence of nosy samples, frst, we re-cluster samples wthn the class, fnd out the cluster centres to form the vrtual classes, and then dvde vrtual classes of all the classes by SVM. Experment results show that ths method follows the dstrbuton law of ponts n hgh-dmensonal space and can acheve better performance than some tradtonal methods. Keywords - face recognton, wthn-class cluster, support vector machne (SVM) I. INTRODUCTION Face recognton s an mportant research topc n Bometrcs and has already become very hot n the feld of pattern recognton. Many scholars at home and abroad are devoted nto researches n ths feld. Maor statstcal pattern recognton methods nowadays nclude dscrmnant functon method, K-nearest neghbor classfcaton method, non-lnear mappng method and feature analyss method, etc. Support Vector Machne (SVM) [] s the statstcal learnng theory put forward by Vapnk based on structural rsk mnmzaton prncple and t has been appled n classfcaton and regresson problems. SVM method has been pad more and more attenton to because of ts outstandng characterstcs and promsng expermental performance. However, the face samples of the same class present great randomness because of a varety of noses n realty, such as sunlght, postures, etc. Many nosy samples (drty samples) n hgh-dmensonal space dffer greatly from other samples of the same class, so these samples can not be clustered together and show great clusterng dstnctveness. Recent research results show that samples of the same knd n hgh-dmensonal space are located n a non-lnear manfold [-3], however, the lmted number of sample ponts determnes ts dscontnuty, ths s, there are several cluster areas wthn the class n space. Therefore, the key s to determne the cluster center. Up to now, many clusterng methods have been proposed, such as Separaton- Based K-Means [4] algorthm, Densty-Based Spatal Clusterng of Applcaton wth Nose (DBSCAN) [5] algorthm, Bottom-up Clusterng usng Representatve (CURE) [6] algorthm and Grddng-and-Densty-based Clusterng In QUEst(CLIQUE)[7] algorthm. Each method has ts own mert and also has some lmtatons. In ths paper a face recognton method based on wthnclass cluster and SVM s proposed. Ths method s along wth the thnkng of CURE algorthm and the merts of SVM algorthm. Conductng cluster analyss wthn each class, several cluster areas can be found as vrtual classes accordng to ts spatal dstrbuton law and all vrtual classes can be classfed wth SVM classfer. Experments show that ths algorthm acheves satsfactory recognton effect. II. CLUSTER ANALYSIS BASED ON THE COVERAGE AND MERGING Defnton Vrtual center pont: In hgh-dmensonal space, for a pont p and a threshold R, wthn a cluster area determned by p (center) and R (radus) the average of all the ponts s x 0 card( R) card ( R) x l l, where s the lth sample pont, s the dstance between a sample pont and pont p, card ( R) s the pont set satsfyng condton R. Defnton Representatve areas: Each vrtual center pont represents the hyper ball area wth ths pont as the center and R as the radus, so ths area s called representatve area. Defnton 3 Adacent center: Gven radus R, f the dstance of two vrtual ponts p and q satsfes dst ( p, q), then p and q are the adacent centers, where s the merger threshold. The key to cluster analyss s to fnd the cluster center and t can be dvded to three steps: Step, determne coverage radus and merger threshold; Step, search for the canddate cluster centers and elmnate the ones that don t meet the densty requrements; Step 3, merge the adacent canddate centers and decde the fnal cluster center. The threshold radus for the coverage represents the dstrbuton law for the sample ponts n hgh-dmensonal space and can be determned by the analyss for the samples of the same knd. The tranng sample set S wth n tranng x l /08 /$ IEEE CIS 008

2 samples can be dvded nto vrtual tranng set and vrtual testng set S and for each x S,,... n, calculates mn x x, x S,,... m () Radus R for coverage s determned as the average dstance between the vrtual tranng set and all the ponts from the vrtual testng set, so t s shown as: card S R card S () Merger threshold s determned by average dstance between couples of vrtual center ponts and the formula s: { max,..., t, t... t} Where,... t are the dstances between vrtual center ponts and t s the mean vector of all the current vrtual center ponts. After R s determned, cluster centers can be searched out by the followng steps: n () Intalzaton:, ntalze canddate center set as candcenter, set the number of teraton as terate ; () Get sample ponts x, x S from the sample queue of current class; (3) Calculate the Eucldean dstances from x to each present vrtual center ponts center. If R, then x (4). belongs to center t t S (3) s representatve area, else swtch to (4) x becomes the new vrtual center pont and add candcenter nto set. (5) If the sample queue for the current class s not empty, then repeat () to (4), else swtch to (6); (6) Obtan the number num of the sample ponts n the representatve area of every canddate center. If num, then ths vrtual center should be elmnated from candcenter to be an unclassfed sample pont. n n and f n terate, then update vrtual center vector and swtch to (), else swtch to (7); (7) Every unclassfed pont becomes a new vrtual center. If two vrtual center ponts are adacent, then the correspondng representatve areas should be merged, that s to say that all ponts n two representatve areas should be x merged, and then the updated vrtual center ponts wll become fnal cluster centers. III. RECOGNITION METHOD CASED ON WITHIN- CLASS CLUSTERING SVM (CCSVM) The basc prncple of SVM s to map the nput space to hgh-dmensonal space through non-lnear transformaton and get the optmal lnear classfcaton plane accordng to mnmal structure crteron n the new space and ths nonlnear transformaton s completed by defnng proper nnerproduct functon. Suppose the Lnear separable sample set n x, y,... n ( x R, y {, }, and y are class labels), the general form for lnear dscrmnant functon s gx w x b and classfcaton plane equaton s w x b 0. Normalze the dscrmnant functon and for, so the nearest all the samples n two classes make g x sample to the classfcaton plane can x classfcaton margn be g to make the / w, therefore, the maxmum margn means the mnmum w ; Requrng that all samples should be correctly classfed by classfer, ths s: w x b 0,, n y... (4) So the classfcaton plane, whch meets the above mentoned condtons and make w mnmum, s the optmal classfcaton plane. The optmal classfcaton plane problem can be consdered as solvng the constraned optmzaton problem that s restrcted by (4), the mnmum value for the functon s: w w w w (5) Ths can be resolved by Lagrange Multpler Method. For non-lnear problem, all the ponts should be consdered to be mapped nto the hgh-dmensonal space n order to be lnear separable n ths space and only conduct nner-product operaton for lnear dscrmnaton n hgh-dmensonal space. There s no need to know the non-lnear transformaton form, so avodng complcated calculaton n hgh-dmensonal transformaton, the problem s greatly smplfed. Accordng to Hlbert-Schmdt prncple, t can be used as nner-product functon only f one operaton can satsfy Mercer condton. SVM s two-class classfer and for multple-class pattern recognton problem, a combnaton scheme for multple SVM classfers can be appled, such as -a-r multple-class classfcaton [8]. Each class n sample set can be clustered based on the coverage and mergence method proposed n secton II. Correspondng to the multple cluster centers, the samples of the same knd can be dvded to several vrtual

3 classes and all of them can be classfed by SVM. Because of the ncrease of the class numbers, after re-clusterng wthn class, f -a-r method [9] proposed by Knerr s appled, then KK / classfers should be constructed and the decson-makng effcency wll decrease. In order to solve ths problem, for the tranng samples n K vrtual classes, let the sample n the frst class postve and the samples n nd, 3rd,, Kth classes negatve to tran the frst SVM. The th SVM(SVM ) s traned by the samples whch take the th class as postve and the th, th, K th classes as negatve. Fnally takng the sample of K th class as postve and the K th class negatve the K th SVM can be traned to be SVMK. Therefore, for a K -classfcaton problem, effcency for classfcaton can be greatly mproved because only K two-class classfers have to be traned. IV. EXPERIMENTS AND ANALYSIS ORL, Yale and YaleB face database have been appled n the experments so as to prove the valdaton of the algorthm. In order to assure the same dmenson for dfferent tranng samples of each face pattern, face mages for each person are pre-processed and the feature dmenson can be reduced through Prncple Component Analyss (PCA). A. Experment results n ORL database ORL database ncludes mages of 40 dfferent persons and 0 x9 gray mages for each person. Ths database s manly nfluenced by posture and expresson. For ORL database, 40 face patterns are appled and k mages for each person are randomly chosen as tranng samples and the rest for each are tranng samples. 0 k The tradtonal K-nearest neghbor (NN) method, SVM method and CCSVM method put forward n ths paper are appled n the experments, n whch the polynomal kernel Kx, x x x s chosen for kernel functon, and multple-class classfcaton mentoned n Secton III s chosen for SVM and CCSVM methods. We conduct several experments wth dfferent k values and calculate the mean of the testng results under the same k value. The results are shown n Fg.. We can see that wth nsuffcent samples mult-cluster areas are not easy to generate n the same-class-sample because dfferent face samples of the same knd only have tny expresson and pose changes.therefore all the methods used n experment dffer a lttle from each other. Fg. recognton rate n ORL database Experment n ORL database for SVM and CCSVM methods wth dfferent kernel functons when k 0 and k 7 are compared n Table I, n whch polynomal kernel wth d, radus kernel functon and Sgmod kernel x x functon: K( x, x ) exp and 56 bx x K( x, x ) tan c are adopted n the 56 experment, and, b and c are set to 0.3, and respectvely. We can see that CCSVM dsplays better recognton effect on the bass of these kernel functons when compared wth SVM n the ORL database through relatvely small abnormal samples. Tme complexty n the recognton process s also compared and we fnd that t s not sgnfcantly ncreased wth the ncrease of total class numbers and ths proves the feasblty of ths algorthm. TABLE I. COMPARISON FOR RECOGNITION RATE WITH DIFFERENT KERNEL FUNCTION (ORL DATABASE) recognton SVM Runnng tme recognton CCSVM Runnng tme Polynomal Radus bass Sgmod B. Experment results n YALE database In order to test the effectveness of the method, Yale database s used n the experment. YALE face database ncludes mages of 7 dfferent persons and 33 mages for each person. We construct the experment wth 7 face patterns n YALE database and use k mages for tranng and the remanng 33-k mages for testng. Compared wth other databases, face mages n YALE database are nfluenced much more by dfferent llumnaton and expressons. Same experment as n ORL database s conducted and the result s shown n Fg..

4 database. We can see that CCSVM dsplays more advantages n recognton rate compared wth SVM because of the ncrease of abnormal samples (such as the llumnaton samples n Fg. 3). If mages (c) and (d) n Fg. 3 are testng samples, SVM wll msclassfy them nto the class represented by sample (a) and (d), but wth CCSVM method they wll be correctly classfed. TABLE II. COMPARISON FOR RECOGNITION RATE WITH DIFFERENT KERNEL FUNCTION (YALE DATABASE) Fg. recognton rate n YALE database We can see from Fg. that some sample ponts nfluenced much by nose (such as the sample ponts wth more llumnaton angle) wll partcpate n the tranng process n YALE database wth the ncrease of k. These samples may be much far way from the samples of the same knd and samples under the same llumnaton condton n space may be often clustered together (such as the face samples n Fg. 3) so as to generate new cluster area. If these abnormal samples are traned together wth other normal samples, msclassfcaton wll be generated wth samples under the same llumnaton condton. However, CCSVM method can separate these abnormal samples from samples of the same knd by clusterng to generate two vrtual classes and tran them respectvely. Ths method better follows the dstrbuton law of the samples n hgh-dmenson space and greatly mproves the recognton effect. recognton SVM Runnng tme recognton CCSVM Runnng tme Polynomal Radus bass Sgmod C. Experment results n YaleB database YaleB database s composed of llumnaton mages wth llumnatons from dfferent drectons, such as vertcal, horzontal and mxed polarzed lghts. In order to compare the robustness of NN, SVM and CCSVM nfluenced by llumnaton, we use 5 face mages of each person affected by llumnaton from all drectons to be a sample set and then dvde t nto 4 subsets randomly. One of them s used as tranng sample set and the others are used as testng sample sets, and then we get the average recognton rates whch are shown n Table III. We can see that the samples of the same person can not be clustered together n hgh-dmensonal space because mages n ths database are greatly nfluenced by llumnaton, however, mult-cluster area are generated accordng to dfferent polarzed angle (for example, samples of the same knd wth the polarzed angle rangng from 0 to 30 degrees may be clustered together, and they keep a relatvely far dstance from samples wth the polarzed angle less than 0 degrees). CCSVM dsplays much more advantages compared wth SVM and recognton rate have been ncreased sgnfcantly (for example, recognton can be ncreased to 7.8% when Subset 3 s used as tranng set). CCSVM based on vrtual class n dfferent cluster area wll obtan relatvely satsfactory recognton effect even wthout the denosng feature extracton process. (a) (b) TABLE III. AVERAGE CORRECT RECOGNITION RATE WITH DIFFERENT SUBSETS AS TRAINING SAMPLE SET (YALEB DATABASE) Subset Subset Subset 3 Subset 4 NN SVM CCSVM (c) (d) Fg. 3 Sample mages under the same llumnaton condton SVM and CCSVM wth dfferent kernel functon are compared n Table II when k 0 and k 7 n YALE database. Parameter selectons are the same as those n ORL V. CONCLUSIONS A face Recognton method based on Wthn-class Cluster and SVM s proposed n ths paper. Based on the dstrbuton dscontnuty of the samples n space and the exstence of nosy samples, ths method forms many cluster areas and searches out the correspondng cluster centers by cluster analyss, dvdng each class nto several vrtual classes, all vrtue class can be classfed by SVM. Many experments show that CCSVM follows the dstrbuton law of feature

5 ponts n hgh-dmensonal space and acheves better recognton effect. REFERENCES [] [] O. Edgar, F. Robert, G. Federco. Tranng support vector machnes: an applcaton to face detecton. IEEE Conference on Computer Vson and Pattern Recognton, San Juan, Puerto Rco.997. [] [] J. B. Tenenbaum, V. DE Slva, J. C. Langford. A global geometrc framework for nonlnear dmensonalty reducton. Scence,000,90(5500): [3] [3] S. Rowes, ll. K. Saul. An ntroducton to locally lnear embeddng. [4] [4] L. Kanfan, PJ. Rousseeuw. Fndng Groups n Data: an Introducton to Cluster Analyss. New York: John Wley& Sons [5] [5] M. Ester, H. Kregel, J. Sander, X. Xu. A densty based algorthm for dscoverng clusters n large spatal databases wth nose. In: Smouds E, Han JW, Fayyad UM. eds. Proceedngs of the nd Internatonal Conference on Knowledge Dscovery and Data Mnng. Portland: AAAI Press, 996, 6-3. [6] [6] G. Sudpto, R. Raeevand R. Kyuseok. Cure: an effcent clusterng algorthm for large databases. Informaton Systems, 00, 6(3):8-37 [7] [7] R. Agrawal, J. Gehrke, D. Gunopolos, P. Raghavan. Automatc subspace clusterng of hgh dmensonal data for data mnng applcaton. In : Haas LM. Twary A, eds. Proceedngs of the ACM SIGM OD Internatonal Conference on Management of Data. Seattle: ACM Press [8] [8] C. W. Hsu, C. J. Ln. A comparson of methods for multclass support vector machnes. IEEE Transactons on Neural Networks,00,3() : [9] [9] U. KreBel. Parwse classfcaton and support vector machnes.advances n Kernel Methods-Support Vector Learnng. Cambrdge, MA;MIT Press,