Appl. Math. Inf. c. 6 No. pp. 8-85 (0) Appled Mathematcs & Informaton cences An Internatonal Journal @ 0 NP Natural cences Publshng Cor. wo-dmensonal upervsed Dscrmnant Proecton Method For Feature Extracton Janguo Wang Department of Computer cence & echnology angshan College angshan Chna Emal Address: wgfwg@63.com Receved: Receved May 05 0; Revsed July 7 0; Accepted eptember 0 Publshed onlne: January 0 Abstract: For supervsed dscrmnant proecton (DP)method the mage matrx data are vectorzed to fnd the ntrnsc manfold structure and the dmenson of matrx data s usually very hgh so DP cannot be performed because of the sngularty of scatter matrx. In addton the matrx-to-vector transform procedure may cause the loss of some useful structural nformaton embeddng n the orgnal mages. hus n ths paper a novel method called D supervsed dscrmnant proecton (DDP) for face recognton s proposed. he proposed method not only takes nto account both the local nformaton of the data and the class nformaton of the data to model the manfold structure but also preserves the useful nformaton of the mage data. o evaluate the performance of the proposed method several experments are conducted on the Yale face database and the FERE face database. he hgh recognton rates demonstrate the effectveness of the proposed method. Keywords: Feature extracton; Face recognton; upervsed dscrmnant proecton (DP); wo-dmensonal DP (DDP); Manfold learnng Introducton Up to now a number of methods for face feature extracton have been proposed []; he most wellknown appearance-based proecton technque s the Egenfaces method orgnally proposed by Krby and rovch and popularzed by urk and Pentland [3]; Another famous proecton method termed Fsherfaces was respectvely put forward by wets [4] and Belhumeur [5]. hs method s based on lnear dscrmnant analyss (LDA) and has shown to yeld better results than Egenfaces under varatons n llumnaton and facal expressons. As we know the PCA and the LDA methods all take consderaton of the global Eucldean structure of the mage data. But they have lttle to do wth the manfold structure of the data. If the data le on a submanfold whch reflects the nherent structure of the data space t s hard for PCA and LDA to fnd the hdden manfold. he UDP (Unsupervsed dscrmnant proecton) method s a recently developed face recognton method [6] whch characterzes the local scatter as well as the non-local scatter and mnmzes the local scatter. hs characterstc makes UDP more ntutve and more powerful than the most up-to-date method such as PCA LDA and LPP methods. Unfortunately a common nherent lmtaton s stll exsted n UDP method.e. class nformaton s not consdered n ths approach. Unlke the unsupervsed learnng scheme of UDP DP [7] follows the supervsed learnng scheme.e. t uses the class nformaton to model the manfold structure. In DP method the local structure of the orgnal data s constructed accordng to a certan knd of smlarty between data ponts whch takes specal consderaton of both the local nformaton and the class nformaton. However due to the hgh-dmensonal and small sample sze problem [89] encountered n face recognton DP method cannot be performed because of the sngularty of scatter matrx. In addton the matrx-to-vector transform procedure may cause the loss of some useful structural
8 W. Janguo: wo-dmensonal upervsed Dscrmnant Proecton... nformaton embeddng n the orgnal mages. Inspred by the successful applcaton of DPCA[0] and DLDA [] to face recognton we proposed a novel method called D supervsed dscrmnant proecton (DDP) to handle the above problems by drectly proectng the D face mage matrces rather than usng the transformed mage vectors. In the experments conducted on two benchmark face databases the proposed face recognton method s shown to outperform the DP UDP and DUDP methods. he paper of the rest s organzed as follows: ecton gves the outlne of the proposed method (DDP) and descrbes the detals of our algorthm. ecton 3 reports experments carred out and the results. he last secton presents our conclusons. he Proposed Method DDP.. Outlne of DP. o make ths paper more selfcontaned the DP procedure s gven n ths secton. In contrast wth UDP method the advantage of DP method s utlzng class nformaton to gude the procedure of feature extracton.e. t uses the class nformaton to model the manfold structure [7]. uppose X = x x L x ] s a set of [ M n M tranng samples n R. We can obtan the lnear transformaton ϕ by calculatng the generalzed egenvectors of the followng generalzed egenequaton: Nϕ = λlϕ (.) where L = H( x )( x ) (.) MM = = N = ( H )( x )( x ) (.3) MM = = he smlarty matrx H n Eq. (.) and Eq. (.3) s defned as follows: x x exp + exp f x s among K nearest neghbors of x and x s among K nearest neghbors of x and ω = ω. H( ) = x x exp exp f x s among K nearest neghbors of x and x s among K nearest neghbors of x and ω ω. 0 otherwse (.4) x where exp denotes the local weght x as UDP does + exp and x exp denotes the ntra-class dscrmnatng weght and nter-class dscrmnatng weght respectvely. In addton the parameter s used as a regulator whch controls the overall scale or the smoothng of the space. But the selecton of the parameter remans an open problem. From Eq. (.4) we know that f the parameter s set very low the value of the H ( ) wll be near zero for all but the closest ponts... Our Method (DDP). Consderng there s a set of M sample mages A = [ A A L AM ] each mage s a m n matrx let ω be an n - dmensonal untary column vector. he DDP method s proectng each mage ( A ) m n onto ω by the followng transformaton: Y = A ω = L M (.5) where Y s n - dmensonal proecton feature vector. o obtan the hghest recognton rate t s mportant to select the optmal proecton vector ω. he obectve functon of the two dmensonal DP method s defned as: Nω = λlω (.6) where (.7) L N = MM = = = = W ( A A )( A A ) = ( W )( A A)( A A) (.8) MM where W s the weght matrx whose elements denote the relatonshp between the mage A and A n the observaton space and s defned as:
. Chang-Guang H. Mnoru: Explctly olvable me-dependent... 83 A A A A exp + exp f A s among k nearest neghbors of A or A s among k nearest neghbors of A and τ = τ. W( ) = A A A A exp exp f A s among k nearest neghbors of A or A s among k nearest neghbors of A and τ τ. 0 otherwse (.9).3. Feature Extracton and Classfcaton. tep. Calculatng the generalzed egenvectors ϕ = ϕ ϕ Lϕ } of Eq. (.6) correspondng to { d the d largest postve egenvalues λ λ L λ d. tep. Mappng the gven sample A usng the proecton matrx ϕ : Y = ( ϕ ) A the feature Y s used to represent the sample A for recognton purposes. 3 Experments on Face Databases 3.. Experments Usng the FERE Face Database. he proposed algorthm s tested on a subset of the FERE database. hs subset ncludes 400 mages of 00 ndvduals (each ndvdual has seven mages). hs subset nvolves varatons n facal expresson llumnaton and pose. It s composed of the mages whose names are marked wth twocharacter strngs: ba bd be bf bg b bk. In our experment the facal porton of each orgnal mage was automatcally cropped based on the locaton of eyes and the cropped mage was reszed to 40*40 pxels. ome example mages of one person are shown n Fgure 3.. Fgure 3.: even mages of one person n the subset of FERE face database. In our test we use the frst the sxth and the seventh mages (.e. ba b and bk ) per class for tranng and the remanng four mages (.e. bd be bf and bg ) for testng. o evaluate the performance of the proposed method (DDP) we conducted the frst experment on FERE face databases. he number of selected egenvectors (proecton vectors) vares from to 8. Here let m denotes the proecton vector number then the dmenson of correspondng proected feature vector s 40* m. And the kernel parameter n DDP s chosen as =. Fnally a nearest neghbor classfer wth Cosne dstance s employed to classfy n the proected feature space. he recognton rates versus m are shown n able. able he recognton rates (RR) of DDP versus the dmensons (40* m ) when the nearest neghbor classfer wth Cosne dstance s used on the FERE database m 3 4 RR(%) 49.5 45 4.75 37.87 m 5 6 7 8 RR(%) 36.3 34.88 33. 30.88 o evaluate the effect of the dfferent kernel parameter we conducted the second experment on the FERE database. In ths test the DP and DDP methods are used for feacture extracton. Fnally a nearest neghbor classfer wth Cosne dstance s employed to classfy n the proected feature space. he recognton rates wth dfferent parameter are shown n able. able he recognton rates (%) and correspondng dmenson (shown n parentheses) of DP and DDP versus the parameter when the nearest neghbor classfer wth Cosne dstance s used on the FERE database 3 samples (the frst the sxth and the seventh) per class are used for tranng 5 0 DP 40.5(40) 46.88(35) 46.38(30) DDP 49.5(40*) 49.75(40*) 48.38(40*) 5 0 5 DP 45.(0) 45(35) 44.37(0) DDP 48.5(40*) 48.5(40*) 48.5(40*) From able we can see that the DDP method outperforms the DP method sgnfcantly. It shows that the proposed method (DDP) can extract more dscrmnatve features than the DP method. ths manly because the proposed one not only takes nto account the local nformaton of the data as DP method does but also preserves the correlatons between varatons of rows and those of columns of face mages. 3.. Experments Usng the Yale Face Database. he Yale database contans 65 grayscale mages of 5 ndvduals each ndvdual has mages wth
84 W. Janguo: wo-dmensonal upervsed Dscrmnant Proecton... 46 * 56 pxels under varous facal expressons and lghtng condtons (one per dfferent facal expresson or confguraton: center-lght w/glasses happy left-lght w/no glasses normal rght-lght sad sleepy surprsed and wnk.). Fgure shows some sample mages of one people n the Yale face database. Fgure 3.: Eleven mages of one person n the Yale face database In the experment the UDP DP DUDP and DDP methods are used for feacture extracton. Frst we used the frst sxth face mages per class for tranng and the rest of the fve face mages for testng. In ths experment the parameter was set as 5. Fnally a nearest neghbor classfer wth Cosne dstance s employed to classfy n the proected feature space. he recognton rates and correspondng dmensons are shown n able 3. able 3 he recognton rates (RR) and correspondng dmenson of UDP DPDUDP and DDP methods when the nearest neghbor classfer wth Cosne dstance s used on the Yale database the frst sxth samples per class are used for tranng Methods UDP DP DUDP DDP RR(%) 96 96 93.33 97.33 Dmenson 5 0 56*3 56*5 hen we reevaluate the recognton performance of the proposed method DDP. We repeated the recognton procedure 0 tmes by randomly choosng sx tranng sets per class. Here the UDP DP DUDP and DDP methods are used for feature extracton. Fnally the nearest neghbor classfer wth Eucldean dstance s employed for classfcaton. he recognton rates wth correspondng standard devatons and correspondng dmensons are shown n able 4. able 4 he average recognton rates (ARR %) wth correspondng standard devatons (td %) of four feature extracton methods (UDP DP DUDP and DDP) across 0 runs on the Yale database under the nearest neghbor classfer wth Cosne dstance Methods UDP DP DUDP DDP ARR(%) td(%) 93.87.8 96.4.88 96.53.43 97.6.76 Dmenson 35 35 56*3 56*5 From able 3 and able 4 we can see that the results are consstent wth those drawn from the experments conducted on the FERE face database. he conclusons are on the whole consstent wth those drawn from the frst experment conducted on the FERE face database. It further shows that the proposed method can extract more dscrmnatve features than the other methods. 4. Conclusons In ths paper we proposed a novel method called D locally dscrmnatng proecton (DLDP) for face recognton. hs method s developed based on the successful applcaton of DPCA (two-dmensonal prncpal component analyss) and DP (supervsed dscrmnant proecton) to face recognton. he proposed one not only takes nto account both the local nformaton of the data and the class nformaton of the data to model the manfold structure but also preserves the useful nformaton of the mage data. Expermental results show that the DDP method acheves hgher recognton than the DP method and other feature extracton methods. Acknowledgements hs work was supported by the Research Program of Hebe Educaton Department under Grant No. Z0074 and the Research Program of Hebe Muncpal cence & echnology Department under Grant No. 0355. References [] W. Zhao R. Chellappa A. Rosenfeld et.al Face recognton: A lterature survey ACM Comput. urv. 35 (003) 399 458. [] M. A. Grudn On nternal representatons n face recognton systems PR. 33 (000) 6-77. [3] M. urk and A. Pentland Egenfaces for recognton J. Cogn. Neurosc. 3 (99) 7-86. [4] D. L. wets and J.Weng Usng dscrmnant egenfeatures for mage retreval IEEE rans. PAMI. 8 (996) 83-836. [5] P. N. Belhumeur J. P. Hespanha D. J. Krengman Egenfaces versus Fsherfaces: recognton usng class specfc lnear proecton IEEE rans. Patt. Anal. Mach. Intell. 9 (997) 7-70. [6] J. Yang D. Zhang J. y. Yang et.al Globally Maxmzng Locally Mnmzng: Unsupervsed Dscrmnant Proecton wth Applcatons to Face and Palm Bometrcs IEEE rans. Pattern Anal. Machne
. Chang-Guang H. Mnoru: Explctly olvable me-dependent... 85 Intell. 9 (007) 650-664. [7] J.G. Wang J.Z. Hua C.Y. Zhou upervsed Dscrmnant Proecton for Feature Extracton ICCAE 0. (0) 00-03. [8] L. F. Chen H.Y.M. Lao M..Ko et al A new LDAbased face recognton system whch can solve the small sample sze problem Pattern Recognton Lett. 33 (000) 73 76. [9] H. Yu and J. Yang A drect LDA algorthm for hghdmensonal data wth applcaton to face recognton Pattern Recognton Lett. 34 (00) 067 070. [0] J. Yang D. Zhang A. F. Frang et al wo-dmensonal PCA: a new approach to appearance- based face representaton and recognton IEEE rans. PAMI. 6 (004) 3-37. [] M. L and B. Yuan D-LDA: a statstcal lnear dscrmnant analyss for mage matrx PRL. 6 (005) 57 53. Janguo Wang receved the M degree n sgnal and nformaton processng at X'an Unversty of technology n 997 and the PhD degree n computer applcaton from the Nanng Unversty of cence and echnology the Department of Computer cence n 008. He s currently an professor n the Department of Computer cence & echnology angshan College. Hs research nterests are n the areas of pattern recognton mage processng face detecton and recognton.