Face Recognition University at Buffalo CSE666 Lecture Slides Resources:

Transcription

1 Face Recognton Unversty at Buffalo CSE666 Lecture Sldes Resources:

2 Overvew of face recognton algorthms Correlaton - Pxel based correspondence between two face mages Structural - Based on dentfyng landmark ponts Lnear models - PCA (Prncpal Component Analyss) - LDA (Lnear Dscrmnant Analyss) - ICA (Independent Component Analyss) - Combnatons of above Non-lnear models -Kernel mappng (and usng PCA, LDA, ICA) -Actve shape/appearance models -Manfold mappng 2D Model vs. 3D Model Matchng vs. Classfcaton

3 Test Image captured by camera FFT Correlaton Matchng Frequency Doman array N x N pxels Resultng Frequency Doman array Correlaton Flter H (Template) N x N pxels IFFT PSR *B.V.K. Vjaya Kumar, Maros Savvdes, C. Xe, K. Venkataraman, J. Thornton and A. Mahalanobs, Bometrc Verfcaton usng Correlaton Flters, Appled Optcs, 2003 *B.V.K. Vjaya Kumar, M. Savvdes, K. Venkataraman, C. Xe, "Spatal frequency doman mage processng for bometrc recognton," IEEE Proc. of Internatonal Conference on Image Processng (ICIP), Vol. I, 53-56, 2002

4 Elastc Bunch Graph Matchng L. Wskott, J.-M. Fellous, N. Krueuger, C. von der Malsburg, Face Recognton by Elastc Bunch Graph Matchng, IEEE Trans. on Pattern Analyss and Machne Intellgence, Vol. 19, No. 7, 1997

5 Elastc Bunch Graph Matchng Heurstc matchng algorthm L. Wskott, J.-M. Fellous, N. Krueuger, C. von der Malsburg, Face Recognton by Elastc Bunch Graph Matchng, IEEE Trans. on Pattern Analyss and Machne Intellgence, Vol. 19, No. 7, 1997

6 Prncpal Component Analyss (PCA) Prevous dscusson: PCA as a feature extracton method Features T a - egenvectors of matrx R = E[ xx ] A = ( ) a - Karhunen-Loeve transform Representaton x y = Ax - projecton of orgnal vector to KL bass x x = n = 1 y a - representaton of orgnal vector n x KL bass vectors y A x m m = - frst PCA coeffcents m best features xˆ = m = 1 y a - the projecton on subspace spanned by egenvectors wth largest egenvalues m E[( x xˆ ) 2 ] = λ n = m sum of the smallest n m egenvalues

7 PCA as appearance model x xˆ y A x m m = - frst PCA coeffcents m best features - should provde a good approxmaton to all the samples (for whch PCA was traned) Dstance from feature space (DFFS) should be small: DFFS = x xˆ PCA provdes a model on typcal faces DFFS can be used t to separate faces from non-faces DIFS = xˆ ˆ 1 x 2 x ˆ = T A m y DIFS (dstance n feature space) can be used to match two faces m

8 PCA for face recognton Collect a set of face mages for tranng Tran PCA Durng face recognton see f the dstance between test and enrolled PCA feature vectors s less than threshold: xˆ xˆ = y < θ y 2 xˆ y - reconstructed face (hyperplane n the orgnal feature space) - projecton nto PCA feature space M. Turk, A. Pentland, Egenfaces for Recognton, Journal of Cogntve Neuroscence, Vol. 3, No. 1, 1991

9 PCA subspace on tranng samples 2 T If x has dmenson N, then R x = E[ xx ] has dmenson 2 N N 2 Suppose we have M tranng samples X = [ x1 x 2 x M ] T R x = XX matrx of dmenson 2 N N 2 T Instead consder L = X X of dmenson M M If v s an egenvector of L then Xv s an egenvector of R x : X T Xv = T λ v XX Xv = λxv The number of tranng samples s usually less than the dmenson of 2 mage vectors M < N, so ths procedure makes sense M. Turk, A. Pentland, Egenfaces for Recognton, Journal of Cogntve Neuroscence, Vol. 3, No. 1, 1991

10 Lnear Dscrmnant Analyss Covarance matrx for class : S = E [( x µ )( x µ Wthn-class scatter matrx: S = M ) w P = 1 T S ] Between-class scatter matrx: where M µ 0 = µ = 1 P S = M b P = 1 ( µ µ 0 )( µ µ 0 ) T Optmzaton crtera: maxmze det( S b ) 1 = det( S w S det( S ) w b )

11 Lnear Dscrmnant Analyss Soluton: projecton s determned by the egenvectors of S 1 S w b Usually has better performance than PCA Requres samples of the same class (same person face) to tran Need to make sure that s non-sngular S w S w Soluton: apply PCA frst to reduce the number of dmensons, then perform LDA.

12 Independent Component Analyss Whereas PCA makes uncorrelated features, there mght be stll dependent ICA tres to reduce hgher order dependence Search for proper projecton s more dffcult Use approxmaton approaches Bartlett et al. Face Recognton by Independent Component Analyss, 2002

13 Comparsons of subspace methods Delac et al. Independent Comparatve Study of PCA, ICA and LDA on the FERET Data Set, 2006

14 Comparsons of subspace methods Delac et al. Independent Comparatve Study of PCA, ICA and LDA on the FERET Data Set, 2006

15 Kernel Methods Kernel mappng: -map orgnal mage feature vectors nto hgher dmensonal space usng some kernel functons: N f Φ : R R, - Covarance matrx n kernel space has elements Φ( x ) Φ( x j ) j - No need to explctly calculate Φ( x ) - use some choosen kernel 2 functon: - The number of egenvectors s stll lmted by the number of tranng samples as n regular PCA f >> - Kernel trck: there exsts a functon k ( x, x ) = Φ( x ) Φ( x ) = K k x x 2σ ( x, x j ) = e 2 j j N j

16 Kernel Methods M.-H. Yang, Face Recognton Usng Kernel Methods, 2002

17 Local Lnear Embeddng Rowes S., Saul L. Nonlnear dmensonalty reducton by locally lnear embeddng, Scence, 2000

18 Local Lnear Embeddng Step 1: fnd optmal local lnear representaton of pont as a combnaton of ts neghbors X ε ( W ) X W X = j W j mn Step 2: fnd lower dmensonal representaton Y of Φ ( Y ) = Y W Y mn j W j = Rowes S., Saul L. Nonlnear dmensonalty reducton by locally lnear embeddng, Scence, 2000 j = 1 0 j j j f j s not a neghbor of j j X X

19 Local Lnear Embeddng Rowes S., Saul L. Nonlnear dmensonalty reducton by locally lnear embeddng, Scence, 2000

20 Local Lnear Embeddng

21 3D Model Based Recognton Blanz, Vetter Face recognton based on fttng a 3D morphable model PAMI

22 3D Model Based Recognton Blanz, Vetter Face recognton based on fttng a 3D morphable model PAMI

23 3D Model Based Recognton Blanz, Vetter Face recognton based on fttng a 3D morphable model PAMI