Recognzng Faces Drk Colbry Outlne Introducton and Motvaton Defnng a feature vector Prncpal Component Analyss Lnear Dscrmnate Analyss
!"" #$""% http://www.nfotech.oulu.f/annual/2004 + &'()*) '+)* 2
! & &" *, " - 3
( Faces wth ntra-subject varatons n pose, llumnaton, expresson, accessores, color, occlusons, and brghtness / Dfferent persons may have very smlar appearance. 4
+ Humans can recognze carcatures and cartoons How can we learn salent facal features? Dscrmnatve vs. descrptve approaches Face Modelng Challenges L Lghtng P Pose E Expresson I = 2D OG ( P( L( OL ( E( faceshape))))) I = 3D OL( E( faceshape)) O G Global Occluson (hands, walls) O L Local Occlusons (Har, makeup) 5
& & 0 /" 0 $!" 0 2 --"%! 0!'" http://www.peterme.com/archves/ 2004_05.html http://www.peterme.com/archves/2004_05.html www.3g.co.uk/pr/ March2005 /09.htm www.customs.gov.au/ ste/page.cfm?u=449 3-3-$45/ $4)678 9 44 - -"! -2* 3-* 6
3- Usng Cogntec engne, pctures of dead subject were compared wth photographs of Dllnger he best matchng score (wth smlar pose) was 0.29 A reference group (a subset of FERE) was used to construct genune and mposter matchng score dstrbutons 0.29 Dead subject Lvng subject Matchng score 0.29 0.29 0.38 0.22 /&+(.:;;: " 94)>7? ):)$<=67>$87>25/" 7
/&+(.:;;: - Left/rght and up/down show dentfcaton rates for the non-frontal mages Left/rght (morphed) and up/down (morphed) show dentfcaton rates for the morphed non-frontal mages. Performance s on a database of 87 ndvduals.,+ Algorthms Pose-dependency Pose-dependent Pose-nvarant Vewer-centered Images Object-centered Models Face representaton Matchng features Appearance -based (Holstc) Hybrd Featurebased (Analytc) -- Elastc Bunch Graph Matchng -- Actve Appearance Models PCA, LDA LFA -- Morphable Model 8
Analytc Approach x = [ d, d 2, d d 3, 2] Analytc Example Can be dffcult to obtan relable and repeatable anchor ponts. 9
Holstc Approach r 95 00 04 0 3 5 6... 9 2 [ p (,), p (,, p( 3), ( 4), ( 5), ( 6), ( 7 ), (, )] 2),, p, p, p, p, p c c x = One long vector of sze d = r c [ p,, (,) p, (,2) p, (,3) p,, (, c) p(2.) p(2,2), p( r, c) ] dx Dealng Wth Large Dmensonal Spaces Curse of Dmensonalty Every pont n the d dmensonal space s a pcture he majorty of the ponts are just nose Goal s to dentfy the regon of ths space wth ponts that are faces 0
Prncpal Components (2D Example) V 2 V Prncpal Component Analyss (a.k.a Karhunen-Loeve Projecton) Gven n tranng mages x, x 2, x 3,, x n Where each x s a d-dmensonal column vector Defne the mage mean as: µ = x n = Shft all mages to the mean: u µ U = u u u, ] = x [, 2, 3, u n dxn Defne d d Covarance Matrx on set X as: C = UU x n
Computaton of Prncpal Component Axes Goal, Calculate egenvectors V such that: UU he scatter matrx could be qute large How bg s a scatter matrx wth 240 rows 320 cols? Solvng for a large scatter matrx s dffcult V We need to use a lnear algebra method to solve a much smaller matrx = λv he followng matrx s much smaller than the scatter matrx as long as n < d: U U Effcent Egenvector Calculaton Calculate the egenvectors (W ) of ths new matrx: U UW = λw It can be shown that: U UUW = λuw hen let V be a unt vector: UW V =, =,2,3,, n UW Solvng for W wll gve you V whch s what we want V V 2
Feature Space Reducton Order the egenvectors based on the magntude of the egenvalues λ λ λ Fnd the smallest b such that: λ α λ where α s the rato of nformaton loss (ex: α=0.5) he b vectors are used to defne the new PCA subspace: V, V2, V3,, V b λ n n = b+ n j= j Ideally b << d Vsualzng the Prncpal Component Axes as an Image µ x V V 2 V 3 V 4 V 5 V 6 V 7 b =5 V 8 V 9 V 0 V V 2 V 3 V 4 V 5 http://vsmod.meda.mt.edu/vsmod/demos/facerec/basc.html 3
Convertng Images to the New PCA Subspace Reorder Subspace Vectors n to a matrx: M = ] [ VV 2V3V b Project orgnal mage nto the new PCA sub-space: y = M U = M ( x µ) Reconstruct an approxmate orgnal mage from a subspace Vector: ' x = µ + M y x dxb Y = M U = M ( X µ) (Matrx Notaton) Reconstructed Images From M and µ x Calculate the new mage vector x new. Convert mage nto new subspace y new. y M ( x µ) new = new Remember, the sze of the y vector s less than the sze of the x vector Reconstruct the orgnal mage x = µ + M ' new y new x new = = x new Images from: http://vsmod.meda.mt.edu/vsmod/demos/facerec/basc.html 4
Problems wth PCA (2D Example) 0.4 0.3 0.2 V 2 V 0. 0-0. -0.2-0.3-0.4-0.3-0.2-0. 0 0. 0.2 0.3 0.4 0.5 0.6 Lnear Dscrmnate Analyss Uses the class nformaton to choose the best subspace ranng examples have class labels Samples of the same class are close together Samples of dfferent classes are far apart Start by calculatng the mean and scatter matrx for each class 5
Defnng LDA Space Compute the mean and covarance for the followng: All ponts: µ x C x Note: µ x = µ from PCA notaton K Classes, each wth µ C Let p be the apror probabltes of each class ypcally: p =/K LDA Goal s to fnd a W matrx that wll project the PCA subspace ponts Y nto a new (more useful) subspace Z: Z = W M ( X µ x ) = W M U = W Y he ransformaton W s defned as: Between class scatter S between max = max W Wthn class scatter W Swthn Where: S wthn = k = p C S between can be thought of as the covarance of data set whose members are the mean of each class k S between = p ( µ µ x )( µ µ x ) = 6
Solvng for W Goal Maxmze the followng: Between class scatter S max = max W Wthn class scatter W S he optmal projecton (W) whch maxmzes the above formula can be obtaned by solvng the generalzed egenvalue problem S between W = λs wthn W between wthn Revew of Lnear ransformatons Prncpal Component Analyss (PCA) Calculates a transformaton from a d-dmensonal space nto a new d-dmensonal space he axes n the new space can be ordered from the most nformatve to the least nformatve axs Smaller feature vectors can be obtaned by only usng the most nformatve of these axes (however some nformaton wll be lost) Lnear Dscrmnate Analyss (LDA) Uses the class nformaton to choose the best subspace whch maxmzes the between class varaton whle mnmzng the wthn class varaton Note: both PCA and LDA are general mathematcal methods and may be useful on any large dmensonal feature space, not just faces or mages 7
Lecture Revew here are two general approaches to face recognton: Analytc Holstc In the Analytc approach, features are determned by heurstc knowledge of the face In the Holstc approach, each pxel n an mage can be vewed as a separate feature dmenson PCA can be used to reduce the sze of the feature space If the faces are labeled, LDA can be used to fnd the most dscrmnatng subspace 8