Eigeimages Uitary trasforms Karhue-Loève trasform ad eigeimages Sirovich ad Kirby method Eigefaces for geder recogitio Fisher liear discrimat aalysis Fisherimages ad varyig illumiatio Fisherfaces vs. eigefaces Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 1
Image recogitio usig liear projectio To recogize complex patters (e.g., faces), large portios of a image (say N pixels) have to be cosidered High dimesioality of image space results i high computatioal burde for may recogitio techiques Example: earest-eigbor search requires pairwise compariso with every image i a database Trasform is a projectio o a J-dimesioal liear subspace that greatly reduces the dimesioality of the image space c = Wf Idea: tailor the projectio to a set of represetative traiig images ad preserve the saliet features by usig Pricipal Compoet Aalysis (PCA) JxN projectio matrix with orthoormal rows. W opt = argmax W ( det( WR ff W H )) J << N Mea squared value of projectio Autocorrelatio matrix of image Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 2
Image recogitio usig liear projectio 2-d example: f 2 Goal: project samples o a 1-d subspace, the perform classificatio. f 1 x Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 3
Image recogitio usig liear projectio 2-d example: f 2 Goal: project samples o a 1-d subspace, the perform classificatio. f 1 Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 4
Image recogitio usig liear projectio 2-d example: f 2 Goal: project samples o a 1-d subspace, the perform classificatio. f 1 x x Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 5
Uitary trasforms! Sort pixels f [x,y] of a image ito colum vector f of legth N Calculate N trasform coefficiets c = Af The trasform A is uitary, iff where A is a matrix of size NxN A 1 = A *T A H Hermitia cojugate If A is real-valued, i.e., A=A*, trasform is orthoormal Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 6
Eergy coservatio with uitary trasforms c = A f For ay uitary trasform we obtai c 2 = c H c = f H A H A f = f 2 Iterpretatio: every uitary trasform is simply a rotatio of the coordiate system (ad, possibly, sig flips) Vector legth is coserved. Eergy (mea squared vector legth) is coserved. Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 7
Eergy distributio for uitary trasforms Eergy is coserved, but, i geeral, uevely distributed amog coefficiets. Autocorrelatio matrix R cc = E c c H = E A f f H A H = AR ff AH Diagoal of R cc comprises mea squared values ( eergies ) of the coefficiets c i 2 E c i = R cc i,i = AR ff A H i,i Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 8
Eigematrix of the autocorrelatio matrix Defiitio: eigematrix Φ of autocorrelatio matrix R ff l Φ is uitary l The colums of Φ form a set of eigevectors of R ff, i.e., R ff Φ = ΦΛ Λ is a diagoal matrix of eigevalues λ ι Λ = λ 0 0 λ 1! 0 λ N 1 l R ff is ormal matrix, i.e., R H H ff R ff = R ff R ff, hece uitary eigematrix exists ( spectral theorem ) l R ff is symmetric oegative defiite, hece λ i 0 for all i Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 9
Karhue-Loève trasform Uitary trasform with matrix A = Φ H Trasform coefficiets are pairwise ucorrelated R cc = AR ff A H = Φ H R ff Φ = Φ H ΦΛ = Λ Colums of Φ are ordered accordig to decreasig eigevalues. Eergy cocetratio property: l No other uitary trasform packs as much eergy ito the first J coefficiets. l Mea squared approximatio error by keepig oly first J coefficiets is miimized. l Holds for ay J. Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 10
Illustratio of eergy cocetratio Strogly correlated samples, equal eergies f 2 f 1 A = cosθ siθ siθ cosθ c 2 c 1 After KLT: ucorrelated samples, most of the eergy i first coefficiet Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 11
Basis images ad eigeimages For ay trasform, the iverse trasform!! f = A 1 c ca be iterpreted i terms of the superpositio of colums of A -1 ( basis images ) For the KL trasform, the basis images are the eigevectors of the autocorrelatio matrix R ff ad are called eigeimages. If eergy cocetratio works well, oly a limited umber of eigeimages is eeded to approximate a set of images with small error. These eigeimages spa a optimal liear subspace of dimesioality J. Eigeimages ca be used directly as rows of the projectio matrix JxN projectio matrix with orthoormal rows W opt = argmax W ( det( WR ff W H )) Mea squared value of projectio Autocorrelatio matrix of image Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 12
New Face Image Eigeimages for face recogitio f c f f Normalizatio + - f Mea Face W Projectio Database of Eigeface Coefficiets p 1! p K Similarity measure!! (e.g., c T pk* ) Class of most similar p k Similarity Matchig Rejectio 1 θ k* Recogitio Result Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 13
Computig eigeimages from a traiig set How to obtai NxN covariace matrix?! l Use traiig set Γ 1, Γ! 2,, Γ! L+1 (each colum vector represets oe image) l Let be the mea image of all L+1 traiig images l µ Defie traiig set matrix S = ( Γ! 1 "! µ, Γ! 2 "! µ, Γ! 3 "! µ,, Γ! L "! µ ), ad calculate scatter matrix R =! Γ l "! µ Problem 1: Traiig set size should be L +1>> N If L < N, scatter matrix R is rak-deficiet Problem 2: Fidig eigevectors of a NxN matrix. L l=1! Γ l "! µ ( )( ) H = SS H Ca we fid a small set of the most importat eigeimages from a small traiig set L << N? Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 14
Sirovich ad Kirby algorithm Istead of eigevectors of SS H, cosider the eigevectors of S H S, i.e., Premultiply both sides by S S H S v i = λ i vi SS H S v! i = λ i S v! i By ispectio, we fid that S v i are eigevectors of SS H Sirovich ad Kirby Algorithm (for L << N ) l Compute the LxL matrix S H S v i l Compute L eigevectors of S H S l Compute eigeimages correspodig to the L 0 L largest eigevalues as a liear combiatio of traiig images S v i L. Sirovich ad M. Kirby, "Low-dimesioal procedure for the characterizatio of huma faces," Joural of the Optical Society of America A, 4(3), pp. 519-524, 1987. Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 15
Example: eigefaces The first 8 eigefaces obtaied from a traiig set of 100 male ad 100 female traiig images Eigeface 1 Eigeface 2 Eigeface 3 Eigeface 4 Mea Face Eigeface 5 Eigeface 6 Eigeface 7 Eigeface 8 Ca be used to geerate faces by adjustig 8 coefficiets. Ca be used for face recogitio by earest-eighbor search i 8-d face space. Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 16
Geder recogitio usig eigefaces Nearest eighbor search i face space Female face samples Male face samples Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 17
Fisher liear discrimiat aalysis Eigeimage method maximizes scatter withi the liear subspace over the etire image set regardless of classificatio task W opt = argmax W ( det( WRW H )) Fisher liear discrimiat aalysis (1936): maximize betwee-class scatter, while miimizig withi-class scatter W opt = argmax W ( ) ( ) det WR B W H det WR W W H R B = Samples i class i R W = c c i=1 i=1 Γ l N ( i µ ) i µ i Class(i) µ ( ) H µ i µ Mea i class i ( Γ ) l Γ l µ i ( ) H Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 18
Fisher liear discrimiat aalysis (cot.)!"! Solutio: Geeralized eigevectors w i correspodig to the J largest eigevalues { λ i i = 1,2,..., J}, i.e.!"! = λi R W w i!"!, i = 1,2,..., J R B w i Problem: withi-class scatter matrix R w at most of rak L-c, hece usually sigular. Apply KLT first to reduce dimesioality of feature space to L-c (or less), proceed with Fisher LDA i lower-dimesioal space Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 19
Eigeimages vs. Fisherimages 2-d example: f 2 Goal: project samples o a 1-d subspace, the perform classificatio. f 1 Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 20
Eigeimages vs. Fisherimages 2-d example: f2 KLT Goal: project samples o a 1-d subspace, the perform classificatio. The KLT preserves maximum eergy, but the 2 classes are o loger distiguishable. f 1 Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 21
2-d example: Eigeimages vs. Fisherimages f2 KLT Goal: project samples o a 1-d subspace, the perform classificatio. The KLT preserves maximum eergy, but the 2 classes are o loger distiguishable. Fisher LDA separates the classes by choosig a better 1-d subspace. f 1 Fisher LDA Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 22
Fisherimages ad varyig iillumiatio Differeces due to varyig illumiatio ca be much larger tha differeces amog faces! Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 23
Fisherimages ad varyig iillumiatio All images of same Lambertia surface with differet illumiatio (without shadows) lie i a 3d liear subspace Sigle poit source at ifiity surface ormal!! l light source Light source itesity ( ) L f ( x, y) = a( x, y) l!! T ( x, y ) directio Surface albedo Superpositio of arbitrary umber of poit sources at ifiity still i same 3d liear subspace, due to liear superpositio of each cotributio to image Fisherimages ca elimiate withi-class scatter Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 24
Fisherface traied to recogize geder Female face samples Mea image Female mea Male mea! µ! µ! 1 µ 2 Male face samples Fisherface Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 25
Geder recogitio usig 1 st Fisherface Error rate = 6.5% Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 26
Geder recogitio usig 1 st eigeface Error rate = 19.0% Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 27
Perso idetificatio with Fisherfaces ad eigefaces ATT Database of Faces 40 classes 10 images per class Digital Image Processig: Berd Girod, 2013-2015 Staford Uiversity -- Eigeimages 28