Eigenimages. Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 1

Transcription

1 Eigeimages Uitary trasforms Karhue-Loève trasform ad eigeimages Sirovich ad Kirby method Eigefaces for geder recogitio Fisher liear discrimat aalysis Fisherimages ad varyig illumiatio Fisherfaces vs. eigefaces Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 1

2 Uitary trasforms! Sort pixels f [x,y] of a image ito colum vector f of legth N Calculate N trasform coefficiets c = Af The trasform A is uitary, iff where A is a matrix of size NxN A 1 = A *T A H Hermitia cojugate If A is real-valued, i.e., A=A*, trasform is orthoormal Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 2

3 Eergy coservatio with uitary trasforms c = A f For ay uitary trasform we obtai c 2 = c H c = f H A H A f = f 2 Iterpretatio: every uitary trasform is simply a rotatio of the coordiate system (ad, possibly, sig flips) Vector legth is coserved. Eergy (mea squared vector legth) is coserved. Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 3

4 Eergy distributio for uitary trasforms Eergy is coserved, but, i geeral, uevely distributed amog coefficiets. Autocorrelatio matrix R cc = E c c H = E A f f H A H = AR ff AH Diagoal of R cc comprises mea squared values ( eergies ) of the coefficiets c i 2 E c i = R cc i,i = AR ff A H i,i for ow: assume R ff is kow or ca be computed Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 4

5 Eigematrix of the autocorrelatio matrix Defiitio: eigematrix Φ of autocorrelatio matrix R ff l Φ is uitary l The colums of Φ form a set of eigevectors of R ff, i.e., R ff Φ = ΦΛ Λ is a diagoal matrix of eigevalues λ i $ & & Λ = & & % & λ 0 0 λ 1! 0 λ N 1 l uitary eigematrix for auto-correlatio matrix always exists ' ) ) ) ) ( ) l R ff is symmetric positive (semi-)defiite, hece λ i 0 for all i Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 5

6 Karhue-Loève trasform Uitary trasform with matrix A = Φ H Trasform coefficiets are pairwise ucorrelated R cc = AR ff A H = Φ H R ff Φ = Φ H ΦΛ = Λ Colums of Φ are ordered accordig to decreasig eigevalues. Eergy cocetratio property: l No other uitary trasform packs as much eergy ito the first J coefficiets. l Mea squared approximatio error by keepig oly first J coefficiets is miimized. l Holds for ay J. Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 6

7 Illustratio of eergy cocetratio Strogly correlated samples, equal eergies f 2 f 1 A = cosθ siθ siθ cosθ c 2 c 1 After KLT: ucorrelated samples, most of the eergy i first coefficiet Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 7

8 Basis images ad eigeimages For ay trasform, the iverse trasform! f = A 1! c ca be iterpreted i terms of the superpositio of colums of A -1 ( basis images ) For the KL trasform, the basis images are the eigevectors of the autocorrelatio matrix R ff ad are called eigeimages. If eergy cocetratio works well, oly a limited umber of eigeimages is eeded to approximate a set of images with small error. These eigeimages spa a optimal liear subspace of dimesioality J. Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 8

9 Eigeimages for recogitio To recogize complex patters (e.g., faces), large portios of a image have to be cosidered High dimesioality of image space meas high computatioal burde for may recogitio techiques Example: earest-eigbor search requires pairwise compariso with every image i a database Trasform ca reduce dimesioality from N to J by represetig the c = Wf image by J coefficiets Idea: tailor a KLT to a specific set of traiig images represetative of the recogitio task to preserve the saliet features Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 9

10 New Face Image f f Normalizatio Eigeimages for recogitio f c + - f Mea Face W Projectio Database of Eigeface Coefficiets p 1! p K Similarity measure!! (e.g., c T pk* ) Class of most similar p k Similarity Matchig Rejectio 1 k * θ Recogitio Result Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 10

11 Computig eigeimages from a traiig set How to obtai NxN covariace matrix?! l Use traiig set Γ 1, Γ! 2,, Γ! L+1 (each colum vector represets oe image) l Let be the mea image of all L+1 traiig images l µ Defie traiig set matrix S = ( Γ! 1 "! µ, Γ! 2 "! µ, Γ! 3 "! µ,, Γ! L "! µ ), ad calculate scatter matrix R =! Γ l "! µ Problem 1: Traiig set size should be L +1>> N If L < N, scatter matrix R is rak-deficiet Problem 2: Fidig eigevectors of a NxN matrix. L l=1! Γ l "! µ ( )( ) H = SS H Ca we fid a small set of the most importat eigeimages from a small traiig set L << N? Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 11

12 Sirovich ad Kirby algorithm Istead of eigevectors of SS H, cosider the eigevectors of S H S, i.e., Premultiply both sides by S S H S v i = λ i vi SS H S v! i = λ i S v! i By ispectio, we fid that S v i are eigevectors of SS H Sirovich ad Kirby Algorithm (for L << N ) l Compute the LxL matrix S H S v i l Compute L eigevectors of S H S l Compute eigeimages correspodig to the L 0 L largest eigevalues as a liear combiatio of traiig images S v i L. Sirovich ad M. Kirby, "Low-dimesioal procedure for the characterizatio of huma faces," Joural of the Optical Society of America A, 4(3), pp , Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 12

13 Example: eigefaces The first 8 eigefaces obtaied from a traiig set of 100 male ad 100 female traiig images Eigeface 1 Eigeface 2 Eigeface 3 Eigeface 4 Mea Face Eigeface 5 Eigeface 6 Eigeface 7 Eigeface 8 Ca be used to geerate faces by adjustig 8 coefficiets. Ca be used for face recogitio by earest-eighbor search i 8-d face space. Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 13

14 Geder recogitio usig eigefaces Nearest eighbor search face space Female face samples Male face samples Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 14

15 Fisher liear discrimiat aalysis Eigeimage method maximizes scatter withi the liear subspace over the etire image set regardless of classificatio task W opt = argmax W ( det( WRW H )) Fisher liear discrimiat aalysis (1936): maximize betwee-class scatter, while miimizig withi-class scatter W opt = argmax W ( ) ( ) det WR B W H det WR W W H R B = Samples i class i R W = c c i=1 i=1 Γ l N i!"!!" µ i µ!"! µ!" ( )( µ ) H i Class(i) Mea i class i µ i ( Γ ) l Γ l µ i ( ) H Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 15

16 Fisher liear discrimiat aalysis (cot.)!"! Solutio: Geeralized eigevectors w i correspodig to the J largest eigevalues { λ i i = 1,2,..., J}, i.e.!"! = λi R W w i!"!, i = 1,2,..., J R B w i à solve eige-problem o this: ( R 1 W R B )w i!"!!"! = λi w i, i = 1,2,..., J Problem: withi-class scatter matrix R w at most of rak L-1 (for L images total i all classes combied), hece usually sigular. Apply KLT first to reduce dimesioality of feature space to L-1 (or less), proceed with Fisher LDA i lower-dimesioal space Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 16

17 Eigeimages vs. Fisherimages 2-d example: f 2 Goal: project samples o a 1-d subspace, the perform classificatio. f 1 Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 17

18 Eigeimages vs. Fisherimages 2-d example: f2 KLT Goal: project samples o a 1-d subspace, the perform classificatio. The KLT preserves maximum eergy, but the 2 classes are o loger distiguishable. f 1 Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 18

19 2-d example: Eigeimages vs. Fisherimages f2 KLT Goal: project samples o a 1-d subspace, the perform classificatio. The KLT preserves maximum eergy, but the 2 classes are o loger distiguishable. Fisher LDA separates the classes by choosig a better 1-d subspace. f 1 Fisher LDA Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 19

20 Fisherimages ad varyig iillumiatio Differeces due to varyig illumiatio ca be much larger tha differeces amog faces! Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 20

21 Fisherimages ad varyig iillumiatio All images of same Lambertia surface with differet illumiatio (without shadows) lie i a 3d liear subspace Sigle poit source at ifiity surface ormal!! l light source Light source itesity ( ) L f ( x, y) = a( x, y) l!! T ( x, y ) directio Surface albedo Superpositio of arbitrary umber of poit sources at ifiity still i same 3d liear subspace, due to liear superpositio of each cotributio to image Fisherimages ca elimiate withi-class scatter Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 21

22 Side Note: Photometric Stereo! N L observed itesity! ormalized lightig directio! I = ρl N albedo! (costat)! ormalized surface ormal! diffuse (Lambertia) surfaces are viewpoit idepedet! [Woodham 1980]!

23 Side Note: Photometric Stereo! N L I = ρ L x L y L z N x N y N z diffuse (Lambertia) surfaces are viewpoit idepedet! [Woodham 1980]!

24 Side Note: Photometric Stereo! N L ( 2) ( ) ( ) I (1) L 1 L 3 I (2) I (3) = ρ L x (1) L x (2) L x (3) L y (1) L y (2) L y (3) L z (1) L z (2) L z (3) N x N y N z =! I =! L diffuse (Lambertia) surfaces are viewpoit idepedet! assume albedo is costat, ivert matrix! N = L 1 I [Woodham 1980]!

25 iput! Side Note: Photometric Stereo! output: recovered ormals! [Woodham 1980]!

26 Fisherface traied to recogize geder Female face samples Mea image Female mea Male mea! µ! µ! 1 µ 2 Male face samples Fisherface Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 26

27 Geder recogitio usig 1 st Fisherface Error rate = 6.5% Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 27

28 Geder recogitio usig 1 st eigeface Error rate = 19.0% Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 28

29 Perso idetificatio with Fisherfaces ad eigefaces ATT Database of Faces 40 classes 10 images per class Digital Image Processig: Berd Girod, 2013 Staford Uiversity -- Eigeimages 29