A Complementary Local Feature Descriptor for Face Identification

Transcription

1 WACV 2012 A Complementary Local Feature Descriptor for Face Identification Jonghyun Choi Dr. William R. Schwartz* Huimin Guo Prof. Larry S. Davis Institute for Advanced Computer Studies, University of Maryland, College Park *Institute of Computing, University of Campinas, Brazil

2 Agenda Background - Brief review of previous works - Mo6va6on Introduc6on of CCS- POP feature Discrimina6ve dimension weigh6ng by par6al least squares Experimental Results Conclusion

3 Existing Feature Descriptors for Face Identification Successful feature descriptors capture - Shape informa6on with different scales e.g. Gabor (D. Gabor, 1946) - Edge informa6on e.g. Histograms of oriented gradients (HOG) (Dalal and Triggs, 2005), SIFT (Lowe 1999) - Micro- edge informa6on e.g. Local binary paxerns (LBP) (Ahonen et al., 2006) Compact representa6on of features - Using histogram in a sub- window

4 Motivation Compact representa6on of features - Using histogram in a sub- window It has a trade- off - Pro Spa6al invariance within the sub- window - Con Losing loca6on specific informa6on [Example: HOG]

5 Motivation Compact representa6on of features is always preferred? - Using histogram in a sub- window It has a trade- off - Pro Spa6al invariance within the sub- window - Con Losing loca6on specific informa6on [Example: HOG]

6 Motivation Compact representa6on of features is always preferred? - Using histogram in a sub- window It has a trade- off - Pro Spa6al invariance within the sub- window - Con Losing loca6on specific informa6on [Example: HOG]

7 Motivation Compact representa6on of features is always preferred? - Using histogram in a sub- window It has a trade- off - Pro Spa6al invariance within the sub- window - Con Losing loca6on specific informa6on [Example: HOG]

8 Motivation Compact representa6on of features is always preferred? - Using histogram in a sub- window It has a trade- off - Pro Spa6al invariance within the sub- window - Con Losing loca6on specific informa6on [Example: HOG] Only Different here!

9 Location Specific Descriptors Gabor (D. Gabor, 1946) - Shape or big edge informa6on centering at a pixel Pairs of Pixels (POP) (Kembhavi et al., 2010) - Pixel- wise informa6on: from micro- edge to distant symmetric pairs - Can have discrimina6ve weights on each pixel pairs by par6al least squares - But only good at well aligned rectangular objects (usually man- made objects)

10 Introducing CCS-POP Circular Center Symmetric- Pairs of Pixels - Generalizing the POP for natural objects with mul6ple radius and various direc6ons - Similar informa6on to LBP and its variants but no histogramming and encoding with magnitude

11 Magnitude Information Due to extreme illumina6on varia6on, pixel difference informa6on (w/ magnitude) in micro scale (few pixels away) might be very noisy - Solu6on: trunca6on threshold (Tt) d (x,y) (r k,a i ) ( d (x,y) (r k,a i ), d (x,y) (r k,a i ) <T t, = sgn(d (x,y) (r k,a i )) T t, d (x,y) (r k,a i ) T t,

12 Curse of Dimensionality CCS- POP gives very high dimensional features - # of dim = # of radii X # of sampled points X # of pixels - It might have noisy informa6on - But less dimensionality than POP

13 Curse of Dimensionality CCS- POP gives very high dimensional features - # of dim = # of radii X # of sampled points X # of pixels - It might have noisy informa6on - But less dimensionality than POP Solution Discrimina6ve dimension reduc6on by Par6al Least Squares [1] [1] W.R.Schwartz et al., A Robust and Scalable Approach to Face Iden6fica6on, ECCV 2010

14 Partial Least Squares A supervised dimension reduc6on technique by maximizing covariance of weighted independent variable (X) and weighted dependent variable (Y) cov(t, u) 2 = max w =1 cov(xw,y )2 Using NIPALS algorithm [1] to obtain the regression solu6on from X to Y [1] H. Wold, Par6al Least Squares, 1985

15 Discriminative Weighting Using Partial Least Squares [1] Model Building (Training) D Gallery A C E F G B Z Build One- vs- All PLS regression models Model A PLSR Model Z PosiOve Samples NegaOve Samples PLSR PosiOve Samples NegaOve Samples TesOng Probe Regression Regression responses Model A Model B Model C Model D Model Z IdenOficaOon Result [1] Schwartz et al., A Robust and Scalable Approach to Face Iden6fica6on, ECCV 2010 C

16 Experimental Results

17 FRGC Ver. 1.0 Datasets Gallery Probe - Exp1: (G) 1 controlled image, (P) 1 controlled image (mostly illumina6on varia6ons) - Exp2: (G) 4 controlled images, (P) 1 controlled image - Exp4: (G) 1 controlled image, (P) mul6ple uncontrolled images FERET - fa: Gallery - m: expression varia6ons - fc: different ligh6ng varia6ons - dup1, dup2: 6me gap (expression and ligh6ng, uncontrolled)

18 Effect of Not Using Histogram Raw LBP is a feature of LBP informa6on (binary pixel difference with a single radius) without histogramming Weighted by PLS regression FRGC FERET Descriptor Dim. Exp.1 Exp.2 Exp.4 LBP 16, Raw LBP 176, Descriptor Dim. fb fc dup1 dup2 LBP 6, Raw LBP 96, Given a discrimina6ve weigh6ng scheme, histogramming prevents bexer performance But the feature dimension of Raw LBP is prohibi6ve

19 CCS-POP Far less number of dimension than Raw LBP - 176,640 44,160 (75% less) - Comparable performance Using color informa6on (only in FRGC dataset) - BeXer performance (Especially in Exp. 4) - Triple the feature dimensions (44, ,480) Less number of dimension by simple color informa6on (CI) - 132,480 60,042 (55% less) - Maintaining the performance FRGC Color Dim. Exp.1 Exp.2 Exp.4 Raw LBP 176, Gray 44, R,G,B 132, Gray+CI 60,

20 Effect of Truncation Threshold Trunca6on threshold improves performance on the experiments with FRGC dataset FRGC T t Exp.1 Exp.2 Exp.4 No threshold

21 Comparison to Other Descriptors FRGC FERET Descriptor Dim. Exp.1 Exp.2 Exp.4 Intensity 22, POP 607, LBP 16, MSLBP 32, HOG 49, Gabor 54, CCS-POP 60, Descriptor Dim. fb fc dup1 dup2 Intensity 12, POP 242, LBP 6, MSLBP 12, HOG 24, Gabor 29, CCS-POP 24, Not Good

22 Complementariness of CCS-POP For the challenging experiments in FRGC dataset - FRGC Exp % 69.2% Improvement by CCS-POP Original Performance 79.3% 82.4% Improvement by CCS-POP Improvement by Gabor HOG 89.0%

23 Complementariness of CCS-POP For the challenging experiments in FRGC dataset - FRGC Exp % 69.2% Improvement by CCS-POP Original Performance 90% 79.3% 82.4% 14.8% 15.8% 67.5% Improvement by CCS-POP Improvement by Gabor HOG 89.0% 6.1% 18.4% 90% 67.5% 55.4% 38.3% 45% 45% 13% LBP 30.9% MSLBP 64.5% 66.6% HOG Gabor 22.5% 0% 64.5% CHG 22.5% 0%

24 Complementariness of CCS-POP For the challenging experiment in FERET dataset - FERET dup2 Improvement by CCS-POP Original Performance Improvement by CCS-POP Improvement by Gabor HOG 84.6% 79.5% 77.4% 58.5%

25 Complementariness of CCS-POP For the challenging experiment in FERET dataset - FERET dup2 58.5% 2.6% Improvement by CCS-POP Original Performance 79.5% 77.4% 1.3% 90% 60% Improvement by CCS-POP Improvement by Gabor HOG 84.6% 1.3% 6.4% 90% 60% 48.2% 76.9% 76.1% 30% 76.9% 30% 10.3% LBP HOG Gabor 0% CHG 0%

26 Complementariness by Feature Weighting An example of heat maps of PLS regression for different features. CCS-POP HOG LBP Gabor CCS- POP captures pixel- wise micro informa6on

27 Comparison to the Previous Works FRGC FERET Method Exp.1 Exp.2 Exp.4 PCA [17] UMD [1] BEE (from [27]) LC 1 C 2 [27] ROCA [10] Liu [12] Tan (from [8]) Holappa [8] LPQ [20] PLS [24] CHG Method fb fc dup1 dup2 LGBPHS [36] HGPP [35] SIS [13] POEM [30] PLS [24] CHG

28 Conclusion New feature descriptor (CCS- POP) generalizing POP for natural object (e.g. face) is proposed Empirically show the importance of not using histogram with a discrimina6ve weigh6ng scheme Complementary to exis6ng descriptors Achieve state- of- the- art performance on both FRGC and FERET datasets

29 code is available in