The Problem. The Problem. What is Face Recognition? The Problem. Quiz. The Context of Face Recognition

Transcription

1 An Overvew of Face Recognton Usng Egenfaces Acknowledgements: Orgnal Sldes from Prof. atthew Turk -- also notes from the web -Egenvalues and Egenvectors -PCA -Egenfaces Outlne Why automated face recognton? Egenfaces and appearance-based approaches to recognton otvaton Revew Why Egenfaces? Why not Egenfaces? Where shall we go from here? Aprl 2004 Egenfaces Why Automated Face Recognton? It s a very vtal and compellng human ablty Faces are mportant to us Severe socal problem for people who lack ths ablty It s fun to work on Better than recognzng tanks and sprockets Good, paradgmatc vson problem It may actually be useful Bometrcs, HCI, survellance, They can do t n the moves! Commercal Interest Image and vdeo ndexng Bometrcs, e-commerce Vsoncs, Vsage, etrue, Survellance Casnos, Super Bowl, Tampa, FL Automated Face Recognton Typcal formulatons: Gven an mage of a face, who s t? (recognton) Is ths an mage of Joe Schmoe? (verfcaton) Why sn t ths easy? The Problem The human face s an extremely complex object, hghly deformable, wth both rgd and non-rgd components that vary over tme, sometmes qute rapdly and sometmes qute slowly The object s covered wth skn, a nonunformly textured materal that s dffcult to model ether geometrcally or photometrcally

2 The Problem Tme-varyng changes nclude: The growth and removal of facal har, wrnkles and saggng of the skn brought about by agng, skn blemshes, changes n skn color and texture caused by exposure to sun, etc. Plus many common artfact-related changes: Glasses, makeup, jewelry, percngs, cuts and scrapes, bandages, etc. Not to menton facal expressons, changes n harstyle, etc. The Problem In general, object recognton s dffcult because of the mmense varablty of object appearance Several factors are all confounded n the mage data Shape, reflectance, pose, occluson, llumnaton Human faces add more factors Expresson, facal har, jewelry, etc. So one may argue that face recognton s harder than most object recognton tasks The Problem Overcomng these dffcultes wll be a sgnfcant step forward for the computer vson communty So, face recognton has been consdered a challengng problem n computer vson for some tme now The amount of effort n the research communty devoted to ths topc has ncreased sgnfcantly over the years (e.g., ths workshop) Real-tme performance s key! What s Face Recognton? What can be observed va the face? Identty, emoton, race, age, sex, gender, attractveness, lp readng, character(?) Does face recognton nclude har? Ears? Are people really very good at face recognton? Drver s lcense photos odels n catalogs Colleagues at ICCV Perhaps we don t do t all that often Clothes, gat, voce, context Quz How many women are n the followng pcture? The Context of Face Recognton Face recognton (n humans and machnes) often coexsts wth other face processng tasks: Face (and head) detecton Face (and head) trackng Face pose estmaton Facal expresson analyss Facal feature detecton, recognton, and trackng It may be unnatural to separate face recognton from these other tasks But we wll anyway 2

3 Egenfaces: otvaton Frst generaton of FR systems Locate features, measure dstances and angles, create feature vector for classfcaton Bledsoe 966, Kelly 970, Kanade 973 d-to-late 980s Is there a dfferent approach to recognton, perhaps makng use of all the mage data (not just solated features)? Revsonary Hstory The Egenfaces approach, based on PCA, was never ntended to be the defntve soluton to face recognton. Rather, t was an attempt to rentroduce the use of nformaton between the features ; that s, t was an attempt to swng back the pendulum somewhat to counterbalance the focus on solated features. features??? otvaton: Bologcal Vson Example: Two face cells n monkey For decades, vson researchers have been nvestgatng mechansms of human face recognton Debate: Are faces specal? I.e., does face processng have a dfferent neural substrate from other vsual recognton? Debate: Confgural vs. holstc processng (featuredrven vs. whole stmulus ntegraton) Evdence from many sources: psychophyscs, snglecell recordng, neuromagng, neurophysologcal case studes Egenfaces: otvaton Levels of Recognton/atchng Bologcal face recognton Is face recognton confgural or holstc? Prevous approaches had all been confgural Abstracton odel Shape Example Shape So let s try an appearance-based approach to face recognton! Appearance models are complementary to shape models Not a replacement Features Pxels Features Pxels 3

4 Egenfaces orgns 980s Burt et al. pyramd-based FR work 987 Srovch and Krby paper PCA-based encodng of face mages Real-tme motvaton Is ths a sutable representaton for face recognton? Detecton? ultple scales? ultple vews? Is t computatonally feasble? So What s (are) Egenfaces? Uses Prncpal Component Analyss (PCA) to construct a Face Space from a tranng set of face mages Subspace of all possble mages Encodes only face mages Some choce n dmensonalty Test mage s projected nto the Face Space Projecton dstance determnes faceness Classfy accordng to projecton coeffcents Effcent mplementaton for face detecton and recognton Explctly handle scale and pose (smply) Implctly handle lghtng, expresson, etc. Intuton Image space s vastly large 8x8 bnary mage 2 64 mage ponts (dstnct mages) bllon mages per second 600 years Assumptons: Images of partcular objects (faces) may occupy a relatvely small but dstnct regon of the mage space Dfferent objects may occupy dfferent regons of mage space Whole classes of objects (all faces under varous transformatons) may occupy a stll relatvely small but dstnct regon of mage space orphng one face to another Questons for Appearance-Based FR What s the shape and dmensonalty of an ndvdual s face space, and how can t be succnctly modeled and used n recognton? What s the shape and dmensonalty of the complete face space, and how can t be succnctly modeled and used n recognton? Wthn the larger space, are the ndvdual spaces separated enough to allow for relable classfcaton among ndvduals? Is the complete face space dstnct enough to allow for relable face/non-face classfcaton? PCA PCA s a statstcal technque useful for dmensonalty reducton Can be used to construct a low-dmensonal lnear model from tranng data Optmal n a least-squares sense Assumes unform nose (an sotropc nose model) nmzes a least-squares energy functon Can be made robust (may be much slower) [Yulle, Black, others] Probablstc PCA [Tppng] Computng Egenfaces Set of face mages {x } of several people X = [ x x x L ] Sample covarance matrx C = = 2 T x x 3 x = XX Egenvectors of C form the prncple components (Egenfaces), ordered by the egenvalues Ce = e, E = T [ e e e L e ] 2 3 4

5 Projectng Into Face Space A potental face mage (y) s multpled by the Egenfaces = E T y ~ y = E Dstance from face space: y ~ y Classfcaton of Face Image Smplest verson: nearest class arg mn There are many better ways to do ths! E.g., dentty surfaces Experments wth Egenfaces Intal and subsequent emprcal results have shown promse Performs reasonably well wth small varatons n most parameters (scale, pose, lghtng, etc.) odular, multscale and multvew approaches Large database experments Early Arguments Aganst EFs EFs s a poor man s correlaton, slghtly more effcent but not as good Only partly true ajor benefts: generalzaton from learnng, ablty to detect new faces, ablty to handle very large databases Features are much better for recognton than are pxel values (lack of nvarants) Examples and counter-examples Some Issues Regardng EFs How to select k, the number of Egenfaces to keep How to effcently update the face space when new mages are added to the data set How to best represent classes and perform classfcaton wthn the face space How to separate ntraclass and nterclass varatons n the ntal calculaton of face space How to generalze from a lmted set of face mages and magng condtons 5

6 Problems wth Basc EFs There are clear shortcomngs wth the early Egenfaces mplementatons Sgnfcant varaton n scale, orentaton, translaton, or lghtng causes t to fal they re not really lnear! Intra- vs. nter-class mxture Not robust bad mages and bad pxels create havoc Senstve to precse algnment of the tranng set an encodng seems to be llumnaton Poor understandng of tranng dependences any, many more.. Bchsel So ths s Why Egenfaces Work To be more accurate Appearance-based technques work PCA, PCA+LDA, AA, ICA, etc. Relatvely smple to mplement and tran Works well n controlled envronments Can be made real-tme runtme s usually the faster part Straghtforward to comprehend and debug Especally when combned wth feature- or shape-based nformaton! E.g., shape- and pose-free technques Where to Go From Here? Better understandng of learnng Implcatons/requrements on sze and dversty of tranng set Convergence propertes Senstvty and robustness Better ntegraton wth feature- and structurebased technques Contnued work on ntra- vs. nter-class modelng Contnued work on llumnaton modelng (beyond lambertan) Where to Go From Here? (cont.) Deep understandng not just applcaton of latest hot PR technques Better understandng of the relatonshp among nner face, har, ears, contour, etc. Dynamc and statc approaches Integraton wth other sources of knowledge (voce, gat, clothes, envronment) Contnue to pursue non-lnear approaches Push toward real-tme, nteractve Where are We Now n FR? Stll a long way to go Stll many good, hard, nterestng problems to solve But we have come a long way n the past ten years 6