WELCOME TO THE NAE US FRONTIERS OF ENGINEERING SYMPOSIUM 2005

Transcription

1 WELCOME TO THE NAE US FRONTIERS OF ENGINEERING SYMPOSIUM 2005

2 Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City

3 How are people identified? People are identified by three basic means: Something they have (identity document or token) Something they know (password, PIN) Something they are (human body)

4 Iris

5 Retina Every eye has its own totally unique pattern of blood vessels.

6 Hand

7 Fingerprint

8 Ear

9 Face

10 Who are these people? [Sinha and Poggio 1996]

11 Who are these people? [Sinha and Poggio 2002]

12 Images as Points in Euclidean Space x 2 x 1 x n Let an n-pixel image to be a point in an n-d space, x R n. Each pixel value is a coordinate of x.

13 Face Recognition: Euclidean Distances D 1 > 0 D 2 > 0 ~ D 3 = 0

14 Face Recognition: Euclidean Distances D 1 > 0 D 2 > 0 D 3 > D 1 or 2 [Hallinan 1994] [Adini, Moses, and Ullman 1994]

15 Same Person or Different People

16

17 Same Person or Different People

18

19 Why is Face Recognition Hard?

20 Challenges: Image Variability Expression Short Term Pose Illumination Facial Hair Makeup Eyewear Long Term Hairstyle Piercings Aging

21 Illumination Invariants? Does there exist a function f s.t. f ( ) = f ( ) = f ( ) = a and f ( ) = f ( ) = f ( ) = b?

22 Can Any Two Images Arise from a Single Surface? s n I(x,y) a I(x,y) = a(x,y) n(x,y) s n l Same Albedo Different and Lighting Surface J(x,y) a J(x,y) = a(x,y) n(x,y) l

23 The Surface PDE I(x,y) = a(x,y) n(x,y) s J(x,y) = a(x,y) n(x,y) l ( I l J s ) n = 0 Nonlinear PDE Linear PDE

24 Non-Existence Theorem for Illumination Invariants Illumination invariants for 3-D objects do not exist. This result does not ignore attached and cast shadows, as well as surface interreflection. [Chen, Belhumeur, and Jacobs 2000]

25 Geometric Invariants? Does there exist a function f s.t. f ( ) = f ( ) = f ( ) = a and f ( ) = f ( ) = f ( ) = b?

26 Non-Existence Theorem for Geometric Invariants Geometric invariants for rigid transformations of 3-D objects viewed under perspective projective projection do not exist. [Burns, Weiss, and Riseman 1992]

27 Image Variability: Appearance Manifolds x 2 x n x 1 Lighting x Pose [Murase and Nayar 1993]

28 Modeling Image Variability Can we model illumination and pose variability in images of a face? Yes, if we can determine the shape and texture of the face. But how?

29 Modeling Image Variability: 3-D Faces Laser Range Scanners Stereo Cameras Structured Light Photometric Stereo [Atick, Griffin, Redlich 1996] [Georghiades, Belhumeur, Kriegman 1996] [Blanz and Vetter 1999] [Zhao and Chellepa 1999] [Kimmel and Sapiro 2003] [Geometrix 2001] [MERL 2005]

30 Illumination Variation Reveals Object Shape s 1 n a s 2 s 3 I 1 I 2 I 3 [Woodham1984]

31 Illumination Movie Illumination Movie

32 Shape Movie Shape Movie

33 Image Variability: From Few to Many Lighting x Pose x 2 x n Real Synthetic x 1 [Georghiades, Belhumeur, and Kriegman 1999]

34 Illumination Dome

35 Real vs. Synthetic Real Synthetic

36 Real vs. Synthetic Real Synthetic

37 A Step Back in Time

38 Albrecht Dürer, Four Books on Human Proportion (1528)

39 D arcy Thompson, On Growth and Form (1917)

40 D arcy Thompson, On Growth and Form (1917)

41 D arcy Thompson, On Growth and Form (1917)

42 But what if we could.? [Blanz and Vetter 1999, 2003]

43 Building a Morphable Face Model [Blanz and Vetter 1999, 2003]

44 3-D Morphaple Models: Semi-Automatic [Blanz and Vetter 1999, 2003]

45 Building Morphable Face Models [Blanz and Vetter 1999, 2003]

46 Fitting Morphable Face Models [Blanz and Vetter 1999, 2003]

47 National Geographic 1984 and

48 Identity Confirmed by IRIS = [Daugman 2002]