Face authentication for low-power mobile devices

Transcription

1 Face authentication for low-power mobile devices Jean-Luc Nagel, CSEM SA, Microelectronics Division, Neuchâtel, Switzerland Patrick Stadelmann, University of Neuchâtel, IMT, Neuchâtel, Switzerland Conference on Biometrical Feature Identification and Analysis, Goettingen, Germany, 6-8 September 2007

2 Agenda Biometrics on mobile devices Motivation / selection of face authentication Challenges of mobile solutions Standard vs. custom hardware solutions for face authentication (camera + processing hardware) Proposed solutions towards mobile face authentication Elastic graph matching (EGM) algorithm A hardware platform for EGM Perspectives and conclusions Copyright 2007 CSEM Face authentication on mobile devices jln Page 1

3 Motivation (1) Beyond the Cell Phone: Mobile Communicators Convergence of PDAs and mobile phones (Smart Phones, Ultra-personal computers) Extended connectivity: IrDA, Bluetooth, GPRS, wireless LAN (802.11b), USB, Ethernet Protection of data stored on communicators Confidential business files (technical, managerial) Sensitive personal files (e.g. contacts + medical records) Banking transaction records Access to remote data / services Tele-maintenance, tele-surveillance Access to sensitive remove data Pay-per-use content M(obile)-business (e.g. weather forecasting, sports, timetables, etc.) Copyright 2007 CSEM Face authentication on mobile devices jln Page 2

4 Motivation (2) Security levels in 3G devices Communication security and data integrity: data encryption User account protection: code exchange Terminal access protection: PIN code Terminal access protection by PIN code is insufficient, because of: Limited security level Easiness to guess or steal PIN codes Limited user-friendliness Time needed to introduce PIN (keystrokes) Difficulty to remember numerous different codes users often do not even utilize PIN on mobile phones Security + user friendliness can be improved by complementing transferable knowledge (PIN codes) with info on user-being (physiological / behavioral traits) Copyright 2007 CSEM Face authentication on mobile devices jln Page 3

5 Motivation (3) Modalities most suitable for mobile terminals: Voice: Face: Fingerprint: (low-cost) microphone already exists on devices, but robustness to noisy environments is difficult Digital camera already exists on almost all mobile devices requires additional dedicated sensor. Face authentication seems a good choice for medium security applications (e.g. consumer devices) Improved robustness could be achieved using multi-modality (e.g. face + fingerprint) Copyright 2007 CSEM Face authentication on mobile devices jln Page 4

6 Motivation (4) Why entirely on the mobile device? (vs. local acquisition and verification on a remote server) Device turned on Authentication Access Advantages template store locally (no central big-brother database) accessible even if no network connection is available Drawbacks higher computational power is necessary attempts at hacking the biometric templates may be undetected Copyright 2007 CSEM Face authentication on mobile devices jln Page 5

7 Challenges Background variation : mostly a face detection problem Scaling view-angle, distance from camera (user holding the camera) Illumination variation (user authentifying himself in different lighting conditions): variations of the perceived intensity light position w.r.t. the geometry of the object; multiple light sources; non-linear effects (e.g. cast and attached shadows, highlights, saturation) variations of the illuminant spectrum Power consumption (energy consumption) Copyright 2007 CSEM Face authentication on mobile devices jln Page 6

8 Standard vs. custom hardware solution Custom Standard Image sensor Robustness to illumination variation; high dynamic range Examples: IR sensor, vision sensors Low-cost Possible reuse for other tasks Processing hardware Higher performance; lower-power consumption; low-cost (if volume is important) Examples: dedicated ASICs Customizable DSPs Examples: icyflex & macgic cores Better development tools Example: CMOS image sensor Examples: high-end DSPs Copyright 2007 CSEM Face authentication on mobile devices jln Page 7

9 Examples of non conventional image/vision sensors CSEM ViSe vision sensor auto-exposure principle : integration time measurement at the pixel level on-chip analogue computation of magnitude and direction of image contrasts. CSEM SwissRanger 3D image sensor time of flight measurement infra-red illumination Copyright 2007 CSEM Face authentication on mobile devices jln Page 8

10 Handling illumination variation on standard sensors Classical approaches: Predict the variability of the features build a statistical model of illumination variation (i.e. stats on the set of images of an object under all pose and illumination) reconstructionist (i.e. similar to image rendering in Computer Graphics) Derive new less sensitive features (i.e. near invariant) Performance of statistical methods is dependent on the quality of training set Reconstructionist approaches perform better than illumination insensitive features, but their computational complexity is not affordable on mobile devices (e.g. Blanz&Vetter several minutes on a 2 GHz PC) derivation of less sensitive features is sufficient for mobile low-power applications and is achievable on these devices. Copyright 2007 CSEM Face authentication on mobile devices jln Page 9

11 Illumination with standard image sensors Normalized mathematical morphology as a simple solution, which is not dependent on face detection results! SE (level 1, 2, 3,, 9) 9 eroded images 9 dilated images E 1 D 1 E 2 D 2 E 3 D 3 E 9 D 9 ( x, y) J ( x, y) J 9 J 8 J J 1 n 19 7 n J 10 J 11 J 12 J 19 J1( x, y) J19( x, y) J ' J ( x, y) Copyright 2007 CSEM Face authentication on mobile devices jln Page 10

12 Normalized mathematical morphology Advantages of normalized MM: Cast shadows produce local effects Attached shadows have a larger extension; feature distance is smaller in normalized morphology than in baseline morphology D 8 Standard mathematical morphology (SE of 17x 17 pixels) E 8 J 2 J 18 Normalized mathematical morphology Copyright 2007 CSEM Face authentication on mobile devices jln Page 11

13 Handling deformations Two approaches 1. Add a pre-processing stage to recover a shape free image (e.g. face warping) 2. Embed the shape matching in the algorithm Pre-processing stage depends on precise face localization (or even facial features localization) Precise detection is affected by deformations! Use the second solution Elastic graph matching is a good candidate Ok if features are near invariant to rotation and translation Copyright 2007 CSEM Face authentication on mobile devices jln Page 12

14 Elastic graph matching (EGM) Originally published by Lades et al. in 1993 In EGM, a face is holistically represented by labelled graphs local information: features associated to a node global information: topography of the graph; penalization of elastic deformations A correspondence is searched between features extracted from a reference graph and from a test graph (Euclidean feature distance minimization), which allows (but penalizes) elastic deformations M F M MxMxF? Copyright 2007 CSEM Face authentication on mobile devices jln Page 13

15 EGM improvements (1) New metrics (invariant to rotation and scaling) as compared to Lades et al. and Kotropoulos et al. Graph size variations a (a, a ) x y Copyright 2007 CSEM Face authentication on mobile devices jln Page 14

16 EGM improvements (2) Occlusion / cast shadows Out of 64 graph node distances, keep only the 32 smallest node feature distances and reject the others from the total graph distance Copyright 2007 CSEM Face authentication on mobile devices jln Page 15

17 EGM results : AVBPA 2003 Lausanne configuration II Automatic registration Copyright 2007 CSEM Face authentication on mobile devices jln Page 16

18 MEGM results : darkened XM2VTS ICBA 2006 Copyright 2007 CSEM Face authentication on mobile devices jln Page 17

19 MEGM results : Yale B database Classification Manual registration subset 1 subset 2 subset 3 subset 4 Method Id. error rate (%) vs. illuminant change Subset 2 Subset 3 Subset 4 Cones-attached (statistical) [Georghiades et al.] Cones-cast (statistical + reconstruc.) [Georghiades et al.] MEGM Copyright 2007 CSEM Face authentication on mobile devices jln Page 18

20 A hardware platform for EGM Two coprocessors (for mathematical morphology + graph matching) Programming tools (assembler, cycle-accurate simulator) Parallel processing improved power consumption compared to standard DSP Image sensor can be embedded in the same system-on-chip (SoC) when using CMOS technology Master processor can be used for other tasks Copyright 2007 CSEM Face authentication on mobile devices jln Page 19

21 FPGA demonstrator FPGA + SRAM VGACAM 146x146 pixels (out of 256x256) Ethernet Level shifter FPGA: Altera APEX-20K600 Logical elements (49 %) Ethernut Ethernet Positive slack (clock period : 100 ns) Face authentication 33.1 ns (ca 33%) 1.6 sec (~0.8 for computation) Copyright 2007 CSEM Face authentication on mobile devices jln Page 20

22 SoC power and area estimation results UMC 0.18 mm technology Power 10 MHz, 1.8 V. A power saving of ca. 3 times can be achieved at 1.0 V (same frequency) total power lower than 40 mw Other state-of-art mobile algorithms Pun et al.(spie 2005) PCA based. 5 sec for a complete authentication Running on an Intel XScale 400 MHz processor Power consumption: not given ( ~ 1 W) Power [mw] Power w/o. Power w. RAM RAM EGM copro 3 9 MM copro 1 3 Embedded SRAM (512 Ko) Micro-controller Image sensor IO pads 5 5 Total Area [mm2] w/o. RAM w. RAM EGM copro MM copro Copyright 2007 CSEM Face authentication on mobile devices jln Page 21

23 Conclusion Clear requirements for a low-cost easy-to-use solution for biometrics verification on mobile devices; Face verification can fit this requirement, but environment variations are challenging in mobile environments, especially lighting variation (indoor/outdoor use, multiple light sources, etc.); Custom cameras can improve the robustness of the solution but are too expensive; standard cameras are affordable in mobile solutions; Dedicated hardware coprocessor can fit the stringent power consumption issues; cost can be maintained low if embedded in a SoC with large volume productions (typical of mobile applications); A platform prototype based on elastic graph matching was demonstrated to correspond to power and speed requirements. Copyright 2007 CSEM Face authentication on mobile devices jln Page 22

24 Thank you for your attention.