Virtual U: Defeating Face Liveness Detection by Building Virtual Models From Your Public Photos

Transcription

1 Virtual U: Defeating Face Liveness Detection by Building Virtual Models From Your Public Photos Yi Xu, True Price, Jan-Michael Frahm, and Fabian Monrose Department of Computer Science, University of North Carolina at Chapel Hill USENIX Security August 11, 2016

2 Face Authentication: Convenient Security image source

3 Evolution of Adversarial Models Attack: Still-image Spoofing

4 Evolution of Adversarial Models Attack: Still-image Spoofing Defense: Liveness Detection

5 Evolution of Adversarial Models Attack: Still-image Spoofing Defense: Liveness Detection Attack:Video Spoofing

6 Evolution of Adversarial Models Attack: Still-image Spoofing Defense: Liveness Detection Attack:Video Spoofing Defense: Motion Consistency

7 Evolution of Adversarial Models Attack: Still-image Spoofing Defense: Liveness Detection Attack:Video Spoofing Defense: Motion Consistency Attack: 3D-Printed Masks

8 Virtual U: A New Attack We introduce a new VR-based attack on face authentication systems solely using publicly available photos of the victim

9 Virtual U: A New Attack ❶ ❷ ❸ ❹ Input Web Photos Landmark Extraction 3D Model Reconstruction Image-based Texturing Gaze Correction ❻ ❺ Viewing with Virtual Reality System Expression Animation

10 Leveraging Social Media

11 Landmark Extraction

12 Identity Variation (e.g., thin-to-heavyset) 3D Face Model Expression Variation (e.g., frowning-to-smiling)

13 Identity Variation (e.g., thin-to-heavyset) 3D Face Model A exp S = S + A id α id + A exp α exp S Expression Variation (e.g., frowning-to-smiling) A id

14 3D Face Model S = S + A id α id + A exp α exp S Reprojection

15 3D Face Model S = S + A id α id + A exp α exp Pose α id α exp

16 3D Face Model S = S + A id α id + A exp α exp Pose α id α exp

17 3D Face Model S = S + A id α id + A exp α exp Pose α id α exp

18 3D Face Model

19 3D Face Model

20 3D Face Model

21 3D Face Model Pose α exp Pose α exp α id Pose α exp Pose α exp

22 Multi-Image Modeling Single image Multiple images

23 Texturing Direct Texturing 2D Poisson Editing

24 Texturing Direct Texturing 2D Poisson Editing 3D Poisson Editing

25 Gaze Correction R R B B G G

26 Gaze Correction

27 Virtual U: A New Attack ❶ ❷ ❸ ❹ Input Web Photos Landmark Extraction 3D Model Reconstruction Image-based Texturing Gaze Correction ❻ ❺ Viewing with Virtual Reality System Expression Animation

28 Expression Animation S = S + A id α id + A exp α exp Smiling Laughing Blinking Raising Eyebrows

29 VR Display Printed Marker VR System Authentication Device

30 VR Display

31 * * Experiments KeyLemon Mobius TrueKey Interaction-based liveness detection Motion-based liveness detection BioID 1U Texture-based liveness detection

32 Experiments 20 participants Aged 24 to males, 6 females Various ethnicities Two tests Indoor photo of the subject in the same environment as registration Publicly accessible photos Anywhere from 3 to 27 photos per person Low-, medium-, and high-quality Potentially strong changes in appearance over time

33 Experiments Indoor Image (Single frontal image) Online Avg. # Tries KeyLemon Mobius TrueKey BioID 1U 100% 100% 100% 100% 100% 85% % % % 1.7 0% --

34 Observations Medium- and high-resolution photos work best Photos from professional photographers (weddings, etc.) Group photos provide consistent frontal views Often lower resolution Only a small number of photos required One or two forward-facing photos One or two higher-resolution photos

35 Experiments How does resolution affect reconstruction quality?

36 Experiments How does rotation affect reconstruction quality?

37 Experiments Combining high-res rotation with low-res front-facing? +

38 Experiments Virtual U is successful against liveness detection

39 Experiments Virtual U is successful against liveness detection Also successful against motion consistency

40 Experiments Seeing Your Face is Not Enough: An Inertial Sensor-Based Liveness Detection for Face Authentication (Li et al., ACM CCS 15) Device motion measured by inertial sensor data Head pose estimated from input video Train a classifier to identify real data (correlated signals) versus spoofed video data

41 Experiments Training Data (Pos. Data vs. Neg. Data) Test Result (Accept Rate) Real Face Video Spoof VR Spoof Real vs. Video 98.0% 1.0% 99.5%

42 Experiments Training Data (Pos. Data vs. Neg. Data) Test Result (Accept Rate) Real Face Video Spoof VR Spoof Real vs. Video 98.0% 1.0% 99.5% Real vs. Video +VR 67.0% 0.0% 50.0%

43 Experiments Training Data (Pos. Data vs. Neg. Data) Test Result (Accept Rate) Real Face Video Spoof VR Spoof Real vs. Video 98.0% 1.0% 99.5% Real vs. Video +VR 67.0% 0.0% 50.0% Real vs. VR 67.0% %

44 Mitigations Alternative/additional hardware Infrared imaging (e.g. Windows Hello) Random structured light projection image source

45 Mitigations Alternative/additional hardware Infrared imaging (e.g. Windows Hello) Random structured light projection Improved defense against low-resolution synthetic textures Original Downsized to 50px

46 Conclusion We introduce a new VR-based attack on face authentication systems solely using publicly available photos of the victim This attack bypasses existing defenses of liveness detection and motion consistency At a minimum, face authentication software must improve against VRbased attacks with low-resolution textures The increasing ubiquity of VR will continue to challenge computervision-based authentication systems

47 Thank you! Questions?

48

49 Overview Face Authentication Virtual U: A VR-based attack Evaluation Mitigations Conclusion

50 Evolution of Adversarial Models Attack: Still-image Spoofing Defense: Liveness Detection Attack:Video Spoofing Defense: Motion Consistency Attack: 3D-Printed Masks Defense: Texture Detection

51 Identity Variation (e.g., thin-to-heavyset) 3D Face Model Expression Variation (e.g., frowning-to-smiling)

52 Identity Variation (e.g., thin-to-heavyset) 3D Face Model A exp S = S + A id α id + A exp α exp S Expression Variation (e.g., frowning-to-smiling) A id

53 3D Face Model S = S + A id α id + A exp α exp S Reprojection min P,α id,α exp i s i PS i 2 + β id α id 2 + β exp α exp 2 Pose Summed over all landmarks Normalization

54 3D Face Model

55 Multi-Image Modeling Single Image min P,α id,α exp i s i PS i 2 + β id α id 2 + β exp α exp 2 Multiple Images min P,α id,α exp m i s mi P m S mi 2 + β id α id 2 + β exp m α m exp 2 Sum over all images

56 Multi-Image Modeling Corners of the eyes and mouth are stable landmarks Contour points are variable landmarks

57 Multi-Image Modeling Multiple Images min P,α id,α exp m i s mi P m S mi 2 + norm. Multiple Images with Landmark Weighting min P,α id,α exp m i 1 σ i s 2 s mi P m S mi 2 + norm. Higher weighting for stable landmarks

58

59

60 Experiments 20 participants Aged 24 to males, 6 females Various ethnicities Two tests Indoor photo of the subject in the same environment as registration Publicly accessible photos Anywhere from 3 to 27 photos per person Low-, medium-, and high-quality Potentially strong changes in appearance over time

61 Experiments How does rotation affect reconstruction quality?

62 Experiments VR System Google Cardboard Authentication Device