Facial Landmark Tracking by Tree-based Deformable Part Model Based Detector

Transcription

1 Facial Landmark Tracking by Tree-based Deformable Part Model Based Detector Michal Uřičář, Vojtěch Franc, and Václav Hlaváč Department of Cybernetics, Faculty Electrical Engineering Czech Technical University in Prague Center for Machine Perception December 18, 215

2 Introduction & Motivation Outline 2/13 Facial Landmark Tracking by Deformable Part Model Based Detector Experiments & Results Demo Conclusions

3 Introduction & Motivation Face registration Essential part of face recognition systems (identity, gender, age,... ). Quality of face recognition and further processing depends on the quality of registration. Other applications Expression analysis. Head-pose estimation.... 3/13

4 Static Detection Model DPM landmark detector learned by SO-SVM Deformable Part based Model [Felzenszwalb et al., 21]. Extension of our previous work [Uřičář et al., 212]. Landmarks and their connections described by a tree graph G = (V, E). 4/

5 Static Detection Model (cont.) Landmarks and their connections described by graph G = (V, E). 5/13 Fixed size input image I IH W (normalized frame). Sought landmark configuration s, i.e. 2D coordinates (x, y). Maximization of the scoring function (max-sum problem) f(i, s; w) = i V s Arg max s S q i (s i, I; w q i ) + f(i, s; w), (i,j) E g ij (s i, s j ; w g ij ) Appearance model qi(si, I; wqi ) = wqi, Ψqi (I, si)... S-LBP features Measures the fitness of the i-th landmark on position s i Deformation cost gφij(si, sj; wgij) = wgij, Ψgij(si, sj)... displacement vector Measures the fitness of mutual connection of landmarks connected by edge (i, j) in G

6 Fixed size normalized frame. Making it Fast Distance transform in max-sum. 6/13 Fast computable features. Exploiting the sparsity of features. Precomputed feature parts in mipmap. Constrain search spaces Si of individual landmarks. Coarse-to-fine detection. Ψ q i (I, s) W H 2W H

7 How to Learn it? SO-SVM famework [Tsochantaridis et al., 25] translates the problem to a convex program w = arg min w R n [ λ 2 w m ] m r i (w) i=1 7/13 s.t. w i c, i W. (1) r i (w) loss incurred by the classifier on the i-th traning example r i (w) = max s S [ (s, s ) + w, Ψ(I i, s) Ψ(I i, s i ) ]. (2) (s, s ) = 1 κ(s) V Solved by BMRM algorithm. V j=1 s j s j, κ(s) is the IOD. Learned on 3-W dataset [Sagonas et al., 213],

8 Extending Static Detector to Tracker Kalman Filter with constant velocity model added. 8/13 Stabilization of imprecise face detector scale and position. Naturally deals with missing face detections in 3VW settings. prediction I face detector Kalman Filter (position, scale) stabilized coarse detector coarse landmarks refinement Kalman Filter (position, scale, rotation) stabilized fine detector landmarks prediction

9 Experiment 1 public 3VW set Experiments 9/13 5 sequences. Comparison to IntraFace [Xiong and la Torre, 213] (on 49 landmarks). Split of 3VW into 2 groups easy and hard (based on the achieved precision of static detector). Comparison to static detector. Experiment 2 non-public 3VW set Comparison to Chehra [Asthana et al., 214] (on 49 landmarks). Evaluation on 3 scenarios. Results provided by the competition authors.

10 3-VW public 49P IntraFace (a) All sequences 3-VW public 68P (d) All sequences coarse-to-fine coarse Results IntraFace (b) Easy sequences (e) Easy sequences coarse-to-fine coarse IntraFace (c) Hard sequences coarse-to-fine coarse (f) Hard sequences 1/13

11 3-VW non-public 49P chehra (g) Scenario 1 3-VW non-public 68P (j) Scenario 1 Results chehra (h) Scenario (k) Scenario chehra (i) Scenario (l) Scenario 3 11/13

12 Demo 12/13 John Oliver Last week tonight (Youtube, resolution px).

13 Conclusions Facial landmark tracker based on DPM. 13/13 Static detector extended to tracker by applying a Kalman filter. Parameters learned from examples by SO-SVM algorithm. Coarse-to-fine strategy for speed up and accuracy improvement. Can be easily modified for real-time performance. Open-source cross-platform implementation provided. Tested on Linux, Windows & Mac and some mobile devices. Interface to MATLAB & Python. Future work Tracker fails mainly on extreme head yaw poses. Mixture of view specific DPMs could help. Better specification of corrected.

14 References [Asthana et al., 214] Asthana, A., Zafeiriou, S., Cheng, S., and Pantic, M. (214). Incremental Face Alignment in the Wild. In The 27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 14, pages , Columbus, OH, USA. [Felzenszwalb et al., 21] Felzenszwalb, P. F., Girshick, R. B., McAllester, D. A., and Ramanan, D. (21). Object Detection with Discriminatively Trained Part-Based Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9): [Sagonas et al., 213] Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., and Pantic, M. (213). A Semi-automatic Methodology for Facial Landmark Annotation. In The 26th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR 13 Workshops, pages , Portland, OR, USA. [Tsochantaridis et al., 25] Tsochantaridis, I., Joachims, T., Hofmann, T., and Altun, Y. (25). Large Margin Methods for Structured and Interdependent Output Variables. Journal of Machine Learning Research, 6: [Uřičář et al., 212] Uřičář, M., Franc, V., and Hlaváč, V. (212). Detector of Facial Landmarks Learned by the Structured Output SVM. In Proceedings of the International Conference on Computer Vision Theory and Applications, VISAPP 12, volume 1, pages , Rome, Italy. [Xiong and la Torre, 213] Xiong, X. and la Torre, F. D. (213). Supervised Descent Method and Its Applications to Face Alignment. In The 26th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 13, pages , Portland, OR, USA.

15

16

17

18 prediction I face detector Kalman Filter (position, scale) stabilized coarse detector coarse landmarks refinement Kalman Filter (position, scale, rotation) stabilized fine detector landmarks prediction