MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
|
|
- Jonathan Edwards
- 5 years ago
- Views:
Transcription
1 MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction Ayush Tewari Michael Zollhofer Hyeongwoo Kim Pablo Garrido Florian Bernard Patrick Perez Christian Theobalt Presented by Suleyman Aslan 1/32
2 Outline Introduction Motivation and Related Work Overview Semantic Code Vector Parametric Model-based Decoder Loss Layer Experiment Results and Comparisons Limitations Conclusion 2/32
3 Introduction Challenging problem of reconstructing 3D human face encode face pose, shape, expression, reflectance and illumination Combining convolutional autoencoder with generative model a novel differentiable parametric decoder Encoder learns to extract semantically meaningful parameters code vector Unsupervised learning 3/32 [1]
4 Motivation A challenging problem in computer vision and computer graphics Previous approaches use mostly calibrated, multi-view data High variability in pose, facial expression, and lighting Face reconstruction from a single uncalibrated image is an open research problem 4/32
5 Related Work Generative methods and regression-based methods Generative approaches, fit a parametric model, optimize alignment between projected model and image require level of control during image capture or additional input data e.g., detected landmarks Regression-based approaches, can estimate pose, shape, expression can only be trained in a supervised fashion, a major obstacle illumination and reflectance do not match best generative methods 5/32
6 Related Work Cootes et al. [2] use Active Appearance Models to match a statistical model of object shape and appearance to a new image, it can be used for matching and tracking faces Roth et al. [3] achieve reconstructing a 3D face model by fitting a 3D Morphable Model (3DMM) Zhou et al. [4] use CNN cascades for the detection of facial landmarks, in a supervised manner and predicts only sparse information Tran et al. [5] obtain robust and discriminative 3D Morphable Models with annotated training data Richardson et al. [6] achieve 3D face reconstruction by learning from synthetic data, lacks realistic features 6/32
7 Related Work Zhmoginov et al [7] generate images from code vectors by using autoencoders Kulkarni et al. [8] learn graphics codes for the reproduction of images under different conditions Yan et al. [9] achieves unsupervised volumetric 3D object reconstruction from a single-view Higher level computer vision task, reconstruction of a full set of meaningful parameters is not considered 7/32
8 Overview New type of model-based deep convolutional autoencoder Makes use of state-of-the-art generative and regression approaches Inspired by deep convolutional autoencoders, features a CNN encoder Unlike previously used CNN decoders, it features newly designed decoder, a generative image formation model on the basis of a parametric 3D face model Input to the decoder, i.e. the semantic meaning of the code vector is ensured by design 8/32
9 Overview Combined end-to-end training of model-based decoder and a CNN encoder Unsupervised training and semantically meaningful face reconstruction Generalizes better to real world data 9/32
10 Overview Decoder generates a realistic synthetic image of a face and enforces semantic meaning Pose, shape, expression, reflectance and illumination are parameterized independently Synthesized image is compared to the input using a photometric loss [1] 10/32
11 Semantic Code Vector Semantic code vector x R 257 facial expression δ R 64 shape α R 80 skin reflectance β R 80 camera rotation T SO(3) (3D rotation group) translation t R 3 scene illumination γ R 27 11/32
12 Semantic Code Vector Face is represented by 24k vertices Normals are computed using one-ring neighborhood A S, average face shape E S and Ee, PCA bases 12/32
13 Semantic Code Vector Per vertex reflectance parameterized based on affine parametric model Ar, average skin reflectance Er, PCA basis 13/32
14 Parametric Model Based Decoder Model generates the synthetic image that corresponds to the face Rendered using a pinhole camera model under full perspective projection mapping from camera space to screen space (R 3 R 2 ) Position and orientation of the camera is given by a rigid transformation an arbitrary point is mapped to camera and screen space Scene illumination is represented using Spherical Harmonics [10] 14/32
15 Parametric Model Based Decoder For the image, screen space and associated pixel color for each vertex is then computed Backward pass that inverts image formation is implemented for backpropagation 15/32
16 Loss Layer Photometric loss function that enables end-to-end training E land, sparse landmark alignment E photo, dense photometric alignment Ereg, statistical plausibility of the modeled faces 16/32
17 Loss Layer Dense photometric alignment (E photo ) Predict parameters that lead to a synthetic face image that matches the input image Photometric alignment 17/32
18 Loss Layer Sparse landmark alignment (E land ) Enforce projected 3D vertices to be close to the 2D detections based on detected facial landmarks [20] 46 landmarks Optional loss 18/32
19 Loss Layer Statistical regularization (Ereg) Enforce plausible facial shape, expression, and skin reflectance prefer values close to the average Pose and illumination is not regularized 19/32
20 Results Encoders based on AlexNet [18] and VGG-Face [19] are tested Last fully connected layer is modified to output 257 model parameters Encoder regressed pose, shape, expression, reflectance and illumination from a single image [1], images from CelebA [11] 20/32
21 Results Combination of four datasets is used for training: CelebA [11], LFW [12], Facewarehouse [13], and 300-VW [14, 15, 16] Facial landmark detection [20] is used and images are cropped to a bounding box using Haar Cascade Face Detection [17] Bad detections are dropped 147k images in total, 142k for training, 5k for evaluation Network trained using, AdaDelta and 200k batch iterations with batch size of 5 Learning rate of 0.1 for all parameters except Z-translation Encoder is initialized with pre-trained weights Last fully connected layer has weights of 0 21/32
22 Comparisons To Richardson et al. [6] Richardson et al. use synthetic images and lacks several dimensions, e.g., facial hair [1] 22/32
23 Comparisons To Tran et al. [5] Tran et al. do not estimate facial expression and illumination, trained in supervised manner [1] 23/32
24 Comparisons To Thies et al. [21] Thies et al. require detected landmarks, is slower, and similar or lower quality [1] 24/32
25 Comparisons To Garrido et al. [22] Garrido et al. require landmark detection, comparable quality [1] 25/32
26 Evaluation Impact of different encoders is evaluated VGG-Face [19] is slightly better than AlexNet [18], lower landmark and photometric error [1] 26/32
27 Evaluation Evaluation of (fully) unsupervised training Landmark error can be reduced even when landmark alignment is not part of the loss function Training with surrogate loss (landmarks) improves landmark alignment and leads to similar photometric error, also improves robustness to occlusions and expressions [1] 27/32
28 Evaluation Evaluation on synthetic data Trained on 100k synthetic images with background augmentation, 5k images for evaluation with known parameters [1] 28/32
29 Evaluation Comparison to convolutional autoencoder Model-based approach obtains sharper reconstruction and better semantic parameters Decoder is trained on synthetic imagery generated by the model to learn parameter-to-image mapping [1] 29/32
30 Limitations Implausible reconstructions are possible outside of the span of training data Can not regress facial hair, eye gaze, accessories Strong occlusions cause approach to fail [1] 30/32
31 Conclusion First deep convolutional model based face autoencoder Trained in an unsupervised manner Learns meaningful set of semantic parameters Semantic meaning in the code vector is enforced by design Pose, shape, expression, skin reflectance, and illumination can be accurately regressed 31/32
32 References 1. A. Tewari M. Zollhofer H. Kim P. Garrido F. Bernard P. Perez C. Theobalt. MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction. arxiv.org: v1, T. F. Cootes, G. J. Edwards, and C. J. Taylor. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6): , June J. Roth, Y. Tong, and X. Liu. Adaptive 3d face reconstruction from unconstrained photo collections. December E. Zhou, H. Fan, Z. Cao, Y. Jiang, and Q. Yin. Extensive Facial Landmark Localization with Coarse-to-Fine Convolutional Network Cascade. In CVPRW, A. T. Tran, T. Hassner, I. Masi, and G. Medioni. Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network. arxiv.org: v1, E. Richardson, M. Sela, and R. Kimmel. 3D face reconstruction by learning from synthetic data. In 3DV, A. Zhmoginov and M. Sandler. Inverting face embeddings with convolutional neural networks. arxiv: , June T. D. Kulkarni, W. Whitney, P. Kohli, and J. B. Tenenbaum. Deep convolutional inverse graphics network. In NIPS, X. Yan, J. Yang, E. Yumer, Y. Guo, and H. Lee. Perspective transformer nets: Learning single-view 3d object reconstruction without 3d supervision. arxiv: , Dec C. Muller. Spherical harmonics. Springer, Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, October C. Cao, Y. Weng, S. Zhou, Y. Tong, and K. Zhou. Facewarehouse: A 3d facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics, 20(3): , Mar G. G. Chrysos, E. Antonakos, S. Zafeiriou, and P. Snape. Offline deformable face tracking in arbitrary videos. In The IEEE International Conference on Computer Vision (ICCV) Workshops, December J. Shen, S. Zafeiriou, G. G. Chrysos, J. Kossaifi, G. Tzimiropoulos, and M. Pantic. The first facial landmark tracking in-the-wild challenge: Benchmark and results. In ICCVW, December G. Tzimiropoulos. Project-out cascaded regression with an application to face alignment. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June G. Bradski. The OpenCV Library. Dr. Dobb s Journal of Software Tools, A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, O. M. Parkhi, A. Vedaldi, and A. Zisserman. Deep face recognition. In British Machine Vision Conference, J. M. Saragih, S. Lucey, and J. F. Cohn. Deformable model fitting by regularized landmark mean-shift. 91(2): , J. Thies, M. Zollhofer, M. Stamminger, C. Theobalt, and M. Niener. Face2Face: Real-time face capture and reenactment of RGB videos. In CVPR, P. Garrido, M. Zollhofer, D. Casas, L. Valgaerts, K. Varanasi, P. Perez, and C. Theobalt. Reconstruction of personalized 3D face rigs from monocular video. ACM Transactions on Graphics, 35(3):28:1 15, June /32
arxiv: v2 [cs.cv] 7 Dec 2017
MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction arxiv:1703.10580v2 [cs.cv] 7 Dec 2017 Ayush Tewari 1 Michael Zollhöfer 1 Hyeongwoo Kim 1 Pablo Garrido 1
More informationSelf-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz Supplemental Material
Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz Supplemental Material Ayush Tewari 1,2 Michael Zollhöfer 1,2,3 Pablo Garrido 1,2 Florian Bernard 1,2 Hyeongwoo
More informationMoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction Ayush Tewari 1 Michael Zollhöfer 1 Hyeongwoo Kim 1 Pablo Garrido 1 Florian Bernard 1,2 Patrick Pérez 3 Christian
More informationarxiv: v1 [cs.cv] 16 Nov 2015
Coarse-to-fine Face Alignment with Multi-Scale Local Patch Regression Zhiao Huang hza@megvii.com Erjin Zhou zej@megvii.com Zhimin Cao czm@megvii.com arxiv:1511.04901v1 [cs.cv] 16 Nov 2015 Abstract Facial
More informationFACIAL POINT DETECTION USING CONVOLUTIONAL NEURAL NETWORK TRANSFERRED FROM A HETEROGENEOUS TASK
FACIAL POINT DETECTION USING CONVOLUTIONAL NEURAL NETWORK TRANSFERRED FROM A HETEROGENEOUS TASK Takayoshi Yamashita* Taro Watasue** Yuji Yamauchi* Hironobu Fujiyoshi* *Chubu University, **Tome R&D 1200,
More informationSelf-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz
Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 50 Hz Ayush Tewari 1, Michael Zollhöfer 1,,3 Pablo Garrido 1, Florian Bernard 1, Hyeongwoo Kim 1, Patrick Pérez 4 Christian
More informationInverseFaceNet: Deep Monocular Inverse Face Rendering
InverseFaceNet: Deep Monocular Inverse Face Rendering Hyeongwoo Kim 1, 2 Michael Zollhöfer 1, 2, 3 Ayush Tewari 1, 2 Justus Thies 4 Christian Richardt 5 Christian Theobalt 1, 2 1 Max-Planck-Institute for
More informationA Hybrid Model for Identity Obfuscation by Face Replacement
A Hybrid Model for Identity Obfuscation by Face Replacement Qianru Sun Ayush Tewari Weipeng Xu Mario Fritz Christian Theobalt Bernt Schiele Max Planck Institute for Informatics, Saarland Informatics Campus
More informationarxiv: v1 [cs.cv] 24 Sep 2018
MobileFace: 3D Face Reconstruction with Efficient CNN Regression Nikolai Chinaev 1, Alexander Chigorin 1, and Ivan Laptev 1,2 arxiv:1809.08809v1 [cs.cv] 24 Sep 2018 1 VisionLabs, Amsterdam, The Netherlands
More informationarxiv: v1 [cs.cv] 29 Sep 2016
arxiv:1609.09545v1 [cs.cv] 29 Sep 2016 Two-stage Convolutional Part Heatmap Regression for the 1st 3D Face Alignment in the Wild (3DFAW) Challenge Adrian Bulat and Georgios Tzimiropoulos Computer Vision
More informationImproved Face Detection and Alignment using Cascade Deep Convolutional Network
Improved Face Detection and Alignment using Cascade Deep Convolutional Network Weilin Cong, Sanyuan Zhao, Hui Tian, and Jianbing Shen Beijing Key Laboratory of Intelligent Information Technology, School
More informationSupplemental Material for Face Alignment Across Large Poses: A 3D Solution
Supplemental Material for Face Alignment Across Large Poses: A 3D Solution Xiangyu Zhu 1 Zhen Lei 1 Xiaoming Liu 2 Hailin Shi 1 Stan Z. Li 1 1 Institute of Automation, Chinese Academy of Sciences 2 Department
More informationDense 3D Modelling and Monocular Reconstruction of Deformable Objects
Dense 3D Modelling and Monocular Reconstruction of Deformable Objects Anastasios (Tassos) Roussos Lecturer in Computer Science, University of Exeter Research Associate, Imperial College London Overview
More informationarxiv: v2 [cs.cv] 24 Jul 2018
A Hybrid Model for Identity Obfuscation by Face Replacement Qianru Sun Ayush Tewari Weipeng Xu Mario Fritz Christian Theobalt Bernt Schiele arxiv:1804.04779v2 [cs.cv] 24 Jul 2018 Max Planck Institute for
More informationFACIAL POINT DETECTION BASED ON A CONVOLUTIONAL NEURAL NETWORK WITH OPTIMAL MINI-BATCH PROCEDURE. Chubu University 1200, Matsumoto-cho, Kasugai, AICHI
FACIAL POINT DETECTION BASED ON A CONVOLUTIONAL NEURAL NETWORK WITH OPTIMAL MINI-BATCH PROCEDURE Masatoshi Kimura Takayoshi Yamashita Yu Yamauchi Hironobu Fuyoshi* Chubu University 1200, Matsumoto-cho,
More informationFROM VIDEO STREAMS IN THE WILD
SEMANTIC FACE SEGMENTATION FROM VIDEO STREAMS IN THE WILD Student: Deividas Skiparis Supervisors: Pascal Landry (Imersivo) and Sergio Escalera (UB) In Collaboration With Imersivo SL 7/5/2017 2 Problem
More informationarxiv: v1 [cs.cv] 15 Jun 2018
Unsupervised Training for 3D Morphable Model Regression Kyle Genova 1,2 Forrester Cole 2 Aaron Maschinot 2 Aaron Sarna 2 Daniel Vlasic 2 William T. Freeman 2,3 1 Princeton University 2 Google Research
More informationDeep Learning for Virtual Shopping. Dr. Jürgen Sturm Group Leader RGB-D
Deep Learning for Virtual Shopping Dr. Jürgen Sturm Group Leader RGB-D metaio GmbH Augmented Reality with the Metaio SDK: IKEA Catalogue App Metaio: Augmented Reality Metaio SDK for ios, Android and Windows
More informationCost-alleviative Learning for Deep Convolutional Neural Network-based Facial Part Labeling
[DOI: 10.2197/ipsjtcva.7.99] Express Paper Cost-alleviative Learning for Deep Convolutional Neural Network-based Facial Part Labeling Takayoshi Yamashita 1,a) Takaya Nakamura 1 Hiroshi Fukui 1,b) Yuji
More informationIntensity-Depth Face Alignment Using Cascade Shape Regression
Intensity-Depth Face Alignment Using Cascade Shape Regression Yang Cao 1 and Bao-Liang Lu 1,2 1 Center for Brain-like Computing and Machine Intelligence Department of Computer Science and Engineering Shanghai
More informationFrom processing to learning on graphs
From processing to learning on graphs Patrick Pérez Maths and Images in Paris IHP, 2 March 2017 Signals on graphs Natural graph: mesh, network, etc., related to a real structure, various signals can live
More informationDetecting facial landmarks in the video based on a hybrid framework
Detecting facial landmarks in the video based on a hybrid framework Nian Cai 1, *, Zhineng Lin 1, Fu Zhang 1, Guandong Cen 1, Han Wang 2 1 School of Information Engineering, Guangdong University of Technology,
More informationDeep learning for dense per-pixel prediction. Chunhua Shen The University of Adelaide, Australia
Deep learning for dense per-pixel prediction Chunhua Shen The University of Adelaide, Australia Image understanding Classification error Convolution Neural Networks 0.3 0.2 0.1 Image Classification [Krizhevsky
More informationDeep Tracking: Biologically Inspired Tracking with Deep Convolutional Networks
Deep Tracking: Biologically Inspired Tracking with Deep Convolutional Networks Si Chen The George Washington University sichen@gwmail.gwu.edu Meera Hahn Emory University mhahn7@emory.edu Mentor: Afshin
More informationAn Exploration of Computer Vision Techniques for Bird Species Classification
An Exploration of Computer Vision Techniques for Bird Species Classification Anne L. Alter, Karen M. Wang December 15, 2017 Abstract Bird classification, a fine-grained categorization task, is a complex
More informationSynscapes A photorealistic syntehtic dataset for street scene parsing Jonas Unger Department of Science and Technology Linköpings Universitet.
Synscapes A photorealistic syntehtic dataset for street scene parsing Jonas Unger Department of Science and Technology Linköpings Universitet 7D Labs VINNOVA https://7dlabs.com Photo-realistic image synthesis
More informationPix2Face: Direct 3D Face Model Estimation
Pix2Face: Direct 3D Face Model Estimation Daniel Crispell Maxim Bazik Vision Systems, Inc Providence, RI USA danielcrispell, maximbazik@visionsystemsinccom Abstract An efficient, fully automatic method
More informationFML: Face Model Learning from Videos
FML: Face Model Learning from Videos Ayush Tewari 1 Florian Bernard 1 Pablo Garrido 2 Gaurav Bharaj 2 Mohamed Elgharib 1 Hans-Peter Seidel 1 Patrick Pérez 3 Michael Zollhöfer 4 Christian Theobalt 1 1 MPI
More informationFacial Landmark Detection via Progressive Initialization
Facial Landmark Detection via Progressive Initialization Shengtao Xiao Shuicheng Yan Ashraf A. Kassim Department of Electrical and Computer Engineering, National University of Singapore Singapore 117576
More informationSupplementary Material for Synthesizing Normalized Faces from Facial Identity Features
Supplementary Material for Synthesizing Normalized Faces from Facial Identity Features Forrester Cole 1 David Belanger 1,2 Dilip Krishnan 1 Aaron Sarna 1 Inbar Mosseri 1 William T. Freeman 1,3 1 Google,
More informationarxiv: v1 [cs.cv] 21 Mar 2018
arxiv:1803.07835v1 [cs.cv] 21 Mar 2018 Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, Xi Zhou Shanghai Jiao Tong University,
More informationDeep Face Feature for Face Alignment and Reconstruction
1 Deep Face Feature for Face Alignment and Reconstruction Boyi Jiang, Juyong Zhang, Bailin Deng, Yudong Guo and Ligang Liu arxiv:1708.02721v1 [cs.cv] 9 Aug 2017 Abstract In this paper, we propose a novel
More informationSelf-supervised CNN for Unconstrained 3D Facial Performance Capture from an RGB-D Camera
1 Self-supervised CNN for Unconstrained 3D Facial Performance Capture from an RGB-D Camera Yudong Guo, Juyong Zhang, Lin Cai, Jianfei Cai and Jianmin Zheng arxiv:1808.05323v2 [cs.cv] 17 Aug 2018 Abstract
More informationVolumetric and Multi-View CNNs for Object Classification on 3D Data Supplementary Material
Volumetric and Multi-View CNNs for Object Classification on 3D Data Supplementary Material Charles R. Qi Hao Su Matthias Nießner Angela Dai Mengyuan Yan Leonidas J. Guibas Stanford University 1. Details
More informationUnsupervised Training for 3D Morphable Model Regression
Unsupervised Training for 3D Morphable Model Regression Kyle Genova 1,2 Forrester Cole 2 Aaron Maschinot 2 Aaron Sarna 2 Daniel Vlasic 2 William T. Freeman 2,3 1 Princeton University 2 Google Research
More informationFML: Face Model Learning from Videos
FML: Face Model Learning from Videos Ayush Tewari1 Florian Bernard1 Pablo Garrido2 Gaurav Bharaj2 Mohamed Elgharib1 Hans-Peter Seidel1 Patrick Pe rez3 Michael Zollho fer4 Christian Theobalt1 1 MPI Informatics,
More informationNonrigid Surface Modelling. and Fast Recovery. Department of Computer Science and Engineering. Committee: Prof. Leo J. Jia and Prof. K. H.
Nonrigid Surface Modelling and Fast Recovery Zhu Jianke Supervisor: Prof. Michael R. Lyu Committee: Prof. Leo J. Jia and Prof. K. H. Wong Department of Computer Science and Engineering May 11, 2007 1 2
More informationarxiv: v3 [cs.cv] 26 Aug 2018 Abstract
Nonlinear 3D Face Morphable Model Luan Tran, Xiaoming Liu Department of Computer Science and Engineering Michigan State University, East Lansing MI 48824 {tranluan, liuxm}@msu.edu arxiv:1804.03786v3 [cs.cv]
More informationRobust FEC-CNN: A High Accuracy Facial Landmark Detection System
Robust FEC-CNN: A High Accuracy Facial Landmark Detection System Zhenliang He 1,2 Jie Zhang 1,2 Meina Kan 1,3 Shiguang Shan 1,3 Xilin Chen 1 1 Key Lab of Intelligent Information Processing of Chinese Academy
More informationPlug-and-Train Loss for Model-Based Single View 3D Reconstruction
Plug-and-Train Loss for Model-Based Single View 3D Reconstruction Eduard Ramon Crisalix SA eduard.ramon@crisalix.com Thomas Batard Crisalix SA thomas.batard@crisalix.com Jordi Villar Crisalix SA jordi.villar@crisalix.com
More information3D Morphable Models as Spatial Transformer Networks
3D Morphable Models as Spatial Transformer Networks Anil Bas *, Patrik Huber, William A. P. Smith *, Muhammad Awais, Josef Kittler * Department of Computer Science, University of York, UK Centre for Vision,
More informationarxiv: v2 [cs.cv] 8 Sep 2017
Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression Aaron S. Jackson 1 Adrian Bulat 1 Vasileios Argyriou 2 Georgios Tzimiropoulos 1 1 The University of Nottingham,
More informationDisentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognition
Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognition Feng Liu 1, Ronghang Zhu 1,DanZeng 1, Qijun Zhao 1,, and Xiaoming Liu 2 1 College of Computer Science, Sichuan University
More informationProceedings of the International MultiConference of Engineers and Computer Scientists 2018 Vol I IMECS 2018, March 14-16, 2018, Hong Kong
, March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong , March 14-16, 2018, Hong Kong TABLE I CLASSIFICATION ACCURACY OF DIFFERENT PRE-TRAINED MODELS ON THE TEST DATA
More informationJoint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network
Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network Yao Feng 1[0000 0002 9481 9783], Fan Wu 2[0000 0003 1970 3470], Xiaohu Shao 3,4[0000 0003 1141 6020], Yanfeng Wang
More informationDeep Learning for Face Recognition. Xiaogang Wang Department of Electronic Engineering, The Chinese University of Hong Kong
Deep Learning for Face Recognition Xiaogang Wang Department of Electronic Engineering, The Chinese University of Hong Kong Deep Learning Results on LFW Method Accuracy (%) # points # training images Huang
More information3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images
1 3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images Yudong Guo, Juyong Zhang, Jianfei Cai, Boyi Jiang and Jianmin Zheng arxiv:1708.00980v2 [cs.cv] 11 Sep 2017
More informationSupplementary Material: Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos
Supplementary Material: Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos Kihyuk Sohn 1 Sifei Liu 2 Guangyu Zhong 3 Xiang Yu 1 Ming-Hsuan Yang 2 Manmohan Chandraker 1,4 1 NEC Labs
More informationOVer the last few years, cascaded-regression (CR) based
1 Random Cascaded-Regression Copse for Robust Facial Landmark Detection Zhen-Hua Feng 1,2, Student Member, IEEE, Patrik Huber 2, Josef Kittler 2, Life Member, IEEE, William Christmas 2, and Xiao-Jun Wu
More informationLearning to generate 3D shapes
Learning to generate 3D shapes Subhransu Maji College of Information and Computer Sciences University of Massachusetts, Amherst http://people.cs.umass.edu/smaji August 10, 2018 @ Caltech Creating 3D shapes
More informationarxiv: v5 [cs.cv] 13 Apr 2018
Fine-Grained Head Pose Estimation Without Keypoints Nataniel Ruiz Eunji Chong James M. Rehg Georgia Institute of Technology {nataniel.ruiz, eunjichong, rehg}@gatech.edu arxiv:1710.00925v5 [cs.cv] 13 Apr
More informationA Fully End-to-End Cascaded CNN for Facial Landmark Detection
2017 IEEE 12th International Conference on Automatic Face & Gesture Recognition A Fully End-to-End Cascaded CNN for Facial Landmark Detection Zhenliang He 1,2 Meina Kan 1,3 Jie Zhang 1,2 Xilin Chen 1 Shiguang
More informationLearning to Recognize Faces in Realistic Conditions
000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
More informationRobust Face Recognition Based on Convolutional Neural Network
2017 2nd International Conference on Manufacturing Science and Information Engineering (ICMSIE 2017) ISBN: 978-1-60595-516-2 Robust Face Recognition Based on Convolutional Neural Network Ying Xu, Hui Ma,
More informationarxiv: v1 [cs.cv] 4 Aug 2011
arxiv:1108.1122v1 [cs.cv] 4 Aug 2011 Leveraging Billions of Faces to Overcome Performance Barriers in Unconstrained Face Recognition Yaniv Taigman and Lior Wolf face.com {yaniv, wolf}@face.com Abstract
More informationSupervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors
Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors Xuanyi Dong 1, Shoou-I Yu 2, Xinshuo Weng 2, Shih-En Wei 2, Yi Yang 1, Yaser Sheikh 2 xuanyi.dong@student.uts.edu.au;
More informationarxiv: v1 [cs.cv] 31 Mar 2017
End-to-End Spatial Transform Face Detection and Recognition Liying Chi Zhejiang University charrin0531@gmail.com Hongxin Zhang Zhejiang University zhx@cad.zju.edu.cn Mingxiu Chen Rokid.inc cmxnono@rokid.com
More informationUnconstrained Face Alignment without Face Detection
Unconstrained Face Alignment without Face Detection Xiaohu Shao 1,2, Junliang Xing 3, Jiangjing Lv 1,2, Chunlin Xiao 4, Pengcheng Liu 1, Youji Feng 1, Cheng Cheng 1 1 Chongqing Institute of Green and Intelligent
More informationBoosting face recognition via neural Super-Resolution
Boosting face recognition via neural Super-Resolution Guillaume Berger, Cle ment Peyrard and Moez Baccouche Orange Labs - 4 rue du Clos Courtel, 35510 Cesson-Se vigne - France Abstract. We propose a two-step
More informationStructured Prediction using Convolutional Neural Networks
Overview Structured Prediction using Convolutional Neural Networks Bohyung Han bhhan@postech.ac.kr Computer Vision Lab. Convolutional Neural Networks (CNNs) Structured predictions for low level computer
More informationLearn to Combine Multiple Hypotheses for Accurate Face Alignment
2013 IEEE International Conference on Computer Vision Workshops Learn to Combine Multiple Hypotheses for Accurate Face Alignment Junjie Yan Zhen Lei Dong Yi Stan Z. Li Center for Biometrics and Security
More informationDe-mark GAN: Removing Dense Watermark With Generative Adversarial Network
De-mark GAN: Removing Dense Watermark With Generative Adversarial Network Jinlin Wu, Hailin Shi, Shu Zhang, Zhen Lei, Yang Yang, Stan Z. Li Center for Biometrics and Security Research & National Laboratory
More informationExtended Supervised Descent Method for Robust Face Alignment
Extended Supervised Descent Method for Robust Face Alignment Liu Liu, Jiani Hu, Shuo Zhang, Weihong Deng Beijing University of Posts and Telecommunications, Beijing, China Abstract. Supervised Descent
More informationDOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION
DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION Yen-Cheng Liu 1, Wei-Chen Chiu 2, Sheng-De Wang 1, and Yu-Chiang Frank Wang 1 1 Graduate Institute of Electrical Engineering,
More informationFace Alignment Across Large Poses: A 3D Solution
Face Alignment Across Large Poses: A 3D Solution Outline Face Alignment Related Works 3D Morphable Model Projected Normalized Coordinate Code Network Structure 3D Image Rotation Performance on Datasets
More informationMultilinear Autoencoder for 3D Face Model Learning
Multilinear Autoencoder for 3D Face Model Learning Victoria Fernández Abrevaya, Stefanie Wuhrer, Edmond Boyer To cite this version: Victoria Fernández Abrevaya, Stefanie Wuhrer, Edmond Boyer. Multilinear
More informationECCV Presented by: Boris Ivanovic and Yolanda Wang CS 331B - November 16, 2016
ECCV 2016 Presented by: Boris Ivanovic and Yolanda Wang CS 331B - November 16, 2016 Fundamental Question What is a good vector representation of an object? Something that can be easily predicted from 2D
More informationarxiv: v1 [cs.cv] 29 May 2018
HeadOn: Real-time Reenactment of Human Portrait Videos JUSTUS THIES, Technical University of Munich MICHAEL ZOLLHÖFER, Stanford University CHRISTIAN THEOBALT, Max-Planck-Institute for Informatics MARC
More informationDetector of Facial Landmarks Learned by the Structured Output SVM
Detector of Facial Landmarks Learned by the Structured Output SVM Michal Uřičář, Vojtěch Franc and Václav Hlaváč Department of Cybernetics, Faculty Electrical Engineering Czech Technical University in
More informationFacial shape tracking via spatio-temporal cascade shape regression
Facial shape tracking via spatio-temporal cascade shape regression Jing Yang nuist yj@126.com Jiankang Deng jiankangdeng@gmail.com Qingshan Liu Kaihua Zhang zhkhua@gmail.com qsliu@nuist.edu.cn Nanjing
More informationLearning Detailed Face Reconstruction from a Single Image
Learning Detailed Face Reconstruction from a Single Image Elad Richardson 1 Matan Sela 1 Roy Or-El 2 Ron Kimmel 1 1 Department of Computer Science, Technion - Israel Institute of Technology 2 Department
More informationFacial Expression Analysis
Facial Expression Analysis Jeff Cohn Fernando De la Torre Human Sensing Laboratory Tutorial Looking @ People June 2012 Facial Expression Analysis F. De la Torre/J. Cohn Looking @ People (CVPR-12) 1 Outline
More informationSpontaneously Emerging Object Part Segmentation
Spontaneously Emerging Object Part Segmentation Yijie Wang Machine Learning Department Carnegie Mellon University yijiewang@cmu.edu Katerina Fragkiadaki Machine Learning Department Carnegie Mellon University
More informationDeep Learning for Visual Manipulation and Synthesis
Deep Learning for Visual Manipulation and Synthesis Jun-Yan Zhu 朱俊彦 UC Berkeley 2017/01/11 @ VALSE What is visual manipulation? Image Editing Program input photo User Input result Desired output: stay
More informationPose-Invariant Face Alignment with a Single CNN
Pose-Invariant Face Alignment with a Single CNN Amin Jourabloo 1, Mao Ye 2, Xiaoming Liu 1, and Liu Ren 2 1 Department of Computer Science and Engineering, Michigan State University, MI 2 Bosch Research
More informationDynamic Routing Between Capsules
Report Explainable Machine Learning Dynamic Routing Between Capsules Author: Michael Dorkenwald Supervisor: Dr. Ullrich Köthe 28. Juni 2018 Inhaltsverzeichnis 1 Introduction 2 2 Motivation 2 3 CapusleNet
More informationNeural Face Editing with Intrinsic Image Disentangling SUPPLEMENTARY MATERIAL
Neural Face Editing with Intrinsic Image Disentangling SUPPLEMENTARY MATERIAL Zhixin Shu 1 Ersin Yumer 2 Sunil Hadap 2 Kalyan Sunkavalli 2 Eli Shechtman 2 Dimitris Samaras 1,3 1 Stony Brook University
More informationShape Augmented Regression for 3D Face Alignment
Shape Augmented Regression for 3D Face Alignment Chao Gou 1,3(B),YueWu 2, Fei-Yue Wang 1,3, and Qiang Ji 2 1 Institute of Automation, Chinese Academy of Sciences, Beijing, China {gouchao2012,feiyue.wang}@ia.ac.cn
More informationDOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION
2017 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 25 28, 2017, TOKYO, JAPAN DOMAIN-ADAPTIVE GENERATIVE ADVERSARIAL NETWORKS FOR SKETCH-TO-PHOTO INVERSION Yen-Cheng Liu 1,
More informationFaceNet. Florian Schroff, Dmitry Kalenichenko, James Philbin Google Inc. Presentation by Ignacio Aranguren and Rahul Rana
FaceNet Florian Schroff, Dmitry Kalenichenko, James Philbin Google Inc. Presentation by Ignacio Aranguren and Rahul Rana Introduction FaceNet learns a mapping from face images to a compact Euclidean Space
More informationon learned visual embedding patrick pérez Allegro Workshop Inria Rhônes-Alpes 22 July 2015
on learned visual embedding patrick pérez Allegro Workshop Inria Rhônes-Alpes 22 July 2015 Vector visual representation Fixed-size image representation High-dim (100 100,000) Generic, unsupervised: BoW,
More informationFace Alignment across Large Pose via MT-CNN based 3D Shape Reconstruction
Face Alignment across Large Pose via MT-CNN based 3D Shape Reconstruction Gang Zhang 1,2, Hu Han 1, Shiguang Shan 1,2,3, Xingguang Song 4, Xilin Chen 1,2 1 Key Laboratory of Intelligent Information Processing
More informationFace2Face Comparing faces with applications Patrick Pérez. Inria, Rennes 2 Oct. 2014
Face2Face Comparing faces with applications Patrick Pérez Inria, Rennes 2 Oct. 2014 Outline Metric learning for face comparison Expandable parts model and occlusions Face sets comparison Identity-based
More informationDeep Alignment Network: A convolutional neural network for robust face alignment
Deep Alignment Network: A convolutional neural network for robust face alignment Marek Kowalski, Jacek Naruniec, and Tomasz Trzcinski Warsaw University of Technology m.kowalski@ire.pw.edu.pl, j.naruniec@ire.pw.edu.pl,
More informationContent-Based Image Recovery
Content-Based Image Recovery Hong-Yu Zhou and Jianxin Wu National Key Laboratory for Novel Software Technology Nanjing University, China zhouhy@lamda.nju.edu.cn wujx2001@nju.edu.cn Abstract. We propose
More informationarxiv: v1 [cs.cv] 12 Mar 2014
Learning Deep Face Representation Haoqiang Fan Megvii Inc. fhq@megvii.com Zhimin Cao Megvii Inc. czm@megvii.com Yuning Jiang Megvii Inc. jyn@megvii.com Qi Yin Megvii Inc. yq@megvii.com arxiv:1403.2802v1
More informationYiqi Yan. May 10, 2017
Yiqi Yan May 10, 2017 P a r t I F u n d a m e n t a l B a c k g r o u n d s Convolution Single Filter Multiple Filters 3 Convolution: case study, 2 filters 4 Convolution: receptive field receptive field
More informationSu et al. Shape Descriptors - III
Su et al. Shape Descriptors - III Siddhartha Chaudhuri http://www.cse.iitb.ac.in/~cs749 Funkhouser; Feng, Liu, Gong Recap Global A shape descriptor is a set of numbers that describes a shape in a way that
More informationTweaked residual convolutional network for face alignment
Journal of Physics: Conference Series PAPER OPEN ACCESS Tweaked residual convolutional network for face alignment To cite this article: Wenchao Du et al 2017 J. Phys.: Conf. Ser. 887 012077 Related content
More informationarxiv: v1 [cs.cv] 3 Mar 2018
Unsupervised Learning of Face Representations Samyak Datta, Gaurav Sharma, C.V. Jawahar Georgia Institute of Technology, CVIT, IIIT Hyderabad, IIT Kanpur arxiv:1803.01260v1 [cs.cv] 3 Mar 2018 Abstract
More informationReconstructive Sparse Code Transfer for Contour Detection and Semantic Labeling
Reconstructive Sparse Code Transfer for Contour Detection and Semantic Labeling Michael Maire 1,2 Stella X. Yu 3 Pietro Perona 2 1 TTI Chicago 2 California Institute of Technology 3 University of California
More informationarxiv: v1 [cs.cv] 17 Apr 2017
End-to-end 3D face reconstruction with deep neural networks arxiv:1704.05020v1 [cs.cv] 17 Apr 2017 Pengfei Dou, Shishir K. Shah, and Ioannis A. Kakadiaris Computational Biomedicine Lab University of Houston
More informationarxiv: v2 [cs.cv] 15 Sep 2017
Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation Matan Sela Elad Richardson Ron Kimmel Department of Computer Science, Technion - Israel Institute of Technology {matansel,eladrich,ron}@cs.technion.ac.il
More informationDD2427 Final Project Report. Human face attributes prediction with Deep Learning
DD2427 Final Project Report Human face attributes prediction with Deep Learning Abstract moaah@kth.se We explore using deep Convolutional Neural Networks (CNN) to predict human attributes like skin tune,
More informationRecovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform. Xintao Wang Ke Yu Chao Dong Chen Change Loy
Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform Xintao Wang Ke Yu Chao Dong Chen Change Loy Problem enlarge 4 times Low-resolution image High-resolution image Previous
More informationarxiv: v5 [cs.cv] 26 Jul 2017
arxiv:6.8657v5 [cs.cv] 26 Jul 27 Convolutional Experts Network for Facial Landmark Detection Amir Zadeh Carnegie Mellon University 5 Forbes Ave, Pittsburgh, PA 523, USA Tadas Baltrus aitis Carnegie Mellon
More informationDeep Learning in Image and Face Intrinsics
Deep Learning in Image and Face Intrinsics Theo Gevers Professor in Computer Vision: University of Amsterdam Co-founder: Sightcorp 3DUniversum Scanm WIC, Eindhoven, February 24th, 2018 Amsterdam A.I. -
More informationarxiv: v1 [cs.cv] 11 Jun 2015
Pose-Invariant 3D Face Alignment Amin Jourabloo, Xiaoming Liu Department of Computer Science and Engineering Michigan State University, East Lansing MI 48824 {jourablo, liuxm}@msu.edu arxiv:506.03799v
More informationLightweight Two-Stream Convolutional Face Detection
Lightweight Two-Stream Convolutional Face Detection Danai Triantafyllidou, Paraskevi Nousi and Anastasios Tefas Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece danaitri22@gmail.com,
More informationSupplementary Material Estimating Correspondences of Deformable Objects In-the-wild
Supplementary Material Estimating Correspondences of Deformable Objects In-the-wild Yuxiang Zhou Epameinondas Antonakos Joan Alabort-i-Medina Anastasios Roussos Stefanos Zafeiriou, Department of Computing,
More information