Synthesizing Realistic Facial Expressions from Photographs
|
|
- Virginia Webster
- 6 years ago
- Views:
Transcription
1 Synthesizing Realistic Facial Expressions from Photographs 1998 F. Pighin, J Hecker, D. Lischinskiy, R. Szeliskiz and D. H. Salesin University of Washington, The Hebrew University Microsoft Research 1
2 Introduction Techniques for creating photorealistic textured 3D facial models from photographs of a human subject and creating smooth transitions between different facial expressions by morphing. Model fitting: 1. User-assisted technique to recover the camera poses corresponding to the views + a sparse set of chosen locations on the subject s face. 2. Scattered data interpolation technique: deforms a generic face mesh to fit the particular geometry of the subject s face. Texture extraction: extracting one or more texture maps from the input images (repeated for several facial expressions). Expression morphing: Generating transitions between the corresponding face models, and blending the textures. 2
3 Motivation Realistic facial synthesis is one of the most fundamental problems in computer graphics, and one of the most difficult. First attempts date back to the 70s. Applications include character animation, computer games, video teleconferencing and facial surgery planning. No perfectly realistic facial animation has ever been generated by a computer. The facial animation Turing test has never been passed. 3
4 Motivation (2) Why is this such a difficult problem? The human face is an extremely complicated geometric form. The face exhibits countless tiny wrinkles and creases, as well as subtle variations in color and texture. Animation is even harder: facial movement is a product of underlying skeletal and muscular forms (+ skin and subcutaneous layers). We look at faces everyday, it is easy for us to detect even the slightest deviation from true faces. We can read facial expressions easily. 4
5 Earlier Work Computer Generated Animation of Faces, Frederic I. Parke (1972): The idea was to produce "realistic" computer generated animated sequences of a human face changing expression. Animation was accomplished using a cosine interpolation scheme to fill in the intermediate frames between expressions. Parke s work introduced simple geometric interpolation between face models. 5
6 Earlier Work (2) Feature-Based Image Metamorphosis (1992): 2D morphing between photographic images. Problems with 2D image morphing: It does not correctly account for changes in viewpoint or object pose. This has been recently addressed by a technique called viewmorphing. 2D morphing still lacks some of the advantages of a 3D model, such as the complete freedom of viewpoint. Michael Jackson s morphing music video 6
7 Similar Work A different approach: performance based animation; measurements from real actors are used to drive synthetic characters. Modeling, Tracking and Interactive Animation of Faces and Heads using Input from Video (1996): Idea: Coupling pixel-by-pixel measurements of surface motion to a physically-based face model and a muscle control model -> detailed spatio-temporal records of both the displacement of each point on the facial surface and the muscle control required to produce the facial motion. 7
8 Advantages of this Technique 1. Gives the user complete freedom in specifying the correspondences, and enables the user to refine the initial fit as needed. 2. Can handle arbitrary camera positions and lenses. 3. Creates realistic face models as well as realistic transitions for face expressions. 4. Is capable of extracting texture maps of high resolution. 5. Can capture subtle changes in illumination and appearance. Price: User intervention in the modeling process. 8
9 General Pipeline Capture multiple views of a human subject (with a given facial expression) using cameras at arbitrary locations. Deform a generic 3D face mesh to fit the face of the particular human subject. Extract texture maps from the photos (single view-independent texture map or view-dependent texture map). Manually mark a small set of initial corresponding points on the face in the different views. Automatically recover the camera parameters (position, focal length, etc.) for each view, and 3D positions. (repeated for different facial expressions) Facial animation: Interpolate between two or more different 3D models, while at the same time blending textures. 9
10 Model Fitting Task: Adapt a generic face model to fit an individual s face and facial expression. Input: 1. Several images of the face from different viewpoints. 2. Generic face model. Output: Face model that has been adapted to fit the face in the input images. User interaction: Selecting feature points and correspondences for refinement. 10
11 Model Fitting (2) 11
12 Model Fitting (3) Consists of 3 stages: 1. Pose Recovery: Apply computer vision techniques to estimate the viewing parameters (position, orientation, focal length) for each input camera -> recover 3D coordinates of feature points. 2. Scattered Data Interpolation: Compute the positions of the remaining face mesh vertices. 3. Shape Refinement: Specify additional correspondences between facial vertices and image coordinates to improve the estimated shape of the face. 12
13 Pose Recovery Technique based on the non-linear least squares structure. The problem is reduced to a series of linear least squares problems that can be solved using very simple and numerically stable techniques. 13
14 Pose Recovery (2) For each camera pose k we have a rotation matrix R k and a translation vector t k. The 3 rows of R k are r k x, rk y and rk z, and the 3 entries in tk are t k x, tk y and tk z. Each feature point p i has 2D screen coordinates in the k th image (x k i, yk i ). Traditional 3D projection equation: 14
15 Pose Recovery (3) Inverse distance World-to-image scale factor Advantage of this formulation: The scale factor can be reliably estimated even if f is too long. In the original formation there is a strong coupling between f and t z. 15
16 Pose Recovery (4) These 2 equations are linear in each of the unknowns p i, t k x, tk y, ηk, s k, and R k. Solve the unknowns in 6 steps in the following order: s k, p i, R k, t k x, tk y, ηk. (maximum numerical stability) Linear least squares is used to solve for each parameter. 16
17 Scattered Data Interpolation Once we have an initial set of 3D feature points p i, we use these values to deform the remaining vertices on the face mesh. Smooth interpolation function: gives 3D displacement between original point positions and the new adapted positions for every vertex in the original generic face mesh. Find u j for every unconstrained vertex j. 17
18 Scattered Data Interpolation (2) Let Radial basis functions: Goal: determine coefficients c i and the affine components M and t. Interpolation constraints: Extra constraints: 18
19 Correspondence-based Shape Refinement After warping the generic face model into its new shape, we can still further improve it by specifying additional correspondences. Re-run the scattered data interpolation step (iteratively) to update the vertices for which no correspondences are given. 19
20 Texture mapping Texture colour for each point on the face model. Methods I. View-independent -> Cylindrical texture map II. View-dependent -> Blending with weights 20
21 21
22 View-independent Feathered visibility map Positional certainty Weight map -> binary visibility map 22
23 Texture Value Weighted map Image function projection coordinates 23
24 24
25 View-dependent texture mapping Render model with different input photo s and blend. Viewing direction d View dependent term 25
26 Results view-(in)dependent blending 26
27 Eyes, teeth, ears, and hair Individual texture map Brightness modulating 27
28 Morphing expressions for animation Linear combination (Vetter and Blanz 1998) -> Simple linear interpolation between coordinates of corresponding vertices of each two face meshes. -> Identical topology: natural correspondence between vertices. ->Blended expression rendered using original texture map with weight at each vertex. 28
29 Local Blending Blend specification: Global Regional Painterly interface 29
30 Facial Action Coding System 30
31 User Interface Keyframe animation Expression gallery Timeline 31
32 Results 32
33 Suggested improvements Color correction. Improved registration. Texture relighting. Automatic modeling. Modeling from video. Complex animations Lip-synching. Performance-driven animation. 33
34 Recent work Facial expressions from video (real-time) Real-Time High-Fidelity Facial Performance Capture 2015 Automatic technique for parameter extraction
35 Dynamic 3D Avatar Creation from Hand-held Video Input 80 images for static reconstruction and > 90 sec. of video for dynamic modeling Dynamic blendshape/morphing Visual feedback still needed Albedo map 35
36 Conclusion Generation of realistic face images. Extract information from face images. Wide range of expressions by linear combination. Specialized for a given person 36
37 Future work More realistic. Behaviour-driven (actor/director and motivation of character). 37
38 Discussion Can this technique be used for anything else? (Example: Animal faces, other body parts, plants, buildings, etc ). How well would this work? We had a good remark that animals have less expressions anyway. Also because of the obstruction of hair it would be probably harder using this technique. But perhaps using this techniques the transformation to a different state (from neural to aggressive) can be generated automatically. Additionally this technique might be used the other way around. Much work has been done for architecture using this technique. 38
39 Discussion (2) How can we improve model fitting, specifically the automation of obtaining valid feature points? We did not discuss this in class but we thought of machine learning as a method to improve the building of the fitted face model. If a computer can learn itself to detect the features based on only the input photographs it can take away the user intervention and this can be used for other applications (architecture, maybe plants?) 39
40 Discussion (3) How far can this technology go? (security/moral issues, fraud ) Based on the reactions during the video presentation, this is not convincing enough, but we can imagine this becoming a problem in the future (especially if speech technology also improves). We already saw in presentation 7 the state of the art techniques that brought single pictures to life and the possible adjustments to that face. 40
41 References F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. H. Salesin Synthesizing realistic facial expressions from photographs. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques(siggraph '98). ACM, New York, NY, USA, A. Ichim, P Kadlecek, L Kavan, M Pauly Physics-based Face Modeling and Animation S. Bouaziz, A. Tagliasacchi, M. Pauly Dynamic 2D/3D Registration. Eurographics Tutorial (Proc. of EG'14) C. Wang, F. Shi, S. Xia Realtime 3D Eye Gaze Animation Using a Single RGB Camera. ACM Transactions on Graphics, 35(4) A.E. Ichim, S. Bouaziz, M Pauly Dynamic 3D Avatar Creation from Hand-held Video Input. ACM Transactions on Graphics (ToG) 34.4 (2015): 45 41
42 References Parke, Frederick I. "Computer generated animation of faces." Proceedings of the ACM annual conference-volume 1. ACM, Beier, Thaddeus, and Shawn Neely. "Feature-based image metamorphosis." ACM SIGGRAPH Computer Graphics. Vol. 26. No. 2. ACM, Essa, Irfan, et al. "Modeling, tracking and interactive animation of faces and heads//using input from video." Computer Animation'96. Proceedings. IEEE,
A Morphable Model for the Synthesis of 3D Faces
A Morphable Model for the Synthesis of 3D Faces Marco Nef Volker Blanz, Thomas Vetter SIGGRAPH 99, Los Angeles Presentation overview Motivation Introduction Database Morphable 3D Face Model Matching a
More informationFaces. Face Modeling. Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 17
Face Modeling Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 17 Faces CS291-J00, Winter 2003 From David Romdhani Kriegman, slides 2003 1 Approaches 2-D Models morphing, indexing, etc.
More informationFacial Image Synthesis 1 Barry-John Theobald and Jeffrey F. Cohn
Facial Image Synthesis Page 1 of 5 Facial Image Synthesis 1 Barry-John Theobald and Jeffrey F. Cohn 1 Introduction Facial expression has been central to the
More informationImage Morphing. Application: Movie Special Effects. Application: Registration /Alignment. Image Cross-Dissolve
Image Morphing Application: Movie Special Effects Morphing is turning one image into another (through a seamless transition) First movies with morphing Willow, 1988 Indiana Jones and the Last Crusade,
More informationSYNTHESIS OF 3D FACES
SYNTHESIS OF 3D FACES R. Enciso, J. Li, D.A. Fidaleo, T-Y Kim, J-Y Noh and U. Neumann Integrated Media Systems Center University of Southern California Los Angeles, CA 90089, U.S.A. Abstract In this paper,
More informationFaces and Image-Based Lighting
Announcements Faces and Image-Based Lighting Project #3 artifacts voting Final project: Demo on 6/25 (Wednesday) 13:30pm in this room Reports and videos due on 6/26 (Thursday) 11:59pm Digital Visual Effects,
More informationImage-Based Deformation of Objects in Real Scenes
Image-Based Deformation of Objects in Real Scenes Han-Vit Chung and In-Kwon Lee Dept. of Computer Science, Yonsei University sharpguy@cs.yonsei.ac.kr, iklee@yonsei.ac.kr Abstract. We present a new method
More informationAnimation of 3D surfaces.
Animation of 3D surfaces Motivations When character animation is controlled by skeleton set of hierarchical joints joints oriented by rotations the character shape still needs to be visible: visible =
More informationComputer Animation Visualization. Lecture 5. Facial animation
Computer Animation Visualization Lecture 5 Facial animation Taku Komura Facial Animation The face is deformable Need to decide how all the vertices on the surface shall move Manually create them Muscle-based
More informationFACIAL ANIMATION FROM SEVERAL IMAGES
International Archives of Photogrammetry and Remote Sensing. Vol. XXXII, Part 5. Hakodate 1998 FACIAL ANIMATION FROM SEVERAL IMAGES Yasuhiro MUKAIGAWAt Yuichi NAKAMURA+ Yuichi OHTA+ t Department of Information
More informationHuman body animation. Computer Animation. Human Body Animation. Skeletal Animation
Computer Animation Aitor Rovira March 2010 Human body animation Based on slides by Marco Gillies Human Body Animation Skeletal Animation Skeletal Animation (FK, IK) Motion Capture Motion Editing (retargeting,
More informationAnimation of 3D surfaces
Animation of 3D surfaces 2013-14 Motivations When character animation is controlled by skeleton set of hierarchical joints joints oriented by rotations the character shape still needs to be visible: visible
More informationPerformance Driven Facial Animation using Blendshape Interpolation
Performance Driven Facial Animation using Blendshape Interpolation Erika Chuang Chris Bregler Computer Science Department Stanford University Abstract This paper describes a method of creating facial animation
More informationImage-based Motion-driven Facial Texture
Image-based Motion-driven Facial Texture Bing Zhang and Hai Tao and Alex Pang Computer Science and Computer Engineering University of California, Santa Cruz 1 INTRODUCTION Facial animation is a fundamental
More informationCSE452 Computer Graphics
CSE452 Computer Graphics Lecture 19: From Morphing To Animation Capturing and Animating Skin Deformation in Human Motion, Park and Hodgins, SIGGRAPH 2006 CSE452 Lecture 19: From Morphing to Animation 1
More informationFacial Expression Analysis for Model-Based Coding of Video Sequences
Picture Coding Symposium, pp. 33-38, Berlin, September 1997. Facial Expression Analysis for Model-Based Coding of Video Sequences Peter Eisert and Bernd Girod Telecommunications Institute, University of
More informationAnimation COM3404. Richard Everson. School of Engineering, Computer Science and Mathematics University of Exeter
Animation COM3404 Richard Everson School of Engineering, Computer Science and Mathematics University of Exeter R.M.Everson@exeter.ac.uk http://www.secamlocal.ex.ac.uk/studyres/com304 Richard Everson Animation
More informationINTERNATIONAL JOURNAL OF GRAPHICS AND MULTIMEDIA (IJGM)
INTERNATIONAL JOURNAL OF GRAPHICS AND MULTIMEDIA (IJGM) International Journal of Graphics and Multimedia (IJGM), ISSN: 0976 6448 (Print) ISSN: 0976 ISSN : 0976 6448 (Print) ISSN : 0976 6456 (Online) Volume
More informationData-Driven Face Modeling and Animation
1. Research Team Data-Driven Face Modeling and Animation Project Leader: Post Doc(s): Graduate Students: Undergraduate Students: Prof. Ulrich Neumann, IMSC and Computer Science John P. Lewis Zhigang Deng,
More informationFace Morphing using 3D-Aware Appearance Optimization
Face Morphing using 3D-Aware Appearance Optimization Fei Yang 1 Eli Shechtman 2 Jue Wang 2 Lubomir Bourdev 2 Dimitris Metaxas 1 1 Rutgers University 2 Adobe Systems Figure 1: Our system can generate fully
More informationHIGH-RESOLUTION ANIMATION OF FACIAL DYNAMICS
HIGH-RESOLUTION ANIMATION OF FACIAL DYNAMICS N. Nadtoka, J.R. Tena, A. Hilton, J. Edge Centre for Vision, Speech and Signal Processing, University of Surrey {N.Nadtoka, J.Tena, A.Hilton}@surrey.ac.uk Keywords:
More informationTopics in Computer Animation
Topics in Computer Animation Animation Techniques Artist Driven animation The artist draws some frames (keyframing) Usually in 2D The computer generates intermediate frames using interpolation The old
More informationAdvanced Graphics and Animation
Advanced Graphics and Animation Character Marco Gillies and Dan Jones Goldsmiths Aims and objectives By the end of the lecture you will be able to describe How 3D characters are animated Skeletal animation
More informationA New Image Based Ligthing Method: Practical Shadow-Based Light Reconstruction
A New Image Based Ligthing Method: Practical Shadow-Based Light Reconstruction Jaemin Lee and Ergun Akleman Visualization Sciences Program Texas A&M University Abstract In this paper we present a practical
More informationInteractive 3D Scene Reconstruction from Images
Interactive 3D Scene Reconstruction from Images Aaron Hertzmann Media Research Laboratory Department of Computer Science New York University 719 Broadway, 12th Floor New York, NY 10003 hertzman@mrl.nyu.edu
More informationInteractive Deformation with Triangles
Interactive Deformation with Triangles James Dean Palmer and Ergun Akleman Visualization Sciences Program Texas A&M University Jianer Chen Department of Computer Science Texas A&M University Abstract In
More informationAbstract We present a system which automatically generates a 3D face model from a single frontal image of a face. Our system consists of two component
A Fully Automatic System To Model Faces From a Single Image Zicheng Liu Microsoft Research August 2003 Technical Report MSR-TR-2003-55 Microsoft Research Microsoft Corporation One Microsoft Way Redmond,
More informationPhysical based Rigging
Physical based Rigging Dinghuang Ji Introduction Computer animation has been a popular research topic since 1970s, when the first parametric facial model is proposed[0]. In the recent few years, a lot
More information3D Active Appearance Model for Aligning Faces in 2D Images
3D Active Appearance Model for Aligning Faces in 2D Images Chun-Wei Chen and Chieh-Chih Wang Abstract Perceiving human faces is one of the most important functions for human robot interaction. The active
More informationLecture 7: Image Morphing. Idea #2: Align, then cross-disolve. Dog Averaging. Averaging vectors. Idea #1: Cross-Dissolving / Cross-fading
Lecture 7: Image Morphing Averaging vectors v = p + α (q p) = (1 - α) p + α q where α = q - v p α v (1-α) q p and q can be anything: points on a plane (2D) or in space (3D) Colors in RGB or HSV (3D) Whole
More informationFeature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who 1 in turn adapted slides from Steve Seitz, Rick Szeliski,
More informationSpeech Driven Synthesis of Talking Head Sequences
3D Image Analysis and Synthesis, pp. 5-56, Erlangen, November 997. Speech Driven Synthesis of Talking Head Sequences Peter Eisert, Subhasis Chaudhuri,andBerndGirod Telecommunications Laboratory, University
More informationVideo based Animation Synthesis with the Essential Graph. Adnane Boukhayma, Edmond Boyer MORPHEO INRIA Grenoble Rhône-Alpes
Video based Animation Synthesis with the Essential Graph Adnane Boukhayma, Edmond Boyer MORPHEO INRIA Grenoble Rhône-Alpes Goal Given a set of 4D models, how to generate realistic motion from user specified
More information2D + 3D FACE MORPHING 1 Yen-Cheng Liu ( ) 2 Pei-Hwai Ciou ( ) Chiou-Shann Fuh ( )
2D + 3D FACE MORPHING 1 Yen-Cheng Liu ( ) 2 Pei-Hwai Ciou ( ) Chiou-Shann Fuh ( ) 1, 2 Graduate Institute of Electrical Engineering National Taiwan University Taipei, Taiwan r49213@ntu.edu.tw r49218@ntu.edu.tw
More informationMotion Synthesis and Editing. Yisheng Chen
Motion Synthesis and Editing Yisheng Chen Overview Data driven motion synthesis automatically generate motion from a motion capture database, offline or interactive User inputs Large, high-dimensional
More informationSCAPE: Shape Completion and Animation of People
SCAPE: Shape Completion and Animation of People By Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, James Davis From SIGGRAPH 2005 Presentation for CS468 by Emilio Antúnez
More informationFree-Form Deformation and Other Deformation Techniques
Free-Form Deformation and Other Deformation Techniques Deformation Deformation Basic Definition Deformation: A transformation/mapping of the positions of every particle in the original object to those
More informationPose Space Deformation A unified Approach to Shape Interpolation and Skeleton-Driven Deformation
Pose Space Deformation A unified Approach to Shape Interpolation and Skeleton-Driven Deformation J.P. Lewis Matt Cordner Nickson Fong Presented by 1 Talk Outline Character Animation Overview Problem Statement
More informationInteractive Shape Metamorphosis
Interactive Shape Metamorphosis David T. Chen Andrei State Department of Computer Science University of North Carolina Chapel Hill, NC 27599 David Banks Institute for Computer Applications in Science and
More informationREALISTIC facial expression synthesis has been one of the
48 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 12, NO. 1, JANUARY/FEBRUARY 2006 Geometry-Driven Photorealistic Facial Expression Synthesis Qingshan Zhang, Zicheng Liu, Senior Member,
More informationGraphics, Vision, HCI. K.P. Chan Wenping Wang Li-Yi Wei Kenneth Wong Yizhou Yu
Graphics, Vision, HCI K.P. Chan Wenping Wang Li-Yi Wei Kenneth Wong Yizhou Yu Li-Yi Wei Background Stanford (95-01), NVIDIA (01-05), MSR (05-11) Research Nominal: Graphics, HCI, parallelism Actual: Computing
More informationK A I S T Department of Computer Science
An Example-based Approach to Text-driven Speech Animation with Emotional Expressions Hyewon Pyun, Wonseok Chae, Yejin Kim, Hyungwoo Kang, and Sung Yong Shin CS/TR-2004-200 July 19, 2004 K A I S T Department
More informationReal-Time Universal Capture Facial Animation with GPU Skin Rendering
Real-Time Universal Capture Facial Animation with GPU Skin Rendering Meng Yang mengyang@seas.upenn.edu PROJECT ABSTRACT The project implements the real-time skin rendering algorithm presented in [1], and
More informationRealistic Human Head Modeling with Multi-view Hairstyle Reconstruction
Realistic Human Head Modeling with Multi-view Hairstyle Reconstruction Xiaolan Li and Hongbin Zha ational Laboratory on Machine Perception, Peing University 100871, Beiing, P.R.China Email: {lixlan,zha}@cis.pu.edu.cn
More informationCS 231. Deformation simulation (and faces)
CS 231 Deformation simulation (and faces) Deformation BODY Simulation Discretization Spring-mass models difficult to model continuum properties Simple & fast to implement and understand Finite Element
More informationNON-RIGID 3D SHAPE RECOVERY USING STEREO FACTORIZATION
NON-IGID D SHAPE ECOVEY USING STEEO FACTOIZATION Alessio Del Bue ourdes Agapito Department of Computer Science Queen Mary, University of ondon ondon, E1 NS, UK falessio,lourdesg@dcsqmulacuk ABSTACT In
More informationReading. Animation principles. Required:
Reading Required: Animation principles John Lasseter. Principles of traditional animation applied to 3D computer animation. Proceedings of SIGGRAPH (Computer Graphics) 21(4): 35-44, July 1987. Recommended:
More informationSupplementary Material for Synthesizing Normalized Faces from Facial Identity Features
Supplementary Material for Synthesizing Normalized Faces from Facial Identity Features Forrester Cole 1 David Belanger 1,2 Dilip Krishnan 1 Aaron Sarna 1 Inbar Mosseri 1 William T. Freeman 1,3 1 Google,
More informationFundamentals of Warping and Morphing
Fundamentals of Warping and Morphing Luiz Velho IMPA - Institututo de Matemática Pura e Aplicada Outline Metamorphosis in Nature Conceptual Framework Overview of Warping and Morphing Applications in Computer
More informationSample Based Texture extraction for Model based coding
DEPARTMENT OF APPLIED PHYSICS AND ELECTRONICS UMEÅ UNIVERISTY, SWEDEN DIGITAL MEDIA LAB Sample Based Texture extraction for Model based coding Zhengrong Yao 1 Dept. Applied Physics and Electronics Umeå
More informationFeature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve Seitz, Rick Szeliski,
More informationIntrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting R. Maier 1,2, K. Kim 1, D. Cremers 2, J. Kautz 1, M. Nießner 2,3 Fusion Ours 1
More informationStereo pairs from linear morphing
Proc. of SPIE Vol. 3295, Stereoscopic Displays and Virtual Reality Systems V, ed. M T Bolas, S S Fisher, J O Merritt (Apr 1998) Copyright SPIE Stereo pairs from linear morphing David F. McAllister Multimedia
More informationAnnouncements: Quiz. Animation, Motion Capture, & Inverse Kinematics. Last Time? Today: How do we Animate? Keyframing. Procedural Animation
Announcements: Quiz Animation, Motion Capture, & Inverse Kinematics On Friday (3/1), in class One 8.5x11 sheet of notes allowed Sample quiz (from a previous year) on website Focus on reading comprehension
More informationCS 231. Deformation simulation (and faces)
CS 231 Deformation simulation (and faces) 1 Cloth Simulation deformable surface model Represent cloth model as a triangular or rectangular grid Points of finite mass as vertices Forces or energies of points
More informationImage stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac
Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration
More informationDeformation Transfer for Triangle Meshes
Deformation Transfer for Triangle Meshes a Paper (SIGGRAPH 2004) by Robert W. Sumner & Jovan Popovic presented by Roni Oeschger Deformation Transfer Source deformed Target deformed 1 Outline of my presentation
More informationMidterm Examination CS 534: Computational Photography
Midterm Examination CS 534: Computational Photography November 3, 2016 NAME: Problem Score Max Score 1 6 2 8 3 9 4 12 5 4 6 13 7 7 8 6 9 9 10 6 11 14 12 6 Total 100 1 of 8 1. [6] (a) [3] What camera setting(s)
More informationReal-time Expression Cloning using Appearance Models
Real-time Expression Cloning using Appearance Models Barry-John Theobald School of Computing Sciences University of East Anglia Norwich, UK bjt@cmp.uea.ac.uk Iain A. Matthews Robotics Institute Carnegie
More informationMultimedia Technology CHAPTER 4. Video and Animation
CHAPTER 4 Video and Animation - Both video and animation give us a sense of motion. They exploit some properties of human eye s ability of viewing pictures. - Motion video is the element of multimedia
More information3D FACE RECONSTRUCTION BASED ON EPIPOLAR GEOMETRY
IJDW Volume 4 Number January-June 202 pp. 45-50 3D FACE RECONSRUCION BASED ON EPIPOLAR GEOMERY aher Khadhraoui, Faouzi Benzarti 2 and Hamid Amiri 3,2,3 Signal, Image Processing and Patterns Recognition
More informationOnline Interactive 4D Character Animation
Online Interactive 4D Character Animation Marco Volino, Peng Huang and Adrian Hilton Web3D 2015 Outline 4D Performance Capture - 3D Reconstruction, Alignment, Texture Maps Animation - Parametric Motion
More informationHigh-Fidelity Facial and Speech Animation for VR HMDs
High-Fidelity Facial and Speech Animation for VR HMDs Institute of Computer Graphics and Algorithms Vienna University of Technology Forecast facial recognition with Head-Mounted Display (HMD) two small
More informationA Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India
A Review of Image- based Rendering Techniques Nisha 1, Vijaya Goel 2 1 Department of computer science, University of Delhi, Delhi, India Keshav Mahavidyalaya, University of Delhi, Delhi, India Abstract
More information3/1/2010. Acceleration Techniques V1.2. Goals. Overview. Based on slides from Celine Loscos (v1.0)
Acceleration Techniques V1.2 Anthony Steed Based on slides from Celine Loscos (v1.0) Goals Although processor can now deal with many polygons (millions), the size of the models for application keeps on
More informationPeripheral drift illusion
Peripheral drift illusion Does it work on other animals? Computer Vision Motion and Optical Flow Many slides adapted from J. Hays, S. Seitz, R. Szeliski, M. Pollefeys, K. Grauman and others Video A video
More informationImage warping/morphing
Image warping/morphing Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Tom Funkhouser and Alexei Efros Image warping Image formation B A Sampling and quantization What
More informationWelcome to CS 231. Topics in Computer Animation. Victor Zordan EBU II 337
Welcome to CS 231 Topics in Computer Animation Victor Zordan EBU II 337 vbz@cs.ucr.edu Overview Administravia Projects History of animation Research Areas About me Mech Engineering(BS) Computer Science
More informationAnimation. CS 465 Lecture 22
Animation CS 465 Lecture 22 Animation Industry production process leading up to animation What animation is How animation works (very generally) Artistic process of animation Further topics in how it works
More information3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images
1 3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images Yudong Guo, Juyong Zhang, Jianfei Cai, Boyi Jiang and Jianmin Zheng arxiv:1708.00980v2 [cs.cv] 11 Sep 2017
More informationNonrigid Surface Modelling. and Fast Recovery. Department of Computer Science and Engineering. Committee: Prof. Leo J. Jia and Prof. K. H.
Nonrigid Surface Modelling and Fast Recovery Zhu Jianke Supervisor: Prof. Michael R. Lyu Committee: Prof. Leo J. Jia and Prof. K. H. Wong Department of Computer Science and Engineering May 11, 2007 1 2
More informationHierarchical Retargetting of Fine Facial Motions
EUROGRAPHICS 2004 / M.-P. Cani and M. Slater (Guest Editors) Volume 23 (2004), Number 3 Hierarchical Retargetting of Fine Facial Motions Kyunggun Na and Moonryul Jung Department of Media Technology, Graduate
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationCHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION
CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION In this chapter we will discuss the process of disparity computation. It plays an important role in our caricature system because all 3D coordinates of nodes
More informationFace Tracking. Synonyms. Definition. Main Body Text. Amit K. Roy-Chowdhury and Yilei Xu. Facial Motion Estimation
Face Tracking Amit K. Roy-Chowdhury and Yilei Xu Department of Electrical Engineering, University of California, Riverside, CA 92521, USA {amitrc,yxu}@ee.ucr.edu Synonyms Facial Motion Estimation Definition
More informationTEXTURE OVERLAY ONTO NON-RIGID SURFACE USING COMMODITY DEPTH CAMERA
TEXTURE OVERLAY ONTO NON-RIGID SURFACE USING COMMODITY DEPTH CAMERA Tomoki Hayashi 1, Francois de Sorbier 1 and Hideo Saito 1 1 Graduate School of Science and Technology, Keio University, 3-14-1 Hiyoshi,
More informationHomework 2 Questions? Animation, Motion Capture, & Inverse Kinematics. Velocity Interpolation. Handing Free Surface with MAC
Homework 2 Questions? Animation, Motion Capture, & Inverse Kinematics Velocity Interpolation Original image from Foster & Metaxas, 1996 In 2D: For each axis, find the 4 closest face velocity samples: Self-intersecting
More informationRENDERING AND ANALYSIS OF FACES USING MULTIPLE IMAGES WITH 3D GEOMETRY. Peter Eisert and Jürgen Rurainsky
RENDERING AND ANALYSIS OF FACES USING MULTIPLE IMAGES WITH 3D GEOMETRY Peter Eisert and Jürgen Rurainsky Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institute Image Processing Department
More informationConsider a partially transparent object that is illuminated with two lights, one visible from each side of the object. Start with a ray from the eye
Ray Tracing What was the rendering equation? Motivate & list the terms. Relate the rendering equation to forward ray tracing. Why is forward ray tracing not good for image formation? What is the difference
More informationMuscle Based facial Modeling. Wei Xu
Muscle Based facial Modeling Wei Xu Facial Modeling Techniques Facial modeling/animation Geometry manipulations Interpolation Parameterizations finite element methods muscle based modeling visual simulation
More informationWhat is Computer Vision? Introduction. We all make mistakes. Why is this hard? What was happening. What do you see? Intro Computer Vision
What is Computer Vision? Trucco and Verri (Text): Computing properties of the 3-D world from one or more digital images Introduction Introduction to Computer Vision CSE 152 Lecture 1 Sockman and Shapiro:
More informationCS 523: Computer Graphics, Spring Shape Modeling. Skeletal deformation. Andrew Nealen, Rutgers, /12/2011 1
CS 523: Computer Graphics, Spring 2011 Shape Modeling Skeletal deformation 4/12/2011 1 Believable character animation Computers games and movies Skeleton: intuitive, low-dimensional subspace Clip courtesy
More informationBlack Desert Online. Taking MMO Development to the Next Level. Dongwook Ha Gwanghyeon Go
Black Desert Online Taking MMO Development to the Next Level Dongwook Ha (dongwook@pearlabyss.com) Gwanghyeon Go (xdotdt@pearlabyss.com) 2018-03-23 Black Desert Online Challenges Massive data and contents
More informationImage Warping and Morphing
Image Warping and Morphing OUTLINE: Image Warping Morphing Beier and Neely s Morphing Method Image Warping Point processing and filtering don t move pixels around. Image warping = rearranging the pixels
More informationVision-based Control of 3D Facial Animation
Eurographics/SIGGRAPH Symposium on Computer Animation (2003) D. Breen, M. Lin (Editors) Vision-based Control of 3D Facial Animation Jin-xiang Chai,1 Jing Xiao1 and Jessica Hodgins1 1 The Robotics Institute,
More informationReal-time Talking Head Driven by Voice and its Application to Communication and Entertainment
ISCA Archive Real-time Talking Head Driven by Voice and its Application to Communication and Entertainment Shigeo MORISHIMA Seikei University ABSTRACT Recently computer can make cyberspace to walk through
More informationWarping and Morphing. Ligang Liu Graphics&Geometric Computing Lab USTC
Warping and Morphing Ligang Liu Graphics&Geometric Computing Lab USTC http://staff.ustc.edu.cn/~lgliu Metamorphosis "transformation of a shape and its visual attributes" Intrinsic in our environment Deformations
More informationPERFORMANCE CAPTURE FROM SPARSE MULTI-VIEW VIDEO
Stefan Krauß, Juliane Hüttl SE, SoSe 2011, HU-Berlin PERFORMANCE CAPTURE FROM SPARSE MULTI-VIEW VIDEO 1 Uses of Motion/Performance Capture movies games, virtual environments biomechanics, sports science,
More information2D Image Morphing using Pixels based Color Transition Methods
2D Image Morphing using Pixels based Color Transition Methods H.B. Kekre Senior Professor, Computer Engineering,MP STME, SVKM S NMIMS University, Mumbai,India Tanuja K. Sarode Asst.Professor, Thadomal
More informationArt Based Rendering of Fur by Instancing Geometry
Art Based Rendering of Fur by Instancing Geometry Abstract Richie Steigerwald In this paper, I describe a non- photorealistic rendering system that uses strokes to render fur and grass in a stylized manner
More informationarxiv: v1 [cs.cv] 29 May 2018
HeadOn: Real-time Reenactment of Human Portrait Videos JUSTUS THIES, Technical University of Munich MICHAEL ZOLLHÖFER, Stanford University CHRISTIAN THEOBALT, Max-Planck-Institute for Informatics MARC
More informationImage-Based Rendering
Image-Based Rendering COS 526, Fall 2016 Thomas Funkhouser Acknowledgments: Dan Aliaga, Marc Levoy, Szymon Rusinkiewicz What is Image-Based Rendering? Definition 1: the use of photographic imagery to overcome
More informationOverview. Video. Overview 4/7/2008. Optical flow. Why estimate motion? Motion estimation: Optical flow. Motion Magnification Colorization.
Overview Video Optical flow Motion Magnification Colorization Lecture 9 Optical flow Motion Magnification Colorization Overview Optical flow Combination of slides from Rick Szeliski, Steve Seitz, Alyosha
More informationAnimated Talking Head With Personalized 3D Head Model
Animated Talking Head With Personalized 3D Head Model L.S.Chen, T.S.Huang - Beckman Institute & CSL University of Illinois, Urbana, IL 61801, USA; lchen@ifp.uiuc.edu Jörn Ostermann, AT&T Labs-Research,
More information3D Face Modelling Under Unconstrained Pose & Illumination
David Bryan Ottawa-Carleton Institute for Biomedical Engineering Department of Systems and Computer Engineering Carleton University January 12, 2009 Agenda Problem Overview 3D Morphable Model Fitting Model
More informationVIDEO FACE BEAUTIFICATION
VIDEO FACE BEAUTIFICATION Yajie Zhao 1, Xinyu Huang 2, Jizhou Gao 1, Alade Tokuta 2, Cha Zhang 3, Ruigang Yang 1 University of Kentucky 1 North Carolina Central University 2 Microsoft Research 3 Lexington,
More informationAAM Based Facial Feature Tracking with Kinect
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 15, No 3 Sofia 2015 Print ISSN: 1311-9702; Online ISSN: 1314-4081 DOI: 10.1515/cait-2015-0046 AAM Based Facial Feature Tracking
More informationNeue Verfahren der Bildverarbeitung auch zur Erfassung von Schäden in Abwasserkanälen?
Neue Verfahren der Bildverarbeitung auch zur Erfassung von Schäden in Abwasserkanälen? Fraunhofer HHI 13.07.2017 1 Fraunhofer-Gesellschaft Fraunhofer is Europe s largest organization for applied research.
More informationGraphics for VEs. Ruth Aylett
Graphics for VEs Ruth Aylett Overview VE Software Graphics for VEs The graphics pipeline Projections Lighting Shading Runtime VR systems Two major parts: initialisation and update loop. Initialisation
More informationImage-Based Modeling and Rendering
Image-Based Modeling and Rendering Richard Szeliski Microsoft Research IPAM Graduate Summer School: Computer Vision July 26, 2013 How far have we come? Light Fields / Lumigraph - 1996 Richard Szeliski
More information