Ping Tan. Simon Fraser University
|
|
- Mitchell McLaughlin
- 5 years ago
- Views:
Transcription
1 Ping Tan Simon Fraser University
2 Photos vs. Videos (live photos) A good photo tells a story Stories are better told in videos
3 Videos in the Mobile Era (mobile & share) More videos are captured by mobile devices Sales of compact cameras have fallen 300 hrs of videos are uploaded to YouTube every minute Cisco predicts videos account for 70% of internet traffic by 2017
4 Challenges in the Mobile Era Mobile: how to produce professional videos with mobile devices? Share: how to create exciting content? the SynthCam app by Marc Levoy [Yu and Gallup 2014]
5 Computational Videography Enhance Video Quality Stabilization Enable Advanced Photography Video defog and stereo TrackCam Auto Fence Removal
6 Pipeline of Video Stabilization Camera Motion Estimation Camera Path Smoothing Re-rendering (by Image Warping) Feature Tracking Camera Path Smoothed Camera Path t t + 1 t + 1 t
7 Digital Video Stabilization Video stabilization techniques can be categorized as: Popular commercial solutions: 2D method [Matsushita et al. PAMI 2006; Grundmann et al. CVPR 2011] 3D method [Liu et al. SIGGRAPH 2009; Liu et al. CVPR 2012; Zhou et al. CVPR 2013] imovie, Apple 2.5D method [Liu et al. TOG 2011; Goldstein and Fattal, TOG 2012] YouTube Stabilizer, Google Relevant rolling shutter correction techniques [Baker et al. CVPR 2010; Karpenko et al. Stanford Tech Report. 2011; Grundmann et al. ICCP 2012] Movie Maker, Microsoft After Effect CS6, Adobe
8 Challenges in Consumer Videos 1. Large depth variation
9 Challenges in Consumer Videos 1. Large depth variation 2. Quick camera motion (rotation, zooming)
10 Challenges in Consumer Videos 1. Large depth variation 2. Quick camera motion (rotation, zooming) 3. Large moving objects
11 Challenges in Consumer Videos 1. Large depth variation 2. Quick camera motion (rotation, zooming) 3. Large moving objects 4. Strong rolling shutter effects
12 Common Artifacts in Stabilized Videos 1. Not stable enough input previous method(virtual dub stabilizer)
13 Common Artifacts in Stabilized Videos 1. Not stable enough input 2. Geometry distortion previous method (YouTube)
14 Common Artifacts in Stabilized Videos 1. Not stable enough input 2. Geometry distortion 3. Cropping previous method (Adobe After Effects)
15 Our Goals Camera Motion Estimation Camera Path Smoothing Re-rendering (by Image Warping) To address the challenges in consumer videos: 1. Large depth variation 2. Quick camera motion 3. Large moving foreground By our novel techniques in: Motion model & estimation Adaptive path smoothing 4. Rolling shutter
16 Contributions Shuaicheng Liu, Lu Yuan, Ping Tan, Jian Sun. SteayFlow: Spatially Smooth Optical Flow for Video Stabilization. IEEE CVPR 2014 Shuaicheng Liu, Lu Yuan, Ping Tan, Jian Sun. Bundled camera paths for video stabilization. ACM SIGGRAPH 2013 Shuaicheng Liu, Yinting Wang, Lu Yuan, Jiajun Bu, Ping Tan, Jian Sun Video Stabilization with a depth camera. IEEE CVPR 2012
17 Contributions Camera Motion Camera Path Re-rendering (by Image Shuaicheng Liu, Lu Yuan, Estimation Ping Tan, Jian Sun. Smoothing Warping) Bundled camera paths for video stabilization. ACM SIGGRAPH 2013 Shuaicheng Liu, Lu Yuan, Ping Tan, Jian Sun. SteayFlow: Spatially Smooth Optical Flow for Video Stabilization. IEEE CVPR 2014 Shuaicheng Liu, Yinting Wang, Lu Yuan, Jiajun Bu, Ping Tan, Jian Sun Video Stabilization with a depth camera. IEEE CVPR 2012
18 Contributions Shuaicheng Liu, Lu Yuan, Ping Tan, Jian Sun. SteayFlow: Spatially Smooth Optical Flow for Video Stabilization. IEEE CVPR 2014 Shuaicheng Liu, Lu Yuan, Ping Tan, Jian Sun. Bundled camera paths for video stabilization. ACM SIGGRAPH 2013 Shuaicheng Liu, Yinting Wang, Lu Yuan, Jiajun Bu, Ping Tan, Jian Sun Video Stabilization with a depth camera. IEEE CVPR 2012 Camera Motion Estimation Camera Path Smoothing Re-rendering (by Image Warping)
19 Contributions Shuaicheng Liu, Lu Yuan, Ping Tan, Jian Sun. SteayFlow: Spatially Smooth Optical Flow for Video Stabilization. IEEE CVPR 2014 Shuaicheng Liu, Lu Yuan, Ping Tan, Jian Sun. Bundled camera paths for video stabilization. ACM SIGGRAPH 2013 Shuaicheng Liu, Yinting Wang, Lu Yuan, Jiajun Bu, Ping Tan, Jian Sun Video Stabilization with a depth camera. IEEE CVPR 2012
20 camera motion estimation Camera Motion Estimation Camera Path Smoothing Re-rendering (by Image Warping) 3D method: [Liu et al. SIGGRAPH 2009] Spatially-variant motion Time-consuming Depends on fragile 3D reconstruction structure from motion
21 camera motion estimation Camera Motion Estimation Camera Path Smoothing Re-rendering (by Image Warping) 2D method: [Matsushita et al. PAMI 2006; Grundmann et al. CVPR 2011] 1 Robust Efficient Homogenous planar motion
22 camera motion estimation Camera Motion Estimation Camera Path Smoothing Re-rendering (by Image Warping) 2.5D method: [Liu et al. TOG 2011; Goldstein and Fattal, TOG 2012] 3D reconstruction 2D feature tracking Spatially-variant motion Feature tracking is fragile to quick camera motion tracking when rotating
23 camera motion estimation Camera Motion Estimation Camera Path Smoothing Re-rendering (by Image Warping) Our Solution: Novel flexible 2D motion model only 2 frames feature correspondence spatially-variant motion
24 our mesh-based motion model conventional single homography our mesh-based motion model Divide the video frame to a 2D regular grid mesh
25 our mesh-based motion model,,,,,,,,,,,,,,,, conventional single homography our mesh-based motion model Divide the video frame to a 2D regular grid mesh Estimate a homography in each cell (now, spatial-variant motion)
26 our mesh-based motion model,,,,,,,,,,,, frame t+1, warped from frame t frame t,,,, Two challenges: Maintain continuity in motion estimation Estimate motion at textureless cells (e.g. in sky)
27 our mesh-based motion model Our solution: frame t frame t + 1 Parameterize by the translations at mesh grid points Estimate all translations by an as-similar-as-possible warping [Igarashi et al. 2005; Liu et al. SIGGRAPH 2009] Estimate at each cell from,,,,,,
28 model estimation Data term: should the same local bilinear coordinates. 2, where 2. frame t frame t+1
29 model estimation Smooth term: should be close to a similarity 0 0
30 comparison with global homography frame t frame t+1 Single homography [Matsushita et al. PAMI 2006] Our method
31 comparison with global homography error = error single homography frame index mesh-based homography
32 comparison with global homography Stabilized with a global homography Stabilized with our method
33 comparison to [Grundmann et al. ICCP 2012] frame t frame t+1 Gaussian smoothness Homography array [Grundmann et al. ICCP 2012] Our method
34 comparison to [Grundmann et al. ICCP 2012] error = error Homography array frame index our method
35 comparison to [Grundmann et al. ICCP 2012] Stabilized by Youtube.com Stabilized with our method
36 camera path smoothing Camera Motion Estimation Camera Path Smoothing Re-rendering (by Image Warping) Low-pass filtering Polynomial curves [Morimoto and Chellappa, ICASSP 1999, Matsushita et al. PAMI 2006] [Chen et al. CG Forum 2008] Piece-wise smoothing L1-norm optimization [Gleicher and Liu, Multimedia 2007] [Grundmann et al. ICCV 2011]
37 camera path t t + 1 t + 2 homographies camera path
38 bundled camera paths
39 adaptive smoothing low-pass smoothing our adaptive smoothing distortion input camera path rapid panning jitters
40 smooth a single path Adaptive smoothing by minimizing: Data term close to original path Smoothness, bilateral weight, temporal range range temporal
41 smooth a single path Adaptive smoothing by minimizing: Iteratively optimized by (according to the Jacobi iterative solver) 1, Initialized as 1,
42 smooth bundled paths + 2 local adaptive path smoothing spatial smoothness 1 1
43 re-rendering.. Input video Input video frame t-1 frame t, Stabilized video Stabilized video frame t-1 frame t..
44 Video Results
45 Video Results
46 + 2 spatial smoothness without spatial constraint with spatial constraint
47 + 2 local path smoothing low-pass local path smoothing adaptive local path smoothing
48 Computational Efficiency CPU: Intel i7 3.2GHz Quad-Core, RAM: 8G 400 ~ 600 SURF features / frame 720P video (resolution: 1280 X 720): 392 ms / frame (~2.5 fps) smooth paths (12 ms) render frame (30 ms) estimate motion (50 ms) extract feature (300 ms)
49 Contributions Shuaicheng Liu, Lu Yuan, Ping Tan, Jian Sun. SteayFlow: Spatially Smooth Optical Flow for Video Stabilization. IEEE CVPR 2014 Shuaicheng Liu, Lu Yuan, Ping Tan, Jian Sun. Bundled camera paths for video stabilization. ACM SIGGRAPH 2013 Shuaicheng Liu, Yinting Wang, Lu Yuan, Jiajun Bu, Ping Tan, Jian Sun Video Stabilization with a depth camera. IEEE CVPR 2012
50 Video Stabilization Pipeline Camera Motion Estimation Camera Path Smoothing Re-rendering (by Image Warping) The bundled path method prefers smaller grid size What if we use 1x1 grid size? optical flow based motion model How to smooth the flow fields?
51 A Naïve Method Obtain feature trajectories from optical flow Smooth feature trajectories Trajectories have irregular shape (which complicates smoothing) Sub-space constraint between trajectories [Liu et al. 2011] a feature trajectory
52 Feature Trajectories vs Pixel Profiles A pixel profile: motion vectors at the same pixel location over time. a feature trajectory a pixel profile Pixel profiles are regular (which simplifies smoothing) Different profiles can be smoothed independently SteayFlow: Spatially Smooth Optical Flow for Video Stabilization. Shuaicheng Liu, Lu Yuan, Ping Tan, Jian Sun. IEEE CVPR 2014
53 Statistics of Pixel Profiles Scene without motion discontinuity feature trajectory pixel profile
54 Statistics of Pixel Profiles Scene with motion discontinuity feature trajectory pixel profile
55 Inpaint Discontinuous Motions input frame optical flow steady-flow motion completion (see paper for more details)
56 Smoothing Pixel Profiles Smooth each pixel profile individually by minimizing: close to original path bilateral weight range temporal
57 Video Results
58 Computational Videography Enhance Video Quality Stabilization Enable Advanced Photography Video defog and stereo TrackCam Auto Fence Removal
59 What is Tracking Shots?
60 How to Take Tracking Shots?
61 Our Solution for Tracking Shots
62 3D Method Object segmentation 3D foreground motion Blur kernels Input video 3D scene reconstruction Result
63 3D Method Object segmentation 3D foreground motion Blur kernels Input video 3D scene reconstruction Result Adobe After Effects RotoBrush
64 3D Method Object segmentation 3D foreground motion Blur kernels Input video 3D scene reconstruction Result
65 3D Method Object segmentation 3D foreground motion Blur kernels Input video 3D scene reconstruction Result
66 3D Method Object segmentation 3D foreground motion Blur kernels Input video 3D scene reconstruction Result foreground motion trajectory virtual cameras
67 3D Method Object segmentation 3D foreground motion Blur kernels Input video 3D scene reconstruction Result
68 3D Method Object segmentation 3D foreground motion Blur kernels Input video 3D scene reconstruction Result
69 3D Method Object segmentation 3D foreground motion Blur kernels Input video 3D scene reconstruction Result Main challenge: recover 3D trajectory of the moving foreground
70 trajectory triangulation Challenge: a static point a moving point? static 3D point dynamic 3D point
71 trajectory triangulation Input: Camera pose (computed according to the static background) A 2D position of foreground object at each frame Output: A 3D position of the foreground object
72 algebraic error constraint the 3D point is projected to by the camera 0,1, :, 0 E, :,
73 linear motion constraint all 3D point roughly form a line 0 0 [Avidan & Shashua 2000] is a 6D vector, Plucker line representation is a 4 4matrix, by rearranging elements in E
74 constant velocity/acceleration constraint the foreground object has near constant velocity/acceleration 2 2 E 2 3 3
75 perspective constraint the foreground s apparent size is proportional to its inverse depth, :,, E 1/ :1/S, is s depth is the foreground s pixel counts is the 3 rd row of [Hartley & Zisserman 2003]
76 final formulation Energy minimization:, Iteratively estimate, applied to overlapping sub-sequences
77 3D results
78 3D results
79 trajectory evaluation without and with perspective constraint without and with constant velocity without and with linear motion
80 Pseudo 3D Method Object segmentation 3D foreground motion Blur kernels Input video 3D scene reconstruction Result 3D scene reconstruction hallucinate background 3D 3D foreground motion hallucinate foreground 3D
81 hallucinate background 3D Principles: faraway points have smaller disparity Algorithm: 1. remove camera rotation by stabilization 2. turn feature disparity to depth directly 2 /
82 hallucinate foreground 3D Principles: faraway object appears smaller Algorithm: turn object size to depth directly γ/s
83 merge foreground and background Hallucinated foreground & background are in different scales. 2 / γ/s We fix and adjust interactively.
84 pseudo 3D examples
85 pseudo 3D examples
86 Evaluation synthetic examples by Maya existing commercial tools a manual tool by user study
87 synthetic examples Hand-held camera Tracking camera rendered in Maya
88 synthetic examples Hand-held Cam Tracking Cam
89 Ground-truth 3D method (PSNR=33.36) Pseudo 3D method (PSNR= 32.28)
90 synthetic examples Hand-held Cam Tracking Cam
91 Ground-truth 3D method (PSNR=34.12) Pseudo 3D method (PSNR= 31.29)
92 synthetic examples Hand-held Cam Tracking Cam
93 Ground-truth 3D method (PSNR = 31.55) Pseudo 3D method (PSNR = 29.48)
94 photoshop blur gallery
95 photoshop blur gallery
96 photoshop blur gallery
97 the Analog Efex 2
98 our manual tool 3x fast
99 User study 3 subjects, each create 20 tracking shots A and B use our manual tool C create use our automatic tool (10 by 3D, 10 by pseudo 3D) 30 viewers: judge the quality Created by A Created by B Created by C
100 User study 3 subjects, each create 20 tracking shots A and B use our manual tool C create use our automatic tool (10 by 3D, 10 by pseudo 3D) 30 viewers: judge the quality Subject A Subject B 3D method 61.8% 90.6% Pseudo 3D method 67.7% 91.2% The numbers are the percentages of viewers who favored our results
101 More Results
102 Summary Enhance Video Quality Stabilization Enable Advanced Photography Video defog and stereo TrackCam Auto Fence Removal
SteadyFlow: Spatially Smooth Optical Flow for Video Stabilization
SteadyFlow: Spatially Smooth Optical Flow for Video Stabilization Shuaicheng Liu 1 Lu Yuan 2 Ping Tan 1 Jian Sun 2 1 National University of Singapore 2 Microsoft Research Abstract We propose a novel motion
More informationVideo Stabilization with a Depth Camera
Video Stabilization with a Depth Camera Shuaicheng Liu 1 Yinting Wang 1 Lu Yuan 2 Ping Tan 1 Jian Sun 2 1 National University of Singapore 2 Microsoft Research Asia Abstract Previous video stabilization
More informationRectangling Panoramic Images via Warping
Rectangling Panoramic Images via Warping Kaiming He Microsoft Research Asia Huiwen Chang Tsinghua University Jian Sun Microsoft Research Asia Introduction Panoramas are irregular Introduction Panoramas
More informationVideo Stabilization with a Depth Camera
Video Stabilization with a Depth Camera Shuaicheng Liu 1 Yinting Wang 1,2 Lu Yuan 3 Jiajun Bu 2 Ping Tan 1 Jian Sun 3 1 National University of Singapore 2 Zhejiang University 3 Microsoft Research Asia
More informationVideo Stabilization by Procrustes Analysis of Trajectories
Video Stabilization by Procrustes Analysis of Trajectories Geethu Miriam Jacob Indian Institute of Technology Chennai, India geethumiriam@gmail.com Sukhendu Das Indian Institute of Technology Chennai,
More informationVIDEO STABILIZATION WITH L1-L2 OPTIMIZATION. Hui Qu, Li Song
VIDEO STABILIZATION WITH L-L2 OPTIMIZATION Hui Qu, Li Song Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University ABSTRACT Digital videos often suffer from undesirable
More informationCS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
More informationDIGITAL VIDEO STABILIZATION LIU SHUAICHENG A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING
DIGITAL VIDEO STABILIZATION LIU SHUAICHENG (M.S. SOC, NUS, 2010) (B.Sc. SICHUAN UNIVERSITY, 2008) A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING
More informationStereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman
Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure
More informationVision Review: Image Formation. Course web page:
Vision Review: Image Formation Course web page: www.cis.udel.edu/~cer/arv September 10, 2002 Announcements Lecture on Thursday will be about Matlab; next Tuesday will be Image Processing The dates some
More informationStructured Light II. Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov
Structured Light II Johannes Köhler Johannes.koehler@dfki.de Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov Introduction Previous lecture: Structured Light I Active Scanning Camera/emitter
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem
More informationCS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
More informationarxiv: v1 [cs.cv] 28 Sep 2018
Camera Pose Estimation from Sequence of Calibrated Images arxiv:1809.11066v1 [cs.cv] 28 Sep 2018 Jacek Komorowski 1 and Przemyslaw Rokita 2 1 Maria Curie-Sklodowska University, Institute of Computer Science,
More information3D Computer Vision. Structured Light II. Prof. Didier Stricker. Kaiserlautern University.
3D Computer Vision Structured Light II Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Introduction
More informationLecture 10: Multi view geometry
Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationStereo vision. Many slides adapted from Steve Seitz
Stereo vision Many slides adapted from Steve Seitz What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape What is
More informationStereo and structured light
Stereo and structured light http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 20 Course announcements Homework 5 is still ongoing. - Make sure
More informationBinocular stereo. Given a calibrated binocular stereo pair, fuse it to produce a depth image. Where does the depth information come from?
Binocular Stereo Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image Where does the depth information come from? Binocular stereo Given a calibrated binocular stereo
More informationContent-Preserving Warps for 3D Video Stabilization
Content-Preserving Warps for 3D Video Stabilization Feng Liu Michael Gleicher University of Wisconsin-Madison Hailin Jin Aseem Agarwala Adobe Systems, Inc. Abstract We describe a technique that transforms
More informationDigital Image Restoration
Digital Image Restoration Blur as a chance and not a nuisance Filip Šroubek sroubekf@utia.cas.cz www.utia.cas.cz Institute of Information Theory and Automation Academy of Sciences of the Czech Republic
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction
More informationWhat have we leaned so far?
What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging What have we learned so far? Image Filtering Image Warping Camera Projection Model Project 2: Panoramic
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 12 130228 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Panoramas, Mosaics, Stitching Two View Geometry
More informationFilter Flow: Supplemental Material
Filter Flow: Supplemental Material Steven M. Seitz University of Washington Simon Baker Microsoft Research We include larger images and a number of additional results obtained using Filter Flow [5]. 1
More informationToday. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography
Computational Photography Matthias Zwicker University of Bern Fall 2009 Today From 2D to 3D using multiple views Introduction Geometry of two views Stereo matching Other applications Multiview geometry
More informationContent-Preserving Warps for 3D Video Stabilization
Content-Preserving Warps for 3D Video Stabilization Feng Liu Michael Gleicher University of Wisconsin-Madison Hailin Jin Aseem Agarwala Adobe Systems, Inc. Abstract We describe a technique that transforms
More informationMulti-view Stereo. Ivo Boyadzhiev CS7670: September 13, 2011
Multi-view Stereo Ivo Boyadzhiev CS7670: September 13, 2011 What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape
More informationAnnouncements. Mosaics. Image Mosaics. How to do it? Basic Procedure Take a sequence of images from the same position =
Announcements Project 2 out today panorama signup help session at end of class Today mosaic recap blending Mosaics Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html
More informationEXAMPLE-BASED MOTION MANIPULATION
EXAMPLE-BASED MOTION MANIPULATION Pin-Ching Su, Hwann-Tzong Chen Chia-Ming Cheng National Tsing Hua University MediaTek ABSTRACT This paper introduces an idea of imitating camera movements and shooting
More informationA Factorization Method for Structure from Planar Motion
A Factorization Method for Structure from Planar Motion Jian Li and Rama Chellappa Center for Automation Research (CfAR) and Department of Electrical and Computer Engineering University of Maryland, College
More information3D Cinematography Principles and Their Applications to
3D Cinematography Principles and Their Applications to Stereoscopic Media Processing Chun-Wei Liu, Tz-Huan Huang, Ming-Hsu Chang, Ken-Yi Lee, Chia-Kai Liang, and Yung-Yu Chuang National Taiwan University
More informationPlane-Based Content-Preserving Warps for Video Stabilization
Plane-Based Content-Preserving Warps for Video Stabilization Zihan Zhou University of Illinois at Urbana-Champaign zzhou7@illinois.edu Hailin Jin Adobe Systems Inc. hljin@adobe.com Yi Ma Microsoft Research
More informationResearch on Evaluation Method of Video Stabilization
International Conference on Advanced Material Science and Environmental Engineering (AMSEE 216) Research on Evaluation Method of Video Stabilization Bin Chen, Jianjun Zhao and i Wang Weapon Science and
More informationStereo. Many slides adapted from Steve Seitz
Stereo Many slides adapted from Steve Seitz Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image image 1 image 2 Dense depth map Binocular stereo Given a calibrated
More informationSingle Image Super-resolution. Slides from Libin Geoffrey Sun and James Hays
Single Image Super-resolution Slides from Libin Geoffrey Sun and James Hays Cs129 Computational Photography James Hays, Brown, fall 2012 Types of Super-resolution Multi-image (sub-pixel registration) Single-image
More informationSubspace Video Stabilization
Subspace Video Stabilization FENG LIU Portland State University MICHAEL GLEICHER University of Wisconsin-Madison and JUE WANG, HAILIN JIN ASEEM AGARWALA Adobe Systems, Inc. We present a robust and efficient
More informationVehicle Dimensions Estimation Scheme Using AAM on Stereoscopic Video
Workshop on Vehicle Retrieval in Surveillance (VRS) in conjunction with 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance Vehicle Dimensions Estimation Scheme Using
More informationEpipolar Geometry and Stereo Vision
CS 1674: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 5, 2016 Announcement Please send me three topics you want me to review next
More informationRendering and Modeling of Transparent Objects. Minglun Gong Dept. of CS, Memorial Univ.
Rendering and Modeling of Transparent Objects Minglun Gong Dept. of CS, Memorial Univ. Capture transparent object appearance Using frequency based environmental matting Reduce number of input images needed
More informationA virtual tour of free viewpoint rendering
A virtual tour of free viewpoint rendering Cédric Verleysen ICTEAM institute, Université catholique de Louvain, Belgium cedric.verleysen@uclouvain.be Organization of the presentation Context Acquisition
More informationSelfie Video Stabilization
Selfie Video Stabilization Jiyang Yu and Ravi Ramamoorthi University of California, San Diego jiy173@eng.ucsd.edu, ravir@cs.ucsd.edu Abstract. We propose a novel algorithm for stabilizing selfie videos.
More informationSemantic Filtering for Video Stabilization
Semantic Filtering for Video Stabilization K. Karageorgos, A. Dimou, A. Axenopoulos, P. Daras Information Technologies Institute CERTH 6th km Harilaou-Thermi, 57001 Thessaloniki, Greece {konstantinkarage,adimou,axenop,daras}@iti.gr
More informationStatic Scene Reconstruction
GPU supported Real-Time Scene Reconstruction with a Single Camera Jan-Michael Frahm, 3D Computer Vision group, University of North Carolina at Chapel Hill Static Scene Reconstruction 1 Capture on campus
More information3D Models from Range Sensors. Gianpaolo Palma
3D Models from Range Sensors Gianpaolo Palma Who Gianpaolo Palma Researcher at Visual Computing Laboratory (ISTI-CNR) Expertise: 3D scanning, Mesh Processing, Computer Graphics E-mail: gianpaolo.palma@isti.cnr.it
More informationJoint Subspace Stabilization for Stereoscopic Video
Joint Subspace Stabilization for Stereoscopic Video Feng iu Portland State University fliu@cs.pdx.edu Yuzhen Niu, Fuzhou University yuzhen@cs.pdx.edu Hailin Jin Adobe esearch hljin@adobe.com Abstract Shaky
More informationStereo Epipolar Geometry for General Cameras. Sanja Fidler CSC420: Intro to Image Understanding 1 / 33
Stereo Epipolar Geometry for General Cameras Sanja Fidler CSC420: Intro to Image Understanding 1 / 33 Stereo Epipolar geometry Case with two cameras with parallel optical axes General case Now this Sanja
More informationFace Recognition At-a-Distance Based on Sparse-Stereo Reconstruction
Face Recognition At-a-Distance Based on Sparse-Stereo Reconstruction Ham Rara, Shireen Elhabian, Asem Ali University of Louisville Louisville, KY {hmrara01,syelha01,amali003}@louisville.edu Mike Miller,
More informationMultiple Motion Scene Reconstruction from Uncalibrated Views
Multiple Motion Scene Reconstruction from Uncalibrated Views Mei Han C & C Research Laboratories NEC USA, Inc. meihan@ccrl.sj.nec.com Takeo Kanade Robotics Institute Carnegie Mellon University tk@cs.cmu.edu
More informationIntrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting R. Maier 1,2, K. Kim 1, D. Cremers 2, J. Kautz 1, M. Nießner 2,3 Fusion Ours 1
More informationWook Kim. 14 September Korea University Computer Graphics Lab.
Wook Kim 14 September 2011 Preview - Seam carving How to choose the pixels to be removed? Remove unnoticeable pixels that blend with their surroundings. Wook, Kim 14 September 2011 # 2 Preview Energy term
More informationAnnouncements. Mosaics. How to do it? Image Mosaics
Announcements Mosaics Project artifact voting Project 2 out today (help session at end of class) http://www.destination36.com/start.htm http://www.vrseattle.com/html/vrview.php?cat_id=&vrs_id=vrs38 Today
More informationMidterm Examination CS 534: Computational Photography
Midterm Examination CS 534: Computational Photography November 3, 2016 NAME: Problem Score Max Score 1 6 2 8 3 9 4 12 5 4 6 13 7 7 8 6 9 9 10 6 11 14 12 6 Total 100 1 of 8 1. [6] (a) [3] What camera setting(s)
More informationImage stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac
Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration
More informationLearning based face hallucination techniques: A survey
Vol. 3 (2014-15) pp. 37-45. : A survey Premitha Premnath K Department of Computer Science & Engineering Vidya Academy of Science & Technology Thrissur - 680501, Kerala, India (email: premithakpnath@gmail.com)
More informationCamera Geometry II. COS 429 Princeton University
Camera Geometry II COS 429 Princeton University Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence
More informationHomographies and RANSAC
Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position
More informationMosaics. Today s Readings
Mosaics VR Seattle: http://www.vrseattle.com/ Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html Today s Readings Szeliski and Shum paper (sections
More informationFinally: Motion and tracking. Motion 4/20/2011. CS 376 Lecture 24 Motion 1. Video. Uses of motion. Motion parallax. Motion field
Finally: Motion and tracking Tracking objects, video analysis, low level motion Motion Wed, April 20 Kristen Grauman UT-Austin Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys, and S. Lazebnik
More informationMotion and Tracking. Andrea Torsello DAIS Università Ca Foscari via Torino 155, Mestre (VE)
Motion and Tracking Andrea Torsello DAIS Università Ca Foscari via Torino 155, 30172 Mestre (VE) Motion Segmentation Segment the video into multiple coherently moving objects Motion and Perceptual Organization
More informationBIL Computer Vision Apr 16, 2014
BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm
More informationComputer Vision Lecture 17
Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester
More informationImage de-fencing using RGB-D data
Image de-fencing using RGB-D data Vikram Voleti IIIT-Hyderabad, India Supervisor: Masters thesis at IIT Kharagpur, India (2013-2014) Prof. Rajiv Ranjan Sahay Associate Professor, Electrical Engineering,
More informationNonrigid Surface Modelling. and Fast Recovery. Department of Computer Science and Engineering. Committee: Prof. Leo J. Jia and Prof. K. H.
Nonrigid Surface Modelling and Fast Recovery Zhu Jianke Supervisor: Prof. Michael R. Lyu Committee: Prof. Leo J. Jia and Prof. K. H. Wong Department of Computer Science and Engineering May 11, 2007 1 2
More informationComputer Vision Lecture 17
Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week
More informationA Versatile Algorithm for Reconstruction of Sports Video Sequences
A Versatile Algorithm for Reconstruction of Sports Video Sequences P.Kannan 1, R.Ramakrishnan 1 Department of Electronics and Communication Panimalar Engineering College, Chennai, India. Department of
More informationImage-based modeling (IBM) and image-based rendering (IBR)
Image-based modeling (IBM) and image-based rendering (IBR) CS 248 - Introduction to Computer Graphics Autumn quarter, 2005 Slides for December 8 lecture The graphics pipeline modeling animation rendering
More information3D Reconstruction with Tango. Ivan Dryanovski, Google Inc.
3D Reconstruction with Tango Ivan Dryanovski, Google Inc. Contents Problem statement and motivation The Tango SDK 3D reconstruction - data structures & algorithms Applications Developer tools Problem formulation
More informationIntroduction à la vision artificielle X
Introduction à la vision artificielle X Jean Ponce Email: ponce@di.ens.fr Web: http://www.di.ens.fr/~ponce Planches après les cours sur : http://www.di.ens.fr/~ponce/introvis/lect10.pptx http://www.di.ens.fr/~ponce/introvis/lect10.pdf
More informationStereo: Disparity and Matching
CS 4495 Computer Vision Aaron Bobick School of Interactive Computing Administrivia PS2 is out. But I was late. So we pushed the due date to Wed Sept 24 th, 11:55pm. There is still *no* grace period. To
More informationChaplin, Modern Times, 1936
Chaplin, Modern Times, 1936 [A Bucket of Water and a Glass Matte: Special Effects in Modern Times; bonus feature on The Criterion Collection set] Multi-view geometry problems Structure: Given projections
More informationImage Based Reconstruction II
Image Based Reconstruction II Qixing Huang Feb. 2 th 2017 Slide Credit: Yasutaka Furukawa Image-Based Geometry Reconstruction Pipeline Last Lecture: Multi-View SFM Multi-View SFM This Lecture: Multi-View
More informationSuper-Resolution. Many slides from Miki Elad Technion Yosi Rubner RTC and more
Super-Resolution Many slides from Mii Elad Technion Yosi Rubner RTC and more 1 Example - Video 53 images, ratio 1:4 2 Example Surveillance 40 images ratio 1:4 3 Example Enhance Mosaics 4 5 Super-Resolution
More informationAugmented and Mixed Reality
Augmented and Mixed Reality Uma Mudenagudi Dept. of Computer Science and Engineering, Indian Institute of Technology Delhi Outline Introduction to Augmented Reality(AR) and Mixed Reality(MR) A Typical
More informationGuided Image Super-Resolution: A New Technique for Photogeometric Super-Resolution in Hybrid 3-D Range Imaging
Guided Image Super-Resolution: A New Technique for Photogeometric Super-Resolution in Hybrid 3-D Range Imaging Florin C. Ghesu 1, Thomas Köhler 1,2, Sven Haase 1, Joachim Hornegger 1,2 04.09.2014 1 Pattern
More informationStructured light 3D reconstruction
Structured light 3D reconstruction Reconstruction pipeline and industrial applications rodola@dsi.unive.it 11/05/2010 3D Reconstruction 3D reconstruction is the process of capturing the shape and appearance
More informationMultiple View Geometry
Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric
More informationPhotometric Stereo with Auto-Radiometric Calibration
Photometric Stereo with Auto-Radiometric Calibration Wiennat Mongkulmann Takahiro Okabe Yoichi Sato Institute of Industrial Science, The University of Tokyo {wiennat,takahiro,ysato} @iis.u-tokyo.ac.jp
More informationSeam-Carving. Michael Rubinstein MIT. and Content-driven Retargeting of Images (and Video) Some slides borrowed from Ariel Shamir and Shai Avidan
Seam-Carving and Content-driven Retargeting of Images (and Video) Michael Rubinstein MIT Some slides borrowed from Ariel Shamir and Shai Avidan Display Devices Content Retargeting PC iphone Page Layout
More information5LSH0 Advanced Topics Video & Analysis
1 Multiview 3D video / Outline 2 Advanced Topics Multimedia Video (5LSH0), Module 02 3D Geometry, 3D Multiview Video Coding & Rendering Peter H.N. de With, Sveta Zinger & Y. Morvan ( p.h.n.de.with@tue.nl
More informationDiscrete Optimization of Ray Potentials for Semantic 3D Reconstruction
Discrete Optimization of Ray Potentials for Semantic 3D Reconstruction Marc Pollefeys Joined work with Nikolay Savinov, Christian Haene, Lubor Ladicky 2 Comparison to Volumetric Fusion Higher-order ray
More informationThere are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...
STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own
More informationImage Base Rendering: An Introduction
Image Base Rendering: An Introduction Cliff Lindsay CS563 Spring 03, WPI 1. Introduction Up to this point, we have focused on showing 3D objects in the form of polygons. This is not the only approach to
More informationMulti-View Stereo for Static and Dynamic Scenes
Multi-View Stereo for Static and Dynamic Scenes Wolfgang Burgard Jan 6, 2010 Main references Yasutaka Furukawa and Jean Ponce, Accurate, Dense and Robust Multi-View Stereopsis, 2007 C.L. Zitnick, S.B.
More informationImage-Based Rendering
Image-Based Rendering COS 526, Fall 2016 Thomas Funkhouser Acknowledgments: Dan Aliaga, Marc Levoy, Szymon Rusinkiewicz What is Image-Based Rendering? Definition 1: the use of photographic imagery to overcome
More informationUrban Scene Segmentation, Recognition and Remodeling. Part III. Jinglu Wang 11/24/2016 ACCV 2016 TUTORIAL
Part III Jinglu Wang Urban Scene Segmentation, Recognition and Remodeling 102 Outline Introduction Related work Approaches Conclusion and future work o o - - ) 11/7/16 103 Introduction Motivation Motivation
More informationTopics to be Covered in the Rest of the Semester. CSci 4968 and 6270 Computational Vision Lecture 15 Overview of Remainder of the Semester
Topics to be Covered in the Rest of the Semester CSci 4968 and 6270 Computational Vision Lecture 15 Overview of Remainder of the Semester Charles Stewart Department of Computer Science Rensselaer Polytechnic
More informationSegmentation and Tracking of Partial Planar Templates
Segmentation and Tracking of Partial Planar Templates Abdelsalam Masoud William Hoff Colorado School of Mines Colorado School of Mines Golden, CO 800 Golden, CO 800 amasoud@mines.edu whoff@mines.edu Abstract
More informationEECS 442 Computer vision. Stereo systems. Stereo vision Rectification Correspondence problem Active stereo vision systems
EECS 442 Computer vision Stereo systems Stereo vision Rectification Correspondence problem Active stereo vision systems Reading: [HZ] Chapter: 11 [FP] Chapter: 11 Stereo vision P p p O 1 O 2 Goal: estimate
More informationSynthesizing Realistic Facial Expressions from Photographs
Synthesizing Realistic Facial Expressions from Photographs 1998 F. Pighin, J Hecker, D. Lischinskiy, R. Szeliskiz and D. H. Salesin University of Washington, The Hebrew University Microsoft Research 1
More informationCS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017
CS 277: Intro to Computer Vision Multiple Views Prof. Adriana Kovashka Universit of Pittsburgh March 4, 27 Plan for toda Affine and projective image transformations Homographies and image mosaics Stereo
More informationAn Improved Image Resizing Approach with Protection of Main Objects
An Improved Image Resizing Approach with Protection of Main Objects Chin-Chen Chang National United University, Miaoli 360, Taiwan. *Corresponding Author: Chun-Ju Chen National United University, Miaoli
More informationLecture 19: Depth Cameras. Visual Computing Systems CMU , Fall 2013
Lecture 19: Depth Cameras Visual Computing Systems Continuing theme: computational photography Cameras capture light, then extensive processing produces the desired image Today: - Capturing scene depth
More informationAdditional Material (electronic only)
Additional Material (electronic only) This additional material contains a presentation of additional capabilities of the system, a discussion of performance and temporal coherence as well as other limitations.
More informationAccurate 3D Face and Body Modeling from a Single Fixed Kinect
Accurate 3D Face and Body Modeling from a Single Fixed Kinect Ruizhe Wang*, Matthias Hernandez*, Jongmoo Choi, Gérard Medioni Computer Vision Lab, IRIS University of Southern California Abstract In this
More information3D Computer Vision. Dense 3D Reconstruction II. Prof. Didier Stricker. Christiano Gava
3D Computer Vision Dense 3D Reconstruction II Prof. Didier Stricker Christiano Gava Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de
More informationPhoto-realistic Renderings for Machines Seong-heum Kim
Photo-realistic Renderings for Machines 20105034 Seong-heum Kim CS580 Student Presentations 2016.04.28 Photo-realistic Renderings for Machines Scene radiances Model descriptions (Light, Shape, Material,
More informationA Non-Linear Filter for Gyroscope-Based Video Stabilization
A Non-Linear Filter for Gyroscope-Based Video Stabilization Steven Bell 1, Alejandro Troccoli 2, and Kari Pulli 2 1 Stanford University, Stanford, CA, USA sebell@stanford.edu 2 NVIDIA Research, Santa Clara,
More informationLecture 9 & 10: Stereo Vision
Lecture 9 & 10: Stereo Vision Professor Fei- Fei Li Stanford Vision Lab 1 What we will learn today? IntroducEon to stereo vision Epipolar geometry: a gentle intro Parallel images Image receficaeon Solving
More information