Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac
|
|
- Todd James
- 5 years ago
- Views:
Transcription
1 Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac
2 Image stitching Stitching = alignment + blending geometrical registration photometric registration
3 Applications of image stitching Video stabilization Video summarization i Video compression Video matting Panorama creation
4 Video summarization
5 Video compression
6 Object removal input video
7 Object removal remove foreground
8 Object removal estimate background
9 Object removal background estimation
10 Panorama creation
11 Why panorama? Are you getting the whole picture? Compact Camera FOV = 50 x 35
12 Why panorama? Are you getting the whole picture? Compact Camera FOV = 50 x 35 Human FOV = 200 x 135
13 Why panorama? Are you getting the whole picture? Compact Camera FOV = 50 x 35 Human FOV = 200 x 135 Panoramic Mosaic = 360 x 180
14 Panorama examples Like HDR, it is a topic of computational photography, seeking ways to build a better camera mostly in software. Most consumer cameras have a panorama mode Mars: Earth:
15 What can be globally aligned? In image stitching, we seek for a matrix to globally warp one image into another. Are any two images of the same scene can be aligned this way? Images captured with the same center of projection A planar scene or far-away scene
16 A pencil of rays contains all views real camera synthetic camera Can generate any synthetic camera view g y y as long as it has the same center of projection!
17 Mosaic as an image reprojection mosaic projection plane The images are reprojected onto a common plane The mosaic is formed on this plane Mosaic is a synthetic wide-angle camera
18 Changing camera center Does it still work? synthetic PP PP1 PP2
19 Planar scene (or a faraway one) PP1 PP3 PP2 PP3 is a projection plane of both centers of projection, so we are OK! This is how big aerial photographs are made
20 Motion models Parametric models as the assumptions on the relation between two images.
21 2D Motion models
22 Motion models Translation Affine Perspective 3D rotation 2 unknowns 6 unknowns 8 unknowns 3 unknowns
23 A case study: cylindrical panorama What if you want a 360 field of view? mosaic projection cylinder
24 Cylindrical panoramas Steps Reproject each image onto a cylinder Blend Output the resulting mosaic
25 Cylindrical panorama 1. Take pictures on a tripod (or handheld) 2. Warp to cylindrical i l coordinate 3. Compute pairwise alignments 4. Fix up the end-to-end alignment 5. Blending 6. Crop the result and import into a viewer It is required to do radial distortion correction for better stitching results!
26 Taking pictures Kaidan panoramic tripod head
27 Translation model Try to align this in PaintShop Pro
28 Where should the synthetic camera be real camera synthetic camera The projection plan of some camera Onto a cylinder
29 Cylindrical projection Adopted from
30 Cylindrical projection
31 Cylindrical projection Adopted from
32 Cylindrical projection unwrapped cylinder y x θ f x
33 Cylindrical projection y x unwrapped cylinder y f θ x z
34 Cylindrical projection y x unwrapped cylinder y f z s=f gives less distortion x
35 Cylindrical reprojection top-down view Focal length the dirty secret Image 384x300 f = 180 (pixels) f = 280 f = 380
36 A simple method for estimating f w p f d Or, you can use other software, such as AutoStich, to help.
37 Input images
38 Cylindrical warping
39 Blending Why blending: parallax, lens distortion, scene motion, exposure difference
40 Blending
41 Blending
42 Blending
43 Assembling the panorama Stitch pairs together, blend, then crop
44 Problem: Drift Error accumulation small errors accumulate over time
45 Problem: Drift (x 1,y 1 ) (x n,y n ) Solution copy of first image add another copy of first image at the end there are a bunch of ways to solve this problem add displacement of (y 1 y n )/(n -1) to each image after the first compute a global warp: y = y + ax run a big optimization problem, incorporating this constraint best solution, but more complicated known as bundle adjustment
46 End-to-end alignment and crop
47 Viewer: panorama example:
48 Viewer: texture mapped model example:
49 Cylindrical panorama 1. Take pictures on a tripod (or handheld) 2. Warp to cylindrical i l coordinate 3. Compute pairwise alignments 4. Fix up the end-to-end alignment 5. Blending 6. Crop the result and import into a viewer
50 Determine pairwise alignment? Feature-based methods: only use feature points to estimate parameters We will study the Recognising panorama paper published in ICCV 2003 Run SIFT (or other feature algorithms) for each Run SIFT (or other feature algorithms) for each image, find feature matches.
51 Determine pairwise alignment p =Mp, where M is a transformation matrix, p and p are feature matches It is possible to use more complicated models such as affine or perspective For example, assume M is a 2x2 matrix x' m11 m12 x y' m 21 m 22 y Find M with the least square error n Mp p' i1 2
52 Determine pairwise alignment x m m x ' ' x m y m x y m m y ' ' y m y m x Overdetermined system ' 0 0 ' y x m y x y x ' x m m y x ' n n n x m m y x ' 0 0 n n n y y x
53 Normal equation Given an overdetermined system Ax b the normal equation is that which minimizes the sum of the square differences between left and right sides A T Ax A T b Why?
54 Normal equation 2 ) ( b Ax x E ) ( b a a m b x a a : : : : : : : 1 m b x : : : n nm n b a a... 1 nxm nequations m variables nxm, n equations, m variables
55 Normal equation m j j m b x a x a 1 1 j j j j j j b x a b x a : : : i m j j ij i m j j ij b x a b x a 1 1 : b Ax m j m j b 1 1 : : : n m j j nj n j j nj b x a b x a 1 1 n i m j ij b x a E 2 2 ) ( b Ax x i i j j ij b x a E 1 1 ) ( b Ax x
56 Normal equation 0 2 Ax E ( x ) b E x 1 i n m aij x j bi 1 j 1 n m 2 aij x j bi ai1 i1 j1 n m n a a x 2 a i1 ij j i1 i1 j1 i1 2 b i 2 T T 0 2( A Ax A b ) x E AA Ax A b T T
57 Normal equation Ax b 2 T Ax b Ax b T T Ax b Ax b T T T x A b Ax b x x T T A A T T Ax b Ax T Ax x T A T b b T b T T T A b x A b T T x b b E x T 2A Ax 2A T b
58 Determine pairwise alignment p =Mp, where M is a transformation matrix, p and p are feature matches For translation model, it is easier. E n ' 2 ' 2 m1 xi xi m2 yi yi i1 0 E m 1 What if the match is false? Avoid impact of outliers.
59 RANSAC RANSAC = Random Sample Consensus An algorithm for robust fitting i of models in the presence of many data outliers Compare to robust statistics Given N data points x i, assume that mjority of them are generated from a model with parameters, try to recover.
60 RANSAC algorithm Run k times: How many times? How big? Smaller is better (1) draw n samples randomly (2) fit parameters with these n samples (3) for each of other N-n points, calculate its distance to the fitted model, count the number of inlier points, c Output t with the largest c How to define? Depends on the problem.
61 How to determine k p: probability of real inliers P: probability of success after k trials P 1 (1 p ) n k n samples are all inliers a failure k log(1 P ) log(1 p ) failure after k trials n p k for P= n
62 Example: line fitting
63 Example: line fitting n=2
64 Model fitting
65 Measure distances
66 Count inliers c=3
67 Another trial c=3
68 The best model c=15
69 RANSAC for Homography
70 RANSAC for Homography
71 RANSAC for Homography
72 Applications of panorama in VFX Background plates Image-based lighting i
73 Troy (image-based lighting) id=2195&page=4
74 Spiderman 2 (background plate)
75
76 Cylindrical projection Map 3D point (X,Y,Z) onto a unit cylinder Y Z X unit cylinder Convert to cylindrical coordinates Convert to cylindrical image coordinates unwrapped cylinder cylindrical image
77 3D 2D perspective projection (X c,y c c,,z c c) ) f u c u
78 Reference Richard Szeliski, Image Alignment and Stitching, unpublished draft, R. Szeliski and H.-Y. Shum. Creating full view panoramic image mosaics and texture-mapped models, SIGGRAPH 1997, pp M. Brown, D. G. Lowe, Recognising Panoramas, ICCV 2003.
79 Direct vs feature-based Direct methods use all information and can be very accurate, but they depend on the fragile brightness constancy assumption Iterative approaches require initialization Not robust to illumination change and noise images In early days, direct method is better. Feature based methods are now more robust and potentially faster Even better, it can recognize panorama without initialization
80 TODO Bundle adjustment LM method Direct method vs feature-based method Frame-rate rate image alignment for stabilization Rick s CGA 1995 paper? LM method
81 Project #2 Image stitching camera availability Ti Tripod? rses/cse590ss/currentqtr/projects.htm norama.html
82 blending Alpha-blending Photomontage Poisson blending Adelson s pyramid blending Hdr?
83 3D interpretation p t R K u p Y X t r r r t r r r f f x Z Y t r r r t r r r f z y u
84 Cylindrical warping Given focal length f and image center (x c,y c ) (X,Y,Z) Y Y (sin,h,cos) Z X
85 Cylindrical projection Map 3D point (X,Y,Z) onto cylinder Y Z X unit cylinder Convert to cylindrical coordinates Convert to cylindrical image coordinates unwrapped cylinder cylindrical image
86 Cylindrical reprojection How to map from a cylinder to a planar image? Apply camera projection matrix w = image width, h = image height Y Z X side view Convert to image coordinates divide by third coordinate (w) top-down view image coords
87 Cylindrical projection Y Z X unit cylinder
88 Levenberg-Marquardt Method
89 Alignment a rotation of the camera is a translation of the cylinder! cylinder! y x I y x J I I I I 2 ), ( ), ( y y x x y y x y x y x y x x y x I y x J I y x I y x J I v u I I I I I I, 2,, ), ( ), ( ), ( ), ( y x y x y x,,,
90 LucasKanadeStep void LucasKanadeStep(CByteImage& img1, CByteImage& img2, float t[2]) { // Transform the image Translation(img2, img2t, t); // Compute the gradients and summed error by comparing img1 and img2t double A[2][2], b[2]; for (int y = 1; y < height-1; y++) { // ignore borders for (int x = 1; x < width-1; x++) { // If both have full alphas, then compute and accumulate the error double e = img2t.pixel(x, y, k) - img1.pixel (x, y, k); // Accumulate the matrix entries double gx = 0.5*(img2t.Pixel(x+1, ( y, k) -img2t.pixel(x-1, y, k)); double gy = 0.5*(img2t.Pixel(x, y+1, k) - img2t.pixel(x, y-1, k)); A[0][0] ] += gx*gx; g A[0][1] ] += gx*gy; gy; A[1][0] += gx*gy; A[1][1] += gy*gy; } } b[0] += e*gx; b[1] += e*gy;
91 LucasKanadeStep (cont.) // Solve for the update At=b and update the vector double det = 1.0 / (A[0][0]*A[1][1] - A[1][0]*A[1][0]); } t[0] += (A[1][1]*b[0] - A[1][0]*b[1]) * det; t[1] += (A[0][0]*b[1] - A[1][0]*b[0]) * det;
92 PyramidLucasKanade void PyramidalLucasKanade(CByteImage& img1, CByteImage& img2, float t[2], int nlevels, int nlucaskanadesteps) { CBytePyramid p1(img1); // Form the two pyramids CBytePyramid p2(img2); // Process in a coarse-to-fine hierarchy for (int l = nlevels-1; l >= 0; l--) { t[0] /= (1 << l); // scale the t vector t[1] /= (1 << l); CByteImage& i1 = p1[l]; CByteImage& i2 = p2[l]; } } for (int k = 0; k < nlucaskanadesteps; k++) LucasKanadeStep(i1, i2, t); t[0] *= (1 << l); // restore the full scaling t[1] *= (1 << l);
93 Gaussian pyramid
94 2D Motion models translation: x = x + t x = (x,y) rotation: x = R x + t similarity: affine: x = s R x + t x = A x + t perspective: x H x x = (x,y,1) y (x is a homogeneous coordinate) These all form a nested group (closed under These all form a nested group (closed under composition w/ inv.)
95 Video matting alpha matte
96 Recognising Panoramas 1D Rotations () Ordering matching images
97 Recognising Panoramas 1D Rotations () Ordering matching images
98 Recognising Panoramas 1D Rotations () Ordering matching images
99 Recognising Panoramas 1D Rotations () Ordering matching images 2D Rotations (q, f) Ordering matching images
100 Recognising Panoramas 1D Rotations () Ordering matching images 2D Rotations (q, f) Ordering matching images
101 Recognising Panoramas 1D Rotations () Ordering matching images 2D Rotations (q, f) Ordering matching images
102 Probabilistic model for verification Compare probability that this set of RANSAC inliers/outliers was generated by a correct/false image match Choosing values for p 1, p 0 and p min
103 Recognising Panoramas
104 Overview SIFT Feature Matching Image Matching Bundle Adjustment Multi-band Blending
105 Nearest Neighbour Matching Find k-nn for each feature k number of overlapping images (we use k = 4) Use k-d tree k-d tree recursively bi-partitions data at mean in the dimension of maximum variance Approximate nearest neighbours found in O(nlogn)
106 Overview SIFT Feature Matching Image Matching For each image, use RANSAC to select inlier features from 6 images with most feature matches Bundle Adjustment Multi-band Blending
107 Finding the panoramas
108 Finding the panoramas
109 Finding the panoramas
110 Finding the panoramas
111 Overview SIFT Feature Matching Image Matching Bundle Adjustment Multi-band Blending
112 Homography for Rotation Parameterise each camera by rotation and focal length This gives pairwise homographies
113 Error function Sum of squared projection errors n = #images I(i) = set of image matches to image i F(i, j) = set of feature matches between images i,j r ijk = residual of k th feature match between images i,j Robust error function
114 Overview SIFT Feature Matching Image Matching Bundle Adjustment Multi-band Blending
115 Multi-band Blending Burt & Adelson 1983 Blend frequency bands over range
116 2-band Blending Low frequency ( > 2 pixels) High frequency ( < 2 pixels)
117 Linear Blending
118 2-band Blending
119 Results
120 Distortion No distortion Pin cushion Barrel Radial distortion of the image Caused by imperfect lenses Deviations are most noticeable for rays that pass through the edge of the lens
121 Radial correction Correct for bending in wide field of view lenses
Image stitching. Announcements. Outline. Image stitching
Announcements Image stitching Project #1 was due yesterday. Project #2 handout will be available on the web later tomorrow. I will set up a webpage for artifact voting soon. Digital Visual Effects, Spring
More informationImage stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac
Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration
More informationMosaics. Today s Readings
Mosaics VR Seattle: http://www.vrseattle.com/ Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html Today s Readings Szeliski and Shum paper (sections
More informationToday s lecture. Image Alignment and Stitching. Readings. Motion models
Today s lecture Image Alignment and Stitching Computer Vision CSE576, Spring 2005 Richard Szeliski Image alignment and stitching motion models cylindrical and spherical warping point-based alignment global
More informationFeature Matching and RANSAC
Feature Matching and RANSAC Recognising Panoramas. [M. Brown and D. Lowe,ICCV 2003] [Brown, Szeliski, Winder, CVPR 2005] with a lot of slides stolen from Steve Seitz, Rick Szeliski, A. Efros Introduction
More informationObject Recognition with Invariant Features
Object Recognition with Invariant Features Definition: Identify objects or scenes and determine their pose and model parameters Applications Industrial automation and inspection Mobile robots, toys, user
More informationImage Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi
Image Stitching Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Combine two or more overlapping images to make one larger image Add example Slide credit: Vaibhav Vaish
More informationAnnouncements. Mosaics. How to do it? Image Mosaics
Announcements Mosaics Project artifact voting Project 2 out today (help session at end of class) http://www.destination36.com/start.htm http://www.vrseattle.com/html/vrview.php?cat_id=&vrs_id=vrs38 Today
More informationAnnouncements. Mosaics. Image Mosaics. How to do it? Basic Procedure Take a sequence of images from the same position =
Announcements Project 2 out today panorama signup help session at end of class Today mosaic recap blending Mosaics Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html
More informationCS6670: Computer Vision
CS6670: Computer Vision Noah Snavely Lecture 7: Image Alignment and Panoramas What s inside your fridge? http://www.cs.washington.edu/education/courses/cse590ss/01wi/ Projection matrix intrinsics projection
More informationMore Mosaic Madness. CS194: Image Manipulation & Computational Photography. Steve Seitz and Rick Szeliski. Jeffrey Martin (jeffrey-martin.
More Mosaic Madness Jeffrey Martin (jeffrey-martin.com) CS194: Image Manipulation & Computational Photography with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 2018 Steve Seitz and Rick
More information6.098 Digital and Computational Photography Advanced Computational Photography. Panoramas. Bill Freeman Frédo Durand MIT - EECS
6.098 Digital and Computational Photography 6.882 Advanced Computational Photography Panoramas Bill Freeman Frédo Durand MIT - EECS Lots of slides stolen from Alyosha Efros, who stole them from Steve Seitz
More informationImage-based Modeling and Rendering: 8. Image Transformation and Panorama
Image-based Modeling and Rendering: 8. Image Transformation and Panorama I-Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung Univ, Taiwan Outline Image transformation How to represent the
More informationPanoramas. Why Mosaic? Why Mosaic? Mosaics: stitching images together. Why Mosaic? Olivier Gondry. Bill Freeman Frédo Durand MIT - EECS
Olivier Gondry 6.098 Digital and Computational Photography 6.882 Advanced Computational Photography Panoramas Director of music video and commercial Special effect specialist (Morphing, rotoscoping) Today
More informationImage Warping and Mosacing
Image Warping and Mosacing 15-463: Rendering and Image Processing Alexei Efros with a lot of slides stolen from Steve Seitz and Rick Szeliski Today Mosacs Image Warping Homographies Programming Assignment
More informationImage warping and stitching
Image warping and stitching May 4 th, 2017 Yong Jae Lee UC Davis Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 2 Alignment problem In alignment, we will
More informationImage warping and stitching
Image warping and stitching May 5 th, 2015 Yong Jae Lee UC Davis PS2 due next Friday Announcements 2 Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 3 Alignment
More informationCSE 527: Introduction to Computer Vision
CSE 527: Introduction to Computer Vision Week 5 - Class 1: Matching, Stitching, Registration September 26th, 2017 ??? Recap Today Feature Matching Image Alignment Panoramas HW2! Feature Matches Feature
More informationHomographies and RANSAC
Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position
More informationChapter 3 Image Registration. Chapter 3 Image Registration
Chapter 3 Image Registration Distributed Algorithms for Introduction (1) Definition: Image Registration Input: 2 images of the same scene but taken from different perspectives Goal: Identify transformation
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 12 130228 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Panoramas, Mosaics, Stitching Two View Geometry
More informationMidterm Examination CS 534: Computational Photography
Midterm Examination CS 534: Computational Photography November 3, 2016 NAME: Problem Score Max Score 1 6 2 8 3 9 4 12 5 4 6 13 7 7 8 6 9 9 10 6 11 14 12 6 Total 100 1 of 8 1. [6] (a) [3] What camera setting(s)
More informationImage warping and stitching
Image warping and stitching Thurs Oct 15 Last time Feature-based alignment 2D transformations Affine fit RANSAC 1 Robust feature-based alignment Extract features Compute putative matches Loop: Hypothesize
More informationAutomatic Image Alignment
Automatic Image Alignment Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 Live Homography DEMO Check out panoramio.com
More informationAutomatic Image Alignment
Automatic Image Alignment with a lot of slides stolen from Steve Seitz and Rick Szeliski Mike Nese CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2018 Live Homography
More information360 Full View Spherical Mosaic
360 Full View Spherical Mosaic Huang Wenfan Huang Yehui Rong Nan U017865B U017844X U018274R Objective Full spherical mosaic 360 x 180. All images are taken with camera mounted on a tripod. Registration
More informationTargil 10 : Why Mosaic? Why is this a challenge? Exposure differences Scene illumination Miss-registration Moving objects
Why Mosaic? Are you getting the whole picture? Compact Camera FOV = 5 x 35 Targil : Panoramas - Stitching and Blending Some slides from Alexei Efros 2 Slide from Brown & Lowe Why Mosaic? Are you getting
More informationAgenda. Rotations. Camera models. Camera calibration. Homographies
Agenda Rotations Camera models Camera calibration Homographies D Rotations R Y = Z r r r r r r r r r Y Z Think of as change of basis where ri = r(i,:) are orthonormal basis vectors r rotated coordinate
More informationAgenda. Rotations. Camera calibration. Homography. Ransac
Agenda Rotations Camera calibration Homography Ransac Geometric Transformations y x Transformation Matrix # DoF Preserves Icon translation rigid (Euclidean) similarity affine projective h I t h R t h sr
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253
Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine
More informationCS4670: Computer Vision
CS467: Computer Vision Noah Snavely Lecture 13: Projection, Part 2 Perspective study of a vase by Paolo Uccello Szeliski 2.1.3-2.1.6 Reading Announcements Project 2a due Friday, 8:59pm Project 2b out Friday
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 10 130221 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Canny Edge Detector Hough Transform Feature-Based
More informationPanoramic Image Stitching
Mcgill University Panoramic Image Stitching by Kai Wang Pengbo Li A report submitted in fulfillment for the COMP 558 Final project in the Faculty of Computer Science April 2013 Mcgill University Abstract
More informationAutomatic Panoramic Image Stitching. Dr. Matthew Brown, University of Bath
Automatic Panoramic Image Stitching Dr. Matthew Brown, University of Bath Automatic 2D Stitching The old days: panoramic stitchers were limited to a 1-D sweep 2005: 2-D stitchers use object recognition
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263
Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine
More informationRecap. DoF Constraint Solver. translation. affine. homography. 3D rotation
Image Blending Recap DoF Constraint Solver translation affine homography 3D rotation Recap DoF Constraint Solver translation 2 affine homography 3D rotation Recap DoF Constraint Solver translation 2 affine
More informationCamera Calibration. COS 429 Princeton University
Camera Calibration COS 429 Princeton University Point Correspondences What can you figure out from point correspondences? Noah Snavely Point Correspondences X 1 X 4 X 3 X 2 X 5 X 6 X 7 p 1,1 p 1,2 p 1,3
More informationImage Warping. Many slides from Alyosha Efros + Steve Seitz. Photo by Sean Carroll
Image Warping Man slides from Alosha Efros + Steve Seitz Photo b Sean Carroll Morphing Blend from one object to other with a series of local transformations Image Transformations image filtering: change
More informationN-Views (1) Homographies and Projection
CS 4495 Computer Vision N-Views (1) Homographies and Projection Aaron Bobick School of Interactive Computing Administrivia PS 2: Get SDD and Normalized Correlation working for a given windows size say
More informationCS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry
CS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry Some material taken from: David Lowe, UBC Jiri Matas, CMP Prague http://cmp.felk.cvut.cz/~matas/papers/presentations/matas_beyondransac_cvprac05.ppt
More informationGlobal Flow Estimation. Lecture 9
Motion Models Image Transformations to relate two images 3D Rigid motion Perspective & Orthographic Transformation Planar Scene Assumption Transformations Translation Rotation Rigid Affine Homography Pseudo
More informationCSE 252B: Computer Vision II
CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribes: Jeremy Pollock and Neil Alldrin LECTURE 14 Robust Feature Matching 14.1. Introduction Last lecture we learned how to find interest points
More informationImage Based Rendering. D.A. Forsyth, with slides from John Hart
Image Based Rendering D.A. Forsyth, with slides from John Hart Topics Mosaics translating cameras reveal extra information, break occlusion Optical flow for very small movements of the camera Explicit
More informationImage Features: Local Descriptors. Sanja Fidler CSC420: Intro to Image Understanding 1/ 58
Image Features: Local Descriptors Sanja Fidler CSC420: Intro to Image Understanding 1/ 58 [Source: K. Grauman] Sanja Fidler CSC420: Intro to Image Understanding 2/ 58 Local Features Detection: Identify
More informationFeature Based Registration - Image Alignment
Feature Based Registration - Image Alignment Image Registration Image registration is the process of estimating an optimal transformation between two or more images. Many slides from Alexei Efros http://graphics.cs.cmu.edu/courses/15-463/2007_fall/463.html
More informationStitching and Blending
Stitching and Blending Kari Pulli VP Computational Imaging Light First project Build your own (basic) programs panorama HDR (really, exposure fusion) The key components register images so their features
More informationInstance-level recognition
Instance-level recognition 1) Local invariant features 2) Matching and recognition with local features 3) Efficient visual search 4) Very large scale indexing Matching of descriptors Matching and 3D reconstruction
More informationIntroduction to Image Processing and Computer Vision. -- Panoramas and Blending --
Introduction to Image Processing and Computer Vision -- Panoramas and Blending -- Winter 2013/14 Ivo Ihrke Panoramas Mosaics and Panoramas - Outline - Perspective Panoramas - Hardware-Based - Software-Based
More informationVideo Mosaics for Virtual Environments, R. Szeliski. Review by: Christopher Rasmussen
Video Mosaics for Virtual Environments, R. Szeliski Review by: Christopher Rasmussen September 19, 2002 Announcements Homework due by midnight Next homework will be assigned Tuesday, due following Tuesday.
More informationOverview. Augmented reality and applications Marker-based augmented reality. Camera model. Binary markers Textured planar markers
Augmented reality Overview Augmented reality and applications Marker-based augmented reality Binary markers Textured planar markers Camera model Homography Direct Linear Transformation What is augmented
More informationGlobal Flow Estimation. Lecture 9
Global Flow Estimation Lecture 9 Global Motion Estimate motion using all pixels in the image. Parametric flow gives an equation, which describes optical flow for each pixel. Affine Projective Global motion
More informationInstance-level recognition
Instance-level recognition 1) Local invariant features 2) Matching and recognition with local features 3) Efficient visual search 4) Very large scale indexing Matching of descriptors Matching and 3D reconstruction
More informationAutomatic Image Alignment (feature-based)
Automatic Image Alignment (feature-based) Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2006 Today s lecture Feature
More informationScene Modeling for a Single View
Scene Modeling for a Single View René MAGRITTE Portrait d'edward James with a lot of slides stolen from Steve Seitz and David Brogan, 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Classes
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational
More informationFeature Detectors and Descriptors: Corners, Lines, etc.
Feature Detectors and Descriptors: Corners, Lines, etc. Edges vs. Corners Edges = maxima in intensity gradient Edges vs. Corners Corners = lots of variation in direction of gradient in a small neighborhood
More informationAutomatic Image Alignment (direct) with a lot of slides stolen from Steve Seitz and Rick Szeliski
Automatic Image Alignment (direct) with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Today Go over Midterm Go over Project #3
More informationCS4670: Computer Vision
CS4670: Computer Vision Noah Snavely Lecture 9: Image alignment http://www.wired.com/gadgetlab/2010/07/camera-software-lets-you-see-into-the-past/ Szeliski: Chapter 6.1 Reading All 2D Linear Transformations
More informationMultiview Stereo COSC450. Lecture 8
Multiview Stereo COSC450 Lecture 8 Stereo Vision So Far Stereo and epipolar geometry Fundamental matrix captures geometry 8-point algorithm Essential matrix with calibrated cameras 5-point algorithm Intersect
More informationCS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017
CS 277: Intro to Computer Vision Multiple Views Prof. Adriana Kovashka Universit of Pittsburgh March 4, 27 Plan for toda Affine and projective image transformations Homographies and image mosaics Stereo
More informationCS664 Lecture #16: Image registration, robust statistics, motion
CS664 Lecture #16: Image registration, robust statistics, motion Some material taken from: Alyosha Efros, CMU http://www.cs.cmu.edu/~efros Xenios Papademetris http://noodle.med.yale.edu/~papad/various/papademetris_image_registration.p
More informationScene Modeling for a Single View
Scene Modeling for a Single View René MAGRITTE Portrait d'edward James CS194: Image Manipulation & Computational Photography with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 2014 Steve
More informationPhoto Tourism: Exploring Photo Collections in 3D
Photo Tourism: Exploring Photo Collections in 3D SIGGRAPH 2006 Noah Snavely Steven M. Seitz University of Washington Richard Szeliski Microsoft Research 2006 2006 Noah Snavely Noah Snavely Reproduced with
More informationDense Image-based Motion Estimation Algorithms & Optical Flow
Dense mage-based Motion Estimation Algorithms & Optical Flow Video A video is a sequence of frames captured at different times The video data is a function of v time (t) v space (x,y) ntroduction to motion
More informationDD2423 Image Analysis and Computer Vision IMAGE FORMATION. Computational Vision and Active Perception School of Computer Science and Communication
DD2423 Image Analysis and Computer Vision IMAGE FORMATION Mårten Björkman Computational Vision and Active Perception School of Computer Science and Communication November 8, 2013 1 Image formation Goal:
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationComputer Vision. Recap: Smoothing with a Gaussian. Recap: Effect of σ on derivatives. Computer Science Tripos Part II. Dr Christopher Town
Recap: Smoothing with a Gaussian Computer Vision Computer Science Tripos Part II Dr Christopher Town Recall: parameter σ is the scale / width / spread of the Gaussian kernel, and controls the amount of
More informationWarping, Morphing and Mosaics
Computational Photograph and Video: Warping, Morphing and Mosaics Prof. Marc Pollefes Dr. Gabriel Brostow Toda s schedule Last week s recap Warping Morphing Mosaics Toda s schedule Last week s recap Warping
More informationInvariant Features from Interest Point Groups
Invariant Features from Interest Point Groups Matthew Brown and David Lowe {mbrown lowe}@cs.ubc.ca Department of Computer Science, University of British Columbia, Vancouver, Canada. Abstract This paper
More informationCOSC579: Scene Geometry. Jeremy Bolton, PhD Assistant Teaching Professor
COSC579: Scene Geometry Jeremy Bolton, PhD Assistant Teaching Professor Overview Linear Algebra Review Homogeneous vs non-homogeneous representations Projections and Transformations Scene Geometry The
More informationFinal Exam Study Guide
Final Exam Study Guide Exam Window: 28th April, 12:00am EST to 30th April, 11:59pm EST Description As indicated in class the goal of the exam is to encourage you to review the material from the course.
More information1-2 Feature-Based Image Mosaicing
MVA'98 IAPR Workshop on Machine Vision Applications, Nov. 17-19, 1998, Makuhari, Chibq Japan 1-2 Feature-Based Image Mosaicing Naoki Chiba, Hiroshi Kano, Minoru Higashihara, Masashi Yasuda, and Masato
More informationHomework #4: Creating Panoramas
Homework #4: Creating Panoramas Assigned: March 21 Due: Wednesday, April 6 The goal of this assignment is to write a simple photo panorama creator. You will take four or more photographs and create a panoramic
More informationImage correspondences and structure from motion
Image correspondences and structure from motion http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 20 Course announcements Homework 5 posted.
More informationLocal Feature Detectors
Local Feature Detectors Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Slides adapted from Cordelia Schmid and David Lowe, CVPR 2003 Tutorial, Matthew Brown,
More informationIntroduction to Computer Vision
Introduction to Computer Vision Michael J. Black Nov 2009 Perspective projection and affine motion Goals Today Perspective projection 3D motion Wed Projects Friday Regularization and robust statistics
More informationOptical flow and tracking
EECS 442 Computer vision Optical flow and tracking Intro Optical flow and feature tracking Lucas-Kanade algorithm Motion segmentation Segments of this lectures are courtesy of Profs S. Lazebnik S. Seitz,
More informationImage-Based Rendering and Modeling. IBR Approaches for View Synthesis
Image-Based Rendering and Modeling l Image-based rendering (IBR): A scene is represented as a collection of images l 3D model-based rendering (MBR): A scene is represented by a 3D model plus texture maps
More informationFeature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve Seitz, Rick Szeliski,
More informationLecture 16: Computer Vision
CS4442/9542b: Artificial Intelligence II Prof. Olga Veksler Lecture 16: Computer Vision Motion Slides are from Steve Seitz (UW), David Jacobs (UMD) Outline Motion Estimation Motion Field Optical Flow Field
More informationImage Stitching using Harris Feature Detection
Image Stitching using Harris Feature Detection Shreyas Mistry 1, Prof. Arpita Patel 2 1M. Tech. Student, Electronics & Communication, Chandubhai S. Patel Institute of Technology 2Assi. Professor, Electronics
More informationarxiv: v1 [cs.cv] 28 Sep 2018
Camera Pose Estimation from Sequence of Calibrated Images arxiv:1809.11066v1 [cs.cv] 28 Sep 2018 Jacek Komorowski 1 and Przemyslaw Rokita 2 1 Maria Curie-Sklodowska University, Institute of Computer Science,
More informationE27 Computer Vision - Final Project: Creating Panoramas David Nahmias, Dan Spagnolo, Vincent Stigliani Professor Zucker Due 5/10/13
E27 Computer Vision - Final Project: Creating Panoramas David Nahmias, Dan Spagnolo, Vincent Stigliani Professor Zucker Due 5/10/13 Sources Brown, M.; Lowe, D.G., "Recognising panoramas," Computer Vision,
More informationUncertainties: Representation and Propagation & Line Extraction from Range data
41 Uncertainties: Representation and Propagation & Line Extraction from Range data 42 Uncertainty Representation Section 4.1.3 of the book Sensing in the real world is always uncertain How can uncertainty
More informationFeature-Based Image Mosaicing
Systems and Computers in Japan, Vol. 31, No. 7, 2000 Translated from Denshi Joho Tsushin Gakkai Ronbunshi, Vol. J82-D-II, No. 10, October 1999, pp. 1581 1589 Feature-Based Image Mosaicing Naoki Chiba and
More informationFeature Tracking and Optical Flow
Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who 1 in turn adapted slides from Steve Seitz, Rick Szeliski,
More informationStep-by-Step Model Buidling
Step-by-Step Model Buidling Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure
More informationBroad field that includes low-level operations as well as complex high-level algorithms
Image processing About Broad field that includes low-level operations as well as complex high-level algorithms Low-level image processing Computer vision Computational photography Several procedures and
More informationImage-Based Modeling and Rendering
Image-Based Modeling and Rendering Richard Szeliski Microsoft Research IPAM Graduate Summer School: Computer Vision July 26, 2013 How far have we come? Light Fields / Lumigraph - 1996 Richard Szeliski
More informationVisual Odometry. Features, Tracking, Essential Matrix, and RANSAC. Stephan Weiss Computer Vision Group NASA-JPL / CalTech
Visual Odometry Features, Tracking, Essential Matrix, and RANSAC Stephan Weiss Computer Vision Group NASA-JPL / CalTech Stephan.Weiss@ieee.org (c) 2013. Government sponsorship acknowledged. Outline The
More informationScene Modeling for a Single View
on to 3D Scene Modeling for a Single View We want real 3D scene walk-throughs: rotation translation Can we do it from a single photograph? Reading: A. Criminisi, I. Reid and A. Zisserman, Single View Metrology
More informationIntroduction to Computer Vision
Introduction to Computer Vision Michael J. Black Oct 2009 Motion estimation Goals Motion estimation Affine flow Optimization Large motions Why affine? Monday dense, smooth motion and regularization. Robust
More informationLucas-Kanade Image Registration Using Camera Parameters
Lucas-Kanade Image Registration Using Camera Parameters Sunghyun Cho a, Hojin Cho a, Yu-Wing Tai b, Young Su Moon c, Junguk Cho c, Shihwa Lee c, and Seungyong Lee a a POSTECH, Pohang, Korea b KAIST, Daejeon,
More informationSingle-view 3D Reconstruction
Single-view 3D Reconstruction 10/12/17 Computational Photography Derek Hoiem, University of Illinois Some slides from Alyosha Efros, Steve Seitz Notes about Project 4 (Image-based Lighting) You can work
More informationProf. Feng Liu. Spring /26/2017
Prof. Feng Liu Spring 2017 http://www.cs.pdx.edu/~fliu/courses/cs510/ 04/26/2017 Last Time Re-lighting HDR 2 Today Panorama Overview Feature detection Mid-term project presentation Not real mid-term 6
More informationFAST-MATCH: FAST AFFINE TEMPLATE MATCHING
Seminar on Sublinear Time Algorithms FAST-MATCH: FAST AFFINE TEMPLATE MATCHING KORMAN, S., REICHMAN, D., TSUR, G., & AVIDAN, S., 2013 Given by: Shira Faigenbaum-Golovin Tel-Aviv University 27.12.2015 Problem
More informationOverview. Video. Overview 4/7/2008. Optical flow. Why estimate motion? Motion estimation: Optical flow. Motion Magnification Colorization.
Overview Video Optical flow Motion Magnification Colorization Lecture 9 Optical flow Motion Magnification Colorization Overview Optical flow Combination of slides from Rick Szeliski, Steve Seitz, Alyosha
More informationLecture 16: Computer Vision
CS442/542b: Artificial ntelligence Prof. Olga Veksler Lecture 16: Computer Vision Motion Slides are from Steve Seitz (UW), David Jacobs (UMD) Outline Motion Estimation Motion Field Optical Flow Field Methods
More informationSingle View Geometry. Camera model & Orientation + Position estimation. What am I?
Single View Geometry Camera model & Orientation + Position estimation What am I? Vanishing point Mapping from 3D to 2D Point & Line Goal: Point Homogeneous coordinates represent coordinates in 2 dimensions
More informationMultiple Views Geometry
Multiple Views Geometry Subhashis Banerjee Dept. Computer Science and Engineering IIT Delhi email: suban@cse.iitd.ac.in January 2, 28 Epipolar geometry Fundamental geometric relationship between two perspective
More information