Homographies and RANSAC
|
|
|
- Estella Cain
- 5 years ago
- Views:
Transcription
1 Homographies and RANSAC Computer vision Bill Freeman and Antonio Torralba March 30, 2011
2 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2
3 Depth-based ambiguity of position Camera A Camera B
4 Under what conditions can you know where to translate each point of image A to where it would appear in camera B (with calibrated cameras), knowing nothing about image depths? Camera A Camera B 4
5 (a) camera rotation 5
6 and (b) imaging a planar surface 6
7 Geometry of perspective projection sensor plane pinhole 7
8 Geometry of perspective projection inverted copy of sensor plane pinhole 7
9 Geometry of perspective projection inverted copy of sensor plane pinhole Let s look at this scene from above... 7
10 Two cameras with same center of projection common pinhole position of the cameras
11 Two cameras with same center of projection camera A common pinhole position of the cameras
12 Two cameras with same center of projection camera A camera B common pinhole position of the cameras
13 Two cameras with same center of projection camera A camera B common pinhole position of the cameras Can generate any synthetic camera view as long as it has the same center of projection!
14 Two cameras with offset centers of projection camera A camera B camera A center
15 Two cameras with offset centers of projection camera A camera B camera A center camera B center
16 Two cameras with offset centers of projection camera A camera B camera A center camera B center
17 Two cameras with offset centers of projection camera A camera B camera A center camera B center
18 Recap When we only rotate the camera (around nodal point) depth does not matter It only performs a 2D warp one-to-one mapping of the 2D plane plus of course reveals stuff that was outside the field of view A B Now we just need to figure out this mapping
19 Aligning images: translation?
20 Aligning images: translation? left on top right on top
21 Aligning images: translation? left on top right on top Translations are not enough to align the images
22 homography
23 homography
24 homography
25 homography How many pairs of points does it take to specify M_10?
26 Homography See Szeliski Sect 2.1.5, Mapping from one camera to another. PP1 PP2
27 Homography See Szeliski Sect 2.1.5, Mapping from one camera to another. Projective mapping between any two projection planes with the same center of projection called Homography represented as 3x3 matrix in homogenous coordinates p H p PP1 PP2
28 Homography See Szeliski Sect 2.1.5, Mapping from one camera to another. Projective mapping between any two projection planes with the same center of projection called Homography represented as 3x3 matrix in homogenous coordinates p H p To apply a homography H Compute p = Hp (regular matrix multiply) Convert p from homogeneous to image coordinates (divide by w) PP1 PP2
29 Images of planar objects, taken by generically offset cameras, are also related by a homography. camera A From Szeliski book camera B
30 Measurements on planes CSE 576, Spring 2008 Projective Geometry 6
31 Measurements on planes CSE 576, Spring 2008 Projective Geometry 6
32 Measurements on planes Approach: unwarp then measure CSE 576, Spring 2008 Projective Geometry 6
33 Measurements on planes Approach: unwarp then measure CSE 576, Spring 2008 Projective Geometry 6
34 Measurements on planes Approach: unwarp then measure CSE 576, Spring 2008 Projective Geometry 6
35 Measurements on planes Approach: unwarp then measure CSE 576, Spring 2008 Projective Geometry 6
36 Measurements on planes Approach: unwarp then measure CSE 576, Spring 2008 Projective Geometry 6
37 Measurements on planes Approach: unwarp then measure How to unwarp? CSE 576, Spring 2008 Projective Geometry 6
38 Image rectification p p To unwarp (rectify) an image solve for homography H given p and p solve equations of the form: wp = Hp linear in unknowns: w and coefficients of H H is defined up to an arbitrary scale factor how many points are necessary to solve for H? CSE 576, Spring 2008 Projective Geometry 7
39 Solving for homographies CSE 576, Spring 2008 Projective Geometry 8
40 Solving for homographies CSE 576, Spring 2008 Projective Geometry 8
41 Solving for homographies CSE 576, Spring 2008 Projective Geometry 8
42 Solving for homographies CSE 576, Spring 2008 Projective Geometry 8
43 Solving for homographies A h 0 CSE 576, Spring 2008 Projective Geometry 9
44 Solving for homographies A h 0 2n 9 9 2n CSE 576, Spring 2008 Projective Geometry 9
45 Solving for homographies Defines a least squares problem: A h 0 2n 9 9 2n CSE 576, Spring 2008 Projective Geometry 9
46 Solving for homographies Defines a least squares problem: A h 0 2n 9 9 2n Since h is only defined up to scale, solve for unit vector ĥ CSE 576, Spring 2008 Projective Geometry 9
47 Solving for homographies Defines a least squares problem: A h 0 2n 9 9 2n Since h is only defined up to scale, solve for unit vector ĥ Solution: ĥ = eigenvector of A T A with smallest eigenvalue Works with 4 or more points CSE 576, Spring 2008 Projective Geometry 9
48 Image warping with homographies homography so that image is parallel to floor homography so that image is parallel to right wall image plane below
49 Image warping with homographies homography so that image is parallel to floor homography so that image is parallel to right wall black area where no pixel maps to image plane below
50 Image warping with homographies homography so that image is parallel to floor homography so that image is parallel to right wall black area where no pixel maps to image plane below
51 automatic image mosaicing
52 automatic image mosaicing Basic Procedure Take a sequence of images from the same position. Rotate the camera about its optical center (entrance pupil). Robustly compute the homography transformation between second image and first. Transform (warp) the second image to overlap with first. Blend the two together to create a mosaic. If there are more images, repeat.
53 Robust feature matching through RANSAC Krister Parmstrand Nikon D70. Stitched Panorama. The sky has been retouched. No other image manipulation. with a lot of slides stolen from Steve Seitz and Rick Szeliski : Computational Photography Alexei Efros, CMU, Fall 2005
54 Feature matching descriptors for left image feature points descriptors for right image feature points?
55 Strategies to match images robustly (a) Working with individual features: For each feature point, find most similar point in other image (SIFT distance) Reject ambiguous matches where there are too many similar points (b) Working with all the features: Given some good feature matches, look for possible homographies relating the two images Reject homographies that don t have many feature matches. 25
56 (a) Feature-space outlier rejection Let s not match all features, but only these that have similar enough matches? How can we do it? SSD(patch1,patch2) < threshold How to set threshold? Not so easy.
57 Feature-space outlier rejection A better way [Lowe, 1999]: 1-NN: SSD of the closest match 2-NN: SSD of the second-closest match Look at how much better 1-NN is than 2-NN, e.g. 1-NN/2-NN That is, is our best match so much better than the rest?
58 Feature-space outlier rejection A better way [Lowe, 1999]: 1-NN: SSD of the closest match 2-NN: SSD of the second-closest match Look at how much better 1-NN is than 2-NN, e.g. 1-NN/2-NN That is, is our best match so much better than the rest?
59 Feature-space outlier rejection
60 Feature-space outlier rejection Can we now compute H from the blue points?
61 Feature-space outlier rejection Can we now compute H from the blue points? No! Still too many outliers What can we do?
62 (b) Matching many features--looking for a good homography Simplified illustration with translation instead of homography What do we do about the bad matches? Note: at this point we don t know which ones are good/bad
63 RAndom SAmple Consensus Select one match, count inliers
64 RAndom SAmple Consensus 0 inliers Select one match, count inliers
65 RAndom SAmple Consensus 4 inliers Select one match, count inliers
66 RAndom SAmple Consensus Select one match, count inliers Keep match with largest set of inliers
67 At the end: Least squares fit Find average translation vector, but with only inliers
68 At the end: Least squares fit Find average translation vector, but with only inliers
69 Reference M. A. Fischler, R. C. Bolles. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Comm. of the ACM, Vol 24, pp , citation.cfm?id=
70 RANSAC for estimating homography
71 RANSAC for estimating homography RANSAC loop:
72 RANSAC for estimating homography RANSAC loop: Select four feature pairs (at random)
73 RANSAC for estimating homography RANSAC loop: Select four feature pairs (at random) Compute homography H (exact)
74 RANSAC for estimating homography RANSAC loop: Select four feature pairs (at random) Compute homography H (exact) Compute inliers where p i, H p i < ε
75 RANSAC for estimating homography RANSAC loop: Select four feature pairs (at random) Compute homography H (exact) Compute inliers where p i, H p i < ε
76 RANSAC for estimating homography RANSAC loop: Select four feature pairs (at random) Compute homography H (exact) Compute inliers where p i, H p i < ε Keep largest set of inliers
77 RANSAC for estimating homography RANSAC loop: Select four feature pairs (at random) Compute homography H (exact) Compute inliers where p i, H p i < ε Keep largest set of inliers Re-compute least-squares H estimate using all of the inliers
78 Simple example: fit a line Rather than homography H (8 numbers) fit y=ax+b (2 numbers a, b) to 2D pairs 37
79 Simple example: fit a line 38
80 Simple example: fit a line Pick 2 points 38
81 Simple example: fit a line Pick 2 points Fit line 38
82 Simple example: fit a line Pick 2 points Fit line Count inliers 3 inlier 38
83 Simple example: fit a line 39
84 Simple example: fit a line Pick 2 points 39
85 Simple example: fit a line Pick 2 points Fit line 39
86 Simple example: fit a line Pick 2 points Fit line Count inliers 4 inlier 39
87 Simple example: fit a line 40
88 Simple example: fit a line Pick 2 points 40
89 Simple example: fit a line Pick 2 points Fit line 40
90 Simple example: fit a line Pick 2 points Fit line Count inliers 9 inlier 40
91 Simple example: fit a line 41
92 Simple example: fit a line Pick 2 points 41
93 Simple example: fit a line Pick 2 points Fit line 41
94 Simple example: fit a line Pick 2 points Fit line Count inliers 8 inlier 41
95 Simple example: fit a line 42
96 Simple example: fit a line Use biggest set of inliers 42
97 Simple example: fit a line Use biggest set of inliers Do least-square fit 42
98 RANSAC red: rejected by 2nd nearest neighbor criterion blue: Ransac outliers yellow: inliers
99 Robustness 44
100 Robustness Proportion of inliers in our pairs is G (for good ) Our model needs P pairs P=4 for homography 44
101 Robustness Proportion of inliers in our pairs is G (for good ) Our model needs P pairs P=4 for homography Probability that we pick P inliers? 44
102 Robustness Proportion of inliers in our pairs is G (for good ) Our model needs P pairs P=4 for homography Probability that we pick P inliers? G P 44
103 Robustness Proportion of inliers in our pairs is G (for good ) Our model needs P pairs P=4 for homography Probability that we pick P inliers? G P Probability that after N RANSAC iterations we have not picked a set of inliers? 44
104 Robustness Proportion of inliers in our pairs is G (for good ) Our model needs P pairs P=4 for homography Probability that we pick P inliers? G P Probability that after N RANSAC iterations we have not picked a set of inliers? (1-G P ) N 44
105 Robustness: example Matlab: p=4; x=0.5; n=1000; (1-x^p)^n Proportion of inliers G=0.5 Probability that we pick P=4 inliers? = (6% chance) Probability that we have not picked a set of inliers? N=100 iterations: ( ) 100 = (1 chance in 600) N=1000 iterations: 1 chance in 1e28 45
106 Robustness: example Proportion of inliers G=0.3 Probability that we pick P=4 inliers? = (0.8% chance) Probability that we have not picked a set of inliers? N=100 iterations: ( ) 100 =0.44 (1 chance in 2) N=1000 iterations: 1 chance in
107 Robustness: example Proportion of inliers G=0.1 Probability that we pick P=4 inliers? = (0.01% chances, 1 in 10,000) Probability that we have not picked a set of inliers? N=100 iterations: ( ) 100 =0.99 N=1000 iterations: 90% N=10,000: 36% 47 N=100,000: 1 in 22,000
108 Robustness: conclusions 48
109 Robustness: conclusions Effect of number of parameters of model/ number of necessary pairs 48
110 Robustness: conclusions Effect of number of parameters of model/ number of necessary pairs Bad exponential 48
111 Robustness: conclusions Effect of number of parameters of model/ number of necessary pairs Bad exponential Effect of percentage of inliers 48
112 Robustness: conclusions Effect of number of parameters of model/ number of necessary pairs Bad exponential Effect of percentage of inliers Base of the exponential 48
113 Robustness: conclusions Effect of number of parameters of model/ number of necessary pairs Bad exponential Effect of percentage of inliers Base of the exponential Effect of number of iterations 48
114 Robustness: conclusions Effect of number of parameters of model/ number of necessary pairs Bad exponential Effect of percentage of inliers Base of the exponential Effect of number of iterations Good exponential 48
115 RANSAC recap For fitting a model with low number P of parameters (8 for homographies) Loop Select P random data points Fit model Count inliers (other data points well fit by this model) Keep model with largest number of inliers 49
116 Example: Recognising Panoramas M. Brown and D. Lowe, University of British Columbia * M. Brown and D. Lowe. Automatic Panoramic Image Stitching using Invariant Features. International Journal of Computer Vision, 74(1), pages 59-73, 2007 (pdf 3.5Mb bib) * M. Brown and D. G. Lowe. Recognising Panoramas. In Proceedings of the 9th International Conference on Computer Vision (ICCV2003), pages , Nice, France, 2003 (pdf 820kb ppt bib)
117 Recognising Panoramas?
118 RANSAC for Homography
119 RANSAC for Homography
120 RANSAC for Homography
121 Probabilistic model for verification
122 Finding the panoramas
123 Finding the panoramas
124 Finding the panoramas
125 Finding the panoramas
126 Results
127 61
128 Benefits of Laplacian image compositing M. Brown and D. Lowe. Automatic Panoramic Image Stitching using Invariant Features. International Journal of Computer Vision, 74(1), pages 59-73,
129 Photo Tourism: Exploring Photo Collections in 3D Noah Snavely Steven M. Seitz University of Washington Richard Szeliski Microsoft Research Noah Snavely Noah Snavely
130 2006 Noah Snavely
131 2006 Noah Snavely
132 15, Noah Snavely
133 15,464 37, Noah Snavely
134 15,464 37,383 76, Noah Snavely
135 Movie 2006 Noah Snavely
136 Photo Tourism overview 2006 Noah Snavely
137 Photo Tourism overview Input photographs 2006 Noah Snavely
138 Photo Tourism overview Scene reconstruction Input photographs 2006 Noah Snavely
139 Photo Tourism overview Scene reconstruction Input photographs Relative camera positions and orientations Point cloud Sparse correspondence 2006 Noah Snavely
140 Photo Tourism overview Input photographs Scene reconstruction Relative camera positions and orientations Point cloud Sparse correspondence Photo Explorer 2006 Noah Snavely
141 Photo Tourism overview Scene reconstruction Input photographs Photo Explorer 2006 Noah Snavely
142 Photo Tourism overview Scene reconstruction Input photographs Photo Explorer 2006 Noah Snavely
143 Scene reconstruction 2006 Noah Snavely
144 Scene reconstruction Automatically estimate position, orientation, and focal length of cameras 3D positions of feature points 2006 Noah Snavely
145 Scene reconstruction Automatically estimate position, orientation, and focal length of cameras 3D positions of feature points Feature detection 2006 Noah Snavely
146 Scene reconstruction Automatically estimate position, orientation, and focal length of cameras 3D positions of feature points Feature detection Pairwise feature matching 2006 Noah Snavely
147 Scene reconstruction Automatically estimate position, orientation, and focal length of cameras 3D positions of feature points Feature detection Pairwise feature matching Correspondence estimation 2006 Noah Snavely
148 Scene reconstruction Automatically estimate position, orientation, and focal length of cameras 3D positions of feature points Feature detection Pairwise feature matching Incremental structure from motion Correspondence estimation 2006 Noah Snavely
149 Feature detection Detect features using SIFT [Lowe, IJCV 2004] 2006 Noah Snavely
150 Feature detection Detect features using SIFT [Lowe, IJCV 2004] 2006 Noah Snavely
151 Feature detection Detect features using SIFT [Lowe, IJCV 2004] 2006 Noah Snavely
152 Feature detection Detect features using SIFT [Lowe, IJCV 2004] 2006 Noah Snavely
153 Feature detection Detect features using SIFT [Lowe, IJCV 2004] 2006 Noah Snavely
154 Feature matching Match features between each pair of images 2006 Noah Snavely
155 Feature matching Match features between each pair of images 2006 Noah Snavely
156 Feature matching Refine matching using RANSAC [Fischler & Bolles 1987] to estimate fundamental matrices between pairs (See 6.801/6.866 for fundamental matrix, or Hartley and Zisserman, Multi-View Geometry. See also the fundamental matrix song: ) 2006 Noah Snavely
157 Feature matching Refine matching using RANSAC [Fischler & Bolles 1987] to estimate fundamental matrices between pairs (See 6.801/6.866 for fundamental matrix, or Hartley and Zisserman, Multi-View Geometry. See also the fundamental matrix song: ) 2006 Noah Snavely
158 Structure from motion 2006 Noah Snavely
159 Structure from motion Camera 1 Camera 3 Camera Noah Snavely
160 Structure from motion Camera 1 Camera 3 Camera Noah Snavely
161 Structure from motion p 1 p 4 p 3 p 2 p 5 p 6 p 7 Camera 1 Camera 3 Camera Noah Snavely
162 Structure from motion p 1 p 4 p 3 p 2 p 5 p 6 p 7 Camera 1 Camera 2 Camera 3 R 1,t 1 R 2,t 2 R 3,t Noah Snavely
163 Structure from motion p 1 p 4 p 3 p 2 p 5 p 6 p 7 Camera 1 Camera 2 Camera 3 R 1,t 1 R 2,t 2 R 3,t Noah Snavely
164 Structure from motion p 1 p 4 p 3 p 2 minimize f (R, T, P) p 5 p 6 p 7 Camera 1 Camera 2 Camera 3 R 1,t 1 R 2,t 2 R 3,t Noah Snavely
165 Links Code available: Noah Snavely
Feature Matching and RANSAC
Feature Matching and RANSAC Recognising Panoramas. [M. Brown and D. Lowe,ICCV 2003] [Brown, Szeliski, Winder, CVPR 2005] with a lot of slides stolen from Steve Seitz, Rick Szeliski, A. Efros Introduction
Image Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi
Image Stitching Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Combine two or more overlapping images to make one larger image Add example Slide credit: Vaibhav Vaish
Image warping and stitching
Image warping and stitching May 4 th, 2017 Yong Jae Lee UC Davis Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 2 Alignment problem In alignment, we will
Image warping and stitching
Image warping and stitching May 5 th, 2015 Yong Jae Lee UC Davis PS2 due next Friday Announcements 2 Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 3 Alignment
CS6670: Computer Vision
CS6670: Computer Vision Noah Snavely Lecture 7: Image Alignment and Panoramas What s inside your fridge? http://www.cs.washington.edu/education/courses/cse590ss/01wi/ Projection matrix intrinsics projection
Image Warping and Mosacing
Image Warping and Mosacing 15-463: Rendering and Image Processing Alexei Efros with a lot of slides stolen from Steve Seitz and Rick Szeliski Today Mosacs Image Warping Homographies Programming Assignment
Image warping and stitching
Image warping and stitching Thurs Oct 15 Last time Feature-based alignment 2D transformations Affine fit RANSAC 1 Robust feature-based alignment Extract features Compute putative matches Loop: Hypothesize
Automatic Image Alignment (feature-based)
Automatic Image Alignment (feature-based) Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2006 Today s lecture Feature
Automatic Image Alignment
Automatic Image Alignment Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 Live Homography DEMO Check out panoramio.com
Image correspondences and structure from motion
Image correspondences and structure from motion http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 20 Course announcements Homework 5 posted.
Automatic Image Alignment
Automatic Image Alignment with a lot of slides stolen from Steve Seitz and Rick Szeliski Mike Nese CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2018 Live Homography
Mosaics. Today s Readings
Mosaics VR Seattle: http://www.vrseattle.com/ Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html Today s Readings Szeliski and Shum paper (sections
Photo Tourism: Exploring Photo Collections in 3D
Photo Tourism: Exploring Photo Collections in 3D SIGGRAPH 2006 Noah Snavely Steven M. Seitz University of Washington Richard Szeliski Microsoft Research 2006 2006 Noah Snavely Noah Snavely Reproduced with
EE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 12 130228 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Panoramas, Mosaics, Stitching Two View Geometry
CSE 527: Introduction to Computer Vision
CSE 527: Introduction to Computer Vision Week 5 - Class 1: Matching, Stitching, Registration September 26th, 2017 ??? Recap Today Feature Matching Image Alignment Panoramas HW2! Feature Matches Feature
Noah Snavely Steven M. Seitz. Richard Szeliski. University of Washington. Microsoft Research. Modified from authors slides
Photo Tourism: Exploring Photo Collections in 3D Noah Snavely Steven M. Seitz University of Washington Richard Szeliski Microsoft Research 2006 2006 Noah Snavely Noah Snavely Modified from authors slides
Instance-level recognition
Instance-level recognition 1) Local invariant features 2) Matching and recognition with local features 3) Efficient visual search 4) Very large scale indexing Matching of descriptors Matching and 3D reconstruction
Instance-level recognition
Instance-level recognition 1) Local invariant features 2) Matching and recognition with local features 3) Efficient visual search 4) Very large scale indexing Matching of descriptors Matching and 3D reconstruction
Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac
Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration
Photo Tourism: Exploring Photo Collections in 3D
Photo Tourism: Exploring Photo Collections in 3D Noah Snavely Steven M. Seitz University of Washington Richard Szeliski Microsoft Research 15,464 37,383 76,389 2006 Noah Snavely 15,464 37,383 76,389 Reproduced
Today s lecture. Image Alignment and Stitching. Readings. Motion models
Today s lecture Image Alignment and Stitching Computer Vision CSE576, Spring 2005 Richard Szeliski Image alignment and stitching motion models cylindrical and spherical warping point-based alignment global
Instance-level recognition part 2
Visual Recognition and Machine Learning Summer School Paris 2011 Instance-level recognition part 2 Josef Sivic http://www.di.ens.fr/~josef INRIA, WILLOW, ENS/INRIA/CNRS UMR 8548 Laboratoire d Informatique,
Midterm Examination CS 534: Computational Photography
Midterm Examination CS 534: Computational Photography November 3, 2016 NAME: Problem Score Max Score 1 6 2 8 3 9 4 12 5 4 6 13 7 7 8 6 9 9 10 6 11 14 12 6 Total 100 1 of 8 1. [6] (a) [3] What camera setting(s)
Stereo and Epipolar geometry
Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka
Camera Geometry II. COS 429 Princeton University
Camera Geometry II COS 429 Princeton University Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence
Stereo Epipolar Geometry for General Cameras. Sanja Fidler CSC420: Intro to Image Understanding 1 / 33
Stereo Epipolar Geometry for General Cameras Sanja Fidler CSC420: Intro to Image Understanding 1 / 33 Stereo Epipolar geometry Case with two cameras with parallel optical axes General case Now this Sanja
Local features: detection and description. Local invariant features
Local features: detection and description Local invariant features Detection of interest points Harris corner detection Scale invariant blob detection: LoG Description of local patches SIFT : Histograms
Image Warping. Many slides from Alyosha Efros + Steve Seitz. Photo by Sean Carroll
Image Warping Man slides from Alosha Efros + Steve Seitz Photo b Sean Carroll Morphing Blend from one object to other with a series of local transformations Image Transformations image filtering: change
arxiv: v1 [cs.cv] 28 Sep 2018
![arxiv: v1 [cs.cv] 28 Sep 2018](/thumbs/93/113542646.jpg "arxiv: v1 [cs.cv] 28 Sep 2018")
Camera Pose Estimation from Sequence of Calibrated Images arxiv:1809.11066v1 [cs.cv] 28 Sep 2018 Jacek Komorowski 1 and Przemyslaw Rokita 2 1 Maria Curie-Sklodowska University, Institute of Computer Science,
Structure from Motion. Introduction to Computer Vision CSE 152 Lecture 10
Structure from Motion CSE 152 Lecture 10 Announcements Homework 3 is due May 9, 11:59 PM Reading: Chapter 8: Structure from Motion Optional: Multiple View Geometry in Computer Vision, 2nd edition, Hartley
Local features: detection and description May 12 th, 2015
Local features: detection and description May 12 th, 2015 Yong Jae Lee UC Davis Announcements PS1 grades up on SmartSite PS1 stats: Mean: 83.26 Standard Dev: 28.51 PS2 deadline extended to Saturday, 11:59
A Systems View of Large- Scale 3D Reconstruction
Lecture 23: A Systems View of Large- Scale 3D Reconstruction Visual Computing Systems Goals and motivation Construct a detailed 3D model of the world from unstructured photographs (e.g., Flickr, Facebook)
Instance-level recognition II.
Reconnaissance d objets et vision artificielle 2010 Instance-level recognition II. Josef Sivic http://www.di.ens.fr/~josef INRIA, WILLOW, ENS/INRIA/CNRS UMR 8548 Laboratoire d Informatique, Ecole Normale
Geometric camera models and calibration
Geometric camera models and calibration http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 13 Course announcements Homework 3 is out. - Due October
N-Views (1) Homographies and Projection
CS 4495 Computer Vision N-Views (1) Homographies and Projection Aaron Bobick School of Interactive Computing Administrivia PS 2: Get SDD and Normalized Correlation working for a given windows size say
CS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017
CS 277: Intro to Computer Vision Multiple Views Prof. Adriana Kovashka Universit of Pittsburgh March 4, 27 Plan for toda Affine and projective image transformations Homographies and image mosaics Stereo
Camera Calibration. COS 429 Princeton University
Camera Calibration COS 429 Princeton University Point Correspondences What can you figure out from point correspondences? Noah Snavely Point Correspondences X 1 X 4 X 3 X 2 X 5 X 6 X 7 p 1,1 p 1,2 p 1,3
Local features and image matching. Prof. Xin Yang HUST
Local features and image matching Prof. Xin Yang HUST Last time RANSAC for robust geometric transformation estimation Translation, Affine, Homography Image warping Given a 2D transformation T and a source
Midterm Wed. Local features: detection and description. Today. Last time. Local features: main components. Goal: interest operator repeatability
Midterm Wed. Local features: detection and description Monday March 7 Prof. UT Austin Covers material up until 3/1 Solutions to practice eam handed out today Bring a 8.5 11 sheet of notes if you want Review
Image stitching. Announcements. Outline. Image stitching
Announcements Image stitching Project #1 was due yesterday. Project #2 handout will be available on the web later tomorrow. I will set up a webpage for artifact voting soon. Digital Visual Effects, Spring
Automatic Image Alignment (direct) with a lot of slides stolen from Steve Seitz and Rick Szeliski
Automatic Image Alignment (direct) with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Today Go over Midterm Go over Project #3
Chaplin, Modern Times, 1936
Chaplin, Modern Times, 1936 [A Bucket of Water and a Glass Matte: Special Effects in Modern Times; bonus feature on The Criterion Collection set] Multi-view geometry problems Structure: Given projections
Epipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Shiv Ram Dubey, IIIT Sri City Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X
Local features and image matching May 8 th, 2018
Local features and image matcing May 8 t, 2018 Yong Jae Lee UC Davis Last time RANSAC for robust fitting Lines, translation Image mosaics Fitting a 2D transformation Homograpy 2 Today Mosaics recap: How
CS4670: Computer Vision
CS4670: Computer Vision Noah Snavely Lecture 6: Feature matching and alignment Szeliski: Chapter 6.1 Reading Last time: Corners and blobs Scale-space blob detector: Example Feature descriptors We know
Epipolar Geometry and Stereo Vision
CS 1699: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 8, 2015 Today Review Projective transforms Image stitching (homography) Epipolar
Feature Based Registration - Image Alignment
Feature Based Registration - Image Alignment Image Registration Image registration is the process of estimating an optimal transformation between two or more images. Many slides from Alexei Efros http://graphics.cs.cmu.edu/courses/15-463/2007_fall/463.html
Epipolar Geometry CSE P576. Dr. Matthew Brown
Epipolar Geometry CSE P576 Dr. Matthew Brown Epipolar Geometry Epipolar Lines, Plane Constraint Fundamental Matrix, Linear solution + RANSAC Applications: Structure from Motion, Stereo [ Szeliski 11] 2
There are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...
STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own
arxiv: v1 [cs.cv] 28 Sep 2018
![arxiv: v1 [cs.cv] 28 Sep 2018](/thumbs/91/106305353.jpg "arxiv: v1 [cs.cv] 28 Sep 2018")
Extrinsic camera calibration method and its performance evaluation Jacek Komorowski 1 and Przemyslaw Rokita 2 arxiv:1809.11073v1 [cs.cv] 28 Sep 2018 1 Maria Curie Sklodowska University Lublin, Poland jacek.komorowski@gmail.com
Srikumar Ramalingam. Review. 3D Reconstruction. Pose Estimation Revisited. School of Computing University of Utah
School of Computing University of Utah Presentation Outline 1 2 3 Forward Projection (Reminder) u v 1 KR ( I t ) X m Y m Z m 1 Backward Projection (Reminder) Q K 1 q Presentation Outline 1 2 3 Sample Problem
Miniature faking. In close-up photo, the depth of field is limited.
Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg
More Mosaic Madness. CS194: Image Manipulation & Computational Photography. Steve Seitz and Rick Szeliski. Jeffrey Martin (jeffrey-martin.
More Mosaic Madness Jeffrey Martin (jeffrey-martin.com) CS194: Image Manipulation & Computational Photography with a lot of slides stolen from Alexei Efros, UC Berkeley, Fall 2018 Steve Seitz and Rick
Large Scale 3D Reconstruction by Structure from Motion
Large Scale 3D Reconstruction by Structure from Motion Devin Guillory Ziang Xie CS 331B 7 October 2013 Overview Rome wasn t built in a day Overview of SfM Building Rome in a Day Building Rome on a Cloudless
Image-based Modeling and Rendering: 8. Image Transformation and Panorama
Image-based Modeling and Rendering: 8. Image Transformation and Panorama I-Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung Univ, Taiwan Outline Image transformation How to represent the
Srikumar Ramalingam. Review. 3D Reconstruction. Pose Estimation Revisited. School of Computing University of Utah
School of Computing University of Utah Presentation Outline 1 2 3 Forward Projection (Reminder) u v 1 KR ( I t ) X m Y m Z m 1 Backward Projection (Reminder) Q K 1 q Q K 1 u v 1 What is pose estimation?
Epipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x
Step-by-Step Model Buidling
Step-by-Step Model Buidling Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure
Model Fitting, RANSAC. Jana Kosecka
Model Fitting, RANSAC Jana Kosecka Fitting: Overview If we know which points belong to the line, how do we find the optimal line parameters? Least squares What if there are outliers? Robust fitting, RANSAC
Automatic Panoramic Image Stitching. Dr. Matthew Brown, University of Bath
Automatic Panoramic Image Stitching Dr. Matthew Brown, University of Bath Automatic 2D Stitching The old days: panoramic stitchers were limited to a 1-D sweep 2005: 2-D stitchers use object recognition
Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263
Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine
Introduction to Computer Vision
Introduction to Computer Vision Michael J. Black Nov 2009 Perspective projection and affine motion Goals Today Perspective projection 3D motion Wed Projects Friday Regularization and robust statistics
Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253
Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine
Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac
Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration
Structure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t R 2 3,t 3 Camera 1 Camera
Single-view 3D Reconstruction
Single-view 3D Reconstruction 10/12/17 Computational Photography Derek Hoiem, University of Illinois Some slides from Alyosha Efros, Steve Seitz Notes about Project 4 (Image-based Lighting) You can work
Final project bits and pieces
Final project bits and pieces The project is expected to take four weeks of time for up to four people. At 12 hours per week per person that comes out to: ~192 hours of work for a four person team. Capstone:
The SIFT (Scale Invariant Feature
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor developed by David Lowe University of British Columbia Initial paper ICCV 1999 Newer journal paper IJCV 2004 Review: Matt Brown s Canonical
Agenda. Rotations. Camera calibration. Homography. Ransac
Agenda Rotations Camera calibration Homography Ransac Geometric Transformations y x Transformation Matrix # DoF Preserves Icon translation rigid (Euclidean) similarity affine projective h I t h R t h sr
Last lecture. Passive Stereo Spacetime Stereo
Last lecture Passive Stereo Spacetime Stereo Today Structure from Motion: Given pixel correspondences, how to compute 3D structure and camera motion? Slides stolen from Prof Yungyu Chuang Epipolar geometry
6.098 Digital and Computational Photography Advanced Computational Photography. Panoramas. Bill Freeman Frédo Durand MIT - EECS
6.098 Digital and Computational Photography 6.882 Advanced Computational Photography Panoramas Bill Freeman Frédo Durand MIT - EECS Lots of slides stolen from Alyosha Efros, who stole them from Steve Seitz
Computational Optical Imaging - Optique Numerique. -- Multiple View Geometry and Stereo --
Computational Optical Imaging - Optique Numerique -- Multiple View Geometry and Stereo -- Winter 2013 Ivo Ihrke with slides by Thorsten Thormaehlen Feature Detection and Matching Wide-Baseline-Matching
Panoramas. Why Mosaic? Why Mosaic? Mosaics: stitching images together. Why Mosaic? Olivier Gondry. Bill Freeman Frédo Durand MIT - EECS
Olivier Gondry 6.098 Digital and Computational Photography 6.882 Advanced Computational Photography Panoramas Director of music video and commercial Special effect specialist (Morphing, rotoscoping) Today
Dense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem
CS201 Computer Vision Camera Geometry
CS201 Computer Vision Camera Geometry John Magee 25 November, 2014 Slides Courtesy of: Diane H. Theriault (deht@bu.edu) Question of the Day: How can we represent the relationships between cameras and the
More Single View Geometry
More Single View Geometry with a lot of slides stolen from Steve Seitz Cyclops Odilon Redon 1904 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Final Projects Are coming up fast! Undergrads
CS 1674: Intro to Computer Vision. Midterm Review. Prof. Adriana Kovashka University of Pittsburgh October 10, 2016
CS 1674: Intro to Computer Vision Midterm Review Prof. Adriana Kovashka University of Pittsburgh October 10, 2016 Reminders The midterm exam is in class on this coming Wednesday There will be no make-up
calibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
Local Feature Detectors
Local Feature Detectors Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Slides adapted from Cordelia Schmid and David Lowe, CVPR 2003 Tutorial, Matthew Brown,
Fundamental matrix. Let p be a point in left image, p in right image. Epipolar relation. Epipolar mapping described by a 3x3 matrix F
Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix F Fundamental
Announcements. Mosaics. How to do it? Image Mosaics
Announcements Mosaics Project artifact voting Project 2 out today (help session at end of class) http://www.destination36.com/start.htm http://www.vrseattle.com/html/vrview.php?cat_id=&vrs_id=vrs38 Today
Computer Vision I. Announcements. Random Dot Stereograms. Stereo III. CSE252A Lecture 16
Announcements Stereo III CSE252A Lecture 16 HW1 being returned HW3 assigned and due date extended until 11/27/12 No office hours today No class on Thursday 12/6 Extra class on Tuesday 12/4 at 6:30PM in
More Single View Geometry
More Single View Geometry with a lot of slides stolen from Steve Seitz Cyclops Odilon Redon 1904 15-463: Computational Photography Alexei Efros, CMU, Fall 2008 Quiz: which is 1,2,3-point perspective Image
Structure from Motion
11/18/11 Structure from Motion Computer Vision CS 143, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from
AUTOMATIC RECTIFICATION OF LONG IMAGE SEQUENCES. Kenji Okuma, James J. Little, David G. Lowe
AUTOMATIC RECTIFICATION OF LONG IMAGE SEQUENCES Kenji Okuma, James J. Little, David G. Lowe The Laboratory of Computational Intelligence The University of British Columbia Vancouver, British Columbia,
Fitting. Fitting. Slides S. Lazebnik Harris Corners Pkwy, Charlotte, NC
Fitting We ve learned how to detect edges, corners, blobs. Now what? We would like to form a higher-level, more compact representation of the features in the image by grouping multiple features according
Object Recognition with Invariant Features
Object Recognition with Invariant Features Definition: Identify objects or scenes and determine their pose and model parameters Applications Industrial automation and inspection Mobile robots, toys, user
Photo by Carl Warner
Photo b Carl Warner Photo b Carl Warner Photo b Carl Warner Fitting and Alignment Szeliski 6. Computer Vision CS 43, Brown James Has Acknowledgment: Man slides from Derek Hoiem and Grauman&Leibe 2008 AAAI
Building a Panorama. Matching features. Matching with Features. How do we build a panorama? Computational Photography, 6.882
Matching features Building a Panorama Computational Photography, 6.88 Prof. Bill Freeman April 11, 006 Image and shape descriptors: Harris corner detectors and SIFT features. Suggested readings: Mikolajczyk
Observations. Basic iteration Line estimated from 2 inliers
Line estimated from 2 inliers 3 Observations We need (in this case!) a minimum of 2 points to determine a line Given such a line l, we can determine how well any other point y fits the line l For example:
CS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
3D Reconstruction from Two Views
3D Reconstruction from Two Views Huy Bui UIUC huybui1@illinois.edu Yiyi Huang UIUC huang85@illinois.edu Abstract In this project, we study a method to reconstruct a 3D scene from two views. First, we extract
Lecture 10: Multi view geometry
Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
Two-view geometry Computer Vision Spring 2018, Lecture 10
Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of
A Summary of Projective Geometry
A Summary of Projective Geometry Copyright 22 Acuity Technologies Inc. In the last years a unified approach to creating D models from multiple images has been developed by Beardsley[],Hartley[4,5,9],Torr[,6]
Agenda. Rotations. Camera models. Camera calibration. Homographies
Agenda Rotations Camera models Camera calibration Homographies D Rotations R Y = Z r r r r r r r r r Y Z Think of as change of basis where ri = r(i,:) are orthonormal basis vectors r rotated coordinate
Improving Initial Estimations for Structure from Motion Methods
Improving Initial Estimations for Structure from Motion Methods University of Bonn Outline Motivation Computer-Vision Basics Stereo Vision Bundle Adjustment Feature Matching Global Initial Estimation Component
3D Geometry and Camera Calibration
3D Geometry and Camera Calibration 3D Coordinate Systems Right-handed vs. left-handed x x y z z y 2D Coordinate Systems 3D Geometry Basics y axis up vs. y axis down Origin at center vs. corner Will often
CSE 252B: Computer Vision II
CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribes: Jeremy Pollock and Neil Alldrin LECTURE 14 Robust Feature Matching 14.1. Introduction Last lecture we learned how to find interest points
Computational Optical Imaging - Optique Numerique. -- Single and Multiple View Geometry, Stereo matching --
Computational Optical Imaging - Optique Numerique -- Single and Multiple View Geometry, Stereo matching -- Autumn 2015 Ivo Ihrke with slides by Thorsten Thormaehlen Reminder: Feature Detection and Matching