Photo by Carl Warner

Size: px
Start display at page:

Download "Photo by Carl Warner"

Transcription

1 Photo b Carl Warner

2 Photo b Carl Warner

3 Photo b Carl Warner

4 Fitting and Alignment Szeliski 6. Computer Vision CS 43, Brown James Has Acknowledgment: Man slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial

5 Project 2 questions?

6 This section: correspondence and alignment Correspondence: matching points, patches, edges, or regions across images

7 Review Fitting: find the parameters of a model that best fit the data Alignment: find the parameters of the transformation that best align matched points

8 Review: Fitting and Alignment Design challenges Design a suitable goodness of fit measure Similarit should reflect application goals Encode robustness to outliers and noise Design an optimization method Avoid local optima Find best parameters quickl

9 Fitting and Alignment: Methods Global optimization / Search for parameters Least squares fit Robust least squares Iterative closest point (ICP) Hpothesize and test Hough transform RANSAC

10 Review: Hough Transform. Create a grid of parameter values 2. Each point votes for a set of parameters, incrementing those values in grid 3. Find maimum or local maima in grid

11 Review: Hough transform P.V.C. Hough, Machine Analsis of Bubble Chamber Pictures, Proc. Int. Conf. High Energ Accelerators and Instrumentation, 959 Given a set of points, find the curve or line that eplains the data points best m = m + b Hough space b Slide from S. Savarese

12 Review: Hough transform m b m Slide from S. Savarese b

13 Hough Transform How would we find circles? Of fied radius Of unknown radius

14 Hough transform conclusions Good Robust to outliers: each point votes separatel Fairl efficient (much faster than tring all sets of parameters) Provides multiple good fits Bad Some sensitivit to noise Bin size trades off between noise tolerance, precision, and speed/memor Can be hard to find sweet spot Not suitable for more than a few parameters grid size grows eponentiall Common applications Line fitting (also circles, ellipses, etc.) Object instance recognition (parameters are affine transform) Object categor recognition (parameters are position/scale)

15 RANSAC (RANdom SAmple Consensus) : Fischler & Bolles in 8. Algorithm:. Sample (randoml) the number of points required to fit the model 2. Solve for model parameters using samples 3. Score b the fraction of inliers within a preset threshold of the model Repeat -3 until the best model is found with high confidence

16 RANSAC Line fitting eample Algorithm:. Sample (randoml) the number of points required to fit the model (#=2) 2. Solve for model parameters using samples 3. Score b the fraction of inliers within a preset threshold of the model Repeat -3 until the best model is found with high confidence Illustration b Savarese

17 RANSAC Line fitting eample Algorithm:. Sample (randoml) the number of points required to fit the model (#=2) 2. Solve for model parameters using samples 3. Score b the fraction of inliers within a preset threshold of the model Repeat -3 until the best model is found with high confidence

18 RANSAC Line fitting eample N I 6 Algorithm:. Sample (randoml) the number of points required to fit the model (#=2) 2. Solve for model parameters using samples 3. Score b the fraction of inliers within a preset threshold of the model Repeat -3 until the best model is found with high confidence

19 RANSAC Algorithm: N I 4. Sample (randoml) the number of points required to fit the model (#=2) 2. Solve for model parameters using samples 3. Score b the fraction of inliers within a preset threshold of the model Repeat -3 until the best model is found with high confidence

20 How to choose parameters? Number of samples N Choose N so that, with probabilit p, at least one random sample is free from outliers (e.g. p=0.99) (outlier ratio: e ) Number of sampled points s Minimum number needed to fit the model Distance threshold Choose so that a good point with noise is likel (e.g., prob=0.95) within threshold Zero-mean Gaussian noise with std. dev. σ: t 2 =3.84σ 2 N log p/ log e s proportion of outliers e s 5% 0% 20% 25% 30% 40% 50% modified from M. Pollefes

21 RANSAC conclusions Good Robust to outliers Applicable for larger number of objective function parameters than Hough transform Optimization parameters are easier to choose than Hough transform Bad Computational time grows quickl with fraction of outliers and number of parameters Not good for getting multiple fits Common applications Computing a homograph (e.g., image stitching) Estimating fundamental matri (relating two views)

22 How do we fit the best alignment?

23 Alignment Alignment: find parameters of model that maps one set of points to another Tpicall want to solve for a global transformation that accounts for *most* true correspondences Difficulties Noise (tpicall -3 piels) Outliers (often 50%) Man-to-one matches or multiple objects

24 Parametric (global) warping T p = (,) p = (, ) Transformation T is a coordinate-changing machine: p = T(p) What does it mean that T is global? Is the same for an point p can be described b just a few numbers (parameters) For linear transformations, we can represent T as a matri p = Tp ' T '

25 Common transformations original Transformed translation rotation aspect affine perspective Slide credit (net few slides): A. Efros and/or S. Seitz

26 Scaling Scaling a coordinate means multipling each of its components b a scalar Uniform scaling means this scalar is the same for all components: 2

27 Scaling Non-uniform scaling: different scalars per component: X 2, Y 0.5

28 Scaling Scaling operation: ' ' a b Or, in matri form: ' ' a 0 0 b scaling matri S

29 2-D Rotation (, ) (, ) = cos() - sin() = sin() + cos()

30 2-D Rotation This is eas to capture in matri form: ' ' cos sin sin cos Even though sin() and cos() are nonlinear functions of, is a linear combination of and is a linear combination of and What is the inverse transformation? Rotation b For rotation matrices R T R R

31 Basic 2D transformations Translate Rotate Shear Scale ' ' cos sin sin cos ' ' s s 0 0 ' ' 0 0 t t f e d c b a Affine Affine is an combination of translation, scale, rotation, shear

32 Affine Transformations Affine transformations are combinations of Linear transformations, and Translations Properties of affine transformations: Lines map to lines Parallel lines remain parallel Ratios are preserved Closed under composition f e d c b a 0 0 ' ' f e d c b a or

33 Projective Transformations Projective transformations are combos of Affine transformations, and Projective warps Properties of projective transformations: ' ' w' Lines map to lines Parallel lines do not necessaril remain parallel Ratios are not preserved Closed under composition Models change of basis Projective matri is defined up to a scale (8 DOF) a d g b e h c f i w

34 2D image transformations (reference table) Szeliski 2.

35 Eample: solving for translation A A 2 A 3 B B 2 B 3 Given matched points in {A} and {B}, estimate the translation of the object A i A i B i B i t t

36 Eample: solving for translation A A 2 A 3 B B 2 B 3 Least squares solution A i A i B i B i t t (t, t ). Write down objective function 2. Derived solution a) Compute derivative b) Compute solution 3. Computational solution a) Write in form A=b b) Solve using pseudo-inverse or eigenvalue decomposition A n B n A n B n A B A B t t

37 Eample: solving for translation A A 5 A 2 A 3 (t, t ) B 4 B A 4 B 2 B 3 B 5 Problem: outliers RANSAC solution. Sample a set of matching points ( pair) 2. Solve for transformation parameters 3. Score parameters with number of inliers 4. Repeat steps -3 N times B i B i A i A i t t

38 Eample: solving for translation B 4 A B 5 B 6 A 2 A 3 (t, t ) B A 4 B 2 B 3 A 5 A 6 Problem: outliers, multiple objects, and/or man-to-one matches Hough transform solution. Initialize a grid of parameter values 2. Each matched pair casts a vote for consistent values 3. Find the parameters with the most votes 4. Solve using least squares with inliers B i B i A i A i t t

39 Eample: solving for translation (t, t ) Problem: no initial guesses for correspondence A i A i B i B i t t

40 What if ou want to align but have no prior matched pairs? Hough transform and RANSAC not applicable Important applications Medical imaging: match brain scans or contours Robotics: match point clouds

41 Iterative Closest Points (ICP) Algorithm Goal: estimate transform between two dense sets of points. Initialize transformation (e.g., compute difference in means and scale) 2. Assign each point in {Set } to its nearest neighbor in {Set 2} 3. Estimate transformation parameters e.g., least squares or robust least squares 4. Transform the points in {Set } using estimated parameters 5. Repeat steps 2-4 until change is ver small

42 Eample: aligning boundaries. Etract edge piels p.. p n and q.. q m 2. Compute initial transformation (e.g., compute translation and scaling b center of mass, variance within each image) 3. Get nearest neighbors: for each point p i find corresponding match(i) = argmin dist(pi, qj) j 4. Compute transformation T based on matches 5. Warp points p according to T 6. Repeat 3-5 until convergence p q

43 Eample: solving for translation (t, t ) Problem: no initial guesses for correspondence A i A i B i B i t t ICP solution. Find nearest neighbors for each point 2. Compute transform using matches 3. Move points using transform 4. Repeat steps -3 until convergence

44 Algorithm Summar Least Squares Fit closed form solution robust to noise not robust to outliers Robust Least Squares improves robustness to noise requires iterative optimization Hough transform robust to noise and outliers can fit multiple models onl works for a few parameters (-4 tpicall) RANSAC robust to noise and outliers works with a moderate number of parameters (e.g, -8) Iterative Closest Point (ICP) For local alignment onl: does not require initial correspondences

45 Object Instance Recognition. Match kepoints to object model 2. Solve for affine transformation parameters 3. Score b inliers and choose solutions with score above threshold A A 2 Matched kepoints A 3 Affine Parameters # Inliers This Class Choose hpothesis with ma score above threshold

46 N piels Overview of Kepoint Matching. Find a set of distinctive kepoints A A 2 A 3 2. Define a region around each kepoint N piels f A e.g. color d( f A, f B ) T f B e.g. color K. Grauman, B. Leibe 3. Etract and normalize the region content 4. Compute a local descriptor from the normalized region 5. Match local descriptors

47 Finding the objects (overview) Input Image Stored Image. Match interest points from input image to database image 2. Matched points vote for rough position/orientation/scale of object 3. Find position/orientation/scales that have at least three votes 4. Compute affine registration and matches using iterative least squares with outlier check 5. Report object if there are at least T matched points

48 Affine Object Model Accounts for 3D rotation of a surface under orthographic projection

49 Affine Object Model Accounts for 3D rotation of a surface under orthographic projection What is the minimum number of matched points that we need? f e d c b a f e d c b a

50 Finding the objects (SIFT, Lowe 2004). Match interest points from input image to database image 2. Get location/scale/orientation using Hough voting In training, each point has known position/scale/orientation wrt whole object Matched points vote for the position, scale, and orientation of the entire object Bins for,, scale, orientation Wide bins (0.25 object length in position, 2 scale, 30 degrees orientation) Vote for two closest bin centers in each direction (6 votes total) 3. Geometric verification For each bin with at least 3 kepoints Iterate between least squares fit and checking for inliers and outliers 4. Report object if > T inliers (T is tpicall 3, can be computed to match some probabilistic threshold)

51 Eamples of recognized objects

Multi-stable Perception. Necker Cube

Multi-stable Perception. Necker Cube Multi-stable Perception Necker Cube Spinning dancer illusion, Nobuuki Kaahara Fitting and Alignment Computer Vision Szeliski 6.1 James Has Acknowledgment: Man slides from Derek Hoiem, Lana Lazebnik, and

More information

10/03/11. Model Fitting. Computer Vision CS 143, Brown. James Hays. Slides from Silvio Savarese, Svetlana Lazebnik, and Derek Hoiem

10/03/11. Model Fitting. Computer Vision CS 143, Brown. James Hays. Slides from Silvio Savarese, Svetlana Lazebnik, and Derek Hoiem 10/03/11 Model Fitting Computer Vision CS 143, Brown James Hays Slides from Silvio Savarese, Svetlana Lazebnik, and Derek Hoiem Fitting: find the parameters of a model that best fit the data Alignment:

More information

Feature Matching and Robust Fitting

Feature Matching and Robust Fitting Feature Matching and Robust Fitting Computer Vision CS 143, Brown Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Project 2 questions? This

More information

Hough Transform and RANSAC

Hough Transform and RANSAC CS4501: Introduction to Computer Vision Hough Transform and RANSAC Various slides from previous courses by: D.A. Forsyth (Berkeley / UIUC), I. Kokkinos (Ecole Centrale / UCL). S. Lazebnik (UNC / UIUC),

More information

EECS 442 Computer vision. Fitting methods

EECS 442 Computer vision. Fitting methods EECS 442 Computer vision Fitting methods - Problem formulation - Least square methods - RANSAC - Hough transforms - Multi-model fitting - Fitting helps matching! Reading: [HZ] Chapters: 4, 11 [FP] Chapters:

More information

Image Warping. Many slides from Alyosha Efros + Steve Seitz. Photo by Sean Carroll

Image Warping. Many slides from Alyosha Efros + Steve Seitz. Photo by Sean Carroll Image Warping Man slides from Alosha Efros + Steve Seitz Photo b Sean Carroll Morphing Blend from one object to other with a series of local transformations Image Transformations image filtering: change

More information

Lecture 9 Fitting and Matching

Lecture 9 Fitting and Matching Lecture 9 Fitting and Matching Problem formulation Least square methods RANSAC Hough transforms Multi- model fitting Fitting helps matching! Reading: [HZ] Chapter: 4 Estimation 2D projective transformation

More information

Fitting. Instructor: Jason Corso (jjcorso)! web.eecs.umich.edu/~jjcorso/t/598f14!! EECS Fall 2014! Foundations of Computer Vision!

Fitting. Instructor: Jason Corso (jjcorso)! web.eecs.umich.edu/~jjcorso/t/598f14!! EECS Fall 2014! Foundations of Computer Vision! Fitting EECS 598-08 Fall 2014! Foundations of Computer Vision!! Instructor: Jason Corso (jjcorso)! web.eecs.umich.edu/~jjcorso/t/598f14!! Readings: FP 10; SZ 4.3, 5.1! Date: 10/8/14!! Materials on these

More information

Lecture 8 Fitting and Matching

Lecture 8 Fitting and Matching Lecture 8 Fitting and Matching Problem formulation Least square methods RANSAC Hough transforms Multi-model fitting Fitting helps matching! Reading: [HZ] Chapter: 4 Estimation 2D projective transformation

More information

Fitting a transformation: Feature-based alignment April 30 th, Yong Jae Lee UC Davis

Fitting a transformation: Feature-based alignment April 30 th, Yong Jae Lee UC Davis Fitting a transformation: Feature-based alignment April 3 th, 25 Yong Jae Lee UC Davis Announcements PS2 out toda; due 5/5 Frida at :59 pm Color quantization with k-means Circle detection with the Hough

More information

Instance-level recognition

Instance-level recognition Instance-level recognition 1) Local invariant features 2) Matching and recognition with local features 3) Efficient visual search 4) Very large scale indexing Matching of descriptors Matching and 3D reconstruction

More information

Miniature faking. In close-up photo, the depth of field is limited.

Miniature faking. In close-up photo, the depth of field is limited. Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg

More information

Instance-level recognition

Instance-level recognition Instance-level recognition 1) Local invariant features 2) Matching and recognition with local features 3) Efficient visual search 4) Very large scale indexing Matching of descriptors Matching and 3D reconstruction

More information

Keypoint-based Recognition and Object Search

Keypoint-based Recognition and Object Search 03/08/11 Keypoint-based Recognition and Object Search Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Notices I m having trouble connecting to the web server, so can t post lecture

More information

Homography estimation

Homography estimation RANSAC continued Homography estimation x w? ~x img H~x w Homography estimation? x img ~x w = H 1 ~x img Homography estimation (0,0) (1,0) (6.4,2.8) (8.0,2.9) (5.6, 4.0) (7.8, 4.2) (0,1) (1,1) Ah =0s.tkhk

More information

Image Warping CSE399b, Spring 07 Computer Vision

Image Warping CSE399b, Spring 07 Computer Vision Image Warping CSE399b, Spring 7 Computer Vision http://maps.a9.com http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html Autostiching on A9.com

More information

Prof. Kristen Grauman

Prof. Kristen Grauman Fitting Prof. Kristen Grauman UT Austin Fitting Want to associate a model with observed features [Fig from Marszalek & Schmid, 2007] For eample, the model could be a line, a circle, or an arbitrary shape.

More information

CS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017

CS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017 CS 277: Intro to Computer Vision Multiple Views Prof. Adriana Kovashka Universit of Pittsburgh March 4, 27 Plan for toda Affine and projective image transformations Homographies and image mosaics Stereo

More information

Image Warping. Some slides from Steve Seitz

Image Warping.   Some slides from Steve Seitz Image Warping http://www.jeffre-martin.com Some slides from Steve Seitz 5-463: Computational Photograph Aleei Efros, CMU, Spring 2 Image Transformations image filtering: change range of image g() = T(f())

More information

Image Warping. Computational Photography Derek Hoiem, University of Illinois 09/28/17. Photo by Sean Carroll

Image Warping. Computational Photography Derek Hoiem, University of Illinois 09/28/17. Photo by Sean Carroll Image Warping 9/28/7 Man slides from Alosha Efros + Steve Seitz Computational Photograph Derek Hoiem, Universit of Illinois Photo b Sean Carroll Reminder: Proj 2 due monda Much more difficult than project

More information

Instance-level recognition part 2

Instance-level recognition part 2 Visual Recognition and Machine Learning Summer School Paris 2011 Instance-level recognition part 2 Josef Sivic http://www.di.ens.fr/~josef INRIA, WILLOW, ENS/INRIA/CNRS UMR 8548 Laboratoire d Informatique,

More information

Image Warping. Some slides from Steve Seitz

Image Warping.   Some slides from Steve Seitz Image Warping http://www.jeffre-martin.com Some slides from Steve Seitz 5-463: Computational Photograph Aleei Efros, CMU, Fall 26 Image Warping image filtering: change range of image g() T(f()) f T f image

More information

Image Warping : Computational Photography Alexei Efros, CMU, Fall Some slides from Steve Seitz

Image Warping : Computational Photography Alexei Efros, CMU, Fall Some slides from Steve Seitz Image Warping http://www.jeffre-martin.com Some slides from Steve Seitz 5-463: Computational Photograph Aleei Efros, CMU, Fall 2 Image Transformations image filtering: change range of image g() T(f())

More information

Image Warping (Szeliski Sec 2.1.2)

Image Warping (Szeliski Sec 2.1.2) Image Warping (Szeliski Sec 2..2) http://www.jeffre-martin.com CS94: Image Manipulation & Computational Photograph Aleei Efros, UC Berkele, Fall 7 Some slides from Steve Seitz Image Transformations image

More information

Instance-level recognition II.

Instance-level recognition II. Reconnaissance d objets et vision artificielle 2010 Instance-level recognition II. Josef Sivic http://www.di.ens.fr/~josef INRIA, WILLOW, ENS/INRIA/CNRS UMR 8548 Laboratoire d Informatique, Ecole Normale

More information

Warping, Morphing and Mosaics

Warping, Morphing and Mosaics Computational Photograph and Video: Warping, Morphing and Mosaics Prof. Marc Pollefes Dr. Gabriel Brostow Toda s schedule Last week s recap Warping Morphing Mosaics Toda s schedule Last week s recap Warping

More information

Image Warping, mesh, and triangulation CSE399b, Spring 07 Computer Vision

Image Warping, mesh, and triangulation CSE399b, Spring 07 Computer Vision http://grail.cs.washington.edu/projects/rotoscoping/ Image Warping, mesh, and triangulation CSE399b, Spring 7 Computer Vision Man of the slides from A. Efros. Parametric (global) warping Eamples of parametric

More information

CS 558: Computer Vision 4 th Set of Notes

CS 558: Computer Vision 4 th Set of Notes 1 CS 558: Computer Vision 4 th Set of Notes Instructor: Philippos Mordohai Webpage: www.cs.stevens.edu/~mordohai E-mail: Philippos.Mordohai@stevens.edu Office: Lieb 215 Overview Keypoint matching Hessian

More information

Interactive Computer Graphics. Warping and morphing. Warping and Morphing. Warping and Morphing. Lecture 14+15: Warping and Morphing. What is.

Interactive Computer Graphics. Warping and morphing. Warping and Morphing. Warping and Morphing. Lecture 14+15: Warping and Morphing. What is. Interactive Computer Graphics Warping and morphing Lecture 14+15: Warping and Morphing Lecture 14: Warping and Morphing: Slide 1 Lecture 14: Warping and Morphing: Slide 2 Warping and Morphing What is Warping

More information

(More) Algorithms for Cameras: Edge Detec8on Modeling Cameras/Objects. Connelly Barnes

(More) Algorithms for Cameras: Edge Detec8on Modeling Cameras/Objects. Connelly Barnes (More) Algorithms for Cameras: Edge Detec8on Modeling Cameras/Objects Connelly Barnes Acknowledgment: Many slides from James Hays, also Derek Hoiem Grauman&Leibe 2008 Outline Edge Detec)on: Canny, etc.

More information

Last Lecture. Edge Detection. Filtering Pyramid

Last Lecture. Edge Detection. Filtering Pyramid Last Lecture Edge Detection Filtering Pramid Toda Motion Deblur Image Transformation Removing Camera Shake from a Single Photograph Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis and William T.

More information

How is project #1 going?

How is project #1 going? How is project # going? Last Lecture Edge Detection Filtering Pramid Toda Motion Deblur Image Transformation Removing Camera Shake from a Single Photograph Rob Fergus, Barun Singh, Aaron Hertzmann, Sam

More information

Feature Detectors and Descriptors: Corners, Lines, etc.

Feature Detectors and Descriptors: Corners, Lines, etc. Feature Detectors and Descriptors: Corners, Lines, etc. Edges vs. Corners Edges = maxima in intensity gradient Edges vs. Corners Corners = lots of variation in direction of gradient in a small neighborhood

More information

Feature Matching + Indexing and Retrieval

Feature Matching + Indexing and Retrieval CS 1699: Intro to Computer Vision Feature Matching + Indexing and Retrieval Prof. Adriana Kovashka University of Pittsburgh October 1, 2015 Today Review (fitting) Hough transform RANSAC Matching points

More information

CS4670: Computer Vision

CS4670: Computer Vision CS467: Computer Vision Noah Snavely Lecture 8: Geometric transformations Szeliski: Chapter 3.6 Reading Announcements Project 2 out today, due Oct. 4 (demo at end of class today) Image alignment Why don

More information

Announcements, schedule. Lecture 8: Fitting. Weighted graph representation. Outline. Segmentation by Graph Cuts. Images as graphs

Announcements, schedule. Lecture 8: Fitting. Weighted graph representation. Outline. Segmentation by Graph Cuts. Images as graphs Announcements, schedule Lecture 8: Fitting Tuesday, Sept 25 Grad student etensions Due of term Data sets, suggestions Reminder: Midterm Tuesday 10/9 Problem set 2 out Thursday, due 10/11 Outline Review

More information

Automatic Image Alignment (feature-based)

Automatic Image Alignment (feature-based) Automatic Image Alignment (feature-based) Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2006 Today s lecture Feature

More information

Image warping. image filtering: change range of image. image warping: change domain of image g(x) = f(h(x)) h(y)=0.5y+0.5. h([x,y])=[x,y/2] f h

Image warping. image filtering: change range of image. image warping: change domain of image g(x) = f(h(x)) h(y)=0.5y+0.5. h([x,y])=[x,y/2] f h Image warping Image warping image filtering: change range of image g() () = h(f()) h(f()) f h g h()=0.5+0.5 image warping: change domain of image g() = f(h()) f h g h([,])=[,/2] Parametric (global) warping

More information

Straight Lines and Hough

Straight Lines and Hough 09/30/11 Straight Lines and Hough Computer Vision CS 143, Brown James Hays Many slides from Derek Hoiem, Lana Lazebnik, Steve Seitz, David Forsyth, David Lowe, Fei-Fei Li Project 1 A few project highlights

More information

Midterm Wed. Local features: detection and description. Today. Last time. Local features: main components. Goal: interest operator repeatability

Midterm Wed. Local features: detection and description. Today. Last time. Local features: main components. Goal: interest operator repeatability Midterm Wed. Local features: detection and description Monday March 7 Prof. UT Austin Covers material up until 3/1 Solutions to practice eam handed out today Bring a 8.5 11 sheet of notes if you want Review

More information

Homographies and RANSAC

Homographies and RANSAC Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position

More information

CSE 527: Introduction to Computer Vision

CSE 527: Introduction to Computer Vision CSE 527: Introduction to Computer Vision Week 5 - Class 1: Matching, Stitching, Registration September 26th, 2017 ??? Recap Today Feature Matching Image Alignment Panoramas HW2! Feature Matches Feature

More information

Fitting: The Hough transform

Fitting: The Hough transform Fitting: The Hough transform Voting schemes Let each feature vote for all the models that are compatible with it Hopefully the noise features will not vote consistently for any single model Missing data

More information

Scale Invariant Feature Transform (SIFT) CS 763 Ajit Rajwade

Scale Invariant Feature Transform (SIFT) CS 763 Ajit Rajwade Scale Invariant Feature Transform (SIFT) CS 763 Ajit Rajwade What is SIFT? It is a technique for detecting salient stable feature points in an image. For ever such point it also provides a set of features

More information

Fitting: The Hough transform

Fitting: The Hough transform Fitting: The Hough transform Voting schemes Let each feature vote for all the models that are compatible with it Hopefully the noise features will not vote consistently for any single model Missing data

More information

Chaplin, Modern Times, 1936

Chaplin, Modern Times, 1936 Chaplin, Modern Times, 1936 [A Bucket of Water and a Glass Matte: Special Effects in Modern Times; bonus feature on The Criterion Collection set] Multi-view geometry problems Structure: Given projections

More information

Fitting: The Hough transform

Fitting: The Hough transform Fitting: The Hough transform Voting schemes Let each feature vote for all the models that are compatible with it Hopefully the noise features will not vote consistently for any single model Missing data

More information

Image Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi

Image Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Image Stitching Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Combine two or more overlapping images to make one larger image Add example Slide credit: Vaibhav Vaish

More information

Wikipedia - Mysid

Wikipedia - Mysid Wikipedia - Mysid Erik Brynjolfsson, MIT Filtering Edges Corners Feature points Also called interest points, key points, etc. Often described as local features. Szeliski 4.1 Slides from Rick Szeliski,

More information

Determining the 2d transformation that brings one image into alignment (registers it) with another. And

Determining the 2d transformation that brings one image into alignment (registers it) with another. And Last two lectures: Representing an image as a weighted combination of other images. Toda: A different kind of coordinate sstem change. Solving the biggest problem in using eigenfaces? Toda Recognition

More information

Fitting. Fitting. Slides S. Lazebnik Harris Corners Pkwy, Charlotte, NC

Fitting. Fitting. Slides S. Lazebnik Harris Corners Pkwy, Charlotte, NC Fitting We ve learned how to detect edges, corners, blobs. Now what? We would like to form a higher-level, more compact representation of the features in the image by grouping multiple features according

More information

Image Features: Local Descriptors. Sanja Fidler CSC420: Intro to Image Understanding 1/ 58

Image Features: Local Descriptors. Sanja Fidler CSC420: Intro to Image Understanding 1/ 58 Image Features: Local Descriptors Sanja Fidler CSC420: Intro to Image Understanding 1/ 58 [Source: K. Grauman] Sanja Fidler CSC420: Intro to Image Understanding 2/ 58 Local Features Detection: Identify

More information

Computer Vision Lecture 20

Computer Vision Lecture 20 Computer Vision Lecture 2 Motion and Optical Flow Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de 28.1.216 Man slides adapted from K. Grauman, S. Seitz, R. Szeliski,

More information

Affine and Projective Transformations

Affine and Projective Transformations CS 674: Intro to Computer Vision Affine and Projective Transformations Prof. Adriana Kovaska Universit of Pittsburg October 3, 26 Alignment problem We previousl discussed ow to matc features across images,

More information

EE795: Computer Vision and Intelligent Systems

EE795: Computer Vision and Intelligent Systems EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 10 130221 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Canny Edge Detector Hough Transform Feature-Based

More information

Feature Matching and RANSAC

Feature Matching and RANSAC Feature Matching and RANSAC Recognising Panoramas. [M. Brown and D. Lowe,ICCV 2003] [Brown, Szeliski, Winder, CVPR 2005] with a lot of slides stolen from Steve Seitz, Rick Szeliski, A. Efros Introduction

More information

CAP 5415 Computer Vision Fall 2012

CAP 5415 Computer Vision Fall 2012 CAP 5415 Computer Vision Fall 2012 Hough Transform Lecture-18 Sections 4.2, 4.3 Fundamentals of Computer Vision Image Feature Extraction Edges (edge pixels) Sobel, Roberts, Prewit Laplacian of Gaussian

More information

Lecture: RANSAC and feature detectors

Lecture: RANSAC and feature detectors Lecture: RANSAC and feature detectors Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 1 What we will learn today? A model fitting method for edge detection RANSAC Local invariant

More information

Stereo and Epipolar geometry

Stereo and Epipolar geometry Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka

More information

Model Fitting, RANSAC. Jana Kosecka

Model Fitting, RANSAC. Jana Kosecka Model Fitting, RANSAC Jana Kosecka Fitting: Overview If we know which points belong to the line, how do we find the optimal line parameters? Least squares What if there are outliers? Robust fitting, RANSAC

More information

Methods for Representing and Recognizing 3D objects

Methods for Representing and Recognizing 3D objects Methods for Representing and Recognizing 3D objects part 1 Silvio Savarese University of Michigan at Ann Arbor Object: Building, 45º pose, 8-10 meters away Object: Person, back; 1-2 meters away Object:

More information

Local Image Features

Local Image Features Local Image Features Computer Vision Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Flashed Face Distortion 2nd Place in the 8th Annual Best

More information

Perception IV: Place Recognition, Line Extraction

Perception IV: Place Recognition, Line Extraction Perception IV: Place Recognition, Line Extraction Davide Scaramuzza University of Zurich Margarita Chli, Paul Furgale, Marco Hutter, Roland Siegwart 1 Outline of Today s lecture Place recognition using

More information

CS231A Midterm Review. Friday 5/6/2016

CS231A Midterm Review. Friday 5/6/2016 CS231A Midterm Review Friday 5/6/2016 Outline General Logistics Camera Models Non-perspective cameras Calibration Single View Metrology Epipolar Geometry Structure from Motion Active Stereo and Volumetric

More information

Image correspondences and structure from motion

Image correspondences and structure from motion Image correspondences and structure from motion http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 20 Course announcements Homework 5 posted.

More information

Local Image Features

Local Image Features Local Image Features Computer Vision CS 143, Brown Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial This section: correspondence and alignment

More information

CS 231A Computer Vision (Winter 2014) Problem Set 3

CS 231A Computer Vision (Winter 2014) Problem Set 3 CS 231A Computer Vision (Winter 2014) Problem Set 3 Due: Feb. 18 th, 2015 (11:59pm) 1 Single Object Recognition Via SIFT (45 points) In his 2004 SIFT paper, David Lowe demonstrates impressive object recognition

More information

Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac

Image stitching. Digital Visual Effects Yung-Yu Chuang. with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Digital Visual Effects Yung-Yu Chuang with slides by Richard Szeliski, Steve Seitz, Matthew Brown and Vaclav Hlavac Image stitching Stitching = alignment + blending geometrical registration

More information

Warping. 12 May 2015

Warping. 12 May 2015 Warping 12 May 2015 Warping, morphing, mosaic Slides from Durand and Freeman (MIT), Efros (CMU, Berkeley), Szeliski (MSR), Seitz (UW), Lowe (UBC) http://szeliski.org/book/ 2 Image Warping Image filtering:

More information

RANSAC and some HOUGH transform

RANSAC and some HOUGH transform RANSAC and some HOUGH transform Thank you for the slides. They come mostly from the following source Dan Huttenlocher Cornell U Matching and Fitting Recognition and matching are closely related to fitting

More information

Prof. Feng Liu. Winter /05/2019

Prof. Feng Liu. Winter /05/2019 Prof. Feng Liu Winter 2019 http://www.cs.pd.edu/~fliu/courses/cs410/ 02/05/2019 Last Time Image alignment 2 Toda Image warping The slides for this topic are used from Prof. Yung-Yu Chuang, which use materials

More information

Image warping and stitching

Image warping and stitching Image warping and stitching May 4 th, 2017 Yong Jae Lee UC Davis Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 2 Alignment problem In alignment, we will

More information

Local features: detection and description. Local invariant features

Local features: detection and description. Local invariant features Local features: detection and description Local invariant features Detection of interest points Harris corner detection Scale invariant blob detection: LoG Description of local patches SIFT : Histograms

More information

CSE328 Fundamentals of Computer Graphics: Theory, Algorithms, and Applications

CSE328 Fundamentals of Computer Graphics: Theory, Algorithms, and Applications CSE328 Fundamentals of Computer Graphics: Theor, Algorithms, and Applications Hong in State Universit of New York at Ston Brook (Ston Brook Universit) Ston Brook, New York 794-44 Tel: (63)632-845; Fa:

More information

Motion illusion, rotating snakes

Motion illusion, rotating snakes Motion illusion, rotating snakes Local features: main components 1) Detection: Find a set of distinctive key points. 2) Description: Extract feature descriptor around each interest point as vector. x 1

More information

Image warping and stitching

Image warping and stitching Image warping and stitching May 5 th, 2015 Yong Jae Lee UC Davis PS2 due next Friday Announcements 2 Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 3 Alignment

More information

Computer Vision Lecture 13

Computer Vision Lecture 13 Computer Vision Lecture 3 Recognition wit Local Features 5.2.205 astian Leibe RWTH acen ttp://www.vision.rwt-aacen.de/ leibe@vision.rwt-aacen.de Course Outline Image Processing asics Segmentation & Grouping

More information

Image warping/morphing

Image warping/morphing Image warping/morphing Digital Visual Effects, Spring 2007 Yung-Yu Chuang 2007/3/20 with slides b Richard Szeliski, Steve Seitz, Tom Funkhouser and Aleei Efros Image warping Image formation B A Sampling

More information

Feature Based Registration - Image Alignment

Feature Based Registration - Image Alignment Feature Based Registration - Image Alignment Image Registration Image registration is the process of estimating an optimal transformation between two or more images. Many slides from Alexei Efros http://graphics.cs.cmu.edu/courses/15-463/2007_fall/463.html

More information

Structure from Motion

Structure from Motion /8/ Structure from Motion Computer Vision CS 43, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from motion

More information

Model Fitting. Introduction to Computer Vision CSE 152 Lecture 11

Model Fitting. Introduction to Computer Vision CSE 152 Lecture 11 Model Fitting CSE 152 Lecture 11 Announcements Homework 3 is due May 9, 11:59 PM Reading: Chapter 10: Grouping and Model Fitting What to do with edges? Segment linked edge chains into curve features (e.g.,

More information

Automatic Image Alignment

Automatic Image Alignment Automatic Image Alignment Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 Live Homography DEMO Check out panoramio.com

More information

An edge is not a line... Edge Detection. Finding lines in an image. Finding lines in an image. How can we detect lines?

An edge is not a line... Edge Detection. Finding lines in an image. Finding lines in an image. How can we detect lines? Edge Detection An edge is not a line... original image Cann edge detector Compute image derivatives if gradient magnitude > τ and the value is a local maimum along gradient direction piel is an edge candidate

More information

Automatic Image Alignment

Automatic Image Alignment Automatic Image Alignment with a lot of slides stolen from Steve Seitz and Rick Szeliski Mike Nese CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2018 Live Homography

More information

Object Reconstruction

Object Reconstruction B. Scholz Object Reconstruction 1 / 39 MIN-Fakultät Fachbereich Informatik Object Reconstruction Benjamin Scholz Universität Hamburg Fakultät für Mathematik, Informatik und Naturwissenschaften Fachbereich

More information

CS 664 Image Matching and Robust Fitting. Daniel Huttenlocher

CS 664 Image Matching and Robust Fitting. Daniel Huttenlocher CS 664 Image Matching and Robust Fitting Daniel Huttenlocher Matching and Fitting Recognition and matching are closely related to fitting problems Parametric fitting can serve as more restricted domain

More information

Uncertainties: Representation and Propagation & Line Extraction from Range data

Uncertainties: Representation and Propagation & Line Extraction from Range data 41 Uncertainties: Representation and Propagation & Line Extraction from Range data 42 Uncertainty Representation Section 4.1.3 of the book Sensing in the real world is always uncertain How can uncertainty

More information

Computing F class 13. Multiple View Geometry. Comp Marc Pollefeys

Computing F class 13. Multiple View Geometry. Comp Marc Pollefeys Computing F class 3 Multiple View Geometr Comp 90-089 Marc Pollefes Multiple View Geometr course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective D Geometr Jan. 4, 6 (no class) Projective

More information

INFO0948 Fitting and Shape Matching

INFO0948 Fitting and Shape Matching INFO0948 Fitting and Shape Matching Renaud Detry University of Liège, Belgium Updated March 31, 2015 1 / 33 These slides are based on the following book: D. Forsyth and J. Ponce. Computer vision: a modern

More information

Recognition Tools: Support Vector Machines

Recognition Tools: Support Vector Machines CS 2770: Computer Vision Recognition Tools: Support Vector Machines Prof. Adriana Kovashka University of Pittsburgh January 12, 2017 Announcement TA office hours: Tuesday 4pm-6pm Wednesday 10am-12pm Matlab

More information

Surface Registration. Gianpaolo Palma

Surface Registration. Gianpaolo Palma Surface Registration Gianpaolo Palma The problem 3D scanning generates multiple range images Each contain 3D points for different parts of the model in the local coordinates of the scanner Find a rigid

More information

Fi#ng & Matching Region Representa3on Image Alignment, Op3cal Flow

Fi#ng & Matching Region Representa3on Image Alignment, Op3cal Flow Fi#ng & Matching Region Representa3on Image Alignment, Op3cal Flow Lectures 5 & 6 Prof. Fergus Slides from: S. Lazebnik, S. Seitz, M. Pollefeys, A. Effros. Facebook 360 photos Panoramas How do we build

More information

Previously. Part-based and local feature models for generic object recognition. Bag-of-words model 4/20/2011

Previously. Part-based and local feature models for generic object recognition. Bag-of-words model 4/20/2011 Previously Part-based and local feature models for generic object recognition Wed, April 20 UT-Austin Discriminative classifiers Boosting Nearest neighbors Support vector machines Useful for object recognition

More information

Prof. Noah Snavely CS Administrivia. A4 due on Friday (please sign up for demo slots)

Prof. Noah Snavely CS Administrivia. A4 due on Friday (please sign up for demo slots) Robust fitting Prof. Noah Snavely CS111 http://www.cs.cornell.edu/courses/cs111 Administrivia A due on Friday (please sign up for demo slots) A5 will be out soon Prelim is coming up, Tuesday, / Roadmap

More information

3D Environment Reconstruction

3D Environment Reconstruction 3D Environment Reconstruction Using Modified Color ICP Algorithm by Fusion of a Camera and a 3D Laser Range Finder The 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems October 11-15,

More information

Introduction to Computer Vision

Introduction to Computer Vision Introduction to Computer Vision Michael J. Black Oct 2009 Motion estimation Goals Motion estimation Affine flow Optimization Large motions Why affine? Monday dense, smooth motion and regularization. Robust

More information

Transformations Between Two Images. Translation Rotation Rigid Similarity (scaled rotation) Affine Projective Pseudo Perspective Bi linear

Transformations Between Two Images. Translation Rotation Rigid Similarity (scaled rotation) Affine Projective Pseudo Perspective Bi linear Transformations etween Two Images Translation Rotation Rigid Similarit (scaled rotation) ffine Projective Pseudo Perspective i linear Fundamental Matri Lecture 13 pplications Stereo Structure from Motion

More information

Fundamental Matrix. Lecture 13

Fundamental Matrix. Lecture 13 Fundamental Matri Lecture 13 Transformations etween Two Images Translation Rotation Rigid Similarit (scaled rotation) ffine Projective Pseudo Perspective i linear pplications Stereo Structure from Motion

More information

Object Recognition with Invariant Features

Object Recognition with Invariant Features Object Recognition with Invariant Features Definition: Identify objects or scenes and determine their pose and model parameters Applications Industrial automation and inspection Mobile robots, toys, user

More information

CS664 Lecture #16: Image registration, robust statistics, motion

CS664 Lecture #16: Image registration, robust statistics, motion CS664 Lecture #16: Image registration, robust statistics, motion Some material taken from: Alyosha Efros, CMU http://www.cs.cmu.edu/~efros Xenios Papademetris http://noodle.med.yale.edu/~papad/various/papademetris_image_registration.p

More information