Computer Vision I - Appearance-based Matching and Projective Geometry
|
|
|
- Sheryl Sharp
- 5 years ago
- Views:
Transcription
1 Computer Vision I - Appearance-based Matching and Projective Geometry Carsten Rother 01/11/2016 Computer Vision I: Image Formation Process
2 Roadmap for next four lectures Computer Vision I: Image Formation Process 01/11/ Appearance-based Matching (sec. 4.1) Projective Geometry - Basics (sec ) Geometry of a Single Camera (sec 2.1.5, 2.1.6) Camera versus Human Perception The Pinhole Camera Lens effects Geometry of two Views (sec. 7.2) The Homography (e.g. rotating camera) Camera Calibration (3D to 2D Mapping) The Fundamental and Essential Matrix (two arbitrary images) Robust Geometry estimation for two cameras (sec ) Multi-View 3D reconstruction (sec ) General scenario From Projective to Metric Space Special Cases
3 Objective Computer Vision I: Image Formation Process 01/11/ Input: Two images which have some common scene geometry. Assume the common scene part is textured. Goal: Match interest points of the common scene geometry Matching of objects which are texture-less is harder (later lecture)
4 Examples: Appearance-based matching Computer Vision I: Image Formation Process 01/11/2016 4
5 Applications Computer Vision I: Image Formation Process 01/11/ D reconstruction: Augmented Realty: Panoramic Stitching: Robotics Camera re-localization Grasping known objects
6 Matching Points between two Images Computer Vision I: Image Formation Process 01/11/ Find interest points Find orientated patches around interest points to capture appearance Encode patch appearance in a descriptor Find matching patches according to appearance (similar descriptors) Verify matching patches according to geometry (later lecture) v
7 Matching Points between two Images Computer Vision I: Image Formation Process 01/11/ Find interest points Find orientated patches around interest points to capture appearance Encode patch appearance in a descriptor Find matching patches according to appearance (similar descriptors) Verify matching patches according to geometry (later lecture) v
8 Reminder: Harris Corner Detector Computer Vision I: Image Formation Process 01/11/ Compute: 1. Based on so-called auto-correlation function: 2. Compute the two eigenvalues of Q, i.e. λ 1 λ 2 3. Harris measure: 4. Take those points (after non-max suppression) with high H value
9 Matching Points between two Images Computer Vision I: Image Formation Process 01/11/ Find interest points Find orientated patches around interest points to capture appearance Encode patch appearance in a descriptor Find matching patches according to appearance (similar descriptors) Verify matching patches according to geometry (later lecture) v
10 How to deal with orientation Computer Vision I: Image Formation Process 01/11/ Orientate with image gradient:
11 Choose a patch around each point Computer Vision I: Image Formation Process 01/11/ How to deal with scale?
12 Choose a patch around each point Computer Vision I: Image Formation Process 01/11/ How to deal with scale?
13 Choose a patch around each point Computer Vision I: Image Formation Process 01/11/ How to deal with scale?
14 Reminder: Edge detection via image gradient Computer Vision I: Image Formation Process 01/11/ Image gradient: ( (D x G) I, (D y G) I) Image Result of (using small sigma for Gaussian) Result of (using large sigma for Gaussian) Final result with canny edge detector
15 Alterative Edge Detector via LoG Operator Computer Vision I: Image Formation Process 01/11/ To find an edge we first smooth is called the LoG (Laplacian of Gaussian operator) 1D example: 2D example (Mexican hat): Find zero-crossing
16 Alterative: Edge detection with LoG Filter Computer Vision I: Image Formation Process 01/11/ Called Laplacian of Gaussian (LoG)
17 Scale selection (illustration) Computer Vision I: Image Formation Process 01/11/ f is Laplacian of Gaussian (LoG) operator. Measures an average edgeness in all directions 2 G(σ) I = 2 (G(σ) I) x (G(σ) I) y 2 (details on page 191)
18 Scale selection (illustration) Computer Vision I: Image Formation Process 01/11/
19 Scale selection (illustration) Computer Vision I: Image Formation Process 01/11/
20 Scale selection (illustration) Computer Vision I: Image Formation Process 01/11/
21 Scale selection (illustration) Computer Vision I: Image Formation Process 01/11/ We could match up these curves and find unique corresponding points
22 Scale selection (illustration) Computer Vision I: Image Formation Process 01/11/ Simpler: Find maxima of the curve
23 Matching Points between two Images Computer Vision I: Image Formation Process 01/11/ Find interest points Find orientated patches around interest points to capture appearance Encode patch appearance in a descriptor Find matching patches according to appearance (similar descriptors) Verify matching patches according to geometry v
24 SIFT features (Scale Invariant Feature Transform) 64 pixels Computer Vision I: Image Formation Process 01/11/ v 64 pixels 4*4=16 cells Each cell has an 8-bin histogram In total: 16*8 values, i.e. 128D vector A cell has 16x16 pixels (here 8x8 for illustration only) (blue circle shows center weighting) [Lowe 2004]
25 Question Computer Vision I: Image Formation Process 01/11/ Assume you want to create a 3D reconstruction form a set of photos form the internet What are important properties for a feature descriptor: 1) Fast to compute 2) Can handle large changes in viewpoint 3) Can handle photometric changes (e.g. day versus night) 4) All of the above
26 SIFT feature is very popular Computer Vision I: Image Formation Process 01/11/ Fast to compute Can handle large changes in viewpoint well (up to 60 o out-of-plane rotation) Can handle photometric changes (even day versus night)
27 Many other feature descriptors Computer Vision I: Image Formation Process 01/11/ MOPS [Brown, Szeliski and Winder 2005] SURF [Herbert Bay et al. 2006] DAISY [Tola, Lepetit, Fua 2010] Shape Context LIFT (Deep Learning) DAISY
28 Matching Points between two Images Computer Vision I: Image Formation Process 01/11/ Find interest points Find orientated patches around interest points to capture appearance Encode patch appearance in a descriptor Find matching patches according to appearance (similar descriptors) Verify matching patches according to geometry (later lecture) v
29 Find matching patches fast Computer Vision I: Image Formation Process 01/11/ N patches (e.g. N = 1000) N patches (e.g. N = 1000) Goal: 1) Find for each patch in left image the closest patch in right image, in terms of similarity of descriptors 2) Accept all those matches that are close enough Methods: Naïve: N 2 tests (e.g. 1 Million) Hashing KD-tree; on average NlogN tests (e.g. ~10,000) Patch (Hashing Function) Index for Patch in DB
30 Matching Points between two Images Computer Vision I: Image Formation Process 01/11/ Find interest points Find orientated patches around interest points to capture appearance Encode patch appearance in a descriptor Find matching patches according to appearance (similar descriptors) Verify matching patches according to geometry (later lecture) v
31 Roadmap for next four lectures Computer Vision I: Image Formation Process 01/11/ Appearance-based Matching (sec. 4.1) Projective Geometry - Basics (sec ) Geometry of a Single Camera (sec 2.1.5, 2.1.6) Camera versus Human Perception The Pinhole Camera Lens effects Geometry of two Views (sec. 7.2) The Homography (e.g. rotating camera) Camera Calibration (3D to 2D Mapping) The Fundamental and Essential Matrix (two arbitrary images) Robust Geometry estimation for two cameras (sec ) Multi-View 3D reconstruction (sec ) General scenario From Projective to Metric Space Special Cases
32 Some Basics Computer Vision I: Image Formation Process 01/11/ Real coordinate space R 2 example: Real coordinate space R 3 example: Operations we need are: scalar product: x y = x t y = x 1 y 1 + x 2 y 2 + x 3 y 3 where x = cross- or vector-product: x y = x y x 1 x 2 x 3 x = 0 x 3 x 2 x 3 0 x 1 x 2 x 1 0 2
33 Euclidean Space Computer Vision I: Image Formation Process 11/11/ Euclidean Space R 2 and R 3 have angles and distances defined x y Angle defined as: θ = arccos x y Length of the vector x: x 2 = x x = n 2 i x i Comment, x implicitly x 2 means Distance of two vectors: d x, y = x y 2 = i n x i y i 2 x x y Θ y origin
34 Projective Space Computer Vision I: Image Formation Process 01/11/ D Point in a real coordinate space: 1 2 R 2 has 2 DoF (degrees of freedom) 3D Point in a real coordinate space: R3 has 3 DoF Definition: A point in 2-dimensional projective space P 2 is defined as p = x y w P 2, such that all vectors kx ky kw k 0 define the same point p in P 2 (equivalent classes) Sometimes written as: We write as: = ~ P
35 Projective Space - Visualization Computer Vision I: Image Formation Process 01/11/ Definition: A point in 2-dimensional projective space P 2 is defined as p = x y w P 2, such that all vectors A point in P 2 is a ray in R 3 that goes through the origin: kx ky kw k 0 define the same point p in P 2 (equivalent classes) w-axis x y w All rays go through (0,0,0) define a point in P 2 Plane w=1 Plane w=0 x y 0
36 Projective Space Computer Vision I: Image Formation Process 01/11/ All points in P 2 are given by: R 3 \ A point x y w P 2 has 2 DoF (3 elements but norm of vector can be set to 1) w-axis x y w All rays go through (0,0,0) define a point in P 2 Plane w=1 Plane w=0 x y 0
37 From R 2 to P 2 and back Computer Vision I: Image Formation Process 01/11/ From R 2 to P 2 : ~ p = x y R 2 p = y x 1 - a point in inhomogeneous coordinates - we soemtimes write ~ p for inhomogeneous coordinates From P 2 to R 2 : P 2 - a point in homogeneous coordinates x y w P 2 what does it mean if w=0? x/w y/w R2 for w 0 We can do similar transformations with all primitives (points, lines, planes)
38 Question Computer Vision I: Image Formation Process 01/11/ How do you convert a point p = y x P 2 back to p = x y R 2? z 1) x = x; y = y 2) x = z x ; y = z x 3) x = x ; z y = y z 4) Ist not possible 5) I don t know ~
39 From R 2 to P 2 and back Computer Vision I: Image Formation Process 01/11/ From R 2 to P 2 : ~ p = x y R 2 p = y x 1 - a point in inhomogeneous coordinates - we soemtimes write ~ p for inhomogeneous coordinates From P 2 to R 2 : P 2 - a point in homogeneous coordinates x y w P 2 what does it mean if w = 0? x/w y/w R2 for w 0 We can do similar transformations with all primitives (points, lines, planes)
40 From R 2 to P 2 and back: Example Computer Vision I: Image Formation Process 01/11/ From R 2 to P 2 : ~ p = x y R 2 p = y x 1 P 2 - a point in inhomogeneous coordinates - a point in homogeneous coordinates p ~ = 3 3 ~ 2 R 2 p = = = P 2 p = 6/2 w-axis (6,4,2) 4/2 R 2 (3,2,1) Plane w=1 (Space R 2 ) Plane w=0
41 Things we would like to have and do Computer Vision I: Image Formation Process 01/11/ We look at these operations in: R 2 /R 3, P 2 /P 3 Primitives: Points in 2D/3D Lines in 2D/3D Planes in 3D Conic in 2D; Quadric in 3D Operations with Primitives: Intersection Tangent Transformations: Rotation Translation Projective.
42 Why bother about P 2? Computer Vision I: Image Formation Process 01/11/ All Primitives, operations and transformations are defined in R 2 and P 2 Advantage of P 2 : Many transformation and operations are written more compactly (e.g. linear transformations) We will introduce new special primitives that are useful when dealing with parallelism This example will come later in detail. In P 2 they meet in a point at inifinty Two parallel lines in R 2
43 Points at infinity Computer Vision I: Image Formation Process 01/11/ w-axis x y w All rays go through (0,0,0) define a point in P 2 Plane w=1 Points with coordinate x y 0 Plane w=0 are so-called ideal points or points at infinity x y 0 x y 0 P 2 Not defined in R 2 since w = 0
44 Lines in R 2 Computer Vision I: Image Formation Process 01/11/ For Lines in coordinate space R 2 we can write l = (n x, n y, d) with n = n x, n y t as normal vector ( n = 1) and d is distance of line to origin A line has 2 DoF A point (x, y) lies on l if: n x x + n y y + d = 0 The normal can also be encoded with an angle θ: n = cos θ, sin θ t
45 Lines in P 2 Computer Vision I: Image Formation Process 11/11/ Points in P 2 : x = (x, y, w) Lines in P 2 : l = a, b, c Again, equivalent class: a, b, c = ka, kb, kc k 0 l = (0,0,0) is not a valid line in P 2 Lines in P 2 also have 2 DoF All points x, y, w on the line (a, b, c) satisfy: ax + by + cw = 0 think of each line being represented by a vector (a, b, c) this is the equation of a plane in R 3 with normal (a,b,c) going through (0,0,0)
46 Converting Lines between P 2 and R 2 Computer Vision I: Image Formation Process 01/11/ Points in P 2 : x = (x, y, w) Lines in P 2 : l = (a, b, c) From P 2 to R 2 : l = (ka, kb, kc) chose k such that (ka, kb) = 1 From R 2 to P 2 : l = n x, n y, d is already a line in P 2
47 Question Computer Vision I: Image Formation Process 01/11/ All points at infinity x, y, 0 lie on the line at infinity. What is the formula for the line at infinity : 1. l = 0,0,0 2. l = 0,0,1 3. l = 0,1,0 4. l = 1,0,0 5. l = 1,1,1 6. I don t know
48 Line at Infininty Computer Vision I: Image Formation Process 01/11/ There is a special line, called line at infinity: (0,0,1) All points at infinity x, y, 0 lie on the line at infinity (0,0,1): x 0 + y = 0 w-axis vector for line at infinity Plane w=1 Plane w=0 x y 0 A point at infinity (w=0)
49 A line is defined by two points in P 2 Computer Vision I: Image Formation Process 01/11/ The line through two points x and x is given by l = x x Proof: It is: x x x = x x x = 0 This is the same as: x l = x l = 0 vectors are orthogonal 2 Hence, the line l goes through points x and x
50 The Intersection of two lines in P 2 Computer Vision I: Image Formation Process 01/11/ Intersection of two lines l and l is the point x = l l Proof: It is: l l l = l l l = 0 This is the same as: lx = l x = 0 vectors are orthogonal 2 Hence, the point x lies onthe lines l and l Note the Theorem and Proofs have been very similiar, we only interchanged meaning of points and lines
51 Duality of points and lines Computer Vision I: Image Formation Process 01/11/ Note lx = xl = 0 (x and l are interchangeable ) Duality Theorem: To any theorem of 2D projective geometry there corresponds a dual theorem which may be derived by interchanging the roles of points and lines in the original theorem. The intersection of two lines l and l is the point x = l l The line through two points x and x is the line l = x x
52 Parallel lines meet in a point at infinity Computer Vision I: Image Formation Process 01/11/ l l l = ; l = (-1,1,1) (-1,0,1) l l In R 2 (Plane w = 1) y (-1/2,1,1) (-1/2,0,1) x intersection l l = = Point at infinty
Computer Vision I - Appearance-based Matching and Projective Geometry
Computer Vision I - Appearance-based Matching and Projective Geometry Carsten Rother 05/11/2015 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation
Computer Vision I - Image Matching and Image Formation
Computer Vision I - Image Matching and Image Formation Carsten Rother 10/12/2014 Computer Vision I: Image Formation Process Computer Vision I: Image Formation Process 10/12/2014 2 Roadmap for next five
Computer Vision I - Algorithms and Applications: Image Formation Process Part 2
Computer Vision I - Algorithms and Applications: Image Formation Process Part 2 Carsten Rother 17/11/2013 Computer Vision I: Image Formation Process Computer Vision I: Image Formation Process 17/11/2013
3D Computer Vision II. Reminder Projective Geometry, Transformations
3D Computer Vision II Reminder Projective Geometry, Transformations Nassir Navab" based on a course given at UNC by Marc Pollefeys & the book Multiple View Geometry by Hartley & Zisserman" October 21,
CSE 252B: Computer Vision II
CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribe: Sameer Agarwal LECTURE 1 Image Formation 1.1. The geometry of image formation We begin by considering the process of image formation when a
CS-9645 Introduction to Computer Vision Techniques Winter 2019
Table of Contents Projective Geometry... 1 Definitions...1 Axioms of Projective Geometry... Ideal Points...3 Geometric Interpretation... 3 Fundamental Transformations of Projective Geometry... 4 The D
Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction
Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction Carsten Rother 09/12/2013 Computer Vision I: Multi-View 3D reconstruction Roadmap this lecture Computer Vision I: Multi-View
Feature descriptors and matching
Feature descriptors and matching Detections at multiple scales Invariance of MOPS Intensity Scale Rotation Color and Lighting Out-of-plane rotation Out-of-plane rotation Better representation than color:
Local features: detection and description. Local invariant features
Local features: detection and description Local invariant features Detection of interest points Harris corner detection Scale invariant blob detection: LoG Description of local patches SIFT : Histograms
Local features and image matching. Prof. Xin Yang HUST
Local features and image matching Prof. Xin Yang HUST Last time RANSAC for robust geometric transformation estimation Translation, Affine, Homography Image warping Given a 2D transformation T and a source
Visual Recognition: Image Formation
Visual Recognition: Image Formation Raquel Urtasun TTI Chicago Jan 5, 2012 Raquel Urtasun (TTI-C) Visual Recognition Jan 5, 2012 1 / 61 Today s lecture... Fundamentals of image formation You should know
Computer Vision for HCI. Topics of This Lecture
Computer Vision for HCI Interest Points Topics of This Lecture Local Invariant Features Motivation Requirements, Invariances Keypoint Localization Features from Accelerated Segment Test (FAST) Harris Shi-Tomasi
Automatic Image Alignment (feature-based)
Automatic Image Alignment (feature-based) Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2006 Today s lecture Feature
CS4670: Computer Vision
CS4670: Computer Vision Noah Snavely Lecture 6: Feature matching and alignment Szeliski: Chapter 6.1 Reading Last time: Corners and blobs Scale-space blob detector: Example Feature descriptors We know
The SIFT (Scale Invariant Feature
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor developed by David Lowe University of British Columbia Initial paper ICCV 1999 Newer journal paper IJCV 2004 Review: Matt Brown s Canonical
BSB663 Image Processing Pinar Duygulu. Slides are adapted from Selim Aksoy
BSB663 Image Processing Pinar Duygulu Slides are adapted from Selim Aksoy Image matching Image matching is a fundamental aspect of many problems in computer vision. Object or scene recognition Solving
Harder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford
Image matching Harder case by Diva Sian by Diva Sian by scgbt by swashford Even harder case Harder still? How the Afghan Girl was Identified by Her Iris Patterns Read the story NASA Mars Rover images Answer
Stereo Vision. MAN-522 Computer Vision
Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in
Harder case. Image matching. Even harder case. Harder still? by Diva Sian. by swashford
Image matching Harder case by Diva Sian by Diva Sian by scgbt by swashford Even harder case Harder still? How the Afghan Girl was Identified by Her Iris Patterns Read the story NASA Mars Rover images Answer
EE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 09 130219 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Feature Descriptors Feature Matching Feature
Projective 2D Geometry
Projective D Geometry Multi View Geometry (Spring '08) Projective D Geometry Prof. Kyoung Mu Lee SoEECS, Seoul National University Homogeneous representation of lines and points Projective D Geometry Line
Homogeneous Coordinates. Lecture18: Camera Models. Representation of Line and Point in 2D. Cross Product. Overall scaling is NOT important.
Homogeneous Coordinates Overall scaling is NOT important. CSED44:Introduction to Computer Vision (207F) Lecture8: Camera Models Bohyung Han CSE, POSTECH bhhan@postech.ac.kr (",, ) ()", ), )) ) 0 It is
Projective geometry for Computer Vision
Department of Computer Science and Engineering IIT Delhi NIT, Rourkela March 27, 2010 Overview Pin-hole camera Why projective geometry? Reconstruction Computer vision geometry: main problems Correspondence
Local Feature Detectors
Local Feature Detectors Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Slides adapted from Cordelia Schmid and David Lowe, CVPR 2003 Tutorial, Matthew Brown,
Image Features. Work on project 1. All is Vanity, by C. Allan Gilbert,
Image Features Work on project 1 All is Vanity, by C. Allan Gilbert, 1873-1929 Feature extrac*on: Corners and blobs c Mo*va*on: Automa*c panoramas Credit: Ma9 Brown Why extract features? Mo*va*on: panorama
3D Computer Vision II. Reminder Projective Geometry, Transformations. Nassir Navab. October 27, 2009
3D Computer Vision II Reminder Projective Geometr, Transformations Nassir Navab based on a course given at UNC b Marc Pollefes & the book Multiple View Geometr b Hartle & Zisserman October 27, 29 2D Transformations
Image matching. Announcements. Harder case. Even harder case. Project 1 Out today Help session at the end of class. by Diva Sian.
Announcements Project 1 Out today Help session at the end of class Image matching by Diva Sian by swashford Harder case Even harder case How the Afghan Girl was Identified by Her Iris Patterns Read the
Local features: detection and description May 12 th, 2015
Local features: detection and description May 12 th, 2015 Yong Jae Lee UC Davis Announcements PS1 grades up on SmartSite PS1 stats: Mean: 83.26 Standard Dev: 28.51 PS2 deadline extended to Saturday, 11:59
Feature Detection. Raul Queiroz Feitosa. 3/30/2017 Feature Detection 1
Feature Detection Raul Queiroz Feitosa 3/30/2017 Feature Detection 1 Objetive This chapter discusses the correspondence problem and presents approaches to solve it. 3/30/2017 Feature Detection 2 Outline
Local invariant features
Local invariant features Tuesday, Oct 28 Kristen Grauman UT-Austin Today Some more Pset 2 results Pset 2 returned, pick up solutions Pset 3 is posted, due 11/11 Local invariant features Detection of interest
Application questions. Theoretical questions
The oral exam will last 30 minutes and will consist of one application question followed by two theoretical questions. Please find below a non exhaustive list of possible application questions. The list
Robot Vision: Projective Geometry
Robot Vision: Projective Geometry Ass.Prof. Friedrich Fraundorfer SS 2018 1 Learning goals Understand homogeneous coordinates Understand points, line, plane parameters and interpret them geometrically
Local Features: Detection, Description & Matching
Local Features: Detection, Description & Matching Lecture 08 Computer Vision Material Citations Dr George Stockman Professor Emeritus, Michigan State University Dr David Lowe Professor, University of British
Augmented Reality VU. Computer Vision 3D Registration (2) Prof. Vincent Lepetit
Augmented Reality VU Computer Vision 3D Registration (2) Prof. Vincent Lepetit Feature Point-Based 3D Tracking Feature Points for 3D Tracking Much less ambiguous than edges; Point-to-point reprojection
3D Modeling using multiple images Exam January 2008
3D Modeling using multiple images Exam January 2008 All documents are allowed. Answers should be justified. The different sections below are independant. 1 3D Reconstruction A Robust Approche Consider
Dense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem
Perspective projection and Transformations
Perspective projection and Transformations The pinhole camera The pinhole camera P = (X,,) p = (x,y) O λ = 0 Q λ = O λ = 1 Q λ = P =-1 Q λ X = 0 + λ X 0, 0 + λ 0, 0 + λ 0 = (λx, λ, λ) The pinhole camera
CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.
CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Section 10 - Detectors part II Descriptors Mani Golparvar-Fard Department of Civil and Environmental Engineering 3129D, Newmark Civil Engineering
Auto-calibration. Computer Vision II CSE 252B
Auto-calibration Computer Vision II CSE 252B 2D Affine Rectification Solve for planar projective transformation that maps line (back) to line at infinity Solve as a Householder matrix Euclidean Projective
Local Patch Descriptors
Local Patch Descriptors Slides courtesy of Steve Seitz and Larry Zitnick CSE 803 1 How do we describe an image patch? How do we describe an image patch? Patches with similar content should have similar
Multiple View Geometry in Computer Vision
Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 More on Single View Geometry Lecture 11 2 In Chapter 5 we introduced projection matrix (which
Midterm Wed. Local features: detection and description. Today. Last time. Local features: main components. Goal: interest operator repeatability
Midterm Wed. Local features: detection and description Monday March 7 Prof. UT Austin Covers material up until 3/1 Solutions to practice eam handed out today Bring a 8.5 11 sheet of notes if you want Review
calibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
Scale Invariant Feature Transform
Scale Invariant Feature Transform Why do we care about matching features? Camera calibration Stereo Tracking/SFM Image moiaicing Object/activity Recognition Objection representation and recognition Image
Image Features: Detection, Description, and Matching and their Applications
Image Features: Detection, Description, and Matching and their Applications Image Representation: Global Versus Local Features Features/ keypoints/ interset points are interesting locations in the image.
Automatic Image Alignment
Automatic Image Alignment with a lot of slides stolen from Steve Seitz and Rick Szeliski Mike Nese CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2018 Live Homography
Robust Geometry Estimation from two Images
Robust Geometry Estimation from two Images Carsten Rother 09/12/2016 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation Process 09/12/2016 2 Appearance-based
Camera model and multiple view geometry
Chapter Camera model and multiple view geometry Before discussing how D information can be obtained from images it is important to know how images are formed First the camera model is introduced and then
Building a Panorama. Matching features. Matching with Features. How do we build a panorama? Computational Photography, 6.882
Matching features Building a Panorama Computational Photography, 6.88 Prof. Bill Freeman April 11, 006 Image and shape descriptors: Harris corner detectors and SIFT features. Suggested readings: Mikolajczyk
Local Image Features
Local Image Features Computer Vision Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Flashed Face Distortion 2nd Place in the 8th Annual Best
Epipolar Geometry Prof. D. Stricker. With slides from A. Zisserman, S. Lazebnik, Seitz
Epipolar Geometry Prof. D. Stricker With slides from A. Zisserman, S. Lazebnik, Seitz 1 Outline 1. Short introduction: points and lines 2. Two views geometry: Epipolar geometry Relation point/line in two
Feature Based Registration - Image Alignment
Feature Based Registration - Image Alignment Image Registration Image registration is the process of estimating an optimal transformation between two or more images. Many slides from Alexei Efros http://graphics.cs.cmu.edu/courses/15-463/2007_fall/463.html
Scale Invariant Feature Transform
Why do we care about matching features? Scale Invariant Feature Transform Camera calibration Stereo Tracking/SFM Image moiaicing Object/activity Recognition Objection representation and recognition Automatic
Dense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction
Image correspondences and structure from motion
Image correspondences and structure from motion http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 20 Course announcements Homework 5 posted.
SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014
SIFT: SCALE INVARIANT FEATURE TRANSFORM SURF: SPEEDED UP ROBUST FEATURES BASHAR ALSADIK EOS DEPT. TOPMAP M13 3D GEOINFORMATION FROM IMAGES 2014 SIFT SIFT: Scale Invariant Feature Transform; transform image
Step-by-Step Model Buidling
Step-by-Step Model Buidling Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure
Local Image Features
Local Image Features Computer Vision CS 143, Brown Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial This section: correspondence and alignment
Visual Tracking (1) Tracking of Feature Points and Planar Rigid Objects
Intelligent Control Systems Visual Tracking (1) Tracking of Feature Points and Planar Rigid Objects Shingo Kagami Graduate School of Information Sciences, Tohoku University swk(at)ic.is.tohoku.ac.jp http://www.ic.is.tohoku.ac.jp/ja/swk/
1 Projective Geometry
CIS8, Machine Perception Review Problem - SPRING 26 Instructions. All coordinate systems are right handed. Projective Geometry Figure : Facade rectification. I took an image of a rectangular object, and
Motion illusion, rotating snakes
Motion illusion, rotating snakes Local features: main components 1) Detection: Find a set of distinctive key points. 2) Description: Extract feature descriptor around each interest point as vector. x 1
EM225 Projective Geometry Part 2
EM225 Projective Geometry Part 2 eview In projective geometry, we regard figures as being the same if they can be made to appear the same as in the diagram below. In projective geometry: a projective point
Feature Detectors and Descriptors: Corners, Lines, etc.
Feature Detectors and Descriptors: Corners, Lines, etc. Edges vs. Corners Edges = maxima in intensity gradient Edges vs. Corners Corners = lots of variation in direction of gradient in a small neighborhood
2D Image Transforms Computer Vision (Kris Kitani) Carnegie Mellon University
2D Image Transforms 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University Extract features from an image what do we do next? Feature matching (object recognition, 3D reconstruction, augmented
Automatic Image Alignment
Automatic Image Alignment Mike Nese with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 Live Homography DEMO Check out panoramio.com
Multiple View Geometry in Computer Vision Second Edition
Multiple View Geometry in Computer Vision Second Edition Richard Hartley Australian National University, Canberra, Australia Andrew Zisserman University of Oxford, UK CAMBRIDGE UNIVERSITY PRESS Contents
SIFT - scale-invariant feature transform Konrad Schindler
SIFT - scale-invariant feature transform Konrad Schindler Institute of Geodesy and Photogrammetry Invariant interest points Goal match points between images with very different scale, orientation, projective
Homogeneous coordinates, lines, screws and twists
Homogeneous coordinates, lines, screws and twists In lecture 1 of module 2, a brief mention was made of homogeneous coordinates, lines in R 3, screws and twists to describe the general motion of a rigid
EE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 10 130221 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Canny Edge Detector Hough Transform Feature-Based
Multi-view 3D reconstruction. Problem formulation Projective ambiguity Rectification Autocalibration Feature points and their matching
Multi-view 3D reconstruction Problem formulation Projective ambiguity Rectification Autocalibration Feature points and their matching Problem Given m images of n scene points captured from different viewpoints,
Lecture 7 Measurement Using a Single Camera. Lin ZHANG, PhD School of Software Engineering Tongji University Fall 2016
Lecture 7 Measurement Using a Single Camera Lin ZHANG, PhD School of Software Engineering Tongji University Fall 2016 If I have an image containing a coin, can you tell me the diameter of that coin? Contents
Camera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration
Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1
Computer Vision. Exercise 3 Panorama Stitching 09/12/2013. Compute Vision : Exercise 3 Panorama Stitching
Computer Vision Exercise 3 Panorama Stitching 09/12/2013 Compute Vision : Exercise 3 Panorama Stitching The task Compute Vision : Exercise 3 Panorama Stitching 09/12/2013 2 Pipeline Compute Vision : Exercise
Computer Vision I - Filtering and Feature detection
Computer Vision I - Filtering and Feature detection Carsten Rother 30/10/2015 Computer Vision I: Basics of Image Processing Roadmap: Basics of Digital Image Processing Computer Vision I: Basics of Image
Image processing and features
Image processing and features Gabriele Bleser gabriele.bleser@dfki.de Thanks to Harald Wuest, Folker Wientapper and Marc Pollefeys Introduction Previous lectures: geometry Pose estimation Epipolar geometry
5LSH0 Advanced Topics Video & Analysis
1 Multiview 3D video / Outline 2 Advanced Topics Multimedia Video (5LSH0), Module 02 3D Geometry, 3D Multiview Video Coding & Rendering Peter H.N. de With, Sveta Zinger & Y. Morvan ( p.h.n.de.with@tue.nl
Multiple View Geometry in Computer Vision
Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 Structure Computation Lecture 18 March 22, 2005 2 3D Reconstruction The goal of 3D reconstruction
CS 4495 Computer Vision A. Bobick. CS 4495 Computer Vision. Features 2 SIFT descriptor. Aaron Bobick School of Interactive Computing
CS 4495 Computer Vision Features 2 SIFT descriptor Aaron Bobick School of Interactive Computing Administrivia PS 3: Out due Oct 6 th. Features recap: Goal is to find corresponding locations in two images.
Two-view geometry Computer Vision Spring 2018, Lecture 10
Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of
C / 35. C18 Computer Vision. David Murray. dwm/courses/4cv.
C18 2015 1 / 35 C18 Computer Vision David Murray david.murray@eng.ox.ac.uk www.robots.ox.ac.uk/ dwm/courses/4cv Michaelmas 2015 C18 2015 2 / 35 Computer Vision: This time... 1. Introduction; imaging geometry;
Mosaics. Today s Readings
Mosaics VR Seattle: http://www.vrseattle.com/ Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html Today s Readings Szeliski and Shum paper (sections
There are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...
STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own
Object and Class Recognition I:
Object and Class Recognition I: Object Recognition Lectures 10 Sources ICCV 2005 short courses Li Fei-Fei (UIUC), Rob Fergus (Oxford-MIT), Antonio Torralba (MIT) http://people.csail.mit.edu/torralba/iccv2005
Multiple View Geometry in Computer Vision
http://www.cs.unc.edu/~marc/ Multiple View Geometry in Computer Vision 2011.09. 2 3 4 5 Visual 3D Modeling from Images 6 AutoStitch http://cs.bath.ac.uk/brown/autostitch/autostitch.html 7 Image Composite
Augmented Reality II - Camera Calibration - Gudrun Klinker May 11, 2004
Augmented Reality II - Camera Calibration - Gudrun Klinker May, 24 Literature Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2. (Section 5,
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
3D Geometric Computer Vision. Martin Jagersand Univ. Alberta Edmonton, Alberta, Canada
3D Geometric Computer Vision Martin Jagersand Univ. Alberta Edmonton, Alberta, Canada Multi-view geometry Build 3D models from images Carry out manipulation tasks http://webdocs.cs.ualberta.ca/~vis/ibmr/
Multiple View Geometry in computer vision
Multiple View Geometry in computer vision Chapter 8: More Single View Geometry Olaf Booij Intelligent Systems Lab Amsterdam University of Amsterdam, The Netherlands HZClub 29-02-2008 Overview clubje Part
Homographies and RANSAC
Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position
Binocular Stereo Vision. System 6 Introduction Is there a Wedge in this 3D scene?
System 6 Introduction Is there a Wedge in this 3D scene? Binocular Stereo Vision Data a stereo pair of images! Given two 2D images of an object, how can we reconstruct 3D awareness of it? AV: 3D recognition
EECS150 - Digital Design Lecture 14 FIFO 2 and SIFT. Recap and Outline
EECS150 - Digital Design Lecture 14 FIFO 2 and SIFT Oct. 15, 2013 Prof. Ronald Fearing Electrical Engineering and Computer Sciences University of California, Berkeley (slides courtesy of Prof. John Wawrzynek)
3D Reconstruction From Multiple Views Based on Scale-Invariant Feature Transform. Wenqi Zhu
3D Reconstruction From Multiple Views Based on Scale-Invariant Feature Transform Wenqi Zhu wenqizhu@buffalo.edu Problem Statement! 3D reconstruction 3D reconstruction is a problem of recovering depth information
Comments on Consistent Depth Maps Recovery from a Video Sequence
Comments on Consistent Depth Maps Recovery from a Video Sequence N.P. van der Aa D.S. Grootendorst B.F. Böggemann R.T. Tan Technical Report UU-CS-2011-014 May 2011 Department of Information and Computing
Edge and corner detection
Edge and corner detection Prof. Stricker Doz. G. Bleser Computer Vision: Object and People Tracking Goals Where is the information in an image? How is an object characterized? How can I find measurements
Fundamentals of 3D. Lecture 3: Debriefing: Lecture 2 Rigid transformations Quaternions Iterative Closest Point (+Kd-trees)
INF555 Fundamentals of 3D Lecture 3: Debriefing: Lecture 2 Rigid transformations Quaternions Iterative Closest Point (+Kd-trees) Frank Nielsen nielsen@lix.polytechnique.fr Harris-Stephens' combined corner/edge
Camera Model and Calibration
Camera Model and Calibration Lecture-10 Camera Calibration Determine extrinsic and intrinsic parameters of camera Extrinsic 3D location and orientation of camera Intrinsic Focal length The size of the
Robotics Programming Laboratory
Chair of Software Engineering Robotics Programming Laboratory Bertrand Meyer Jiwon Shin Lecture 8: Robot Perception Perception http://pascallin.ecs.soton.ac.uk/challenges/voc/databases.html#caltech car
More on single-view geometry class 10
More on single-view geometry class 10 Multiple View Geometry Comp 290-089 Marc Pollefeys Multiple View Geometry course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective 2D Geometry Jan.
HISTOGRAMS OF ORIENTATIO N GRADIENTS
HISTOGRAMS OF ORIENTATIO N GRADIENTS Histograms of Orientation Gradients Objective: object recognition Basic idea Local shape information often well described by the distribution of intensity gradients
CS231A Midterm Review. Friday 5/6/2016
CS231A Midterm Review Friday 5/6/2016 Outline General Logistics Camera Models Non-perspective cameras Calibration Single View Metrology Epipolar Geometry Structure from Motion Active Stereo and Volumetric