Structure from Motion
|
|
- Dayna Horn
- 5 years ago
- Views:
Transcription
1 Structure from Motion
2 Outline Bundle Adjustment Ambguities in Reconstruction Affine Factorization Extensions
3 Structure from motion Recover both 3D scene geoemetry and camera positions
4 SLAM: Simultaneous Localization and Mapping
5 Recall: 2-view stereo
6 An annoying detail What to do when ray s don t intersect?
7 Minimize reprojection error X? x x 2 2 O O 2 min X f(x) = x Proj(X,M ) 2 + x 2 Proj(X,M 2 ) 2 Perspective projection equations where M = 3x4 matrix of extrinics (R,t) and intrinsics (f) of camera For this equation, its easier to parameterize camera position in absolute coordinates instead of relative ones
8 Minimize reprojection error X? x x 2 2 O O 2 min X f(x) = x Proj(X,M ) 2 + x 2 Proj(X,M 2 ) 2 Perspective projection equations where M = 3x4 matrix of extrinics (R,t) and intrinsics (f) of camera For this equation, its easier to parameterize camera position in absolute coordinates instead of relative ones
9 Generalize triangulation to multiple points from multiple cameras min X,X 2,... mx i= nx j= x ij Proj(X j,m i ) 2 As written, could this minimization done independantly for each 3D point?
10 Bundle adjustment Minimize reprojection error over multiple 3D points and cameras min X,X 2,...,M,M 2,... mx i= nx x ij Proj(X j,m i ) 2 j=
11 Basic SFM pipeline.find candidate correspondences (interest points + descriptor matches) 2.Select subset that are consistent with epipolar constraints on pairs of images (RANSAC + fundmental matrix) 3.Solve for 3D points and camera that minimize reprojection error (Lots of variants; e.g., iteratively build map of 3D points and cameras as new images arrive)
12 min X,X 2,...,M,M 2,... mx i= nx x ij Proj(X j,m i ) 2 j= M,..,M m X,..,X n
13 Bundle adjustment: nonlinear least-squares Encode visibility graphs of what points are seen in what images, i.e. Features: A,B,C,D,E Images:,2,3 Implies jacobian of error function is sparse
14 Excellent reference Bundle Adjustment A Modern Synthesis Bill Triggs,PhilipMcLauchlan 2, Richard Hartley 3 and Andrew Fitzgibbon 4 INRIA Rhône-Alpes, 655 avenue de l Europe, Montbonnot, France. Bill.Triggs@inrialpes.fr 2 School of Electrical Engineering, Information Technology & Mathematics University of Surrey, Guildford, GU2 5XH, U.K. P.McLauchlan@ee.surrey.ac.uk 3 General Electric CRD, Schenectady, NY, 230 hartley@crd.ge.com 4 Dept of Engineering Science, University of Oxford, 9 Parks Road, OX 3PJ, U.K. awf@robots.ox.ac.uk awf
15 Outline Bundle Adjustment Ambguities in Reconstruction Affine Factorization Extensions
16 Recall camera projection X x fs x fs o x r r 2 r 3 t x 4y5 = 4 0 fs y o y 5 4r 2 r 22 r 23 t y 5 6Y 7 4Z r 3 r 32 r 33 t z 2 3 X = K 3 3 R3 3 T 3 6Y 7 4Z X = M 3 4 6Y 7 4Z 5 x MX x = mt X m T 3 X y = mt 2 X m T 3 X
17 Structure from motion ambiguity If we scale the entire scene by some factor k and, at the same time, scale the camera matrices by the factor of /k, the 2D homogenous vector remains exactly the same: x MX MX =( k M)(kX) It is impossible to recover the absolute scale of the scene!
18 Structure from motion ambiguity If we scale the entire scene by some factor k and, at the same time, scale the camera matrices by the factor of /k, the 2D homogenous vector remains exactly the same: More generally: if we transform the scene using a transformation Q and apply the inverse transformation to the camera matrices, nothing changes x MX MX =(MQ )(QX)
19 Recall: transformations in 3D Make use of 3D homogenous coordinates 2 3 x 6y 7 4z5
20 Recall: transformations in 3D
21 Recall: transformations in 3D Normalize by last coordinate to recover 3D points x y 7 z5
22 Back to ambiguities for 3D reconstruction x = PX = ( - PQ )( QX) Projective A T v t v Preserves intersection Affine A T 0 t Preserves parallellism, Similarity s R T 0 t Preserves angles, ratios Euclidean R T 0 t Preserves angles, With no constraints on the camera calibration matrix or on the scene, we get a projective reconstruction Need additional information to upgrade the reconstruction to affine, similarity, or Euclidean
23 Projective ambiguity x MX =(MQ )(QX) Q p = A T v t v
24 Projective ambiguity (straight line are preserved)
25 Affine ambiguity x MX =(MQ )(QX) Affine Q A = A 0 T t
26 Affine ambiguity (parallel lines are preserved)
27 Similarity ambiguity x MX =(MQ )(QX) Q s = sr 0 T t
28 Similarity ambiguity angles+lengths preserved (but can t recover world coordinate system)
29 Outline Bundle Adjustment Ambguities in Reconstruction Affine Factorization Large-scale SFM
30 Structure from motion Let s start with affine cameras (the math is easier) center at infinity
31 Recall: Orthographic Projection Special case of perspective projection Image World Another common notation for a projection matrix: 2 3 x 4y5 = X 6Y 7 4Z 5 ) (x, y) Sometimes notationally convenient because 2D homogenous coordinates allow us to write 2D transformations as matrix multiplication
32 Recall: affine cameras Model as 3D affine transformation + orthographic projection + 2D affine transformation apple x y = apple X Y Z = 2 4 a a 2 a 3 b a 2 a 22 a 23 b X Y Z Affine camera defined by 8 parameters
33 Affine cameras Model as 3D affine transformation + orthographic projection + 2D affine transformation apple x y = = = X apple Y Z X a a 2 a 3 b 4a 2 a 22 a 23 b 2 5 6Y 7 4Z X a a 2 a 3 b 4a 2 a 22 a 23 b 2 5 Q a Qa 6Y 7 4Z 5, Q a = Qa must be 3D affine transformation (2 parameters) inorder to maintain structure of affine projection matrix
34 Affine cameras Model as 3D affine transformation + orthographic projection + 2D affine transformation apple x y = = = = X apple Y Z X a a 2 a 3 b 4a 2 a 22 a 23 b 2 5 6Y 7 4Z X a a 2 a 3 b 4a 2 a 22 a 23 b 2 5 Q a Qa 6Y 7 4Z 5, Q a = apple a a 2 a 3 a 2 a 22 a 23 x = AX + b 2 3 X 4Y Z 5 + apple b b 2 2D points = linear transformation of 3D points + 2D translation
35 Affine structure from motion (my favorite vision algorithm) Given: m images of n fixed 3D points: x ij = A i X j + b i, i =,, m, j =,, n Problem: use the mn correspondences x ij to estimate m projection matrices A i and translation vectors b i, and n points X j # of images # of points 2 3 x x x 4 2 x
36 Affine structure from motion Given: m images of n fixed 3D points: x ij = A i X j + b i, i =,, m, j =,, n Problem: use the mn correspondences x ij to estimate m projection matrices A i and translation vectors b i, and n points X j The reconstruction is defined up to an arbitrary 3D affine transformation Q (2 degrees of freedom): A 0 b A 0 b Q How many knowns are there? How many unknowns?, X Q X
37 Affine structure from motion Given: m images of n fixed 3D points: x ij = A i X j + b i, i =,, m, j =,, n Problem: use the mn correspondences x ij to estimate m projection matrices A i and translation vectors b i, and n points X j The reconstruction is defined up to an arbitrary 3D affine transformation Q (2 degrees of freedom): A 0 b A 0 b Q, Q We have 2mn knowns and 8m + 3n unknowns (minus 2 dof for affine ambiguity) Thus, we must have 2mn >= 8m + 3n 2 For m=2 views, we need n=4 point correspondences X X
38 Affine structure from motion Let s try centering the points in each 2D image # of images # of points 2 3 x x x 4 2 x ˆx ij = x ij n nx k x ik Plug the following into the above expression: x ij = A i X j + b i [on board]
39 Affine structure from motion Centering: subtract the centroid of the image points After centering, each normalized point x ij is related to the 3D point X i by ( ) j i n k k j i n k i k i i j i n k ik ij ij n n n A X X X A b A X b A X x x x ˆ ˆ = = + + = = = = = ˆx ij = A i ˆX j Given a set of 2D correspondences, simply center them in each image Affine image projection now becomes linear (represent Ai by a 2x3 matrix and x^ij is 2 vector)
40 Affine structure from motion Let s create a 2m n data (measurement) matrix: D = xˆ xˆ xˆ 2 m xˆ xˆ xˆ 2 22 m2!! "! xˆ xˆ xˆ n 2n mn cameras (2 m) points (n) C. Tomasi and T. Kanade. Shape and motion from image streams under orthography: A factorization method. IJCV, 9(2):37-54, November 992.
41 Affine structure from motion Let s create a 2m n data (measurement) matrix: cameras (2 m 3) points (3 n) The measurement matrix D = MS must have rank 3! C. Tomasi and T. Kanade. Shape and motion from image streams under orthography: A factorization method. IJCV, 9(2):37-54, November 992. [ ] n m mn m m n n X X X A A A x x x x x x x x x D! "! #!! ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ = =
42 Fundamental Decomposition 2m Measurements = Motion Shape 3 n D = MS n 3
43 n n n n 2m D = U V T n 3 3 To reduce to rank 3, we just need to set all the singular values to 0 except for the first 3 2m D = U V 3 T 3
44 3 n 2m D U 3 3 V T 3 3 = 3 3 Possible decomposition: M = U 3, S = 3 V T 3 D = M S This decomposition minimizes D-MS 2
45 Affine ambiguity The decomposition is not unique. We get the same D by using any 3 3 matrix C and applying the transformations M MC, S C - S That is because we have only an affine transformation and we have not enforced any Euclidean constraints (like forcing the image axes to be perpendicular, for example) Source: M. Hebert
46 Eliminating the affine ambiguity enforce image axes to be ortthonogal and length a a 2 = 0 x a 2 = a 2 2 = a 2 a M = A A X We want C such that Ai CC T A T i = Id Write out optimization as min min L A i LA T i Id 2
47 Orthographic: image axes are perpendicular and scale is 2 3 a a 2 = 0 M = Eliminating the affine ambiguity 6 4 A A a 2 x a 2 = a 2 2 = a X This translates into 3m equations in L = CC T : A i L A T i = Id, i =,, m Solve for L Recover C from L by Cholesky decomposition: L = CC T (in practice, easy to do) Update M and S: M = MC, S = C - S Source: M. Hebert
48 Algorithm summary Given: m images and n features x ij For each image i, center the feature coordinates Construct a 2m n measurement matrix D: Column j contains the projection of point j in all views Row i contains one coordinate of the projections of all the n points in image i Factorize D: Compute SVD: D = U W V T Create U 3 by taking the first 3 columns of U Create V 3 by taking the first 3 columns of V Create W 3 by taking the upper left 3 3 block of W Create the motion and shape matrices: M = U 3, S = 3 V3 T Eliminate affine ambiguity Source: M. Hebert
49 Results C. Tomasi and T. Kanade. Shape and motion from image streams under orthography: A factorization method.
50 Results
51 Outline Bundle Adjustment Ambguities in Reconstruction Affine Factorization Extensions
52 Dealing with missing data So far, we have assumed that all points are visible in all views In reality, the measurement matrix typically looks something like this: cameras points
53 Sequential structure from motion Intuition: exploit low-rank redundancy of matrix Possible solution: decompose matrix into dense sub-blocks, factorize each sub-block, and fuse the results Finding dense maximal sub-blocks of the matrix is NP-complete (equivalent to finding maximal cliques in a graph) Incremental bilinear refinement points () Perform factorization on a dense subblock cameras
54 Sequential structure from motion Intuition: exploit low-rank redundancy of matrix Possible solution: decompose matrix into dense sub-blocks, factorize each sub-block, and fuse the results Finding dense maximal sub-blocks of the matrix is NP-complete (equivalent to finding maximal cliques in a graph) Incremental bilinear refinement points (2) Solve for a new 3D point visible by at least two known cameras (linear least squares) min S D MS 2 cameras
55 Sequential structure from motion Intuition: exploit low-rank redundancy of matrix Possible solution: decompose matrix into dense sub-blocks, factorize each sub-block, and fuse the results Finding dense maximal sub-blocks of the matrix is NP-complete (equivalent to finding maximal cliques in a graph) Incremental bilinear refinement points (3) Solve for a new camera that sees at least three known 3D points (linear least squares) min M D MS 2 cameras F. Rothganger, S. Lazebnik, C. Schmid, and J. Ponce. Segmenting, Modeling, and Matching Video Clips Containing Multiple Moving Objects. PAMI 2007.
56 Nonrigid structure motion D = X S S2 M 2 M Assume 3D shapes are a linear combination of K basis shapes S = KX k= k S k For each image, we need to solve for camera matrix and scaling coefficients Simply relax rank constraint on measurement matrix from 3 to 3K!
57
58 Projective structure from motion Given: m images of n fixed 3D points x ij M i X j, i =,, m, j =,, n Problem: estimate m projection matrices M i and n 3D points X j from the mn correspondences x ij With no calibration info, cameras and points can only be recovered up to a 4x4 projective transformation Q: X QX, M MQ - We can solve for structure and motion when 2mn >= m +3n 5 For two cameras, at least 7 points are needed
59 Projective SFM: Two-camera case Compute fundamental matrix F between the two views (from 7 or more correspondences) First camera matrix: M = [I 0] Second camera matrix: M = [A3x3 b3x] Then one can compute A,b from F as follows: b is the right null vector, or epipole (F T b = 0) A = ˆbF (where hat notation refers to skew symmetric matrix) For proof, see F & P Sec 8.3 For more than 2 images, things get more implicated
60 Back to bundle adjustment Minimize reprojection error over multiple 3D points and cameras min X,X 2,...,M,M 2,... mx i= nx x ij Proj(X j,m i ) 2 j=
61 Self-calibration Self-calibration (auto-calibration) is the process of determining intrinsic camera parameters directly from uncalibrated images For example, when the images are acquired by a single moving camera, we can use the constraint that the intrinsic parameter matrix remains fixed for all the images Compute initial projective reconstruction and find 3D projective transformation matrix Q such that all camera matrices are in the form M i = K [R i t i ] Can use constraints on the form of the calibration matrix: orthogonal image axis Can use vanishing points
62 Review: Structure from motion Ambiguity Affine structure from motion Factorization Dealing with missing data Incremental structure from motion Projective structure from motion Bundle adjustment Self-calibration
63 Summary: 3D geometric vision Single-view geometry The pinhole camera model Variation: orthographic projection The perspective projection matrix Intrinsic parameters Extrinsic parameters Calibration Multiple-view geometry Triangulation The epipolar constraint Essential matrix and fundamental matrix Stereo Binocular, multi-view Structure from motion Reconstruction ambiguity Affine SFM Projective SFM
Structure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t R 2 3,t 3 Camera 1 Camera
More informationStructure from Motion CSC 767
Structure from Motion CSC 767 Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R,t R 2,t 2 R 3,t 3 Camera??
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera
More informationBIL Computer Vision Apr 16, 2014
BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm
More informationC280, Computer Vision
C280, Computer Vision Prof. Trevor Darrell trevor@eecs.berkeley.edu Lecture 11: Structure from Motion Roadmap Previous: Image formation, filtering, local features, (Texture) Tues: Feature-based Alignment
More informationStructure from Motion
11/18/11 Structure from Motion Computer Vision CS 143, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from
More informationStructure from Motion
/8/ Structure from Motion Computer Vision CS 43, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from motion
More informationCS 532: 3D Computer Vision 7 th Set of Notes
1 CS 532: 3D Computer Vision 7 th Set of Notes Instructor: Philippos Mordohai Webpage: www.cs.stevens.edu/~mordohai E-mail: Philippos.Mordohai@stevens.edu Office: Lieb 215 Logistics No class on October
More informationCS231M Mobile Computer Vision Structure from motion
CS231M Mobile Computer Vision Structure from motion - Cameras - Epipolar geometry - Structure from motion Pinhole camera Pinhole perspective projection f o f = focal length o = center of the camera z y
More informationLecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15
Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15 Lecture 6 Stereo Systems Multi- view geometry Stereo systems
More informationComputer Vision I - Algorithms and Applications: Multi-View 3D reconstruction
Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction Carsten Rother 09/12/2013 Computer Vision I: Multi-View 3D reconstruction Roadmap this lecture Computer Vision I: Multi-View
More informationCEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.
CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Session 4 Affine Structure from Motion Mani Golparvar-Fard Department of Civil and Environmental Engineering 329D, Newmark Civil Engineering
More informationLecture 10: Multi-view geometry
Lecture 10: Multi-view geometry Professor Stanford Vision Lab 1 What we will learn today? Review for stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationCS231A Course Notes 4: Stereo Systems and Structure from Motion
CS231A Course Notes 4: Stereo Systems and Structure from Motion Kenji Hata and Silvio Savarese 1 Introduction In the previous notes, we covered how adding additional viewpoints of a scene can greatly enhance
More informationUnit 3 Multiple View Geometry
Unit 3 Multiple View Geometry Relations between images of a scene Recovering the cameras Recovering the scene structure http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook1.html 3D structure from images Recover
More informationLecture 10: Multi view geometry
Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationLecture 9: Epipolar Geometry
Lecture 9: Epipolar Geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Why is stereo useful? Epipolar constraints Essential and fundamental matrix Estimating F (Problem Set 2
More informationStructure from Motion. Introduction to Computer Vision CSE 152 Lecture 10
Structure from Motion CSE 152 Lecture 10 Announcements Homework 3 is due May 9, 11:59 PM Reading: Chapter 8: Structure from Motion Optional: Multiple View Geometry in Computer Vision, 2nd edition, Hartley
More informationStructure from Motion
Structure from Motion Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz, N Snavely, and D. Hoiem Administrative stuffs HW 3 due 11:55 PM, Oct 17 (Wed) Submit your alignment results!
More informationCS 664 Structure and Motion. Daniel Huttenlocher
CS 664 Structure and Motion Daniel Huttenlocher Determining 3D Structure Consider set of 3D points X j seen by set of cameras with projection matrices P i Given only image coordinates x ij of each point
More informationCS231A Midterm Review. Friday 5/6/2016
CS231A Midterm Review Friday 5/6/2016 Outline General Logistics Camera Models Non-perspective cameras Calibration Single View Metrology Epipolar Geometry Structure from Motion Active Stereo and Volumetric
More informationStereo Vision. MAN-522 Computer Vision
Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in
More informationWeek 2: Two-View Geometry. Padua Summer 08 Frank Dellaert
Week 2: Two-View Geometry Padua Summer 08 Frank Dellaert Mosaicking Outline 2D Transformation Hierarchy RANSAC Triangulation of 3D Points Cameras Triangulation via SVD Automatic Correspondence Essential
More informationLast lecture. Passive Stereo Spacetime Stereo
Last lecture Passive Stereo Spacetime Stereo Today Structure from Motion: Given pixel correspondences, how to compute 3D structure and camera motion? Slides stolen from Prof Yungyu Chuang Epipolar geometry
More information1 Projective Geometry
CIS8, Machine Perception Review Problem - SPRING 26 Instructions. All coordinate systems are right handed. Projective Geometry Figure : Facade rectification. I took an image of a rectangular object, and
More informationEpipolar geometry. x x
Two-view geometry Epipolar geometry X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections
More informationTwo-view geometry Computer Vision Spring 2018, Lecture 10
Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of
More informationLecture 6 Stereo Systems Multi-view geometry
Lecture 6 Stereo Systems Multi-view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-5-Feb-4 Lecture 6 Stereo Systems Multi-view geometry Stereo systems
More informationMetric Structure from Motion
CS443 Final Project Metric Structure from Motion Peng Cheng 1 Objective of the Project Given: 1. A static object with n feature points and unknown shape. 2. A camera with unknown intrinsic parameters takes
More informationStructure from Motion and Multi- view Geometry. Last lecture
Structure from Motion and Multi- view Geometry Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 5 Last lecture S. J. Gortler, R. Grzeszczuk, R. Szeliski,M. F. Cohen The Lumigraph, SIGGRAPH,
More informationComputer Vision Projective Geometry and Calibration. Pinhole cameras
Computer Vision Projective Geometry and Calibration Professor Hager http://www.cs.jhu.edu/~hager Jason Corso http://www.cs.jhu.edu/~jcorso. Pinhole cameras Abstract camera model - box with a small hole
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationAugmented Reality II - Camera Calibration - Gudrun Klinker May 11, 2004
Augmented Reality II - Camera Calibration - Gudrun Klinker May, 24 Literature Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2. (Section 5,
More informationStructure from Motion
04/4/ Structure from Motion Comuter Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Many slides adated from Lana Lazebnik, Silvio Saverese, Steve Seitz his class: structure from motion Reca
More informationAgenda. Rotations. Camera models. Camera calibration. Homographies
Agenda Rotations Camera models Camera calibration Homographies D Rotations R Y = Z r r r r r r r r r Y Z Think of as change of basis where ri = r(i,:) are orthonormal basis vectors r rotated coordinate
More informationThe end of affine cameras
The end of affine cameras Affine SFM revisited Epipolar geometry Two-view structure from motion Multi-view structure from motion Planches : http://www.di.ens.fr/~ponce/geomvis/lect3.pptx http://www.di.ens.fr/~ponce/geomvis/lect3.pdf
More informationA Factorization Method for Structure from Planar Motion
A Factorization Method for Structure from Planar Motion Jian Li and Rama Chellappa Center for Automation Research (CfAR) and Department of Electrical and Computer Engineering University of Maryland, College
More informationStereo and Epipolar geometry
Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka
More informationAgenda. Rotations. Camera calibration. Homography. Ransac
Agenda Rotations Camera calibration Homography Ransac Geometric Transformations y x Transformation Matrix # DoF Preserves Icon translation rigid (Euclidean) similarity affine projective h I t h R t h sr
More informationRecovering structure from a single view Pinhole perspective projection
EPIPOLAR GEOMETRY The slides are from several sources through James Hays (Brown); Silvio Savarese (U. of Michigan); Svetlana Lazebnik (U. Illinois); Bill Freeman and Antonio Torralba (MIT), including their
More informationN-View Methods. Diana Mateus, Nassir Navab. Computer Aided Medical Procedures Technische Universität München. 3D Computer Vision II
1/66 N-View Methods Diana Mateus, Nassir Navab Computer Aided Medical Procedures Technische Universität München 3D Computer Vision II Inspired by Slides from Adrien Bartoli 2/66 Outline 1 Structure from
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Shiv Ram Dubey, IIIT Sri City Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X
More informationVisual Recognition: Image Formation
Visual Recognition: Image Formation Raquel Urtasun TTI Chicago Jan 5, 2012 Raquel Urtasun (TTI-C) Visual Recognition Jan 5, 2012 1 / 61 Today s lecture... Fundamentals of image formation You should know
More informationMultiple Views Geometry
Multiple Views Geometry Subhashis Banerjee Dept. Computer Science and Engineering IIT Delhi email: suban@cse.iitd.ac.in January 2, 28 Epipolar geometry Fundamental geometric relationship between two perspective
More information3D Computer Vision. Structure from Motion. Prof. Didier Stricker
3D Computer Vision Structure from Motion Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Structure
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationEpipolar Geometry Prof. D. Stricker. With slides from A. Zisserman, S. Lazebnik, Seitz
Epipolar Geometry Prof. D. Stricker With slides from A. Zisserman, S. Lazebnik, Seitz 1 Outline 1. Short introduction: points and lines 2. Two views geometry: Epipolar geometry Relation point/line in two
More informationCS 231A: Computer Vision (Winter 2018) Problem Set 2
CS 231A: Computer Vision (Winter 2018) Problem Set 2 Due Date: Feb 09 2018, 11:59pm Note: In this PS, using python2 is recommended, as the data files are dumped with python2. Using python3 might cause
More information3D Geometry and Camera Calibration
3D Geometry and Camera Calibration 3D Coordinate Systems Right-handed vs. left-handed x x y z z y 2D Coordinate Systems 3D Geometry Basics y axis up vs. y axis down Origin at center vs. corner Will often
More informationMei Han Takeo Kanade. January Carnegie Mellon University. Pittsburgh, PA Abstract
Scene Reconstruction from Multiple Uncalibrated Views Mei Han Takeo Kanade January 000 CMU-RI-TR-00-09 The Robotics Institute Carnegie Mellon University Pittsburgh, PA 1513 Abstract We describe a factorization-based
More informationEpipolar Geometry in Stereo, Motion and Object Recognition
Epipolar Geometry in Stereo, Motion and Object Recognition A Unified Approach by GangXu Department of Computer Science, Ritsumeikan University, Kusatsu, Japan and Zhengyou Zhang INRIA Sophia-Antipolis,
More informationGeometry for Computer Vision
Geometry for Computer Vision Lecture 5b Calibrated Multi View Geometry Per-Erik Forssén 1 Overview The 5-point Algorithm Structure from Motion Bundle Adjustment 2 Planar degeneracy In the uncalibrated
More informationHomogeneous Coordinates. Lecture18: Camera Models. Representation of Line and Point in 2D. Cross Product. Overall scaling is NOT important.
Homogeneous Coordinates Overall scaling is NOT important. CSED44:Introduction to Computer Vision (207F) Lecture8: Camera Models Bohyung Han CSE, POSTECH bhhan@postech.ac.kr (",, ) ()", ), )) ) 0 It is
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263
Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine
More informationMysteries of Parameterizing Camera Motion - Part 1
Mysteries of Parameterizing Camera Motion - Part 1 Instructor - Simon Lucey 16-623 - Advanced Computer Vision Apps Today Motivation SO(3) Convex? Exponential Maps SL(3) Group. Adapted from: Computer vision:
More informationCamera Geometry II. COS 429 Princeton University
Camera Geometry II COS 429 Princeton University Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence
More informationUndergrad HTAs / TAs. Help me make the course better! HTA deadline today (! sorry) TA deadline March 21 st, opens March 15th
Undergrad HTAs / TAs Help me make the course better! HTA deadline today (! sorry) TA deadline March 2 st, opens March 5th Project 2 Well done. Open ended parts, lots of opportunity for mistakes. Real implementation
More informationMulti-view geometry problems
Multi-view geometry Multi-view geometry problems Structure: Given projections o the same 3D point in two or more images, compute the 3D coordinates o that point? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera
More informationMultiple View Geometry in Computer Vision
Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 Structure Computation Lecture 18 March 22, 2005 2 3D Reconstruction The goal of 3D reconstruction
More informationPin Hole Cameras & Warp Functions
Pin Hole Cameras & Warp Functions Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Pinhole Camera. Homogenous Coordinates. Planar Warp Functions. Motivation Taken from: http://img.gawkerassets.com/img/18w7i1umpzoa9jpg/original.jpg
More informationEECS 442: Final Project
EECS 442: Final Project Structure From Motion Kevin Choi Robotics Ismail El Houcheimi Robotics Yih-Jye Jeffrey Hsu Robotics Abstract In this paper, we summarize the method, and results of our projective
More informationPin Hole Cameras & Warp Functions
Pin Hole Cameras & Warp Functions Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Pinhole Camera. Homogenous Coordinates. Planar Warp Functions. Example of SLAM for AR Taken from:
More informationCamera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration
Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1
More information3D reconstruction class 11
3D reconstruction class 11 Multiple View Geometry Comp 290-089 Marc Pollefeys Multiple View Geometry course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective 2D Geometry Jan. 14, 16
More informationStep-by-Step Model Buidling
Step-by-Step Model Buidling Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253
Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine
More informationStereo II CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo II CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Camera parameters A camera is described by several parameters Translation T of the optical center from the origin of world
More informationStructure from motion
Multi-view geometry Structure rom motion Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera 3 R 3,t 3 Figure credit: Noah Snavely Structure rom motion? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera 3 R 3,t 3 Structure:
More informationLecture 14: Basic Multi-View Geometry
Lecture 14: Basic Multi-View Geometry Stereo If I needed to find out how far point is away from me, I could use triangulation and two views scene point image plane optical center (Graphic from Khurram
More informationVision par ordinateur
Epipolar geometry π Vision par ordinateur Underlying structure in set of matches for rigid scenes l T 1 l 2 C1 m1 l1 e1 M L2 L1 e2 Géométrie épipolaire Fundamental matrix (x rank 2 matrix) m2 C2 l2 Frédéric
More informationCamera models and calibration
Camera models and calibration Read tutorial chapter 2 and 3. http://www.cs.unc.edu/~marc/tutorial/ Szeliski s book pp.29-73 Schedule (tentative) 2 # date topic Sep.8 Introduction and geometry 2 Sep.25
More informationC / 35. C18 Computer Vision. David Murray. dwm/courses/4cv.
C18 2015 1 / 35 C18 Computer Vision David Murray david.murray@eng.ox.ac.uk www.robots.ox.ac.uk/ dwm/courses/4cv Michaelmas 2015 C18 2015 2 / 35 Computer Vision: This time... 1. Introduction; imaging geometry;
More informationStereo CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Why do we perceive depth? What do humans use as depth cues? Motion Convergence When watching an object close to us, our eyes
More informationCamera Calibration. COS 429 Princeton University
Camera Calibration COS 429 Princeton University Point Correspondences What can you figure out from point correspondences? Noah Snavely Point Correspondences X 1 X 4 X 3 X 2 X 5 X 6 X 7 p 1,1 p 1,2 p 1,3
More information3D Reconstruction from Two Views
3D Reconstruction from Two Views Huy Bui UIUC huybui1@illinois.edu Yiyi Huang UIUC huang85@illinois.edu Abstract In this project, we study a method to reconstruct a 3D scene from two views. First, we extract
More informationMultiple View Geometry in Computer Vision Second Edition
Multiple View Geometry in Computer Vision Second Edition Richard Hartley Australian National University, Canberra, Australia Andrew Zisserman University of Oxford, UK CAMBRIDGE UNIVERSITY PRESS Contents
More informationRobust Geometry Estimation from two Images
Robust Geometry Estimation from two Images Carsten Rother 09/12/2016 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation Process 09/12/2016 2 Appearance-based
More informationProjective geometry for 3D Computer Vision
Subhashis Banerjee Computer Science and Engineering IIT Delhi Dec 16, 2015 Overview Pin-hole camera Why projective geometry? Reconstruction Computer vision geometry: main problems Correspondence problem:
More informationEpipolar Geometry and the Essential Matrix
Epipolar Geometry and the Essential Matrix Carlo Tomasi The epipolar geometry of a pair of cameras expresses the fundamental relationship between any two corresponding points in the two image planes, and
More informationAn idea which can be used once is a trick. If it can be used more than once it becomes a method
An idea which can be used once is a trick. If it can be used more than once it becomes a method - George Polya and Gabor Szego University of Texas at Arlington Rigid Body Transformations & Generalized
More informationCamera calibration. Robotic vision. Ville Kyrki
Camera calibration Robotic vision 19.1.2017 Where are we? Images, imaging Image enhancement Feature extraction and matching Image-based tracking Camera models and calibration Pose estimation Motion analysis
More informationAnnouncements. Stereo
Announcements Stereo Homework 2 is due today, 11:59 PM Homework 3 will be assigned today Reading: Chapter 7: Stereopsis CSE 152 Lecture 8 Binocular Stereopsis: Mars Given two images of a scene where relative
More informationEpipolar Geometry and Stereo Vision
CS 1674: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 5, 2016 Announcement Please send me three topics you want me to review next
More informationCS201 Computer Vision Camera Geometry
CS201 Computer Vision Camera Geometry John Magee 25 November, 2014 Slides Courtesy of: Diane H. Theriault (deht@bu.edu) Question of the Day: How can we represent the relationships between cameras and the
More informationReminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches
Reminder: Lecture 20: The Eight-Point Algorithm F = -0.00310695-0.0025646 2.96584-0.028094-0.00771621 56.3813 13.1905-29.2007-9999.79 Readings T&V 7.3 and 7.4 Essential/Fundamental Matrix E/F Matrix Summary
More informationLecture 3: Camera Calibration, DLT, SVD
Computer Vision Lecture 3 23--28 Lecture 3: Camera Calibration, DL, SVD he Inner Parameters In this section we will introduce the inner parameters of the cameras Recall from the camera equations λx = P
More informationEuclidean Reconstruction Independent on Camera Intrinsic Parameters
Euclidean Reconstruction Independent on Camera Intrinsic Parameters Ezio MALIS I.N.R.I.A. Sophia-Antipolis, FRANCE Adrien BARTOLI INRIA Rhone-Alpes, FRANCE Abstract bundle adjustment techniques for Euclidean
More informationPerception and Action using Multilinear Forms
Perception and Action using Multilinear Forms Anders Heyden, Gunnar Sparr, Kalle Åström Dept of Mathematics, Lund University Box 118, S-221 00 Lund, Sweden email: {heyden,gunnar,kalle}@maths.lth.se Abstract
More informationComputer Vision Lecture 17
Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester
More informationComputer Vision Lecture 17
Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week
More informationLecture 8.2 Structure from Motion. Thomas Opsahl
Lecture 8.2 Structure from Motion Thomas Opsahl More-than-two-view geometry Correspondences (matching) More views enables us to reveal and remove more mismatches than we can do in the two-view case More
More informationRectification and Distortion Correction
Rectification and Distortion Correction Hagen Spies March 12, 2003 Computer Vision Laboratory Department of Electrical Engineering Linköping University, Sweden Contents Distortion Correction Rectification
More informationMultiview Stereo COSC450. Lecture 8
Multiview Stereo COSC450 Lecture 8 Stereo Vision So Far Stereo and epipolar geometry Fundamental matrix captures geometry 8-point algorithm Essential matrix with calibrated cameras 5-point algorithm Intersect
More informationMultiple-View Structure and Motion From Line Correspondences
ICCV 03 IN PROCEEDINGS OF THE IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION, NICE, FRANCE, OCTOBER 003. Multiple-View Structure and Motion From Line Correspondences Adrien Bartoli Peter Sturm INRIA
More informationInvariance of l and the Conic Dual to Circular Points C
Invariance of l and the Conic Dual to Circular Points C [ ] A t l = (0, 0, 1) is preserved under H = v iff H is an affinity: w [ ] l H l H A l l v 0 [ t 0 v! = = w w] 0 0 v = 0 1 1 C = diag(1, 1, 0) is
More informationComputational Optical Imaging - Optique Numerique. -- Single and Multiple View Geometry, Stereo matching --
Computational Optical Imaging - Optique Numerique -- Single and Multiple View Geometry, Stereo matching -- Autumn 2015 Ivo Ihrke with slides by Thorsten Thormaehlen Reminder: Feature Detection and Matching
More informationCSE 252B: Computer Vision II
CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribe: Haowei Liu LECTURE 16 Structure from Motion from Tracked Points 16.1. Introduction In the last lecture we learned how to track point features
More information