Structure from motion
|
|
- Gwen Booker
- 5 years ago
- Views:
Transcription
1 Structure from motion
2 Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera 2 Camera 3?? Slide credit: Noah Snavely
3 Structure from motion Given: m images of n fixed 3D points λ ij x ij P i X j, i 1,, m, j 1,, n Problem: estimate m projection matrices P i and n 3D points X j from the mn correspondences x ij X j x 1j P 1 x 2j x 3j P 2 P 3
4 Outline Reconstruction ambiguities Affine structure from motion Factorization Projective structure from motion Bundle adjustment Modern structure from motion pipeline
5 Is SfM always uniquely solvable? Necker cube Source: N. Snavely
6 Structure from motion ambiguity If we scale the entire scene by some factor k and, at the same time, scale the camera matrices by the factor of 1/k, the projections of the scene points in the image remain exactly the same: x PX æ ç è 1 k ö P ( k ø X) It is impossible to recover the absolute scale of the scene!
7 Structure from motion ambiguity If we scale the entire scene by some factor k and, at the same time, scale the camera matrices by the factor of 1/k, the projections of the scene points in the image remain exactly the same More generally, if we transform the scene using a transformation Q and apply the inverse transformation to the camera matrices, then the images do not change: x PX ( PQ -1)( QX)
8 Projective ambiguity With no constraints on the camera calibration matrix or on the scene, we can reconstruct up to a projective ambiguity Q p é A ê T ëv tù v ú û x PX ( PQ -1 )( Q X) P P
9 Projective ambiguity
10 Affine ambiguity If we impose parallelism constraints, we can get a reconstruction up to an affine ambiguity Affine Q A é A ê T ë0 tù 1 ú û x PX ( PQ -1 )( Q X) A A
11 Affine ambiguity
12 Similarity ambiguity A reconstruction that obeys orthogonality constraints on camera parameters and/or scene Q s ésr ê T ë0 tù 1 ú û x PX ( PQ -1)( Q X) S S
13 Similarity ambiguity
14 Affine structure from motion Let s start with affine or weak perspective cameras (the math is easier) center at infinity
15 Recall: Orthographic Projection Image World Projection along the z direction
16 Affine cameras Orthographic Projection Parallel Projection
17 Affine cameras A general affine camera combines the effects of an affine transformation of the 3D space, orthographic projection, and an affine transformation of the image: Affine projection is a linear mapping + translation in non-homogeneous coordinates ú û ù ê ë é ú ú ú û ù ê ê ê ë é ú ú ú û ù ê ê ê ë é affine] [ affine] 3 [ b A P b a a a b a a a x X a 1 a 2 b AX x + ø ö ç ç è æ + ø ö ç ç ç è æ ú û ù ê ë é ø ö ç è æ b b Z Y X a a a a a a y x Projection of world origin
18 Affine structure from motion Given: m images of n fixed 3D points: xij A i X j + b i, i 1,, m, j 1,, n Problem: use the mn correspondences x ij to estimate m projection matrices A i and translation vectors b i, and n points X j The reconstruction is defined up to an arbitrary affine transformation Q (12 degrees of freedom): éa -1 ê ë 0 bù 1 ú û éa ê ë 0 bù 1 úq û, æ Xö Qç è 1 ø We have 2mn knowns and 8m + 3n unknowns (minus 12 dof for affine ambiguity) Thus, we must have 2mn > 8m + 3n 12 For two views, we need four point correspondences æ ç è X 1 ö ø
19 Affine structure from motion Centering: subtract the centroid of the image points in each view xˆ ij x ij - 1 n n å k 1 x ik A X i j + b i - 1 n n å k 1 ( A X + b ) i k i A i æ ç è X j - 1 n n å k 1 X k ö ø A Xˆ i j For simplicity, set the origin of the world coordinate system to the centroid of the 3D points After centering, each normalized 2D point is related to the 3D point X j by x ˆ A ij i X j
20 Affine structure from motion D Let s create a 2m n data (measurement) matrix: éxˆ ê ê xˆ ê ê ëxˆ m1 xˆ xˆ xˆ m2!! "! xˆ xˆ xˆ 1n 2n mn ù ú ú ú ú û cameras (2m) points (n) C. Tomasi and T. Kanade. Shape and motion from image streams under orthography: A factorization method. IJCV, 9(2): , November 1992.
21 Affine structure from motion Let s create a 2m n data (measurement) matrix: [ ] n m mn m m n n X X X A A A x x x x x x x x x D! "! #!! ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ú ú ú ú û ù ê ê ê ê ë é ú ú ú ú û ù ê ê ê ê ë é cameras (2m 3) points (3 n) The measurement matrix D MS must have rank 3! C. Tomasi and T. Kanade. Shape and motion from image streams under orthography: A factorization method. IJCV, 9(2): , November 1992.
22 Factorizing the measurement matrix Source: M. Hebert
23 Factorizing the measurement matrix Singular value decomposition of D: Source: M. Hebert
24 Factorizing the measurement matrix Singular value decomposition of D: Source: M. Hebert
25 Factorizing the measurement matrix Obtaining a factorization from SVD: Source: M. Hebert
26 Factorizing the measurement matrix Obtaining a factorization from SVD: This decomposition minimizes D-MS 2 Source: M. Hebert
27 Affine ambiguity The decomposition is not unique. We get the same D by using any 3 3 matrix C and applying the transformations M MC, S C -1 S That is because we have only an affine transformation and we have not enforced any Euclidean constraints (like forcing the image axes to be perpendicular, for example) Source: M. Hebert
28 Eliminating the affine ambiguity Transform each projection matrix A to another matrix AC to get orthographic projection Image axes are perpendicular and scale is 1 a 2 a 1 x a 1 a 2 0 a 1 2 a X This translates into 3m equations: (A i C)(A i C) T A i (CC T )A i Id, i 1,, m Solve for L CC T Recover C from L by Cholesky decomposition: L CC T Update M and S: M MC, S C -1 S Source: M. Hebert
29 Reconstruction results C. Tomasi and T. Kanade, Shape and motion from image streams under orthography: A factorization method, IJCV 1992
30 Dealing with missing data So far, we have assumed that all points are visible in all views In reality, the measurement matrix typically looks something like this: cameras points Possible solution: decompose matrix into dense subblocks, factorize each sub-block, and fuse the results Finding dense maximal sub-blocks of the matrix is NPcomplete (equivalent to finding maximal cliques in a graph)
31 Dealing with missing data Incremental bilinear refinement (1) Perform factorization on a dense sub-block (2) Solve for a new 3D point visible by at least two known cameras (triangulation) (3) Solve for a new camera that sees at least three known 3D points (calibration) F. Rothganger, S. Lazebnik, C. Schmid, and J. Ponce. Segmenting, Modeling, and Matching Video Clips Containing Multiple Moving Objects. PAMI 2007.
32 Projective structure from motion Given: m images of n fixed 3D points λijxij Pi Xj, i 1,, m, j 1,, n Problem: estimate m projection matrices P i and n 3D points Xj from the mn correspondences xij X j x 1j P 1 x 2j x 3j P 2 P 3
33 Projective structure from motion Given: m images of n fixed 3D points λijxij Pi Xj, i 1,, m, j 1,, n Problem: estimate m projection matrices P i and n 3D points Xj from the mn correspondences xij With no calibration info, cameras and points can only be recovered up to a 4x4 projective transformation Q: X QX, P PQ -1 We can solve for structure and motion when 2mn > 11m +3n 15 For two cameras, at least 7 points are needed
34 Projective SFM: Two-camera case Compute fundamental matrix F between the two views First camera matrix: [I 0] Second camera matrix: [A b] Then b is the epipole (F T b 0), A [b ]F F&P sec
35 Incremental structure from motion Initialize motion from two images using fundamental matrix Initialize structure by triangulation points For each additional view: Determine projection matrix of new camera using all the known 3D points that are visible in its image calibration cameras
36 Incremental structure from motion Initialize motion from two images using fundamental matrix Initialize structure by triangulation points For each additional view: Determine projection matrix of new camera using all the known 3D points that are visible in its image calibration cameras Refine and extend structure: compute new 3D points, re-optimize existing points that are also seen by this camera triangulation
37 Incremental structure from motion Initialize motion from two images using fundamental matrix Initialize structure by triangulation points For each additional view: Determine projection matrix of new camera using all the known 3D points that are visible in its image calibration cameras Refine and extend structure: compute new 3D points, re-optimize existing points that are also seen by this camera triangulation Refine structure and motion: bundle adjustment
38 Bundle adjustment Non-linear method for refining structure and motion Minimize reprojection error m n w ij i1 j1 x ij 1 λ ij P i X j 2 X j visibility flag: is point j visible in view i? P 1 X j P 1 x 1j P 2 X j x 2j P 3 X j x 3j P 2 P 3
39 Representative SFM pipeline N. Snavely, S. Seitz, and R. Szeliski, Photo tourism: Exploring photo collections in 3D, SIGGRAPH 2006.
40 Feature detection Detect SIFT features Source: N. Snavely
41 Feature detection Detect SIFT features Source: N. Snavely
42 Feature matching Match features between each pair of images Source: N. Snavely
43 Feature matching Use RANSAC to estimate fundamental matrix between each pair Source: N. Snavely
44 Feature matching Use RANSAC to estimate fundamental matrix between each pair Image source
45 Feature matching Use RANSAC to estimate fundamental matrix between each pair Source: N. Snavely
46 Image connectivity graph (graph layout produced using the Graphviz toolkit: Source: N. Snavely
47 Incremental SFM Pick a pair of images with lots of inliers (and preferably, good EXIF data) Initialize intrinsic parameters (focal length, principal point) from EXIF Estimate extrinsic parameters (R and t) using five-point algorithm Use triangulation to initialize model points While remaining images exist Find an image with many feature matches with images in the model Run RANSAC on feature matches to register new image to model Triangulate new points Perform bundle adjustment to re-optimize everything
48 The devil is in the details Handling degenerate configurations (e.g., homographies) Eliminating outliers Dealing with repetitions and symmetries Handling multiple connected components Closing loops Making the whole thing efficient! See, e.g., Towards Linear-Time Incremental Structure from Motion
49 SFM software Bundler OpenSfM OpenMVG VisualSFM See also Wikipedia s list of toolboxes
50 Review: Structure from motion Ambiguity Affine structure from motion Factorization Dealing with missing data Incremental structure from motion Projective structure from motion Bundle adjustment Modern structure from motion pipeline
Structure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t R 2 3,t 3 Camera 1 Camera
More informationStructure from Motion CSC 767
Structure from Motion CSC 767 Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R,t R 2,t 2 R 3,t 3 Camera??
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera
More informationBIL Computer Vision Apr 16, 2014
BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm
More informationStructure from Motion
Structure from Motion Outline Bundle Adjustment Ambguities in Reconstruction Affine Factorization Extensions Structure from motion Recover both 3D scene geoemetry and camera positions SLAM: Simultaneous
More informationC280, Computer Vision
C280, Computer Vision Prof. Trevor Darrell trevor@eecs.berkeley.edu Lecture 11: Structure from Motion Roadmap Previous: Image formation, filtering, local features, (Texture) Tues: Feature-based Alignment
More informationStructure from Motion
/8/ Structure from Motion Computer Vision CS 43, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from motion
More informationStructure from Motion
11/18/11 Structure from Motion Computer Vision CS 143, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from
More informationLecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15
Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15 Lecture 6 Stereo Systems Multi- view geometry Stereo systems
More informationCEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.
CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Session 4 Affine Structure from Motion Mani Golparvar-Fard Department of Civil and Environmental Engineering 329D, Newmark Civil Engineering
More informationLecture 10: Multi view geometry
Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationLecture 10: Multi-view geometry
Lecture 10: Multi-view geometry Professor Stanford Vision Lab 1 What we will learn today? Review for stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationCS 532: 3D Computer Vision 7 th Set of Notes
1 CS 532: 3D Computer Vision 7 th Set of Notes Instructor: Philippos Mordohai Webpage: www.cs.stevens.edu/~mordohai E-mail: Philippos.Mordohai@stevens.edu Office: Lieb 215 Logistics No class on October
More informationStructure from Motion
Structure from Motion Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz, N Snavely, and D. Hoiem Administrative stuffs HW 3 due 11:55 PM, Oct 17 (Wed) Submit your alignment results!
More informationCS231M Mobile Computer Vision Structure from motion
CS231M Mobile Computer Vision Structure from motion - Cameras - Epipolar geometry - Structure from motion Pinhole camera Pinhole perspective projection f o f = focal length o = center of the camera z y
More informationLecture 6 Stereo Systems Multi-view geometry
Lecture 6 Stereo Systems Multi-view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-5-Feb-4 Lecture 6 Stereo Systems Multi-view geometry Stereo systems
More informationStructure from Motion
04/4/ Structure from Motion Comuter Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Many slides adated from Lana Lazebnik, Silvio Saverese, Steve Seitz his class: structure from motion Reca
More informationImage correspondences and structure from motion
Image correspondences and structure from motion http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 20 Course announcements Homework 5 posted.
More informationComputer Vision I - Algorithms and Applications: Multi-View 3D reconstruction
Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction Carsten Rother 09/12/2013 Computer Vision I: Multi-View 3D reconstruction Roadmap this lecture Computer Vision I: Multi-View
More informationUnit 3 Multiple View Geometry
Unit 3 Multiple View Geometry Relations between images of a scene Recovering the cameras Recovering the scene structure http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook1.html 3D structure from images Recover
More informationCamera Geometry II. COS 429 Princeton University
Camera Geometry II COS 429 Princeton University Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence
More informationCS231A Midterm Review. Friday 5/6/2016
CS231A Midterm Review Friday 5/6/2016 Outline General Logistics Camera Models Non-perspective cameras Calibration Single View Metrology Epipolar Geometry Structure from Motion Active Stereo and Volumetric
More informationCamera Calibration. COS 429 Princeton University
Camera Calibration COS 429 Princeton University Point Correspondences What can you figure out from point correspondences? Noah Snavely Point Correspondences X 1 X 4 X 3 X 2 X 5 X 6 X 7 p 1,1 p 1,2 p 1,3
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Shiv Ram Dubey, IIIT Sri City Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X
More informationLecture 9: Epipolar Geometry
Lecture 9: Epipolar Geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Why is stereo useful? Epipolar constraints Essential and fundamental matrix Estimating F (Problem Set 2
More informationLecture 5 Epipolar Geometry
Lecture 5 Epipolar Geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 5-24-Jan-18 Lecture 5 Epipolar Geometry Why is stereo useful? Epipolar constraints Essential
More informationLast lecture. Passive Stereo Spacetime Stereo
Last lecture Passive Stereo Spacetime Stereo Today Structure from Motion: Given pixel correspondences, how to compute 3D structure and camera motion? Slides stolen from Prof Yungyu Chuang Epipolar geometry
More informationStep-by-Step Model Buidling
Step-by-Step Model Buidling Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure
More informationThe end of affine cameras
The end of affine cameras Affine SFM revisited Epipolar geometry Two-view structure from motion Multi-view structure from motion Planches : http://www.di.ens.fr/~ponce/geomvis/lect3.pptx http://www.di.ens.fr/~ponce/geomvis/lect3.pdf
More informationStructure from Motion. Introduction to Computer Vision CSE 152 Lecture 10
Structure from Motion CSE 152 Lecture 10 Announcements Homework 3 is due May 9, 11:59 PM Reading: Chapter 8: Structure from Motion Optional: Multiple View Geometry in Computer Vision, 2nd edition, Hartley
More informationStereo and Epipolar geometry
Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka
More informationCS231A Course Notes 4: Stereo Systems and Structure from Motion
CS231A Course Notes 4: Stereo Systems and Structure from Motion Kenji Hata and Silvio Savarese 1 Introduction In the previous notes, we covered how adding additional viewpoints of a scene can greatly enhance
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationGeometric camera models and calibration
Geometric camera models and calibration http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 13 Course announcements Homework 3 is out. - Due October
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationMultiple Views Geometry
Multiple Views Geometry Subhashis Banerjee Dept. Computer Science and Engineering IIT Delhi email: suban@cse.iitd.ac.in January 2, 28 Epipolar geometry Fundamental geometric relationship between two perspective
More informationVisual Recognition: Image Formation
Visual Recognition: Image Formation Raquel Urtasun TTI Chicago Jan 5, 2012 Raquel Urtasun (TTI-C) Visual Recognition Jan 5, 2012 1 / 61 Today s lecture... Fundamentals of image formation You should know
More informationCS 664 Structure and Motion. Daniel Huttenlocher
CS 664 Structure and Motion Daniel Huttenlocher Determining 3D Structure Consider set of 3D points X j seen by set of cameras with projection matrices P i Given only image coordinates x ij of each point
More informationEpipolar Geometry and Stereo Vision
CS 1674: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 5, 2016 Announcement Please send me three topics you want me to review next
More informationStereo Vision. MAN-522 Computer Vision
Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in
More informationStereo II CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo II CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Camera parameters A camera is described by several parameters Translation T of the optical center from the origin of world
More informationEpipolar geometry. x x
Two-view geometry Epipolar geometry X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections
More informationLecture 8.2 Structure from Motion. Thomas Opsahl
Lecture 8.2 Structure from Motion Thomas Opsahl More-than-two-view geometry Correspondences (matching) More views enables us to reveal and remove more mismatches than we can do in the two-view case More
More informationProjective geometry for 3D Computer Vision
Subhashis Banerjee Computer Science and Engineering IIT Delhi Dec 16, 2015 Overview Pin-hole camera Why projective geometry? Reconstruction Computer vision geometry: main problems Correspondence problem:
More informationStructure from Motion and Multi- view Geometry. Last lecture
Structure from Motion and Multi- view Geometry Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 5 Last lecture S. J. Gortler, R. Grzeszczuk, R. Szeliski,M. F. Cohen The Lumigraph, SIGGRAPH,
More informationMulti-view geometry problems
Multi-view geometry Multi-view geometry problems Structure: Given projections o the same 3D point in two or more images, compute the 3D coordinates o that point? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera
More information1 Projective Geometry
CIS8, Machine Perception Review Problem - SPRING 26 Instructions. All coordinate systems are right handed. Projective Geometry Figure : Facade rectification. I took an image of a rectangular object, and
More informationMei Han Takeo Kanade. January Carnegie Mellon University. Pittsburgh, PA Abstract
Scene Reconstruction from Multiple Uncalibrated Views Mei Han Takeo Kanade January 000 CMU-RI-TR-00-09 The Robotics Institute Carnegie Mellon University Pittsburgh, PA 1513 Abstract We describe a factorization-based
More informationSrikumar Ramalingam. Review. 3D Reconstruction. Pose Estimation Revisited. School of Computing University of Utah
School of Computing University of Utah Presentation Outline 1 2 3 Forward Projection (Reminder) u v 1 KR ( I t ) X m Y m Z m 1 Backward Projection (Reminder) Q K 1 q Presentation Outline 1 2 3 Sample Problem
More informationWeek 2: Two-View Geometry. Padua Summer 08 Frank Dellaert
Week 2: Two-View Geometry Padua Summer 08 Frank Dellaert Mosaicking Outline 2D Transformation Hierarchy RANSAC Triangulation of 3D Points Cameras Triangulation via SVD Automatic Correspondence Essential
More informationEpipolar Geometry and Stereo Vision
CS 1699: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 8, 2015 Today Review Projective transforms Image stitching (homography) Epipolar
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationN-View Methods. Diana Mateus, Nassir Navab. Computer Aided Medical Procedures Technische Universität München. 3D Computer Vision II
1/66 N-View Methods Diana Mateus, Nassir Navab Computer Aided Medical Procedures Technische Universität München 3D Computer Vision II Inspired by Slides from Adrien Bartoli 2/66 Outline 1 Structure from
More informationCS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
More informationCreating 3D Models with Uncalibrated Cameras
Creating D Models with Uncalibrated Cameras Mei Han Takeo Kanade Robotics Institute, Carnegie Mellon University meihan,tk@cs.cmu.edu Abstract We describe a factorization-based method to recover D models
More information3D Computer Vision. Structure from Motion. Prof. Didier Stricker
3D Computer Vision Structure from Motion Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Structure
More informationComputer Vision in a Non-Rigid World
2 nd Tutorial on Computer Vision in a Non-Rigid World Dr. Lourdes Agapito Prof. Adrien Bartoli Dr. Alessio Del Bue Non-rigid Structure from Motion Non-rigid structure from motion To extract non-rigid 3D
More informationStructure from motion
Multi-view geometry Structure rom motion Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera 3 R 3,t 3 Figure credit: Noah Snavely Structure rom motion? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera 3 R 3,t 3 Structure:
More informationPin Hole Cameras & Warp Functions
Pin Hole Cameras & Warp Functions Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Pinhole Camera. Homogenous Coordinates. Planar Warp Functions. Motivation Taken from: http://img.gawkerassets.com/img/18w7i1umpzoa9jpg/original.jpg
More informationTwo-view geometry Computer Vision Spring 2018, Lecture 10
Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of
More informationHomogeneous Coordinates. Lecture18: Camera Models. Representation of Line and Point in 2D. Cross Product. Overall scaling is NOT important.
Homogeneous Coordinates Overall scaling is NOT important. CSED44:Introduction to Computer Vision (207F) Lecture8: Camera Models Bohyung Han CSE, POSTECH bhhan@postech.ac.kr (",, ) ()", ), )) ) 0 It is
More informationMultiple Motion Scene Reconstruction from Uncalibrated Views
Multiple Motion Scene Reconstruction from Uncalibrated Views Mei Han C & C Research Laboratories NEC USA, Inc. meihan@ccrl.sj.nec.com Takeo Kanade Robotics Institute Carnegie Mellon University tk@cs.cmu.edu
More informationInvariance of l and the Conic Dual to Circular Points C
Invariance of l and the Conic Dual to Circular Points C [ ] A t l = (0, 0, 1) is preserved under H = v iff H is an affinity: w [ ] l H l H A l l v 0 [ t 0 v! = = w w] 0 0 v = 0 1 1 C = diag(1, 1, 0) is
More informationHomographies and RANSAC
Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position
More informationA Factorization Method for Structure from Planar Motion
A Factorization Method for Structure from Planar Motion Jian Li and Rama Chellappa Center for Automation Research (CfAR) and Department of Electrical and Computer Engineering University of Maryland, College
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263
Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine
More informationEpipolar Geometry class 11
Epipolar Geometry class 11 Multiple View Geometry Comp 290-089 Marc Pollefeys Multiple View Geometry course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective 2D Geometry Jan. 14, 16
More informationContents. 1 Introduction Background Organization Features... 7
Contents 1 Introduction... 1 1.1 Background.... 1 1.2 Organization... 2 1.3 Features... 7 Part I Fundamental Algorithms for Computer Vision 2 Ellipse Fitting... 11 2.1 Representation of Ellipses.... 11
More informationComputer Vision CSCI-GA Assignment 1.
Computer Vision CSCI-GA.2272-001 Assignment 1. September 22, 2017 Introduction This assignment explores various methods for aligning images and feature extraction. There are four parts to the assignment:
More informationTwo-View Geometry (Course 23, Lecture D)
Two-View Geometry (Course 23, Lecture D) Jana Kosecka Department of Computer Science George Mason University http://www.cs.gmu.edu/~kosecka General Formulation Given two views of the scene recover the
More informationSrikumar Ramalingam. Review. 3D Reconstruction. Pose Estimation Revisited. School of Computing University of Utah
School of Computing University of Utah Presentation Outline 1 2 3 Forward Projection (Reminder) u v 1 KR ( I t ) X m Y m Z m 1 Backward Projection (Reminder) Q K 1 q Q K 1 u v 1 What is pose estimation?
More informationFinal project bits and pieces
Final project bits and pieces The project is expected to take four weeks of time for up to four people. At 12 hours per week per person that comes out to: ~192 hours of work for a four person team. Capstone:
More informationThere are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...
STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own
More information3D Reconstruction from Two Views
3D Reconstruction from Two Views Huy Bui UIUC huybui1@illinois.edu Yiyi Huang UIUC huang85@illinois.edu Abstract In this project, we study a method to reconstruct a 3D scene from two views. First, we extract
More informationToday s lecture. Structure from Motion. Today s lecture. Parameterizing rotations
Today s lecture Structure from Motion Computer Vision CSE576, Spring 2008 Richard Szeliski Geometric camera calibration camera matrix (Direct Linear Transform) non-linear least squares separating intrinsics
More informationVision 3D articielle Multiple view geometry
Vision 3D articielle Multiple view geometry Pascal Monasse monasse@imagine.enpc.fr IMAGINE, École des Ponts ParisTech Contents Multi-view constraints Multi-view calibration Incremental calibration Global
More informationEECS 442: Final Project
EECS 442: Final Project Structure From Motion Kevin Choi Robotics Ismail El Houcheimi Robotics Yih-Jye Jeffrey Hsu Robotics Abstract In this paper, we summarize the method, and results of our projective
More informationPin Hole Cameras & Warp Functions
Pin Hole Cameras & Warp Functions Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Pinhole Camera. Homogenous Coordinates. Planar Warp Functions. Example of SLAM for AR Taken from:
More informationSingle-view metrology
Single-view metrology Magritte, Personal Values, 952 Many slides from S. Seitz, D. Hoiem Camera calibration revisited What if world coordinates of reference 3D points are not known? We can use scene features
More informationComputer Vision Projective Geometry and Calibration. Pinhole cameras
Computer Vision Projective Geometry and Calibration Professor Hager http://www.cs.jhu.edu/~hager Jason Corso http://www.cs.jhu.edu/~jcorso. Pinhole cameras Abstract camera model - box with a small hole
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253
Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine
More informationFundamental Matrix & Structure from Motion
Fundamental Matrix & Structure from Motion Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Transformations between images Structure from Motion The Essential Matrix The Fundamental
More informationMulti-view 3D reconstruction. Problem formulation Projective ambiguity Rectification Autocalibration Feature points and their matching
Multi-view 3D reconstruction Problem formulation Projective ambiguity Rectification Autocalibration Feature points and their matching Problem Given m images of n scene points captured from different viewpoints,
More informationLecture 3: Camera Calibration, DLT, SVD
Computer Vision Lecture 3 23--28 Lecture 3: Camera Calibration, DL, SVD he Inner Parameters In this section we will introduce the inner parameters of the cameras Recall from the camera equations λx = P
More informationInstance-level recognition part 2
Visual Recognition and Machine Learning Summer School Paris 2011 Instance-level recognition part 2 Josef Sivic http://www.di.ens.fr/~josef INRIA, WILLOW, ENS/INRIA/CNRS UMR 8548 Laboratoire d Informatique,
More informationStereo CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Why do we perceive depth? What do humans use as depth cues? Motion Convergence When watching an object close to us, our eyes
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem
More informationCS 231A: Computer Vision (Winter 2018) Problem Set 2
CS 231A: Computer Vision (Winter 2018) Problem Set 2 Due Date: Feb 09 2018, 11:59pm Note: In this PS, using python2 is recommended, as the data files are dumped with python2. Using python3 might cause
More informationIntroduction to Computer Vision
Introduction to Computer Vision Michael J. Black Nov 2009 Perspective projection and affine motion Goals Today Perspective projection 3D motion Wed Projects Friday Regularization and robust statistics
More informationGeometry for Computer Vision
Geometry for Computer Vision Lecture 5b Calibrated Multi View Geometry Per-Erik Forssén 1 Overview The 5-point Algorithm Structure from Motion Bundle Adjustment 2 Planar degeneracy In the uncalibrated
More informationComputational Optical Imaging - Optique Numerique. -- Single and Multiple View Geometry, Stereo matching --
Computational Optical Imaging - Optique Numerique -- Single and Multiple View Geometry, Stereo matching -- Autumn 2015 Ivo Ihrke with slides by Thorsten Thormaehlen Reminder: Feature Detection and Matching
More informationEECS 442 Computer vision. Stereo systems. Stereo vision Rectification Correspondence problem Active stereo vision systems
EECS 442 Computer vision Stereo systems Stereo vision Rectification Correspondence problem Active stereo vision systems Reading: [HZ] Chapter: 11 [FP] Chapter: 11 Stereo vision P p p O 1 O 2 Goal: estimate
More informationLarge Scale 3D Reconstruction by Structure from Motion
Large Scale 3D Reconstruction by Structure from Motion Devin Guillory Ziang Xie CS 331B 7 October 2013 Overview Rome wasn t built in a day Overview of SfM Building Rome in a Day Building Rome on a Cloudless
More informationA Systems View of Large- Scale 3D Reconstruction
Lecture 23: A Systems View of Large- Scale 3D Reconstruction Visual Computing Systems Goals and motivation Construct a detailed 3D model of the world from unstructured photographs (e.g., Flickr, Facebook)
More informationImproving Initial Estimations for Structure from Motion Methods
Improving Initial Estimations for Structure from Motion Methods University of Bonn Outline Motivation Computer-Vision Basics Stereo Vision Bundle Adjustment Feature Matching Global Initial Estimation Component
More informationMetric Structure from Motion
CS443 Final Project Metric Structure from Motion Peng Cheng 1 Objective of the Project Given: 1. A static object with n feature points and unknown shape. 2. A camera with unknown intrinsic parameters takes
More informationStructure from Motion
Structure from Motion Lecture-13 Moving Light Display 1 Shape from Motion Problem Given optical flow or point correspondences, compute 3-D motion (translation and rotation) and shape (depth). 2 S. Ullman
More informationLecture 9 & 10: Stereo Vision
Lecture 9 & 10: Stereo Vision Professor Fei- Fei Li Stanford Vision Lab 1 What we will learn today? IntroducEon to stereo vision Epipolar geometry: a gentle intro Parallel images Image receficaeon Solving
More informationSegmenting, Modeling, and Matching Video Clips Containing Multiple Moving Objects
1 Segmenting, Modeling, and Matching Video Clips Containing Multiple Moving Objects Fred Rothganger, Member, IEEE, Svetlana Lazebnik, Member, IEEE Cordelia Schmid, Senior Member, IEEE Jean Ponce, IEEE
More information