Stereo Vision. MAN-522 Computer Vision
|
|
- Lesley Gray
- 5 years ago
- Views:
Transcription
1 Stereo Vision MAN-522 Computer Vision
2 What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in space. The term binocular vision is used when two cameras are employed.
3 Stereo setup Cameras in arbitrary position and orientation: More convenient setup:
4 Fixation point: the point of intersection of the optical axes. Baseline: the distance between the centers of projection. Stereo terminology Conjugate pair: any point in the scene that is visible in both cameras will be projected to a pair of image points in the two images (corresponding points).
5 Stereo terminology (cont d) Disparity: the distance between corresponding points when the two images are superimposed. Disparity map: the disparities of all points form the disparity map.
6 Triangulation Determines the position of a point in space by finding the intersection of the two lines passing through the center of projection and the projection of the point in each image.
7 The two problems of stereo (1) The correspondence problem. (2) The reconstruction problem.
8 The correspondence problem Finding pairs of matched points such that each point in the pair is the projection of the same 3D point. left camera right camera
9 The correspondence problem (cont d) Triangulation depends crucially on the solution of the correspondence problem! Ambiguous correspondences may lead to several different consistent interpretations of the scene.
10 The reconstruction problem Given the corresponding points, we compute the disparity map. The disparity map can be converted to a 3D map of the scene assuming that the stereo geometry is known.
11 Recovering depth (i.e., reconstruction) Consider recovering the position of P from its projections p l and p r. P l : P in left camera coordinates P r : P in right camera coordinates right camera P r =(X r,y r,z r ) left camera P l =(X l,y l,z l )
12 Recovering depth (cont d) Suppose that the two cameras are related by the following transformation: Using Z r = Z l = Z and X r = X l - T we have: i.e., (R, -T) aligns right camera with left camera where d=x l x r is the disparity (i.e., the difference in the position between the corresponding points in the two images)
13 Stereo camera parameters Extrinsic parameters (R, T): describe the relative position and orientation of the two cameras. Intrinsic parameters: characterize the transformation from image plane coordinates to pixel coordinates, in each camera.
14 Stereo camera parameters (cont d) R and T can be determined from the extrinsic parameters of each camera: (T l,r l ) align left camera with world (T r, R r ) align right camera with world (see homework)
15 Correspondence problem Some points in each image will have no corresponding points in the other image. i.e., cameras might have different field of view. A stereo system must be able to determine the image parts that should not be matched.
16 Correspondence problem (cont d) Two main approaches: Intensity-based: attempt to establish a correspondence by matching image intensities. Feature-based: attempt to establish a correspondence by matching a sparse sets of image features.
17 Intensity-based Methods Match image sub-windows between the two images (e.g., using correlation). left right
18 Correlation-based Methods p l R(p l ) I l I r
19 Correlation-based Methods (cont d) Normalized cross-correlation: Normalize c(d) by subtracting the mean and dividing by the standard deviation.
20 Correlation-based Methods (cont d) The success of correlation-based methods depends on whether the image window in one image exhibits a distinctive structure that occurs infrequently in the search region of the other image. left right
21 Correlation-based Methods (cont d) How to choose the window size W? Too small a window may not capture enough image structure, and may be too noise sensitive (i.e., less distinctive, many false matches). Too large a window makes matching less sensitive to noise (desired) but also harder to match. left right
22 Correlation-based Methods (cont d) An adaptive searching window has been proposed in the literature (i.e., adapt both shape and size). original disparities 128 x 128 images 3 x 3 7 x 7 3D info recovered using adaptive window adaptive T. Kanade and M. Okutomi, A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 16 No. 9, 1994.
23 Correlation-based Methods (cont d) How to choose the size and location of R( p l )? If the distance of the fixating point from the cameras is much larger than the baseline, the location of R( p l ) can be chosen to be the same as the location of p l. The size of R( p l ) can be estimated from the maximum range of disparities (or depths) we expect to find in the scene. p l R(p l ) I l I r
24 Wide Baseline Stereo Matching More powerful methods are needed when dealing with wide baseline stereo images (i.e., affine invariant region matching). T. Tuytelaars and L. Van Gool, "Wide Baseline Stereo Matching based on Local, Affinely Invariant Regions", British Machine Vision Conference, 2000.
25 Feature-based Methods Look for a feature in an image that matches a feature in the other. Edge points Line segments Corners A set of geometric features are used for matching.
26 Feature-based Methods (cont d) For example, a line feature descriptor could contain: the length, l the orientation, θ the coordinates of the midpoint, m the average intensity along the line, i Similarity measures are based on matching feature descriptors, e.g., line matching can be done as follows: where w 0,..., w 3 are weights
27 Intensity-based vs feature-based approaches Intensity-based methods Provide a dense disparity map. Need textured images to work well. Sensitive to illumination changes. Feature-based methods: Faster than correlation-based methods. Provide sparse disparity maps. Relatively insensitive to illumination changes.
28 Structured lighting Feature-based methods cannot be used when objects have smooth surfaces or surfaces of uniform intensity. Patterns of light can be projected onto the surface of objects, creating interesting points even in regions which would be otherwise smooth.
29 Search Region R(p l ) Finding corresponding points is time consuming! Can we reduce the size of the search region? YES p l R(p l ) I l I r
30 Epipolar Plane and Lines Epipolar plane: the plane passing through the centers of projection and the point in the scene. Epipolar line: the intersection of the epipolar plane with the image plane.
31 Epipoles Left epipole: the projection of O r on the left image plane. Right epipole: the projection of O l on the right image plane.
32 Epipoles (cont d) All epipolar lines go through the camera s epipole
33 Epipoles (cont d) All epipolar lines go through the camera s epipole
34 Epipoles (cont d) If the line through the center of projection is parallel to one of the image planes, the corresponding epipole is at infinity. one epipole at infinity two epipoles at infinity
35 Epipolar constraint Given p l, P can lie anywhere on the ray from O l through p l. The image of this ray in the right image image is the epipolar line through the corresponding point p r. p l Mapping between points in the left image and lines in the right image and vice versa. O l O r
36 Importance of epipolar constraint The search for correspondences is reduced to 1D. Very effective for rejecting false matches due to occlusion. p l p l R(p l ) I l I r O l O r
37 Epipolar Lines: Example
38 Ordering of corresponding points Conjugate points along corresponding epipolar lines have the same order in each image. Exception: when corresponding points lie on the same epipolar plane and imaged from different sides.
39 Estimating epipolar geometry Given a point in one image, how do we find its epipolar line in the other image? Need to estimate the epipolar geometry which is encoded by two matrices: Essential matrix Fundamental matrix p l O l O r
40 Quick Review Cross product Homogeneous representation of lines
41 Vector Cross Product
42 Homogeneous (projective) representation of lines A line ax + by + c = 0 is represented by the homogeneous vector below (projective line): Any vector represents the same line.
43 Homogeneous (projective) representation of lines Duality: in homogeneous (projective) coordinates, points and lines are dual (we can interchange their roles). Some properties involving points and lines: (1) The point x lies on the line iff x T l = 0 (2) Two lines l, m define a point p: p = l x m (3) Two points p, q define a line l: l = p x q
44 Homogeneous (projective) representation of lines: Quick Review
45 Homogeneous (projective) representation of lines
46 Homogeneous (projective) representation of lines w=0, therefore, they intersect at infinity!
47 The essential matrix: E The equation of the epipolar plane is given by the following co-planarity condition (assume that the world coordinate system is aligned with the left camera): normal to epipolar plane
48 The essential matrix: E (cont d) Using P r = R(P l - T) we have: (R T P r ) T (T x P l ) = 0 Expressing cross product as matrix multiplication we have: where E = RS is called the essential matrix. (the matrix S is always rank deficient, i.e., rank(s) = 2)
49 The essential matrix: E (cont d) Using image plane coordinates: p l =fp l /Z l p r =fp r /Z r Z l =Z r The equation defines a mapping between points and epipolar lines (in homogeneous coordinates). p l p r We will revisit this later
50 The essential matrix: E (cont d) Properties of the essential matrix E: (1) Encodes the extrinsic parameters only (2) Has rank 2 (3) Its two nonzero singular values are equal
51 The essential matrix: Example
52 The fundamental matrix: F Suppose that M l and M r are the matrices of the intrinsic parameters of the left and right camera, then the pixel coordinates and of p l and p r are: Using the above equations and we have: p M p 1 r r r p M p 1 l l l 1 1 ( ) T Mr pr E( Ml pl ) 0 where fundamental matrix is called the
53 The fundamental matrix: F (cont d) Properties of the fundamental matrix: (1)Encodes both the extrinsic and intrinsic parameters (2) Has rank 2
54 The fundamental matrix: F (cont d)
55 Estimating F (or E): eight-point algorithm We can estimate the fundamental matrix from point correspondences only (i.e., without information at all on the extrinsic or intrinsic camera parameters). Each correspondence leads to a homogeneous equation of the form: or
56 Estimating F (or E): the eight-point algorithm (cont d) We can determine the entries of the matrix F (up to an unknown scale factor) by establishing correspondences: A is rank deficient (i.e., rank(a) = 8) The solution is unique up to a scale factor (i.e., proportional to the last column of V where A = UDV T ).
57 Estimating F (or E): the eight-point algorithm (cont d) (i.e., corresponding to the smallest singular value)
58 Estimating F (or E): the eight-point algorithm (cont d) Uncorrected F Corrected F Epipolar lines must intersect at the epipole!
59 Estimating F (or E): the eight-point algorithm (cont d) Normalization R. Hartley, "In Defense of the Eight-Point Algorithm", IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(6): , 1997.
60 Finding the epipolar lines The equation below defines a mapping between points and epipolar lines: u l u r p l p r Reminder: a point x lies on a line l iff x T l = 0
61 Finding the epipolar lines (cont d) u l u r p l p r ( p T Fp ) T 0 or p T F T p 0 r l l r
62 Locating the epipoles from F (or E)
63 Locating the epipoles from F (or E) (cont d) The solution is proportional to the last column of U in the SVD decomposition of F:
64 Rectification Epipolar lines are at arbitrary locations and orientations Epipolar lines are colinear and parallel to baseline
65 Rectification (cont d) Rectification is a transformation which makes pairs of conjugate epipolar lines become collinear and parallel to the horizontal axis (i.e., baseline) Searching for corresponding points becomes much simpler for the case of rectified images u l u r rectification u l u r p l p r p l p r
66 Rectification (cont d) Disparities between the images are in the x-direction only (i.e., no y disparity)
67 Rectification: Example before rectification after rectification
68 Rectification (cont d) Main steps (assuming knowledge of the extrinsic/intrinsic stereo parameters): (1) Rotate the left camera so that the epipolar lines become parallel to the horizontal axis (i.e., epipole is mapped to infinity).
69 Rectification (cont d) (2) Apply the same rotation to the right camera to recover the original geometry. (3) Align right camera with left camera using rotation R. (4) Adjust scale in both camera frames.
70 Rectification (cont d) Consider step (1) only (i.e., other steps are easy): Construct a coordinate system (e 1, e 2, e 3 ) centered at O l. Aligning it with image plane coordinate system.
71 Rectification (cont d) (1.1) e 1 is a unit vector along the vector T (baseline)
72 Rectification (cont d) (1.2) e 2 must be perpendicular to e 1 (i.e., cross product of e 1 with the optical axis) z=[0,0,1] T
73 Rectification (cont d) (1.3) choose e 3 as the cross product of e 1 and e 2
74 Rectification (cont d) The rotation matrix that maps the left epipole to infinity is the transformation that aligns (e 1, e 2, e 3 ) with (i,j, k):
75 The reconstruction problem (1) Both intrinsic and extrinsic parameters are known: we can solve the reconstruction problem unambiguously by triangulation. (2) Only the intrinsic parameters are known: we can solve the reconstruction problem only up to an unknown scaling factor. (3) Neither the extrinsic nor the intrinsic parameters are available: we can solve the reconstruction problem only up to an unknown, global projective transformation.
76 (1) Reconstruction by triangulation Assumptions and problem statement (1) Both the extrinsic and intrinsic camera parameters are known. (2) Compute the location of the 3D points from their projections p l and p r
77 What is the solution? The point P lies at the intersection of the two rays from O l through p l and from O r through p r
78 Practical issues The two rays will not intersect exactly in space because of errors in the location of the corresponding features. Find the point that is closest to both rays (i.e., midpoint P of line segment being perpendicular to both ray)s.
79 Parametric line representation The parametric representation of the line passing through P 1 and P 2 is given by: t Є [0,1] The direction of the line is given by the vector t(p 2 - P 1 )
80 Computing P Parametric representation of the line l passing through O l and p l : Parametric representation of the line r passing through O r and p r : since R L T R L L T R P R( P T) or R P P T or P R P T
81 Computing P (cont d) Suppose the P 1 and P 2 are given by l: αp l r: T+bR T p r The parametric equation of the line passing through P 1 and P 2 is given by: The desired point P (midpoint) is computed for c = 1/2
82 Computing a 0 and b 0 Consider the vector w orthogonal to both l and r is given by: The line s going through P 1 with direction w is given by: s: P 1 +cw or
83 Computing a 0 and b 0 (cont d) The lines s and r intersect at P 2 : P 1 +c 0 w = P 2 or (assume s passes through P 2 for c = c 0 ) We can obtain a 0 and b 0 by solving the following system of equations:
84 (2) Reconstruction up to a scale factor Only the intrinsic camera parameters are known. We cannot recover the true scale of the viewed scene since we do not know the baseline T (recall that Z = ft/d). Reconstruction is unique only up to an unknown scaling factor. This factor can be determined if we know the distance between two points in the scene.
85 (2) Reconstruction up to a scale factor (cont d) Estimate E Estimate E using the 8-point algorithm. The solution is unique up to an unknown scale factor. Recover T (from E)
86 (2) Reconstruction up to a scale factor (cont d) To simplify the recovery of T, consider where Ê Recover R (from E) It can be shown that
87 (2) Reconstruction up to a scale factor (cont d) Ambiguity in ( Tˆ, R) - The equations for recovering Tˆ are quadratic - The sign of Ê is not fixed (from SVD) The ambiguity will be resolved during 3D reconstruction - Only one combination of ( Tˆ, R) gives consistent reconstructions (positive Z l,z r )
88 (2) Reconstruction up to a scale factor (cont d) 3D reconstruction P R( P Tˆ ) r l
89 (2) Reconstruction up to a scale factor (cont d) 3D reconstruction and solving for Z l
90 (2) Reconstruction up to a scale factor (cont d) Algorithm Note: the algorithm should not go through more than 4 iterations
91 (3) Reconstruction up to a projective transformation More complicated - see book chapter
Multiple View Geometry
Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric
More informationReminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches
Reminder: Lecture 20: The Eight-Point Algorithm F = -0.00310695-0.0025646 2.96584-0.028094-0.00771621 56.3813 13.1905-29.2007-9999.79 Readings T&V 7.3 and 7.4 Essential/Fundamental Matrix E/F Matrix Summary
More informationMachine vision. Summary # 11: Stereo vision and epipolar geometry. u l = λx. v l = λy
1 Machine vision Summary # 11: Stereo vision and epipolar geometry STEREO VISION The goal of stereo vision is to use two cameras to capture 3D scenes. There are two important problems in stereo vision:
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationCorrespondence and Stereopsis. Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri]
Correspondence and Stereopsis Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri] Introduction Disparity: Informally: difference between two pictures Allows us to gain a strong
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationThe reconstruction problem. Reconstruction by triangulation
The reconstruction problem Both intrinsic and extr insic parameters are known: we can solve the reconstruction problem unambiguously by triangulation. Only the intrinsic parameters are known: we can solve
More informationLecture 14: Basic Multi-View Geometry
Lecture 14: Basic Multi-View Geometry Stereo If I needed to find out how far point is away from me, I could use triangulation and two views scene point image plane optical center (Graphic from Khurram
More informationThe end of affine cameras
The end of affine cameras Affine SFM revisited Epipolar geometry Two-view structure from motion Multi-view structure from motion Planches : http://www.di.ens.fr/~ponce/geomvis/lect3.pptx http://www.di.ens.fr/~ponce/geomvis/lect3.pdf
More informationTwo-view geometry Computer Vision Spring 2018, Lecture 10
Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of
More informationUnit 3 Multiple View Geometry
Unit 3 Multiple View Geometry Relations between images of a scene Recovering the cameras Recovering the scene structure http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook1.html 3D structure from images Recover
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera
More informationBIL Computer Vision Apr 16, 2014
BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm
More information3D Sensing and Reconstruction Readings: Ch 12: , Ch 13: ,
3D Sensing and Reconstruction Readings: Ch 12: 12.5-6, Ch 13: 13.1-3, 13.9.4 Perspective Geometry Camera Model Stereo Triangulation 3D Reconstruction by Space Carving 3D Shape from X means getting 3D coordinates
More informationLecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15
Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15 Lecture 6 Stereo Systems Multi- view geometry Stereo systems
More informationComputer Vision Lecture 17
Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester
More informationComputer Vision Lecture 17
Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week
More informationToday. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography
Computational Photography Matthias Zwicker University of Bern Fall 2009 Today From 2D to 3D using multiple views Introduction Geometry of two views Stereo matching Other applications Multiview geometry
More information1 (5 max) 2 (10 max) 3 (20 max) 4 (30 max) 5 (10 max) 6 (15 extra max) total (75 max + 15 extra)
Mierm Exam CS223b Stanford CS223b Computer Vision, Winter 2004 Feb. 18, 2004 Full Name: Email: This exam has 7 pages. Make sure your exam is not missing any sheets, and write your name on every page. The
More informationLecture 9: Epipolar Geometry
Lecture 9: Epipolar Geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Why is stereo useful? Epipolar constraints Essential and fundamental matrix Estimating F (Problem Set 2
More informationCamera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration
Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1
More informationStereo and Epipolar geometry
Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka
More informationMAPI Computer Vision. Multiple View Geometry
MAPI Computer Vision Multiple View Geometry Geometry o Multiple Views 2- and 3- view geometry p p Kpˆ [ K R t]p Geometry o Multiple Views 2- and 3- view geometry Epipolar Geometry The epipolar geometry
More informationEpipolar Geometry and the Essential Matrix
Epipolar Geometry and the Essential Matrix Carlo Tomasi The epipolar geometry of a pair of cameras expresses the fundamental relationship between any two corresponding points in the two image planes, and
More informationEpipolar Geometry Prof. D. Stricker. With slides from A. Zisserman, S. Lazebnik, Seitz
Epipolar Geometry Prof. D. Stricker With slides from A. Zisserman, S. Lazebnik, Seitz 1 Outline 1. Short introduction: points and lines 2. Two views geometry: Epipolar geometry Relation point/line in two
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x
More information1 Projective Geometry
CIS8, Machine Perception Review Problem - SPRING 26 Instructions. All coordinate systems are right handed. Projective Geometry Figure : Facade rectification. I took an image of a rectangular object, and
More informationDepth from two cameras: stereopsis
Depth from two cameras: stereopsis Epipolar Geometry Canonical Configuration Correspondence Matching School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem
More informationLecture 6 Stereo Systems Multi-view geometry
Lecture 6 Stereo Systems Multi-view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-5-Feb-4 Lecture 6 Stereo Systems Multi-view geometry Stereo systems
More informationFundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision
Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision What Happened Last Time? Human 3D perception (3D cinema) Computational stereo Intuitive explanation of what is meant by disparity Stereo matching
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t R 2 3,t 3 Camera 1 Camera
More informationStereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman
Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure
More informationImage Rectification (Stereo) (New book: 7.2.1, old book: 11.1)
Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1) Guido Gerig CS 6320 Spring 2013 Credits: Prof. Mubarak Shah, Course notes modified from: http://www.cs.ucf.edu/courses/cap6411/cap5415/, Lecture
More informationStructure from Motion and Multi- view Geometry. Last lecture
Structure from Motion and Multi- view Geometry Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 5 Last lecture S. J. Gortler, R. Grzeszczuk, R. Szeliski,M. F. Cohen The Lumigraph, SIGGRAPH,
More informationMultiple View Geometry in Computer Vision
Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 Structure Computation Lecture 18 March 22, 2005 2 3D Reconstruction The goal of 3D reconstruction
More informationDepth from two cameras: stereopsis
Depth from two cameras: stereopsis Epipolar Geometry Canonical Configuration Correspondence Matching School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Shiv Ram Dubey, IIIT Sri City Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X
More informationComputer Vision Projective Geometry and Calibration. Pinhole cameras
Computer Vision Projective Geometry and Calibration Professor Hager http://www.cs.jhu.edu/~hager Jason Corso http://www.cs.jhu.edu/~jcorso. Pinhole cameras Abstract camera model - box with a small hole
More informationStereo CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Why do we perceive depth? What do humans use as depth cues? Motion Convergence When watching an object close to us, our eyes
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationAnnouncements. Stereo
Announcements Stereo Homework 2 is due today, 11:59 PM Homework 3 will be assigned today Reading: Chapter 7: Stereopsis CSE 152 Lecture 8 Binocular Stereopsis: Mars Given two images of a scene where relative
More informationRecap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?
Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple
More informationCS231A Course Notes 4: Stereo Systems and Structure from Motion
CS231A Course Notes 4: Stereo Systems and Structure from Motion Kenji Hata and Silvio Savarese 1 Introduction In the previous notes, we covered how adding additional viewpoints of a scene can greatly enhance
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction
More informationMultiple Views Geometry
Multiple Views Geometry Subhashis Banerjee Dept. Computer Science and Engineering IIT Delhi email: suban@cse.iitd.ac.in January 2, 28 Epipolar geometry Fundamental geometric relationship between two perspective
More informationRectification and Distortion Correction
Rectification and Distortion Correction Hagen Spies March 12, 2003 Computer Vision Laboratory Department of Electrical Engineering Linköping University, Sweden Contents Distortion Correction Rectification
More informationLecture 9 & 10: Stereo Vision
Lecture 9 & 10: Stereo Vision Professor Fei- Fei Li Stanford Vision Lab 1 What we will learn today? IntroducEon to stereo vision Epipolar geometry: a gentle intro Parallel images Image receficaeon Solving
More informationStep-by-Step Model Buidling
Step-by-Step Model Buidling Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure
More informationWeek 2: Two-View Geometry. Padua Summer 08 Frank Dellaert
Week 2: Two-View Geometry Padua Summer 08 Frank Dellaert Mosaicking Outline 2D Transformation Hierarchy RANSAC Triangulation of 3D Points Cameras Triangulation via SVD Automatic Correspondence Essential
More informationLecture'9'&'10:'' Stereo'Vision'
Lecture'9'&'10:'' Stereo'Vision' Dr.'Juan'Carlos'Niebles' Stanford'AI'Lab' ' Professor'FeiAFei'Li' Stanford'Vision'Lab' 1' Dimensionality'ReducIon'Machine'(3D'to'2D)' 3D world 2D image Point of observation
More informationStereo imaging ideal geometry
Stereo imaging ideal geometry (X,Y,Z) Z f (x L,y L ) f (x R,y R ) Optical axes are parallel Optical axes separated by baseline, b. Line connecting lens centers is perpendicular to the optical axis, and
More informationLecture 5 Epipolar Geometry
Lecture 5 Epipolar Geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 5-24-Jan-18 Lecture 5 Epipolar Geometry Why is stereo useful? Epipolar constraints Essential
More informationStereo II CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo II CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Camera parameters A camera is described by several parameters Translation T of the optical center from the origin of world
More informationCOMPARATIVE STUDY OF DIFFERENT APPROACHES FOR EFFICIENT RECTIFICATION UNDER GENERAL MOTION
COMPARATIVE STUDY OF DIFFERENT APPROACHES FOR EFFICIENT RECTIFICATION UNDER GENERAL MOTION Mr.V.SRINIVASA RAO 1 Prof.A.SATYA KALYAN 2 DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING PRASAD V POTLURI SIDDHARTHA
More informationStructure from Motion
Structure from Motion Outline Bundle Adjustment Ambguities in Reconstruction Affine Factorization Extensions Structure from motion Recover both 3D scene geoemetry and camera positions SLAM: Simultaneous
More informationC / 35. C18 Computer Vision. David Murray. dwm/courses/4cv.
C18 2015 1 / 35 C18 Computer Vision David Murray david.murray@eng.ox.ac.uk www.robots.ox.ac.uk/ dwm/courses/4cv Michaelmas 2015 C18 2015 2 / 35 Computer Vision: This time... 1. Introduction; imaging geometry;
More informationStereo matching. Francesco Isgrò. 3D Reconstruction and Stereo p.1/21
Stereo matching Francesco Isgrò 3D Reconstruction and Stereo p.1/21 Structure of a stereo vision system Extract interesting point from each image Determine a set of matching points Compute the fundamental
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253
Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine
More information3D Reconstruction from Two Views
3D Reconstruction from Two Views Huy Bui UIUC huybui1@illinois.edu Yiyi Huang UIUC huang85@illinois.edu Abstract In this project, we study a method to reconstruct a 3D scene from two views. First, we extract
More informationRectification and Disparity
Rectification and Disparity Nassir Navab Slides prepared by Christian Unger What is Stereo Vision? Introduction A technique aimed at inferring dense depth measurements efficiently using two cameras. Wide
More informationA Factorization Method for Structure from Planar Motion
A Factorization Method for Structure from Planar Motion Jian Li and Rama Chellappa Center for Automation Research (CfAR) and Department of Electrical and Computer Engineering University of Maryland, College
More informationRectification. Dr. Gerhard Roth
Rectification Dr. Gerhard Roth Problem Definition Given a pair of stereo images, the intrinsic parameters of each camera, and the extrinsic parameters of the system, R, and, compute the image transformation
More informationAnnouncements. Stereo
Announcements Stereo Homework 1 is due today, 11:59 PM Homework 2 will be assigned on Thursday Reading: Chapter 7: Stereopsis CSE 252A Lecture 8 Binocular Stereopsis: Mars Given two images of a scene where
More informationECE 470: Homework 5. Due Tuesday, October 27 in Seth Hutchinson. Luke A. Wendt
ECE 47: Homework 5 Due Tuesday, October 7 in class @:3pm Seth Hutchinson Luke A Wendt ECE 47 : Homework 5 Consider a camera with focal length λ = Suppose the optical axis of the camera is aligned with
More informationLecture 10: Multi view geometry
Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationComputer Vision I - Algorithms and Applications: Multi-View 3D reconstruction
Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction Carsten Rother 09/12/2013 Computer Vision I: Multi-View 3D reconstruction Roadmap this lecture Computer Vision I: Multi-View
More informationStereo. Outline. Multiple views 3/29/2017. Thurs Mar 30 Kristen Grauman UT Austin. Multi-view geometry, matching, invariant features, stereo vision
Stereo Thurs Mar 30 Kristen Grauman UT Austin Outline Last time: Human stereopsis Epipolar geometry and the epipolar constraint Case example with parallel optical axes General case with calibrated cameras
More informationEpipolar geometry. x x
Two-view geometry Epipolar geometry X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections
More informationRecovering structure from a single view Pinhole perspective projection
EPIPOLAR GEOMETRY The slides are from several sources through James Hays (Brown); Silvio Savarese (U. of Michigan); Svetlana Lazebnik (U. Illinois); Bill Freeman and Antonio Torralba (MIT), including their
More informationThere are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...
STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own
More informationStructure from Motion. Introduction to Computer Vision CSE 152 Lecture 10
Structure from Motion CSE 152 Lecture 10 Announcements Homework 3 is due May 9, 11:59 PM Reading: Chapter 8: Structure from Motion Optional: Multiple View Geometry in Computer Vision, 2nd edition, Hartley
More informationComputer Vision I. Announcement. Stereo Vision Outline. Stereo II. CSE252A Lecture 15
Announcement Stereo II CSE252A Lecture 15 HW3 assigned No class on Thursday 12/6 Extra class on Tuesday 12/4 at 6:30PM in WLH Room 2112 Mars Exploratory Rovers: Spirit and Opportunity Stereo Vision Outline
More informationEpipolar Geometry and Stereo Vision
CS 1674: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 5, 2016 Announcement Please send me three topics you want me to review next
More informationarxiv: v1 [cs.cv] 28 Sep 2018
Camera Pose Estimation from Sequence of Calibrated Images arxiv:1809.11066v1 [cs.cv] 28 Sep 2018 Jacek Komorowski 1 and Przemyslaw Rokita 2 1 Maria Curie-Sklodowska University, Institute of Computer Science,
More information3D FACE RECONSTRUCTION BASED ON EPIPOLAR GEOMETRY
IJDW Volume 4 Number January-June 202 pp. 45-50 3D FACE RECONSRUCION BASED ON EPIPOLAR GEOMERY aher Khadhraoui, Faouzi Benzarti 2 and Hamid Amiri 3,2,3 Signal, Image Processing and Patterns Recognition
More informationFinal Exam Study Guide CSE/EE 486 Fall 2007
Final Exam Study Guide CSE/EE 486 Fall 2007 Lecture 2 Intensity Sufaces and Gradients Image visualized as surface. Terrain concepts. Gradient of functions in 1D and 2D Numerical derivatives. Taylor series.
More informationEpipolar Geometry CSE P576. Dr. Matthew Brown
Epipolar Geometry CSE P576 Dr. Matthew Brown Epipolar Geometry Epipolar Lines, Plane Constraint Fundamental Matrix, Linear solution + RANSAC Applications: Structure from Motion, Stereo [ Szeliski 11] 2
More informationComputer Vision I - Appearance-based Matching and Projective Geometry
Computer Vision I - Appearance-based Matching and Projective Geometry Carsten Rother 05/11/2015 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation
More informationLecture 10: Multi-view geometry
Lecture 10: Multi-view geometry Professor Stanford Vision Lab 1 What we will learn today? Review for stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationFlexible Calibration of a Portable Structured Light System through Surface Plane
Vol. 34, No. 11 ACTA AUTOMATICA SINICA November, 2008 Flexible Calibration of a Portable Structured Light System through Surface Plane GAO Wei 1 WANG Liang 1 HU Zhan-Yi 1 Abstract For a portable structured
More informationCS231A Midterm Review. Friday 5/6/2016
CS231A Midterm Review Friday 5/6/2016 Outline General Logistics Camera Models Non-perspective cameras Calibration Single View Metrology Epipolar Geometry Structure from Motion Active Stereo and Volumetric
More informationBut First: Multi-View Projective Geometry
View Morphing (Seitz & Dyer, SIGGRAPH 96) Virtual Camera Photograph Morphed View View interpolation (ala McMillan) but no depth no camera information Photograph But First: Multi-View Projective Geometry
More informationCS 664 Slides #9 Multi-Camera Geometry. Prof. Dan Huttenlocher Fall 2003
CS 664 Slides #9 Multi-Camera Geometry Prof. Dan Huttenlocher Fall 2003 Pinhole Camera Geometric model of camera projection Image plane I, which rays intersect Camera center C, through which all rays pass
More information3D Reconstruction with two Calibrated Cameras
3D Reconstruction with two Calibrated Cameras Carlo Tomasi The standard reference frame for a camera C is a right-handed Cartesian frame with its origin at the center of projection of C, its positive Z
More informationStructure from Motion
/8/ Structure from Motion Computer Vision CS 43, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from motion
More information3D Geometry and Camera Calibration
3D Geometry and Camera Calibration 3D Coordinate Systems Right-handed vs. left-handed x x y z z y 2D Coordinate Systems 3D Geometry Basics y axis up vs. y axis down Origin at center vs. corner Will often
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263
Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine
More informationComputer Vision I - Appearance-based Matching and Projective Geometry
Computer Vision I - Appearance-based Matching and Projective Geometry Carsten Rother 01/11/2016 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation
More informationDepth. Common Classification Tasks. Example: AlexNet. Another Example: Inception. Another Example: Inception. Depth
Common Classification Tasks Recognition of individual objects/faces Analyze object-specific features (e.g., key points) Train with images from different viewing angles Recognition of object classes Analyze
More informationStereo Epipolar Geometry for General Cameras. Sanja Fidler CSC420: Intro to Image Understanding 1 / 33
Stereo Epipolar Geometry for General Cameras Sanja Fidler CSC420: Intro to Image Understanding 1 / 33 Stereo Epipolar geometry Case with two cameras with parallel optical axes General case Now this Sanja
More informationCS201 Computer Vision Camera Geometry
CS201 Computer Vision Camera Geometry John Magee 25 November, 2014 Slides Courtesy of: Diane H. Theriault (deht@bu.edu) Question of the Day: How can we represent the relationships between cameras and the
More informationFundamental Matrix & Structure from Motion
Fundamental Matrix & Structure from Motion Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Transformations between images Structure from Motion The Essential Matrix The Fundamental
More informationStereo Image Rectification for Simple Panoramic Image Generation
Stereo Image Rectification for Simple Panoramic Image Generation Yun-Suk Kang and Yo-Sung Ho Gwangju Institute of Science and Technology (GIST) 261 Cheomdan-gwagiro, Buk-gu, Gwangju 500-712 Korea Email:{yunsuk,
More informationModel-Based Stereo. Chapter Motivation. The modeling system described in Chapter 5 allows the user to create a basic model of a
96 Chapter 7 Model-Based Stereo 7.1 Motivation The modeling system described in Chapter 5 allows the user to create a basic model of a scene, but in general the scene will have additional geometric detail
More informationCompositing a bird's eye view mosaic
Compositing a bird's eye view mosaic Robert Laganiere School of Information Technology and Engineering University of Ottawa Ottawa, Ont KN 6N Abstract This paper describes a method that allows the composition
More informationMulti-view stereo. Many slides adapted from S. Seitz
Multi-view stereo Many slides adapted from S. Seitz Beyond two-view stereo The third eye can be used for verification Multiple-baseline stereo Pick a reference image, and slide the corresponding window
More informationFinal Exam Study Guide
Final Exam Study Guide Exam Window: 28th April, 12:00am EST to 30th April, 11:59pm EST Description As indicated in class the goal of the exam is to encourage you to review the material from the course.
More informationStereo Vision Computer Vision (Kris Kitani) Carnegie Mellon University
Stereo Vision 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University What s different between these two images? Objects that are close move more or less? The amount of horizontal movement is
More informationHomogeneous Coordinates. Lecture18: Camera Models. Representation of Line and Point in 2D. Cross Product. Overall scaling is NOT important.
Homogeneous Coordinates Overall scaling is NOT important. CSED44:Introduction to Computer Vision (207F) Lecture8: Camera Models Bohyung Han CSE, POSTECH bhhan@postech.ac.kr (",, ) ()", ), )) ) 0 It is
More information