Machine vision. Summary # 11: Stereo vision and epipolar geometry. u l = λx. v l = λy
|
|
- Emil Bates
- 6 years ago
- Views:
Transcription
1 1 Machine vision Summary # 11: Stereo vision and epipolar geometry STEREO VISION The goal of stereo vision is to use two cameras to capture 3D scenes. There are two important problems in stereo vision: Correspondence problem: finding matching pairs (conjugate pairs) of the two images that represent the same point in the 3D scene. Reconstruction problem: obtain the 3D structure from the images. For a single pinhole camera we wrote: u = x v = y A simple camera geometry for stereo vision is shown in figure 1, from which we have: (1) (2) where u r = u l = x v r = y v l = y (x b) is the focal length. The distance from the image plane to the center of projection. b is the baseline, distance between the centers of the two cameras. We assume that the optical axes are aligned. By subtraction, we get u l u r = b and therefore = b u l u r (8) It is common to attach the origin to the left camera as shown in figure 1. We assume that both cameras are calibrated and that they are identical. We also assume that the relative orientation of the two cameras is the same. It is also possible to attach the origin to the middle point between the two cameras reference frames. The equations will be slightly different. Equation (8) gives the distance to the 3D point from the camera. Note that The difference u l u r is called the horiontal disparity, retinal disparity, or binocular disparity. In order to get a feel for the disparity, put one finger in front of you, close one eye, then open it and close the other eye. Distance is inversely proportional to disparity Disparity is proportional to the base line Accuracy of depth determination increases with increasing baseline. Images become less similar when the baseline increases. For a given baseline, the accuracy is better for closer objects than for farther objects. Example Using equation (8), we can determine the x and y coordinates of point P as follows: u l x = b (9) u l u r v r y = b (10) u l u r (3) (4) (5) (6) (7)
2 Fig. 1. Stereo vision geometry, C l is the reference point. Example Consider images 2 and 3 obtained from a stereo vision system (in this problem we use subscripts 1 and 2 for right and left images, respectively). The image sie is 3456 by 4608 pixels. The pixel coordinates of the dot are r 1 = 749, r 2 = 4271, c 1 = 420. The origin of the pixel coordinate system is the bottom left of the image. (u 0, v 0 ) is located in the middle of the image. 1) Deduce a formula to find the distance to point P. 2) Calculate when the intrinsic parameters are The distance between the two cameras is 30cm. 3) Find the (x, y) coordinates of point P. The solution is shown as a code below. % Camera p a r a m e t e r s u0= 2304 v0= 1728 a l p h a v =3450 =3700 s x (11) =3450 s y (12) u 0 =2304 (13) v 0 =1728 (14) 2
3 Point P Fig. 2. Left image Point P Fig. 3. Right image a l p h a u =3700 %R i g h t r1 =749 c1 =420 %L e f t r2 =4271 c2 =420 b=300 p o i n t =( a l p h a u b ) / ( r2 r1 ) y p o i n t = p o i n t ( c2 v0 ) / a l p h a v y p o i n t= y p o i n t %T r a n s f o r m i n g t h e o r i g i n x p o i n t = p o i n t ( r2 u0 ) / a l p h a u The resiul;ts are = mm (15) x = mm (16) y = mm (17) 3
4 Fig. 4. Relative geometry between two cameras RELATIVE GEOMETRY BETWEEN TWO CAMERAS The assumption of perfectly aligned cameras is violated in practice. Also, two identical cameras do not exist. In general, the first step in stereo vision is to determine the relationship between the two cameras. By relationship we mean the relative orientation and position (cameras are not aligned any more). Consider the geometric representation of figure 4. Let S l = (x l, y l, ) T be the position of point P in the left camera coordinate system and S r = (x r, y r, r ) T be the position of point P in the right camera coordinate system. It is possible to relate the coordinates by the following equation where R is a rotation matrix, it satisfies R T R = I. System (18) can be written as S r = RS l + T (18) r 11 x l + r 12 y l + r 13 + T x = x r r 21 x l + r 22 y l + r 23 + T y = y r r 31 x l + r 32 y l + r 33 + T = r (19) We do not know R or T but we know the the left and right image projections u r, v r, u l, v l. Knowing the focal length, it is possible to write u l = x l (20) v l = y l (21) u r = x r r (22) v r = y r r (23) Now and r are regarded as additional unknowns. After substituting x l, y l, x r, y r by their formulae in terms of the focal length and the depth distance, we get u l r 11 + r v l 12 + r 13 + T x = u r r (24) u l r 21 + r v l 22 + r 23 + T y = v r r (25) u l r 31 + r v l 32 + r 33 + T = r (26) 4
5 and r 11 u l + r 12 v l + r 13 + T x = u r r (27) r 21 u l + r 22 v l + r 23 + T y = v r r (28) r 31 u l + r 32 v l + r 33 + T = r (29) There are three equations and fourteen unknowns (r ij, T x, T y, T., r ). Each additional point provides three more equations, but at the same time introduces two unknown variables:, r. For one point, we can write the system as For N points, we obtain Therefore, we need at least 12 points to solve. 3 equations 1 point = 12 unkowns + 2 unkowns 1 point (30) 3 equations N points = 12 unkowns + 2 unkowns N points (31) COMPUTING THE DEPTH If we know the translation and the rotation matrix as well as the image coordinate u l, u r, v l, v r, we can calculate the depths and r : u l r 11 + r v l 12 + r r 13 + T x = u r (32) u l r 21 + r v l 22 + r r 23 + T y = v r (33) u l r 31 + r v l 32 + r 33 + T = r (34) Since we have two unknowns and three equations we can use any two equations to solve for and r. In the particular case when the cameras have the same orientation, we have: ul ur + T x = r (35) vl vr + T y = r (36) + T = r (37) EPIPOLAR GEOMETRY AND FUNDAMENTAL MATRIX Consider the stereo vision geometry of figures 5 and 6. We want to solve the correspondence problem. Point P in the 3D space is imaged in the left camera at q l and in the right camera at q r. Rays C l q l and C r q r intersect at point P and they both lie in the same plane. As a result the image points q l, q r, the space point P, and the camera centers are coplanar, i.e., they belong to the same plane. The plane defined by these three points (C l, C r, P ) is called the epipolar plane and is denoted by Π. The correspondence problem can be formulated as follows: knowing q l, is what are the coordinates and the constraints on the location of q r. Point q r lies in the right image plane and at the same time, it lies in the plane Π. The intersection of the epipolar plane Π and the image plane form a line l r. Now the search for q r is reduced to line l r. Line l r is called the epipolar line. Points e l, e r are called the epipoles. Figure 6 shows the stereo vision geometry. Points P 1, P 2, P 3 have the same projection in the left image plane, but different projections in the right image. The epipolar constraint reduces the problem to 1D search. Example The images in figure 7 and 8 are taken using a stereo vision system with b = 300mm. The coordinates of the point of interest in the pixel coordinate system are (916, 686) in the right image and (97, 701) in the left image. The blue line is the right epipolar line and the red line is the left epipolar line. Assume we have canonical cameras THE ESSENTIAL MATRIX A l = A r = I 5
6 Machine vision, spring 2017 Summary 11 Fig. 5. Stereo vision geometry Fig. 6. Stereo vision geometry X: 916 Y: Fig. 7. Examples of the epipolar lines
7 Machine vision, spring 2017 Summary X: 97 Y: Fig. 8. Examples of the epipolar lines where A` and Ar are the intrinsic matrixes of the left and right follows M` = I Mr = R camera, respectively. We define the projection matrixes as 0 T (38) (39) The essential matrix is defines as E = TX R where TX is the translation vector represented under matrix form 0 T 0 TX = T Ty Tx Ty Tx 0 (40) T HE FUNDAMENTAL MATRIX The fundamental matrix is a algebraic representation of the epipolar geometry. It represents a mapping between the right and left image. In general A` 6= I and Ar 6= I. The fundamental matrix is given by 1 T F = A 1 r TX RA` (41) The most important property of the fundamental matrix is summaried in the following theorem. Theorem: The fundamental matrix satisfies the following condition: for any pair of corresponding image points qrt F q` = 0 (42) qr lies on the epilopar line `r = F q` (43) `` = F T qr (44) q` lies on the epilopar line Equations (43) and (44) show that the fundamental matrix represents a mapping between a point and a line. The correspondence problem is formulated in terms of matrix F. Solving the correspondence problem means solving for matrix F, which is a unique 3 3 matrix of rank 2. 7
8 COMPUTING THE FUNDAMENTAL MATRIX: THE EIGHT-POINT ALGORITHM Equation (41) gives the fundamental matrix in terms of the intrinsic and extrinsic parameters. As mentioned previously, each pair of points gives a scalar constraint as follows q T r i F q l i = 0 (45) The eight-point algorithm proposes to use at least eight points to calculate matrix F. Equation (42) can be written as ur v r 1 f 11 f 12 f 13 u l f 21 f 22 f 23 v l = 0 (46) f 31 f 32 f 33 1 This is a scalar equation that can be reduced to: ul u r u l v r u l v l u r v l v r v l u r v r 1 At least eight points are needed to solve. If we take N points, we obtain N constraint that can be put under matrix form as follow W f = 0 where W is given by and f 11 f 12 f 13 f 21 f 22 f 23 f 31 f 32 f 33 = 0 u l1 u r1 u l1 v r1 u l1 v l1 u r1 v l1 v r1 v l1 u r1 v r1 1 u l2 u r2 u l2 v r2 u l2 v l2 u r2 v l2 v r2 v l2 u r2 v r2 1 u l3 u r3 u l3 v r3 u l3 v l3 u r3 v l3 v r3 v l3 u r3 v r3 1 u l4 u r4 u l4 v r4 u l4 v l4 u r4 v l4 v r4 v l4 u r4 v r4 1 u l5 u r5 u l5 v r5 u l5 v l5 u r5 v l5 v r5 v l5 u r5 v r5 1 u l6 u r6 u l6 v r6 u l6 v l6 u r6 v l6 v r6 v l6 u r6 v r6 1 u l7 u r7 u l7 v r7 u l7 v l7 u r7 v l7 v r7 v l7 u r7 v r u ln u rn u ln v rn u ln v ln u rn v ln v rn v ln u rn v rn 1 f = f 11 f 12 f 13 f 21 f 22 f 23 f 31 f 32 f 33 One possible way to solve is bu using the singular value decomposition method. The solution consists of two steps in general Step 1: Linear solution: Use the singular value decomposition to obtain a first estimation of matrix F by soling W f = 0. This estimation may not satisfy the rank requirement for the fundamental matrix. The following commands can be used: U, S, V = svd (A ) ; f = V ( :, end ) ; F = r e s h a p e ( f, 3 3 ) ; Step 2: Constrain enforcement: Find the closest approximation to F that has rank 2. Again SVD is used as follows U, S, V = svd ( F ) ; S ( 3, 3 ) = 0 ; F = U S V ; (47) (48) 8
9 Example We want to find the epipolar lines for the pair of images given in figures 7 8. The desired points are p 1 =(916, 686) (49) p 2 =(97, 701) (50) For 8 points, matrix W is given by: W = The fundamental matrix is given by The epipolar lines are given by f = l r = l l = (51) (52) (53) (54) 9
Stereo Vision. MAN-522 Computer Vision
Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in
More informationTwo-view geometry Computer Vision Spring 2018, Lecture 10
Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of
More informationMultiple View Geometry
Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric
More informationUnit 3 Multiple View Geometry
Unit 3 Multiple View Geometry Relations between images of a scene Recovering the cameras Recovering the scene structure http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook1.html 3D structure from images Recover
More informationStereo CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Why do we perceive depth? What do humans use as depth cues? Motion Convergence When watching an object close to us, our eyes
More informationEpipolar geometry. x x
Two-view geometry Epipolar geometry X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationMAPI Computer Vision. Multiple View Geometry
MAPI Computer Vision Multiple View Geometry Geometry o Multiple Views 2- and 3- view geometry p p Kpˆ [ K R t]p Geometry o Multiple Views 2- and 3- view geometry Epipolar Geometry The epipolar geometry
More informationCS201 Computer Vision Camera Geometry
CS201 Computer Vision Camera Geometry John Magee 25 November, 2014 Slides Courtesy of: Diane H. Theriault (deht@bu.edu) Question of the Day: How can we represent the relationships between cameras and the
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationDepth from two cameras: stereopsis
Depth from two cameras: stereopsis Epipolar Geometry Canonical Configuration Correspondence Matching School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture
More informationAnnouncements. Stereo
Announcements Stereo Homework 2 is due today, 11:59 PM Homework 3 will be assigned today Reading: Chapter 7: Stereopsis CSE 152 Lecture 8 Binocular Stereopsis: Mars Given two images of a scene where relative
More informationDepth from two cameras: stereopsis
Depth from two cameras: stereopsis Epipolar Geometry Canonical Configuration Correspondence Matching School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture
More informationLecture 14: Basic Multi-View Geometry
Lecture 14: Basic Multi-View Geometry Stereo If I needed to find out how far point is away from me, I could use triangulation and two views scene point image plane optical center (Graphic from Khurram
More informationEpipolar Geometry and the Essential Matrix
Epipolar Geometry and the Essential Matrix Carlo Tomasi The epipolar geometry of a pair of cameras expresses the fundamental relationship between any two corresponding points in the two image planes, and
More informationAnnouncements. Stereo
Announcements Stereo Homework 1 is due today, 11:59 PM Homework 2 will be assigned on Thursday Reading: Chapter 7: Stereopsis CSE 252A Lecture 8 Binocular Stereopsis: Mars Given two images of a scene where
More informationThe end of affine cameras
The end of affine cameras Affine SFM revisited Epipolar geometry Two-view structure from motion Multi-view structure from motion Planches : http://www.di.ens.fr/~ponce/geomvis/lect3.pptx http://www.di.ens.fr/~ponce/geomvis/lect3.pdf
More informationEpipolar Geometry and Stereo Vision
CS 1674: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 5, 2016 Announcement Please send me three topics you want me to review next
More informationEpipolar Geometry Prof. D. Stricker. With slides from A. Zisserman, S. Lazebnik, Seitz
Epipolar Geometry Prof. D. Stricker With slides from A. Zisserman, S. Lazebnik, Seitz 1 Outline 1. Short introduction: points and lines 2. Two views geometry: Epipolar geometry Relation point/line in two
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationLecture 9: Epipolar Geometry
Lecture 9: Epipolar Geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Why is stereo useful? Epipolar constraints Essential and fundamental matrix Estimating F (Problem Set 2
More informationGeometry of Multiple views
1 Geometry of Multiple views CS 554 Computer Vision Pinar Duygulu Bilkent University 2 Multiple views Despite the wealth of information contained in a a photograph, the depth of a scene point along the
More informationCS 664 Slides #9 Multi-Camera Geometry. Prof. Dan Huttenlocher Fall 2003
CS 664 Slides #9 Multi-Camera Geometry Prof. Dan Huttenlocher Fall 2003 Pinhole Camera Geometric model of camera projection Image plane I, which rays intersect Camera center C, through which all rays pass
More informationEpipolar Geometry CSE P576. Dr. Matthew Brown
Epipolar Geometry CSE P576 Dr. Matthew Brown Epipolar Geometry Epipolar Lines, Plane Constraint Fundamental Matrix, Linear solution + RANSAC Applications: Structure from Motion, Stereo [ Szeliski 11] 2
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x
More informationMultiple View Geometry in Computer Vision
Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 Structure Computation Lecture 18 March 22, 2005 2 3D Reconstruction The goal of 3D reconstruction
More informationMulti-view geometry problems
Multi-view geometry Multi-view geometry problems Structure: Given projections o the same 3D point in two or more images, compute the 3D coordinates o that point? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera
More informationComputer Vision Lecture 17
Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera
More informationStereo II CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo II CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Camera parameters A camera is described by several parameters Translation T of the optical center from the origin of world
More informationComputer Vision Lecture 17
Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester
More informationRecovering structure from a single view Pinhole perspective projection
EPIPOLAR GEOMETRY The slides are from several sources through James Hays (Brown); Silvio Savarese (U. of Michigan); Svetlana Lazebnik (U. Illinois); Bill Freeman and Antonio Torralba (MIT), including their
More informationReminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches
Reminder: Lecture 20: The Eight-Point Algorithm F = -0.00310695-0.0025646 2.96584-0.028094-0.00771621 56.3813 13.1905-29.2007-9999.79 Readings T&V 7.3 and 7.4 Essential/Fundamental Matrix E/F Matrix Summary
More informationStructure from Motion and Multi- view Geometry. Last lecture
Structure from Motion and Multi- view Geometry Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 5 Last lecture S. J. Gortler, R. Grzeszczuk, R. Szeliski,M. F. Cohen The Lumigraph, SIGGRAPH,
More information1 Projective Geometry
CIS8, Machine Perception Review Problem - SPRING 26 Instructions. All coordinate systems are right handed. Projective Geometry Figure : Facade rectification. I took an image of a rectangular object, and
More information3D Sensing and Reconstruction Readings: Ch 12: , Ch 13: ,
3D Sensing and Reconstruction Readings: Ch 12: 12.5-6, Ch 13: 13.1-3, 13.9.4 Perspective Geometry Camera Model Stereo Triangulation 3D Reconstruction by Space Carving 3D Shape from X means getting 3D coordinates
More informationStructure from motion
Multi-view geometry Structure rom motion Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera 3 R 3,t 3 Figure credit: Noah Snavely Structure rom motion? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera 3 R 3,t 3 Structure:
More information1 (5 max) 2 (10 max) 3 (20 max) 4 (30 max) 5 (10 max) 6 (15 extra max) total (75 max + 15 extra)
Mierm Exam CS223b Stanford CS223b Computer Vision, Winter 2004 Feb. 18, 2004 Full Name: Email: This exam has 7 pages. Make sure your exam is not missing any sheets, and write your name on every page. The
More informationCamera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration
Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1
More informationCS 231A Computer Vision (Winter 2015) Problem Set 2
CS 231A Computer Vision (Winter 2015) Problem Set 2 Due Feb 9 th 2015 11:59pm 1 Fundamental Matrix (20 points) In this question, you will explore some properties of fundamental matrix and derive a minimal
More informationMultiple Views Geometry
Multiple Views Geometry Subhashis Banerjee Dept. Computer Science and Engineering IIT Delhi email: suban@cse.iitd.ac.in January 2, 28 Epipolar geometry Fundamental geometric relationship between two perspective
More informationBIL Computer Vision Apr 16, 2014
BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Shiv Ram Dubey, IIIT Sri City Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X
More informationLecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15
Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15 Lecture 6 Stereo Systems Multi- view geometry Stereo systems
More informationImage Rectification (Stereo) (New book: 7.2.1, old book: 11.1)
Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1) Guido Gerig CS 6320 Spring 2013 Credits: Prof. Mubarak Shah, Course notes modified from: http://www.cs.ucf.edu/courses/cap6411/cap5415/, Lecture
More informationLUMS Mine Detector Project
LUMS Mine Detector Project Using visual information to control a robot (Hutchinson et al. 1996). Vision may or may not be used in the feedback loop. Visual (image based) features such as points, lines
More informationCorrespondence and Stereopsis. Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri]
Correspondence and Stereopsis Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri] Introduction Disparity: Informally: difference between two pictures Allows us to gain a strong
More informationComputer Vision Projective Geometry and Calibration. Pinhole cameras
Computer Vision Projective Geometry and Calibration Professor Hager http://www.cs.jhu.edu/~hager Jason Corso http://www.cs.jhu.edu/~jcorso. Pinhole cameras Abstract camera model - box with a small hole
More informationTake Home Exam # 2 Machine Vision
1 Take Home Exam # 2 Machine Vision Date: 04/26/2018 Due : 05/03/2018 Work with one awesome/breathtaking/amazing partner. The name of the partner should be clearly stated at the beginning of your report.
More informationStereo Vision Computer Vision (Kris Kitani) Carnegie Mellon University
Stereo Vision 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University What s different between these two images? Objects that are close move more or less? The amount of horizontal movement is
More informationCamera Geometry II. COS 429 Princeton University
Camera Geometry II COS 429 Princeton University Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence
More informationComputer Vision I. Announcement. Stereo Vision Outline. Stereo II. CSE252A Lecture 15
Announcement Stereo II CSE252A Lecture 15 HW3 assigned No class on Thursday 12/6 Extra class on Tuesday 12/4 at 6:30PM in WLH Room 2112 Mars Exploratory Rovers: Spirit and Opportunity Stereo Vision Outline
More informationMulti-View Geometry Part II (Ch7 New book. Ch 10/11 old book)
Multi-View Geometry Part II (Ch7 New book. Ch 10/11 old book) Guido Gerig CS-GY 6643, Spring 2016 gerig@nyu.edu Credits: M. Shah, UCF CAP5415, lecture 23 http://www.cs.ucf.edu/courses/cap6411/cap5415/,
More informationToday. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography
Computational Photography Matthias Zwicker University of Bern Fall 2009 Today From 2D to 3D using multiple views Introduction Geometry of two views Stereo matching Other applications Multiview geometry
More informationRectification and Disparity
Rectification and Disparity Nassir Navab Slides prepared by Christian Unger What is Stereo Vision? Introduction A technique aimed at inferring dense depth measurements efficiently using two cameras. Wide
More information3D Geometry and Camera Calibration
3D Geometry and Camera Calibration 3D Coordinate Systems Right-handed vs. left-handed x x y z z y 2D Coordinate Systems 3D Geometry Basics y axis up vs. y axis down Origin at center vs. corner Will often
More informationMERGING POINT CLOUDS FROM MULTIPLE KINECTS. Nishant Rai 13th July, 2016 CARIS Lab University of British Columbia
MERGING POINT CLOUDS FROM MULTIPLE KINECTS Nishant Rai 13th July, 2016 CARIS Lab University of British Columbia Introduction What do we want to do? : Use information (point clouds) from multiple (2+) Kinects
More informationRecap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?
Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t R 2 3,t 3 Camera 1 Camera
More informationCS231A Course Notes 4: Stereo Systems and Structure from Motion
CS231A Course Notes 4: Stereo Systems and Structure from Motion Kenji Hata and Silvio Savarese 1 Introduction In the previous notes, we covered how adding additional viewpoints of a scene can greatly enhance
More informationUndergrad HTAs / TAs. Help me make the course better! HTA deadline today (! sorry) TA deadline March 21 st, opens March 15th
Undergrad HTAs / TAs Help me make the course better! HTA deadline today (! sorry) TA deadline March 2 st, opens March 5th Project 2 Well done. Open ended parts, lots of opportunity for mistakes. Real implementation
More information3D Reconstruction with two Calibrated Cameras
3D Reconstruction with two Calibrated Cameras Carlo Tomasi The standard reference frame for a camera C is a right-handed Cartesian frame with its origin at the center of projection of C, its positive Z
More informationComputer Vision I - Algorithms and Applications: Multi-View 3D reconstruction
Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction Carsten Rother 09/12/2013 Computer Vision I: Multi-View 3D reconstruction Roadmap this lecture Computer Vision I: Multi-View
More informationLecture 6 Stereo Systems Multi-view geometry
Lecture 6 Stereo Systems Multi-view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-5-Feb-4 Lecture 6 Stereo Systems Multi-view geometry Stereo systems
More informationComputer Vision cmput 428/615
Computer Vision cmput 428/615 Basic 2D and 3D geometry and Camera models Martin Jagersand The equation of projection Intuitively: How do we develop a consistent mathematical framework for projection calculations?
More informationComments on Consistent Depth Maps Recovery from a Video Sequence
Comments on Consistent Depth Maps Recovery from a Video Sequence N.P. van der Aa D.S. Grootendorst B.F. Böggemann R.T. Tan Technical Report UU-CS-2011-014 May 2011 Department of Information and Computing
More informationCSE 252B: Computer Vision II
CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribe: Sameer Agarwal LECTURE 1 Image Formation 1.1. The geometry of image formation We begin by considering the process of image formation when a
More informationThere are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...
STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253
Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine
More informationCHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION
CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION In this chapter we will discuss the process of disparity computation. It plays an important role in our caricature system because all 3D coordinates of nodes
More informationCameras and Stereo CSE 455. Linda Shapiro
Cameras and Stereo CSE 455 Linda Shapiro 1 Müller-Lyer Illusion http://www.michaelbach.de/ot/sze_muelue/index.html What do you know about perspective projection? Vertical lines? Other lines? 2 Image formation
More informationStereo imaging ideal geometry
Stereo imaging ideal geometry (X,Y,Z) Z f (x L,y L ) f (x R,y R ) Optical axes are parallel Optical axes separated by baseline, b. Line connecting lens centers is perpendicular to the optical axis, and
More informationThe real voyage of discovery consists not in seeking new landscapes, but in having new eyes.
The real voyage of discovery consists not in seeking new landscapes, but in having new eyes. - Marcel Proust University of Texas at Arlington Camera Calibration (or Resectioning) CSE 4392-5369 Vision-based
More information3D Computer Vision. Structure from Motion. Prof. Didier Stricker
3D Computer Vision Structure from Motion Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Structure
More informationStereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman
Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263
Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine
More informationLecture 5 Epipolar Geometry
Lecture 5 Epipolar Geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 5-24-Jan-18 Lecture 5 Epipolar Geometry Why is stereo useful? Epipolar constraints Essential
More informationVision 3D articielle Multiple view geometry
Vision 3D articielle Multiple view geometry Pascal Monasse monasse@imagine.enpc.fr IMAGINE, École des Ponts ParisTech Contents Multi-view constraints Multi-view calibration Incremental calibration Global
More informationComputer Vision Project-1
University of Utah, School Of Computing Computer Vision Project- Singla, Sumedha sumedha.singla@utah.edu (00877456 February, 205 Theoretical Problems. Pinhole Camera (a A straight line in the world space
More informationEpipolar Geometry and Stereo Vision
CS 1699: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 8, 2015 Today Review Projective transforms Image stitching (homography) Epipolar
More informationCS223b Midterm Exam, Computer Vision. Monday February 25th, Winter 2008, Prof. Jana Kosecka
CS223b Midterm Exam, Computer Vision Monday February 25th, Winter 2008, Prof. Jana Kosecka Your name email This exam is 8 pages long including cover page. Make sure your exam is not missing any pages.
More informationStereo Epipolar Geometry for General Cameras. Sanja Fidler CSC420: Intro to Image Understanding 1 / 33
Stereo Epipolar Geometry for General Cameras Sanja Fidler CSC420: Intro to Image Understanding 1 / 33 Stereo Epipolar geometry Case with two cameras with parallel optical axes General case Now this Sanja
More informationGeometric camera models and calibration
Geometric camera models and calibration http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 13 Course announcements Homework 3 is out. - Due October
More informationC / 35. C18 Computer Vision. David Murray. dwm/courses/4cv.
C18 2015 1 / 35 C18 Computer Vision David Murray david.murray@eng.ox.ac.uk www.robots.ox.ac.uk/ dwm/courses/4cv Michaelmas 2015 C18 2015 2 / 35 Computer Vision: This time... 1. Introduction; imaging geometry;
More informationThree-Dimensional Sensors Lecture 2: Projected-Light Depth Cameras
Three-Dimensional Sensors Lecture 2: Projected-Light Depth Cameras Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inria.fr http://perception.inrialpes.fr/ Outline The geometry of active stereo.
More informationStereo matching. Francesco Isgrò. 3D Reconstruction and Stereo p.1/21
Stereo matching Francesco Isgrò 3D Reconstruction and Stereo p.1/21 Structure of a stereo vision system Extract interesting point from each image Determine a set of matching points Compute the fundamental
More informationFlexible Calibration of a Portable Structured Light System through Surface Plane
Vol. 34, No. 11 ACTA AUTOMATICA SINICA November, 2008 Flexible Calibration of a Portable Structured Light System through Surface Plane GAO Wei 1 WANG Liang 1 HU Zhan-Yi 1 Abstract For a portable structured
More informationRobust Geometry Estimation from two Images
Robust Geometry Estimation from two Images Carsten Rother 09/12/2016 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation Process 09/12/2016 2 Appearance-based
More informationA Factorization Method for Structure from Planar Motion
A Factorization Method for Structure from Planar Motion Jian Li and Rama Chellappa Center for Automation Research (CfAR) and Department of Electrical and Computer Engineering University of Maryland, College
More informationFundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision
Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision What Happened Last Time? Human 3D perception (3D cinema) Computational stereo Intuitive explanation of what is meant by disparity Stereo matching
More informationCS231M Mobile Computer Vision Structure from motion
CS231M Mobile Computer Vision Structure from motion - Cameras - Epipolar geometry - Structure from motion Pinhole camera Pinhole perspective projection f o f = focal length o = center of the camera z y
More informationPerception II: Pinhole camera and Stereo Vision
Perception II: Pinhole camera and Stereo Vision Davide Scaramuzza Margarita Chli, Paul Furgale, Marco Hutter, Roland Siegwart 1 Mobile Robot Control Scheme knowledge, data base mission commands Localization
More informationElements of Computer Vision: Multiple View Geometry. 1 Introduction. 2 Elements of Geometry. Andrea Fusiello
Elements of Computer Vision: Multiple View Geometry. Andrea Fusiello http://www.sci.univr.it/~fusiello July 11, 2005 c Copyright by Andrea Fusiello. This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike
More informationTwo-View Geometry (Course 23, Lecture D)
Two-View Geometry (Course 23, Lecture D) Jana Kosecka Department of Computer Science George Mason University http://www.cs.gmu.edu/~kosecka General Formulation Given two views of the scene recover the
More informationStructure from Motion
/8/ Structure from Motion Computer Vision CS 43, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from motion
More informationCamera Models and Image Formation. Srikumar Ramalingam School of Computing University of Utah
Camera Models and Image Formation Srikumar Ramalingam School of Computing University of Utah srikumar@cs.utah.edu Reference Most slides are adapted from the following notes: Some lecture notes on geometric
More informationLecture 10: Multi view geometry
Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationCamera Model and Calibration
Camera Model and Calibration Lecture-10 Camera Calibration Determine extrinsic and intrinsic parameters of camera Extrinsic 3D location and orientation of camera Intrinsic Focal length The size of the
More informationStructure from Motion. Prof. Marco Marcon
Structure from Motion Prof. Marco Marcon Summing-up 2 Stereo is the most powerful clue for determining the structure of a scene Another important clue is the relative motion between the scene and (mono)
More informationFundamental Matrix & Structure from Motion
Fundamental Matrix & Structure from Motion Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Transformations between images Structure from Motion The Essential Matrix The Fundamental
More information