Lecture 5 Epipolar Geometry
|
|
- Marjory Lindsey
- 5 years ago
- Views:
Transcription
1 Lecture 5 Epipolar Geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 5-24-Jan-18
2 Lecture 5 Epipolar Geometry Why is stereo useful? Epipolar constraints Essential and fundamental matrix Estimating F Examples Reading: [AZ] Chapter: 4 Estimation 2D perspective transformations Chapter: 9 Epipolar Geometry and the Fundamental Matrix Transformation Chapter: 11 Computation of the Fundamental Matrix F [FP] Chapter: 7 Stereopsis Chapter: 8 Structure from Motion Silvio Savarese Lecture 5-24-Jan-18
3 Recovering structure from a single view Pinhole perspective projection P p O w Calibration rig Scene C Camera K From calibration rig location/pose of the rig, K From points and lines at infinity + orthogonal lines and planes structure of the scene, K Knowledge about scene (point correspondences, geometry of lines & planes, etc
4 Recovering structure from a single view Pinhole perspective projection P p O w Calibration rig Scene C Camera K Why is it so difficult? Intrinsic ambiguity of the mapping from 3D to image (2D)
5 Recovering structure from a single view Intrinsic ambiguity of the mapping from 3D to image (2D) Courtesy slide S. Lazebnik
6 Two eyes help!
7 Two eyes help! P = l l" [Eq. 1] P l l K =known p p K =known R, T O 1 O 2 This is called triangulation
8 Triangulation Find P* that minimizes d(p, M P*)+ d(p, M P*) [Eq. 2] P* P MP* p M P* p O 1 O 2
9 Multi (stereo)-view geometry Camera geometry: Given corresponding points in two images, find camera matrices, position and pose. Scene geometry: Find coordinates of 3D point from its projection into 2 or multiple images. Correspondence: Given a point p in one image, how can I find the corresponding point p in another one?
10 Epipolar geometry P p p e e Epipolar Plane Epipoles e, e Baseline Epipolar Lines O 1 O 2 = intersections of baseline with image planes = projections of the other camera center
11 Example of epipolar lines
12 Example: Parallel image planes P e v p p v e u u O 1 O 2 Baseline intersects the image plane at infinity Epipoles are at infinity Epipolar lines are parallel to u axis
13 Example: Parallel Image Planes
14 Example: Forward translation e P O 2 e O 1 The epipoles have same position in both images Epipole called FOE (focus of expansion)
15 Epipolar Constraint p Where is p? - Two views of the same object - Given a point on left image, how can I find the corresponding point on right image?
16 Epipolar geometry P Epipolar line 2 p p O 1 O 2
17 Epipolar Constraint p p Epipolar line 2
18 v Epipolar Constraint P p p u O 1 R, T O 2 M = K! " I 0! # M P = # # " u v 1 $ & & = p & % # $ " M = K R T R T T #$ " u! % $ [Eq. 3] M! P = $ v! = p [Eq. 4] $ # 1 & % &
19 Epipolar Constraint P! # K canonical = # # " $ & & & % p p O 1 R, T O 2 M = K! " I 0 # $ K = K are known (canonical cameras) " M = K R T R T T #$ % & M = [ I 0] [Eq. 5] " M = R T R T T #$ % & [Eq. 6]
20 The cameras are related by R, T Camera 2 p T O 2 Camera 1 O 1 T = O 2 in the camera 1 reference system R is the rotation matrix such that a vector p in the camera 2 is equal to R p in camera 1.
21 Epipolar Constraint P p p O 1 R, T O 2 p in first camera reference system is T ((R p ")+T) = T (R p ") = R p! +T is perpendicular to epipolar plane [ ( )] = 0 T R p p T [Eq. 7]
22 b a b a ] [ = ú ú ú û ù ê ê ê ë é ú ú ú û ù ê ê ê ë é = z y x x y x z y z b b b a a a a a a Cross product as matrix multiplication
23 Epipolar Constraint P p p O 1 R, T O 2 [ T (R p )] 0 p T = [ T ] R p 0 p T = [Eq. 8] [Eq. 9] E = Essential matrix (Longuet-Higgins, 1981)
24 Epipolar Constraint p T E p = 0 P [Eq. 10] p l l e e p O 1 O 2 l = E p is the epipolar line associated with p l = E T p is the epipolar line associated with p E e = 0 and E T e = 0 E is 3x3 matrix; 5 DOF E is singular (rank two)
25 Epipolar Constraint P p p O 1 O 2 M = K[ I 0] R, T " M = K R T R T T #$ p c = K 1 p p c! = K 1 p! [Eq. 11] [Eq. 12] % &
26 p c = K 1 p [Eq. 11] Epipolar Constraint P p c! = K 1 p! [Eq. 12] p p [Eq.9] O 1 O 2 p T c [ T ] R # -1 T p = 0 (K p) [ T ] R K p = 0 c -1 p T K [ T ] R K -1 p = 0 p T = - T F p 0 [Eq. 13]
27 Epipolar Constraint P p p O 1 O 2 [Eq. 13] p T F p = 0 F F = Fundamental Matrix (Faugeras and Luong, 1992) [ ] -1 = K - T T R K [Eq. 14]
28 Epipolar Constraint p T F p = 0 P p l l p e e O 1 O 2 l = F p is the epipolar line associated with p l = F T p is the epipolar line associated with p F e = 0 and F T e = 0 F is 3x3 matrix; 7 DOF F is singular (rank two)
29 Why F is useful? p p l = F T p - Suppose F is known - No additional information about the scene and camera is given - Given a point on left image, we can compute the corresponding epipolar line in the second imag
30 Why F is useful? F captures information about the epipolar geometry of 2 views + camera parameters MORE IMPORTANTLY: F gives constraints on how the scene changes under view point transformation (without reconstructing the scene!) Powerful tool in: 3D reconstruction Multi-view object/scene matching
31 The Eight-Point Algorithm Estimating F (Longuet-Higgins, 1981) (Hartley, 1995) P p p O 1 O 2 p T F p = 0
32 [Eq. 13] ( ) p T F p = Estimating F 0 æf11 F12 F13 öæu ö ç ç uv,,1 F F F v = ç ç F31 F32 F ç 33 1 è øè ø ( uu uv u vu vv v u v ) Let s take 8 corresponding points! # p = # # " æf11 ö ç ç F12 ç F 13 ç F ç 21,,,,,,,,1 ç F = 0 u v 1 22 ç $ & & & % ç F23 ç ç F31 ç F 32 ç F è 33 ø " $ p! = $ $ # u! v! 1 % & [Eq. 14]
33 Estimating F
34 Estimating F! # # # # # # ( u i u i,u i v i,u i,v i u i,v i v i,v i,u i,v i,1) # # # # # # # " F 11 F 12 F 13 F 21 F 22 F 23 F 31 F 32 F 33 $ & & & & & & & = 0 [Eq. 14] & & & & & & %
35 W Homogeneous system Rank 8 If N>8 Estimating F æf11 ö æuu uv u vu vv v u v 1öç F ç uu 2 2 uv 2 2 u2 vu 2 2 vv 2 2 v2 u 2 v 2 1 ç 12 ç ç F 13 ç uu 3 3 uv 3 3 u3 vu 3 3 vv 3 3 v3 u 3 v 3 1 ç ç ç F21 uu 4 4 uv 4 4 u4 vu 4 4 vv 4 4 v4 u 4 v 4 1 F ç 22 uu 5 5 uv 5 5 u5 vu 5 5 vv 5 5 v5 u 5 v 5 1 ç ç ç F23 ç uu 6 6 uv 6 6 u6 vu 6 6 vv 6 6 v6 u 6 v 6 1 ç ç F31 uu 7 7 uv 7 7 u7 v7u 7 v7v 7 v7 u 7 v 7 1 ç ç ç F 32 uu 8 8 uv 8 8 u8 vu 8 8 vv 8 8 v8 u 8 v 8 1 ç ç ç = 0 è øç F è 33 ø A non-zero solution exists (unique) Lsq. solution by SVD! f =1 W f = 0 f [Eqs. 15] Fˆ
36 Fˆ satisfies: ^ p T Fˆ p = and estimated F may have full rank (det(f) 0) But remember: fundamental matrix is Rank2 0 ^ Find F that minimizes F - Fˆ = 0 Frobenius norm (*) Subject to det(f)=0 SVD (again!) can be used to solve this problem (*) Sq. root of the sum of squares of all entries
37 Find F that minimizes F - Fˆ = 0 Frobenius norm (*) Subject to det(f)=0! # F =U# # "# s s $ & & & %& V T [HZ] pag 281, chapter 11, Computation of F Where:! # U# # "# s s s 3 $ & & V T = SVD( ˆF) & %&
38 Data courtesy of R. Mohr and B. Boufama.
39 Mean errors: 10.0pixel 9.1pixel
40 Problems with the 8-Point Algorithm W f = 0, Lsq solution by SVD f =1 F - Recall the structure of W: - do we see any potential (numerical) issue?
41 Problems with the 8-Point Algorithm Wf = 0 æf11 ö æuu uv u vu vv v u v 1öç F ç uu 2 2 uv 2 2 u2 vu 2 2 vv 2 2 v2 u 2 v 2 1 ç 12 ç ç F 13 ç uu 3 3 uv 3 3 u3 vu 3 3 vv 3 3 v3 u 3 v 3 1 ç ç ç F21 uu 4 4 uv 4 4 u4 vu 4 4 vv 4 4 v4 u 4 v 4 1 F ç 22 uu 5 5 uv 5 5 u5 vu 5 5 vv 5 5 v5 u 5 v 5 1 ç ç ç F23 ç uu 6 6 uv 6 6 u6 vu 6 6 vv 6 6 v6 u 6 v 6 1 ç ç F31 uu 7 7 uv 7 7 u7 v7u 7 v7v 7 v7 u 7 v 7 1 ç ç ç F 32 uu 8 8 uv 8 8 u8 vu 8 8 vv 8 8 v8 u 8 v 8 1 ç ç ç = è øç F è 33 ø Highly un-balanced (not well conditioned) Values of W must have similar magnitude This creates problems during the SVD decomposition 0 HZ pag 108
42 Normalization IDEA: Transform image coordinates such that the matrix W becomes better conditioned (pre-conditioning) For each image, apply a transformation T (translation and scaling) acting on image coordinates such that: Origin = centroid of image points Mean square distance of the image points from origin is ~ 2 pixels
43 Example of normalization T Coordinate system of the image before applying T Coordinate system of the image after applying T Origin = centroid of image points Mean square distance of the image points from origin is ~ 2 pixels
44 Normalization q i = T p i q = T p i i
45 The Normalized Eight-Point Algorithm 0. Compute T and T for image 1 and 2, respectively 1. Normalize coordinates in images 1 and 2: q = T i p i 2. Use the eight-point algorithm to compute from the corresponding points q and q. i q = T p i i i ˆF q 1. Enforce the rank-2 constraint: 2. De-normalize F q : F F q such that: q T F q q = 0 T = T Fq T det( Fq ) = 0
46 With normalization Without normalization Mean errors: 10.0pixel 9.1pixel Mean errors: 1.0pixel 0.9pixel
47 The Fundamental Matrix Song
48 Next lecture: Stereo systems Silvio Savarese Lecture 5-24-Jan-18
49
50 Example: Parallel image planes P e v p p e y u z x O 1 O 2 Hint : K E=? 1 =K 2 = known R = I T = (T, 0, 0) x parallel to O 1 O 2
51 Essential matrix for parallel images E = [ T ] R " $ E = $ $ # $ 0 T z T y T z 0 T x T y T x 0 T = [ T 0 0 ] R = I % " $ R = $ $ & # T 0 T 0 % & [Eq. 20]
52 Example: Parallel image planes P v v e l p p l e y u u z O 1 What are the directions of epipolar lines? x " $ l = E p = $ $ # T 0 T 0 %" $ $ $ &# u v 1 % " $ = $ $ & # 0 T T v % & horizontal!
53 Example: Parallel image planes P v v e p p e y u u z O 1 x How are p and p related? p T E p = 0
54 Example: Parallel image planes P v v e p p e y u u z O 1 x How are p and p related? é0 0 0 ùæu ö æ 0 ö ( u v ê ) T úç ç = Þ v ( u v ) T Tv Tv ê - ú = Þ - = Þ = ê 0 T 0 úç 1 ç Tv ë ûè ø è ø v = v" T pep
55 Example: Parallel image planes P v v e p p e y u u z O 1 x Rectification: making two images parallel Why it is useful? Epipolar constraint v = v New views can be synthesized by linear interpolation
56 Application: view morphing S. M. Seitz and C. R. Dyer, Proc. SIGGRAPH 96, 1996, 21-30
57 Rectification P H 1 I 1 I s î 1 v u I 2 î S v u C 1 v u î 2 H 2 C s C 2
58
59
60
61 From its reflection!
62
63 Deep view morphing D. Ji, J. Kwon, M. McFarland, S. Savarese, CVPR 2017
64 Deep view morphing D. Ji, J. Kwon, M. McFarland, S. Savarese, CVPR 2017
65 Deep view morphing D. Ji, J. Kwon, M. McFarland, S. Savarese, CVPR 2017
Lecture 9: Epipolar Geometry
Lecture 9: Epipolar Geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Why is stereo useful? Epipolar constraints Essential and fundamental matrix Estimating F (Problem Set 2
More informationRecovering structure from a single view Pinhole perspective projection
EPIPOLAR GEOMETRY The slides are from several sources through James Hays (Brown); Silvio Savarese (U. of Michigan); Svetlana Lazebnik (U. Illinois); Bill Freeman and Antonio Torralba (MIT), including their
More informationLecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15
Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15 Lecture 6 Stereo Systems Multi- view geometry Stereo systems
More informationLecture 6 Stereo Systems Multi-view geometry
Lecture 6 Stereo Systems Multi-view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-5-Feb-4 Lecture 6 Stereo Systems Multi-view geometry Stereo systems
More informationCS231M Mobile Computer Vision Structure from motion
CS231M Mobile Computer Vision Structure from motion - Cameras - Epipolar geometry - Structure from motion Pinhole camera Pinhole perspective projection f o f = focal length o = center of the camera z y
More informationLecture 10: Multi-view geometry
Lecture 10: Multi-view geometry Professor Stanford Vision Lab 1 What we will learn today? Review for stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationTwo-view geometry Computer Vision Spring 2018, Lecture 10
Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of
More informationLecture'9'&'10:'' Stereo'Vision'
Lecture'9'&'10:'' Stereo'Vision' Dr.'Juan'Carlos'Niebles' Stanford'AI'Lab' ' Professor'FeiAFei'Li' Stanford'Vision'Lab' 1' Dimensionality'ReducIon'Machine'(3D'to'2D)' 3D world 2D image Point of observation
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x
More informationEpipolar geometry. x x
Two-view geometry Epipolar geometry X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Shiv Ram Dubey, IIIT Sri City Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X
More informationMulti-view geometry problems
Multi-view geometry Multi-view geometry problems Structure: Given projections o the same 3D point in two or more images, compute the 3D coordinates o that point? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera
More informationLecture 9 & 10: Stereo Vision
Lecture 9 & 10: Stereo Vision Professor Fei- Fei Li Stanford Vision Lab 1 What we will learn today? IntroducEon to stereo vision Epipolar geometry: a gentle intro Parallel images Image receficaeon Solving
More informationStructure from motion
Multi-view geometry Structure rom motion Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera 3 R 3,t 3 Figure credit: Noah Snavely Structure rom motion? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera 3 R 3,t 3 Structure:
More informationReminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches
Reminder: Lecture 20: The Eight-Point Algorithm F = -0.00310695-0.0025646 2.96584-0.028094-0.00771621 56.3813 13.1905-29.2007-9999.79 Readings T&V 7.3 and 7.4 Essential/Fundamental Matrix E/F Matrix Summary
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera
More informationUndergrad HTAs / TAs. Help me make the course better! HTA deadline today (! sorry) TA deadline March 21 st, opens March 15th
Undergrad HTAs / TAs Help me make the course better! HTA deadline today (! sorry) TA deadline March 2 st, opens March 5th Project 2 Well done. Open ended parts, lots of opportunity for mistakes. Real implementation
More informationWeek 2: Two-View Geometry. Padua Summer 08 Frank Dellaert
Week 2: Two-View Geometry Padua Summer 08 Frank Dellaert Mosaicking Outline 2D Transformation Hierarchy RANSAC Triangulation of 3D Points Cameras Triangulation via SVD Automatic Correspondence Essential
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationCS231A Course Notes 4: Stereo Systems and Structure from Motion
CS231A Course Notes 4: Stereo Systems and Structure from Motion Kenji Hata and Silvio Savarese 1 Introduction In the previous notes, we covered how adding additional viewpoints of a scene can greatly enhance
More informationCS201 Computer Vision Camera Geometry
CS201 Computer Vision Camera Geometry John Magee 25 November, 2014 Slides Courtesy of: Diane H. Theriault (deht@bu.edu) Question of the Day: How can we represent the relationships between cameras and the
More informationLecture 14: Basic Multi-View Geometry
Lecture 14: Basic Multi-View Geometry Stereo If I needed to find out how far point is away from me, I could use triangulation and two views scene point image plane optical center (Graphic from Khurram
More informationAnnouncements. Stereo
Announcements Stereo Homework 2 is due today, 11:59 PM Homework 3 will be assigned today Reading: Chapter 7: Stereopsis CSE 152 Lecture 8 Binocular Stereopsis: Mars Given two images of a scene where relative
More informationAnnouncements. Stereo
Announcements Stereo Homework 1 is due today, 11:59 PM Homework 2 will be assigned on Thursday Reading: Chapter 7: Stereopsis CSE 252A Lecture 8 Binocular Stereopsis: Mars Given two images of a scene where
More informationCamera Geometry II. COS 429 Princeton University
Camera Geometry II COS 429 Princeton University Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence
More informationMulti-View Geometry Part II (Ch7 New book. Ch 10/11 old book)
Multi-View Geometry Part II (Ch7 New book. Ch 10/11 old book) Guido Gerig CS-GY 6643, Spring 2016 gerig@nyu.edu Credits: M. Shah, UCF CAP5415, lecture 23 http://www.cs.ucf.edu/courses/cap6411/cap5415/,
More informationStructure from Motion and Multi- view Geometry. Last lecture
Structure from Motion and Multi- view Geometry Topics in Image-Based Modeling and Rendering CSE291 J00 Lecture 5 Last lecture S. J. Gortler, R. Grzeszczuk, R. Szeliski,M. F. Cohen The Lumigraph, SIGGRAPH,
More informationBIL Computer Vision Apr 16, 2014
BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm
More informationStructure from Motion
/8/ Structure from Motion Computer Vision CS 43, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from motion
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera
More informationStereo Vision. MAN-522 Computer Vision
Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in
More informationLecture 10: Multi view geometry
Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationStructure from Motion
11/18/11 Structure from Motion Computer Vision CS 143, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from
More informationUnit 3 Multiple View Geometry
Unit 3 Multiple View Geometry Relations between images of a scene Recovering the cameras Recovering the scene structure http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook1.html 3D structure from images Recover
More informationCS 231A Computer Vision (Winter 2015) Problem Set 2
CS 231A Computer Vision (Winter 2015) Problem Set 2 Due Feb 9 th 2015 11:59pm 1 Fundamental Matrix (20 points) In this question, you will explore some properties of fundamental matrix and derive a minimal
More informationEpipolar Geometry Prof. D. Stricker. With slides from A. Zisserman, S. Lazebnik, Seitz
Epipolar Geometry Prof. D. Stricker With slides from A. Zisserman, S. Lazebnik, Seitz 1 Outline 1. Short introduction: points and lines 2. Two views geometry: Epipolar geometry Relation point/line in two
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t R 2 3,t 3 Camera 1 Camera
More informationThe end of affine cameras
The end of affine cameras Affine SFM revisited Epipolar geometry Two-view structure from motion Multi-view structure from motion Planches : http://www.di.ens.fr/~ponce/geomvis/lect3.pptx http://www.di.ens.fr/~ponce/geomvis/lect3.pdf
More informationStereo CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Why do we perceive depth? What do humans use as depth cues? Motion Convergence When watching an object close to us, our eyes
More information1 Projective Geometry
CIS8, Machine Perception Review Problem - SPRING 26 Instructions. All coordinate systems are right handed. Projective Geometry Figure : Facade rectification. I took an image of a rectangular object, and
More informationComputer Vision I. Announcement. Stereo Vision Outline. Stereo II. CSE252A Lecture 15
Announcement Stereo II CSE252A Lecture 15 HW3 assigned No class on Thursday 12/6 Extra class on Tuesday 12/4 at 6:30PM in WLH Room 2112 Mars Exploratory Rovers: Spirit and Opportunity Stereo Vision Outline
More informationCS 231A: Computer Vision (Winter 2018) Problem Set 2
CS 231A: Computer Vision (Winter 2018) Problem Set 2 Due Date: Feb 09 2018, 11:59pm Note: In this PS, using python2 is recommended, as the data files are dumped with python2. Using python3 might cause
More informationMultiple View Geometry. Frank Dellaert
Multiple View Geometry Frank Dellaert Outline Intro Camera Review Stereo triangulation Geometry of 2 views Essential Matrix Fundamental Matrix Estimating E/F from point-matches Why Consider Multiple Views?
More informationComputer Vision Lecture 17
Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester
More informationComputer Vision Lecture 17
Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationEECS 442 Computer vision. Stereo systems. Stereo vision Rectification Correspondence problem Active stereo vision systems
EECS 442 Computer vision Stereo systems Stereo vision Rectification Correspondence problem Active stereo vision systems Reading: [HZ] Chapter: 11 [FP] Chapter: 11 Stereo vision P p p O 1 O 2 Goal: estimate
More informationStructure from Motion. Introduction to Computer Vision CSE 152 Lecture 10
Structure from Motion CSE 152 Lecture 10 Announcements Homework 3 is due May 9, 11:59 PM Reading: Chapter 8: Structure from Motion Optional: Multiple View Geometry in Computer Vision, 2nd edition, Hartley
More informationStereo II CSE 576. Ali Farhadi. Several slides from Larry Zitnick and Steve Seitz
Stereo II CSE 576 Ali Farhadi Several slides from Larry Zitnick and Steve Seitz Camera parameters A camera is described by several parameters Translation T of the optical center from the origin of world
More informationStereo and Epipolar geometry
Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka
More informationBut First: Multi-View Projective Geometry
View Morphing (Seitz & Dyer, SIGGRAPH 96) Virtual Camera Photograph Morphed View View interpolation (ala McMillan) but no depth no camera information Photograph But First: Multi-View Projective Geometry
More information3D Computer Vision. Structure from Motion. Prof. Didier Stricker
3D Computer Vision Structure from Motion Prof. Didier Stricker Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de 1 Structure
More informationLast lecture. Passive Stereo Spacetime Stereo
Last lecture Passive Stereo Spacetime Stereo Today Structure from Motion: Given pixel correspondences, how to compute 3D structure and camera motion? Slides stolen from Prof Yungyu Chuang Epipolar geometry
More informationTwo-View Geometry (Course 23, Lecture D)
Two-View Geometry (Course 23, Lecture D) Jana Kosecka Department of Computer Science George Mason University http://www.cs.gmu.edu/~kosecka General Formulation Given two views of the scene recover the
More informationIntroduction à la vision artificielle X
Introduction à la vision artificielle X Jean Ponce Email: ponce@di.ens.fr Web: http://www.di.ens.fr/~ponce Planches après les cours sur : http://www.di.ens.fr/~ponce/introvis/lect10.pptx http://www.di.ens.fr/~ponce/introvis/lect10.pdf
More informationEpipolar geometry contd.
Epipolar geometry contd. Estimating F 8-point algorithm The fundamental matrix F is defined by x' T Fx = 0 for any pair of matches x and x in two images. Let x=(u,v,1) T and x =(u,v,1) T, each match gives
More informationThere are many cues in monocular vision which suggests that vision in stereo starts very early from two similar 2D images. Lets see a few...
STEREO VISION The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Silvio Savarese (U. of Michigan); Bill Freeman and Antonio Torralba (MIT), including their own
More informationComputer Vision I - Algorithms and Applications: Multi-View 3D reconstruction
Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction Carsten Rother 09/12/2013 Computer Vision I: Multi-View 3D reconstruction Roadmap this lecture Computer Vision I: Multi-View
More informationCEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt.
CEE598 - Visual Sensing for Civil Infrastructure Eng. & Mgmt. Session 4 Affine Structure from Motion Mani Golparvar-Fard Department of Civil and Environmental Engineering 329D, Newmark Civil Engineering
More informationStructure from Motion CSC 767
Structure from Motion CSC 767 Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R,t R 2,t 2 R 3,t 3 Camera??
More informationRecap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?
Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple
More informationCamera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration
Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1
More informationMachine vision. Summary # 11: Stereo vision and epipolar geometry. u l = λx. v l = λy
1 Machine vision Summary # 11: Stereo vision and epipolar geometry STEREO VISION The goal of stereo vision is to use two cameras to capture 3D scenes. There are two important problems in stereo vision:
More informationRobust Geometry Estimation from two Images
Robust Geometry Estimation from two Images Carsten Rother 09/12/2016 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation Process 09/12/2016 2 Appearance-based
More informationEpipolar Geometry and Stereo Vision
CS 1699: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 8, 2015 Today Review Projective transforms Image stitching (homography) Epipolar
More informationMultiple View Geometry
Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric
More informationMAPI Computer Vision. Multiple View Geometry
MAPI Computer Vision Multiple View Geometry Geometry o Multiple Views 2- and 3- view geometry p p Kpˆ [ K R t]p Geometry o Multiple Views 2- and 3- view geometry Epipolar Geometry The epipolar geometry
More informationRectification and Disparity
Rectification and Disparity Nassir Navab Slides prepared by Christian Unger What is Stereo Vision? Introduction A technique aimed at inferring dense depth measurements efficiently using two cameras. Wide
More informationImage Transfer Methods. Satya Prakash Mallick Jan 28 th, 2003
Image Transfer Methods Satya Prakash Mallick Jan 28 th, 2003 Objective Given two or more images of the same scene, the objective is to synthesize a novel view of the scene from a view point where there
More informationComputer Vision Lecture 20
Computer Perceptual Vision and Sensory WS 16/17 Augmented Computing Computer Perceptual Vision and Sensory WS 16/17 Augmented Computing Computer Perceptual Vision and Sensory WS 16/17 Augmented Computing
More informationComputer Vision Lecture 20
Computer Perceptual Vision and Sensory WS 16/76 Augmented Computing Many slides adapted from K. Grauman, S. Seitz, R. Szeliski, M. Pollefeys, S. Lazebnik Computer Vision Lecture 20 Motion and Optical Flow
More informationC / 35. C18 Computer Vision. David Murray. dwm/courses/4cv.
C18 2015 1 / 35 C18 Computer Vision David Murray david.murray@eng.ox.ac.uk www.robots.ox.ac.uk/ dwm/courses/4cv Michaelmas 2015 C18 2015 2 / 35 Computer Vision: This time... 1. Introduction; imaging geometry;
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263
Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine
More informationZ (cm) Y (cm) X (cm)
Oceans'98 IEEE/OES Conference Uncalibrated Vision for 3-D Underwater Applications K. Plakas, E. Trucco Computer Vision Group and Ocean Systems Laboratory Dept. of Computing and Electrical Engineering Heriot-Watt
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253
Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine
More informationStereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman
Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure
More informationHomogeneous Coordinates. Lecture18: Camera Models. Representation of Line and Point in 2D. Cross Product. Overall scaling is NOT important.
Homogeneous Coordinates Overall scaling is NOT important. CSED44:Introduction to Computer Vision (207F) Lecture8: Camera Models Bohyung Han CSE, POSTECH bhhan@postech.ac.kr (",, ) ()", ), )) ) 0 It is
More informationMultiple Views Geometry
Multiple Views Geometry Subhashis Banerjee Dept. Computer Science and Engineering IIT Delhi email: suban@cse.iitd.ac.in January 2, 28 Epipolar geometry Fundamental geometric relationship between two perspective
More informationEpipolar Geometry and Stereo Vision
CS 1674: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 5, 2016 Announcement Please send me three topics you want me to review next
More informationStructure from Motion
Structure from Motion Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz, N Snavely, and D. Hoiem Administrative stuffs HW 3 due 11:55 PM, Oct 17 (Wed) Submit your alignment results!
More informationComputer Vision Lecture 20
Computer Vision Lecture 2 Motion and Optical Flow Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de 28.1.216 Man slides adapted from K. Grauman, S. Seitz, R. Szeliski,
More information3D Reconstruction with two Calibrated Cameras
3D Reconstruction with two Calibrated Cameras Carlo Tomasi The standard reference frame for a camera C is a right-handed Cartesian frame with its origin at the center of projection of C, its positive Z
More informationCameras and Stereo CSE 455. Linda Shapiro
Cameras and Stereo CSE 455 Linda Shapiro 1 Müller-Lyer Illusion http://www.michaelbach.de/ot/sze_muelue/index.html What do you know about perspective projection? Vertical lines? Other lines? 2 Image formation
More information3D Geometry and Camera Calibration
3D Geometry and Camera Calibration 3D Coordinate Systems Right-handed vs. left-handed x x y z z y 2D Coordinate Systems 3D Geometry Basics y axis up vs. y axis down Origin at center vs. corner Will often
More informationDepth from two cameras: stereopsis
Depth from two cameras: stereopsis Epipolar Geometry Canonical Configuration Correspondence Matching School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture
More information3D reconstruction class 11
3D reconstruction class 11 Multiple View Geometry Comp 290-089 Marc Pollefeys Multiple View Geometry course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective 2D Geometry Jan. 14, 16
More informationGeometry of Multiple views
1 Geometry of Multiple views CS 554 Computer Vision Pinar Duygulu Bilkent University 2 Multiple views Despite the wealth of information contained in a a photograph, the depth of a scene point along the
More informationImage Rectification (Stereo) (New book: 7.2.1, old book: 11.1)
Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1) Guido Gerig CS 6320 Spring 2013 Credits: Prof. Mubarak Shah, Course notes modified from: http://www.cs.ucf.edu/courses/cap6411/cap5415/, Lecture
More informationDepth from two cameras: stereopsis
Depth from two cameras: stereopsis Epipolar Geometry Canonical Configuration Correspondence Matching School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture
More informationMultiple View Geometry in Computer Vision
Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 Structure Computation Lecture 18 March 22, 2005 2 3D Reconstruction The goal of 3D reconstruction
More informationEpipolar Geometry CSE P576. Dr. Matthew Brown
Epipolar Geometry CSE P576 Dr. Matthew Brown Epipolar Geometry Epipolar Lines, Plane Constraint Fundamental Matrix, Linear solution + RANSAC Applications: Structure from Motion, Stereo [ Szeliski 11] 2
More informationToday. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography
Computational Photography Matthias Zwicker University of Bern Fall 2009 Today From 2D to 3D using multiple views Introduction Geometry of two views Stereo matching Other applications Multiview geometry
More informationComputer Vision I. Announcements. Random Dot Stereograms. Stereo III. CSE252A Lecture 16
Announcements Stereo III CSE252A Lecture 16 HW1 being returned HW3 assigned and due date extended until 11/27/12 No office hours today No class on Thursday 12/6 Extra class on Tuesday 12/4 at 6:30PM in
More informationStructure from Motion
Structure from Motion Outline Bundle Adjustment Ambguities in Reconstruction Affine Factorization Extensions Structure from motion Recover both 3D scene geoemetry and camera positions SLAM: Simultaneous
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem
More informationRectification. Dr. Gerhard Roth
Rectification Dr. Gerhard Roth Problem Definition Given a pair of stereo images, the intrinsic parameters of each camera, and the extrinsic parameters of the system, R, and, compute the image transformation
More informationCorrespondence and Stereopsis. Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri]
Correspondence and Stereopsis Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri] Introduction Disparity: Informally: difference between two pictures Allows us to gain a strong
More informationCamera Model and Calibration
Camera Model and Calibration Lecture-10 Camera Calibration Determine extrinsic and intrinsic parameters of camera Extrinsic 3D location and orientation of camera Intrinsic Focal length The size of the
More informationEpipolar Geometry and the Essential Matrix
Epipolar Geometry and the Essential Matrix Carlo Tomasi The epipolar geometry of a pair of cameras expresses the fundamental relationship between any two corresponding points in the two image planes, and
More information