Camera Geometry II. COS 429 Princeton University

Size: px
Start display at page:

Download "Camera Geometry II. COS 429 Princeton University"

Transcription

1 Camera Geometry II COS 429 Princeton University

2 Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence Application: structure from motion revisited

3 Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence Application: structure from motion revisited

4 Review: Camera projection matrix Z Y X t r r r t r r r t r r r y x y y x x x PX intrinsic parameters extrinsic parameters world coordinate system camera coordinate system

5 Review: Camera parameters Intrinsic parameters Image center (p x,p y ) Focal length (f) Pixel magnification (m x,m y ) Skew (non-rectangular pixels) Radial distortion y y x x y x y x p f p f m m K t P K R

6 Review: Camera parameters Intrinsic parameters Principal point coordinates Focal length Pixel magnification Skew (non-rectangular pixels) Radial distortion Extrinsic parameters Rotation (R) and translation (t) relative to world coordinate system P K R t

7 Review: Camera calibration * * * * * * * * * * * * Z Y X y x X t x K R Source: D. Hoiem

8 Review: Camera calibration Given n points with known 3D coordinates X i and known image projections x i, estimate the camera parameters X i x i P

9 Camera calibration What if don t know correspondences? Can you determine the camera intrinsic parameters (focal length, center) or extrinsic parameters (rotation, translation)?

10 Camera calibration Let s see what we can get from vanishing points Vanishing line Vertical vanishing point (at infinity) Vanishing point Vanishing point Slide from Efros, Photo from Criminisi

11 Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence Application: structure from motion revisited

12 Review: Vanishing Points Any set of parallel lines on a plane define a vanishing point The union of all of these vanishing points is the horizon line also called vanishing line Different planes (can) define different vanishing lines v l v 2 Noah Snavely

13 Review: Vanishing points image plane vanishing point v camera center line in the scene All lines having the same direction share the same vanishing point

14 Computing Vanishing Points How can we find lines in an image?

15 Computing Vanishing Points Edge detection Edges Papusha & Ho

16 Computing Vanishing Points Edge detection + Hough transform Strong Lines Hough Transform Papusha & Ho

17 Computing Vanishing Points v q q 2 p 2 For a set of parallel lines, how can we find where they intersect? p

18 Computing Vanishing Points v q q 2 p 2 p Intersect p q with p 2 q 2

19 Computing Vanishing Points v q q 2 p 2 p Intersect p q with p 2 q 2

20 Computing Vanishing Points v q q 2 p 2 p Intersect p q with p 2 q 2

21 Computing Vanishing Points v q q 2 p 2 p Intersect p q with p 2 q 2 Least squares version Better to use more than two lines and compute the closest point of intersection

22 Computing Vanishing Points Alternative: vanishing points can be extracted directly from Hough transform (fit sine curves) Strong Lines Hough Transform Papusha & Ho

23 Computing Vanishing Points Vanishing points can be extracted directly from Hough transform (fit sine curves) Hough Transform Strong Lines Papusha & Ho

24 Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence Application: structure from motion revisited

25 Calibration from vanishing points What camera parameters can we calibrate using three orthogonal vanishing directions (points)? v 2. v 3 v

26 Calibration from vanishing points Let us align the world coordinate system with three orthogonal vanishing directions in the scene: Each pair of vanishing points gives us a constraint on the focal length and principal point 0 0, 0 0, e e e i i i KRe e t K R v 0 0, j T i i T i e e v K e R 0 j T T i j T T T i v K K v v K RR K v

27 Intrinsic calibration from vanishing points Cannot recover focal length, image center is the finite vanishing point Can solve for focal length, image center

28 Rotation from vanishing points Thus, Get λ by using the constraint r i 2 =. i i i KRe e t K R v ] [ r r r r Re v K. v i r i K

29 Calibration from vanishing points: Summary Solve for K (focal length, principal point) using three orthogonal vanishing points Get rotation directly from vanishing points once K is known Advantages No need for calibration chart (2D-3D correspondences) Could be completely automatic Disadvantages Only applies to certain kinds of scenes Inaccuracies in computation of vanishing points Problems due to infinite vanishing points

30 Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence Application: structure from motion revisited

31 How Tall is the Man in this Image? Svetlana Lazebnik

32 How Tall is the Man in this Image? 0 meters Svetlana Lazebnik

33 How Tall is the Man in this Image? 0 meters Svetlana Lazebnik

34 How Tall is the Man in this Image? O Z 0 meters ground plane Goal: compute Z Problem: depends on camera angle and distance Svetlana Lazebnik

35 The cross-ratio A projective invariant: quantity that does not change under projective transformations (including perspective projection) The cross-ratio of four points: P 3 P P 4 P 2 P 3 P 2 P 4 P P 3 P 4 P 2 P Svetlana Lazebnik

36 Measuring height C b r t H T (top of object) R (reference point) R T B R R B T scene cross ratio t b r b v v Z Z r t image cross ratio H R H R v Z B (bottom of object) ground plane Svetlana Lazebnik

37 How Tall is the Man in this Image? 0 meters Svetlana Lazebnik

38 v z r vanishing line (horizon) v x v t 0 H t R H v y b 0 t r b b v v Z Z r t image cross ratio H R b Svetlana Lazebnik

39 2D lines in homogeneous coordinates Line equation: ax + by + c = 0 Line passing through two points: Intersection of two lines:, where 0 y x c b a T x l x l x x 2 l l l 2 x Svetlana Lazebnik

40 v z r vanishing line (horizon) v x v t 0 H t R H v y b 0 t r b b v v Z Z r t image cross ratio H R b Svetlana Lazebnik

41 How Long is this Line Segment? Can we measure distances between two points on same plane? Svetlana Lazebnik

42 How Long is this Line Segment? What if we know distances between some pairs of points on the same plane? Svetlana Lazebnik

43 Measurements on planes p p Approach: unwarp then measure What kind of warp is this? Svetlana Lazebnik

44 Measurements on planes p p Approach: unwarp then measure Homography! Svetlana Lazebnik

45 Application: Image rectification Piero della Francesca, Flagellation, ca. 455 Svetlana Lazebnik

46 Application: Image editing Inserting synthetic objects into images: K. Karsch and V. Hedau and D. Forsyth and D. Hoiem, Rendering Synthetic Objects into Legacy Photographs, SIGGRAPH Asia 20 Svetlana Lazebnik

47 Application: Object recognition D. Hoiem, A.A. Efros, and M. Hebert, "Putting Objects in Perspective", CVPR 2006 Svetlana Lazebnik

48 Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence Application: structure from motion revisited

49 Correspondence constraints? Given p in left image, where can corresponding point p be? Kristen Grauman

50 Correspondence constraints? Kristen Grauman

51 Epipolar constraint Geometry of two views constrains where the corresponding pixel for some image point in the first view must occur in the second view. It must be on the line carved out by a plane connecting the world point and optical centers. Kristen Grauman

52 Epipolar geometry Epipolar Plane Epipolar Line Epipole Baseline Epipole Kristen Grauman

53 Epipolar geometry: terms Baseline: line joining the camera centers Epipole: point of intersection of baseline with image plane Epipolar plane: plane containing baseline and world point Epipolar line: intersection of epipolar plane with the image plane All epipolar lines intersect at the epipole An epipolar plane intersects the left and right image planes in epipolar lines Why is the epipolar constraint useful? Kristen Grauman

54 Epipolar constraint This is useful because it reduces the correspondence problem to a D search along an epipolar line. Image from Andrew Zisserman

55 Example Kristen Grauman

56 What do the epipolar lines look like?. O l O r 2. O l O r Kristen Grauman

57 Epipolar lines: converging cameras Figure from Hartley & Zisserman

58 Epipolar lines: parallel cameras Where are the epipoles? Figure from Hartley & Zisserman

59 Epipolar lines So far, we have the explanation in terms of geometry. Now, how to express the epipolar constraints algebraically? Kristen Grauman

60 Stereo geometry Main idea Kristen Grauman

61 Stereo geometry If the stereo rig is calibrated, we know : how to rotate and translate camera reference frame to get to camera reference frame 2. Rotation: 3 x 3 matrix R; translation: 3 vector T. Kristen Grauman

62 Stereo geometry If the stereo rig is calibrated, we know : how to rotate and translate camera reference frame to get to camera reference frame 2. X ' c RX c T Kristen Grauman

63 An aside: cross product Vector cross product takes two vectors and returns a third vector that s perpendicular to both inputs. So here, c is perpendicular to both a and b, which means the dot product = 0. Kristen Grauman

64 From geometry to algebra X' RX T TX Normal to the plane TRX TRX TT X T X X T RX 0 Kristen Grauman

65 Another aside: Matrix form of cross product Can be expressed as a matrix multiplication. c b b b a a a a a a b a a a a a a a a x Kristen Grauman

66 From geometry to algebra X' RX T TX Normal to the plane TRX TRX TT X T X X T RX 0 Kristen Grauman

67 Let X X T RX 0 [T ] RX 0 x E [T x] R X T EX 0 Essential matrix E is called the essential matrix, and it relates corresponding image points between both cameras, given the rotation and translation. If we observe a point in one image, its position in other image is constrained to lie on line defined by above. Note: these points are in camera coordinate systems. Kristen Grauman

68 Fundamental matrix Relates pixel coordinates in the two views p im, rightfp im, left 0 More general form than essential matrix: we remove need to know intrinsic parameters If we estimate fundamental matrix from correspondences in pixel coordinates, can reconstruct epipolar geometry without intrinsic or extrinsic parameters.

69 Fundamental Matrix, right c, left p c Ep 0 From before, the essential matrix E. M p E M p 0 p int,right im, right int,left im, left M EM p 0 im, right int,right int,left im, left F Fundamental matrix p im, rightfp im, left 0

70 Properties of the Fundamental Matrix is the epipolar line associated with T is the epipolar line associated with and is rank 2 p q F T q Fp e e 2

71 Estimating the Fundamental Matrix If we don t know K, K 2, R, or t, can we estimate F for two images? Yes, given enough correspondences

72 Estimating F 8-point algorithm The fundamental matrix F is defined by x' Fx 0 for any pair of matches x and x in two images. Let x=(u,v,) T and x =(u,v,) T, each match gives a linear equation f F f f 2 3 f f f f f f uu' f vu' f2 u' f3 uv' f2 vv' f22 v' f23 uf3 vf32 f33 0

73 f f f f f f f f f v u v v v v u u u v u u v u v v v v u u u v u u v u v v v u v u v u u u n n n n n n n n n n n n In reality, instead of solving, we seek f to minimize, least eigenvector of. 0 Af Af A A Estimating F 8-point algorithm

74 F should have rank 2 To enforce that F is of rank 2, F is replaced by F that minimizes subject to the rank constraint. F F' This is achieved by SVD. Let, where, let then is the solution. V U F Σ Σ Σ' 2 V U F Σ' ' Estimating F 8-point algorithm

75 Estimating F 8-point algorithm Pros: it is linear, easy to implement and fast Cons: susceptible to noise

76 Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence Application: structure from motion revisited

77 Stereo Correspondence Goal: Find correspondences (pairs of points (u,v ) (u,v)). ) Find interest points in image 2) Compute correspondences 3) Compute epipolar geometry 4) Refine Example from Andrew Zisserman

78 Stereo Correspondence ) Find interest points

79 Stereo Correspondence 2) Match points within proximity to get putative matches

80 Stereo Correspondence 3) Compute epipolar geometry -- robustly with RANSAC Select random sample of putative correspondences Compute F using them - determines epipolar constraint Evaluate amount of support - inliers within threshold distance of epipolar line Choose F with most support (inliers)

81 Using window search to get putative matches: noisy, but enough to compute F using RANSAC Pruned matches: those consistent with epipolar geometry

82 Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence Application: structure from motion revisited

83 Structure from Motion. Detect features using SIFT 2. Match features between pairs of images 3. Refine matching using RANSAC to find correspondences between each pair 4. Massive bundle adjustment Noah Snavely

84 Structure from Motion. Detect features using SIFT 2. Match features between pairs of images 3. Refine matching using RANSAC to find correspondences between each pair 4. Massive bundle adjustment Use fundamental matrix to detect inliers during RANSAC! Noah Snavely

85 Structure from Motion Results Noah Snavely

86 Structure from Motion Results Noah Snavely

87 Recap Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence Application: structure from motion revisited

Single-view metrology

Single-view metrology Single-view metrology Magritte, Personal Values, 952 Many slides from S. Seitz, D. Hoiem Camera calibration revisited What if world coordinates of reference 3D points are not known? We can use scene features

More information

Epipolar Geometry and Stereo Vision

Epipolar Geometry and Stereo Vision CS 1674: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 5, 2016 Announcement Please send me three topics you want me to review next

More information

Epipolar Geometry and Stereo Vision

Epipolar Geometry and Stereo Vision CS 1699: Intro to Computer Vision Epipolar Geometry and Stereo Vision Prof. Adriana Kovashka University of Pittsburgh October 8, 2015 Today Review Projective transforms Image stitching (homography) Epipolar

More information

Epipolar Geometry and Stereo Vision

Epipolar Geometry and Stereo Vision Epipolar Geometry and Stereo Vision Computer Vision Shiv Ram Dubey, IIIT Sri City Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X

More information

Structure from motion

Structure from motion Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t R 2 3,t 3 Camera 1 Camera

More information

Computer Vision Lecture 17

Computer Vision Lecture 17 Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week

More information

Computer Vision Lecture 17

Computer Vision Lecture 17 Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester

More information

1 Projective Geometry

1 Projective Geometry CIS8, Machine Perception Review Problem - SPRING 26 Instructions. All coordinate systems are right handed. Projective Geometry Figure : Facade rectification. I took an image of a rectangular object, and

More information

Recovering structure from a single view Pinhole perspective projection

Recovering structure from a single view Pinhole perspective projection EPIPOLAR GEOMETRY The slides are from several sources through James Hays (Brown); Silvio Savarese (U. of Michigan); Svetlana Lazebnik (U. Illinois); Bill Freeman and Antonio Torralba (MIT), including their

More information

CS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017

CS 2770: Intro to Computer Vision. Multiple Views. Prof. Adriana Kovashka University of Pittsburgh March 14, 2017 CS 277: Intro to Computer Vision Multiple Views Prof. Adriana Kovashka Universit of Pittsburgh March 4, 27 Plan for toda Affine and projective image transformations Homographies and image mosaics Stereo

More information

Camera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration

Camera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1

More information

Epipolar Geometry and Stereo Vision

Epipolar Geometry and Stereo Vision Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x

More information

Epipolar Geometry Prof. D. Stricker. With slides from A. Zisserman, S. Lazebnik, Seitz

Epipolar Geometry Prof. D. Stricker. With slides from A. Zisserman, S. Lazebnik, Seitz Epipolar Geometry Prof. D. Stricker With slides from A. Zisserman, S. Lazebnik, Seitz 1 Outline 1. Short introduction: points and lines 2. Two views geometry: Epipolar geometry Relation point/line in two

More information

Two-view geometry Computer Vision Spring 2018, Lecture 10

Two-view geometry Computer Vision Spring 2018, Lecture 10 Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of

More information

Single-view 3D Reconstruction

Single-view 3D Reconstruction Single-view 3D Reconstruction 10/12/17 Computational Photography Derek Hoiem, University of Illinois Some slides from Alyosha Efros, Steve Seitz Notes about Project 4 (Image-based Lighting) You can work

More information

55:148 Digital Image Processing Chapter 11 3D Vision, Geometry

55:148 Digital Image Processing Chapter 11 3D Vision, Geometry 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence

More information

Camera Calibration. COS 429 Princeton University

Camera Calibration. COS 429 Princeton University Camera Calibration COS 429 Princeton University Point Correspondences What can you figure out from point correspondences? Noah Snavely Point Correspondences X 1 X 4 X 3 X 2 X 5 X 6 X 7 p 1,1 p 1,2 p 1,3

More information

Last lecture. Passive Stereo Spacetime Stereo

Last lecture. Passive Stereo Spacetime Stereo Last lecture Passive Stereo Spacetime Stereo Today Structure from Motion: Given pixel correspondences, how to compute 3D structure and camera motion? Slides stolen from Prof Yungyu Chuang Epipolar geometry

More information

Structure from motion

Structure from motion Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera

More information

Lecture 9: Epipolar Geometry

Lecture 9: Epipolar Geometry Lecture 9: Epipolar Geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Why is stereo useful? Epipolar constraints Essential and fundamental matrix Estimating F (Problem Set 2

More information

Undergrad HTAs / TAs. Help me make the course better! HTA deadline today (! sorry) TA deadline March 21 st, opens March 15th

Undergrad HTAs / TAs. Help me make the course better! HTA deadline today (! sorry) TA deadline March 21 st, opens March 15th Undergrad HTAs / TAs Help me make the course better! HTA deadline today (! sorry) TA deadline March 2 st, opens March 5th Project 2 Well done. Open ended parts, lots of opportunity for mistakes. Real implementation

More information

Stereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman

Stereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure

More information

Unit 3 Multiple View Geometry

Unit 3 Multiple View Geometry Unit 3 Multiple View Geometry Relations between images of a scene Recovering the cameras Recovering the scene structure http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook1.html 3D structure from images Recover

More information

CS231A Midterm Review. Friday 5/6/2016

CS231A Midterm Review. Friday 5/6/2016 CS231A Midterm Review Friday 5/6/2016 Outline General Logistics Camera Models Non-perspective cameras Calibration Single View Metrology Epipolar Geometry Structure from Motion Active Stereo and Volumetric

More information

Pinhole Camera Model 10/05/17. Computational Photography Derek Hoiem, University of Illinois

Pinhole Camera Model 10/05/17. Computational Photography Derek Hoiem, University of Illinois Pinhole Camera Model /5/7 Computational Photography Derek Hoiem, University of Illinois Next classes: Single-view Geometry How tall is this woman? How high is the camera? What is the camera rotation? What

More information

Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1)

Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1) Image Rectification (Stereo) (New book: 7.2.1, old book: 11.1) Guido Gerig CS 6320 Spring 2013 Credits: Prof. Mubarak Shah, Course notes modified from: http://www.cs.ucf.edu/courses/cap6411/cap5415/, Lecture

More information

Multi-view geometry problems

Multi-view geometry problems Multi-view geometry Multi-view geometry problems Structure: Given projections o the same 3D point in two or more images, compute the 3D coordinates o that point? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera

More information

BIL Computer Vision Apr 16, 2014

BIL Computer Vision Apr 16, 2014 BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm

More information

Recap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views?

Recap: Features and filters. Recap: Grouping & fitting. Now: Multiple views 10/29/2008. Epipolar geometry & stereo vision. Why multiple views? Recap: Features and filters Epipolar geometry & stereo vision Tuesday, Oct 21 Kristen Grauman UT-Austin Transforming and describing images; textures, colors, edges Recap: Grouping & fitting Now: Multiple

More information

Structure from Motion CSC 767

Structure from Motion CSC 767 Structure from Motion CSC 767 Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R,t R 2,t 2 R 3,t 3 Camera??

More information

calibrated coordinates Linear transformation pixel coordinates

calibrated coordinates Linear transformation pixel coordinates 1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial

More information

Structure from Motion

Structure from Motion /8/ Structure from Motion Computer Vision CS 43, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from motion

More information

Structure from motion

Structure from motion Multi-view geometry Structure rom motion Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera 3 R 3,t 3 Figure credit: Noah Snavely Structure rom motion? Camera 1 Camera 2 R 1,t 1 R 2,t 2 Camera 3 R 3,t 3 Structure:

More information

CHAPTER 3. Single-view Geometry. 1. Consequences of Projection

CHAPTER 3. Single-view Geometry. 1. Consequences of Projection CHAPTER 3 Single-view Geometry When we open an eye or take a photograph, we see only a flattened, two-dimensional projection of the physical underlying scene. The consequences are numerous and startling.

More information

Structure from motion

Structure from motion Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera

More information

Structure from Motion

Structure from Motion 11/18/11 Structure from Motion Computer Vision CS 143, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from

More information

Week 2: Two-View Geometry. Padua Summer 08 Frank Dellaert

Week 2: Two-View Geometry. Padua Summer 08 Frank Dellaert Week 2: Two-View Geometry Padua Summer 08 Frank Dellaert Mosaicking Outline 2D Transformation Hierarchy RANSAC Triangulation of 3D Points Cameras Triangulation via SVD Automatic Correspondence Essential

More information

Geometric camera models and calibration

Geometric camera models and calibration Geometric camera models and calibration http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 13 Course announcements Homework 3 is out. - Due October

More information

Homographies and RANSAC

Homographies and RANSAC Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position

More information

Lecture 5 Epipolar Geometry

Lecture 5 Epipolar Geometry Lecture 5 Epipolar Geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 5-24-Jan-18 Lecture 5 Epipolar Geometry Why is stereo useful? Epipolar constraints Essential

More information

Epipolar geometry. x x

Epipolar geometry. x x Two-view geometry Epipolar geometry X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections

More information

Miniature faking. In close-up photo, the depth of field is limited.

Miniature faking. In close-up photo, the depth of field is limited. Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg

More information

Projective Geometry and Camera Models

Projective Geometry and Camera Models /2/ Projective Geometry and Camera Models Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Note about HW Out before next Tues Prob: covered today, Tues Prob2: covered next Thurs Prob3:

More information

Epipolar geometry contd.

Epipolar geometry contd. Epipolar geometry contd. Estimating F 8-point algorithm The fundamental matrix F is defined by x' T Fx = 0 for any pair of matches x and x in two images. Let x=(u,v,1) T and x =(u,v,1) T, each match gives

More information

CS201 Computer Vision Camera Geometry

CS201 Computer Vision Camera Geometry CS201 Computer Vision Camera Geometry John Magee 25 November, 2014 Slides Courtesy of: Diane H. Theriault (deht@bu.edu) Question of the Day: How can we represent the relationships between cameras and the

More information

55:148 Digital Image Processing Chapter 11 3D Vision, Geometry

55:148 Digital Image Processing Chapter 11 3D Vision, Geometry 55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence

More information

More Single View Geometry

More Single View Geometry More Single View Geometry with a lot of slides stolen from Steve Seitz Cyclops Odilon Redon 1904 15-463: Computational Photography Alexei Efros, CMU, Fall 2008 Quiz: which is 1,2,3-point perspective Image

More information

arxiv: v1 [cs.cv] 28 Sep 2018

arxiv: v1 [cs.cv] 28 Sep 2018 Camera Pose Estimation from Sequence of Calibrated Images arxiv:1809.11066v1 [cs.cv] 28 Sep 2018 Jacek Komorowski 1 and Przemyslaw Rokita 2 1 Maria Curie-Sklodowska University, Institute of Computer Science,

More information

3D Geometry and Camera Calibration

3D Geometry and Camera Calibration 3D Geometry and Camera Calibration 3D Coordinate Systems Right-handed vs. left-handed x x y z z y 2D Coordinate Systems 3D Geometry Basics y axis up vs. y axis down Origin at center vs. corner Will often

More information

More Single View Geometry

More Single View Geometry More Single View Geometry with a lot of slides stolen from Steve Seitz Cyclops Odilon Redon 1904 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Final Projects Are coming up fast! Undergrads

More information

3D Sensing and Reconstruction Readings: Ch 12: , Ch 13: ,

3D Sensing and Reconstruction Readings: Ch 12: , Ch 13: , 3D Sensing and Reconstruction Readings: Ch 12: 12.5-6, Ch 13: 13.1-3, 13.9.4 Perspective Geometry Camera Model Stereo Triangulation 3D Reconstruction by Space Carving 3D Shape from X means getting 3D coordinates

More information

But First: Multi-View Projective Geometry

But First: Multi-View Projective Geometry View Morphing (Seitz & Dyer, SIGGRAPH 96) Virtual Camera Photograph Morphed View View interpolation (ala McMillan) but no depth no camera information Photograph But First: Multi-View Projective Geometry

More information

Multiple View Geometry

Multiple View Geometry Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric

More information

Projective Geometry and Camera Models

Projective Geometry and Camera Models Projective Geometry and Camera Models Computer Vision CS 43 Brown James Hays Slides from Derek Hoiem, Alexei Efros, Steve Seitz, and David Forsyth Administrative Stuff My Office hours, CIT 375 Monday and

More information

Augmented Reality II - Camera Calibration - Gudrun Klinker May 11, 2004

Augmented Reality II - Camera Calibration - Gudrun Klinker May 11, 2004 Augmented Reality II - Camera Calibration - Gudrun Klinker May, 24 Literature Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2. (Section 5,

More information

Multi-View Geometry Part II (Ch7 New book. Ch 10/11 old book)

Multi-View Geometry Part II (Ch7 New book. Ch 10/11 old book) Multi-View Geometry Part II (Ch7 New book. Ch 10/11 old book) Guido Gerig CS-GY 6643, Spring 2016 gerig@nyu.edu Credits: M. Shah, UCF CAP5415, lecture 23 http://www.cs.ucf.edu/courses/cap6411/cap5415/,

More information

Rectification and Disparity

Rectification and Disparity Rectification and Disparity Nassir Navab Slides prepared by Christian Unger What is Stereo Vision? Introduction A technique aimed at inferring dense depth measurements efficiently using two cameras. Wide

More information

6.819 / 6.869: Advances in Computer Vision Antonio Torralba and Bill Freeman. Lecture 11 Geometry, Camera Calibration, and Stereo.

6.819 / 6.869: Advances in Computer Vision Antonio Torralba and Bill Freeman. Lecture 11 Geometry, Camera Calibration, and Stereo. 6.819 / 6.869: Advances in Computer Vision Antonio Torralba and Bill Freeman Lecture 11 Geometry, Camera Calibration, and Stereo. 2d from 3d; 3d from multiple 2d measurements? 2d 3d? Perspective projection

More information

Machine vision. Summary # 11: Stereo vision and epipolar geometry. u l = λx. v l = λy

Machine vision. Summary # 11: Stereo vision and epipolar geometry. u l = λx. v l = λy 1 Machine vision Summary # 11: Stereo vision and epipolar geometry STEREO VISION The goal of stereo vision is to use two cameras to capture 3D scenes. There are two important problems in stereo vision:

More information

Computer Vision I. Announcement. Stereo Vision Outline. Stereo II. CSE252A Lecture 15

Computer Vision I. Announcement. Stereo Vision Outline. Stereo II. CSE252A Lecture 15 Announcement Stereo II CSE252A Lecture 15 HW3 assigned No class on Thursday 12/6 Extra class on Tuesday 12/4 at 6:30PM in WLH Room 2112 Mars Exploratory Rovers: Spirit and Opportunity Stereo Vision Outline

More information

CS231M Mobile Computer Vision Structure from motion

CS231M Mobile Computer Vision Structure from motion CS231M Mobile Computer Vision Structure from motion - Cameras - Epipolar geometry - Structure from motion Pinhole camera Pinhole perspective projection f o f = focal length o = center of the camera z y

More information

Agenda. Rotations. Camera calibration. Homography. Ransac

Agenda. Rotations. Camera calibration. Homography. Ransac Agenda Rotations Camera calibration Homography Ransac Geometric Transformations y x Transformation Matrix # DoF Preserves Icon translation rigid (Euclidean) similarity affine projective h I t h R t h sr

More information

Image warping and stitching

Image warping and stitching Image warping and stitching May 4 th, 2017 Yong Jae Lee UC Davis Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 2 Alignment problem In alignment, we will

More information

Lecture 14: Basic Multi-View Geometry

Lecture 14: Basic Multi-View Geometry Lecture 14: Basic Multi-View Geometry Stereo If I needed to find out how far point is away from me, I could use triangulation and two views scene point image plane optical center (Graphic from Khurram

More information

Lecture 10: Multi-view geometry

Lecture 10: Multi-view geometry Lecture 10: Multi-view geometry Professor Stanford Vision Lab 1 What we will learn today? Review for stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from

More information

CS223b Midterm Exam, Computer Vision. Monday February 25th, Winter 2008, Prof. Jana Kosecka

CS223b Midterm Exam, Computer Vision. Monday February 25th, Winter 2008, Prof. Jana Kosecka CS223b Midterm Exam, Computer Vision Monday February 25th, Winter 2008, Prof. Jana Kosecka Your name email This exam is 8 pages long including cover page. Make sure your exam is not missing any pages.

More information

C280, Computer Vision

C280, Computer Vision C280, Computer Vision Prof. Trevor Darrell trevor@eecs.berkeley.edu Lecture 11: Structure from Motion Roadmap Previous: Image formation, filtering, local features, (Texture) Tues: Feature-based Alignment

More information

Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction

Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction Carsten Rother 09/12/2013 Computer Vision I: Multi-View 3D reconstruction Roadmap this lecture Computer Vision I: Multi-View

More information

COMPARATIVE STUDY OF DIFFERENT APPROACHES FOR EFFICIENT RECTIFICATION UNDER GENERAL MOTION

COMPARATIVE STUDY OF DIFFERENT APPROACHES FOR EFFICIENT RECTIFICATION UNDER GENERAL MOTION COMPARATIVE STUDY OF DIFFERENT APPROACHES FOR EFFICIENT RECTIFICATION UNDER GENERAL MOTION Mr.V.SRINIVASA RAO 1 Prof.A.SATYA KALYAN 2 DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING PRASAD V POTLURI SIDDHARTHA

More information

Computer Vision Projective Geometry and Calibration. Pinhole cameras

Computer Vision Projective Geometry and Calibration. Pinhole cameras Computer Vision Projective Geometry and Calibration Professor Hager http://www.cs.jhu.edu/~hager Jason Corso http://www.cs.jhu.edu/~jcorso. Pinhole cameras Abstract camera model - box with a small hole

More information

Today. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography

Today. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography Computational Photography Matthias Zwicker University of Bern Fall 2009 Today From 2D to 3D using multiple views Introduction Geometry of two views Stereo matching Other applications Multiview geometry

More information

Image warping and stitching

Image warping and stitching Image warping and stitching May 5 th, 2015 Yong Jae Lee UC Davis PS2 due next Friday Announcements 2 Last time Interactive segmentation Feature-based alignment 2D transformations Affine fit RANSAC 3 Alignment

More information

More Single View Geometry

More Single View Geometry More Single View Geometry 5-463: Rendering and Image rocessing Alexei Efros with a lot of slides stolen from Steve Seitz and Antonio Criminisi Quiz! Image B Image A Image C How can we model this scene?.

More information

Project 4 Results. Representation. Data. Learning. Zachary, Hung-I, Paul, Emanuel. SIFT and HoG are popular and successful.

Project 4 Results. Representation. Data. Learning. Zachary, Hung-I, Paul, Emanuel. SIFT and HoG are popular and successful. Project 4 Results Representation SIFT and HoG are popular and successful. Data Hugely varying results from hard mining. Learning Non-linear classifier usually better. Zachary, Hung-I, Paul, Emanuel Project

More information

Reminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches

Reminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches Reminder: Lecture 20: The Eight-Point Algorithm F = -0.00310695-0.0025646 2.96584-0.028094-0.00771621 56.3813 13.1905-29.2007-9999.79 Readings T&V 7.3 and 7.4 Essential/Fundamental Matrix E/F Matrix Summary

More information

Agenda. Rotations. Camera models. Camera calibration. Homographies

Agenda. Rotations. Camera models. Camera calibration. Homographies Agenda Rotations Camera models Camera calibration Homographies D Rotations R Y = Z r r r r r r r r r Y Z Think of as change of basis where ri = r(i,:) are orthonormal basis vectors r rotated coordinate

More information

Multiple View Geometry. Frank Dellaert

Multiple View Geometry. Frank Dellaert Multiple View Geometry Frank Dellaert Outline Intro Camera Review Stereo triangulation Geometry of 2 views Essential Matrix Fundamental Matrix Estimating E/F from point-matches Why Consider Multiple Views?

More information

Projective geometry for Computer Vision

Projective geometry for Computer Vision Department of Computer Science and Engineering IIT Delhi NIT, Rourkela March 27, 2010 Overview Pin-hole camera Why projective geometry? Reconstruction Computer vision geometry: main problems Correspondence

More information

Structure from Motion

Structure from Motion Structure from Motion Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz, N Snavely, and D. Hoiem Administrative stuffs HW 3 due 11:55 PM, Oct 17 (Wed) Submit your alignment results!

More information

Stereo Vision. MAN-522 Computer Vision

Stereo Vision. MAN-522 Computer Vision Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in

More information

CIS 580, Machine Perception, Spring 2016 Homework 2 Due: :59AM

CIS 580, Machine Perception, Spring 2016 Homework 2 Due: :59AM CIS 580, Machine Perception, Spring 2016 Homework 2 Due: 2015.02.24. 11:59AM Instructions. Submit your answers in PDF form to Canvas. This is an individual assignment. 1 Recover camera orientation By observing

More information

Dense 3D Reconstruction. Christiano Gava

Dense 3D Reconstruction. Christiano Gava Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem

More information

Camera model and multiple view geometry

Camera model and multiple view geometry Chapter Camera model and multiple view geometry Before discussing how D information can be obtained from images it is important to know how images are formed First the camera model is introduced and then

More information

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253 Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine

More information

Scene Modeling for a Single View

Scene Modeling for a Single View on to 3D Scene Modeling for a Single View We want real 3D scene walk-throughs: rotation translation Can we do it from a single photograph? Reading: A. Criminisi, I. Reid and A. Zisserman, Single View Metrology

More information

Image correspondences and structure from motion

Image correspondences and structure from motion Image correspondences and structure from motion http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 20 Course announcements Homework 5 posted.

More information

Camera Registration in a 3D City Model. Min Ding CS294-6 Final Presentation Dec 13, 2006

Camera Registration in a 3D City Model. Min Ding CS294-6 Final Presentation Dec 13, 2006 Camera Registration in a 3D City Model Min Ding CS294-6 Final Presentation Dec 13, 2006 Goal: Reconstruct 3D city model usable for virtual walk- and fly-throughs Virtual reality Urban planning Simulation

More information

Announcements. Stereo

Announcements. Stereo Announcements Stereo Homework 2 is due today, 11:59 PM Homework 3 will be assigned today Reading: Chapter 7: Stereopsis CSE 152 Lecture 8 Binocular Stereopsis: Mars Given two images of a scene where relative

More information

Stereo Epipolar Geometry for General Cameras. Sanja Fidler CSC420: Intro to Image Understanding 1 / 33

Stereo Epipolar Geometry for General Cameras. Sanja Fidler CSC420: Intro to Image Understanding 1 / 33 Stereo Epipolar Geometry for General Cameras Sanja Fidler CSC420: Intro to Image Understanding 1 / 33 Stereo Epipolar geometry Case with two cameras with parallel optical axes General case Now this Sanja

More information

CS4670/5670: Computer Vision Kavita Bala. Lec 19: Single- view modeling 2

CS4670/5670: Computer Vision Kavita Bala. Lec 19: Single- view modeling 2 CS4670/5670: Computer Vision Kavita Bala Lec 19: Single- view modeling 2 Today Vanishing points in images are useful Recover size Camera calibrakon Vanishing points image plane vanishing point V camera

More information

CS 664 Slides #9 Multi-Camera Geometry. Prof. Dan Huttenlocher Fall 2003

CS 664 Slides #9 Multi-Camera Geometry. Prof. Dan Huttenlocher Fall 2003 CS 664 Slides #9 Multi-Camera Geometry Prof. Dan Huttenlocher Fall 2003 Pinhole Camera Geometric model of camera projection Image plane I, which rays intersect Camera center C, through which all rays pass

More information

Lecture 6 Stereo Systems Multi-view geometry

Lecture 6 Stereo Systems Multi-view geometry Lecture 6 Stereo Systems Multi-view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-5-Feb-4 Lecture 6 Stereo Systems Multi-view geometry Stereo systems

More information

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263

Index. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263 Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine

More information

Step-by-Step Model Buidling

Step-by-Step Model Buidling Step-by-Step Model Buidling Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure

More information

Image Based Rendering. D.A. Forsyth, with slides from John Hart

Image Based Rendering. D.A. Forsyth, with slides from John Hart Image Based Rendering D.A. Forsyth, with slides from John Hart Topics Mosaics translating cameras reveal extra information, break occlusion Optical flow for very small movements of the camera Explicit

More information

Multiple Views Geometry

Multiple Views Geometry Multiple Views Geometry Subhashis Banerjee Dept. Computer Science and Engineering IIT Delhi email: suban@cse.iitd.ac.in January 2, 28 Epipolar geometry Fundamental geometric relationship between two perspective

More information

More Single View Geometry

More Single View Geometry More Single View Geometry with a lot of slides stolen from Steve Seitz Cyclops Odilon Redon 1904 15-463: Computational hotography Alexei Efros, CMU, Fall 2011 Quiz: which is 1,2,3-point perspective Image

More information

Rectification and Distortion Correction

Rectification and Distortion Correction Rectification and Distortion Correction Hagen Spies March 12, 2003 Computer Vision Laboratory Department of Electrical Engineering Linköping University, Sweden Contents Distortion Correction Rectification

More information

3D Reconstruction from Two Views

3D Reconstruction from Two Views 3D Reconstruction from Two Views Huy Bui UIUC huybui1@illinois.edu Yiyi Huang UIUC huang85@illinois.edu Abstract In this project, we study a method to reconstruct a 3D scene from two views. First, we extract

More information

Fundamental matrix. Let p be a point in left image, p in right image. Epipolar relation. Epipolar mapping described by a 3x3 matrix F

Fundamental matrix. Let p be a point in left image, p in right image. Epipolar relation. Epipolar mapping described by a 3x3 matrix F Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix F Fundamental

More information