3D Photography. Marc Pollefeys, Torsten Sattler. Spring 2015
|
|
- Dayna Grant
- 6 years ago
- Views:
Transcription
1 3D Photography Marc Pollefeys, Torsten Sattler Spring 2015
2 Schedule (tentative) Feb 16 Feb 23 Mar 2 Mar 9 Mar 16 Mar 23 Mar 30 Apr 6 Apr 13 Apr 20 Apr 27 May 4 May 11 May 18 Apr 6 Introduction Geometry, Camera Model, Calibration Features, Tracking / Matching Project Proposals by Students Structure from Motion (SfM) + 2 papers Dense Correspondence (stereo / optical flow) + 2 papers Bundle Adjustment & SLAM + 2 papers Easter Multi-View Stereo & Volumetric Modeling + 2 papers Project Updates 3D Modeling with Depth Sensors + 2 papers 3D Scene Understanding + 2 papers 4D Video & Dynamic Scenes + 2 papers Final Demos Pentecost
3 Important Dates Decide for a project by Feb. 27 Form groups and send us your topics Proposal submission by March 6 Talk to your supervisor and send proposals Proposal presentation on March 9 5min per group
4 3D Photography Class 2 Projective Geometry and Camera model points, lines, planes conics and quadrics transformations camera model Read tutorial chapter 2 and Chapters 1, 2 and 5 in Hartley and Zisserman Chapters 2, 3 and 6 in 2 nd edition
5 Overview 2D Projective Geometry 3D Projective Geometry Camera Models & Calibration
6 2D Projective Geometry? Projections of planar surfaces
7 2D Projective Geometry? Measure distances A. Criminisi. Accurate Visual Metrology from Single and Multiple Uncalibrated Images. PhD Thesis 1999.
8 2D Projective Geometry? Discovering details Piero della Francesca, La Flagellazione di Cristo (1460) A. Criminisi. Accurate Visual Metrology from Single and Multiple Uncalibrated Images. PhD Thesis 1999.
9 2D Projective Geometry? Image Stitching
10 2D Projective Geometry? Image Stitching
11 Homogeneous coordinates y t x t y x' y' x y As matrix multiplication x Homogeneous coordinates: y x' cos sin t x x y' sin cos t y y Equivalence class of vectors, any vector is representative Projective Space P 2 = R 3 (0,0,0) T x,y,1 T ~ k x,y,1 z 1 x Projective transformation = matrix multiplication k x,y,1 T T,k 0 z
12 Homogeneous coordinates l a b (Homogeneous) representation of 2D line: ax by c 0 a,b,c T x,y,1 0 c / a 2 b 2 The point x lies on the line l if and only if l T x 0 Note that scale is unimportant for incidence relation a,b,c T ~ ka,b,c T,k 0 x,y,1 T ~ k x,y,1 T x 1 x 3, x 2 x 3 T,k 0 Homogeneous coordinates x 1,x 2,x 3 T but only 2DOF Inhomogeneous coordinates x, y T
13 Points from lines and vice-versa Intersections of lines The intersection of two lines l and l' is Line joining two points The line through two points x and x' is x l l' l x x' Example Note: x x' x x' x (0,1,-1) y 1 y 1 x 1 x (1,0,-1) y 1 with x 0 z -y -z 0 x y -x 0
14 Ideal points and the line at infinity Intersections of parallel lines l a,b,c Example T and l' a,b,c' T l l' b,a,0 T b,-a a,b tangent vector normal direction Ideal points x =1 x = 2 x 1,x 2,0 Line at infinity l 0,0,1 T T P 2 R 2 l Note that in P 2 there is no distinction between ideal points and others
15 Conics Curve described by 2 nd -degree equation in the plane ax 2 bxy cy 2 dx ey f 0 or homogenized x a x 1 x3,y a x 2 x3 ax 2 1 bx 1 x 2 cx 2 2 dx 1 x 3 ex 2 x 3 fx or in matrix form a b/2 d /2 x T Cx 0 with C b/2 c e /2 d /2 e /2 f 5DOF: a :b :c : d :e : f (defined up to scale)
16 Five points define a conic For each point the conic passes through ax i 2 bx i y i cy i 2 dx i ey i f 0 or x 2 i,x i y i,y 2 i,x i,y i,1.c 0 c a,b,c,d,e, f T stacking constraints yields x 1 2 x 2 2 x 3 2 x 4 2 x 5 2 x 1 y 1 y 1 2 x 2 y 2 y 2 2 x 3 y 3 y 3 2 x 4 y 4 y 4 2 x 5 y 5 y 5 2 x 1 y 1 1 x 2 y 2 1 x 3 y 3 1 c 0 x 4 y 4 1 x 5 y 5 1
17 Tangent lines to conics The line l tangent to C at point x on C is given by l=cx x l C
18 Dual conics A line tangent to the conic C satisfies l T * C l = 0 In general (C full rank): * -1 C C Dual conics = line conics = conic envelopes
19 Degenerate conics A conic is degenerate if matrix C is not of full rank e.g. two lines (rank 2) C = lm T + ml e.g. repeated line (rank 1) T m l C ll T l Degenerate line conics: 2 points (rank 2), double point (rank1) Note that for degenerate conics ( * C ) * C
20 2D projective transformations Definition: A projectivity is an invertible mapping h from P 2 to itself such that three points x 1,x 2,x 3 lie on the same line if and only if h(x 1 ),h(x 2 ),h(x 3 ) do. Theorem: A mapping h:p 2 P 2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P 2 reprented by a vector x it is true that h(x)=hx Definition: Projective transformation x' 1 x' 2 x' 3 h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 x 1 x 2 x 3 or x' Hx 8DOF projectivity=collineation=projective transformation=homography
21 Transformation of 2D points, lines and conics For a point transformation x' Hx Transformation for lines l' H -T l Transformation for conics C' H -T CH -1 Transformation for dual conics C' * HC * H T
22 Fixed points and lines He = λe (eigenvectors H = fixed points) ( 1 = 2 pointwise fixed line) H -T l l (eigenvectors H - T = fixed lines)
23 Hierarchy of 2D transformations Projective 8dof Affine 6dof Similarity 4dof Euclidean 3dof h h h a a 0 sr sr 0 r11 r h h h a a r r sr sr h13 h 23 h 33 tx t y 1 tx t y 1 tx t y 1 transformed squares invariants Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l Ratios of lengths, angles. The circular points I,J Lengths, areas.
24 Application: Removing Perspective Two stages: From perspective to affine transformation via the line at infinitiy From affine to similarity transformation via the circular points
25 The line at infinity 0 T T A 0 l H A l 0 l T T t 1 A 1 The line at infinity l is a fixed line under a projective transformation H if and only if H is an affinity Note: not fixed pointwise
26 Affine properties from images projection rectification H P H A l 1 l 2 l 3 l l 1 l 2 l 3 T,l 3 0
27 Affine Rectification v 1 v 2 l l 1 l 3 l 2 l 4 v v 2 = l3 l4 1 l1 l2 l v 1 v 2
28 The circular points 1 I i 0 1 J i 0 I H S I scos ssin t x 1 1 ssin scos t y i sei i I 0 0 The circular points I, J are fixed points under the projective transformation H iff H is a similarity
29 The circular points circular points l x 1 2 x 2 2 dx 1 x 3 ex 2 x 3 fx I 1,0,0 x 3 0 T i0,1,0 T x 1 2 x I 1,i,0 T J 1,-i,0 T Algebraically, encodes orthogonal directions
30 Conic dual to the circular points C C * * H IJ T C JI T * T S H S * C J I l * C The dual conic is fixed conic under the projective transformation H iff H is a similarity * C Note: has 4DOF (det = 0) l is the nullvector
31 Angles Euclidean: l l 1,l 2,l 3 cos T m m 1,m 2,m 3 l 1 m 1 l 2 m 2 l l 2 m m 2 T Projective: cos l T C * m l T C * l m T C * m l T C * m 0 (orthogonal) Once we know dual conic on projective plane, we can measure Euclidean angles!
32 Metric Rectification Dual conic under affinity C * A t 0 T I T AT 0 AA T 0 1 t T 1 0 T 0 S=AA -T symmetric, estimate from two pairs of orthogonal lines: l 1 m 1,l 1 m 2 l 2 m 1,l 2 m 2 s 0 Note: Result defined up to similarity
33 Overview 2D Projective Geometry 3D Projective Geometry Camera Models & Calibration
34 3D points and planes Homogeneous representation of 3D points and planes π1x1 + π2x 2 + π3x3 + π4x 4 = 0 The point X lies on the plane π if and only if π T X = 0 The plane π goes through the point X if and only if π T X = 0
35 Planes from points 0 π X X X T T T π = 0 X π = 0 X π = 0, X π T T T and from Solve (solve as right nullspace of ) π T T T X X X
36 Solve π π π T 1 T 2 T 3 X Points from planes T T T X fromπ1 X = 0, π2x = 0 and π3x = 0 0 X = Mx M X X π T M = 0 X (solve as right nullspace of T ) Representing a plane by its span 1 2X3 π π π T 1 2 T 3
37 Lines (4dof) Representing a line by its span A W B T T Dual representation W * P Q T T λa μb λp μq * T W W = WW * T = Example: X-axis 0 W W * (Alternative: Plücker representation, details see e.g. H&Z)
38 Points, lines and planes W M T M π = 0 X X W W * M T MX 0 π W π
39 Quadrics and dual quadrics X T QX 0 (Q : 4x4 symmetric matrix) 9 DOF (up to scale) In general, 9 points define quadric det(q)=0 degenerate quadric tangent plane Dual quartic: ( adjoint) relation to quadric (non-degenerate) QX T Q * 0 Q * Q -1 Q * o Q o o o o o
40 Transformation of 3D points, planes and quadrics For a point transformation X' HX (cfr. 2D equivalent) x' Hx Transformation for lines ' H -T Transformation for quadrics Q' H -T QH -1 l' H -T l C' H -T CH -1 Transformation for dual quadrics Q' * HQ * H T C' * HC * H T
41 Hierarchy of 3D transformations Projective 15dof A v T t v Intersection and tangency Affine 12dof Similarity 7dof Euclidean 6dof A t 0 T 1 sr t 0 T 1 R t 0 T 1 Parallellism of planes, Volume ratios, centroids, The plane at infinity π Angles, ratios of length The absolute conic Ω Volume
42 Hierarchy of 3D transformations projective Plane at infinity affine Absolute conic similarity
43 The plane at infinity 0 T T A 0 0 π H A π π T T - t A The plane at infinity π is a fixed plane under a projective transformation H iff H is an affinity 1. canonical position 2. contains directions D X 1,X 2,X 3,0 T 3. two planes are parallel line of intersection in π 4. line // line (or plane) point of intersection in π 0,0,0,1 T
44 The absolute conic The absolute conic Ω is a (point) conic on π. In a metric frame: or conic for directions: (with no real points) X 1 2 X 2 2 X 3 2 X 4 0 X 1,X 2,X 3 I X 1,X 2,X 3 The absolute conic Ω is a fixed conic under the projective transformation H iff H is a similarity T 1. Ω is only fixed as a set 2. Circle intersect Ω in two circular points 3. Spheres intersect π in Ω
45 The absolute dual quadric * I 0 0 T 0 The absolute dual quadric Ω * is a fixed quadric under the projective transformation H iff H is a similarity 1. 8 dof 2. plane at infinity π is the nullvector of Ω 3. Angles: cos 1 T * 2 T 1 * 1 T 2 * 2
46 Overview 2D Projective Geometry 3D Projective Geometry Camera Models & Calibration
47 Camera Model Relation between pixels and rays in space?
48 Pinhole Camera
49 Pinhole Camera Model (X,Y,Z) T a ( fx /Z, fy /Z) T X Y a Z 1 fx fy Z X f 0 f 0 Y Z linear projection in homogeneous coordinates!
50 Pinhole Camera Model X X fx f 10 0 Y 0 fy Y x PX f 1 0 Z Z Z P diag( f, f,1) I 0
51 Principal Point Offset (X,Y,Z) T a ( fx /Z p x, fy /Z p y ) T X Y a Z 1 (p x, p y ) T fx Zp x fy Zp x Z principal point X f p x 0 f p y 0 Y Z 1 0 1
52 Principal Point Offset X fx Zp x f p x 0 fy Zp Y x KI 0f X p y 0 cam Z Z f p x K f p y 1 calibration matrix
53 Camera Rotation and Translation C X cam RX - C X X cam R -R C Y R -R C X 0 1 Z x KR I 0 I X -C cam X x PX P KR t t -RC C -R T t
54 CCD Camera K m x m y p x f p y 1 f p x p y 1
55 General Projective Camera K x s p x y p y 1 P KRI -C non-singular P KR t In practice: 11 dof (5+3+3) x y, s 0 p x, p y intrinsic camera parameters extrinsic camera parameters = image center
56 Affine Cameras
57 Camera Model Relation between pixels and rays in space?
58 Projector Model Relation between pixels and rays in space (dual of camera)? (main geometric difference is vertical principal point offset to reduce keystone effect)
59 Meydenbauer Camera vertical lens shift to allow direct ortho-photographs
60 Projector-Camera Systems IR projector IR camera
61 Action of Projective Camera on Points and Lines forward projection of point x PX image point back-projects to line / ray forward projection of line (not pointwise!) X P(A B) PA PB a b back-projection of line P T l X R T K 1 x T X l T PX l T x 0; x PX
62 Action of Projective Camera on Conics and Quadrics back-projection to cone Q co P T CP x T Cx X T P T CPX 0 x PX projection of quadric C * PQ * P T T Q * l T PQ * P T l 0 P T l
63 Image of Absolute Conic * P * P T I 0 K R t KK T K -T K RT t T K T
64 A Simple Calibration Device (i) compute H for each square (0,0),(1,0),(0,1),(1,1)) (ii) compute the imaged circular points H(1,±i,0) T (iii) fit a conic to 6 circular points (iv) compute K from through Cholesky factorization ( Zhang s calibration method)
65 Practical Camera Calibration Method and Pictures from Zhang (ICCV 99): Flexible Camera Calibration By Viewing a Plane From Unknown Orientations Unknown: constant camera intrinsics K (varying) camera pose R,t Known: 3D coordinates of chessboard corners => Define to be the z=0 plane ( X=[X 1 X 2 0 1] T ) Point is mapped as λx = K (r 1 r 2 r 3 t) X x = K (r 1 r 2 t) [X 1 X 2 1] K Homography H between image and chess coordinates, estimate from known X i and measured x i f x f y p x p y 1
66 Direct Linear Transformation (DLT) x i Hx i 0 x i x i, y i, w i y i h 3T x i w i h 2T x i x i Hx i w i h 1T x i x i h 3T x i x i h 2T x i y i h 1T x i T w i x i T y i x i 0 T T w i x i T y i x i 0 T T x i x i T x i x i 0 T A i h 0 T h 1 h 2 h 3 h 1T x i Hx i h 2T x i h 3T x i 0
67 Direct Linear Transformation (DLT) Equations are linear in h A i h 0 Only 2 out of 3 are linearly independent (indeed, 2 eq/pt) 0 T T T i x i y i x i 0 T T T T w w i x i 0 T i x i y T x i x i x i T i w T T y i x i i 0 T T x i x i x i 0 T i x i h 1 h 2 h 3 (only x i Adrop 1 i y third i A 2 i row w i Aif 3 i w i 0 0) Holds for any homogeneous representation, e.g. (x i,y i,1) 0
68 Direct Linear Transformation (DLT) Solving for H A 1 A 2 A 3 A 4 Ah h 0 size A is 8x9 or 12x9, but rank 8 Trivial solution is h=0 9 T is not interesting 1-D null-space yields solution of interest pick for example the one with h 1
69 Direct Linear Transformation (DLT) Over-determined solution No exact solution because of inexact measurement i.e. noise Ah 0 A 1 A 2 Ah h 0 M A n Find approximate solution - Additional constraint needed to avoid 0, e.g. - not possible, so minimize Ah h 1
70 DLT algorithm Objective Given n 4 2D to 2D point correspondences {x i x i }, determine the 2D homography matrix H such that x i =Hx i Algorithm (i) (ii) For each correspondence x i x i compute A i. Usually only two first rows needed. Assemble n 2x9 matrices A i into a single 2nx9 matrix A (iii) Obtain SVD of A. Solution for h is last column of V (iv) Determine H from h
71 Importance of Normalization x i y i 1 y i x i x i y i x i x i y i y i x i y i y i x i ~10 2 ~ ~10 2 ~ ~10 4 ~10 4 ~10 2 orders of magnitude difference! h 1 h 2 h 3 0 Monte Carlo simulation for identity computation based on 5 points (not normalized normalized)
72 Normalized DLT Algorithm Objective Given n 4 2D to 2D point correspondences {x i x i }, determine the 2D homography matrix H such that x i =Hx i Algorithm (i) (ii) Normalize points Apply DLT algorithm to (iii) Denormalize solution x i T norm x i, x i x i x i, -1 H T norm H T norm Normalization (independently per image): T norm x i Translate points such that centroid is at origin Isotropic scaling such that mean distance to origin is 2
73 x xˆ x Geometric Distance measured coordinates estimated coordinates true coordinates d(.,.) Euclidean distance (in image) Error in one image H ˆ argmin d H H i i 2 x i,hx i e.g. calibration pattern Symmetric transfer error H ˆ argmin dx i,h -1 x 2 i d x i,hx i Reprojection error H ˆ, x ˆ i, x ˆ i argmin d x i, ˆ H ˆ, x ˆ i, ˆ x i x' i 2 x i 2 d x ˆ i x i, 2 subject to x ˆ i H ˆ ˆ x i
74 Reprojection Error 2 d d x,h -1 x x,hx 2 2 dx, x ˆ 2 d x, ˆ x
75 Statistical Cost Function and Maximum Likelihood Estimation Optimal cost function related to noise model Assume zero-mean isotropic Gaussian noise (assume outliers removed) 2 / 2 2 Prx 1 x,x 2 2 ed Error in one image Pr x 1 H i i log Pr x H 1 i 2 2 ed x i,hx i 2 / 2 2 Maximum Likelihood Estimate d 2 x i,hx i 2 2 i d( x i,hx i ) 2 const
76 Statistical Cost Function and Maximum Likelihood Estimation Optimal cost function related to noise model Assume zero-mean isotropic Gaussian noise (assume outliers removed) 2 / 2 2 Prx 1 x,x 2 2 ed Error in both images Pr x 1 H i i / 2 2 d x 2 2 e i,x i 2 d x,hx 2 i i Maximum Likelihood Estimate x i 2 x i 2 d x i, ˆ d x i, ˆ
77 Objective Gold Standard Algorithm Given n 4 2D to 2D point correspondences {x i x i }, determine the Maximum Likelihood Estimation of H (this also implies computing optimal x i =Hx i ) Algorithm (i) Initialization: compute an initial estimate using normalized DLT or RANSAC (ii) Geometric minimization of symmetric transfer error: Minimize using Levenberg-Marquardt over 9 entries of h or reprojection error: compute initial estimate dx for optimal {x i } i, x ˆ i 2 dx i, x ˆ i 2 minimize cost over {H,x 1,x 2,,x n } if many points, use sparse method
78 Radial Distortion Due to spherical lenses (cheap) Model: R R ( x, y) (1 K1( x y ) K2( x y )...) x y straight lines are not straight anymore
79 Calibration with Radial Low radial distortion: Distortion Ignore radial distortion during initial calibration Estimate distortion parameters, refine full calibration High radial distortion: Simultaneous estimation Fitzgibbon, Simultaneous linear estimation of multiple view geometry and lens distortion, CVPR 2001 Kukelova et al., Real-Time Solution to the Absolute Pose Problem with Unknown Radial Distortion and Focal Length, ICCV 2013
80 Bouguet Toolbox
81 Rolling Shutter Cameras Image build row by row Distortions based on depth and speed Many mobile phone cameras have rolling shutter Video credit: Olivier Saurer
82 Rolling Shutter Cameras Calibration more complex: Internal calibration + shutter speed Oth et al., Rolling Shutter Camera Calibration, CVPR 2013 Non-central projection under motion Google StreetView Camera Setup Klinger et al., Street View Motion-from-Structurefrom-Motion, ICCV 2013
83 Event Cameras Kim et al., Simultaneous Mosaicing and Tracking with an Event Camera, BMVC 2014
84 Reminder Project presentation in 2 weeks Find team mate / decide topic soon Submit proposal
85 Next class: Features, Tracking / Matching
Camera models and calibration
Camera models and calibration Read tutorial chapter 2 and 3. http://www.cs.unc.edu/~marc/tutorial/ Szeliski s book pp.29-73 Schedule (tentative) 2 # date topic Sep.8 Introduction and geometry 2 Sep.25
More informationProjective geometry for Computer Vision
Department of Computer Science and Engineering IIT Delhi NIT, Rourkela March 27, 2010 Overview Pin-hole camera Why projective geometry? Reconstruction Computer vision geometry: main problems Correspondence
More informationProjective 2D Geometry
Projective D Geometry Multi View Geometry (Spring '08) Projective D Geometry Prof. Kyoung Mu Lee SoEECS, Seoul National University Homogeneous representation of lines and points Projective D Geometry Line
More informationHomogeneous Coordinates. Lecture18: Camera Models. Representation of Line and Point in 2D. Cross Product. Overall scaling is NOT important.
Homogeneous Coordinates Overall scaling is NOT important. CSED44:Introduction to Computer Vision (207F) Lecture8: Camera Models Bohyung Han CSE, POSTECH bhhan@postech.ac.kr (",, ) ()", ), )) ) 0 It is
More informationMore on single-view geometry class 10
More on single-view geometry class 10 Multiple View Geometry Comp 290-089 Marc Pollefeys Multiple View Geometry course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective 2D Geometry Jan.
More informationComputer Vision. 2. Projective Geometry in 3D. Lars Schmidt-Thieme
Computer Vision 2. Projective Geometry in 3D Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim, Germany 1 / 26 Syllabus Mon.
More informationAugmented Reality II - Camera Calibration - Gudrun Klinker May 11, 2004
Augmented Reality II - Camera Calibration - Gudrun Klinker May, 24 Literature Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2. (Section 5,
More informationComputer Vision I - Algorithms and Applications: Multi-View 3D reconstruction
Computer Vision I - Algorithms and Applications: Multi-View 3D reconstruction Carsten Rother 09/12/2013 Computer Vision I: Multi-View 3D reconstruction Roadmap this lecture Computer Vision I: Multi-View
More informationRobot Vision: Projective Geometry
Robot Vision: Projective Geometry Ass.Prof. Friedrich Fraundorfer SS 2018 1 Learning goals Understand homogeneous coordinates Understand points, line, plane parameters and interpret them geometrically
More informationProjective geometry, camera models and calibration
Projective geometry, camera models and calibration Subhashis Banerjee Dept. Computer Science and Engineering IIT Delhi email: suban@cse.iitd.ac.in January 6, 2008 The main problems in computer vision Image
More informationInvariance of l and the Conic Dual to Circular Points C
Invariance of l and the Conic Dual to Circular Points C [ ] A t l = (0, 0, 1) is preserved under H = v iff H is an affinity: w [ ] l H l H A l l v 0 [ t 0 v! = = w w] 0 0 v = 0 1 1 C = diag(1, 1, 0) is
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 253
Index 3D reconstruction, 123 5+1-point algorithm, 274 5-point algorithm, 260 7-point algorithm, 255 8-point algorithm, 253 affine point, 43 affine transformation, 55 affine transformation group, 55 affine
More informationIndex. 3D reconstruction, point algorithm, point algorithm, point algorithm, point algorithm, 263
Index 3D reconstruction, 125 5+1-point algorithm, 284 5-point algorithm, 270 7-point algorithm, 265 8-point algorithm, 263 affine point, 45 affine transformation, 57 affine transformation group, 57 affine
More information3D reconstruction class 11
3D reconstruction class 11 Multiple View Geometry Comp 290-089 Marc Pollefeys Multiple View Geometry course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective 2D Geometry Jan. 14, 16
More informationComputer Vision Projective Geometry and Calibration. Pinhole cameras
Computer Vision Projective Geometry and Calibration Professor Hager http://www.cs.jhu.edu/~hager Jason Corso http://www.cs.jhu.edu/~jcorso. Pinhole cameras Abstract camera model - box with a small hole
More informationVisual Recognition: Image Formation
Visual Recognition: Image Formation Raquel Urtasun TTI Chicago Jan 5, 2012 Raquel Urtasun (TTI-C) Visual Recognition Jan 5, 2012 1 / 61 Today s lecture... Fundamentals of image formation You should know
More informationcalibrated coordinates Linear transformation pixel coordinates
1 calibrated coordinates Linear transformation pixel coordinates 2 Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial
More informationMultiple View Geometry in Computer Vision
Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 More on Single View Geometry Lecture 11 2 In Chapter 5 we introduced projection matrix (which
More informationProjective geometry for 3D Computer Vision
Subhashis Banerjee Computer Science and Engineering IIT Delhi Dec 16, 2015 Overview Pin-hole camera Why projective geometry? Reconstruction Computer vision geometry: main problems Correspondence problem:
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t R 2 3,t 3 Camera 1 Camera
More informationComputer Vision I - Appearance-based Matching and Projective Geometry
Computer Vision I - Appearance-based Matching and Projective Geometry Carsten Rother 05/11/2015 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation
More informationCSE 252B: Computer Vision II
CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribe: Sameer Agarwal LECTURE 1 Image Formation 1.1. The geometry of image formation We begin by considering the process of image formation when a
More informationVision Review: Image Formation. Course web page:
Vision Review: Image Formation Course web page: www.cis.udel.edu/~cer/arv September 10, 2002 Announcements Lecture on Thursday will be about Matlab; next Tuesday will be Image Processing The dates some
More informationAgenda. Rotations. Camera calibration. Homography. Ransac
Agenda Rotations Camera calibration Homography Ransac Geometric Transformations y x Transformation Matrix # DoF Preserves Icon translation rigid (Euclidean) similarity affine projective h I t h R t h sr
More informationUnit 3 Multiple View Geometry
Unit 3 Multiple View Geometry Relations between images of a scene Recovering the cameras Recovering the scene structure http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook1.html 3D structure from images Recover
More information3D Photography: Epipolar geometry
3D Photograph: Epipolar geometr Kalin Kolev, Marc Pollefes Spring 203 http://cvg.ethz.ch/teaching/203spring/3dphoto/ Schedule (tentative) Feb 8 Feb 25 Mar 4 Mar Mar 8 Mar 25 Apr Apr 8 Apr 5 Apr 22 Apr
More informationMultiple View Geometry in Computer Vision
http://www.cs.unc.edu/~marc/ Multiple View Geometry in Computer Vision 2011.09. 2 3 4 5 Visual 3D Modeling from Images 6 AutoStitch http://cs.bath.ac.uk/brown/autostitch/autostitch.html 7 Image Composite
More informationMETR Robotics Tutorial 2 Week 2: Homogeneous Coordinates
METR4202 -- Robotics Tutorial 2 Week 2: Homogeneous Coordinates The objective of this tutorial is to explore homogenous transformations. The MATLAB robotics toolbox developed by Peter Corke might be a
More informationStructure from motion
Structure from motion Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R 1,t 1 R 2,t 2 R 3,t 3 Camera 1 Camera
More informationStructure from Motion
Structure from Motion Outline Bundle Adjustment Ambguities in Reconstruction Affine Factorization Extensions Structure from motion Recover both 3D scene geoemetry and camera positions SLAM: Simultaneous
More informationGeometric camera models and calibration
Geometric camera models and calibration http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 13 Course announcements Homework 3 is out. - Due October
More informationCamera model and calibration
and calibration AVIO tristan.moreau@univ-rennes1.fr Laboratoire de Traitement du Signal et des Images (LTSI) Université de Rennes 1. Mardi 21 janvier 1 AVIO tristan.moreau@univ-rennes1.fr and calibration
More informationMultiple View Geometry in computer vision
Multiple View Geometry in computer vision Chapter 8: More Single View Geometry Olaf Booij Intelligent Systems Lab Amsterdam University of Amsterdam, The Netherlands HZClub 29-02-2008 Overview clubje Part
More informationEpipolar Geometry class 11
Epipolar Geometry class 11 Multiple View Geometry Comp 290-089 Marc Pollefeys Multiple View Geometry course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective 2D Geometry Jan. 14, 16
More informationRobotics - Projective Geometry and Camera model. Matteo Pirotta
Robotics - Projective Geometry and Camera model Matteo Pirotta pirotta@elet.polimi.it Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico di Milano 14 March 2013 Inspired from Simone
More informationAgenda. Rotations. Camera models. Camera calibration. Homographies
Agenda Rotations Camera models Camera calibration Homographies D Rotations R Y = Z r r r r r r r r r Y Z Think of as change of basis where ri = r(i,:) are orthonormal basis vectors r rotated coordinate
More information(Geometric) Camera Calibration
(Geoetric) Caera Calibration CS635 Spring 217 Daniel G. Aliaga Departent of Coputer Science Purdue University Caera Calibration Caeras and CCDs Aberrations Perspective Projection Calibration Caeras First
More informationMultiple View Geometry in Computer Vision
Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 Projective 3D Geometry (Back to Chapter 2) Lecture 6 2 Singular Value Decomposition Given a
More informationCamera Geometry II. COS 429 Princeton University
Camera Geometry II COS 429 Princeton University Outline Projective geometry Vanishing points Application: camera calibration Application: single-view metrology Epipolar geometry Application: stereo correspondence
More informationBIL Computer Vision Apr 16, 2014
BIL 719 - Computer Vision Apr 16, 2014 Binocular Stereo (cont d.), Structure from Motion Aykut Erdem Dept. of Computer Engineering Hacettepe University Slide credit: S. Lazebnik Basic stereo matching algorithm
More informationToday. Stereo (two view) reconstruction. Multiview geometry. Today. Multiview geometry. Computational Photography
Computational Photography Matthias Zwicker University of Bern Fall 2009 Today From 2D to 3D using multiple views Introduction Geometry of two views Stereo matching Other applications Multiview geometry
More informationReminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches
Reminder: Lecture 20: The Eight-Point Algorithm F = -0.00310695-0.0025646 2.96584-0.028094-0.00771621 56.3813 13.1905-29.2007-9999.79 Readings T&V 7.3 and 7.4 Essential/Fundamental Matrix E/F Matrix Summary
More informationComputing F class 13. Multiple View Geometry. Comp Marc Pollefeys
Computing F class 3 Multiple View Geometr Comp 90-089 Marc Pollefes Multiple View Geometr course schedule (subject to change) Jan. 7, 9 Intro & motivation Projective D Geometr Jan. 4, 6 (no class) Projective
More informationPart 0. The Background: Projective Geometry, Transformations and Estimation
Part 0 The Background: Projective Geometry, Transformations and Estimation La reproduction interdite (The Forbidden Reproduction), 1937, René Magritte. Courtesy of Museum Boijmans van Beuningen, Rotterdam.
More informationComputer Vision Projective Geometry and Calibration. Pinhole cameras
Computer Vision Projective Geometry and Calibration Professor Hager http://www.cs.jhu.edu/~hager Jason Corso http://www.cs.jhu.edu/~jcorso. Pinhole cameras Abstract camera model - box with a small hole
More informationOutline. ETN-FPI Training School on Plenoptic Sensing
Outline Introduction Part I: Basics of Mathematical Optimization Linear Least Squares Nonlinear Optimization Part II: Basics of Computer Vision Camera Model Multi-Camera Model Multi-Camera Calibration
More information3D Computer Vision II. Reminder Projective Geometry, Transformations
3D Computer Vision II Reminder Projective Geometry, Transformations Nassir Navab" based on a course given at UNC by Marc Pollefeys & the book Multiple View Geometry by Hartley & Zisserman" October 21,
More informationCamera Model and Calibration
Camera Model and Calibration Lecture-10 Camera Calibration Determine extrinsic and intrinsic parameters of camera Extrinsic 3D location and orientation of camera Intrinsic Focal length The size of the
More informationComputer Vision Lecture 17
Announcements Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics Seminar in the summer semester Current Topics in Computer Vision and Machine Learning Block seminar, presentations in 1 st week
More informationMultiple View Geometry in Computer Vision Second Edition
Multiple View Geometry in Computer Vision Second Edition Richard Hartley Australian National University, Canberra, Australia Andrew Zisserman University of Oxford, UK CAMBRIDGE UNIVERSITY PRESS Contents
More informationCamera Calibration. Schedule. Jesus J Caban. Note: You have until next Monday to let me know. ! Today:! Camera calibration
Camera Calibration Jesus J Caban Schedule! Today:! Camera calibration! Wednesday:! Lecture: Motion & Optical Flow! Monday:! Lecture: Medical Imaging! Final presentations:! Nov 29 th : W. Griffin! Dec 1
More informationPin Hole Cameras & Warp Functions
Pin Hole Cameras & Warp Functions Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Pinhole Camera. Homogenous Coordinates. Planar Warp Functions. Motivation Taken from: http://img.gawkerassets.com/img/18w7i1umpzoa9jpg/original.jpg
More informationCS231A Midterm Review. Friday 5/6/2016
CS231A Midterm Review Friday 5/6/2016 Outline General Logistics Camera Models Non-perspective cameras Calibration Single View Metrology Epipolar Geometry Structure from Motion Active Stereo and Volumetric
More informationStructure from Motion. Prof. Marco Marcon
Structure from Motion Prof. Marco Marcon Summing-up 2 Stereo is the most powerful clue for determining the structure of a scene Another important clue is the relative motion between the scene and (mono)
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x
More informationStructure from Motion CSC 767
Structure from Motion CSC 767 Structure from motion Given a set of corresponding points in two or more images, compute the camera parameters and the 3D point coordinates?? R,t R 2,t 2 R 3,t 3 Camera??
More informationCS201 Computer Vision Camera Geometry
CS201 Computer Vision Camera Geometry John Magee 25 November, 2014 Slides Courtesy of: Diane H. Theriault (deht@bu.edu) Question of the Day: How can we represent the relationships between cameras and the
More informationComputer Vision Lecture 17
Computer Vision Lecture 17 Epipolar Geometry & Stereo Basics 13.01.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Seminar in the summer semester
More informationCS 664 Slides #9 Multi-Camera Geometry. Prof. Dan Huttenlocher Fall 2003
CS 664 Slides #9 Multi-Camera Geometry Prof. Dan Huttenlocher Fall 2003 Pinhole Camera Geometric model of camera projection Image plane I, which rays intersect Camera center C, through which all rays pass
More informationRobust Geometry Estimation from two Images
Robust Geometry Estimation from two Images Carsten Rother 09/12/2016 Computer Vision I: Image Formation Process Roadmap for next four lectures Computer Vision I: Image Formation Process 09/12/2016 2 Appearance-based
More informationCS-9645 Introduction to Computer Vision Techniques Winter 2019
Table of Contents Projective Geometry... 1 Definitions...1 Axioms of Projective Geometry... Ideal Points...3 Geometric Interpretation... 3 Fundamental Transformations of Projective Geometry... 4 The D
More informationLecture 10: Multi-view geometry
Lecture 10: Multi-view geometry Professor Stanford Vision Lab 1 What we will learn today? Review for stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationCamera Projection Models We will introduce different camera projection models that relate the location of an image point to the coordinates of the
Camera Projection Models We will introduce different camera projection models that relate the location of an image point to the coordinates of the corresponding 3D points. The projection models include:
More informationN-View Methods. Diana Mateus, Nassir Navab. Computer Aided Medical Procedures Technische Universität München. 3D Computer Vision II
1/66 N-View Methods Diana Mateus, Nassir Navab Computer Aided Medical Procedures Technische Universität München 3D Computer Vision II Inspired by Slides from Adrien Bartoli 2/66 Outline 1 Structure from
More informationCS231A Course Notes 4: Stereo Systems and Structure from Motion
CS231A Course Notes 4: Stereo Systems and Structure from Motion Kenji Hata and Silvio Savarese 1 Introduction In the previous notes, we covered how adding additional viewpoints of a scene can greatly enhance
More informationInstance-level recognition I. - Camera geometry and image alignment
Reconnaissance d objets et vision artificielle 2011 Instance-level recognition I. - Camera geometry and image alignment Josef Sivic http://www.di.ens.fr/~josef INRIA, WILLOW, ENS/INRIA/CNRS UMR 8548 Laboratoire
More informationCamera calibration. Robotic vision. Ville Kyrki
Camera calibration Robotic vision 19.1.2017 Where are we? Images, imaging Image enhancement Feature extraction and matching Image-based tracking Camera models and calibration Pose estimation Motion analysis
More informationRectification and Distortion Correction
Rectification and Distortion Correction Hagen Spies March 12, 2003 Computer Vision Laboratory Department of Electrical Engineering Linköping University, Sweden Contents Distortion Correction Rectification
More informationMultiple Views Geometry
Multiple Views Geometry Subhashis Banerjee Dept. Computer Science and Engineering IIT Delhi email: suban@cse.iitd.ac.in January 2, 28 Epipolar geometry Fundamental geometric relationship between two perspective
More informationMetric Rectification for Perspective Images of Planes
789139-3 University of California Santa Barbara Department of Electrical and Computer Engineering CS290I Multiple View Geometry in Computer Vision and Computer Graphics Spring 2006 Metric Rectification
More information3D Sensing and Reconstruction Readings: Ch 12: , Ch 13: ,
3D Sensing and Reconstruction Readings: Ch 12: 12.5-6, Ch 13: 13.1-3, 13.9.4 Perspective Geometry Camera Model Stereo Triangulation 3D Reconstruction by Space Carving 3D Shape from X means getting 3D coordinates
More informationLecture 10: Multi view geometry
Lecture 10: Multi view geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Stereo vision Correspondence problem (Problem Set 2 (Q3)) Active stereo vision systems Structure from
More informationStereo Vision. MAN-522 Computer Vision
Stereo Vision MAN-522 Computer Vision What is the goal of stereo vision? The recovery of the 3D structure of a scene using two or more images of the 3D scene, each acquired from a different viewpoint in
More informationPin Hole Cameras & Warp Functions
Pin Hole Cameras & Warp Functions Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Pinhole Camera. Homogenous Coordinates. Planar Warp Functions. Example of SLAM for AR Taken from:
More informationMultiple View Geometry in Computer Vision
Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 Structure Computation Lecture 18 March 22, 2005 2 3D Reconstruction The goal of 3D reconstruction
More informationStructure from Motion
11/18/11 Structure from Motion Computer Vision CS 143, Brown James Hays Many slides adapted from Derek Hoiem, Lana Lazebnik, Silvio Saverese, Steve Seitz, and Martial Hebert This class: structure from
More informationImproving camera calibration
Applying a least squares method to control point measurement 27th May 2005 Master s Thesis in Computing Science, 20 credits Supervisor at CS-UmU: Niclas Börlin Examiner: Per Lindström UmeåUniversity Department
More informationVision par ordinateur
Epipolar geometry π Vision par ordinateur Underlying structure in set of matches for rigid scenes l T 1 l 2 C1 m1 l1 e1 M L2 L1 e2 Géométrie épipolaire Fundamental matrix (x rank 2 matrix) m2 C2 l2 Frédéric
More informationPart I: Single and Two View Geometry Internal camera parameters
!! 43 1!???? Imaging eometry Multiple View eometry Perspective projection Richard Hartley Andrew isserman O p y VPR June 1999 where image plane This can be written as a linear mapping between homogeneous
More informationCritical Motion Sequences for the Self-Calibration of Cameras and Stereo Systems with Variable Focal Length
Critical Motion Sequences for the Self-Calibration of Cameras and Stereo Systems with Variable Focal Length Peter F Sturm Computational Vision Group, Department of Computer Science The University of Reading,
More information1 Projective Geometry
CIS8, Machine Perception Review Problem - SPRING 26 Instructions. All coordinate systems are right handed. Projective Geometry Figure : Facade rectification. I took an image of a rectangular object, and
More informationRigid Body Motion and Image Formation. Jana Kosecka, CS 482
Rigid Body Motion and Image Formation Jana Kosecka, CS 482 A free vector is defined by a pair of points : Coordinates of the vector : 1 3D Rotation of Points Euler angles Rotation Matrices in 3D 3 by 3
More informationCSE 252B: Computer Vision II
CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribe : Martin Stiaszny and Dana Qu LECTURE 0 Camera Calibration 0.. Introduction Just like the mythical frictionless plane, in real life we will
More informationC280, Computer Vision
C280, Computer Vision Prof. Trevor Darrell trevor@eecs.berkeley.edu Lecture 11: Structure from Motion Roadmap Previous: Image formation, filtering, local features, (Texture) Tues: Feature-based Alignment
More informationComputer Vision. Marc Pollefeys Vittorio Ferrari Luc Van Gool
Computer Vision Marc Pollefeys Vittorio Ferrari Luc Van Gool Lecturers Marc Pollefeys Vittorio Ferrari Luc Van Gool Teaching assistants Olivier Saurer Petri Tanskanen Administrivia Classes: Wed, 13-16,
More informationCamera model and multiple view geometry
Chapter Camera model and multiple view geometry Before discussing how D information can be obtained from images it is important to know how images are formed First the camera model is introduced and then
More informationLecture 9: Epipolar Geometry
Lecture 9: Epipolar Geometry Professor Fei Fei Li Stanford Vision Lab 1 What we will learn today? Why is stereo useful? Epipolar constraints Essential and fundamental matrix Estimating F (Problem Set 2
More informationA Stratified Approach for Camera Calibration Using Spheres
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. Y, MONTH YEAR 1 A Stratified Approach for Camera Calibration Using Spheres Kwan-Yee K. Wong, Member, IEEE, Guoqiang Zhang, Student-Member, IEEE and Zhihu
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationCamera Calibration from the Quasi-affine Invariance of Two Parallel Circles
Camera Calibration from the Quasi-affine Invariance of Two Parallel Circles Yihong Wu, Haijiang Zhu, Zhanyi Hu, and Fuchao Wu National Laboratory of Pattern Recognition, Institute of Automation, Chinese
More informationCS 664 Structure and Motion. Daniel Huttenlocher
CS 664 Structure and Motion Daniel Huttenlocher Determining 3D Structure Consider set of 3D points X j seen by set of cameras with projection matrices P i Given only image coordinates x ij of each point
More informationWeek 2: Two-View Geometry. Padua Summer 08 Frank Dellaert
Week 2: Two-View Geometry Padua Summer 08 Frank Dellaert Mosaicking Outline 2D Transformation Hierarchy RANSAC Triangulation of 3D Points Cameras Triangulation via SVD Automatic Correspondence Essential
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Shiv Ram Dubey, IIIT Sri City Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X
More informationAuto-calibration. Computer Vision II CSE 252B
Auto-calibration Computer Vision II CSE 252B 2D Affine Rectification Solve for planar projective transformation that maps line (back) to line at infinity Solve as a Householder matrix Euclidean Projective
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationTwo-view geometry Computer Vision Spring 2018, Lecture 10
Two-view geometry http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 10 Course announcements Homework 2 is due on February 23 rd. - Any questions about the homework? - How many of
More informationStereo and Epipolar geometry
Previously Image Primitives (feature points, lines, contours) Today: Stereo and Epipolar geometry How to match primitives between two (multiple) views) Goals: 3D reconstruction, recognition Jana Kosecka
More informationSINGLE VIEW GEOMETRY AND SOME APPLICATIONS
SINGLE VIEW GEOMERY AND SOME APPLICAIONS hank you for the slides. hey come mostly from the following sources. Marc Pollefeys U. on North Carolina Daniel DeMenthon U. of Maryland Alexei Efros CMU Action
More informationCOSC579: Scene Geometry. Jeremy Bolton, PhD Assistant Teaching Professor
COSC579: Scene Geometry Jeremy Bolton, PhD Assistant Teaching Professor Overview Linear Algebra Review Homogeneous vs non-homogeneous representations Projections and Transformations Scene Geometry The
More informationA Summary of Projective Geometry
A Summary of Projective Geometry Copyright 22 Acuity Technologies Inc. In the last years a unified approach to creating D models from multiple images has been developed by Beardsley[],Hartley[4,5,9],Torr[,6]
More information