Para-catadioptric Camera Auto Calibration from Epipolar Geometry Branislav Mičušík and Tomáš Pajdla Center for Machine Perception http://cmp.felk.cvut.cz Department of Cybernetics Faculty of Electrical Engeneering Czech Technical University in Prague
Motivation automatic 3D metric reconstruction from two uncalibrated para-catadioptric images 2/15 PCD camera
Motivation automatic 3D metric reconstruction from two uncalibrated para-catadioptric images 2/15 PCD camera
Contribution Para-catadioptric (PCD) camera auto calibration from epipolar geometry 3/15
Contribution Para-catadioptric (PCD) camera auto calibration from epipolar geometry 3/15 Solution for F and the mirror parameter a by solving Polynomial Eigenvalue Problem (D 1 + ad 2 +... + a 4 D 5 ) f = 0
Contribution Para-catadioptric (PCD) camera auto calibration from epipolar geometry 3/15 Solution for F and the mirror parameter a by solving Polynomial Eigenvalue Problem (D 1 + ad 2 +... + a 4 D 5 ) f = 0 ransac with 9-point correspondences possible WBS RANSAC 9 pts (Matas et al BMVC 2002)
Contribution Para-catadioptric (PCD) camera auto calibration from epipolar geometry 3/15 Solution for F and the mirror parameter a by solving Polynomial Eigenvalue Problem (D 1 + ad 2 +... + a 4 D 5 ) f = 0 ransac with 9-point correspondences possible WBS RANSAC 9 pts (Matas et al BMVC 2002) linearization of projection model avoided
Previous work 4/15 Geyer&Daniilidis ICCV 2003 Encoding a PCD camera model in a bilinear form by lifting coordinates to 4 dimensional space 15 pts ransac Micusik&Pajdla CVPR 2003 Estimation of omnidirectional camera model from EG linearization of projection model (no linearization for PCD) Fitzgibbon CVPR 2001 Polynomial Eigenvalue Problem for simultaneous estimation of small radial distortion & EG Kang CVPR 2000 Catadioptric camera self-calibration calibration by bundle adjustment on good correspondences
Para-catadioptric camera Para-catadioptric camera = parabolic mirror + orthographic camera 5/15 optical axis PSfrag replacements π r θ u p Assumption: camera projection is orthographic & parallel to opt. axis central camera
Para-catadioptric camera model 6/15 y F z x p u, v, p expressed in Cartesian coordinate system g replacements π u v u p = u v a 2 r 2 2a 2 au 2 a v a 2 r 2 Coordinate system of the PCD camera. r = u 2 + v 2 ag replacements u π v The coordinate system in the pre-calibrated image.
u PCD camera auto-calibration y x u : (u = A u + t ) 7/15 Digitization Pre calibration u : (u = 1 ρ R 1 u ) Calibration from EG p = ( u, v, a2 r 2 2a )
Epipolar Geometry 8/15 PSfrag replacements F 1 F 2 p 1 X p 2 π u 1 π u 2 central omnidirectional cameras possess EG (Svoboda et al ECCV 1998) p 2 F p 1 = 0
Theory of PCD camera calibration 9/15 3D vector p = ( u v a 2 r 2 2a ) PSfrag replacements F y z x p π u v u
Theory of PCD camera calibration 9/15 3D vector p = ( u v a 2 r 2 2a ) PSfrag replacements F y z x p Epipolar constraint for rays π u v u ag replacements F 1 F 2 p 1 X p 2 p 2 Fp 1 = 0 π u 1 π u 2 ( u 2 v 2 a 2 r 2 2 2a ) ( F u 1 v 1 a 2 r 2 1 2a ) = 0
Theory of PCD camera calibration 9/15 3D vector p = ( u v a 2 r 2 2a ) PSfrag replacements F y z x p Epipolar constraint for rays π u v u ag replacements F 1 F 2 p 1 X p 2 p 2 Fp 1 = 0 π u 1 π u 2 ( u 2 v 2 a 2 r 2 2 2a ) ( F u 1 v 1 a 2 r 2 1 2a ) = 0 leads to the Polynomial Eigenvalue Problem (PEP) (D 1 + ad 2 +... + a 4 D 5 )f = 0, where D i R 9 9 are known and f = [ F 11 F 12 F 13 F 21... F 33 ]. - PEP can be solved by Matlab using polyeig details 1
Results of PCD camera auto-calibration the calibrated camera, i.e. the parameters of a PCD camera model: a, A, t p = ( u v a 2 r 2 2a ), 10/15
Results of PCD camera auto-calibration the calibrated camera, i.e. the parameters of a PCD camera model: a, A, t p = ( u v a 2 r 2 2a ), 10/15 the essential matrix E = [t] R,
Results of PCD camera auto-calibration the calibrated camera, i.e. the parameters of a PCD camera model: a, A, t p = ( u v a 2 r 2 2a ), 10/15 the essential matrix E = [t] R, correct point correspondences (inliers)
Finding correspondences 11/15 Pair of images with regions detected by Matas & Chum & Urban & Pajdla BMVC 2002 Tentative correspondences using Inliers satisfying epipolar geometry similarity (Matas et al BMVC 2002) (Micusik & Pajdla ACCV 2004) (many outliers) (inliers/outliers selected)
EXPERIMENTS
Trajectory estimation PCD camera mounted on a turntable and rotated along a circle 13/15 correctly recovered positions and orientations of cameras details 1 2
3D reconstruction 14/15
Main contribution: Conclusion 15/15 Epipolar geometry estimation and para-catadioptric camera calibration and correspondences validation by the autocalibration method based on ransac estimation technique. Good initial estimate for a bundle adjustment. Experiments show: The method designed for central PCD cameras can be used for real (slightly non-central) PCD cameras to solve the correspondence problem and obtain initial estimate of camera positions. Stable 3D metric reconstruction just from two images.
Solution of the PEP (back) (D 1 + ad 2 +... + a 4 D 5 )f = 0 36 (9 4) solutions of a, f many of them are zero, infinite or complex cements usually 1-3 solutions remain choose the pair which has smallest angular error (Oliensis PAMI 2002) τ X p 1 φ1 n ˆp 1 ˆp 2 ( p 2 ɛ(p 1, p 2, F) = min sin 2 φ 1 + sin 2 ) φ 2 n φ 2 C 2 = min n ( n.p1 2 + n.p 2 2) C 1 PEP can be easily incorporated in 9-point ransac
Real non-central PCD camera (back) pi pi F F u u p p ideal real
Non-central vs. Central model (back) Central model (angles are wrong) Non-central model (angles are correct)
optical axis π r u p ents θ
F x y z p rag replacements π u v u
ts u v π
x y Digitization Pre calibration Calibration from EG
F 1 F 2 p 1 X p 2 π u 1 π u 2
F x y z p rag replacements π u v u
F 1 F 2 p 1 X p 2 π u 1 π u 2
X n τ p 2 ˆp 1 ˆp 2 φ 2 p 1 φ1 C 2 C 1
F p pi u
F p pi u