Single-shot Extrinsic Calibration of a Generically Configured RGB-D Camera Rig from Scene Constraints Jiaolong Yang*, Yuchao Dai^, Hongdong Li^, Henry Gardner^ and Yunde Jia* *Beijing Institute of Technology (BIT), ^The Australian National University (ANU)
Introduction Calibration are required to fuse the color and depth images for AR/MR applications. Intrinsic parameters are often fixed while extrinsic parameters can be subject to change. Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 2
Related Work Calibration with checkerboard pattern Q. Zhang and R. Pless IROS 04 D. Herrera et al. PAMI 12 C. Zhang and Z. Zhang. ICME 11
Related Work Calibration with checkerboard pattern (single shot) A. Geiger et al. ICRA 12 Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 4
Single-shot Calibration from Scene Constraints Our target: Single-shot verus multi-shot Scene constraints verus calibration patterns The principles: Colour image and depth map measure the scene in different modalities from different positions Scene constraints are invariant to view and modalities Single-shot with scene constraints provide enough information for extrinsic calibration Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 5
Single-shot Calibration from Scene Constraints Scene constraints: Distance: known distance, distance equivalency Angle: parallel, orthogonal, known angle Evaluation: Inverse projection Scene knowledge measurement Scene knowledge discrepancy Benefits: Hand held camera application Post processing / permit varying extrinsic parameters Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 6
Scene Constraints Evaluation Scene Constraints (Ground Truth) Measurements with Poor Calibration Measurements with Good Calibration Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 7
Single-shot Extrinsic Calibration Θ [0.1, -0.1, 1.3] 0.9m [0.2, 0.4, 0.5] Inverse Projection Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 8
Single-shot Extrinsic Calibration Inverse projection: from color image to 3D 1 ( d, d, d x ) ( ) ( c, c i yi zi = g Θ ui vi ) Infer 3D position for 2D colour image pixel Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 9
Scene constraints evaluation Given a we can evaluate the discrepancy between a scene constraint and its measurement. Known distance constraint: Distance equivalency constraint: Angular constraints Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 10
Single-shot Extrinsic Calibration Our goal is to find the optimal transformation minimizing the total geometric error: Non-linear Minimization: Levenberg-Marquardt algorithm (numerical gradients) Nelder-Mead simplex downhill on SE(3) manifold (gradient free) Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 11
Initialization Build 3D points from the color camera by Single View Reconstruction (SVR) with Scene Constraints Find initial transformation between color and depth point clouds by Pointset Registration Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 12
Initialization: Single View Reconstruction (SVR) The corresponding 3D point of lies on the ray with direction with unknown projective depth to determine. Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 13
Initialization: Single View Reconstruction (SVR) The known distance (see paper for more constraints and minimal configuration) between two 3D points gives constraint on the projective depth. Semi-Definite Programming (SDP) problem Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 14
Initialization: Pointset registration Two point clouds:, Iterative Closest Point (ICP) Go-ICP(Globally-optimal ICP) Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 15
Experiments 1) Synthesized scene Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 16
Experiments 1) Synthetic data: two cylinders Quantitative results Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 17
Experiments 1) Synthetic data: two cylinders Qualitative results Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 18
Experiments 2) Real-world scene: three A4 paper Single view reconstruction Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 19
Experiments 2) Real-world scene: three A4 paper Quantitative results Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 20
Applications Warping of real-world scene: (after calibration with three sheets of A4 paper) Qualitative results Herrera et al 2012 Ours Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 21
Applications AR application An augmented reality demonstration Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 22
Conclusions: Single-shot extrinsic calibration of a generically configured RGB-D camera rig from scene constraints Single view reconstruction + pointset registration for initialization Without specifically designed patterns Correspondence-free Geometric error minimization Less human intervention Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 23
Future Works Efficient implementation Extrinsic calibration for hand held camera video sequences Deal with large displacement
Thanks Support: ARC LP--- with Microsoft Research Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 25
Questions? Single-shot Extrinsic Calibration of a RGB-D Camera Rig from Scene Constraints 26