ROBUST ESTIMATION TECHNIQUES IN COMPUTER VISION Half-day tutorial at ECCV 14, September 7 Olof Enqvist! Fredrik Kahl! Richard Hartley!
Robust Optimization Techniques in Computer Vision! Olof Enqvist! Chalmers University of Technology! Fredrik Kahl! Chalmers University of Technology! Lund University! Richard Hartley! Australian National University! NICTA!
Approximate Methods - Fast approximate methods and outlier rejection schemes!
+1! -1! +1! -1! A Simple Idea!
Approximate Inlier Maximization! 0! For every point - Compute angle intersections - Sort the angles - Compute maximum inliers 0! -1! +1! 1! -1! +1! 0! 1!
Fast Outlier Rejection!
Useful Observation! Given a solution with N inliers within -error. Then, there exists an N inlier solution with one error-free correspondence and others within -error
Upper Bound on Inliers! 0! +1! For every point - Propagate error to other points - Compute angle intersections - Sort the angles - Compute maximum inliers -1! 0! 1! +1! 2! -1! 1!
Fast Outlier Rejection! Given lower bound L on number of inliers (i) Assume one correspondence is an inlier (ii) Compute upper bound U with zero error on this correspondence and other correspondences within -error If U < L then contradiction: Correspondence guaranteed to be outlier!
Fast Outlier Rejection!
! City-Scale Localization Find corresponding points! up to 99% outliers! L. Svärm, O. Enqvist, M. Oskarsson, F. Kahl,! Accurate Localization and Pose Estimation for Large 3D Models, CVPR 2014.!
! Are we doing something wrong?
The Camera Pose Problem
The Camera Pose Problem
Two Key Ideas! 1. Simplified model 2. Fast outlier rejection
Simplified Model! Assumption: - Gravity vector of camera and 3D model known
Fast Outlier Rejection! Observation: Given a solution with N inliers within -error * * * * * * * * * * * * * * * Then, there exists an N inlier solution with one error-free correspondence and others within -error
Minkowski sum of two sets of vectors A and B is the set Minkowski!
Error propagation via Minkowski differences: Error Propagation!
Registration Problem! Assumption: - Lower and upper bounds of height
Error propagation via Minkowski differences: Error propagation!
Fast Outlier Rejection! Algorithm: (i) Assume one correspondence is an inlier (ii) Perform error propagation with Minkowski differences (iii) Compute a solution with zero error on this inlier correspondence (iv) Contradiction?
Algorithm! 1. Compute a solution with the approximate method 2. Perform outlier rejection 3. Optimal estimation (optional)
Application: City-Scale Localization! up to 99 % outliers!
Application: City-Scale Localization! Sattler et al! 2012! Sattler et al! 2011! Sattler et al! 2011! Li et al! 2010! Choudhary et al! 2012! Li et al! 2012! Dubrovnik!
Application: City-Scale Localization!
! Other Applications Inter-modal registration Multiview geometry E. Ask, O. Enqvist, L. Svärm, F. Kahl, G. Lippolis, Tractable and Reliable Registration of 2D Point Sets, ECCV 2014.! O. Enqvist, E. Ask, F. Kahl, K. Åström, Robust Fitting for Multiple View Geometry, ECCV 2012.!
Branch and Bound - global optimization method by iteratively branching and bounding!
Outline! Introduction to branch & bound Registration and geometric matching problems Rotation space search Applications
Model: Measurements:
Model: Measurements:
Model: Measurements:
Model: Measurements: - unknown parameters - source points - target points Inlier if
Find transformation that minimizes the number of outliers
2D-2D Registration! Transformations: Euclidean Parametrization: Residual error: Domain: T. Breuel, Implementation techniques for geometric! branch-and-bound matching methods, CVIU 2003.!
2D-2D Registration! Lower bound on the number of outliers. Consider translation only. Suppose there exists in rectangle such that
2D-2D Registration! Lower bound. Now also with rotation uncertainty: Suppose there exists in cube such that, then
Branch & Bound Algorithm! 1. Initialize a queue with a rectangle domain 2. Execute until no more rectangles - Choose rectangle with best lower bound - Perform branching - Perform bounding on each branch - Update best upper bound - Remove rectangles with lower bound above best upper bound
2D-2D Registration!
2D-2D Registration! Discussion: Empirical performance often good, though worst-case exponential Can be done optimally in Approximate methods in
2D-2D Registration! Related work: Polynomial-time algorithms for truncated L1 and L2 norms E. Ask, O. Enqvist, L. Svärm, F. Kahl, G. Lippolis, Tractable and Reliable Registration of 2D Point Sets, ECCV 2014.!
Branch and Bound on Rotation Space!
Epipolar Geometry! Encodes relative displacement between two cameras - Rotation - Translation - 5 degrees of freedom!
Epipolar Geometry! Use angular reprojection error: α Find transformation that minimizes the number of outliers:
3D Rotation Estimation! Consider simpler problem first. Transformations: Rotation Residual error: Domain: R. Hartley, F. Kahl, Global Optimization through! Rotation Space Search, IJCV 2009.!
Isometry of Rotations and Quaternions! Angle between two quaternions is half the angle between the corresponding rotations, defined by All rotations within a deltaneighbourhood of a reference rotation form a circle on the quaternion sphere.
Angle-axis representation of Rotations! Flatten out the meridians (longitude lines)! Azimuthal Equidistant Projection! Rotations are represented by a ball of radius pi in 3- dimensional space.!
Subdividing and testing rotation space!
3D Rotation Estimation! Lower bound. Consider a cube in angle-axis representation with half side-length : Suppose there exists in cube such that :
Applications of 3D Rotation Estimation! Vanishing point detection J.-C. Bazin, Y. Seo, C. Demonceaux, P. Vasseur, K. Ikeuchi, I. Kweon, M. Pollefeys,! Globally Optimal Line Clustering and Vanishing Point Estimation, CVPR 2012.!
! Applications of 3D Rotation Estimation Panorama stitching J.-C. Bazin, Y. Seo, M. Pollefeys,! Globally Optimal Consensus Set Maximization through Rotation Search, CVPR 2012! J.-C. Bazin, Y. Seo, R. Hartley, M. Pollefeys,! Globally Optimal Inlier Set Maximization with Unknown Rottion and Focal Length, ECCV 2014!
Applications of 3D Rotation Estimation! As a subroutine for 3D-3D registration J. Yang, H. Li, Y. Jia,! Go-ICP: Solving 3D Registration Efficiently and Globally Optimally, ICCV 2013.! A. P. Bustos, T.-J. Chin, D. Suter,! Fast Rotation Search with Stereographic Projections for 3D Registration, CVPR 2014.!
As a subroutine for two-view epipolar geometry Applications of 3D Rotation Estimation! O. Enqvist, F. Kahl,! Two-View Geometry Estimation with Outliers, BMVC 2009.! J. Yang, H. Li, Y. Jia,! Optimal Essential Matrix Estimation via Inlier-Set Maximization, ECCV 2014.!
Conclusions! - Branch-and-bound practical for small dimensions only - Good bounding functions important - Worst-case performance exponential!