Multiple View Geometry in Computer Vision Second Edition

Transcription

1 Multiple View Geometry in Computer Vision Second Edition Richard Hartley Australian National University, Canberra, Australia Andrew Zisserman University of Oxford, UK CAMBRIDGE UNIVERSITY PRESS

2 Contents Foreword Preface page xi xiii 1 Introduction - a Tour of Multiple View Geometry Introduction - the ubiquitous projective geometry Camera projections Reconstruction from more than one view Three-view geometry Four view geometry and n-view reconstruction Transfer Euclidean reconstruction Auto-calibration The reward 1: 3D graphical models The reward II: video augmentation 19 PART 0: The Background: Projective Geometry, Transformations and Estimation 23 Outline 24 2 Projective Geometry and Transformations of 2D Planar geometry The 2D projective plane Projective transformations A hierarchy of transformations The projective geometry of ID Topology of the projective plane Recovery of affine and metric properties from images More properties of conies Fixed points and lines Closure 62 3 Projective Geometry and Transformations of 3D Points and projective transformations Representing and transforming planes, lines and quadrics 66 v

3 VI Contents 3.3 Twisted cubics The hierarchy of transformations The plane at infinity The absolute conic The absolute dual quadric Closure 85 4 Estimation - 2D Projective Transformations The Direct Linear Transformation (DLT) algorithm Different cost functions Statistical cost functions and Maximum Likelihood estimation Transformation invariance and normalization Iterative minimization methods Experimental comparison of the algorithms Robust estimation Automatic computation of a homography Closure Algorithm Evaluation and Error Analysis Bounds on performance Covariance of the estimated transformation Monte Carlo estimation of covariance Closure 150 PART I: Camera Geometry and Single View Geometry 151 Outline Camera Models Finite cameras The projective camera Cameras at infinity Other camera models Closure Computation of the Camera Matrix P * Basic equations Geometric error Restricted camera estimation Radial distortion Closure More Single View Geometry Action of a projective camera on planes, lines, and conies Images of smooth surfaces Action of a projective camera on quadrics The importance of the camera centre Camera calibration and the image of the absolute conic 208

4 Contents Vll 8.6 Vanishing points and vanishing lines Affine 3D measurements and reconstruction Determining camera calibration К from a single view Single view reconstruction The calibrating conic Closure 233 PART II: Two-View Geometry 237 Outline Epipolar Geometry and the Fundamental Matrix Epipolar geometry The fundamental matrix F Fundamental matrices arising from special motions Geometric representation of the fundamental matrix Retrieving the camera matrices The essential matrix Closure D Reconstruction of Cameras and Structure Outline of reconstruction method Reconstruction ambiguity The projective reconstruction theorem Stratified reconstruction Direct reconstruction - using ground truth Closure v * Computation of the Fundamental Matrix F Basic equations The normalized 8-point algorithm The algebraic minimization algorithm Geometric distance Experimental evaluation of the algorithms Automatic computation of F Special cases of F-computation Correspondence of other entities Degeneracies A geometric interpretation of F-computation The envelope of epipolar lines Image rectification Closure Structure Computation Problem statement Linear triangulation methods Geometric error cost function Sampson approximation (first-order geometric correction) 314

5 Vlll Contents 12.5 An optimal solution Probability distribution of the estimated 3D point Line reconstruction Closure Scene planes and homographies Homographies given the plane and vice versa Plane induced homographies given F and image correspondences Computing F given the homography induced by a plane The infinite homography H^ Closure Äffine Epipolar Geometry Affine epipolar geometry The affine fundamental matrix Estimating F A from image point correspondences Triangulation Affine reconstruction Necker reversal and the bas-relief ambiguity Computing the motion Closure 360 PART III: Three-View Geometry 363 Outline The Trifocal Tensor The geometric basis for the trifocal tensor The trifocal tensor and tensor notation Transfer The fundamental matrices for three views Closure Computation of the Trifocal Tensor T Basic equations The normalized linear algorithm The algebraic minimization algorithm Geometric distance Experimental evaluation of the algorithms Automatic computation of T Special cases of T-computation Closure 406 PART IV: N-View Geometry 409 Outline iv-linearities and Multiple View Tensors Bilinear relations Trilinear relations 414

6 Contents 17.3 Quadrilinear relations Intersections of four planes Counting arguments Number of independent equations Choosing equations Closure TV-View Computational Methods Projective reconstruction - bundle adjustment Affine reconstruction - the factorization algorithm Non-rigid factorization Projective factorization Projective reconstruction using planes Reconstruction from sequences Closure Auto-Calibration Introduction Algebraic framework and problem statement Calibration using the absolute dual quadric The Kruppa equations A stratified solution Calibration from rotating cameras Auto-calibration from planes Planar motion Single axis rotation - turntable motion Auto-calibration of a stereo rig Closure Duality Carlsson-Weinshall duality Reduced reconstruction Closure Cheirality Quasi-affine transformations Front and back of a camera Three-dimensional point sets Obtaining a quasi-affine reconstruction Effect of transformations on cheirality Orientation The cheiral inequalities Which points are visible in a third view Which points are in front of which Closure 531 ix

7 X Contents 22 Degenerate Configurations Camera resectioning Degeneracies in two views Carlsson-Weinshall duality Three-view critical configurations Closure 558 PART V : Appendices 561 Appendix 1 Tensor Notation 562 Appendix 2 Gaussian (Normal) and x 2 Distributions 565 Appendix 3 Parameter Estimation 568 Appendix 4 Matrix Properties and Decompositions 578 Appendix 5 Least-squares Minimization 588 Appendix 6 Iterative Estimation Methods 597 Appendix 7 Some Special Plane Projective Transformations 628 Bibliography 634 Index 646