The Geometry of Multiple Images The Laws That Govern the Formation of Multiple Images of a Scene and Some of Thcir Applications

Transcription

1 The Geometry of Multiple Images The Laws That Govern the Formation of Multiple Images of a Scene and Some of Thcir Applications Olivier Faugeras QUC1ng-Tuan Luong with contributions from Theo Papadopoulo The MIT Press Cambridge, Massachusetts London, England

2 Contents Preface Notation 1 A tour into multiple image geometry 1.1 Multiple image geometry and three-dimensional vision 1.2 Projective geometry " D arid 3-D. 1.4 Calibrated and uncalibrated capabilities The plane-to-image hornography as a projective transforrnation 1.6 Affine description of the projection. 1.7 Structure and motion. 1.8 The homography between two images of a plane 1.9 Stationary cameras The epipolar constraint between corresponding points 1.11 The Fundamental matrix Computing thc Fundamental matrix Planar homographlos and the Fundamental matrix 1.14 A st.ratified approach to reconstruction Projective reconstruction Reconstruction is not always nec:essary 1.17 Affine reconstruc:tion Euc:lidean reconstruction The geometry of throe images 1.20 The Trifocal tensor. xiii xix

3 vi Contents 1.21 Computing the Trifocal tensor Reconstruction from N images Self-calibration of a moving carnera using the absolute conic 1.24 From affine to Euclidean From projective to Euclidean References and further reading 2 Projeetive, affine and Euclidean geometries 2.1 Motivations for the approach and overview Projective spaces: basic definitions Projeetive geometry Affine geometry Euclidean geometry 2.2 Affine spaces and affine geometry Definition of an affine space and an affine basis Affine morphisrns, affine group Change of affine basis Affine subspaces, parallelism 2.3 Euclidean spaces and Euclidean geomctry Euelidean spaces, rigid displacements, similarities The isotropic eone. 2.4 Projective spaces and projective geometry Basic definitions Projective bases, projective morphisms, homographies Projective subspaces. 2.5 Affine and projective geometry Projective completion of an affine space Affine and projective bases Affine subspace X n of a projective space lpm Relation between PDJ(X) and AQ(X). 2.6 More projective geometry Cross-ratlos Duality Conics, quadrics and their duals 2.7 Projective, affine and Euclidean geometry Relation between pc.qet) and S(X) Angles as cross-ratlos 2.8 Summary. 2.9 References and further reading ')

4 Contents vii 3 Exterior and double or Grassmann-Cayley algebras Definition of the exterior algebra of the join First definitions: The join operator Properties of the join operator Plüeker relations Derivation of the Plüeker relations The example of 3D lines: II The example of 3D planes: II The meet operator: The Grassmann-Cayley algebra Definition of the meet Some planar examples Some 3D examples Duality and the Hodge operator Duality The example of 3D lines: III The Hodge operator The example of 2D lines: II The example of 3D planes: III The exarnple of 3D lines: IV Summary and eonclusion References and further reading One camera The Projeetive model The pinhole camera The projeetion matrix The inverse projeetion matrix Viewing a plane in spaee: The single view hornography Projeetion of a line The affine model: The case of perspective projeetion The projeetion rnatrix The inverse perspeetive projeetion matrix Vanishing points and lines The Euelidean model: The case of perspective projeetion Intrinsic and Extrinsic parameters The absolute eonie arid the intrinsie parameters The affine and Euelidean models: The ease of parallel projeetion Orthographie, weak perspective, para-perspeetive projections The general model: The affine projection matrix Euclidean interpretation of the parallel projeetion Departures from the pinhole model: Nonlinear distortion Nonlinear distort.ion of the pinhole model Distortion eorrection within a projeetive model.. 234

5 viii Contents 4.6 Calibration teehniques Coordinates-bascd methods Using single view homographies. 4.7 Summary and discussion Rcferences and further reading Two views: The Fundamental matrix Configurations with no parallax The correspondence between the two irnages of a plane Identical optical centers: Application to mosaicing The Fundamental matrix Geometry: Tho cpipolar constraint Algebra: The bilinear constraint The epipolar hornography Relations between the Fundamental matrix and planar homographies The S-matrix and the intrinsic planes Perspective projeetion The affine case ö.3.2 The Euelidean ease: Epipolar geometry The Essential rnatrix Structure and rnotion parameters for a plane Sorne partieular cases Parallel projection Affine epipolar geometry Cyclopean and affine viewing The Euclidean case Arnbiguity and the critical surface The critical surfaces The quadratic transforrnation bctwccn two ambiguous images The planar case G Summary References and further reading Estimating the Fundamental matrix Linear methods An unportant normalization proeedure The basic algorithm Enforcing the rank constraint by approximation Enforcing the rank constraint by parameterization Parameterizing by the epipolar hornography Computing the Jacobian of the pararneterization Choosing the best rnap 325

6 Contents ix 6.3 The distance minimization approach The distance to epipolar lines The Gradient criterion arid an interpretation as a distance Thc "optimal" method Robust Methode M-Estimators Monte-Carlo methods An example of Fundamental matrix estimation with comparison Computing the uncertainty of t.he Fundamental matrix Thc case of an explicit function The case of an implicit function The error function is a sum of squares The hyper-ellipsoid of uncertainty The case of the Fundamental matrix Sorne applications of the computation of A F Unccrtainty of the epipoles Epipolar Band References and further reading Stratification of binocular stereo and applications Canonical representations of two views Projective stratum The projection rnatrices Projective reconstruction Dealing with real correspondences Planar parallax Image rectification Application to obstacle det.ection Applic:ation to image based renclering from two views Affine stratum The projec:tion matric:es Affine reconstruc:tion Affine parallax Estimating H ov Application to affine measurernents Euclidean stratum The projection matrices Euclidean reconstruction Euc:lidean parallax Recovery of the intrinsic parameters Using knowledge about the world: Point c:oordinates Summary Referenccs and further reading

7 x Contents -~ Three views: The trifocal geometry The geometry of three views from the viewpoint of two Transfer Trifocal geometry Optical centers aligned The Trifocal tensors Geometrie derivation of the Trifocal tensors The six intrinsic planar morphisms Changing the reference view Properties of the Trifocal matrices Gi' Relation with planar homographies Prediction revisited Prediction in the Trifocal plane Optical centers aligned Constraints satisfied by the tensors Rank and epipolar constraints The 27 axes constraints The extended rank constraints Constraints that characterize the Trifocal tensor The Affine case The Euclidcan case Computing the directions of the translation vectors and the rotation matrices Computing the ratio of the norms of the translation vectors Affine cameras X1 Projective setting Eudidean setting Summary and Conclusion Perspective projection matrices, Fundamental matrices and Trifocal tensors Transfer References anel further reading Determining the Trifocal tensor The linear algorithm Normalization again! The basic algorithm Discussion Some results Parameterizing the Trifocal tensor The parameterization by projection matrices The six-point parameterization The tensorial parameterization

8 Contents Xl The minimal one-to-one parameterization 9.3 Imposing the constraints Projecting by parameterizing Projecting using the algebraic constraints Some results. 9.4 A note about the "change of view" operar.ion. 9.5 Nonlinear methods The nonlinear scheme A note about the geometrie criterion Results. 9.6 References and further reading Stratification of ti 2 3 views and appiications Canonical representations of n views Projeetive stratum Beyond the Fundamental matrix and the Trifocal tensor The projection matrices: Three views The projection rnatrices: An arbitrary nurnber of views Affine and Euclidean strata Stereo rigs Affine calibration Euclidean calibration References and further reading Self-caIibration of a moving camera: From affine or projective caiibration to full EucIidean caiibration From affine to Euclidean Theoretical analysis Practical computation A numerical example Application to panoramic mosaicing From projective to Euclidean The rigidity constraints: Algebraic: formulations using the Essential matrix The Kruppa equations: A geometrie interpretation of the rigidity constraint Using two rigid displacernents of a carnera: A method for self-calibration Computing the intrinsic parameters using the Kruppa equations Rerovering the focal lengths for two views Solving the Kruppa equations for three views Nonlinear optimization to accumulate the Kruppa equations for ti > 3 views: The "Kruppa" method 563

9 xii Contents 11.4 Computing the Euclidean canonical form The affine camera case The general forrnulation in the perspective case Computing all the Euclidean parameters Simultaneous cornputation of motion and intrinsic parameters: The "Epipolar/Motion" method Global optimization on structure, motion, and calibration parameters More applications Degeneracies in self-calibration G The spurious absolute conics lie in the real plane at infinity G Degeneracies of the Kruppa equations Discussion References and further reading G89 A Appendix A.1 Solution of min ; /IAxl/2 subject to A.2 A note about rank-2 matrices References Index