Geometry and Computing

Transcription

1 Geometry and Computing Volume 11 Series editors Herbert Edelsbrunner, Department Computer Science, Durham, NC, USA Leif Kobbelt, RWTH Aachen University, Aachen, Germany Konrad Polthier, AG Mathematical Geometry Processing, Freie Universität Berlin, Berlin, Germany

2 Geometric shapes belong to our every-day life, and modeling and optimization of such forms determine biological and industrial success. Similar to the digital revolution in image processing, which turned digital cameras and online video downloads into consumer products, nowadays we encounter a strong industrial need and scientific research on geometry processing technologies for 3D shapes. Several disciplines are involved, many with their origins in mathematics, revived with computational emphasis within computer science, and motivated by applications in the sciences and engineering. Just to mention one example, the renewed interest in discrete differential geometry is motivated by the need for a theoretical foundation for geometry processing algorithms, which cannot be found in classical differential geometry. Scope: This book series is devoted to new developments in geometry and computation and its applications. It provides a scientific resource library for education, research, and industry. The series constitutes a platform for publication of the latest research in mathematics and computer science on topics in this field. Discrete geometry Computational geometry Differential geometry Discrete differential geometry Computer graphics Geometry processing CAD/CAM Computer-aided geometric design Geometric topology Computational topology Statistical shape analysis Structural molecular biology Shape optimization Geometric data structures Geometric probability Geometric constraint solving Algebraic geometry Graph theory Physics-based modeling Kinematics Symbolic computation Approximation theory Scientific computing Computer vision More information about this series at

3 Wolfgang Förstner Bernhard P. Wrobel Photogrammetric Computer Vision Statistics, Geometry, Orientation and Reconstruction 123

4 Wolfgang Förstner Institut für Geodäsie und Geoinformation Rheinische Friedrich-Wilhelms-Universität Bonn Bonn Germany Bernhard P. Wrobel Institut für Geodäsie Technische Universität Darmstadt Darmstadt Germany ISSN ISSN (electronic) Geometry and Computing ISBN ISBN (ebook) DOI / Library of Congress Control Number: Springer International Publishing Switzerland 2016 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. Printed on acid-free paper This Springer imprint is published by Springer Nature The registered company is Springer International Publishing AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

5 Preface This textbook on Photogrammetric Computer Vision Statistics, Geometry, Orientation and Reconstruction provides a statistical treatment of the geometry of multiple view analysis useful for camera calibration, orientation, and geometric scene reconstruction. The book is the first to offer a joint view of photogrammetry and computer vision, two fields that have converged in recent decades. It is motivated by the need for a conceptually consistent theory aiming at generic solutions for orientation and reconstruction problems. Large parts of the book result from teaching bachelor s and master s courses for students of geodesy within their education in photogrammetry. Most of these courses were simultaneously offered as subjects in the computer science faculty. The book provides algorithms for various problems in geometric computation and in vision metrology, together with mathematical justification and statistical analysis allowing thorough evaluation. The book aims at enabling researchers, software developers, and practitioners in the photogrammetric and GIS industry to design, write, and test their own algorithms and application software using statistically founded concepts to obtain optimal solutions and to realize self-diagnostics within algorithms. This is essential when applying vision techniques in practice. The material of the book can serve as a source for different levels of undergraduate and graduate courses in photogrammetry, computer vision, and computer graphics, and for research and development in statistically based geometric computer vision methods. The sixteen chapters of the book are self-contained, are illustrated with numerous figures, have exercises, and are supported by an appendix and an index. Many of the examples and exercises can be verified or solved using the Matlab routines available on the home page of the book, which also contains solutions to some of the exercises. Acknowledgements: The book gained a lot through the significant support of numerous colleagues. We thank Camillo Ressl and Jochen Meidow for their careful reading of the manuscript and Carl Gerstenecker and Boris Kargoll for their critical review of Part I on statistics. The language proofreading by Silja Weber, Indiana University, is highly appreciated. Thanks for fruitful comments, discussions and support of the accompanying Matlab Software to Martin Drauschke, Susanne Wenzel, Falko Schindler, Thomas Läbe, Richard Steffen, Johannes Schneider, and Lutz Plümer. We thank the American Society for Photogrammetry and Remote Sensing for granting us permission to use material of the sixth edition of the Manual of Photogrammetry. Wolfgang Förstner Bernhard P. Wrobel Bonn, 2016

6 Contents 1 Introduction Tasks for Photogrammetric Computer Vision Modelling in Photogrammetric Computer Vision The Book On Notation Part I Statistics and Estimation 2 Probability Theory and Random Variables Notions of Probability Axiomatic Definition of Probability Random Variables Distributions Moments Quantiles of a Distribution Functions of Random Variables Stochastic Processes Generating Random Numbers Exercises Testing Principles of Hypothesis Testing Testability of an Alternative Hypothesis Common Tests Exercises Estimation Estimation Theory The Linear Gauss Markov Model Gauss Markov Model with Constraints The Nonlinear Gauss Markov Model Datum or Gauge Definitions and Transformations Evaluation Robust Estimation and Outlier Detection Estimation with Implicit Functional Models Methods for Closed Form Estimations Estimation in Autoregressive Models Exercises vii

7 viii Contents Part II Geometry 5 Homogeneous Representations of Points, Lines and Planes Homogeneous Vectors and Matrices Homogeneous Representations of Points and Lines in 2D Homogeneous Representations in IP n Homogeneous Representations of 3D Lines On Plücker Coordinates for Points, Lines and Planes The Principle of Duality Conics and Quadrics Normalizations of Homogeneous Vectors Canonical Elements of Coordinate Systems Exercises Transformations Structure of Projective Collineations Basic Transformations Concatenation and Inversion of Transformations Invariants of Projective Mappings Perspective Collineations Projective Correlations Hierarchy of Projective Transformations and Their Characteristics Normalizations of Transformations Conditioning Exercises Geometric Operations Geometric Operations in 2D Space Geometric Operations in 3D Space Vector and Matrix Representations for Geometric Entities Minimal Solutions for Conics and Transformations Exercises Rotations Rotations in 3D Concatenation of Rotations Relations Between the Representations for Rotations Rotations from Corresponding Vector Pairs Exercises Oriented Projective Geometry Oriented Entities and Constructions Transformation of Oriented Entities Exercises Reasoning with Uncertain Geometric Entities Motivation Representing Uncertain Geometric Elements Propagation of the Uncertainty of Homogeneous Entities Evaluating Statistically Uncertain Relations Closed Form Solutions for Estimating Geometric Entities Iterative Solutions for Maximum Likelihood Estimation Exercises

8 Contents ix Part III Orientation and Reconstruction 11 Overview Scene, Camera, and Image Models The Setup of Orientation, Calibration, and Reconstruction Exercises Geometry and Orientation of the Single Image Geometry of the Single Image Orientation of the Single Image Inverse Perspective and 3D Information from a Single Image Exercises Geometry and Orientation of the Image Pair Motivation The Geometry of the Image Pair Relative Orientation of the Image Pair Triangulation Absolute Orientation and Spatial Similarity Transformation Orientation of the Image Pair and Its Quality Exercises Geometry and Orientation of the Image Triplet Geometry of the Image Triplet Relative Orientation of the Image Triplet Exercises Bundle Adjustment Motivation for Bundle Adjustment and Its Tasks Block Adjustment Sparsity of Matrices, Free Adjustment and Theoretical Precision Self-calibrating Bundle Adjustment Camera Calibration Outlier Detection and Approximate Values View Planning Exercises Surface Reconstruction Introduction Parametric 2 1 /2D Surfaces Models for Reconstructing One-Dimensional Surface Profiles Reconstruction of 2 1 /2D Surfaces from 3D Point Clouds Examples for Surface Reconstruction Exercises Appendix: Basics and Useful Relations from Linear Algebra A.1 Inner Product A.2 Determinant A.3 Inverse, Adjugate, and Cofactor Matrix A.4 Skew Symmetric Matrices A.5 Eigenvalues A.6 Idempotent Matrices A.7 Kronecker Product, vec( ) Operator, vech( ) Operator

9 x Contents A.8 Hadamard Product A.9 Cholesky and QR Decomposition A.10 Singular Value Decomposition A.11 The Null Space and the Column Space of a Matrix A.12 The Pseudo-inverse A.13 Matrix Exponential A.14 Tensor Notation A.15 Variance Propagation of Spectrally Normalized Matrix References Index

10 List of Algorithms 1 Estimation in the linear Gauss Markov model Estimation in the Gauss Markov model with constraints Random sample consensus Robust estimation in the Gauss Helmert model with constraints Reweighting constraints Estimation in the model with constraints between the observations only Algebraic solution for estimating 2D homography from point pairs Direct LS estimation of 2D line from points with isotropic accuracy Direct LS estimation of a 2D point from lines with positional uncertainty Direct LS estimation of the mean of directions with isotropic uncertainty Direct LS estimation of the mean of axes with isotropic uncertainty Direct LS estimation of a rotation from direction pairs Direct LS estimation of similarity Direct LS estimation of 3D line from points Estimation in the Gauss Helmert model with reduced coordinates Algebraic estimation of uncertain projection from six or more points Optimal estimation of a projection matrix from observed image points Decomposition of uncertain projection matrix D circle with given radius determined from its image Base direction and rotation from essential matrix Optimal triangulation from two images and spherical camera model Sequential spatial resections Sequential similarity transformations xi

11 List of Symbols Table 0.1 List of symbols: A M symbol meaning A, B, C names of planes, sets A, B, C homogeneous vectors of planes A 0, A h Euclidean, homogeneous part of the homogeneous coordinate vector A of plane A A X, A Y, A Z homogeneous vectors of coordinate planes, perpendicular to the axes X, Y, and Z B d d-dimensional unit ball in IR d Cov(.,.) covariance operator CR(.,.,.,. ) cross ratio D 6 6 matrix dualizing a line δ(x) Dirac s delta function Diag(. ) diagonal matrix of vector or list of matrices diag(.) vector of diagonal elements of a matrix det(.) =. determinant e [d] i ith basic unit vector in d-space, e.g., e [3] 2 = [0, 1, 0]T D(.) dispersion operator E(.) expectation operator I (L) Plücker matrix of a 3D line I (L) dual Plücker matrix of a 3D line I (s) (L) 2 4 matrix of selected independent rows I n n n unit matrix J = {1,..., j,..., J} set of indices J xy, J x,y Jacobian x/ y J r Jacobian x/ x r, with reduced vector x r of x J s Jacobian x s / x of spherical normalization Hf or H(f) Hessian matrix [ 2 f(x)/( x i x j )] of function f(x) H name of homography H general homography, 2 2, 3 3, or 4 4 matrix l vector of observations in an estimation procedure l, m, n names of 2D lines l, m, n homogeneous vectors of 2D lines L, M, N names of 3D lines L, M, N homogeneous vectors of 3D lines l 0, l h Euclidean, homogeneous part of homogeneous coordinate vector l of 2D line l L 0, L h Euclidean, homogeneous part of homogeneous coordinate vector L of 3D line L l x, l y, L X, L Y, L Z line parameters of coordinate axes L coordinates of 3D line L dual to 3D line L M motion, special homography in 2D or 3D IN set of natural numbers M (µ, Σ) distribution characterized only by mean µ and covariance matrix Σ xiii

12 Table 0.2 List of symbols: N Z symbol meaning N (µ, Σ) normal distribution with mean µ and covariance matrix Σ N normal equation matrix N(.) operator for achieving Frobenius norm 1, for vectors: spherical normalization N e (.) operator for Euclidean normalization of homogeneous vectors N σ (. ) operator for spectral normalization of matrices null(.), null T (.) orthonormal matrix: basis vectors of null space as columns, transpose o origin of coordinate system O(Z), o(z) coordinates of the centre of perspectivity IP n n-dimensional projective space IP n dual n-dimensional projective space I I (X), I I (A) Pi-matrix of a 3D point or a plane I I (X), I I (A) dual Pi-matrix of a 3D point or a plane I I (s) (X), I I (s) (A) 3 4 matrix of selected independent rows r(x a, b) rectangle function in the range [a, b] r xy correlation coefficient of x and y R rotation matrix, correlation matrix IR n n-dimensional Euclidean space over IR IR n \ 0 n-dimensional Euclidean space without origin s(x) step function SL(n) special group of linear transformations with determinant 1 SO(n) special group of orthogonal transformations (rotations) so(n) Lie group of skew matrices S a, S(a), S a, S(a) inhomogeneous, homogeneous skew symmetric matrix depending on a 3-vector S i 3 3 skew symmetric matrix of 3 1 vector e [3] i S (s) (x) 2 3 matrix with two selected independent rows S d unit sphere of dimension d in IR d+1, set of points x IR d+1 with x = 1 σ x standard deviation of x σ xy covariance of x and y Σ xy covariance matrix of x and y T n oriented projective space T n dual oriented projective space W xx weight matrix of parameters x x unknown parameters in an estimation procedure x, y, z names of 2D points x, y, z homogeneous vectors of points in 2D X, Y, Z names of 3D points X, Y, Z homogeneous vectors of points in 3D x 0, x h Euclidean, homogeneous part of the homogeneous coordinate vector x of point x X 0, X h Euclidean, homogeneous part of the homogeneous coordinate vector X of point X

13 Table 0.3 List of symbols: fonts, operators symbol meaning % permille x, µ n 1 inhomogeneous vectors, with indicated size x, µ homogeneous vectors A m n inhomogeneous matrices, with indicated size, generally n m K, P homogeneous matrices λ max(. ) largest eigenvalue (.) entity at infinity, transformation referring to entities at infinity i I = {1,..., I} index and index set (. ) T transpose (.) T transpose of inverse matrix (.) + pseudo-inverse matrix (. ) a approximated vector or matrix within iterative estimation procedure (.) adjugate matrix (.) O cofactor matrix (.) r reduced, minimal vector (.) (s) reduced matrix with selected independent rows. absolute value of scalar, Euclidean norm of a vector, determinant of matrix. Frobenius norm.,. A inner product, e.g., x, y A = x T Ay.,.,. triple product of three 3-vectors, identical to the determinant of their 3 3 matrix [.,.,. ].,.,.,. cross ratio of four numbers operation, defined locally (.) dualizing or Hodge operator x vector perpendicular to x x(p) nabla operator, gradient, Jacobian x/ p (. ) stochastic variable veca vec operator x H (q) stochastic variable x follows distribution H (q) (.) estimated value (. ) true value there exists A B Hadamard product A B Kronecker product intersection operator ( cap ) join operator ( wedge ) = proportional to (vectors, matrices) proportional to (functions) not, antipode of an entity having negative homogeneous coordinates if and only if. = defining equation := assignment a =! b constraint: a should be equal to b, or E(a) = b a = + b, a = + b two elements are equivalent in oriented projective geometry [.,. ] closed interval (.,.] semi-open interval x floor function, largest integer smaller than x x ceiling function, smallest integer larger than x

14 Table 0.4 List of Symbols in Part III (1) abbreviation meaning α parallactic angle between two rays A infinite homography, mapping from plane at infinity to image plane, also called H A = [C, D] design matrix, Jacobian w.r.t. parameters, partitioned for scene coordinates and orientation parameters (A, B, C) principal planes of camera coordinate system, rows of projection matrix P A l (A l ) projection plane to image line l B Jacobian of constraints w.r.t. observations b, B base vector c principal distance (.) coordinate in camera coordinate system c(x) function to derive inhomogeneous from homogeneous coordinates D E number of parameters of observed image feature C 3 3 matrix for conics D T number of parameters for transformation or projection D I number of parameters for scene feature (it) E index set E I T for observed image features f it e (e ), e (e ) epipoles of image pair E epipolar plane E, E tt essential matrix, of images t and t E it matrix for selecting scene points observed in images F, F tt fundamental matrix, of images t and t F i (k i ) scene feature F i with coordinates k i, indices i I F i0 (k i0 ) control scene feature F i0 with coordinates k i0, indices i I 0 f it (l it ) image feature f it with observed coordinates l it, indices (it) E f it projection function for scene feature i and image t g it projection relation for scene feature i and image t G 3, G 4 G 6 d d selection matrix Diag([1 T d, 0]) 6 6 selection matrix Diag({I 3, }) H (H) homography, perspective mapping H infinite homography, mapping from plane at infinity to image plane, also called A H A homography, mapping plane A in object space to image plane H (x H ) principal point H matrix of constraints for fixing gauge H g flight height over ground (.) coordinate in image coordinate system {1,..., i,...i} = I index set for scene features (i, j) discrete image coordinates, unit pixels l (X ), l (x ) epipolar lines of image pair, depending on scene or on image point κ rotation angle around Z-axis of camera system, gear angle κ 1, κ 2 principal curvatures of surface k vector of unknown coordinates K 1, K 2 principal points of optics K calibration matrix l(l, l ) projection operator to obtain line l L x (L x ) projection ray to image point x (.) coordinate in model coordinate system of two or more images m scale difference of x - and y -image coordinates M (M) motion or similarity

15 Table 0.5 List of Symbols in Part III (2) abbreviation meaning n (. ) coordinate in normal camera coordinate system (parallel to scene coordinate system) N normal equation matrix N pp, N kk normal equation matrices reduced to orientation parameters and coordinates ω rotation angle around X-axis of camera, roll angle O(Z) coordinates of projection centre 2 (x, l ), 3 (x, l ) prediction of point from point and line in two other images φ rotation around Y -axis of camera, tilt angle P projection with projection matrix for points P t(p t ) tth image with parameters of projection P 0t (p 0t ) image with observed parameters p 0t of projection P 0t, indices t T 0 p vector of unknown orientation parameters P projection matrix for points P d (d 1) d unit projection matrix [I d 1 0] q vector of parameters modelling non-linear image distortions Q 3 6-projection matrix for lines, 4 4 matrix for quadrics Q unit projection matrix [I ] R (R) rotation matrix s, S image scale s and image scale number 1/S s shear of image coordinate system s vector of additional parameters for modelling systematic errors {1,..., t,...t } = T index set for images (time) T = [[T i,jk ]] trifocal tensor v (v ) vanishing point x, x observable image point, ideal image [ point (without ] distortion) a1 a Z(a) 2 2 matrix operator Z : a a 2 a 1 Table 0.6 Abbreviations abbreviation meaning AO absolute orientation AR autoregressive BLUE best linear unbiased estimator DLT direct linear transformation EO exterior orientation GIS geoinformation system(s), geoinformation science GHM Gauss Helmert model GPS global positioning system GSD ground sampling distance IMU inertial measuring unit IO interior orientation LS least squares MAD median absolute difference MAP maximum a posteriori ML maximum likelihood MSE mean square error PCA principal component analysis RANSAC random sample consensus RMSE root mean square error SLERP spherical linear interpolation SVD singular value decomposition