Quasi-Euclidean Uncalibrated Epipolar Rectification

Transcription

1 Dipartimento di Informatica Università degli Studi di Verona Rapporto di ricerca Research report September 2006 RR 43/2006 Quasi-Euclidean Uncalibrated Epipolar Rectification L. Irsara A. Fusiello Questo rapporto è disponibile su Web all indirizzo: This report is available on the web at the address:

2 Abstract This paper deals with the problem of epipolar rectification in the uncalibrated case. First the calibrated (Euclidean) case is recognized as the ideal one, then we observe that in that case images are transformed with a collineation induced by the plane at infinity, which has a special structure. Hence, that structure is imposed to the sought transformation while minimizing a rectification error. We call our method quasi-euclidean epipolar rectification. Experiments show that it yields images that are qualitatively close to the one produced by Euclidean rectification. Keywords: research reports, L A TEX

3 1 Introduction Epipolar rectification is an important stage in dense stereo matching, as practically any stereo algorithm requires rectified images, i.e., images where epipolar lines are parallel and horizontal, and corresponding points have the same vertical coordinates. In the case of calibrated cameras the Euclidean epipolar rectification is unique up to trivial transformations. The problem is addressed in [2]. On the contrary, in the case of uncalibrated cameras, there are more degrees of freedom in choosing the rectifying transformation [3] and a few competing methods are present in the literature [8, 9]. Each aims at producing a good rectification by minimizing a measure of distortion, but none is clearly superior to the others, not to mention the fact that there is no agreement on what the distortion criterion should be. Above all, none seems to achieve results comparable to the calibrated case, which can be undoubtedly taken as the target result. This paper aims at approaching this target: given the lack of knowledge of the internal camera parameters, we will achieve an approximation of the Euclidean epipolar rectification, which we refer to as quasi-euclidean epipolar rectification. Geometrically, in the Euclidean frame, rectification is achieved by a suitable rotation of the image plane. Hence, the image transformation is the collineation induced by the plane at infinity. This is tantamount to say that rectification is done with respect to the plane at infinity. Calibration is always referred to a plane, which can be characterized as the locus of zero-disparity in the rectified stereo pair. In the uncalibrated case, however, the reference plane is generic (any plane can play the role of the infinity plane in the projective space). Our approach can be regarded as seeking an approximation for the plane at infinity as the reference plane. 1.1 Previous work The first work on uncalibrated rectification (called matched-epipolar projection ) is [4], followed several years later by [3] where the author tidied up the theory. Hartley uses the condition that one of the two collineations should be close to a rigid transformation in the neighborhood of a selected point, while the remaining degrees of freedom are fixed by minimizing the distance between corresponding points (disparity). In the meantime, [10] also proposed a distortion criterion based on simple geometric heuristics. The state-of-the-art papers in projective rectification [9, 8] appeared simultaneously in Loop and Zhang [9] decompose each collineation into similarity, shearing and projective factors and attempt to make the projective component as affine as possible. Isgrò and Trucco [8] build upon [3] and propose a method that avoids computation of the fundamental matrix, using the same distortion criterion as in [3]. The practice has shown that the 1

4 rectification produced by these methods is not always satisfactory, if compared to results obtained in the calibrated case. Recently, Al-Zahrani et al. [7] argued that minimizing the disparity might be the cause of the problem, and proposed a technique which is similar to [8] but uses a different distortion criterion derived from [9]. Rectification ends up in a non-linear minimization with 6 d.o.f. A different approach is followed in [1]: they design the collineations so as to minimize the relative distortion between the rectified images (instead of the distortion of each rectified image with respect to the original one), and the remaining degrees of freedom are fixed by choosing the reference plane in the scene, that will have zero disparity in the rectified pair. This choice, which affects sensibly the quality of rectification, is left to the user. The merit of this approach, however, is to have made the role of the reference plane explicit. 2 Background In this section we shall briefly recapitulate the theory of calibrated (or Euclidean) epipolar rectification; please refer to [2] for more details. Given two camera matrices P or and P o, the idea behind rectification is to define two new virtual cameras P nr and P n obtained by rotating the actual ones around their optical centers until focal planes become coplanar. The rectification method describes how to compute the new cameras. Then (we concentrate on the right camera, but the same reasoning applies to the left one), the rectifying transformation that is to be applied to images is given by the 3 3 matrix: H r = P nr1:3 P 1 or 1:3 (1) where the subscript denotes a range of columns. It is easy to see that H r is the collineation induced by the plane at infinity between the old and the new view, hence it can be written as: H r = K nr R r K 1 or (2) where K or and K nr are the internal parameters of the old and new camera respectively, and R r is the rotation that is applied to the old camera in order to rectify it. In the calibrated case, K or is known, K nr is set arbitrarily (typically to be close to K or ) and R r is prescribed by the rectification algorithm. When considering the left camera, care must be taken that K nr and K n have the same vertical focal length and the same vertical coordinate of the principal point. The rectified images are as if they were taken by a pair of cameras related by a translation along the baseline. Hence, the zero-disparity plane is at infinity. 2

5 3 Method We henceforth concentrate on the uncalibrated case. We assume that intrinsic parameters are unknown and that a number of conjugate points m j mj r are available. Let H r and H be the unknown rectifying collineations. The transformed conjugate points must satisfy the epipolar geometry of a rectified pair, (H r m j r) T F (H m j )=0, (3) which is represented by a fundamental matrix of a particular, known form [8]: F = As this equation must hold for any conjugate pair, it is used in a non-linear least squares fashion to constraint H r and H, as in [8]. The way in which H r and H are parametrized is crucial, and characterizes our approach. We force the rectifying collineations to have the same structure as in the calibrated (Euclidean) case, i.e., to be collineations induced by the plane at infinity, namely H r = K nr R r K 1 or H = K n R K 1 o. (4) The old intrinsic parameters and the rotation matrices are unknown, whereas the new intrinsic parameters can be set arbitrarily; we used K nr = K n = K o + K or. (5) 2 Scaling or horizontal translation for centering can be added afterward to both images. Each collineation depends in principle on five (intrinsic) plus three (rotation) unknown parameters. We reduce the number to four by making an educated guess on the internal parameters: no skew, principal point in the centre of the image, aspect ratio equal to one. The only remaining unknowns are the focal lengths α r and α : α r 0 w/2 α 0 w/2 K or = 0 α r h/2 K o = 0 α h/2 (6) where w and h are width and height (in pixel) of the image. The minimization is carried out using Levenberg-Marquardt and follows an alternation strategy: first we fix α r and α and solve for the three rotation parameters, then we fix R r and R and solve for the focal lengths α r and α, until 3

6 convergence. We start with α r = α = w + h and all the three Euler angles set to zero. Since in real stereo pairs we expect the YZ plane to be already nearly vertical, the roll angle is fixed to zero for the first iteration. Convergence is not guaranteed. As a fall-back strategy, we detect divergence when the focal length falls outside the interval [1/3(w+h), 3(w+h)] (this has been suggested in [6] as a reasonable range of values, considering standard cameras) and in that case we re-start the algorithm using a random value inside [1/3(w + h), 3(w + h)] for the focal lengths. As a bonus, one can compute an approximation of the the collineation induced by the plane at infinity between the two original cameras, which is given by 4 Results H = Hr 1 K rn K 1 n H = Hr 1 H (7) We present here some results of our rectification algorithm applied to the SYN- TIM images 1. Calibration data (provided with the images) was used for computing the ground truth Euclidean rectification using [2] 2. Unfortunately, a quantitative measure of distortion that captures the desired behaviour of rectification does not exist: If it existed, one could simply minimize it to obtain the best rectification. Therefore we are forced to leave the assessment of our quasi-euclidean rectification to visual comparison with ground truth. As the reader can recognize in Fig. 1, our results are very close to the Euclidean rectification. With respect to [8], which is one of the state-of-the-art algorithms and uses the SYNTIM images as well, our rectified images looks generally less distorted, after a visual inspection. No direct comparison was possible with [9] because they used their own image pair. The second column of Table 1 reports the RMS rectification error, computed as 1 d( 2N FH m j,h rm j r) 2 + d( F T H r m j r,h m j )2 (8) j where d() here is the Euclidean distance between a line and a point (in homogeneous coordinates) and N is the number of points. A low error indicates that the pair is rectified. It is well known that, in principle, the two focal lengths can be computed from two uncalibrated images [5, 11]. Our technique, as a side-effect, yields a reasonable estimate of the focal length. The third and fourth column of Table 1 1 Available from 2 Code available from 4

7 5 Figure 1: Euclidean (left) and quasi-euclidean rectification (right).

8 Image pair Residual True focal Computed focal Aout BalMire BalMouss Rubik Angle Tot Color Sport Table 1: Residuals of quasi-euclidean rectification and comparison of focal length. Image pairs are in the same order as in Fig. 1) report the true and the computed focal length, respectively. In these experiments we set α = α r hence we computed only one value (the true focal length is the average of the right and left ones). 5 Discussion We presented a new method for the epipolar rectification of uncalibrated stereo pairs which approximates the Euclidean (calibrated) case by enforcing the rectifying transformation to be a collineation induced by the plane at infinity. The results are close to the target Euclidean rectification and compares favorably with state-of-the-art uncalibrated methods, in terms of distortion applied to the rectified images. This method will be part of a view synthesis application that we are developing. Given the general utility of rectification, we intend to tidy up the code and make it available on the web. References [1] A. Al-Zahrani, S. S. Ipson, and J. G. B. Haigh. Applications of a direct algorithm for the rectification of uncalibrated images. Information Sciences Informatics and Computer Science, 160(1-4):53 71, [2] A. Fusiello, E. Trucco, and A. Verri. A compact algorithm for rectification of stereo pairs. Machine Vision and Applications, 12(1):16 22, [3] R. Hartley. Theory and practice of projective rectification. International Journal of Computer Vision, 35(2):1 16, November

9 [4] R. Hartley and R. Gupta. Computing matched-epipolar projections. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages , New York, NY, June [5] R. I. Hartley. Estimation of relative camera position for uncalibrated cameras. In Proceedings of the European Conference on Computer Vision, pages , Santa Margherita L., [6] A. Heyden and M. Pollefeys. Multiple view geometry. In G. Medioni and S. B. Kang, editors, Emerging Topics in Computer Vision, pages Prentice Hall, [7] Y.-H. Y. Hsien-Huang P. Wu. Projective rectification with reduced geometric distortion for stereo vision and stereoscopic video. Journal of Intelligent and Robotic Systems, 42(1):Pages 71 94, Jan [8] F. Isgrò and E. Trucco. Projective rectification without epipolar geometry. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages I:94 99, Fort Collins, CO, June [9] C. Loop and Z. Zhang. Computing rectifying homographies for stereo vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages I: , Fort Collins, CO, June [10] L. Robert, M. Buffa, and M. Hebert. Weakly-calibrated stereo perception for rover navigation. In ICCV, pages 46 51, [11] P. Sturm. On focal length calibration from two views. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, volume II, pages , Kauai, USA,

10 University of Verona Department of Computer Science Strada Le Grazie, 15 I Verona Italy