Arm coordinate system. View 1. View 1 View 2. View 2 R, T R, T R, T R, T. 12 t 1. u_ 1 u_ 2. Coordinate system of a robot

Transcription

1 Czech Technical University, Prague The Center for Machine Perception Camera Calibration and Euclidean Reconstruction from Known Translations Tomas Pajdla and Vaclav Hlavac Computer Vision Laboratory Czech Technical University CZ Karlovo nam. 13, Praha 2 tel: fax: pajdla@vision.felk.cvut.cz October 4, 1996 Reference [1] Tomas Pajdla and Vaclav Hlavac. Camera calibration and Euclidean reconstruction from known translations. Presented at the workshop Computer Vision and Applied Geometry, Nordfjordeid, Norway, August 1{ This publication can be obtained via anonymous ftp from ftp://cmp.felk.cvut.cz/pub/cvl/articles/pajdla/cvag95.ps.gz Czech Technical University, Faculty of Electrical Engineering Department of Control Engineering, Center for Machine Perception, Computer Vision Laboratory Prague 2, Karlovo namest 13, Czech Republic, FAX , phone

2 Camera Calibration and Euclidean Reconstruction from Known Translations? Tomas Pajdla and Vaclav Hlavac Computer Vision Laboratory Czech Technical University CZ Karlovo nam. 13, Praha 2 tel: fax: pajdla@vision.felk.cvut.cz Abstract. We present a technique for camera calibration and Euclidean reconstruction from multiple images of the same scene. Unlike standard Tsai's camera calibration from known scene, we exploit controlled known motions of the camera to obtain the calibration and Euclidean reconstruction without any knowledge about the scene. We consider three translations of an uncalibrated but same camera mounted on the robot arm providing us with four views of the scene. We also assume to measure the translations of some Euclidean coordinate system rigidly attached to the camera. This special, but still realistic, arrangement brings us a linear algorithm for recovery of all intrinsic camera calibration parameters, rotation parameters of the camera with respect to the robot coordinate system, and proper scaling factors for all points allowing their Euclidean reconstruction. The experiments show that an ecient and robust algorithm is obtained by exploiting Total Least Squares in combination with careful normalization of image coordinates. 1 Introduction Standard stereo [1] delivers Euclidean reconstructions if calibrations of the cameras are available and if their mutual positions and orientations are known. Tsai's \bundle adjustment" camera calibration technique [8] recovers camera parameters by nding a projection of a known scene, i.e. three-dimensional coordinates of some points in the scene are explicitly measured, into observed images. If the scene is not known other knowledge must be exploited to calibrate the cameras and reconstruct the scene.? This work has been partially done during the visit of T. Pajdla at ESAT, K.U.Leuven. Support by Esprit Basic Research Action `VIVA' and the Belgian project IUAP-50 on Robotics and Industrial automation is gratefully acknowledged. This research was also supported by the Grant Agency of the Czech Republic, grant 102/95/1378, European Union grant Copernicus No RECCAD, and by Czech Ministry of Education project No. VS96049.

3 Faugeras has shown [2] that if there is no information whatsoever about the scene and the cameras, only projective reconstruction can be obtained. Maybank and Faugeras have also developed [6] an algorithm for Euclidean reconstruction from three images of a scene taken by the same camera. This method assumes no extra knowledge about the scene and camera motions but requires to solve an overdetermined system of nonlinear equations. Moons et al. have used knowledge about the motion to decrease an uncertainty in reconstructions. They presented [7] an ane reconstruction for the case when the images were taken by a translated camera. Horaud, Mohr et al. [5] exploited controlled motion of a camera to get a Euclidean reconstruction of the scene. They rst computed the projective reconstruction and then got a camera calibration by solving a set of quadratic equations. In this work we deal with the linear camera calibration from known motions. Our approach is similar to the work of Horaud et al. [5] in that respect that known motions and an unknown scene is assumed. Similarly, our experiments are carried out with a camera mounted on a robot arm. On the other hand, we show how the calibration from known pure translations reduces into the solving a set of linear equations. This allows to construct ecient and robust camera calibration algorithm that does not require any special calibration objects. In the next section we describe a controlled motion motions and dene the model of a perspective camera. The calibration method is presented in the section 2.1. Finally, experiments corroborating feasibility of the proposed method are shown in the section 3. 2 Camera calibration from controlled motion We consider a camera with xed internal parameters rigidly attached to a positioning device like the arm of a robot, see Figure 1. We expect that the robot is equipped with a cartesian coordinate system in which the positions T i and the orientations R i of the arm can be measured. The rigid transformation between the camera ane coordinate system and the local coordinate system of the arm R, T is not known but is assumed to remain constant during the measurement. A perspective linear camera projects points from a 3-dimensional projective space P 3 into a 2-dimensional projective space P 2. Points from a 3-dimensional projective space will be represented by homogeneous 4-vectors, X, as well as points from retina plane are regarded to be homogeneous 3-vectors, U. The fourth element of X, X 4 can be for nite points set to 1, and therefore nite points will be henceforth considered to be in the form (x 1) T. Vectors U are measured in images up to some non-zero scale. It is often desirable to express nite points U = (p; q; r) T in the form U = u, where u = (u; v; 1) T, so that u a v have meaning of ane, pixel image coordinates. The space-image mapping by a perspective camera can be represented by a 3 4 matrix M of rank 3. If U and X are corresponding points in P 3 resp. P 2,

4 Arm coordinate system R, 12 t 1 View 1 R, T R, T View 2 R, T View 1 View 2 t 1 R, T R 1, T1 A t 1 A R 2, T 2 u_ 1 u_ 2 Coordinate system of a robot Fig. 1. Images of the scene are taken by the camera rigidly mounted mounted on a robot's arm. Fig. 2. In the case of pure translations, the translation vector between the camera centers equals to the translation vector between the arm coordinate systems. x_ then the mapping is explicitly given by U = M X; or u = M x x 4 : (1) The matrix M can be decomposed as M = K R (I j? t), where t represents the position of the camera, R is a rotation matrix representing the orientation of the camera and K is an upper triangular matrix, camera calibration matrix. It suits to our purposes to further rearrange terms in the equation (1), so that unknown entities which are to be identied become more explicit u = K R (I j t) x x 4 = K R x? K R t x 4 = A (x? x 4 t): (2) Ane matrix A represents the transformation from an Euclidean coordinate system attached to the camera into its retina, in general ane, coordinate system. In the next sections we will seek for matrix A using image coordinates of corresponding points and known relative motions. Having a matrix A it is a simple matter to obtain the matrix K, using the QR-decomposition of matrices. 2.1 Calibration from three known translations and two points in four views Let us assume that we can measure three relative translations t i, i = 1 : : : 3, in some Euclidean coordinate system rigidly attached to the camera, yet in an unknown relation to the camera ane coordinates. In our case, when the camera is mounted on a robot arm, the translation vectors T i, i = 0 : : : 3 are available

5 in the robot coordinate system. Therefore, by setting t i = T i? T 0, relative translations are obtained. Vectors t i sre measured as the dierences of arm positions. They equal to the translations vectors between camera focal points since there is no rotation of the camera, see Figure 2. Moreover, let two unknown points X 1, X 2 from the scene project into the image points u i1, u i2, where i = 0 : : : 3 numbers the views. The idea here is to rst obtain the ane reconstruction of the points which posseses itself as a linear problem. Having ane reconstruction it will be shown that the camera calibration matrix can be recovered by solving linear equations if relative motions are known. Substituting T i and x 4 = 1 into the equation (2) yields i1 u i1 = KR x 1? KR T i ; i2 u i2 = KR x 2? KR T i ; i = 0 : : : 3:(3) If all equations, i = 1 : : : 3 are subtracted from the zeroth one, i = 0, the set of equations where unknown points x 1 and x 2 are eliminated is obtained: 01 u 01? i1 u i1 =? A t i ; 02 u 02? i2 u i2 =? A t i ; i = 1 : : : 3:(4) Now, if the equations on the right hand side are subtracted from the left ones, the following sets of homogeneous linear equations is obtained 01 u 01? i1 u i1? 02 u 02 + i2 u i2 = 0 ; i = 1 : : : 3: (5) Assuming that the u ij, i = 1 : : : 3; j = 1 : : : 2 are measured in images implies that they are nite points and can be expressed as u ij = [u ij v ij 1] T. Therefore the above equations can be rewritten as u 01?u i1?u 02 u i2 v 01?v i1?v 02 v i2 1?1? A 01 i1 02 i2 1 C A = 0 ; i = 1 : : : 3 ; (6) and resolved for unknown -s provided that one of them, e.g. 01, is kept xed for all i. Given -s, the points x i in (3) are determined up to some unknown ane transformation. Exactly this is used to obtain an ane reconstruction of a scene from pure translations in [7]. Similarly, given -s, the left hand sides of the equations in (4) are all determined up to some unknown, but common, scale. Thus, once -s are determined and if t-s spanning a 3-dimensional linear space are known, A can be computed from (4) up to a scale. Two image points are needed to form equation (6). It means that at least two corresponding points must be tracked in four images since the vectors minimally three t i, i.e. four images, span three-dimensional space.

6 2.2 Computing the calibration from many points Equations (6) were derived mainly to show the relation to an ane reconstruction and to prove that -s can be computed from image coordinates alone. In actual computation, both steps, i.e. computing the -s and the matrix A, are grouped together, so that one system of homogeneous linear equations is solved. After some manipulation with equations (4) the set of linear equations in the form has C b = 0 is obtained. By using the notation t i = (t i1 t i2 t i3 ) T, and u ij = (u ij v ij 1) T, C b = 0 can be rewritten as 0 u 01?u 11 t 11 t 12 t v 01?v 11 t 11 t 12 t ? t 11 t 12 t 13 u 01?u 21 t 21 t 22 t v 01?v 21 t 21 t 22 t ? t 21 t 22 t 23 u 01?u 31 t 31 t 32 t v 01?v 31 t 31 t 32 t ? t 31 t 32 t 33 u 02?u 12 t 11 t 12 t v 02?v 12 t 11 t 12 t ? t 11 t 12 t 13 u 02?u 22 t 21 t 22 t v 02?v 22 t 21 t 22 t ? t 21 t 22 t 23 u 02?u 32 t 31 t 32 t v 02?v 32 t 31 t 32 t ? t 31 t 32 t C B a11 a12 a13 a21 a22 a23 a31 a32 a33 1 C A = 0 : We have seen that only two points in four images suce to uniquely determine matrix A as well as all -s. However, it is always desirable to use all points available and, if possible, to design the calibration in such a way that the whole eld of view is covered by data. As more points are added, the equations (7) can be generated for each pair n2 of points. Thus, for n points, there would be = n (n?1) systems of equations 2 similar to (7) each giving rise to one (ideally same) A and four -s. On the other hand we could group all these equations together into some global matrix C in order to nd common consensus on A and -s. There is yet another reason why to solve all the equations altogether. Since all point pairs sharing common point also share its equations most of the equations are same. The same equations can be left out without any loss. It is easy to see that if all redundant equations are removed, all what remains is just one set (i.e. left equations in (4)) per one point. Hence, each point tracked in four views contributes by 9 new equations but only 4 new variables, -s, to be solved. By that, the global C will have only 9 n rows and 4 n + 9 columns (extra 9 counts for unknown entries of A). It also means that we solve only for 4 n + 9 unknowns and not for n (n?1) as it would be necessary if all point pairs 2 were treated independently. Left hand sides of the equations (4) are always linearly independent if t i span three-dimensional space since we assume that A is not singular. This could be violated only in the presence of extreme noise or if model was invalid. Equation C b = 0 is a homogeneous equation and can be numerically solved by SVD [3] decomposition since b equals to the right singular of C corresponding to its smallest singular value. This solution corresponds to the Total Least Squares solution of an overdetermined linear system [9]. In order to get stable (7)

7 results one has to scale coordinates in images and translation vectors so that they have similar ranges of magnitudes [4]. 2.3 Euclidean reconstruction By setting x 4 to 1 in (2) we can reconstruct calibration points in the robot coordinate system as x i = A?1 0i u 0i + T 0 : (8) 3 Experiments The rst experiment shows calibration of the camera mounted on the robot arm using a planar scene. Fig. 3. A camera is mounted on the robot arm and translated three times so that four images of the scene can be captured. The translation vectors span a three dimensional vector space. The camera Sony XC-75E with the Computar lens was zoomed to have focus length about 15 mm and mounted on the ABB robot arm. The arm was moved three times while keeping the rotation of the camera xed and four images of translated scene were captured. The translation vectors have been chosen to span three-dimensional vector space and so that the disparity vectors of projected points were not colinear, see Figure 3. The precision of the robot positioning was about 0:1 mm. 24 calibration points were extracted as the intersections of the lines tted to sides of six black squares in each of the four images. Each point in four views contributes to the linear system (7) by 9 equations. Therefore A is obtained by

8 sx sk sy u0 v0 Zoom NaN NaN NaN Tsai Trans Fig. 4. An Euclidean reconstruction of the calibration points is shown so that the view direction is perpendicular to the plane in which the points lie. Fig. 5. Intrinsic camera parameters recovered by the calibration from known translations (the third row) are compared to the parameters obtained by zooming (the rst row) and by Tsai's camera calibration method (the second row) Fig. 6. All reconstructed points indeed lie in a plane as the plane t residuals do not exceed 0:1 mm. Fig. 7. The lengths of the sides of squares are reconstructed with error in the range of 0:5 mm. The full line shows average reconstructed length. Two dashed lines show the range of expected length. SVD of 24 9 = 216 equations. Matrices K and R are then recovered by QR decomposition of A. Figure 5 shows camera intrinsic parameters K = sx sk u0 sy v0 1 recovered by dierent methods. The principal point given by the calibration from the known translations is much closer to the principal point measured by zooming than to the principal point obtained from Tsai's calibration method [8]. This shows the stability of the proposed algorithm since zooming delivers very accurate estimate of the principal point. 1 A

9 Fig. 8. An electricity outlet placed on the planar grid. Fig. 9. A frontal view of the reconstruction. Fig. 10. An oblique view of the reconstruction. Figure 4 shows the reconstructed points. The quality of reconstruction is supported by small residuals of plane t, Figure 6, and by good agreement of reconstructed and true size of the calibration squares, Figure 7. The second experiment shows a reconstruction of the scene consisting of the electricity outlet placed on the planar grid, Figure 8. The points were extracted manually with the error about 2 pixels. The camera was kept about half a meter from the scene while the scene was moved by hand in the range o 5 cm. The precision of translation measurements was about 1 mm. Figures 9 and 10 show a frontal and an oblique view of the reconstructed scene respectively. The overall shape of the outlet as well as angles and sizes were well reconstructed although particular points are noisy. The largest errors emerges on two small circles inside the outlet where the precision of correspond-

10 ing points was poor. 4 Conclusion The method for camera calibration and Euclidean reconstruction from known translations was presented. Our approach allows to calibrate the camera by tracking two points in four images of an unknown scene. We have shown that in this case a linear algorithm solving the calibration and reconstruction exists. Experiments suggest that the performance of the algorithm compares to the standard bundle adjustment camera calibration method. We believe that our approach is especially useful for the reconstruction of shape from long image sequences in a controlled environment. Acknowledgement We would like to thank to Dorin Ungureanu for fruitful discussions and for nding the principal point of the camera by zooming. We also thank to Bert Van den Berghe for assistance with the experiments. References 1. Olivier Faugeras. Three-Dimensional Computer Vision: A Geometric Viewpoint. The MIT Press, Olivier D. Faugeras. What can be seen in three dimensions with an uncalibrated stereo rig? In European Conference on Computer Vision, pages 563{578, G.H. Golub and C.F. Van Loan. Matrix Computations. The John Hopkins University Press, R. Hartley. In defence of the 8-point algorithm. In E. Grimson, editor, Proc. of the Fifth International Conference on Computer Vision, volume 1, pages 1064{1070, R. Horaud, R. Mohr, F. Dornaika, and B. Boufama. The advantage of mounting a camera onto a robot arm. In Proc. of the Europe-China Workshop on Geometrical Modelling and Invariants for Computer Vision, Xian, China, pages 206{213, Stephen J. Maybank and Olivier D. Faugeras. A theory of self-calibration of a moving camera. International Journal of Computer Vision, 8(2):123{151, T. Moons, L. Van Gool, M. Van Diest, and E. Pauwels. Euclidean reconstruction from uncalibrated views. In 2nd ESPRIT - ARPA Workshop on Invariants in Computer Vision, pages 297 { 316, Ponta Delgada, Azores, October R.Y. Tsai. A versatile camera calibration technique for high accuracy 3D machine vision metrology using o-the-shelf cameras and lenses. IEEE Journal of Robotics and Automation, 3(4):323{344, S. Van Huel and J. Vandewalle. The total least squares problem: Computational aspects and analysis. SIAM, Philadelphia, This article was processed using the LaT E X macro package with LLNCS style