A Calibration Algorithm for POX-Slits Camera

Transcription

1 A Calibration Algorithm for POX-Slits Camera N. Martins 1 and H. Araújo 2 1 DEIS, ISEC, Polytechnic Institute of Coimbra, Portugal 2 ISR/DEEC, University of Coimbra, Portugal Abstract Recent developments have suggested alternative multiperspective camera models potentially advantageous for the analysis of the scene structure. Two-slit cameras are one such case. These cameras collect all rays passing through two lines. The projection model for these cameras is non-linear, and in this model every 3D point is projected by a line that passes through that point and intersects two slits. In this paper we propose a robust non-iterative linear method for the calibration of this type of cameras. For that purpose a calibrating object with known dimensions is required. A solution for the calibration can be obtained using at least thirteen world to image correspondences. To achieve a higher level of accuracy data normalization and a non-linear technique based on the maximum likelihood criterion can be used to refine the estimated solution. 1 Introduction Projection models constitute a relevant issue in computer vision. The mathematical model that describes the formation of the most common type of images is the perspective projection model. Most of the commercialized optical devices generate images whose geometrical properties are described in this model. Therefore, the classic pinhole and orthographic camera models have long been used in 3D imaging applications. However certain special vision problems can benefit from the application of alternative projection models, as recent developments have suggested. Besides, those developments in image sensing make the perspective model highly restrictive. These multiperspective models have been providing advantageous imaging systems for understanding the structure of observed 3D scenes. Examples of such camera models are bi-centric [13], crossed-slits (also known as x-slits) [15], general linear [14] and rational polynomial [4] models. Multiperspective imaging has also been explored in computer graphics[8]. In the bi-centric model the centers of horizontal and vertical projections lie in different locations on the camera s optical axis. Perspective and pushbroom cameras [3] are particular cases of this model, if the horizontal and vertical projections lie in the same locations and if only the horizontal projection resides on the infinity (corresponds to a vertical strip of a sensor translating sideways), respectively. In [13] it was also shown that a straight line in the scene is projected

2 2 into a hyperbole in the image. The pushbroom model collects rays along parallel planes from points swept along a linear trajectory [3]. The most visible distortion in the images that followthe pushbroom model is the variation of aspect-ratio. General linear and cubic camera are general models. The general linear camera model unifies most projection models used in computer vision, including perspective and affine models, optical distortions models, x-slits models, stereo systems and catadioptric systems [14]. A cubic camera maps the image points as rational polynomial functions, of degree less than four, of the coordinates of a world point [4]. This camera model treats projective, affine, pushbroom and x-slits cameras as particular cases. In the x-slits model the projection ray of a generic 3D point is defined by the 3D line that passes through the point and two lines, referred as slits. The image is obtained by the intersection of every projective ray with the image plane. This model was initially designed by one of the pioneers of the color photography, Ducos du Hauron, in 1888 [7], under the name transformisme en photographie [6]. He thought that his device would be used in the 20 th century to create visions of another world [7]. However, it was a restricted model in terms of the slits positions, which were parallel and orthogonal between each other (this situation was later referred as parallel-orthogonal x-slits, or pox-slits [15]). An interesting aspect is that pox-slits projection equations are similar to the bi-centric model [1]. A particular case of the pox-slits camera, in which the vertical slit resides at infinity, is the pushbroom camera. One century later, the pox-slits model was revised and generalized by Kingslake, who concluded that it was similar to the perspective projection model in which the image is stretched or compressed in one direction more than the other [6]. This fact shows its adequacy to the use in wide-screen technologies. Zomet et al, in [15], expanding the Kingslake generalization, introduced the x- slits projection model. According to their study, one advantage of of this model is the fact that x-slits images can be easily generated by perspective images. Shortly, this procedure is performed by pasting together vertical or horizontal samplings of a sequence of images captured from a perspective camera, which moves, respectively, along a horizontal or vertical line. With a more complex procedure new x-slits views can be generated even when the camera motion is not parallel to the image plane [1]. The idea of sampling columns from images has been explored before, but using a constant sampling function [10]. This traditional mosaicing technique is similar to the one used to create pushbroom panoramas [13]. Another remarkable aspect of this camera is that perspective model is a particular case of the x-slits camera, in which the vertical and horizontal slits lie in the same plane. The optical center of the perspective camera is the intersection of the slits. In spite of the extensive analysis of x-slits cameras by [15], they have only focused on aspects related to image generation. In this paper we deal with the problem of calibrating this type of cameras. Grossberg et al, in [2], presented a different camera calibration algorithm, referred to as the generic imaging model. In that case calibration consists in de-

3 3 termining, for every image pixel, the associated 3D projection ray. This method is also used in [8] and [10]. This mapping can be conveniently described using a set of virtual sensing elements, called raxels. Raxels include geometric, radiometric and optical properties. As an extension to [2], [12] introduce a generic calibration approach. In this method at least three images of a calibration object are acquired. The fact that a projective ray is a 3D line yields a constraint that allows the recovery of both the motion and the camera s parameters. This constraint is formulated via a set of trifocal tensors that can be estimated linearly. In [9] this calibration method is used in a 3D reconstruction process, with a parametric reprojection to refine the obtained solution, based on bundle adjustment. In the calibration method described in this paper, we use the non-linear x- slits equations. For estimation purposes the equations are rewritten so that linear estimation methods can be used. For good levels of accuracy in the estimates, data normalization and a non-linear technique based on the maximum likelihood criterion can be used [5]. 2 X-slits projection model Consider the x-slits projection configuration represented in figure 1. The pro- Fig. 1. X-slits projection model. jective ray of a generic 3D point, P, must intersect two lines, or slits, l 1 and l 2. Point P together with each slit defines one plane. The intersection of those planes defines the projective ray. The projection of the 3D point in the image, p, is obtained by the intersection of the projective ray with the image plane. To define the two slits, let u i and v i (with i =1, 2) be two generic planes defined in a space of 3 dimensions, given by their parametric coordinates. The slits, l i, are defined through the intersection of those planes. These slits can be represented by the dual Plucker matrix [5], whose equation is 0 L i34 L i42 L i23 L i = u iv T i v iu T L i34 0 L i14 L i13 i = L i42 L i14 0 L i12 L i23 L i13 L i12 0

4 4 if we use the Plucker coordinates of the slits. The projective ray, l, is the intersection between two planes, defined by each slit l i and the 3D point P (L 1 P and L 2 P, respectively), and can be defined by the dual Plucker matrix, through L =(L 1 P )(L 2 P ) T (L 2 P )(L 1 P ) T Assuming that the image plane, I, is defined by the points P 0, P 1 and P 2, any point that belongs to I can be expressed by the linear combination of those points, given by kxp 0 + kyp 1 + kp 2 As a result any point from a 2D space vector defined in the image plane, in homogeneous co-ordinates, is given by p = [ kx ky k ] T [11]. The projection of a 3D generic point P in the image plane I generates a 2D point p. This projection is given by the intersection of the projective ray l with the image plane I. Therefore l must belong to both planes L i P. Therefore, [ P T L 1 P 0 P T L 1 P 1 P T L ] 1 P 2 P T L 2 P 0 P T L 2 P 1 P T L p = 0 (1) 2 P 2 The solution for equation (1) is the right null space of the matrix. This solution is obtained by using the cross product between the elements of the matrix, as P T L ( 1 P 1 P T T 2 P 2 P ) 1 L 2 P kp = P T L ( 1 P 2 P T T 0 P 0 P ) 2 L 2 P P T L ( 1 P 0 P T T 1 P 1 P ) 0 L 2 P The homogeneous relation between 3D world scene points and 2D image points, in pixels, for the x-slits projection model is kp = k x γ c x P T L 1 I 0 L 2 P P T L 1 I 1 L 2 P (2) P T L 1 I 2 L 2 P 0 k y c y where I 0, I 1 and I 2 are the Plucker matrices corresponding to the x and y axis of the image plane and the line at infinity. k x and k y are the focal lengths. (c x,c y ) are the coordinates of the principal point and γ is the image skew. According to [15], this solution is unique unless it resides on the line joining the intersections of the two slits with the image plane. 3 Calibrating X-slits projection model In this section we describe an algorithm to calibrate the x-slits camera. We begin with a particular case of this camera, known as pox-slits, and then we address the general case.

5 5 3.1 Pox-slits case We showhowto calibrate the pox-slits camera because this camera is similar to the bi-centric camera and a generalization of the pushbroom camera. Let us define, u 1 = [ 1000 ] T v 1 = [ ] T 001 Z 1 u 2 = [ 0100 ] T v 2 = [ ] T 001 Z 2 Let us also consider three homogeneous points P 0 = [ 1000 ] T P 1 = [ 0100 ] T P 2 = [ 0001 ] T which belong to the image plane. As a result, equation (2) is given by k x y = k X x γ c x Z 1 Z Z 1 0 k y c y Y Z 2 Z Z 2 (3) The calibration algorithm aims at estimating the intrinsic camera parameters k x, k y, c x, c y and γ and the slits parameters Z 1 and Z 2. From equation (3) we can obtain k x Z 1 XZ + k x Z 1 Z 2 X γz 2 YZ+ γz 1 Z 2 Y + c x Z 2 c x Z 1 Z c x Z 2 Z+ +c x Z 1 Z 2 + Z 1 xz + Z 2 xz Z 1 Z 2 x = xz 2 (4) and c y Z c y Z 2 k y Z 2 Y + Z 2 y = Zy (5) Assuming, without loss of generality,, C 1 = c y Z 2 and C 2 = k y Z 2,wecan rewrite equation (5), matrix form, as c y [ ] Z 1 Y y C 1 C 2 = Zy Z 2 Using, at least, four world to image correspondences, we obtain a system of equations whose solution can be obtained using any numerical linear method, e.g, SVD. The solutions of this system of equations yield estimates for the intrinsic parameters k y and c y, and the slit parameter Z 2. Assuming now, without loss of generality, C 3 = k x Z 1, C 4 = c x Z 1, C 5 = Z 1 γ and C 6 = c x Z 1, and substituting the estimated parameters in equation (4) we get xz 2 xzz 2 =( XZ + XZ 2 )C 3 YZZ 2 γ + YZ 2 C 5 +(Z 2 ZZ 2 )c x + +( Z + Z 2 )C 6 +(xz xz 2 )Z 1 Similarly, and using at least six world to image correspondences, we obtain a system of equations whose solutions yield estimates for the the intrinsic parameters k x, c x and γ, and the slit parameter Z 1. To obtain a higher level of accuracy, Hartley et al, in [5], suggest data normalization and a non-linear technique based on the maximum likelihood criterion. Therefore to calibrate the pox-slits camera six world to image correspondences, at least, must be used.

6 6 3.2 General case Let us nowaddress the case of the general x-slits camera. Assuming C 1 = L 142L 234 L 134L 242 C 2 = L 114L 234 L 134L 214 C 3 = L 142L 213 L 113L 242 C 4 = L 134L 213 L 113L 234 C 5 = L 114L 223 L 123L 214 C 6 = L 123L 234 L 134L 223 C 7 = L 114L 242 L 142L 214 C 8 = L 123L 213 L 113L 223 C 9 = L 114L 213 L 113L 214 C 10 = L 134L 212 L 112L 234 C 11 = L 112L 214 L 114L 212 C 12 = L 113L 212 L 112L 213 C 13 = L 142L 223 L 123L 242 C 14 = L 123L 212 L 112L 223 C 15 = L 142L 212 L 112L 242 V 1 = C 3 + C 5 V 2 = c xc 1 k xc 5 + k xc 10 + γc 13 V 8 = c xc 8 V 20 = c yc 8 V 3 = c xc 2 γc 3 + k xc 9 + γc 10 V 4 = c xv 1 + k xc 12 + γc 14 V 5 = c xc 4 + γc 8 V 6 = c xc 6 + k xc 8 V 7 = c xc 7 + k xc 11 + γc 15 V 9 = k xc 4 + γc 6 V 10 = k xc 6 V 12 = k yc 3 + k yc 10 c yc 2 V 16 = k yc 13 c yc 1 V 17 = k yc 15 c yc 7 V 13 = k yc 4 V 11 = γc 4 V 15 = k yc 8 + c yc 4 V 14 = k yc 6 V 18 = k yc 14 c yv 1 V 19 = c yc 6 without loss of generality, equation (2) can be rewritten as V 10 V 9 xc 1 + V 2 xc 6 V 6 P T 0 V 11 xc 2 + V 3 xc 4 V xc 7 + V 7 xv 1 + V P = xc 8 V 8 and P T 0 V 14 yc 1 + V 16 yc 6 V 19 0 V 13 yc 2 + V 12 yc 4 V yc 7 + V 17 yv 1 + V P = yc 8 V 20 The general model of this camera is specified by 15 parameters and therefore the total number of unknowns is also 15. However, as a result of rewriting the equations so that a linear numerical method can be used, we end up with 26 unknowns. Therefore at least 13 world to image correspondences are required. 4 Experimental Results The experimental results presented in this paper use synthetically generated data. In addition we only present results for the case of the pox-slits model. Results for the general case are still being obtained. As it can be seen in figure 2(a), a sphere is used as calibration object. This random sphere, with radius 20 and center ( 22.33, 43.37, ), is made up of D known points. In the figure we also show the image plane (bottom) and the planes that contain the slits (the two upper planes). Figure 2(b) represents the pox-slits image of the sphere points, with resolution Using equation (3), the pox-slits camera is defined with Z 1 = 100, Z 2 = 50, k x =47, k y = 63, c x = 320, c y = 240 and γ = 25. To calibrate the camera we start by normalizing the image coordinates as suggested by Hartley [5]. Gaussian white noise with 0 mean and σ 2 variance

7 7 (a) (b) Fig. 2. (a) Visualization of the calibration object, with the image plane and the planes that contain the slits; (b) Pox-slit image of calibration object. was added to the image coordinates of the points. The noise variance was varied between 0.1 pixels and 20 pixels. For each value of noise variance 150 runs were performed. The percent error in the estimates for each parameter was computed. The averages (for each noise variance level) of the percent errors are presented in Figure 3. As it can be seen in the figure, errors increase almost linearly with the noise level. We also computed the variance of errors in the estimates of the parameters. The values of the error variances are belowthe floating point precision. Therefore we can assume that this algorithm can be used to estimate this type of camera. 5 Conclusions In this paper we present a robust non-iterative linear algorithm to calibrate a pox-slits camera. The algorithm requires at least with six world to image correspondences. Normalization of the coordinates of the image points is an essential step of the algorithm. The algorithm for the general x-slit camera is also described briefly. References 1. D. Feldman, A. Zomet, S. Peleg, and D. Weinshall, Video synthesis made simple with the x-slits projection, IEEE Workshop on Motion and Video Computing, pp , Orlando, December M. Grossberg and S. Nayar, A General Imaging Model and a Method for Finding its Parameters, Proceedings of IEEE International Conference on Computer Vision, Vancouver - Canada, July R. Gupta and R. Hartley, Linear Pushbroom Cameras, IEEE Transactions Pattern Analysis and Machine Intelligence, Vol. 19, N. 9, pp , R. Hartley and T. Saxena, The cubic rational polynomial camera model, Image Understanding Workshop, pp , R. Hartley and A. Zisserman, Multiple View Geometry, Cambridge University Press, R. Kingslake, Optics in Photography, SPIE Optical Eng. Press, B. Newhall, The History of Photography, from 1839 to the present day, The Museum of Modern Art, pp. 162, 1964.

8 8 Fig. 3. Relative mean error in the estimation of the camera parameters plotted as function of the noise variance. 8. S. Peleg, M. Ben-Ezra, and Y. Pritch, Omnistereo: Panoramic Stereo Imaging, IEEE Transactions Pattern Analysis and Machine Intelligence, vol. 23, n. 3, pp , March S. Ramalingam, S. K. Lodha and P. Sturm, A generic structure-from-motion algorithm for cross-camera scenarios, OMNIVIS - Workshop on Omnidirectional Vision, Camera Networks and Non-Classical Cameras, pp , Prague, Czech Republic, May S. Seitz, The Space of All Stereo Images, Proceedings of the 8th IEEE International Conference on Computer Vision, vol. 1, pp , Vancouver - Canada, July J. Semple and G. Kneebone, Algebraic Projective Geometry, Oxford University Press, P. Sturm and S. Ramalingam, A generic concept for camera calibration, Proceedings of 8th European Conference on Computer Vision, Prague - Czech, D. Weinshall, M. Lee, T. Brodsky, M. Trajkovic, and D. Feldman, New View Generation with a Bi-Centric Camera, Proceedings of 7th European Conference on Computer Vision, pp , Copenhagen - Denmark, May J. Yu and L. McMillan, General Linear Cameras, Proceedings of 8th European Conference on Computer Vision, Prague - Czech, A. Zomet, D. Feldman, S. Peleg, and D. Weinshall, Mosaicing New Views: The Crossed-Slits Projection, IEEE Transactions Pattern Analysis and Machine Intelligence, pp , 2003.