Camera Self-calibration Based on the Vanishing Points*

Transcription

1 Camera Self-calibration Based on the Vanishing Points* Dongsheng Chang 1, Kuanquan Wang 2, and Lianqing Wang 1,2 1 School of Computer Science and Technology, Harbin Institute of Technology, Harbin , China dongshengchang@gmail.com, wangkq@hit.edu.cn, lianqingw@gmail.com 2 Xi an Communication Institute Xi an , China Abstract. In this article, a new method for camera self-calibration is presented. A correspondent matrix can be built by matching the corresponding anishing points between two images and the proposed method must use at least two of these matrices for calibration. The images are all taken from different orientations in any locations for the same obect in space. The anishing points in the image are the proectie points of the infinity points in space. So in the calibration, it has no constraints about the location and the orientation of the camera and the calibration procedure is easy. Since when the internal parameters of camera are changed during the task, the camera can be easily and effectiely calibrated if the anishing points can be coneniently gotten from the task images or the camera can insert three or more images about the same obect during the ongoing task. Compared with the traditional methods of camera self-calibration, this method is an easier and more effectie calibration method and also it is an online calibration with accurate results. Keywords: anishing points, online self-calibration, proection matrix. 1 Introduction The camera self-calibration is a method that only uses self-constraints of cameras to estimate their internal parameters. It has been widely applied in the mobile robot naigation, machine ision, biomedical and isual sureillance. The camera calibration techniques can be diided into two categories: online calibration and offline calibration. The online camera calibration means that the camera can be re-calibrated without stopping the ongoing task when its parameters change when being used. In other words, online camera self-calibration is a method which doesn t limit locations and orientations of the camera and doesn t need to know any proectie geometry or epipolar structure. Howeer, the offline calibration is on a contrary that it must interrupt the ongoing task for re-calibration. There are many self-calibration methods. And the common method of camera self-calibration is based on the proectie geometry theory and epipolar structure [1, 2]. * Supported by NSFC proect (grant no ). H. Deng et al. (Eds.): AICI 2011, Part III, LNAI 7004, pp , Springer-Verlag Berlin Heidelberg 2011

2 48 D. Chang, K. Wang, and L. Wang And Zhengyou Zhang proposed a method which was based on plane precise template [3]. Howeer, these kinds of methods need a complex process of calibration. Some other methods of camera self-calibration on actie ision are based on knowing the moing or rotation of the cameras [4, 5, 6]. And another method had been proposed by Richard that requires the camera at the same location take the images of the same obect in different orientations [7, 8]. For these methods hae constraints to the camera, they are not suitable for online re-calibration. In this article, the proposed calibration method is not essential to interrupt the task for re-calibration. And it does not need to know the camera s location or to track the motion of the camera. This method uses at least three iews for calibration with three steps. Firstly, it extracts the corner points, and then the anishing points of the specific direction are computed in eery image. Secondly it uses the anishing points to compute the correspondent matrix between any two images. Finally, it can use at least two of these correspondent matrixes to compute the calibration matrix. In specific application areas, as long as there is a method for coneniently computing the anishing points or the camera can insert at least three images for the same obect in the ongoing task, the proposed method can be calibrated online. This paper is organized as follows: the section 2 simply introduces the camera model. The section 3 shows the proposed algorithm of self-calibration. And the experiment results are gien in section 4. Section 5 gies the conclusion. 2 The Camera Model The camera model is pinhole, as indicated in Figure 1. It is a linear mapping from 3D proectie space points to 2D proectie space points. The points are parameterized by homogeneous coordinate, and the proection transformation can be represented by a matrix M = 3 4, the rank of which is 3. The matrix M may be decomposed as M = K(R t), wherein t is a 3 1 ector, representing the location of the camera, R is a 3 3 rotation matrix, representing the orientation of the camera with respect to an absolute coordinate frame, and K is the internal parameter of the camera, which is the purpose of camera self-calibration. The real camera has uncertain principal point. And in the manufacturing process of the CCD camera, the pixel of the camera is not exactly square. So the coordinate of the image is has a skew and the magnifications of the horizontal and the ertical directions are not exactly equal. The matrix K can be written as ku s u0 K = 0 k 0 (1) wherein k u and k the indicate the magnification in the u and coordinate directions, u 0 and 0 are the coordinates of the principal point, and s is the skew of the image coordinate axes.

3 Camera Self-calibration Based on the Vanishing Points 49 3 Algorithm of Camera Self-calibration 3.1 The Proection Matrix The anishing point is an important feature in the proectie geometric. B. Caprile and V. Torre firstly proided the calibration method based on anishing points [9]. They proed three properties of anishing points about camera calibration. The first two properties are useful for the determination of the extrinsic parameters, and the third property is useful for estimating the internal parameters. X c y c z c x c m z c p y x π O c x c y c Fig. 1. Pinhole camera model In the proposed method, the first property will be used. It says the parallel planes are correspondent to the same infinity line in space and the parallel lines are correspondent to the same infinity point in space. So either the infinity line or the infinity point has no relation with the location of the camera in space. Let m = ( u,,1) be the point of image coordinate and X = ( x, y, z,0) be the infinity point of space coordinates. The proectie transformation can be written as x up11 p12 p13 p14 x y w p21 p22 p23 p 24 KR y = = z (2) 1p31 p32 p33 p 34 z 0 wherein w is a non-zero factor and m is a anishing point. The displacement ector is eliminated and the calibration matrix is only related to the rotation matrix and the internal parameters. The rank of the matrix KR is 3, so it is inertible. 3.2 Calculation of Vanishing Points In space, the infinity point represents the direction of the line. For the linear camera model, the anishing point correspondent to the infinity point in space is the intersection of a set of lines in image which is in correspondence with the parallel lines. In this paper, the corner points should be extracted firstly. And then it uses the corner points to fit the straight lines in different directions. Let ε = [ ε1, ε2, ε3] and μ = [ μ1, μ2, μ3] be two lines in image, then the intersection of the two lines can be calculated by

4 50 D. Chang, K. Wang, and L. Wang ε2 ε3 ε3 ε1 ε1 ε2 ε μ =,, μ2 μ3 μ3 μ1 μ1 μ (3) 2 In order to obtain and maintain higher accuracy of anishing points, a set of lines which are correspondence to a set of parallel lines in space are used to compute the intersection of them which is the anishing point. 3.3 Determining the Calibration Matrix Two images are taken from different orientation for the same target. Let M1 = KR1 and M2 = KR2 be the proection matrix of the two images, taking any point x in the world coordinate, its corresponding points in two images are m 1 = KR 1 x and m2 = KR2x. The relationship between m 1 and m 2 is m = KR( KR) m = KRR K m = KRK m (4) By the equation aboe, it can skillfully and reasonably eliminate the displacement of the camera between two images. Let P = KRK 1, which is a conugate rotation matrix, wherein K is the conugating element[7, 8], and P is normalized, so its determinant is 1. A pair of points can proide two equations for the camera calibration, since it must use four pairs of points at least to compute the proectie transformation P, and any three of them can t be at a line. The experiment uses the same camera with the same internal parameters to take two iews. Since there is only the rotation matrix is the ariable in the proection transformation. 1 For P = KRK 1 and K is the calibration matrix, so K PK = R. For a rotation matrix R, it meets the relation that R = R T T, wherein R is the inerse transpose of R. 1 T T From the relation R = K PK, it can be attained that R = K PK. So an equation about R can be expressed that T T T ( KK ) P = P ( KK ) (5) Let C T = KK, then C can be written that a b c = = c e f T C KK b d e The equation (5) gies nine linear equations in six independent entries of C. Howeer, the nine linear equations are redundant, which are not sufficient to sole C. If two or more of such P are known, then the C can be found by using least-squares to compute an oer-determined set of equations. When C has been got, the calibration matrix K can be unique computed by the Cholesky factorization. Attention, the Cholesky factorization must require C to be positie-definite. Howeer, for the noisy input data, the C is not positie-definite when the points matching (6)

5 Camera Self-calibration Based on the Vanishing Points 51 hae gross errors. So as long as the anishing points are computed accurately, the matrix C almost is positie-definite and an accurate calibration result can be obtained. 3.4 Algorithm Idea 1. Extract points of the images and compute the anishing points with 4 directions 2. Compute the correspondent matrix P between two images with anishing points. 3. Use at least two different matrices P to sole the equation (5). 4. Find the calibration matrix K by using Cholesky factorization to decompose C. 4 Experiment Result In the experiment, Canon EOS 500D camera is used to take a set of images. The size of all images is pixels. The checkerboard grid map was employed as the calibration obect, ust because it was easy to find the anishing points in different directions. The images were all taken by the camera with different locations and orientations, as shown in Figure 2. The calibration obect should contain at least two checkerboard grid maps (neither two of these maps is parallel). Fig. 2. Calibration images of proposed method Fig. 3. Calibration images of zhang s method

6 52 D. Chang, K. Wang, and L. Wang In order to compare the performances of the proposed method, the same camera with the same internal parameters was calibrated by Zhengyou Zhang s method [10], because this method has been proed an effectie method with acceptable errors in practice. The calibration results of these two methods are summarized in table images were used for calibration in Zhang s method, and ust 4 images were used in the proposed method eery time. By comparing the calibration results of these two methods from table 1, it can be seen that the proposed method has achieed a ery good accuracy. If the result data of Zhang s method were marked the ideal data, the relatie error of the proposed method can be computed, as shown in table 2. As illustrated in table 2, the data are the relatie errors of eery element of calibration matrix in each experiment. It also proes that the proposed method has achieed a good accuracy. Table 1. The results of the calibration Method Sample k u k p u p s Zhang Figure This paper 1 7; ; ; ; ; Experiment Samples Table 2. The errors of the proposed method ku 1 7; ; ; ; ; k pu p 5 Conclusions In this article, the proposed method uses the anishing points for camera calibration which can make the camera freely moe and rotate in space. It is also independent on the proectie geometry and epipolar structure. The calibration result accuracy is depended on the corner extraction accuracy. With the increase of experimental images, the experimental results are more stable and accurate. Besides, if there is a method for coneniently computing the anishing points or inserting three or more images about the same obect during the ongoing task, the proposed method will be an online camera self-calibration approach.

7 Camera Self-calibration Based on the Vanishing Points 53 References 1. Maybank, S.J., Faugeras, O.D.: A theory of self-calibration of a moing camera. International Journal of Computer Vision 8(2), (1992) 2. Faugeras, O.D., Luong, Q.-T., Maybank, S.J.: Camera Self-calibration: Theory and experiments. In: Sandini, G. (ed.) ECCV LNCS, ol. 588, pp Springer, Heidelberg (1992) 3. Zhang, Z.: A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, (2002) 4. Dron, L.: Dynamic camera self-calibration from controlled motion sequences. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp (1993) 5. Basu, A.: Actie calibration: Alternatie strategy and analysis. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp (1993) 6. Du, F., Brady, M.: Self-calibration of the intrinsic parameters of cameras for actie ision systems. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp (1992) 7. Hartley, R.I.: Self-calibration of stationary cameras. International Journal of Computer Vision 22, 5 23 (1997) 8. Hartley, R.I.: Self-calibration from multiple iews with a rotation camera. In: Proc. of the 3rd, European Conference on Computer Vision, ol. I, pp (1994) 9. Caprile, B., Torre, V.: Using Vanishing Points for Camera Calibration. International Journal of Computer Vision 4(2), V127 V140 (1990) 10. Bouguet, J.Y.: Complete camera calibration toolbox for matlab [EB/OL], (2002), Retrieed from the World Wide Web