1D camera geometry and Its application to circular motion estimation. Creative Commons: Attribution 3.0 Hong Kong License

Transcription

1 Title D camera geometry and Its application to circular motion estimation Author(s Zhang, G; Zhang, H; Wong, KKY Citation The 7th British Machine Vision Conference (BMVC, Edinburgh, U.K., 4-7 September In Proceedings of the British Machine Vision Conference, 2006, v., p Issued Date 2006 URL Rights Creative Commons: Attribution 3.0 Hong Kong License

2 D Camera Geometry and Its Application to Circular Motion Estimation Guoqiang Zhang, Hui Zhang and Kwan-Yee K. Wong Department of Computer Science, University of Hong Kong, Hong Kong SAR, China {gqzhang, hzhang, Abstract This paper describes a new and robust method for estimating circular motion geometry from an uncalibrated image sequence. Under circular motion, all the camera centers lie on a circle, and the mapping of the plane containing this circle to the horizon line in the image can be modelled as a D projection. A 2 2 homography is introduced in this paper to relate the projections of the camera centers in two D views. It is shown that the two imaged circular points and the rotation angle between the two views can be derived directly from the eigenvectors and eigenvalues of such a homography respectively. The proposed D geometry can be nicely applied to circular motion estimation using either point correspondences or silhouettes. The method introduced here is intrinsically a multiple view approach as all the sequence geometry embedded in the epipoles is exploited in the computation of the homography for a view pair. This results in a robust method which gives accurate estimated rotation angles and imaged circular points. Experimental results are presented to demonstrate the simplicity and applicability of the new method. Introduction In computer vision and computer graphics, a CCD camera is commonly modelled as a 2D projective device. On the other hand, many imaging systems using laser beams, infra-red or ultra-sound act only on a source plane, and can be modelled as a D projective device. Many work has been done to solve the structure and motion problem for both calibrated and uncalibrated D projective cameras [2,, 5]. It has been shown that under some special situations, a 2D camera model can be reduced to a D camera to simplify the vision problem. For instance, Faugeras et al. reduced 2D images under planar motion to the trifocal line images and derived a simple solution for self-calibration [5]. This paper considers the problem of circular motion estimation. Circular motion can be viewed as a special kind of planar motion, in which the camera rotates around a fixed axis with the object remaining stationary. Under this setup, all the camera centers lie on a circle, which is the trajectory of the rotating camera center. The projection of the plane containing the camera centers onto the image plane gives a line known as the horizon. It follows that the images of the camera centers (i.e., the epipoles must all lie on this line.

3 2 This results in a D projective geometry which will be exploited here to solve the motion problem. In this paper, a 2 2 homography is introduced to relate the projections of the camera centers in two views. It will be shown that both the imaged circular points and the rotation angle between these two views can be extracted directly from this homography simultaneously. This proposed geometry can ease the job of circular motion estimation. Many studies have been conducted for circular motion [0, 6, 7, 9, 2, 4]. Traditional method obtained the rotation angles by careful calibration [0]. In [6], Fitzgibbon et al. developed a method to handle the case of uncalibrated camera with unknown rotation angles based on a projective reconstruction. Their method requires tracking points in 2 and 3 views, respectively, for estimating the fundamental matrices and the trifocal tensors. This method was further extended by Jiang et al. in [7], where the computation of the trifocal tensor is avoided. However, their method requires tracking a minimum of 2 points in 4 images. An alternative approach is to exploit the silhouettes of the object. In [2], Wong and Cipolla proposed a method which requires 2 outer epipolar tangents. Their method involves a nonlinear optimization in a high dimensional space, and requires the knowledge of camera intrinsics. In [9], Mendonça et al. proposed to recover the structure and motion in several steps, each of which only involves a low dimensional optimization. However, the camera intrinsics are still required in the procedure for recovering the rotation angles and the subsequent Euclidean reconstruction. The work in [4] is an extension of Mendonça s method, and it removes the requirement of known camera intrinsics. However, the computation of each rotation angle in this method only involves 3 views, which sometimes results in bad estimation. Our proposed D geometry can be nicely applied to solve the circular motion problem using either point correspondences or silhouettes. The key point is to obtain the epipoles of all the view pairs. For the former case, the epipoles can be extracted from the fundamental matrices computed from point correspondences. Due to the special form of the fundamental matrix under circular motion, after obtaining one fundamental matrix using general methods, the minimum data required to compute the fundamental matrix for another view pair is only one point correspondence. It is evident that our method is more flexible than the approach proposed in [7] in that it requires no tracking of point in several images. Further, our method converts the geometry embedded in the corresponding points to the epipoles, and nicely exploits them in motion recovery. As for the silhouettebased motion estimation, our method can be directly integrated with the work of [9]. The advantage of our method over the existing one is that it is intrinsically a multiple view approach as all geometric information from the whole sequence is nicely exploited. This paper is organized as follows. Section 2 briefly reviews the D camera model. The D homography relating two views in the circular motion is introduced in Section 3. Section 4 illustrates the application of our proposed method to circular motion. Experiments on 3 sequences are presented in Section 5, followed by a short conclusion in Section 6. 2 D Projective Camera This section gives a short review of the D camera model (see [5]. Similar to a 2D projective camera which projects a point in P 3 to a point in P 2, a D camera maps a point in P 2 to a point in P. A D camera can be modelled by a 2 3 matrix [5] in the

4 3 X Z Z X X X u' u camera center image pane (a(b X v u X camera center image plane Figure : (a The relative positions of a D camera and the points X and X in the coordinate system. (b Special setup of X and X in the coordinate system where X overlaps with the camera center. form P = K ( R t, ( ( f u0 where K = is the calibration matrix with focal length f and principle point 0 u0, and R and t are the rotation matrix and translation vector. R can be expressed as ( cosθ sinθ R =. sinθ cosθ The scene space for a D camera is a projective plane, and the two circular points I and J on the line at infinity l are invariant under any similarity transformation. Similarly to the 2D camera case where the image of the absolute conic can be expressed in terms of the intrinsic parameters, the imaged circular points i and j can be expressed in terms of the intrinsic parameters of a D camera. The expression of j follows directly by projecting the circular point J = ( j 0 T onto the image plane: ( f uo j = 0 ( R t j ( = e jθ u0 + j f 0. (2 Equation (2 shows that j can be formulated in terms of the camera intrinsics f and u0. The real part of the ratio of the two components is u0 and the imaginary part is the focal length f. 3 D Homography in Circular Motion Suppose the world coordinates and the camera position are as in Fig. (a. The projection matrix is given by P = KR ( I t, where t = ( 0 T. Proposition : The 2D points X, X and the camera center are on a circle, as shown in Fig. (a. The rotation angle of the two points with respect to the center of the circle is θ.

5 4 Their projections u and u in the image are related by a D homography H as follows u = Hu = KR( θ 2 K u. (3 If X overlaps with the camera center as illustrated in Fig. (b, its expression in (3 is the vanishing point v of the X-axis direction. The formula for the special case in Fig. (b is v = Hu. Proof: Image point u can be expressed as u = PX ( sinφ = KR cosφ + ( tan φ = KR 2. (4 ( Similarly the image point u can be expressed as u tan( θ+φ = KR 2 tan( θ+φ 2 = tan 2 θ +tan φ 2, the above equation can be rewritten as tan 2 θ tan φ 2 ( u = KR Substituting (4 into (5 gives ( = KR tan θ 2 + tan φ 2 tan θ 2 tan φ 2 tan θ 2 tan θ 2 cos θ = KR( 2 sin 2 θ sin 2 θ cos 2 θ = KR R( θ ( tan φ 2 2 ( tan φ 2 ( tan φ 2 u = KR R( θ 2 R K u = KR( θ 2 K u.. Since. (5 The special case where X overlaps with the camera center can be proved in a similar manner. Note that the eigenvalues of H are {e j θ 2,e j θ 2 }, which are functions of the rotation angle θ, and the eigenvectors are the two imaged circular points i and j. H has only three degrees of freedom,which indicates that only 3 constraints are required to determine it uniquely. 4 Application of the D Homography to Circular Motion Estimation Circular motion refers to the case of a stationary camera viewing an object rotating around an axis, or a camera rotating around an axis with the object being stationary. These two

6 5 X X X u u' view2 u' u view (a (b view Figure 2: (a Two views and one point configuration. View 2 is obtained by rotating view by an angle of θ. (b One view and two points configuration. The setup of X and the camera is the same as in (a. cases are geometrically equivalent and the rotating camera case is used here for the ease of explanation. As mentioned in Section, all the camera centers should lie on a circle, and the projection of the plane containing this circle onto the image plane can be modeled by a D camera projection. Consider the projection of one camera center X into two views, as shown in Fig. 2(a. The rotation angle between the two views is θ. It is easy to see that the projection of X in view 2 in Fig. 2(a is equivalent to the projection of X in view in Fig. 2(b where the rotation angle between X and X is θ. From Proposition, u and u satisfies u = Hu, where H = KR( θ 2 K. It indicates that one such pair of u and u induced by a camera center X provides a constraint on the homography H relating a pair of views. In practice, there are many images taken under circular motion. There should be enough constraints for estimating H relating any pair of views. The rotation angle and the imaged circular points can thus be recovered easily from the eigenvalues and eigenvectors of H. Now let s focus on the minimum data requirement for computing H relating a view pair. Consider two views, if the vanishing point v as illustrated in Proposition is known, the two epipoles and v together can provide 2 constraints on H independently. Under this situation, one more constraint provided by another view is enough to determine H. This should be the minimum data required for determining H. The computation of H only involves 3 views, which is the same as the work in []. Practically, H relating any view pair can be computed more accurately by incorporating constraints induced by as many views as possible. 4. Acquisition of Epipoles from Point Correspondences Previous section describes how the proposed D homography can be used to solve the circular motion problem. In this section, we will discuss how to compute this homography from the images. The key point here is to get the epipoles from the view pairs. Since the epipoles can be extracted from the fundamental matrices, the computation of epipoles relating any pair of views is equivalent to computing the associated fundamental matrix. In this section, we will summarize the idea on how to compute epipoles in an elegant algebraic description resulting in a concise computational algorithm using the information of point correspondences.

7 6 The projective geometry of circular motion has been nicely studied by Fitzgibbon et al. [6]. There are a number of fixed features under circular motion. The fundamental matrix F can be parameterized in terms of these fixed features as [6] F = [v x ] + k tan θ 2 (l sl T h + l hl T s. (6 where the fixed features v x, l h and l s are scaled to unit norm and k is an unknown scalar which is fixed throughout the sequence. l s is the image of the rotation axis, and l h is the vanishing line of the horizontal plane and also the image line of the plane containing the camera centers. v x is the vanishing point of the normal direction of the plane π d defined by the camera center and the rotation axis. Besides, the two imaged circular points on l h are also fixed features which are to be recovered from the proposed D homography. Equation (6 reveals that the fundamental matrix relating different view pairs can be obtained by changing the scalar k tan 2 θ, with θ being the rotation angle between the chosen views. Given a fundamental matrix ˆF relating a particular view pair, the fixed features v x, l s and l h can be obtained by decomposing ˆF [6]. With the resulting v x, l s and l h, only a minimum of one point correspondence is needed to compute the fundamental matrix relating any two views in the sequence. This makes the computation of fundamental matrix with large θ possible. Note that the computation of epipoles here requires no point to be tracked over several images, as opposed to the method in [7] which requires 2 points being available in 4 images. Based on the above analysis, a simple algorithm is developed, which is summarized below:. Compute one fundamental matrix from any two given views. The two selected views should be as distant as possible while sharing enough corresponding points. 2. Decompose the fundamental matrix obtained in step into the form of (6 to get v x, l s and l h. 3. Compute the fundamental matrices relating other view pairs using corresponding points. Extract the epipoles from the fundamental matrices. 4. Compute the D homography H for each neighboring view pair from the epipoles. Recover the rotation angles and imaged circular points from the eigenvalues and eigenvectors of H. 4.2 Acquisition of Epipoles from Silhouettes In [9], Mendonça et al. demonstrated that the epipoles relating any pair of views can be computed by using the information of silhouettes. The degenerate case happens only when l h passes through the silhouettes and the rotation angle from two views are close to π. Hence, the geometric information from the whole sequence can be nicely exploited in the computation of H relating a pair of views. In this section, we will briefly summarize the algorithm as described in [9]. Consider a pair of camera matrices P and P 2 with epipole e i, formed in the image of camera P i. The fundamental matrix under circular motion can be written in a plane plus parallax representation, given by F = [e 2 ] W, (7

8 7 where W = I 2 v xl T s v T x l s is the homography induced by the plane Π W that contains the axis of rotation and bisects the line segment joining the 2 camera centers [9]. Note that W is also the harmonic homology associated with image of the surface of revolution swept by the rotating object [3]. The imaged rotation axis l s becomes the image of the axis of the surface of revolution. It has been shown in [9] that given a dense image sequence taken under complete circular motion, say that the rotation angles are about 5, the imaged envelope, obtained by overlapping all the images, can be viewed as an imaged surface of revolution. As indicated in [3], the associated harmonic homology W or equivalently l s and v x can be estimated from this imaged surface of revolution by intersecting the bitangents to the image profile. Since W satisfies W = W, it follows that the corresponding epipolar lines l and l 2 are mapped by W T mutually, as l 2 = W T l. (8 Note that the outer epipolar tangents are special epipolar lines in that they are tangent to the silhouettes of the object. This geometric constraint together with (8 makes the computation of the two outer epipolar tangents possible. The search for one outer epipolar tangent proceeds as a one-dimensional optimization of its orientation (see [9] for details. The epipoles can be directly computed as the intersection of the two recovered outer epipolar tangents, and l h can then be obtained by robustly fitting a line to these estimated epipoles. Finally, the epipoles can be refined by constraining them to lie on l h. Given the epipoles, the D homography can be recovered and the solution to the motion problem follows. 5 Experimental Results The new method of computing rotation angles and imaged circular points from D projective geometry is implemented and tested using both the information of point correspondences and silhouettes, respectively. 5. Point-Based Experiment The sequence tested is the popular dinosaur sequence from the University of Hannover. It contains 36 images taken under circular motion with a constant rotation angle of 0. The angular accuracy is about 0.05 [0]. The fundamental matrix F from the 7th and 0th images was computed to extract the fixed features. To improve the antinoise ability, robust linear regression algorithm was exploited to estimate the homograhy. Since the D imaged circular points (a ± jb, T should be fixed over the sequence, they were estimated from the histograms of the components a and b obtained by randomly selecting the view pairs in the computation of the D homography, as shown in Fig. 3. The distributions are close to a Gaussian distribution. Note that the best solutions are very distinct which are denoted by the dashed lines. The final computed results of the 2D imaged circular points are (254.2 ± j, j, T. To get the best solution of each rotation angle, the optimized imaged circular points were exploited. Note that the D homography H relating a view pair has only 3 degrees of freedom. This number is far less than the constraints on H provided by the projections

9 8 Figure 3: Histogram of a and b which are the coordinates of the D imaged circular points (a ± jb, T. The dashed lines indicate the best solutions computed. Figure 4: (a Recovered rotation angles for the dinosaur sequence. (b Recovered rotation angles for the David sequence. of other camera centers. For a pair of neighboring views, H was computed many times, each time with a portion of the constraints randomly selected from the whole set. The homography that gives the imaged circular points closest to the optimized one was taken as the best solution. The rotation angle was then extracted from it. Figure 4(a gives the recovered angles for the whole sequence. The RMS error is 0.7, which indicates the estimation was very good. Assuming that the camera satisfies unit aspect ratio and zeroskew restriction, the intrinsics of the camera were estimated from the imaged circular points and the constraints provided by l s and v x [4]. The visual hull of the dinosaur were computed as in [8], and three views of the 3D model are shown in Fig Silhouette-Based Experiment In this experiment, the silhouettes are represented by the Cubic B-spline snake [3]. It provides a compact representation of silhouettes of various complexity, and can achieve sub-pixel localization accuracy. The epipoles were estimated using the method as described in Section 4.2. After obtaining the epipoles, the remaining steps are similar to the point-based experiment. Experiment on a David sequence was carried out to test the applicability of our proposed approach. The sequence consists of 72 images with a resolution of , acquired with an electronic turntable. The rotation angle between successive views is 5 with a resolution of 0.2. Figure 6 shows two selected images. The distributions of the histograms on the estimated components of the D circular points are similar to those

10 9 Figure 5: 3D reconstruction of the dinosaur using motion estimated from point correspondences. Figure 6: Two images of David under circular motion. of the dinosaur sequence. Due to space limit, it is not shown here. In the procedure of rotation angle estimation and 3D reconstruction, a subsequence of 36 views were selected from the whole set with a rotation angle of 0. The estimated rotation angles are shown in Fig. 4(b, with a RMS error of 0.2. Some views of the 3D model constructed are shown in Fig. 7. The experiments on other sequences were also performed, and the results were impressive. 6 Conclusion In this paper, we have introduced a D homography between two D projective cameras, induced by the points lying on a circle passing through the two camera centers. We have shown that the eigenvectors of such a homography are the imaged circular points and the rotation angle between the two cameras with respect to the center of the circle can be extracted from the eigenvalues of the homography. We have demonstrated that this D geometry can be nicely applied to circular motion estimation. Circular motion is a special kind of planar motion with all the camera centers lying on a circle. This configuration exactly satisfies the requirement of the proposed D geometry. Experiments using both point correspondences and silhouettes, respectively, demonstrate that the recovered rotation angles achieve a very high precision. This is because that the algorithm efficiently and effectively exploits the underlying multiple view information. The high quality of the reconstructed models demonstrates the practicality of our work. References [] M. Armstrong, A. Zisserman, and R. Hartley. Self-calibration from image triplets. In B. Buxton and R. Cipolla, editors, Proc. 4th European Conf. on Computer Vision, volume 064

11 0 Figure 7: 3D reconstruction of David using motion estimated from silhouettes. of Lecture Notes in Computer Science, pages 3 6, Cambridge, UK, April 996. Springer Verlag. [2] K. Astrom and M. Oskarsson. Solutions and ambiguities of the structure and motion problem for d retinal vision. Journal of Mathematical Image and Vision, 2(2:2 35, April [3] R. Cipolla and A. Blake. Surface shape from the deformation of apparent contours. Int. Journal of Computer Vision, 9(2:83 2, November 992. [4] C. Colombo, A. del Bimbo, and F. Pernici. Metric 3d reconstruction and texture acquisition of surfaces of revolution from a single uncalibrated view. PAMI, 27(:99 4, January [5] O.D. Faugeras, L. Quan, and P.F. Sturm. Self-calibration of a d projective camera and its application to the self-calibration of a 2d projective camera. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(0:79 85, October [6] A. W. Fitzgibbon, G. Cross, and A. Zisserman. Automatic 3D model construction for turntable sequences. In R. Koch and L. Van Gool, editors, 3D Structure from Multiple Images of Large-Scale Environments, European Workshop SMILE 98, volume 506 of Lecture Notes in Computer Science, pages 55 70, Freiburg, Germany, June 998. Springer Verlag. [7] G. Jiang, L. Quan, and H.T. Tsui. Circular motion geometry by minimal 2 points in 4 images. In Proc. 9th Int. Conf. on Computer Vision, pages , [8] C. Liang and K.-Y. K. Wong. Complex 3d shape recovery using a dual-space approach. In Proc. Conf. Computer Vision and Pattern Recognition, pages II: , [9] P. R. S. Mendonça, K.-Y. K. Wong, and R. Cipolla. Epipolar geometry from profiles under circular motion. IEEE Trans. on Pattern Analysis and Machine Intelligence, 23(6:604 66, June 200. [0] W. Niem. Robust and fast modelling of 3d natural objects from multiple views. In SPIE Proceedings - Image and Video Processing II, volume 282, pages , 994. [] L. Quan and T. Kanade. Affine structure from line correspondences with uncalibrated affine cameras. IEEE Trans. on Pattern Analysis and Machine Intelligence, 9(8: , 997. [2] K.-Y. K. Wong and R. Cipolla. Structure and motion from silhouettes. In Proc. 8th Int. Conf. on Computer Vision, volume II, pages , Vancouver, BC, Canada, July 200. [3] K.-Y. K. Wong, P. R. S. Mendonça, and R. Cipolla. Camera calibration from symmetry. In R. Cipolla and R. Martin, editors, Proc. 9th IMA Conference on the Mathematics of Surfaces, pages , Cambridge, UK, September Springer Verlag. [4] H. Zhang, G. Zhang, and K.-Y. K. Wong. Auto-calibration and motion recovery from silhouettes for turntable sequences. In Proc. British Machine Vision Conference, volume I, pages 79 88, Oxford, UK, September 2005.