Fitting conics to paracatadioptric projections of lines

Transcription

1 Computer Vision and Image Understanding 11 (6) Fitting onis to paraatadioptri projetions of lines João P. Barreto *, Helder Araujo Institute for Systems and Robotis, Department of Eletrial and Computer Engineering, University of Coimbra, Coimbra, Portugal Reeived 1 May ; aepted 1 July Available online Otober Abstrat The paraatadioptri amera is one of the most popular panorami systems urrently available in the market. It provides a wide field of view by ombining a paraboli shaped mirror with a amera induing an orthographi projetion. Previous work proved that the paraatadioptri projetion of a line is a oni urve, and that the sensor an be fully alibrated from the image of three or more lines. However, the estimation of the oni urves where the lines are projeted is hard to aomplish beause of the partial olusion. In general only a small ar of the oni is visible in the image, and onventional oni fitting tehniques are unable to aurately estimate the urve. The present work provides methods to overome this problem. We show that in unalibrated paraatadioptri views a set of oni urves is a set of line projetions if and only if ertain properties are verified. These properties are used to onstrain the searh spae and orretly estimate the urves. The oni fitting is solved naturally by an eigensystem whenever the amera is skewless and the aspet ratio is known. For the general situation the line projetions are estimated using non-linear optimization. The set of paraatadioptri lines is used in a geometri onstrution to determine the amera parameters and alibrate the system. We also propose an algorithm to estimate the oni lous orresponding to a line projetion in a alibrated paraatadioptri image. It is proved that the set of all line projetions is a hyperplane in the spae of oni urves. Sine the position of the hyperplane depends only on the sensor parameters, the auray of the estimation an be improved by onstraining the searh to onis lying in this subspae. We show that the fitting problem an be solved by an eigensystem, whih leads to a robust and omputationally effiient method for paraatadioptri line estimation. Ó Elsevier In. All rights reserved. Keywords: Catadioptri; Paraatadioptri; Omnidiretional vision; Calibration; Lines; Line estimation 1. Introdution The approah of ombining mirrors with onventional ameras to enhane the sensor field of view is referred as atadioptri image formation. The use of atadioptri systems to ahieve panorami vision is simple and fast enabling the apture of dynami senes. The entire lass of atadioptri onfigurations satisfying the single viewpoint onstraint is derived in [1]. Panorami entral atadioptri systems an be built by ombining a hyperboli mirror with a perspetive amera, and a paraboli mirror with an orthographi amera (paraatadioptri sensor). * Corresponding author. Fax: addresses: jpbar@dee.u.pt (J.P. Barreto), helder@isr.u.pt (H. Araujo). URL: (J.P. Barreto). The onstrution of the former requires a areful alignment between the mirror and the imaging devie. The amera projetion enter must be positioned in the outer fous of the hyperboli refletive surfae. The paraatadioptri amera is easier to onstrut being broadly used in appliations requiring omnidiretional vision [ 8]. In[9], Geyer and Daniilidis introdue for the first time a unifying theory for general entral atadioptri image formation. A modified version of this mapping model is proposed in [1]. It is shown that entral atadioptri projetion is isomorphi to a projetive mapping from a sphere, entered in the effetive viewpoint, to a plane with a projetion enter on the perpendiular to the plane. For the partiular ase of paraatadioptri sensors the projetion enter lies on the sphere and the projetive mapping is a stereographi projetion. The plane and the final atadioptri image are related by a ollineation depending on the mirror and 177-1/$ - see front matter Ó Elsevier In. All rights reserved. doi:1.116/j.viu..7.

2 1 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) amera intrinsi parameters. The system is alibrated when this ollineation is known. It has already been proved that the entral atadioptri projetion of a line is a oni urve [11,9,1], and that any entral panorami system an be fully alibrated from the image of three lines in general position [1]. However, sine lines are mapped into oni urves whih are only partially visible, the aurate estimation of atadioptri line images is far from being a trivial task [1,1]. The present work addresses the problem of aurately estimate paraatadioptri lines using image points. Several authors have already proposed algorithms to alibrate a paraatadioptri amera [16 18]. The approah presented in [17] requires a sequene of paraatadioptri images. The system is alibrated using the onsisteny of pair wise traked point features aross the sequene, based on the harateristis of atadioptri imaging. In [18], the enter and foal length are determined by fitting a irle to the image of the mirror boundary. The method is simple and an be easily automated, however it is not very aurate and requires the visibility of the mirror boundary. Its major drawbak is that it is only appliable to the situation of a skewless amera with unitary aspet ratio. Geyer and Daniilidis [16] propose an algorithm to alibrate the sensor from an image of at least three lines. They present a losedform solution for foal length, image enter, and aspet ratio for skewless ameras, and a polynomial root solution in the presene of skew. Certain properties of paraboli projetion are used to get aurate line estimates, but the oni urves verifying these properties are not neessarily the paraatadioptri projetion of lines. This has an impat on the global performane of the method as will be disussed in Setion. We derive for the first time the neessary and suffiient onditions that must be satisfied by a set of oni urves to be the paraatadioptri projetion of lines. The derived onditions an be used to aurately estimate the line images using non-linear optimization. Moreover, if the system is skewless and the aspet ratio is known, then the line projetions an be omputed by solving an eigensystem. Given the image of at least three lines the paraatadioptri amera is easily alibrated using the geometri onstrution proposed in [1]. The alibration algorithm is evaluated using both syntheti and real images. The experimental results show that it out performs the method proposed in [16]. Additionally, we present a oni fitting algorithm that opes with the olusion problem and aurately estimates the paraatadioptri image of single lines. The method is speifi for line projetions in paraboli systems and requires the sensor to be alibrated. We prove that a oni urve is the paraatadioptri image of a line if and only if the image of the irular points lie on the urve, and two speifi points are harmoni onjugate with respet to the oni. The paraatadioptri amera maps lines in the sene into oni urves lying in a hyperplane in the spae of all onis. Therefore, the line projetion an be aurately determined by onstraining the searh spae. The estimation algorithm is stable and omputationally effiient beause the fitting problem an be solved by an eigensystem. The proposed method is useful for many appliations suh as D reonstrution and visual ontrol of motion using paraatadioptri images. Experimental results show that the approah is very robust and the estimation results are muh better than the ones obtained by performing perspetive retifiation [].. Paraatadioptri projetion of lines A general mapping model for entral atadioptri systems has been introdued for the first time in [9]. This setion briefly reviews the image formation model for the partiular ase of paraatadioptri ameras (for a detailed derivation see [1]). The equations for paraatadioptri line projetion are derived, and it is shown that the image of a line is a oni urve [9,11]. The estimation of these oni urves is in general hard to aomplish due to partial olusion. We ompare and disuss the performane of five standard oni fitting methods..1. Paraatadioptri projetion model Assume a paraatadioptri system ombining a paraboli mirror, with latus retum p, and an orthographi amera. The prinipal axis of the amera is aligned with the symmetry axis of the paraboloid. The paraatadioptri projetion an be modeled by a stereographi projetion from an unitary sphere, entered in the effetive viewpoint, into a plane P 1 as shown in Fig. 1. The mapping an be desribed as follows. Eah visible sene point defines an oriented projetive ray x =(x,y,z) t, joining the D point with the projetion enter O. Consider Π world point x x Ω Ω P X n X Y Y O Z Z O O Ο H Π x Paraatadioptri Image Plane Fig. 1. Paraatadioptri projetion model. The unitary sphere is entered at point O and P is the point where the projetive ray x intersets the sphere. The new projetive point x is defined by O and P. The paraatadioptri image point x is related to x by a projetive transformation H.

3 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) the south pole of the sphere O with oordinates (,, 1) t. To eah x orresponds a projetive ray x going through O and P (the intersetion point of x and the sphere). This mapping between projetive points is non-linear as shown in Eq. (1). Funtion is equivalent to projeting the sene on the unitary sphere, followed by a re-projetion from the sphere to the plane P 1 with enter O. Points in atadioptri image plane x are obtained after a ollineation H of the D projetive points x. Eq. () shows that H is always an affine transformation depending on the amera intrinsi parameters K, and the latus retum of the paraboli mirror. p hðxþ ¼ðx; y; z þ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi x þ y þ z Þ t ; ð1þ p 6 7 x ¼ K p x. ðþ 1 fflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflffl} H Consider plane P =(n,) t going through the effetive viewpoint O as depited in Fig. 1 (n =(n x,n y,n z ) t ). The paraatadioptri image of any line lying in P is the oni urve X. The line in the sene is projeted into a great irle on the sphere surfae. This great irle is the urve of intersetion of plane P, ontaining both the line and the projetion enter O, and the unit sphere. The projetive rays x, joining O with points on the great irle, form a entral one surfae. The entral one, with vertex at O, projets into the oni X in plane P 1 Eq. (). Sine the image plane and P 1 are related by ollineation H, the result of Eq. () omes in a straightforward manner. n z n x n z 6 X ¼ n 7 z n y n z ; ðþ n x n z n y n z n z a b d 6 7 X ¼ b e ¼ H t d e f XH 1. ðþ The paraatadioptri system is alibrated whenever the ollineation H is known. Assume that the image enter of the orthographi amera is C =( x, y ) t, and that r, f o, and s k are respetively the aspet ratio, the foal length and the skew. Sine the matrix of intrinsi parameters K is upper triangular, then H is always an affine transformation (Eq. ()). Matrix H is provided in Eq. () where f =f o p is a measurement in pixels of the ombined foal length of the amera and the mirror. As a final remark notie that sine X is a irle, then the oni urve X is in general an ellipse (Eqs. () and ()). Moreover, the paraatadioptri image of a line is a irle if and only if the orthographi amera has unitary aspet ratio (r = 1). H ¼ 6 r f s k x r 1 7 f y. ðþ 1.. Estimation of paraatadioptri line images using standard oni fitting methods Eq. () shows that the paraatadioptri projetion of a line is a oni urve whih is parameterized by a symmetri matrix X. Sine a oni has five independent degrees of freedom (DOF), it an also be represented by a point x in the D projetive spae } (Eq. (6) [19]). Heneforth, we will assume both representations without distintion. The present setion disusses the problem of fitting oni urves to image points to estimate the loi where lines are projeted. A oni fitting algorithm determines the urve that best fits the data points aording to a ertain distane metri. We review some standard oni fitting methods [1,1] and evaluate their performane in estimating paraatadioptri line projetions. x ¼ða; b; ; d; e; f Þ t. ð6þ The algebrai distane a i from the image point x i ¼ðx i ; y i Þ to the oni x is defined by a i ¼ ax i þ bx iy i þ y i þ dx i þ ey i þ f. ð7þ Consider the set of N distint image points x 1 ; x...x N. The orresponding vetor of algebrai distanes is ða 1 ; a ;...; a N Þ t ¼ Ax where A is the following N 6 matrix x 1 x 1y 1 y 1 x 1 y 1 1 x x y y x y 1 A ¼ ð8þ 6 7 x N x Ny N y N x N y N 1 The sum of the square of the algebrai distanes between the data points and the oni urve x is /ðxþ ¼ XN i¼1 a i ¼ x t A t Ax. The algebrai distane a i is zero whenever point x i lies on the oni urve x. Thus, if x 1 ; x...x N are points on the oni then matrix A is rank defiient and x is the respetive right null spae. In general the data points are noisy and A is full rank. In this ase the square matrix A t A is non-singular and funtion /ðxþ has a single root x ¼. There are several methods that minimize the objetive funtion of Eq. (9) to fit a a oni to the data points. The solution x ¼ is a global minimum of the funtion that must be avoided. The following algorithms differ in the way that the searh spae is onstrained. ð9þ (1) The normal least squares (LMS) method determines the unit vetor x whih minimizes the sum of the square distanes to the data points. The ost funtion is / lms ðx; kþ ¼/ðxÞþkðx t x 1Þ where k is a Lagrange multiplier. The minimizer is the eigenvetor orresponding to the smallest eigenvalue of matrix A t A [].

4 1 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) () The approximate mean square (AMS) metri was introdued by Taubin [1]. The AMS method minimizes the algebrai distane / under the onstraint x t ða t x A x þ A t y A yþx ¼ 1 where A x and A y are the partial derivatives of A. In this ase the minimizer is determined by solving the generalized eigensystem A t Ax ¼ kða t x A x þ A t y A yþx. The oni urve estimate is provided by the eigenvetor orresponding to the smallest eigenvalue. () In the method proposed by Fitzgibbon and Fisher (FF) the searh spae is onstrained to the spae of the ellipses []. The algorithm finds the ellipse x that minimizes the algebrai distane to the data points. The minimization an be stated as a generalized eigenproblem with a losed form solution. The data points are usually obtained using an image proessing algorithm (edge detetion, ontours, et). It is reasonable to assume that the ation of the noise is similar for all points and independent from one point to another [1]. Let the error in a generi point x i ¼ðx i ; y i Þ t be Gaussian with zero mean and ovariane matrix r I (I is the identity matrix). Therefore, the noise variane in the algebrai distane is r i ¼r i r, with $ i denoting the salar Laplaian of a i Eq. (7). The algorithms enumerated above provide the optimal solution in terms of the minimum ovariane if and only if the N equations a i = have the same variane and are statistially independent [,]. Sine the Laplaian $ i is a funtion of the point oordinates, the varianes r i are not equal and the estimation results obtained using LMS, AMS, and FF are statistially biased []. This problem an be avoided by applying the following algorithm. () The gradient weighted least square fitting algorithm (GRAD) divides the algebrai distanes a i by the salar Laplaian $ i to normalize the varianes r i. The orresponding objetive funtion / grad is stated below. In this ase, the solution an not be found solving an eigensystem and the problem has no longer a losed form solution. The minimization of / grad must be performed using iterative gradient desent methods suh as Gauss Newton or Levenberg Marquardt [,]. / grad ðxþ ¼ XN i¼1 a i r i ¼ XN i¼1 oa i ox i a i. þ oa i Methods based on algebrai distanes, like the LMS, AMS, and FF algorithms, have a losed form solution beause the estimation problem an be naturally solved by an eigensystem. However, eah data point may ontribute differently to the parameter estimation depending on its position on the oni. The problem of statistial bias is avoided in the GRAD method, but the objetive funtion is not invariant under Eulidean transformations whih oy i auses undesired effets [1]. The following algorithm uses the geometri distane and the orresponding estimation results are invariant to rotation and translation. () The ORTHO method minimizes the sum of the square of the orthogonal distanes b i between the oni and the points (for further details about omputing the orthogonal distanes please onsult the [1,1]). The objetive funtion is / ortho and the minimum solution is determined using an iterative gradient desent method []: / ortho ðxþ ¼ XN b i. i¼1 Fig. A shows the robustness to noise of the desribed oni fitting methods. The performane suffers a graeful degradation in the presene of inreasing noise. The GRAD and ORTHO algorithms are learly more robust than the methods based on algebrai distanes. Among the methods with losed form solution, the FF algorithm seems to be the most robust. In the experiment of Fig A the data points are distributed over the the entire oni. Fig. B shows the performane of the different algorithms when the oni urve is partially oluded. In this experiment the noise standard deviation is kept onstant (r = pixel), and the samples are extrated from a partial ar with a ertain amplitude. As expeted, an inrease in the angle of olusion orresponds to a derease in the performane of the estimators. All methods perform poorly when the amplitude of the olusion is above. None of the algorithms provide an useful estimate when the visible ar is less than 1. Setion.1 proves that a line in the sene is projeted into a oni lous in the paraatadioptri image plane. However, in most real images of lines, only a small oni ar is atually visible. Taking into aount that in average the visible ar of a paraatadioptri line projetion has an amplitude below, we may onlude that these standard oni fitting tehniques are unsuitable to estimate the oni lous where lines are mapped.. Paraatadioptri amera alibration using lines Any entral atadioptri system an be fully alibrated using the image of a minimum of three lines in general position [1,1,16]. In this work, we fous on the partiular ase of alibrating a paraatadioptri amera. Given the image of the lines, matrix H (Eq. ()) an be determined using the geometri onstrution proposed in [1]. This approah is straightforward whenever the onis orresponding to the line projetions are aurately known. In the previous setion we saw that, due to partial olusion, the estimation of the onis using image points is hard to aomplish. To solve this problem, we derive for the first time the neessary and suffiient onditions that must be met by a set of oni

5 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) A Mean Error (pixel).. 1. ERROR FOR THE PRINCIPAL POINTS LMS AMS FF GRAD ORTHO B Mean Error (pixel) LMS AMS FF GRAD ORTHO ERROR FOR THE PRINCIPAL POINTS Oluded Ar (degree) Fig.. Comparing the performane of standard oni fitting algorithms using syntheti data. The ar of the test oni is uniformly sampled by 1 points. Two-dimensional Gaussian noise with zero mean and standard deviation r is added to eah sample point used in the estimation. The prinipal points of the estimated urve are ompared with the ground truth and the mean error is omputed over 1 runs of eah experiment. (A) Robustness to noise. (B) Robustness to partial olusion. urves to be the paraatadioptri projetion of lines. These onditions an be used to aurately estimate the line projetions using non-linear minimization. Moreover, if the system is skewless and the aspet ratio is known, then the lines an be omputed by solving an eigensystem..1. Calibration by geometri onstrution Table 1 summarizes the steps to alibrate a paraatadioptri system from the image of K lines in general position. Fig. is a sheme of the required geometri onstrution when the number of lines is minimum (K = ). Assume that the K lines are projeted into the set of oni urves X 1 ; X,... X K. Any two line projetions X i ; X j interset in two real points B ij ; F ij. The image enter O must always lie on the line l ij defined by the intersetion points B ij and F ij [16,1]. Consider the plane P i ontaining both the original D line and the effetive viewpoint O (Fig. 1). If the line is projeted into the oni X i, then the polar line p i with respet to O is the image of the vanishing line of P i. Line p i intersets the oni urve X i in two points I i and J i. It an be proved that these two points lie on the oni X 1, whih is the lous where the absolute oni is mapped by ollineation H [1]. Coni X 1 an be estimated using the K pairs of points I i ; J i (K P ). Sine H is an upper triangular matrix Eq. () and X 1 ¼ H t H 1, then H an be determined from the Cholesky deomposition of X 1... Properties of a set of paraatadioptri line images A oni urve has five DOF and it an be represented either by a symmetri matrix X, or by a point x in P (Eqs. () and (6)). Replaing X and H in Eq. () by the results of Eqs. () and () yields a a 6 b rs k 7 6 f a 7 x ¼ d ¼ 6 7 e 6 f with 8 a ¼ n z ; r f >< d ¼ nxnz >: e ¼ rnynz f ð r s k f r f n z ðrs k y f xþ r f þ r Þa d e r f a xd y e þ r n z y s kn xn z f ; 7 þ s kn z ðrs k y f xþ r f. ð1þ The paraatadioptri image of a line depends both on the system intrinsi parameters and the orientation of the D plane P (Fig. 1). Assume K lines in the sene that are projeted in the set of oni urves x i x i ¼ða i ; b i ; i ; d i ; e i ; f i Þ t ; i ¼ 1; ; ;...; K. ð11þ Table 1 Calibration of a paraatadioptri system using K lines (K P ). Step 1 Determine the atadioptri line images X i for i =1,,,...,K Step For eah pair of onis X i, X j, ompute the intersetion points F ij, B ij, and determine the orresponding line l ij ¼ F ij B ij Step Estimate the image enter O whih is the intersetion point of lines l ij Step For eah oni X i ompute the polar line p i of the image enter O (i =1,,,...,K). Step For eah oni urve obtain the points I i and J i where line p i intersets X i (i =1,,,...,K) Step 6 Estimate the oni X 1 going through points I i, J i (i =1,,,...,K) Step 7 Perform the Cholesky deomposition of X 1 to estimate matrix H For a detailed proof of the method please onsult [1,1].

6 16 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) π D 1 µ D J Ω F I µ 1 I B1 B 1 F 1 O I 1 B π π 1 Ω 1 F 1 J J 1 D 1 µ 1 Ω Fig.. Geometri onstrution to alibrate a paraatadioptri amera from the projetion of three lines in general position. The lines are mapped into the onis X 1, X, and X that must be aurately determined. From Eq. (1) follows that b 1 ¼ b ¼ b ¼¼ b K ¼ r s k ; a 1 a a a K f 1 ¼ ¼ ¼¼ K ¼ r s k þ r a 1 a a a K f. with 8 >< >: a ¼ n z ; r f d ¼ nxnz r f n z ðrs k y f xþ r f e ¼ rnynz f þ r n z y s kn xn z f ; þ s kn z ðrs k y f xþ r f. From the first expression it follows that g i = for i =,,...,K, withg i obtained from Eq. (1). Moreover, using the seond expression in a similar manner, one obtains v i = for i =,,...,K with v i given by Eq. (1). g i ¼ a 1 b i a i b 1 ; i ¼ ; ;...; K; ð1þ v i ¼ a 1 i a i 1 ; i ¼ ; ;...; K. ð1þ From the result of Eq. (1) it follows that eah line projetion x i verifies r f a i þ x d i þ y e i þ f i ¼. Consider the first three elements x 1 ; x,andx, of the set of line projetions. The parameters r f, x, and y an be estimated from r f a 1 d 1 e 1 f x ¼ a d e f. y a d e fflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflffl} U 1 f fflffl{zfflffl} C Therefore eah oni urve x i,withi =,...,K, must verify the onstraint m i = Eq. (1). " # U 1 C m i ¼ ½a i d i e i f i Š. ; i ¼ ;...; K. ð1þ 1 If a set of K oni urves orresponds to the paraatadioptri projetion of K lines, then g i, v i, and m i, provided in Eqs. (1) (1), must be equal to zero. We have derived K independent onditions that are neessary for a set of K oni urves to be the paraatadioptri projetion of a set of K lines in the sene. However it has not been proved that these onditions are also suffiient. By suffiient we mean that, if a ertain set of oni urves satisfies these onditions then it an be the paraatadioptri projetion of a set of lines. Consider the unalibrated image of K lines that are mapped in the same number of onis. Sine eah oni has five DOF then a set of K onis has a total of K DOF. Eah line introdues two unknowns (DOF), whih orrespond to the orientation of the assoiated plane P (see Fig. 1). Moreover, the five parameters of matrix H are also unknown Eq. (). Therefore, we have a total of K + unknowns (DOF). Sine K >K + then it is obvious that there are sets of oni urves that an never be the paraatadioptri projetion of lines. The onis that an orrespond to the image of the lines lie in a subspae of dimension K +. This means that there are K independent onstraints, whih proves the suffiieny of the onditions derived above... Estimation of a set of K paraatadioptri line images Consider the image of K lines aquired by a non-alibrated paraatadioptri sensor. The lines are mapped into a set of K oni urves x i Eq. (11) that we aim to estimate. To eah urve x i orresponds a set of image points x i j, with

7 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) j = 1,,...,N i and N i P. The data points are used to build a matrix B with the sub-matries A i similar to matrix A of Eq. (8). A 1 A B ¼ A ð1þ 6 7 A K The sum of the square of the algebrai distanes between the data points and the set of onis x i is provided by funtion (p) with p ¼ðx t 1 ; xt ;...; xt K Þt ðpþ ¼p t B t Bp. ð16þ As disussed in Setion., the set of oni urves an be estimated by finding the minimum of funtion. The problem is that in general the oni urves orresponding to the paraatadioptri projetion of lines are strongly oluded in the image. The standard oni fitting tehniques do not work properly in these onditions sine the data points do not provide enough information to orretly estimate the onis. We propose to use the derived neessary and suffiient onditions to onstrain the searh spae and improve the estimation results...1. General ase In general nothing is known about the system alibration. The skew s k an be non-null and the aspet ratio r an be different from one (Eq. ()). Aording to Setion., a set of K oni urves p ¼ðx t 1 ; xt ;...; xt K Þt is the paraatadioptri projetion of a set of lines if and only if the onstraints of Eqs. (1) (1) are verified. We aim to determine the 6k 1 vetor p, that minimizes funtion, and verifies g i =, v i = and m i =. The onstraints an be introdued using a Lagrange multiplier k, and the objetive funtion is! g ðp; kþ ¼ðpÞþk XK g i þ XK v i þ XK. ð17þ i¼ i¼ The minimization of g an be stated as a non-linear least squares problem, and the solution found using Gauss Newton or Levenberg Marquardt algorithms [,].... Skewless images with known aspet ratio Assume that the orthographi amera is skewless and that the aspet ratio r is known. Replaing s k by in Eq. (1) yields b = and ¼ r a. The onstraints g i = and v i = for i =,...,K, beome b i = and i r a i ¼ for i =1,...,K. There are two additional onstraints beause two of the alibration parameters are known. A new objetive funtion s is derived from Eq. (17)! s ðp; kþ ¼ðpÞþk XK b i þ XK ð i r a iþ þ XK. i¼1 i¼1 i¼ m i i¼ m i ð18þ Eah urve in the set has a matrix A i assoiated with it (Eq. (8)). Sine the amera is skewless and the aspet ratio is known, then b i = and i ¼ r a i. Omitting the seond olumn of A i and adding the third olumn, multiplied by r,to the first olumn yields x 1 þ r y 1 x 1 y 1 1 x _A þ r y x y 1 i ¼ x N i þ r y N i x N i y N i 1 The sum of the squares of the algebrai distanes between the oni urve and the data points is _ / i ¼ _x t i _ A t i _ A i _x i with _x i ¼ða i ; d i ; e i ; f i Þ t. By ombining the K onis we obtain the sum of squares of the algebrai distanes between the set of urves and the data points. This sum is given by the funtion _ðqþ ¼q t B _ t Bq; _ ð19þ where matrix B _ is obtained by replaing A i by A _ i in Eq. (1), and q ¼ð_x t 1 ; _xt ;... _xt K Þt. Sine B _ impliitly enodes the onstraints b i = and i r a i ¼, the objetive funtion s of Eq. (18) an be rewritten as _ s ðq; kþ ¼_ðqÞþk XK i¼ m i. ðþ As disussed in Setion., the eigenvetor orresponding to the smallest eigenvalue of matrix _ B t _ B is the solution q that minimizes _ under the onstraint q t q =1 []. If K = then the seond term of Eq. disappears and the minimization problem has a losed form solution. If K > then the minimum of funtion _ s must be found using an iterative gradient desent method. In this situation the eigenvetor solution an be used as an initial estimate.. Diret least square fitting of paraatadioptri line images The paraatadioptri image of a line is a oni urve that in general is hard to estimate due to partial olusion. Setion shows that a set of oni urves must verify ertain properties to be a oherent paraatadioptri projetion of a set of lines. The onditions of Eqs. (1) (1) depend neither on the system alibration nor on the D position of the lines. These onditions are used to onstrain the searh spae and estimate the set of oni urves in the unalibrated image. The set of line images is used to alibrate the paraatadioptri amera using the geometri onstrution presented in [1,1]. While Setion fouses in the estimation of a set of line projetions in an unalibrated paraatadioptri image, the present setion disusses the problem of determining the projetion of a single line in a alibrated paraatadioptri view. Considering the spae of all oni urves, it is proved that the paraatadioptri projetion of any line lies in a hyperplane defined by the alibration parameters. The line image an be estimated by fitting the data points by a oni

8 18 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) lying in this linear subspae. The proposed approah is omputationally effiient beause the fitting problem an be solved by an eigensystem..1. The neessary and suffiient onditions The sheme of Fig. 1 shows the paraatadioptri projetion of a line. The line lies in a plane P going through the effetive viewpoint O. The mapping from the sphere into the plane P 1 is a stereographi projetion. Plane P intersets the sphere in a great irle that is projeted into a irle X Eq.. Points in plane P 1 are mapped into points in the image by an affine transformation H Eq. (). Sine an affine transformation does not hange the type of oni, then the paraatadioptri projetion of a line X is always a irle/ellipse Eq. (). Consider the following points lying in plane P 1 : I1 ¼ð1; i; Þ t ; J 1 ¼ð1; i; Þ t ; G 1 ¼ð1; ; iþ t ; H 1 ¼ð1; ; iþ t. Assume that the paraatadioptri system is alibrated and the affine transformation H is known. The above points are mapped in the paraatadioptri image plane in points: I1 ¼ H I1 ¼ði x ; i y ; i z Þ t ; J 1 ¼ H J1 ¼ðj x ; j y ; j z Þ t ; G 1 ¼ H G1 ¼ðg x ; g y ; g z Þ t ; H 1 ¼ H H1 ¼ðh x ; h y ; h z Þ t. ð1þ Using these points we an state the following proposition Proposition 1. A oni urve X is the paraatadioptri image of a line in the sene if and only if it ontains points I1 and J 1 ði t XI 1 1 ¼ ; J t XJ 1 1 ¼ Þ, and points G 1, H 1 are onjugate with respet to Xð G t X H 1 1 ¼ Þ. Proof. Consider the oni urve X ¼ H t XH, lying in plane P 1 (Fig. 1). Coni X is a funtion of the normal n to the plane P Eq. (). Sine X is a irle, then it must go through the irular points I 1 and J 1. Moreover, from Eq. (), it follows that points G 1 and H 1 are always harmoni onjugate with respet to oni X. Remark that these properties are independent of the orientation n of plane P. Sine ollineation H preserves inidene and harmoni relations, then oni X must satisfy It XI ¼ ; Jt XJ ¼ and Gt X H ¼. The onditions derived are neessary, nevertheless it is not lear that they are suffiient. By suffiient we mean that if a oni urve verifies these three onstraints, then it is the lous where a ertain line in the sene is projeted. By negleting the sale fator, the oni urve X of Eq. () is a funtion of two independent parameters. These two degrees of freedom (DOF) are assoiated with the pose of plane P ontaining both the original line and the effetive viewpoint (Fig. 1). Sine in general a oni urve has five DOF, then we must be able to find three, and no more than three, independent onstraints. This proves the suffiieny of the statement. h The established proposition has an interesting geometri interpretation. It has already been stated that any oni urve X an be parameterized by a point x in } (Eq. (6)). Consider the 6 matrix! provided in Eq. (). The matrix is defined by points I 1 ; J 1 ; G 1,and H 1 Eq. (1). Aording to Proposition 1, a oni urve is the paraatadioptri image of a line if and only if the orresponding 6 1 vetor x lies in the null spae of! (!x ¼ ). This means that all points x, parameterizing the paraatadioptri projetion of a line, must lie in a ertain plane in the five-dimensional projetive spae. This plane is defined by matrix! and depends exlusively on the amera alibration. i x i x i y i y i x i z i y i z i z 6! ¼ j x j x j y j y j x j z j y j z j 7 z. g x h x g x g y þ h x h y g y h y g x g z þ h x h z g y g z þ h y h z g z h z.. The fitting algorithm ðþ Consider a set of image points x i ¼ðx i ; y i Þ t with i=1,,...,n. Our goal is to fit a oni urve x, orresponding to the paraatadioptri projetion of a line, to the set of data points. From Setion. it follows that the sum of the squares of the algebrai distanes between the urve and the image points is /ðxþ ¼x t A t Ax (matrix A is provided in Eq. (8)). Sine a oni has five DOF, the spae of all onis has five dimensions. The standard oni fitting algorithms of Setion. searh the entire spae for the oni that best fits the data points. However, and aording to Proposition 1, not all onis an be the paraatadioptri projetion of a line. The line projetion x must be in the null spae of matrix! Eq. (). The null spae of! is a linear subspae (hyperplane) in the spae of all oni urves. Our approah fits the data by the oni urve in this hyperplane that minimizes the algebrai distane to the image points. Consider the singular value deomposition of matrix!.! ¼ USV t. Matries U, S, and V have respetively dimension, 6, and 6 6. Matrix V is full rank and orthonormal (V 1 = V t ). The three last olumns of V are an orthonormal basis of the null spae of! [,6]. Consider the hange on the base of representation x v ¼ Vx. Ifx belongs to the null spae of matrix!, then the orresponding x v has the following struture: x v ¼ð; ; ; d v ; e v ; f fflfflfflffl{zfflfflfflffl} v Þ t. q ðþ

9 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) Rewriting the algebrai distane of Eq. (9) in terms of the new oordinates one obtains / ¼ x t v VAt AV t x v. Taking into aount the struture of x v Eq. (), the algebrai distane beomes / = q t K t Kq with K the bottom right sub matrix of VA t AV t. We aim to determine the solution q whih minimizes the algebrai distane / under the onstraint q t q = 1. The orresponding objetive funtion is wðq; kþ ¼q t K t Kq þ kðq t q 1Þ. ðþ The minimum of the objetive funtion w is the eigenvetor of matrix K t K orresponding to the smallest eigenvalue. The final oni x is omputed by replaing q in Eq. () and making x ¼ V t x v.. Performane evaluation using syntheti data The performane of the proposed algorithms is evaluated using synthetially generated images. This setion starts by introduing the sheme to generate syntheti data. In Setion. a skewless system with unitary aspet ratio is alibrated using just three lines (K = ). It is shown that our approah outperforms the method proposed in [16]. The alibration of a general paraatadioptri system (unknown skew and aspet ratio) is disussed in Setion.. The setion ends with the performane evaluation of the fitting method to estimate the oni lous where a line is projeted in a alibrated paraatadioptri image..1. Simulation sheme Assume a paraatadioptri amera with a field of view (FOV) of 18, orresponding to a full hemisphere, and predefined intrinsi parameters. The image of a set of K lines is generated as follows. As depited in Fig. 1, to eah line in the sene orresponds a plane P with normal n. The K normals are unitary and randomly hosen from an uniform distribution in the sphere. Eah normal defines a plane that intersets the unit sphere on a great irle. Notie that half of the great irle is within the amera field of view (the FOV is 18 ). An angle h, less than or equal to the FOV, is hosen to be the amplitude of the ar that is atually visible in the paraatadioptri image. The ar is randomly and uniformly positioned along the part of the great irle whih is within the FOV. The visible ar is uniformly sampled by a fixed number N of sample points. To eah sample point orresponds a projetive ray x. The sample rays are projeted using formula (1), and transformed using Eq. () with the hosen intrinsi parameters. Two-dimensional Gaussian noise with zero mean and standard deviation r is added to eah image point x. Fig. depits two simulated images of three randomly generated lines. In Fig. A the visible ar has an amplitude h =7 and is sampled by points. The amera intrinsi parameters appear in the bottom left orner. In Fig. B the visible ar is h = 1 and the number of sample points is N = 1. In this ase the amera is not skewless. As a final remark notie that the amplitude of the visible ar is measured in the great irle where plane P intersets the sphere, and not in the oni urve where the line is projeted. In general the visible angle of the paraatadioptri line image is muh less than h... Calibration of skewless amera with known aspet ratio Consider a skewless paraboli amera with aspet ratio 1.1 (s k = and r = 1.1). Both the skew and the aspet ratio are assumed to be known. We aim to determine the foal length (f = ) and the image enter (( x, y ) = (,8)) using the image of three lines (K = ). The line projetions are estimated by minimizing the sum of the square of the algebrai distanes _. The objetive funtion is provided in Eq. (19) and the minimization problem has a losed form solution. The system is alibrated using the algorithm presented in Table 1 after estimating the oni urves where the lines are projeted. The data points are synthetially generated using the simulation sheme explained above. The image of Fig. A is an example of a test image. The estimated alibration parameters are ompared with the ground truth and the RMS error is omputed over 1 runs of eah experiment. Fig. shows the results for different hoies of h (amplitude of the visible ar) and N (number of sample points). A N = Θ = 7 B N = 1 Θ = 1 α = 1.1 ; s k = ; f = ( x, y ) = (,8) α = 1.1 ; s k = ; f = ( x, y ) = (,8) Fig.. Syntheti generation of 8 6 test images. (A) Test image (Setion.). (B) Test image (Setion.).

10 16 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) A Foal Length RMS Error (pixel) 1 N = ; Θ = 17 N = 8; Θ = 17 N = ; Θ = 9 N = ; Θ = 7 B Image Center RMS Error (pixel) N = ; Θ = 17 N = 8; Θ = 17 N = ; Θ = 9 N = ; Θ = Fig.. Calibration of a skewless paraatadioptri amera with known aspet ratio using the image of three lines. The set of line projetions is estimated using the losed form algorithm of setion... The graphis show the root mean square (RMS) error for the foal length and image enter. (A) Foal length. (B) Image enter. For eah hoie of h and N the standard deviation of the additive Gaussian noise varies between. and 6 pixels by inrements of. pixels. For h = 17 the algorithm presents an exellent performane. The derease in the number of sample points from to 8 only slightly affets the robustness to noise. Sine we are only using three lines, the derease in the amplitude of the visible ar h and in the number of points N has a strong impat on the performane. Even so the alibration using ars of 9 is still pratiable. The situation of h = 7 and N =, shown in Fig. A, is very extreme and leads to a bad estimation of the intrinsi parameters. An alternative alibration approah is presented in [16]. The authors evaluate the performane of their algorithm using similar simulation onditions. A diret omparison an be made between the results presented in here and the ones presented in [16]. In general terms they estimate the oni urves by exploiting the fat that the image enter must lie in the line going through the intersetion points of any two line images. As disussed in Setion, this ondition is neessary, but not suffiient, for a set of oni urves to be the paraatadioptri projetion of lines. Sine the searh spae is not fully onstrained, they need muh more than three lines to alibrate the sensor. The results presented in Fig. are obtained using the minimum theoretial number of lines for alibration [1,16]. Even so, and as far as we are able to judge from the results presented in [16], the performane of our approah seems to be signifiantly better... Calibration of general paraatadioptri systems In this setion, it is assumed that nothing is known about the alibration parameters. We aim to determine the aspet ratio, skew, foal length and image enter using the paraatadioptri image of a set of K lines. The test images are generated using the simulation sheme explained in Setion.1. Fig. B shows an example of a test image with the assumed amera intrinsi parameters at the bottom left orner. The set of line projetions is estimated by minimizing the funtion g provided in Eq. (17). As disussed, the minimization is stated as a non-linear least squares problem, and the minimum is found using iterative gradient desent methods (typially Gauss Newton or Lenvenberg Marquardt [,]). The initial estimate for the iterative method is the losed form solution that minimizes funtion Eq. (16). The set of line projetions is used to determine the alibration parameters following the steps enuniated in Table 1. The results are ompared with the ground truth and the median error is omputed over 1 runs. In the first experiment, the system alibration is performed using the projetion of three lines (K = ). The appliation of gradient desent methods to minimize funtion g an be problemati in many ways [,]. The hoie of the initial estimate is ruial to assure a orret onvergene. The iterative proess must start from a point lose enough to the global minimum to avoid possible loal minima and saddle points. The objetive funtion has this type of singularities when there is not enough information and/ or the searh problem is not properly onstrained. By insuffiient information we mean a small number of lines, data points strongly orrupted with noise and visible ars with small amplitude or not sampled enough. Fig. 6A shows the failure of onvergene over the 1 runs for eah experiment. The run fails when the absolute oni X 1, determined following the steps in Table 1, is not positive definite and the Cholesky deomposition is not possible. This an only happen when the set of paraatadioptri line images is far from being orretly estimated. As expeted, the onvergene is strongly affeted by the noise. Moreover, the derease in the number of sample points also auses an inrease in the number of failures. Remark that we are using the minimum number of lines required to alibrate a paraatadioptri system. Therefore, it is natural that often the minimization proess does not onverge orretly. Fig. 6B shows the median error in the estimation

11 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) A 6 N= N=17 N=8 B N= N=17 N=8 Perentage of Failure 1 Foal Length MEDIAN Error (pixel) Fig. 6. Calibration using the projetion of three lines (K = ). The visible ar h has 17 of amplitude and the number of sample points N is variable (N =,17,8). For eah hoie of h and N the standard deviation of the additive Gaussian noise varies between. and 6 pixels by inrements of. pixels. (A) Failure rate in the onvergene of the iterative gradient desent method. (B) Median error in the alibration of the foal length. of the foal length with the performane dereasing with the number N of sample points. The results are not very impressive beause we are using only three line images. The experiment of Fig. 7 ompares the performane of the alibration algorithm when using the projetion of,, 7, and 9 lines. The inrease of the number of lines dramatially improves the robustness of the alibration... Estimating paraatadioptri line projetions in alibrated images The present setion evaluates the performane of the fitting algorithm (CATPARB) proposed in Setion. The syntheti data is generated using the simulation sheme explained in Setion.1. Fig. 8 shows the estimation of a A Aspet Ration MEDIAN Error Lines Lines 7 Lines 9 Lines B Skew MEDIAN Error (pixel) 1 1 Lines Lines 7 Lines 9 Lines C Foal Length MEDIAN Error (pixel) 1 1 Lines Lines 7 Lines 9 Lines D Image Center MEDIAN Error (pixel) Lines Lines 7 Lines 9 Lines Fig. 7. Calibration using the projetion of K =,, 7, and 9 lines. The amplitude of the visible ar is h = 1 and the number of sample points is N = 1. The standard deviation of the additive Gaussian noise varies between. and 6 pixels. The graphis show the median error for the different alibration parameters. (A) Aspet ratio. (B) Camera skew. (C) Foal length. (D) Image enter.

12 16 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) N = Θ = σ = (pixel) ξ = 1 α = 1.1 ; s k = ; f = ( x, y ) = (,8) AMS FF CATPARB Figs. 9 and 1 ompare the performanes of the AMS, FF, DLE, and CATPARB algorithms. The DLE method performs muh better than the standard oni fitting tehniques (AMS and FF). This is explained by the fat that the DLE uses not only the data points, but also impliit information about the sensor and its alibration. However, the performane of the DLE is learly worse than the performane of the CATPARB method. As explained in Setion., it is reasonable to assume that the noise in the image points x i ¼ðx i ; y i Þ t is Gaussian, two-dimensional and with zero mean. It is also reasonable to assume that Fig. 8. Estimation of the oni urve orresponding to a line projetion in a alibrated paraatadioptri image. The figure ompares the estimation results using AMS, FF, and CATPARB. The amplitude of the visible ar is h =, the number of sample points is N = and the standard deviation of the additive noise is r = pixels. paraatadioptri line projetion using AMS, FF (Setion.), and CATPARB. The standard oni fitting methods perform poorly, but the CATPARB algorithm is able to orretly reover the oni urve. To ertain extent the omparison of Fig. 8 is not entirely fair. While AMS and FF are generi methods to fit a oni urve to data points, CATPARB uses information about the sensor geometry and alibration to perform the estimation. The CATPARB algorithm is a speifi method for the estimation of paraatadioptri line projetions that requires the alibration matrix H to be known. If the system alibration is known, then the line projetion an be easily determined by performing the perspetive retifiation of the data points. Consider the image points x i, lying in the paraatadioptri line projetion X, the alibration matrix H (Eq. ()) and the inverse of funtion (Eq. (1)). Sine x i ¼ H 1 x i and x i ¼ h 1 x i, then the formula to ompute the retified data points is x i ¼ h 1 ðh 1 x i Þ i ¼ 1;...; N ðþ with h 1 ðxþ ¼ðxz; yz;z x y Þ t. The line projeted into oni X lies in a plane P with normal n =(n x,n y,n z ) t (Fig. 1). The retified points x i, with i =1,...,N, are projetive rays in the plane P that satisfy n t x i =. Therefore, the normal n an be estimated from the set of retified image points x i using normal least squares. The solution is the eigenvetor of the matrix C (Eq. (6)) orresponding to the smallest eigenvalue. The oni lous X in the image plane is omputed from n =(n x,n y,n z ) t and H using the relations established in Eqs. () and (). Heneforth, we will refer this approah as the diret line estimation (DLE) method. x 1 y 1 1 x y 1 C ¼ ð6þ x N y N 1 Prinipal Points RMS Error (pixel) AMS FF DLE CATPARB Fig. 9. Comparing the performane of AMS, FF, DLE, and CATPARB algorithms in estimating the oni lous where a line is projeted. The data points are synthetially generated using the simulation sheme of Setion.1. The visible ar has amplitude h =8 and is uniformly sampled by N = points. Eah algorithm fits a oni to the syntheti data points. The estimated oni is ompared with the ground truth and the RMS error in the prinipal points is omputed over 1 runs. Prinipal Points RMS Error (pixel) 6 1 N = ; Θ = 9 N = ; Θ = N = 1 ; Θ = N = ; Θ = Fig. 1. Charaterization of the performane of the CATPARB algorithm for different amplitudes of the visible ar (h) and number of sample points (N). As expeted the performane is worse when the number of samples and/or the amplitude of the visible ar derease. The results provide a general idea of the robustness of the CATPARB algorithm.

13 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) the error is equal in both diretions and unorrelated. Therefore, the noise ovariane matrix is r I, where r is a salar and I the identity matrix. Consider the retified point x i and the line n, both lying in the onventional perspetive plane. The algebrai distane between the point and the line is / i = n t x i. Replaing x i by the result of Eq. () and propagating the variane of the image point x i follows that the noise variane in the algebrai distane is r i ¼ ðn x þ n y Þðx i þy i Þ þ 8n x n yx iy i þ ðn x n y Þðx i y i Þ r. ð1 x i y i Þ ð7þ The least square estimator omputes the line n that minimizes the sum of the squares of the algebrai distanes / i (i = 1,...,N). The estimation is optimal when the algebrai distanes / i have the same noise variane and are statistially independent [,]. From Eq. (7) it follows that the variane r i is a funtion of the oordinates of the original image point x i. Thus, the variane of the algebrai distanes / i is not onstant and the line estimation using least squares is statistially biased []. The effets of the statistial bias are muh stronger in the DLE method than in the CATPARB algorithm, whih explains the poorer performane of the former (Fig. 9). 6. Experimental results using real images In this setion we apply the alibration method proposed in Setion to determine the parameters of a real paraatadioptri sensor. Five images were taken using a paraatadioptri amera. Fig. 11 shows one of those images where a set of line projetions is learly visible. For eah image we used an edge detetor and seleted points belonging to six different lines. Eah one of the five images was independently alibrated using,, and 6 lines. The results are summarized in Table. In this ase, the alibration parameters are all unknown. The estimation of the line projetions is performed by finding the solution that minimizes funtion g of Eq. (17). The initial estimation for the iterative proess is obtained using Table Calibration results using the projetion of,, and 6 lines r f s k x y Lines Mean STD Lines Mean STD Lines Mean STD For eah situation we performed five independent alibrations using five different images. The table shows the mean and standard deviation for eah alibration parameter. the AMS algorithm whih, among the standard oni fitting methods with losed form solution, is the one that better performs in the presene of olusion (see Setion.). Fig. 11A shows the initial estimates for the paraatadioptri line projetions. If the oni loi were aurately determined then all the lines going through the intersetion points should meet in the image enter ([16,1]). Fig. 11B exhibits the result orresponding to the minimum of funtion g. The alibration results are summarized in Table. Notie that the estimated values for the alibration parameters are more or less the same for the different K (number of lines). The standard deviation ats as a measure of onfidene. If the standard deviation takes high values then the results obtained for eah image are very different and the ahieved alibration is not trustful. As expeted the standard deviation dereases when the number of lines inreases. Heneforth, we will assume that the amera is alibrated. The experiment of Fig. 1 uses the CATPARB algorithm to determine the paraatadioptri projetion of lines. Remark that the estimation results impliitly depend on the alibration auray. Therefore, we are simultaneously testing the CATPARB algorithm and the proposed alibration method. Sine a line image has two independent degrees of freedom (Setion ), we selet by hand two points lying on the oni lous where a sene line is projet- A B Fig. 11. Estimation of a set of line projetions in an unalibrated paraatadioptri image. The image resolution is 17 7 and the FOV is 18. (A) Initial estimate (AMS). (B) Final estimate.

14 16 J.P. Barreto, H. Araujo / Computer Vision and Image Understanding 11 (6) Fig. 1. Estimating the projetion of lines in a alibrated paraatadioptri image using two points seleted by hand. are estimated using the CATPARB algorithm (see Fig. 1). The vanishing point of eah pair is determined in a straightforward manner using the results of [1,1]. Sine the alibration matrix H is known, then the image of the absolute oni an be omputed making X 1 ¼ H t H 1. The estimation of the angles between the pairs of parallel lines from the vanishing points and the absolute oni is trivial [7,19]. Table shows the errors in estimating these angles. There is an alternative approah to estimate the angles between the pairs of parallel lines. We an perform the perspetive retifiation of the image points, estimate the lines using normal linear least squares and ompute the angles using standard projetive relations. The estimation errors are exhibited in the seond line of Table. As expeted, estimating the lines diretly in the paraatadioptri plane presents better results. We may onlude that the bias introdued by the perspetive retifiation has a strong impat on the performane of the DLE method. 7. Conlusions b Fig. 1. Estimating the angles between pairs of parallel lines from a paraatadioptri image of those lines. ed. Fig. 1 shows the seleted points and the orresponding projeted line estimated using CATPARB. Fig. 1 is the paraatadioptri image of four pairs of parallel lines denote by a, b,, and d. The polar of the image enter with respet to the oni lous where eah line is mapped is the horizon of the orresponding plane P ontaining the original D line and the effetive viewpoint (see Fig. 1) [1,1]. Moreover, if two imaged lines are parallel then the intersetion of the orresponding horizons is the vanishing point of their ommon diretion. The eight line images, orresponding to the four pairs of parallel lines, a d This artile presents an effetive way to alibrate a paraatadioptri amera using the image of three or more lines in general position. It is shown that the aurate estimation of the oni urves where the lines are projeted is hard to aomplish due to partial olusion. We propose a strategy to overome this diffiulty. The neessary and suffiient onditions that must be verified by a set of oni urves to be the image of a set of lines are derived. These onditions are used to onstrain the searh spae and aurately estimate the set of oni urves required to alibrate the paraatadioptri sensor. If the amera is skewless and the aspet ratio is known then the oni fitting problem is solved naturally by an eigensystem. Otherwise the estimation is performed using non-linear optimization tehniques. Experimental results show that the proposed alibration method performs muh better than the ones appearing in the literature [16 18]. The seond ontribution is the CATPARB algorithm to estimate the projetion of a line in a alibrated paraatadioptri plane. It is proved that a oni urve, parameterized by a point in P, is the paraatadioptri image of a line if and only if it lies in a hyperplane defined by the system parameters. Thus, there are three neessary and suffiient onditions whih define a linear subspae in the spae of all oni urves. The line image is estimated within this subspae by solving an eigensystem. The method is Table Reovering the angles between pairs of parallel lines a b a a d b b d d Mean STD G. Truth Error CAT Error PER CAT denotes the results obtained by estimating the paraatadioptri projetion of the lines. PER are the angles estimated after performing perspetive retifiation.