Self-Calibration from Image Triplets. 1 Robotics Research Group, Department of Engineering Science, Oxford University, England

Transcription

1 Self-Calbraton from Image Trplets Martn Armstrong 1, Andrew Zsserman 1 and Rchard Hartley 2 1 Robotcs Research Group, Department of Engneerng Scence, Oxford Unversty, England 2 The General Electrc Corporate Research and Development Laboratory, Schenectady, NY, USA. Abstract. We descrbe a method for determnng ane and metrc calbraton of a camera wth unchangng nternal parameters undergong planar moton. It s shown that ane calbraton s recovered unquely, and metrc calbraton up to a two fold ambguty. The novel aspects of ths work are: rst, relatng the dstngushed objects of 3D Eucldean geometry to xed enttes n the mage second, showng that these xed enttes can be computed unquely va the trfocal tensor between mage trplets thrd, a robust and automatc mplementaton of the method. Results are ncluded of ane and metrc calbraton and structure recovery usng mages of real scenes. 1 Introducton From an mage sequence acqured wth an uncalbrated camera, structure of 3- space can be recovered up to a projectve ambguty[5, 7]. However, f the camera s constraned to have unchangng nternal parameters then ths ambguty can be reduced to metrc by calbratng the camera usng only mage correspondences (no calbraton grd). Ths process s termed \self-calbraton" [5, 11]. Prevous attempts to make use of the constrant for mage pars have generated sets of polynomal equatons that are solved by homotopy contnuaton [8] or teratvely [8, 17] over a sequence. In ths paper we demonstrate the advantages of utlzng mage trplets drectly n the case of planar moton, both n the reduced complexty of the equatons, and n a practcal and robust mplementaton. To reduce the ambguty of reconstructon from projectve to ane t s necessary to dentfy the plane at nnty, 1, and to reduce further to a metrc ambguty the absolute conc 1 on 1 must also be dented [4, 13]. Both 1 and 1 are xed enttes under Eucldean motons of 3-space. The key dea n ths paper s that these xed enttes can be accessed va xed enttes (ponts, lnes, concs) n the mage. To determne the xed mage enttes we utlse geometrc relatons between mages that are ndependent of three dmensonal structure. The fundamental geometrc relaton between two vews s the eppolar geometry, represented by the fundamental matrx [3]. Ths provdes a mappng from ponts n one mage to lnes n the other, and consequently s not a sutable mappng for determnng xed enttes drectly. However, between three vews the fundamental geometrc

2 relaton s the trfocal tensor [6, 14, 15], whch provdes a mappng of ponts to ponts, and lnes to lnes. It s therefore possble to solve drectly for xed mage enttes as xed ponts and lnes under transfer by the trfocal tensor. In the followng we obtan these xed mage enttes, and thence the camera calbraton, from a trplet of mages acqured by a camera wth unchangng nternal parameters undergong \planar moton". Planar moton s the typcal moton undergone by avehcle movng on a plane the camera translates n a plane and rotates about an axs perpendcular to that plane. Ths extends the work of Moons et al. who showed that ane structure can be obtaned n the case of purely translatonal moton [12]. We show that 1. Ane structure s computed unquely. 2. Metrc structure can be computed up to a one parameter famly, and ths ambguty resolved usng addtonal constrants. Secton 2, descrbes the xed mage enttes and ther relaton to 1 and 1, and descrbes how these are related to ane and metrc structure recovery. Secton 3 gves an algorthm for computng the mage xed ponts and lnes unquely usng the trfocal tensor. Secton 4.1 descrbes results of an mplementaton of ths algorthm, and secton 4.2 results for ane and metrc structure recovery based on these xed ponts from mage trplets. All results are for real mage sequences. Notaton We wll not dstngush between the Eucldean and smlarty cases, both wll be loosely referred to as metrc. Generally vectors wll be denoted by x, matrces as H, and tensors as T jk. Image coordnates are lower case 3-vectors, e.g. x, world coordnates are upper case 4-vectors, e.g. X. For homogeneous quanttes, = ndcates equalty up to a non-zero scale factor. 2 Fxed Image Enttes for Image Trplets Planar Moton Any rgd transformaton of space may be nterpreted as a rotaton about a screw axs and a smultaneous translaton n the drecton of the axs [2]. There are two specal cases { pure translaton and pure rotaton. In ths paper we consder the latter case. A planar moton of a camera conssts of a rotaton and a translaton perpendcular to the rotaton axs. Ths s equvalent to a pure rotaton about a screw axs parallel to the rotaton axs, but not n general passng through the camera centre. The plane through the camera centre and perpendcular to the rotaton axs s the plane of moton of the camera. We consder sequences of planar motons of a camera, by whch we mean a sequence of rotatons about parallel but generally dstnct rotaton axes. The plane of moton s common to all the motons. For vsualsaton, we assume the plane of moton s horzontal and the rotaton axes vertcal. 3D xed enttes The plane at nnty and absolute conc are nvarant under all Eucldean actons. These are the enttes that we desre to nd n order

3 to compute respectvely ane or metrc structure. These enttes can not be observed drectly, however, so we attempt to nd them ndrectly. To ths end we consder xed ponts of a sequence of planar motons. A sngle planar moton has addtonal xed enttes, the screw axs (xed pontwse), and the plane of moton (xed setwse). In fact, any plane parallel to the plane of moton s xed. The ntersecton of ths pencl of planes wth the plane at nnty salne (xed setwse). Although ths lne s xed only as a set, ts ntersecton wth the absolute conc, 1, consstng of two ponts, s xed pontwse by the moton. These two ponts are known as the crcular ponts, denoted I and J, and le on every plane parallel to the plane of moton. Knowledge of these two crcular ponts s equvalent toknowng the metrc structure n each of these planes ([13]). The two crcular ponts are xed for all motons n a sequence of planar motons wth common plane of moton. Ths s not true of the xed screw axes, snce we assume n general that the screw axs s not the same for all motons. However, snce the screw axes are parallel, they all ntersect at the plane at nnty atapontwhchwe shall denote by V. The ponts I, J and V and ther relaton to 1 s shown n gure 1. They are xed by all motons n the sequence. V Plane at nfnty I J L absolute conc Fg. 1. The xed enttes on 1 of a sequence of Eucldean planar motons of 3-space. V s the deal pont of the screw axs, and L the deal lne of the pencl of planes, orthogonal to the screw axs. I and J are the crcular ponts for these planes, dened by thentersecton of L wth 1. V and L are pole and polar wth respect to the absolute conc. If by some means we are able to nd the locatons of the ponts I, J and V n space, then we are able to determne the plane at nnty 1 as the unque plane passng through all three of them. Ths s equvalent to determnng the ane structure of space. Although we do not know 1, and hence can not determne metrc structure, we at least know two ponts on ths absolute conc, and hence know the Eucldean geometry n every plane parallel to the plane of moton.

4 Fxed mage enttes Our goal s to nd the three ponts I, J and V. Snce they are xed by the sequence of motons, ther mages wll appear at the same locaton n all mages taken by themovng camera (assumng xed nternal calbraton). We are led to nqure whch ponts are xed n all mages of a sequence. A xed pont n a par of mages s the mage of a pont n space that appears at the same locaton n the two mages. It wll be seen that apart from the mages of I, J and V there are other xed mage enttes. We wll be led to consder both xed ponts and lnes. The locus of all ponts n space that map to the same pont ntwo mages s known as the horopter curve. Generally ths s a twsted cubc curve n3- space passng through the two camera centres [1]. One can nd the mage of the horopter usng the fundamental matrx of the par of mages, snce a pont on the horopter satses the equaton x > Fx =. Hence, the mage of the horopter s a conc dened by the symmetrc part of F, namely F s = F + F >. In the case of planar moton, the horopter degenerates to a conc n the plane of moton, plus the screw axs. The conc passes through (and s hence dened by): the two camera centres, the two crcular ponts, and the ntersecton of the screw axs wth the plane of moton. It can be shown that for planar moton F s s rank 2 [1], and the conc x > Fx = x > F s x =, whch s the mage of the horopter, degenerates to two lnes. These lnes are the mage of the screw axs and the mage of the plane of moton the horzon lne n the mage [1]. The eppoles and maged crcular ponts le on ths horzon lne. The apex (the vanshng pont of the rotaton axs) les on the maged screw axs. These ponts are shown n gure 2a. Although the lnes can be computed from F s, and the maged crcular ponts and apex le on these lnes, we have not yet explaned how torecover these ponts. v mage v mage mage of screw axs horzon maged screw axes between mage pars l e s e / l j a Fg. 2. Fxed mage enttes for planar moton. (a) For two vews the maged screw axs s a lne of xed ponts n the mage under the moton. The horzon s a xed lne under the moton. (b) The relaton between the xed lnes obtaned parwse for three mages under planar moton. The mage horzon lnes for each par are concdent, and the maged screw axes for each par ntersect n the apex. All the eppoles le on the horzon lne. b

5 We now consder xed ponts n three vews connected va planar motons. To do ths, we need to consder the ntersecton of the horopter for cameras 1 and 2 wth that for cameras 2 and 3. Each horopter conssts of a conc n the plane of moton, plus the vertcal axs. The two vertcal axes, supposed dstnct, meet at nnty at the pont V.Thetwo concs meet n 4 ponts, namely the crcular ponts I and J, the centre of the second camera, plus one further pont that s xed n all three vews. The horopter for cameras 1 and 3 wll not pass through the second camera centre. Thus we are left wth 4 ponts that are xed n all three mages. These are the crcular ponts I and J, a thrd pont X lyng on the plane of moton, and the deal pont V. Any two xed ponts dene a xed lne, the lne passng through them. Snce three of the ponts, namely the mages of ponts I, J and the thrd pont are collnear, there are just 4 xed lnes. There can be no others, snce the ntersecton of two xed lnes must be a xed pont. We have sketched a geometrc proof of the followng theorem. Theorem For three vews from a camera wth xed nternal parameters undergong general planar moton, there are four xed ponts, three of whch are collnear: 1. The vanshng pont of the rotaton axes, v (the apex). 2. Two complex ponts, the mages of the two crcular ponts I J on the horzon lne. 3. A thrd pont x on the horzon lne and pecular to the mage trplet. There are four xed lnes passng through pars of xed ponts. 3D Structure Determnaton A method for determnng ane and metrc structure s as follows. One determnes the xed ponts n the three mages usng the trfocal tensor as descrbed n the followng secton. The thrd real collnear xed pont x can be dstngushed from the complex crcular ponts, the mages of I and J. Ths thrd pont s dscarded. The 3-D ponts I, J and V correspondng to these xed mage ponts may be reconstructed. These three ponts dene the plane at nnty, and hence ane structure. Planar metrc structure s determned by the crcular ponts I and J. Thus, n the absence of other constrants, 3D structure s determned up to a Eucldean transformaton n planes parallel to the plane of moton, and up to a one dmensonal ane transformaton perpendcular to the plane of moton. Followng Luong and Vevlle [9] an addtonal constrant s provded by assumng the skew parameter s zero.e. that the mage axes are perpendcular. Ths s a very good approxmaton n practce. Ths constrant results n a quadratc polynomal gvng two solutons for the nternal calbraton matrx, and hence for the recovery of metrc structure. Alternatvely, an assumpton of equal scale factors n the two coordnate axs drectons wll allow for unque metrc reconstructon. We have now descrbed the structure ambguty once the xed mage enttes are dented. The next secton descrbes a method of dentfyng the xed mage enttes usng the trfocal tensor.

6 3 Fxed mage enttes va the trfocal tensor Suppose the 3 4 camera projecton matrces for the three vews are P, P and P. Let a lne n space be mapped to lnes l, l and l n three mages. A trlnear relatonshp exsts between the coordnates of the three lnes, as follows : l = l j l k Tjk (1) where T jk s the trfocal tensor [6]. Here and elsewhere we observe the conventon that ndces repeated n the contravarant andcovarant postons mply summaton over the range (1 ::: 3) of the ndex. A smlar relatonshp holds between coordnates of correspondng ponts n three mages. The trfocal tensor can be computed drectly from pont and lne matches over three vews. It can also be drectly constructed from the camera projecton matrces P, P and P as follows. Assumng that P =[Ij], we have theformula T jk = p j pk ; (2) 4 pj 4 pk where p and p are the (j)-th entry of the respectve camera matrces, ndex j j beng the contravarant (row) ndex and j beng the covarant (column) ndex. Now n order to nd xed lnes, we seek solutons l to the equatons (from (1)) l = l j l k T jk (3) In (1) as well as (3) the equalty sgn represents equalty up to a non-zero scale factor. We mayremove the unknown scale factor n (3) by takng the cross product of the two sdes and equatng the result to the zero vector. Ths results n three smultaneous homogeneous cubc equatons for the components of l. In the followng we dscuss methods for obtanng the solutons to these cubcs. Frst we descrbe the general case, and then show that ths can be transformed to a specal case where the soluton reduces to a sngle cubc n one varable. The transformaton requred s a plane projectve transformaton of the mages. Fnally, we arrve at a two step algorthm, talored to real mages, for determnng the xed mage ponts and lnes. 3.1 General Planar Moton We consder three vews taken by a camera undergong planar moton. Wthout loss of generalty, we may assume that the camera s movng n the plane Y =. The rotaton axes are perpendcular to ths plane, and meet at the pont at nnty ( 1 ) >.We assume that the camera has xed, but unknown calbraton. The orgn of coordnates may be chosen at the locaton of the rst camera, whch means that the camera has matrx P = H[I j ], for some matrx H. The other two camera der by a planar moton from ths rst camera, whch means that the three camera have the form P = H[I j ] P = H[R j t ] P = H[R j t ] (4)

7 where R and R are rotatons about the Y axs, and t and t are translatons n the plane Y =. One may solve for the xed lnes usng the trfocal tensor T jk. Denotng a xed lne l for convenence as l =(x y z) nstead of (l 1 l 2 l 3 ), the xed lne equaton l = l j l k T jk may be wrtten as 1 1 x h (2) (x y y h (2) (x y z) A (5) z h (2) (x y z) where the superscrpt (2) denotes the degree of the polynomal. Settng the crossproduct of the two sdes of ths equaton to zero, one obtans a set of three cubc equatons n x, y and z. By the dscusson of secton 2, there should be four xed lnes as solutons to ths set of equatons. The rst thng to note, however, s that the three equatons derved from (5) are not lnearly ndependent. There are just two lnearly ndependent cubcs. Inevtably, for a trfocal tensor computed from real mage correspondences, the solutons obtaned depend on just whch par of the three equatons one chooses. Furthermore, f there s nose present n the mage measurements, then the number of solutons to these equatons ncreases. In general, two smultaneous cubcs can have up to 9 solutons. What happens s that one obtans a number of dfferent solutons close to the four deal solutons. Thus, for nstance, there are a number of solutons close to the deal horzon lne. Generally speakng, proceedng n ths way wll lead nto a mre of unpleasant numercal computaton. 3.2 Normalzed Planar Moton One can smplfy the problem by applyng a projectve transformaton to each mage before attemptng to nd the xed lnes. The transformaton to be appled wll be the same for each of the mages, and hence wll map the xed lnes to xed lnes of the transformed mages. The transformaton that we apply wll have the eect of mappng the apex pont v to the pont at nnty ( 1 ) > n the drecton of the y-axs. In addton, t wll map the horzon lne to the x-axs, whch has coordnates ( 1 ) >. The transformed mages wll correspond to camera matrces ~P = GH[I j ] ~P = GH[R j t ] ~P = GH[R j t ] where G represents the appled mage transformaton. We consderng now the rst camera matrx ~P. Ths matrx maps ( 1 ) >,thevanshng pont ofthe Y axs, to the apex ( 1 ) > n the mage. Furthermore the plane Y = wth coordnates ( 1 ) s mapped to the horzon lne ( 1 ) > as requred. Ths constrans the camera matrx ~P =[GH j ] to be of the form 2 3 ~P =[GH j ] = 4 5 (6)

8 where represents a zero entry and represents a non-zero entry. Consder now the other camera matrces ~P and ~P. Snce R and R are rotatons about the Y axs, and t and t are translatons n the plane Y =, both [R j t ]and[r j t ] are of the form [16] (7) Premultplyng by GH, we nd that both ~P and ~P are of the same form (7). Ths partcularly smple form of the camera matrces allows us to nd a smple form for the trfocal tensor as well. In order to apply formula (2), we requre matrx P to be of the form P =[Ij]. Ths can be acheved by rghtmultplcaton of all the camera matrces by the 3D transformaton matrx (GH) ;1.It 1 may beobserved that ths multplcaton does not change the format (7) of the matrces ~P and ~P.Now, for = 1 or 3, we see that ~p and ~p are of the form ( ) >, whereas for = 2, they are of the form ( ) >.Further, ~p s of 4 the form ( ) >. One easly computes the followng form for T ~ ~T = for =1 3 ~ T 2 = (8) Usng ths specal form of the trfocal tensor, we see that (3) may be wrtten as 1 x 1 a 1 x 2 + b 1 xz + c 1 z y A z d 2 xy + e 2 yz a 3 x 2 + b 3 xz + c 3 z 2 A (9) where l = (x y z) > represents a xed lne. Ths set of equatons has eght parameters fa 1 :::c 3 g. The xed lnes may be found by solvng ths system of equatons. One xed pont n the three vews s the apex, v =( 1 ) >. Let us consder only lnes passng through the apex xed n all three vews. Such a lne has coordnates (x z). Thus, we may assume that y =. The equatons (9) now reduce to the form x a1 x 2 + b 1 xz + c 1 z 2 (1) z a 3 x 2 + b 3 xz + c 3 z 2 Cross-multplyng reduces ths to a sngle equaton z(a 1 x 2 + b 1 xz + c 1 z 2 )=x(a 3 x 2 + b 3 xz + c 3 z 2 ) (11) Ths s a homogeneous cubc, and may be easly solved for the rato x : z. The solutons to ths cubc are the three lnes passng through the apex jonng t to three ponts lyng on the horzon lne. These three ponts are the mages of the two crcular ponts, and the thrd xed pont. The thrd xed pont may be dstngushed by the fact that t s a real soluton, whereas the two crcular

9 ponts are a par of complex conjugate solutons. The thrd xed pont s of no specal nterest, and s dscarded. Ths analyss s an example of a generally useful technque of applyng geometrc transformatons to smplfy algebrac computaton. 3.3 Algorthm Outlne We now put the parts of the algorthm together. The followng algorthm determnes the xed lnes n three vews, and hence the apex and crcular ponts on the horzon lne. The rst four steps reduce to the case of normalzed planar moton. The xed ponts and lnes are then computed n steps 5 to 7, and the last step relates the xed ponts back to the xed ponts n the orgnal mages. 1. Compute the fundamental matrx for all pars of mage, and obtan the eppoles. 2. Fnd the orthogonal regresson lne t to the eppoles. Ths s the horzon lne l. 3. Decompose the symmetrc part of the F's nto two lnes, ths generates the mage of the screw axs for each par. Fnd the ntersecton of the maged screw axes, or n the presence of nose, the pont wth mnmum squared dstance to all the maged screw axes. Ths determnes the apex v. 4. Fnd a projectve transformaton G takng the horzon lne to the lne ( 1 ) and the apex to the pont ( 1 ). Apply ths projectve transform to all mages. 5. For three vews, compute the trfocal tensor from pont and lne matches, enforcng the constrant that t be of the form descrbed n (8). 6. Compute the cubc polynomal dened n (9) and (11), and solve for the rato x : z. There wll be two magnary and one real soluton. Dscard the real soluton. The magnary solutons wll be lnes wth coordnates (1 z) and (1 z) passng through the apex and the two crcular ponts on the horzon lne. 7. Compute the ntersecton of the horzon lne ( 1 ) and the lne (1 z). Ths s the pont (;z 1). Do the same for the other soluton (1 z). 8. Apply the nverse transform G ;1 to the two crcular ponts to nd the mage of the two crcular ponts n the orgnal mages. 4 Results Numercal results are mproved sgncantly by enforcng that both F and F s are rank 2 durng the mnmzaton to compute the fundamental matrx. Implementaton detals of the algorthms are gven n [18]. 4.1 Fxed mage ponts and lnes In ths secton we descrbe the results of obtanng the xed ponts/lnes over mage trplets. These ponts are used for ane and metrc calbraton, whch s descrbed n secton 4.2.

10 The mage sequences used are shown n gure 3 (sequence I) and gure 4 (sequence II). The rst sequence s acqured by a camera mounted on an Adept robot, the second by a derent camera mounted on an AGV. The latter sequence has consderably more camera shake, and consequently s not perfect planar moton. Fgure 5 shows the two vew xed lnes obtaned from the seven sequental mage pars from sequence I. The trfocal tensor s computed for the sx sequental mage trplets, and the crcular ponts computed from the xed ponts of the tensor. The results are gven n table 1a. The crcular ponts are certanly stable, but t s dcult to quantfy ther accuracy drectly because they are complex. In the next secton the crcular ponts are used to upgrade projectve structure to ane and metrc. The accuracy of the crcular ponts s hence measured ndrectly by the accuracy of the recovered structure. For comparson, the estmated crcular ponts, based on approxmate nternal parameters, are (4 11 ;255 1) >. The camera undergoes a smaller rotaton n sequence II and the mages are noser due to camera shake. Superor results are obtaned by usng fundamental matrces from mage pars whch are separated by 2 tme steps (.e. pars f1,4g,f1,5g,f2,4g,...), rather than sequental mage pars. Fgure 6 shows these results. The tensor s calculated usng mage trplets separated by one tme step (.e. trplets f1,3,5g,f2,4,6g,...). The crcular ponts computed are shown n table 1b. The estmated crcular ponts, based on approxmate nternal parameters, are ( ) >. 4.2 Structure Recovery Secton 4.1 obtaned the 3 xed ponts of mage trplets usng the algorthm of secton 3.3. These ponts dene the poston of the plane at nnty 1, whch allows ane structure to be recovered. In ths secton we descrbe the results of an mplementaton of ane and metrc structure recovery, and assess the accuracy by comparng wth ground truth. Ane Structure Usng the mage sequence n gure 3, and the crcular ponts lsted n table 1a, ane structure s recovered. We can quantfy the accuracy of Fg. 3. Image sequence I: four mages from an eght mage sequence acqured by a camera mounted on an Adept robot arm. Planar moton wth the rotaton axs at approxmately 25 o to the mage y axs and perpendcular to the mage x axs. 149 corners are automatcally matched and tracked through the 8 mages. The Tsa grds are used only to provde ground truth, not to calbrate.

11 Fg. 4. Image sequence II: four mages from a nne mage sequence. Planar moton wth the rotaton axs approxmately algned wth the mage y axs. 75 corners are automatcally matched across the 9 mages. The sequence was acqured by a camera mountedonanagv Fg. 5. The xed ponts and lnes obtaned from all 7 possble sequental mage pars from sequence I (gure 3), wth axes n pxels. are the eppoles, dashed and sold lnes are the screw axes and the horzon respectvely, s the apex at (394,237). The horzon lne s at y = ;255. the ane structure by comparng the values of ane nvarants measured on the recovered structure to ther verdcal values. The ane nvarant used s the rato of lne segment lengths on parallel lnes. The lnes n the scene are dened by the corners on the partally obscured calbraton grd shown n gure 3. The verdcal value of these ratos s 1., and the results obtaned are shown n table 2. Clearly, the projectve skewng has been largely removed. Metrc Structure Metrc structure n planes parallel to the moton plane (the ground plane here) s recovered for the sequence n gure 3. The accuracy of the metrc structure s measured by computng an angle n the recovered structure wth a known verdcal value. We compute two angles for each mage trplet. Frst s the angle between the planes of the calbraton grd. We t planes to 23 and 18 ponts on the left and rght faces respectvely and compare the nterplane angle to a verdcal value of 9 o. Second s the angle between the three computed camera centres for each mage trplet whch s known from the robot moton. The

12 Image Trplet Crcular ponts 123 ( ;255 1) > 234 ( ;255 1) > 345 ( ;255 1) > 456 ( ;255 1) > 567 ( ;255 1) > 678 ( ;255 1) > a Image Trplet Crcular ponts 135 ( ) > 246 ( ) > 357 ( ) > 468 ( ) > 579 ( ) > Table 1. The crcular ponts obtaned (complex conjugates) for (a) sequence I (gure 3) and (b) sequence II (gure 4). Note, the stablty of the ponts estmated from derent trplets. b 1 x x 1 4 Fg. 6. The xed ponts and lnes sequence II, gure 4, wth axes n pxels. are the eppoles, dashed and sold lnes are the screw axes and the horzon lne respectvely, s the mage apex at (143,4675). The horzon lne passes through (,182). camera centres are computed from the camera projecton matrces. Table 2 shows the computed angles for the 6 mage trplets from sequence I, whle gure 7 shows a plan vew of the recovered metrc structure from the rst mage trplet. 5 Conclusons and extensons We have demonstrated the geometrc mportance of xed ponts and lnes n an mage sequence as calbraton tools. These xed enttes have been measured, and used to recover ane and partal metrc structure from mage sequences of real scenes. There are a number of outstandng questons, both numercal and theoretcal: 1. We have demonstrated that estmates of the plane at nnty and camera nternal parameters can be computed from mage trplets. It now remans to derve the varance of these quanttes. Then a recursve estmator can

13 Ane Invarants Metrc Invarants standard Plane angle Moton Angle Image Trplet max mn average devaton (9 o 1 o ) Actual Computed Table 2. Ane Invarants The rato of lengths of parallel lnes measured on the recovered ane structure of the calbraton grd. The verdcal value s unty. Metrc Invarants Angles measured n the ground plane. The nterplane angle for the calbraton grd, and the angle between the computed camera centres. Fg. 7. Plan vew of the structure recovered from the rst mage trplet of sequence I. The rst three camera centres are marked wth. The calbraton grd and other objects are clearly shown. be bult, such as an Extended Kalman Flter, whch updates the plane at nnty and camera calbraton throughout an mage sequence. 2. The mage of the absolute conc s a xed entty over all mages wth unchangng nternal parameters. The study of xed mage enttes opens up the possblty of solvng for ths drectly as the xed conc of a sequence. Acknowledgements Fnancal support for ths work was provded by EU ACTS Project VANGUARD, the EPSRC and GE Research and Development. References 1. Beardsley, P. and Zsserman, A. Ane calbraton of moble vehcles. In Mohr, R. and Chengke, W., edtors, Europe-Chna workshop on Geometrcal Modellng and Invarants for Computer Vson. X'an, Chna, Bottema, O. and Roth, B. Theoretcal Knematcs. Dover, New York, 1979.

14 3. Faugeras, O. What can be seen n three dmensons wth an uncalbrated stereo rg? In Proc. ECCV, LNCS 588, pages 563{578. Sprnger-Verlag, Faugeras, O. Stratcaton of three-dmensonal vson: projectve, ane, and metrc representaton. J. Opt. Soc. Am., A12:465{484, Faugeras, O., Luong, Q., and Maybank, S. Camera self-calbraton: theory and experments. In Proc. ECCV, LNCS 588, pages 321{334. Sprnger-Verlag, Hartley, R. A lnear method for reconstructon from lnes and ponts. In Proc. ICCV, pages 882{887, Hartley, R., Gupta, R., and Chang, T. Stereo from uncalbrated cameras. In Proc. CVPR, Luong, Q. Matrce Fondamentale et Autocalbraton en Vson par Ordnateur. PhD thess, Unverste de Pars-Sud, France, Luong, Q. and Vevlle, T. Canonc representatons for the geometres of multple projectve vews. In Proc. ECCV, pages 589{597. Sprnger-Verlag, Maybank, S. Theory of reconstructon from mage moton. Sprnger-Verlag, Berln, Maybank, S. and Faugeras, O. A theory of self-calbraton of a movng camera. Internatonal Journal of Computer Vson, 8:123{151, Moons, T., Van Gool, L., Van Dest, M., and Pauwels, E. Ane reconstructon from perspectve mage pars. In Applcatons of Invarance n Computer Vson, LNCS 825. Sprnger-Verlag, Semple, J. and Kneebone, G. Algebrac Projectve Geometry. Oxford Unversty Press, Shashua, A. Trlnearty n vsual recognton by algnment. In Proc. ECCV, Spetsaks, M.E. and Alomonos, J. Structure from moton usng lne correspondences. Internatonal Journal of Computer Vson, pages 171{183, Wles, C and Brady, J.M. Ground plane moton camera models. In Proc. ECCV, Zeller, C. and Faugeras, O. Camera self-calbraton from vdeo sequences: the kruppa equatons revsted. Techncal report, INRIA, Zsserman, A., Armstrong, M., and Hartley, R. Self-calbraton from mage trplets. Techncal report, Dept. of Engneerng Scence, Oxford Unversty, Ths artcle was processed usng the LaT E X macro package wth ECCV'96 style