The Interdeciplinary Center Herzliya Technical Report CS-TR

Transcription

1 The Interdeciplinry Center Herzliy Technicl Report CS-TR Recovering Epipolr Geometry from Imges of Smooth Surfces Yel Moses nd Iln Shimshoni Computer Science Deprtment The Interdisciplinry Center Herzliy 46150, Isrel Industril Engineering nd Mngement The Technion, Hif 32000, Isrel Abstrct. We present method for recovering the epipolr geometry from imges of smooth surfces. Existing methods for recovering epipolr geometry use corresponding feture points tht cnnot be found in such imges. Other methods for recovering the epipolr geometry of smooth objects re bsed on the objects outline. Such methods re limited either to specil type of scenes or to restricted cmer motion. Our method computes the epipolr geometry by serching for epipole loctions nd correspondence between epipolr lines. The min technicl contributions of our method re twofold. The first consists of bounding the serch spce for epipole loctions. On the fce of it, this spce is infinite nd unbounded. We suggest method to prtition the infinite plne into finite number of regions. This prtition is bsed on the desired ccurcy nd mintins properties tht yield n efficient serch over the infinite plne. The second is n efficient method for finding correspondence between two infinite sets of epipolr lines. The correspondence is defined between smll subsets of cndidte epipolr lines using properties of tngents to isophoto curves in the two imges. Our method is pplicble to ny pir of imges of smooth objects tken under the wek or full perspective projection models, s long s the constnt brightness ssumption holds. We show how the method cn be simplified when deling with specil cses, where the cmer s prmeters or the cmer s motion re prtilly known. The method hs been implemented nd tested on pirs of rel imges, which re relted by cmer motion prllel to the imge plne. This reserch ws prtily suported by grnt from The Isrel Science Foundtion.

2 1 Introduction Recovering three dimensionl shpe from sequence of 2D imges hs mny pplictions in res s diverse s utonomous nvigtion, object recognition, nd computer grphics. Solving this problem requires the cmer prmeters nd the correspondence between points in the different imges. Epipolr geometry plys centrl role in extrcting correspondence between points in different imges. For ech point in one imge epipolr geometry determines single line, clled n epipolr line, in the other imge on which its corresponding point is incident. This pper presents method for determining the epipolr geometry of pir of imges under wek nd full perspective projection models. We ssume unclibrted imges of smooth surfces. The pir of imges my be tken from ny two viewpoints distnt from ech other s long s they stisfy the constnt brightness ssumption, which presupposes tht corresponding points in the different imges hve the sme intensity vlue. This ssumption holds when the reflectnce model of the imged surfce is independent of the viewpoint. We lso discuss less generl cses such s the wek perspective projection model, clibrted cmers, nd setup similr to the one suggested by [4], where the imges contin plne whose homogrphy cn be computed. Epipolr geometry is often represented by the fundmentl mtrix [5, 13, 6]. The stndrd method for recovering the epipolr geometry is by computing the fundmentl mtrix from set of corresponding fetures in the two imges such s points or lines (e.g., [9, 15, 13, 12, 17, 16]). However, for imges of smooth surfces, which we consider in this pper, relible extrction of imge fetures is often impossible. Existing methods for recovering the epipolr geometry of smooth objects re bsed solely on the objects outline (e.g., [2, 4]). Such methods re limited to either restricted motion or to reltively rich scene with sufficient number of specil points long the objects outline. Our pproch is more generl nd is independent of the occluding contour nd the cmer motion. However, our method must consider significntly lrger spce serch, nd cn work only when the constnt brightness ssumption holds. In Section 3 we discuss specil cses under which our method cn be simplified nd combined with existing methods. The fundmentl mtrix cn lso be computed from the epipole loctions in the two imges nd the correspondence between smll subset of epipolr lines [13, 7, 10]. The epipole loction defines the set of epipolr lines in ech of the imges, since ll epipolr lines intersect t the epipole. The correspondence between smll subset of epipolr lines defines the 1D-homogrphy between the sets of epipolr lines. Our scheme recovers the epipolr geometry by serching nd finding such corresponding sets. We use properties of epipolr geometry to efficiently determine for ech pir of epipoles the best correspondence between two smll subsets of epipolr lines. The subsets re determined bsed on epipole-dependent fetures: set of epipolr lines tht re tngent to given isophoto. Using geometric nd photometric properties of epipolr lines, the mtching of the subsets correspondence is tested. The test is used to find the correct sets of epipolr lines nd the correspondence between them, which is then used to compute the fundmentl mtrix. The min technicl contributions of our method re twofold. The first is n efficient method for finding correspondence between two sets of epipolr lines. Since we

3 () (b) (c) (d) (e) Fig. 1. ()-(b) Two imges of the sme object from two different viewpoints. The isophoto contours of one vlue re mrked on the two imges. Two epipolr lines which re tngent to this isophoto re lso mrked on the imges. (c) nd (d) re the intensity profile long two corresponding epipolr lines, nd (e) is the intensity profile long different epipolr line. re deling with imges of smooth surfces, we cnnot use feture points. Our method defines smll subsets of epipolr lines nd their correspondence. The subsets re determined bsed on the observtion tht two corresponding epipolr lines must be both tngent to the sme intensity isophoto contour (see Figure 1-b). Actully, this property is correct for ny imge curve which is visible in both imges. This property ws used by Porril & Pollrd [14] for finding corresponding points for motion estimtion. This observtion voids using the impossible exhustive serch of ll possible correspondences between the infinite sets of epipolr lines. The second contribution consists of bounding the serch spce for epipole loctions. On the fce of it, this spce is infinite nd unbounded. We suggest method to prtition the infinite plne into finite number of regions (see Figure 2). The suggested plne prtition mintins the system desired resolution, when possible. In ddition, it mintins probbilistic equl hit mesure of epipolr lines. Roughly speking, probbilisticly the size of ech subset of epipolr lines we ssigned for ech region is equl. This property contributes to the efficiency of the serch. The rest of the pper is orgnized s follows. We begin by presenting our method for recovering the epipolr geometry from pir of unclibrted imges of smooth surfces (Section 2). The method is presented for imges tken under the perspective projection model. We then consider less generl setup such s imges tht were tken under the wek perspective projection model nd imges with restricted cmer motion (Section 3). The implementtion of our method nd the results of running our lgorithm on rel imges re presented in Section 4. Finlly we summrize nd conclude in Section 5.

4 54 r3 r2 δθ Fig. 2. A schemtic drwing of the infinite plne prtitioned into regions. 2 The method In this section we present our method for recovering the epipolr geometry of two imges, by determining the fundmentl mtrix. (A pir of corresponding points in.) To determine it is suffi- the two imges, nd, must stisfies, cient to determine the epipole loction for ech of the imges nd t lest three corresponding epipolr lines. The three pirs of corresponding epipolr lines define the 1D-homogrphy between the epipolr lines of the two imges [13]. Let,, nd be the 1D-homogrphy prmeters, nd nd be the epipoles in the two imges. The fundmentl mtrix cn be expressed using these prmeters s follows [17, 13, 16, 10, 7]):!"#$ $&%'( ) *+$, $- *./ $ - 0.(- 1%2( 0%23, Our method consists of serching the spce of epipole loctions. The epipole loction defines n infinite set of epipolr lines for ech of the imges. Clerly it is impossible to consider ll possible epipole loctions, we therefore tessellte the imge plne into finite number of regions. A single point in ech region represents ll possible epipoles in the region. This tesselltion is described in Section 2.2. Next, we efficiently find subset of lines from ech set of epipolr lines such tht if the epipoles re correct, then these subsets must be corresponding. The subsets re found bsed on the tngents to smll number of isophoto imge contours tht re incident on the epipole, nd therefore constitute epipolr lines. This method is described in Section 2.1. Bsed on photometric nd geometric properties of epipolr lines, we test the correspondence between the two subsets of lines. This test is used to determine the correct epipole loctions nd the correspondence between epipolr lines. The testing method is described in Section 2.3. If more thn single solution is found, lrger set of epipolr lines cn be used to evlute the solution. A lrger set of corresponding epipolr lines is vilble from the 1D-homogrphy defined by smll number of corresponding lines. It is therefore possible to verify correspondence using the photometric constrints on s mny epipolr lines s needed. In the rest of this section we describe ech of the prts of the method. (1)

5 2.1 Subsets of corresponding epipolr lines Epipolr geometry sttes tht ll points on given epipolr line correspond to points on its corresponding epipolr line. Under the constnt brightness ssumption, we ssume, corresponding points in the two imges hve the sme intensity vlue. Therefore, the intensity profile long corresponding epipolr lines must be identicl up to the foreshortening effect, nd occlusion (see Figure 1c-d). Consider imge points with mximl or miniml intensity vlues long n epipolr line, nmely extremum points. Given corresponding epipolr lines, the extremum points long the lines correspond. We next show method to determine the extremum points on corresponding epipolr lines, without exploring the intensity profile long n epipolr line. To do so we use the following property of extremum points long epipolr lines: Clim: At extremum points long epipolr lines, the tngents to the isophoto coincide with the epipolr line direction. Proof: Let 6798:* be the imge intensity function. Consider n extremum point ; long n epipolr line in direction <. Since= is n extremum point, Ï it follows tht the derivtive of 6798:* NM in the direction of < must be zero: QP >@?BADCFE. On the other hnd, ssume 6JLK7. Consider the isophoto curve, O R8-:*BS >HG TMVU 6798:*. Let <@ be the tngent direction of O t =. Loclly, the intensity in the tngent direction does Ï not chnge. It follows tht the derivtive vnishes in this direction, tht is, >@?BADCFE, When the >HGXW intensity in neighborhood of point is not constnt, the derivtive is zero in only one direction. It follows tht, < <3. To define the subsets of epipolr lines, it is sufficient to consider smll number of isophoto curves in the two imges. The subsets re then defined by the set of epipolr lines tht re tngent to the chosen isophoto. The property of extremum points long such lines, ensures tht the corresponding epipolr lines in the second imge hve lso been chosen. Our method exploits this property in less nive nd more efficient mnner. We consider finite number of regions of the infinite plne. A single point in ech region represents ll possible epipoles in the region (see Section 2.2). For ech region we compute nd sve the set of epipolr lines tht re tngent to given isophoto. To do so we compute the tngent lines for ech point on the chosen isophoto nd insert the tngent point prmeters into ech region entry which the tngent intersects. At the end of this step, we obtin lookup tble with smll number of epipolr lines for ech region. Given pir of cndidte epipoles ( region from ech imge), the two cndidte subsets re extrcted from the entry nd mtched. To determine the possible correspondence between lines in the two subsets, we consider lines tht re tngent to the isophoto of the sme vlue. In ddition, corresponding extremum points must both be mximl or both miniml. These two conditions re insufficient to fully determine the correctness of the correspondence between the subsets of cndidte epipolr lines. Further testing the correctness of the mtch is explined in Section Serching for epipoles In this section we present prtition of the infinite plne into finite number of regions. All points in given region will be treted, under this prtition, s single epipole.

6 M % Therefore, ll lines intersecting given region will be members of its epipolr line set. As we explined below, the size of ech region is proportionl to its loction with respect to the imge. PBY[\U^] `_ YBcbdYBfe Our plne prtition is defined by concentric circles with set of rdii, where for ech gihkjlh Y^m n nd. The center of YB these YBfe circles is the imge center. Let ech region be defined by the 4-tuple, 9o*Hpqo, where o nd o pqo YB YBfe re two ngles, nd nd Y Xr re two rdii Xr(see Figure 2). The prmeters of this prtition re the set of nd the set of pqo). For given ring, there YB rey[`e two degrees of freedom to define the set of regions in tht ring: the ring length, nd the region width, pqo. These prmeters re set such tht the prtition mintins the system resolution nd the equl hit mesure. We next define these two properties. For simplicity, we ssume circulr imge with rdius of one. System resolution: Any vision system is limited by the ccurcy of the mesurements. We define the system resolution to be s if it does not discriminte between two imge lines pssing through point when the difference in the line directions is s!tch's. We sy tht the prtition mintins the system resolution when the system cnnot discriminte between two cndidte epipoles which re locted in the sme region. Formlly, Assume the system resolution is s. Let u be given region nd v v*wyx.u be two points (see Figure 3 ). Let z nd z{w be two lines connecting n imge point nd the two points v nd v}w, respectively. When the ngle between z nd z{w is less thn the system resolution s, the system cnnot discriminte between these lines. More generlly, for given region u we define,~ R V to be the mximl m ngle between the imge point nd ny two points in the region. Let ~ ƒ),~ 7. If,~Šh s?@ $ˆ$ then the system resolution is mintined for u. p 3 p 1 Imge e 1 O δθ q β l1 l2 G e 2 p 2 p 4 Fig. 3. Œ is n imge point, nd re two points in given region Ž. The ngle d q q 0 Œ*š! "Œ}šXœV )žÿž / &Œ}š@žŸžŸžŸž ( 1Œ žÿž š. If + where is the system resolution, then the system cnnot discriminte between the epipoles / nd (. Note tht the system resolution cnnot be mintined in regions which overlp the imge or re very close to the imge. The closer the point Š ^ is to the region u the lrger ~ 7 becomes. In prticulr when x1u then ~ R V. Hit mesure: Roughly speking, the equl hit mesure property gurntees tht the number of epipolr lines considered for ech region is probbilisticly equl. We define the hit mesure of region, ul, to be the probbility of rndom epipolr

7 Y t ª t g Y Y line intersecting region u. Ech cndidte epipolr line we consider is defined by n imge point nd single direction. We ssume tht these epipolr lines re generted by uniform distribution of points in the imge nd uniform distribution of directions. Formlly, the following integrl computes Rul. ul «ª ª e ~ R8-:0)87): (2) [ F By chnging the integrl vribles from 8 nd : to the polr coordintes of the imge points, ± Y nd, we obtin: ² Rul ª g wh³ Yµ F ( Yµ f¹ ~ ±V$ ±V!±7 (3) ( Y is the determinnt of the Jcobin of the exchnge vribles). In order to hve probbilisticly equl number of points in ech region, we would like ² Rul to be identicl for ll regions., in order to mintin the bove conditions. We distinguish between three types of regions, which we nlyze seprtely. They include infinite regions on the outermost ring, regions within the imge nd close to the imge such tht the system resolution condition cnnot be mintined, nd intermedite regions. The prmeters depend on the desired system resolution, s. In the Appendix we describe how to set the prtition prmeters, Y nd pqo 2.3 Testing correspondence In this section we describe the tests used by our scheme to determine whether pir of cndidte sets of epipolr lines, defined by the epipoles, nd their correspondence is consistent solution. We use two types of tests. One is bsed on geometric properties of epipolr lines nd the other is bsed on photometric properties of epipolr lines. Geometric properties: We begin with the geometric tests. We use the cross-rtio of corresponding epipolr lines to verify the correctness of cndidte corresponding subsets of epipolr lines. The cross-rtio of ny four epipolr lines in one imge is equl to the cross-rtio of the corresponding epipolr lines in the other imge [11, 8]. The cross-rtio of four coplnr lines, z, z{w, z{º, nd z», tht meet t single point is defined by: where K lines, nd 7ŸK Rz z{w)z{º Hz»[ JLK RK º \7ŸK w K» JLKJw(RK7ºB\7ŸK K}»[ `¹ `¹ `¹ 1%+ `¹ 9¼%#s R Ri%#¼%#s, K w, K º nd K» re the intersection points of given line, z, with the four RK*½[ is the distnce between pir of points K nd K0½. The three ngles between the four lines re given by, nd s. Consider two subsets of epipolr lines, one from ech imge, nd mtch between them. To test the correctness of the correspondence, bsed on the geometric properties, (4)

8 ¾ Á ¾ it is sufficient to compre the cross-rtio of every four epipolr lines from one imge with the cross-rtio of the four corresponding epipolr lines in the other set. We next describe liner test, which is equivlent to the cross-rtio test, but is esier nd more robust to verify. The test is bsed on the 1D-homogrphy which reltes the direction of ech epipolr line in one imge to the direction of its corresponding )¹ epipolr line ¹ in the other imge [7, 10]. The directions of two corresponding lines, ¾ 9 } nd 9 * {, re relted by: )¹ 9 ¹ À¾ )¹ R }%' 7¾ 9 }%2 where,, nd re the coefficients of the 1D-homogrphy. These coefficients together with the epipole loctions in the two imges, prmeterize the fundmentl mtrix s )¹ `¹ 3! shown in Eq 1 [17, 7]. For numericl stbility we use the nd insted of the ¾. Reordering the terms of Eq 5 yields for ech pir of nd * : F ( * f¹ 3! * 3 ( f¹ `¹ * `¹ * `¹ } à à 4FÄ Ä 5 Š Given set of Å corresponding epipolr lines yields Å liner equtions tht reltes between the directions of these lines, s in Eq 6. These Å liner equtions define n Å ÆmÇ mtrix. To stisfy the 1D-homogrphy constrint, the rnk of this mtrix must be less thn four. This property cn esily be verified using SVD. The 1D-homogrphy test is performed in our method on smll number of lines. The test is reltively insensitive to smll errors in the epipolr line directions. Note tht for regions in distnt rings, the 1D-homogrphy test is insufficient since ll epipolr lines re lmost prllel. In Section 3.1 we consider this cse, which is similr to wek perspective projection model. Photometric properties: For cses in which more thn single set of corresponding epipolr lines pss the geometric test, we use n dditionl test to verify the correspondence, the photometric test. The photometric test is bsed on the observtion tht the intensity profile long the corresponding epipolr lines is identicl up to the foreshortening effect, due to the chnge of viewpoint nd occlusion (see Section 2.1). Cox & Roy [3] suggested to use this property to verify correctness of correspondence between epipolr lines. In their study they compred the histogrm of the intensity vlues long epipolr lines. We compre only the mximl nd miniml intensity vlues long the epipolr lines. These vlues re gurnteed to be the sme s long s there is no occlusion. The mtch should therefore llow for missing points due to occlusions. In Figure 1 we demonstrte the difference between the intensity profile long both corresponding epipolr lines nd non-corresponding epipolr lines. The photometric test is sensitive to smll errors in the epipolr lines direction. In ddition, it cnnot be used to test lrge number of lines, since it is reltively expensive test. We therefore suggest tht the geometric test be used first to eliminte incorrect cndidte sets of epipolr lines. Then the photometric test should be used s fine-tuning procedure for determining corresponding epipolr lines. (5) (6)

9 3 Specil cses The method presented so fr mkes no ssumptions bout the cmer prmeters. In this section we present severl specil cses in which we ssume knowledge of the cmer or the cmer s motion. In these cses, much more efficient vrints of the lgorithm cn be devised. We begin with the study of imges under the wek perspective projection model. We then del with internlly clibrted cmers, motion consisting of only trnsltion, nd scene contining plne. 3.1 The wek perspective projection model Under wek perspective projection, ll epipolr lines re prllel to ech other (the epipole is t infinity). The epipolr geometry in this cse is defined r by the directions of the epipolr lines in ech of the imges, nd scle fctor,, between the imges. The serch for the epipolr directions is equivlent to considering only the outer ring of the prtition defined in Section 2.2. If the system resolution is s then the serch tble hs only È É Ê s, which reduces considerbly the serch. To determine the subsets of corresponding epipolr lines, the sme method s in the full perspective cse cn be used. In this cse, ech epipolr line is entered into only two entries. This sves computtion of testing the intersection of given line with ech ring s in the full perspective cse. To verify whether the correspondence of cndidte subsets of epipolr lines is correct, we replce the cross-rtio test with the rtio test. Under the wek perspective projection r model, the rtio of the distnces between corresponding epipolr lines is the scle,. Therefore r two pirs of corresponding epipolr lines re sufficient to determine the scle,, nd dditionl pirs of lines cn be used to verify the consistency of the correspondence nd the correctness of the pir of epipolr directions. We next present liner test which is equivlent to the rtio test. Ech epipolr line cn be prmeterized by RËJ } where Ë is the distnce of the line from the origin nd is the line direction. In this cse the corresponding line 9Ë/ R * is relted to 9Ë7 } by liner trnsformtion Ë/!ËÌ%+, i.e. the prmeters 9Ë7Ë/ my be regrded s points tht lie on line. The liner test determining if these points ctully lie close to line is performed using the totl lest squres method. It cn be concluded tht, in the wek perspective cse, the complexity of both the serch nd the testing is substntilly lower. 3.2 Internlly clibrted cmers In cse of the internlly clibrted cmers, the epipolr geometry is determined by the two epipoles nd one dditionl prmeter. Thus, like in the wek perspective cse, only two pirs of corresponding lines re needed to recover the essentil mtrix. Therefore, the serch complexity in this cse is the sme s in the generl cse but the testing stge is fster. The problem of recovering the epipolr geometry from surfce outline (or rigid curves) ws studied by Cipoll et l. [2]. They suggested to use n itertive scheme to recover the epipolr geometry from n imge sequence.

10 3.3 Trnsltion When the motion between the two imges is known to not contin rottion component, gin more efficient lgorithms cn be derived. In this cse the epipoles nd corresponding epipolr lines in the two imges coincide. The only prmeter which is left to be found is the epipole loction. This cse is similr to Cross et l s method [4] to recover the epipolr geometry for smooth objects tht lie on plnr surfce. The plnr surfce is used to compute the homogrphy of the plne ppering in the two imges. Applying the homogrphy to one of the imges elimintes the rottion component of the motion, nd the only motion left is the trnsltion. They suggested computing the epipole s loction bsed on the bitngents of the silhouette in the homogrphy registered occluding contours. This method requires reltively complex outline where more thn two such bitngents exist. We suggest to use the bitngents to corresponding isophotos of the two imges. Since we cn use s mny isophoto curves s needed, the number of such bitngents is much lrger thn the bitngents to the object outline. In prticulr, when the object outline is too simple or invisible, our method cn still be used. 4 Implementtion We now turn to describe the implementtion of our method nd its result on pir of imges. In its current form, the implementtion is intended s proof of concept, showing tht the theory cn be pplied to rel-world imges. There is no ttempt t optimizing the implementtion in this version. The size of our serch spce, for exmple, is still too lrge to be prcticl tool. However, there re number of techniques tht cn be pplied to improve the overll performnce of the lgorithm. These include reducing the serch spce nd improving the stbility. Improving the lgorithm is topic for further reserch, which we re currently ctively pursuing. The implementtion of our lgorithm is divided into three prts: we first compute the prmeters of the lookup tbles for serching the epipole loction. We then extrct the isophoto curves from ech of the imges nd insert them into the lookup tbles. These two steps re liner in the size of the tble. Finlly we test pirs of regions to find the correct epipole loction nd the correct 1D-homogrphy which together define the fundmentl mtrix. This prt of the system is polynomil in the number of the tble entries, but is exponentil with the number of cndidte lines in ech entry. We next briefly describe ech of these components. Setting the lookup P[YB\Uq] tble f_ prmeters: P \Uq] f_ The lookup tble is defined by the plne prtition prmeters, nd pqo which re described in Section 2.2 nd in the Appendix. These prmeters re determined bsed on the chosen system resolution s nd the vlue Y ². The computtion is done by solving numericlly for ech ring the prmeter nd pqo such tht the system resolution Í*Î nd the Í*Î equl hit properties re mintined. The prmeters used here re s g nd pqo Ç. We obtined 137 rings, ech ring is prtitioned into sections.

11 Inserting epipolr lines into the lookup tble: We first compute the isophoto contours for few intensity vlues. For given intensity vlue, 6, the imge is trnsformed such tht the isophoto contours become strong intensity edges. It is done by pplying the function 6( 98:* e wðhð where 6JR8-:* is the intensity t pixel 98-:*, nd is smoothing fctor HÑ(ÒHÓDÔ\ÓŸÕ3Ö (see Figure ØÑ)Ô ÙÚXØ r[ï 4 for 6 g[û)û ). A stndrd Cnny edge detector [1] is then used to extrct the edges. The outcome of the Cnny edge detector consists of list of edgels long the isophoto curves. At ech edgel the grdient direction is estimted. The tngent direction is perpendiculr to tht direction. This tngent line which is cndidte epipolr line is inserted into entries in the lookup tble corresponding to regions in the epipolr tesselltion which it trverses. In Figure 5 we show the cndidte epipolr lines for the correct (-b) nd incorrect (c) regions in the exmple we tested. This process is repeted for severl isophoto vlues. r[ï (b) () (c) (d) Fig. 4. () The originl imge; (b) The sigmoid function for Iso=188; (c) The result of pplying the sigmoid function opertor to n imge; (d) The isophoto computed by the Cnny edge detector. Determine the correct epipoles nd correspondence: To determine the fundmentl mtrix the correct epipoles nd 1D-homogrphy between epipolr lines must be found. To find the correct pir of epipoles, serch is performed over the Crtesin product of the entries of the two lookup tbles. For ech pir of entries we test if the epipolr line subsets re comptible. In Figure 5 we show the cndidte epipolr lines for the correct epipoles (-b) nd for incorrect epipoles (c) in the exmple we tested. As cn be seen from these imges not ll the cndidte epipolr lines in one imge hve mtches in the other. This cn be due to the fct tht the corresponding epipolr line ws not detected or tht the epipolr line ws erroneously detected. Even so there exists quite lrge subset of corresponding lines.

12 () (b) (c) (d) Fig. 5. The results of running the epipolr line detection step on two imges with one of the isophoto vlues. The imges show the cndidte epipolr lines found for the two correct regions (-b), nd the cndidte epipolr lines found for n incorrect correct region (c). (d) The mtched distnces of corresponding lines from the origin yield points in the plne. The lines for the two isophoto vlues re mrked differently. The line fitted to these points is lso shown. In the experiment tht we performed we delt with the cse of distnt epipoles. In this cse the wek-perspective model cn be used. We therefore compred pirs of regions from the outermost ring of the plne tesselltion. We extrcted isophoto curves for two vlues 188 nd 235. For lines to be considered s corresponding they hve to be ssocited with the sme isophoto vlue nd hve the sme grdient direction (i.e. when the epipolr line is directed from the epipole both grdient directions should be to the sme side of the line. We looked for cndidte mtches for ech set of curves seprtely. The pirs of distnces of lines from the origin RË Ë/ should lie on stright line (this test is equivlent to the cross-rtio test of the full perspective projection model). For ech pir of regions we took the best mtch for ech of the two isophoto vlues nd fitted single line to both sets. The result obtined for the correct pir of regions is shown in Figure 5(d). As expected the best score ws obtined for the correct two regions. The finl results re shown in Figure 6. Corresponding epipolr lines hve been given the sme color. The next chllenge which hs to be delt with is to devise methods to reduce the complexity of the mtching lgorithm, especilly when deling with regions closer to the imges. In those regions we cn not ssume tht the wek-perspective line-fitting

13 Fig. 6. The set of corresponding epipolr lines re overlyed over the originl imges. Corresponding lines hve the sme color. test will work but need to rely on the 1D-homogrphy test. The min problem however is not this but the fct tht we hve to compre mny more pirs of regions. 5 Summry nd discussion The method presented in this pper ddresses the problem of epipolr geometry recovery from imges of smooth surfces. The min problem with such imges is tht no imge fetures clssiclly used for recovering epipolr geometry exist. Our method suggests epipole-dependent fetures which cn be used insted. These fetures re epipolr lines which re tngent to isophoto curves. Such epipolr lines cn be found nd mtched relibly in the two imges. For ech possible epipole set of epipolr lines of this type is esily found. Two such sets, one for ech imge, re then used to test the correctness of pir of epipoles. The testing is bsed on both geometric nd photometric properties of epipolr lines. Since these fetures re epipole-dependent, serch over the infinite spce of epipole loctions must be conducted. The tesselltion of the plne into regions suggested in this pper llows for n efficient serch for cndidte epipoles. This tesselltion cn lso be used for other problems tht involve serching for specil points in the infinite plne (e.g., vnishing points). There re two min limittions to our method. The first is the ssumption of constnt brightness. For this ssumption to hold, only the cmer is llowed to move while the scene nd the light sources remin sttionry. The second limittion is tht the timecomplexity of the implementtion is reltively lrge. However, in specil cses where knowledge on the cmers or the cmer s motion is known, the method my be considerbly simplified. In future reserch we intend to explore the possibilities for improving the performnce of the implementtion. This includes incorporting scle-spce techniques to reduce time-complexity.

14 Y Y w w r n Y Appendix: Setting the prtition prmeters, in order to mintin the system resolution nd the equl hit properties. We distinguish between three types of regions: regions on the outermost ring, regions within the imge nd close to the imge such tht the system resolution condition cnnot be mintined, nd intermedite regions. The prmeters depend on the desired system resolution, s. We next describe how to set the prtition prmeters, Y nd pqo In this cse the regions re defined such Y Y Ün \ tht, w. We next show tht it is possible to choose w nd pqo such tht the region, u, stisfies the system resolution. Outer ring regions: For regions in the outer ring the epipolr lines re lmost prllel. nd u 9o pqo* p 1 t 1 Imge O δθ q 1 αg ( q 1 ) G q 2 α G ( q 2 ) p 2 t 2 Fig. 7. This figure shows region in the outermost ring, Ž. The mximl ngle between n imge point nd two points in the region, Ý Þ, is obtined t one of the two imge points ß or ß. The point ß is the intersection of the imge circle with the ngle bisector à[á. The point ß is point on the imge circle such tht the tngent to the imge circle through ßJ psses through â-. Using trigonometric considertions, it cn be shown tht for n outer ring region, u, the mximl ngle between n imge point nd two points in the region, ~, is obtined t one of the two imge points ã-ä or Y ã å s illustrted in Figure 7 (the point t which the mximum is obtined depends on w nd pqo ). We next define these points. Let æ nd æ\w be the rys tht define n outer region, u. Let ; ä nd ; å be the delineting Y points of the region (the intersection points of the region rys nd the circle w ). The point ã ä is the intersection of the imge circle with the bisector of the ngle pqo. The point ã å is the intersection point of the imge circle with the tngent from the ç point ; ä to the circle. It cn be shown using trigonometric considertions tht ~!8 ~ Rã ä $ ~ ã å where,~ ã-ä N æxè7é f¹ F ( FêHë w Fêë w µ g )¹}ì ~ ã å( pqoì% jxåµé \g ÊqY w (7) It is therefore possible to set one of the two vribles, Y w or pqo, nd compute the other vrible such tht ~ í s where s is the system resolution. In our system we set

15 O U A û ö pqo Y, nd solve for w. The hit mesure, ² Rul, is then set to the vlue computed on u (using Eq 3). This vlue depends on the initil choice of pqo. YBfe Intermedite regions: Two prmeters define n intermedite region, nd pqo. We next show how to define these prmeters bsed on the system resolution, s, nd the hit mesure, Šç ul. Similrly to the outer ring cse, t n intermedite ring region u, ~ (8,R,~ Rã-ä($H~îRã å), where ã-ä is defined s in the outer region cse, nd ã å will be defined next. p 3 p 1 G Imge o δθ p 4 q 2 Fig. 8. This figure shows n intermedite region, Ž. The point ß stisfies tht the circumscribed circle of the points â- ï â7ð nd ßJ is tngent to the imge bounding circle. The ngle Ý Þ ß} ^š â ï\ß7 ^ï âvð is mximl for ll imge points which re outside the circulr section defined by à[á. Let O be the set of imge points on the imge bounding circle, outside the Qy VP imge section defined by o nd pqo. It cn be shown tht ~ ã å ~ Rã FS ãòx iff the circumscribed circle of the points ; ä ;ó nd ã å is tngent to the imge bounding circle (see Figure 8). This is true since t this point the circumscribed circle hs the smller rdius, nd therefore the ngle ô 9; ä Hã ä ;,ó! is mximl (by the sine theorem). It is now possible to write down the equtions tht define ã å. Let õ be the circumscribed circle center of ã å ; ä ;,ó. Assume TY[`e, tht thešyb imge center is t nd the imge rdius is one, in this cse, SfS ; ä S`S, S`S ;,ó7sfs, nd ã å D ö D. Under this setup the rdius of the circumscribed circle nd the ngles pqo nd s re given by: 3! ø/ùø(ú3û Rpqo! D ø0ù D D D ø(ú D F ( ø ù ø ú {s D ø0ù éjü EXA D D D ø(úéjü E D (8) éjü é7ü S`S ; äì.ýtsfs SfS ; ó.ýts`s SfS ýns`s[ g Given Y nd s nd one of the region prmeters Y `e, or pqo it is possible to solve these equtions for the other prmeter, s long s YBfe is lrge enough. The region must lso stisfy the equl hit mesure, which is defined in Eq 3. Our system solves these non. liner equtions by numericl methods yielding vlues of the prmeters Y[`e nd pqo m

16 Inner regions: In the inner regions, the system resolution constrint cnnot be mintined. We cn only mintin the equl hit mesure. Y[`e We therefore hve only one constrint. We set one of the unknown prmeters, nd pqo, rbitrrily, nd solve for the other one. In our system we set pqo to be constnt equl to the lst computed vlue, nd solve for Y `e. Acknowledgment We would like to thnk Nir Goldschmidt for his ssistnt in implementing the system. References 1. CANNY, J. F. A computtionl pproch to edge detection. IEEE Trnsctions on Pttern Anlysis nd Mchine Intelligence 8 (1986), CIPOLLA, R., ASTROM, K., AND GIBLIN, P. Motion from the frontier of curved surfces. In Procceding of the 5th Interntionl Conference on Computer Vision,Cmbridge, Msschusetts (1995), pp COX, I. J., AND ROY, S. Sttisticl modelling of epipolr mislignment. In Interntionl Workshop on Stereoscopic nd Three-Dimensionl Imging (1995). 4. CROSS, G., FITGIBBON, A. W., AND ISSERMAN, A. Prllx geometry of smooth surfces in multiple views. In Proceeding of the 7th Interntionl Conference on Computer Vision, Kerkyr, Greece (1999), pp FAUGERAS, O. Wht cn be seen in three dimensions with n unclibrted stereo rig? In Proc. Europen Conference on Computer Vision (1992), Springer-Verlf, pp FAUGERAS, O. Three-Dimensionl Computer Vision. MIT Press, Boston MA, FORSYTH, D., AND PONCE, J. Computer Vision - A Modern Approch. Prentice Hll, to pper. 8. FORSYTH, D. A., MUNDY, J. L., ISSERMAN, A., AND ROTHWELL, C. A. Recognizing rottionl symmetric objects from their outlines. In Proc. ECCV-92 (1992), G. Sndini, Ed., Springer-Verlg, pp HARTLEY, R. In defence of the 8-point lgorithm. In Proceedings of the Interntionl Conference on Computer Vision (1995), pp HARTLEY, R., AND ISSERMAN, A. Multiple Views Geoemtry in Computer Vision. Cmbridge University Press, KANATANI, K. Computtionl cross rtio for computer vision. Computer Vision, Grphics, nd Imge Processing 60, 3 (1994), LONGUET-HIGGINS, H. A computer lgorithm for reconstructing scene from two projections. Nture 293 (1981), LUONG, Q., AND FAUGERAS, O. The fundmentl mtrix: Theory, lgorithms, nd stbility nlysis. Interntionl Journl of Computer Vision 17, 1 (Jnury 1996), PORRILL, J., AND POLLARD, S. Curve mtching nd stereo clibrtion. Imge nd Vision Computing 9, 1 (1991), TORR, P., AND ISSERMAN, A. Roust prmeteriztion nd computtion of the trifocl tensor. Imge nd vision Computing 15 (1997), HANG,. Determining the epiplor geometry nd its uncertinty - review. Interntionl Journl of Computer Vision 27, 2 (1998), HANG,., R.DERICHE, O.FAUGERAS, AND LOUNG, Q. A robust technique for mtching two unclibrted imges through the recovery of unknown epipolr geometry. Artificil Inteligence 78, 1-2 (1995),