A Robust Method for Estimating the Fundamental Matrix

Transcription

1 Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney A Robust Method for Estmatng the Fundamental Matrx C.L. Feng and Y.S. Hung Department of Electrcal and Electrc Engneerng The Unversty of Hong Kong Pokfulam Road, Hong Kong {clfeng, yshung}@eee.hku.hk Abstract. In ths paper, we propose a robust method to estmate the fundamental matrx n the presence of outlers. The new method uses random mnmum subsets as a search engne to fnd nlers. The fundamental matrx s computed from a mnmum subset and subsequently evaluated over the entre data set by means of the same measure, namely mnmzaton of D reprojecton error. A mxture model of Gaussan and Unform dstrbutons s used to descrbe the mage errors. An teratve algorthm s developed for estmatng the outler percentage and nose level n the mxture model. Smulaton results are provded to llustrate the performance of the proposed method. Introducton A basc problem n computer vson s recoverng 3D constructon of a world scene or object from a par of mages. There are many approaches developed to solve ths problem, whch can be classfed nto stratfed reconstructon methods [] and drect reconstructon methods []. Projectve reconstructon s a necessary frst step n all these approaches. The eppolar geometry descrbes two-vew projectve geometry. The eppolar geometry can be expressed n terms of the fundamental matrx. The fundamental matrx contans all geometrc nformaton necessary for establshng pont correspondences between two mages, from whch projectve reconstructon of the scene or object can be nferred. Therefore, projectve reconstructon for two vews becomes a problem of estmatng the fundamental matrx. The fundamental matrx s estmated usng pont correspondences between two mages. The dffcultes n estmatng the fundamental matrx les n the fact that there are often a far proporton of msmatches n a gven set of pont correspondences. It s therefore mportant that the method used for estmatng the fundamental matrx should be robust n the presence of msmatches. There exstng several methods for the robust estmaton of the fundamental matrx [3]. M-Estmators [4, 5] reduce effect of nlers wth large nose or outlers by applyng weght functons, reducng the problem to a WLS (weghted least-squares) problem. Practcally, M-Estmators are not so robust to outlers. RANSAC (RANdom Sample Consensus) technque [] s a smple and successful method n robust estmaton. RANSAC removes the effect of outlers by usng random samplng as a search engne for nlers n the data set. Each soluton s determned by a random 633

2 Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney mnmum subset and evaluated aganst the entre data set for consstency. In detal, RANSAC calculates the number of supportng correspondences for each soluton and the one that maxmzes the support s chosen. A group of robust methods are developed based on RANSAC. LMedS [6] (Least Medan Squares) s smlar to RANSAC, except for the way to determne the best soluton. LMedS evaluates each soluton n terms of the medan Symmetrc Eppolar Dstances of the data set and chooses the one whch mnmzes ths medan. The MLESAC [7] (Maxmum Lkelhood SAmple Consensus) s a generalzaton of RANSAC usng the same pont selecton strategy. MLESAC maxmzes a lkelhood whch s a mxture model of normal dstrbuton (for nlers) and unform dstrbuton (for outlers) nstead of the number of supportng correspondences. The parameter of the model s estmated by expectaton maxmzaton (EM). MAPSAC [8] (Maxmum A Posteror SAmple Consensus) mproves MLESAC by maxmzng the posteror estmaton of the fundamental matrx and matches. MAPSAC provdes new evaluatons to check the consstency of each soluton wth the data set. In ths paper, we wll propose two mprovements to the robust estmaton of the fundamental matrx. Frst, n RANSAC lke methods, the result wll depend crtcally on the choce of a scorng functon for evaluatng solutons. The scorng functon s defned n terms of parameters such as the proporton of nlers γ and the nose level σ corruptng the nlers. It s essental that reasonable estmates of these parameters be provded. In the MLESAC method of [7], γ s estmated teratvely, but σ s estmated only a pror and a posteror (before and after) the estmaton of γ. Snce the scorng functon depends drectly on both γ and σ, we propose n ths paper that a better way s to estmate γ and σ together wthn an teratve algorthm. Ths would provde a more accurate estmate for both parameters. The accuracy of these parameters has an mplcaton on the relablty of the scorng functon. Secondly, n most methods, the fundamental matrx s estmated from a mnmum subset usng the seven-pont [] or the eght-pont algorthm[9], whch mnmze an algebrac error wth no geometrc meanng. We propose that a mnmzaton of the D reprojecton error n computng the fundamental matrx s more approprate. Ths s because the fundamental matrx s then evaluated n terms of reprojecton errors assocated wth mage ponts of the entre data set. It s mportant that a consstent measure s used n the ntal determnaton and subsequent evaluaton of the fundamental matrx, as these errors wll be used n a scorng functon for dscrmnatng nlers and outlers. Ths paper s organzed as follows. Secton brefly ntroduces robust estmaton of the fundamental matrx. In secton 3, the proposed robust method s descrbed n detal. In secton 4, smulaton results are provded to llustrate the performance of the proposed method. In secton 5, a concluson s gven. Robust estmaton of the fundamental matrx Consder pont correspondences n two mages projected from an object. The set of ponts { x}, =, K, n n the frst mage and the set of correspondng ponts { x '}, =, K, n n the second mage are related by 634

3 Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney T x ' Fx = 0, =, K, n () where the 3 3 matrx F s the fundamental matrx and x = ( xy,,) s n homogeneous form. The fundamental matrx has seven degrees of freedom because t s of rank two and defned only up to scale. Therefore at least seven pont correspondences are requred for calculatng the fundamental matrx. The fundamental matrx s computed from a set of pont correspondences S = {( x, x ') =, K, n}. Ponts are recognzed as corner ponts by a corner detector and matched as correspondences by a matchng algorthm. In practce a set of pont correspondences may be corrupted by nose or contan msmatched correspondences. So t s necessary to dscrmnate the set nto nlers and outlers. Inlers refer to correctly matched correspondences and outlers refer to msmatched correspondences. For a data set wth a gven proporton γ of nlers, the number of trals N requred to gve suffcently hgh probablty p to pck an outler-free subset consstng of r pont correspondences s r N = log( p) / log( γ ) () The general framework of exstng robust methods can be summarzed as followngs:. Repeat N tmes where N s adjusted adaptvely usng () when γ s updated. a. Select a mnmal subset S sub from the data set S accordng to a sample selecton strategy. b. Compute F j from the subset S sub (e.g. seven-pont algorthm, eghtpont algorthm) c. Evaluate the consstency of F wth all pont correspondences of the data set S.. Calculate error { } e for each pont of the data set S wth F j.. Evaluate F j by a scorng functon whch dscrmnates the data set nto nlers and outlers.. Select best soluton over a set of solutons { Fj}, j =, K, N accordng to ther scores. 3. Steps may be performed to mprove the result of step. Varous robust estmaton methods (lke RANSAC, MLESAC and MAPSAC) dffer n one step or another, wth step (c) beng the focus of recent research nterest. T 635

4 Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney 3 A new robust method We propose a robust method whch makes two mprovements wthn the above general framework, namely n stage (b) and (c)(). In stage (b), exstng methods typcally use the seven-pont or the eght-pont algorthm to determne F by mnmzng an algebrac error and a dfferent error measure may be used n step (c) for evaluatng F. For reasons of consstency, we would suggest (see Secton 3.) to use the mnmzaton of the D reprojecton error both n the determnaton and the evaluaton of F. In stage (c)(), the evaluaton of the scorng functon requres the estmaton of the γ and σ. We wll propose a new teratve algorthm for estmatng these parameters (see Secton 3.). 3. Determnaton and evaluaton of F by mnmzaton of D reprojecton error The D reprojecton error of all correspondng ponts n a par of mages s defned as e = e = d( x, x ) + d( x ', x ') subject to x ' Fx = 0. (3) It can be nterpreted geometrcally. Each correspondence x ( x, y,), x '( x ', y ',) 4 defnes a sngle pont denoted X ( x, y, x ', y ') n a measurement space. The corrected measurement denoted X ( x, y, x ', y ') les on a varety υ H defned by the constrant n (3). d(, ) s the Eucldean dstance between the ponts. The task of mnmzng the D reprojecton error s to fnd a varety υ and X whch are H { } closest ponts on the varety H X. The D reprojecton error s proven to be more superor than other geometrc error. The mnmzaton of D reprojecton error s often appled n evaluaton step (c). But t s usually not appled n other steps of the algorthm. In our algorthm, we wll solve the fundamental matrx by mnmzng the D reprojecton error. The problem of the fundamental matrx estmaton can be descrbed as υ to { } mn d(, ) + d( ', ') F = a subset of data { } x x x x subject to x ' Fx = 0. We use the factorzaton method of Tang and Hung [] to solve ths problem. The method teratvely mnmzes the D reprojecton error by evaluatng estmated camera matrx, 3D ponts and projectve depth each at a tme. Gven an estmate of the fundamental matrx, we wll evaluate the consstency of x, x ' n S wth F by the optmal trangulaton method correspondence ( ) x, x ' + d mn d( x, x ) ( x ', x ') subject to (4) x' Fx = 0. (5) 636

5 Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney Optmal trangulaton [0] s a lnear trangulaton method whch converts the least square functons to one parameter functons and fnds the global mnmum. 3. Mxture model parameters estmaton A scorng functon s ntroduced to evaluate how the fundamental matrx fts wth the data set (more exactly nlers) [8]. The scorng functon s defned as: C = ρ( e ) where e e σ T σ ρ( e ) = e T σ > T σ wth a modfcaton on the scorngs compared wth the one n[8]. In (6), T s a threshold for dscrmnatng outlers from nlers, defned by γ L T = log( ) + ( D d)log( ) γ πσ (6) (7) where L represents the sze of the search wndow for performng matches n the two mages, D s the dmenson of pont correspondences { X } and d s the dmenson of the varety υ H. Note that the contrbuton of each nler to the scorng functon depends on ts error whereas each outler ncreases the scorng functon by a constant. The scorng functon performs well when all the components { e },σ and γ are estmated correctly. In the prevous secton, we have consdered how the D reprojecton error { e } may be mnmzed. In ths secton, we wll propose a new teratve estmator of σ and γ. Frst, we wll model the probablty dstrbuton of D reprojecton errors for both nlers and outlers of the data set as a mxture densty []. We assume that the nose of nlers s a zero-mean Gaussan dstrbuton, whereas the matchng error for outlers s a unform dstrbuton. Thus, the error for each correspondence n the entre data set s a multvarate mxture model of the Gaussan and unform dstrbuton n D dmensons gven by D e pe ( σ ) = γ exp( ) + ( γ) πσ σ v In (8), γ s the pror probablty of the nlers occurrng, σ s the standard devaton of the Gaussan dstrbuton on each coordnate of pont correspondences, and { } e are the D reprojecton errors assocated wth pont correspondences n the data (8) 637

6 Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney set, whch can be regard as unlabeled samples drawn ndependently from the mxture densty. v ndcates the volume of the bounded space wthn whch outlers are expected to fall unformly. Put 4 D = as { } e s observed n 4. Regardng the state-condtonal probablty functon (8) as a lkelhood functon and then by maxmzng the lkelhood of the mxture model, σ and γ can be estmated usng an teratve method. Suppose ( σ k, γ k ) are already avalable, ( σ k +, γ k + computed usng the followng recursve formulas: ) can be n k z n γ + = (9) where z = γ k D e γ k exp πσk σ k D e exp ( γ ) + k πσk σ v k (0) ( z e ) ( z e ) and σ = = k + n n D z n D γ k+ () Algorthm The algorthm of mxture model parameters estmaton can be summarzed n followng steps:. Gven a fundamental matrx F, the optmal trangulaton s performed to obtan e for the entre data set. the D reprojecton errors { } ( ) medan e. Set the ntal estmate γ 0 = and σ 0 = D 3. Estmate posteror probablty { z}, =,, n for the data set from the current estmate γ k and σ k usng (0). 4. Make a new estmaton of γ k + (9) and σ k + () from the current Maxmum lkelhood estmaton { z }. 5. Step 3 and step 4 are repeated untl both γ k + and σ k + are convergent (typcally need fve tmes). 638

7 Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney 3.3 Algorthm summary Based on the above dscusson on the determnaton and evaluaton of the fundamental matrx and the mxture model parameter estmaton, we suggest a new robust algorthm for computng fundamental matrx. The objectve of the algorthm s to determne the maxmum lkelhood estmaton of the fundamental matrx whch s optmal under the assumpton that nlers obey Gaussan dstrbuton and outlers obey unform dstrbuton. Algorthm The algorthm of robust estmaton of the fundamental matrx. Randomly sample a mnmum subset of pont correspondences S j = {( x, x ') =, K, n} and estmate F j by factorzaton method of [].. Perform the projectve reconstructon to obtan X n 3D space and calculate the D reprojecton error { } { } e for the data set by the optmal trangulaton. 3. Use Algorthm to estmate the percentage of nlers γ and the standard devaton σ of measurement nose on each coordnate of mage ponts. 4. Compute the threshold T by (7) to dscrmnate nlers and outlers. 5. Compute the scorng functon for F j usng the equaton (6). 6. Go to step untl N trals. Update N by () usng γ f necessary. 7. Select the best soluton for whch the scorng functon has the smallest value. 4 Expermental Results Two sets of expermental results wll be gven to llustrate how Algorthm (for mxture model parameter estmaton) and Algorthm (for robust estmaton of the fundamental matrx ncorporaton the mxture model parameter estmaton) work. 4. Experment on parameter estmaton algorthm In ths experment, Algorthm for estmatng the parameters of a mxture model s tested. The expermental results are derved from ndependent tests on 00 sets of 00 pont correspondences wth varyng percentages of outlers. Gaussan nose s added to each coordnate of the mage ponts. The standard devaton of nose s from 0.5 to 3 pxels (n mages) wth 0.5 pxel ncrement. The ground truth fundamental matrx s used n Step of Algorthm to compute D reprojecton error, from whch the percentage of outlers and nose level are estmated usng Algorthm. The mean and standard devaton of the percentage errors n the estmaton of γ are gven n Fg.. For the purpose of comparson, the result of estmatng γ usng the MLESAC method of [7] s gven n Fg.. We note that the method used n MLESAC requres 639

8 Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney an exact estmate of nose level to acheve good results. Our method performs notceably better than the MLESAC method, reducng the percentage error n the estmated parameter γ by a factor of.5 n most cases. Mean percentage of error Gamma estmaton (00 tests) (our method) nlers 95% nlers 85% nlers 75% Gaussan nose level/pxel Standard devaton of percentage of error Gamma estmaton (00 tests) (our method) nlers 95% nlers 85% nlers 75% Gaussan nose level/pxel (a) (b) Fg.. (a) The mean and (b) the standard devaton of percentage error n the estmated γ (relatve to ground truth) for our method. Mean percentage of error Gamma estmaton (00 tests) (MLESAC) nlers 95% nlers 85% nlers 75% Gaussan nose level/pxel Standard devaton of percentage of error Gamma estmaton (00 tests) (MLESAC) nlers 95% nlers 85% nlers 75% Gaussan nose level/pxel (a) (b) Fg.. (a) The mean and (b) the standard devaton of percentage error n the estmated γ (relatve to ground truth) for MLESAC method. 4. Experment on our robust method In ths experment, the performance of Algorthm s tested usng synthetc data. The accuracy of the soluton F wll be assessed usng the Sampson dstance measure []: d sampson = n n 'T ( x Fx) T T ' ' ( Fx) + ( Fx) + F F x + x () 640

9 Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney ( F ) where x refers to the j-th entry of F x. The underlnng symbol j x ndcates the nose-free data. The expermental results are derved from ndependent tests on 50 sets of 500 pont correspondences. The mage sze s On each coordnate, the standard devaton of Gaussan nose σ s pxel. The percentage of outlers γ s ncreased from 5% to 30% n ncrements of 5% steps. Our method s compared wth MAPSAC. The smulaton results show that our method performs notceably better than MAPSAC method though both methods have a small proporton (about 6%) of falures n reconstructng acceptable solutons. These bad solutons have been excluded from the calculaton of the mean Sampson dstance shown n Fgure 3. Mean Sampson Error of nose-free data The Evaluaton of F our method MAPSAC Percentage of outlers (%) Fg. 3. The mean of the Sampson dstance of the solutons estmated by our method and MAPSAC (after elmnatng 3 bad solutons among total 50 solutons). 5 Concluson In ths paper, a robust method has been developed to gve an accurate estmaton of the fundamental matrx n the presence of outlers. Our method dffers from exstng method n two ways. Frst, we mnmze the D reprojecton error n both the determnaton and evaluaton of the fundamental matrx, n contrast to other methods whch do not necessarly use a consstent measure throughout the entre process. Secondly, we estmate both unknown parameters γ and σ n a mxture model for mage errors of nlers and outlers. These mxture model parameters are estmated together wthn an teratve algorthm, whch provdes better results than exstng ones. 64

10 Proc. VIIth Dgtal Image Computng: Technques and Applcatons, Sun C., Talbot H., Ourseln S. and Adraansen T. (Eds.), 0- Dec. 003, Sydney References. M. Pollefeys. Self-calbraton and metrc 3D reconstructon from uncalbrated mage sequences. PhD thess, EAST_PSI, K.U.Leuven, R.I. Hartley and A. Zsserman. Multple Vew Geometry n Computer Vson. Cambrdge Unversty Press, X. Armangue and J. Salv. Overall vew regardng fundamental matrx estmaton. Image and Vson Computng (003), pp Z. Zhang, R. Derche, O. Faugeras and Q.-T. Luog. A robust technque to mage matchng: Recovery of the eppolar geometry. In Proc. Internatonal Symposum of Young Investgators on Informaton\Computer\Control, Bejng, Chna, 99, pp P.H.S. Torr and D.W. Murray. The development and comparson of robust methods for estmatng the fundamental matrx. Internatonal Journal of Computer Vson 4 3 (997), pp Z. Zhang. Determnng the eppolar geometry and ts uncertanty: a revew. Internatonal Journal of Computer Vson 7 (998), pp P.H.S. Torr and A. Zsserman. MLESAC: a new robust estmator wth applcaton to estmatng mage geometry. Computer Vson and Image Understandng 78 (000), pp P.H.S. Torr. Bayesan model estmaton and selecton for eppolar geometry and generc manfold fttng. Internatonal Journal of Computer Vson 50 (00), pp R.I. Hartley. In defense of the eght-pont algorthm. IEEE Transactons on Pattern Analyss and Machne Intellgence, 9 6 (997), pp R.I. Hartley and P. Sturm. Trangulaton. Computer Vson and Image Understandng, 68 (997), pp W.K. Tang and Y.S. Hung. A factorzaton-based method for projectve reconstructon wth mnmzaton of -D reprojecton errors. In Proceedngs of DAGM 00, Zurch, Sept, 00, pp R.O. Duda, P.E. Hart and D.G. Stock. Pattern Classfcaton (nd ed.). John Wley & Sons Inc,