Homography Estimation with L Norm Minimization Method

Transcription

1 Homography Estmaton wth L orm Mnmzaton Method Hyunjung LEE Yongdue SEO and Rchard Hartley : Department o Meda echnology Sogang Unversty, Seoul, Korea, whtetob@sogang.ac.r : Department o Meda echnology Sogang Unversty, Seoul, Korea, ynd@sogang.ac.r : Australan atonal Unversty Canberra, Australa Rchard.Hartley@anu.edu.au Abstract Mnmzng L error norm or some geometrc vson problems provdes global optmzaton usng the well developed algorthm called SOCP (second order cone programmng). Because the error norm belongs to quas-convex unctons, bsecton method s utlzed to attan the global optmum. It tests the easblty o the ntersecton o all the second order cones due to measurements, repeatedly adjustng the global error level. he computaton tme ncreases accordng to the sze o measurement data snce the number o second order cones or the easblty test nlates correspondngly. We observe n ths paper that not all the data need be ncluded or the easblty test because we mnmze the maxmum o the errors; we may use only a subset o the measurements to obtan the optmal estmate, and thereore we obtan a decreased computaton tme. In addton, by usng L mage error nstead o L Eucldean dstance, we show that the problem s stll a quas-convex problem and can be solved by bsecton method but wth lnear programmng (LP). Our algorthm and expermental results are provded. Introducton Many problems n multple vew geometry have been modeled as optmzaton problems and solved usng lnear methods or teratve estmaton algorthms based on the L norm such as Levenberg-Marquadt [5]. Recently, convex programmng technques have been ntroduced as another ecent tool o optmzaton. By swtchng rom L sum o squared error uncton to L, we are now able to nd the global optmum o the error uncton snce, or several mult-vew geometry problems, the mage re-projecton error has ound to be o quas-convex type that can be ecently mnmzed wth bsecton method [6]. hs L norm mnmzaton, a type o convex optmzaton, s advantageous because we do not need to buld a lnearzed ormulaton to nd an ntal soluton or teratve optmzaton le Levenberg-Marquadt, but also t provdes the global optmum o geometrcally meanngul error wth a well-developed optmzaton algorthm. hat s, we are guaranteed that there s no chance o allng nto a local mnmum. Applyng an dea o L norm mnmzaton was presented by Hartley and Schaaltzy n [], where t was observed that many geometrc vson problems have a sngle global mnmum under L error norm. Kahl, and Ke & Kanade, respectvely, extended the wor by showng that the error unctons are o quas-convex and can be solved by Second Order Cone Programmng (SOCP) [6, 7]. here are readly avalable sotware pacages such as SeDuM, CSDP, etc, that can solve SOCP [4, ]. Recent growth n ths subject has ound that many vson problems can be solved under L ormulaton. hey are two vew trangulaton problem [], the multvew trangulaton problem [], homography estmaton and camera resectonng [6, 7], multvew reconstructon nowng rotatons or homographes nduced by a plane [6, 7], camera moton recovery [], and

2 removng outlers []. In all these cases, a sngle global mnmum exsts and t may be ound by SOCP. Other varous applcatons can be ound n [8]. Bsecton method. Snce the problems above-mentoned are all quas-convex type, bsecton method s utlzed to obtan solutons. It repettvely solves a easblty problem gven a multple o second order cones each o whch s due to a measurement. Algorthm shows the bsecton method. Gven a bound γ that corresponds to the radus o every second order cone, the thrd step n the repetton loop nds a parameter vector X nsde the ntersecton o all the cones. From ths, one may see that the computatonal complexty o bsecton method reles on two thngs. One s the number o data snce one data s equvalent to one second order cone. he other s the gap between the upper and lower bounds. hereore, reducng the number o measurement data or the easblty test wll decrease the computatonal cost, whch s our strategy. he dea o usng only a small number o nput data could be caught rom the wor o Sm and Hartley on removng outlers []. In that, the measurements o maxmum error are dscarded because at least one o them belongs to the outlers. Bascally, ths s due to the act that L optmzaton mnmzes not a nd o average but the maxmum; that s, only the maxmum error s consdered but the others are gnored. However, they used all the nput data or the optmzaton n order to nd the data havng the maxmum error. Regardless o the exstence o outlers, we would not have to nclude all the data we new the data set o maxmum error. Mnmzng the maxmum error or the partal data set would produce all the same result. We run the optmzaton wth a small number o data. I the error s maxmum or all the measurement data, then no more optmzaton s necessary. Otherwse, we nclude one more measurement to run another optmzaton. hs requres a new teraton loop but because the number o measurements s ept small we wll obtan reduced computaton. hs s man dea we espouse n ths paper. Algorthm Bsecton method to mnmze L norm Input: ntal upper(u )/lower( L ) bounds, tolerance ε >. repeat untl γ : = ( L + U ) / Solve the easblty problem (7) easble then U L ε U := γ else L := γ Image space error. Eucldean dstance has been adopted as the mage error or parameter estmaton. he lterature has shown that Eucldean ( L ) dstance uncton as the reprojecton error s a quas-convex uncton and denes a second order cone or the parameters. I we tae L uncton, the error uncton s ound to be stll o quas-convex. However, the constrants due to the error uncton are lnear nequaltes n place o a second order cone. We use lnear programmng (LP) n ths paper or nter-mage homography rather than second order cone programmng (SOCP). In theory, second order cone can be approxmated by nnte number o hal-spaces dened by those supportng hyperplanes. Hence, our usng LP by ntroducng L mage error can be thought o as an approxmaton to SOCP as well. Mnmzng L Error orm hrough an optmzaton problem, we see to t a model, parameterzed by a vector X, to a set o data measurements u ndexed by n a nte ndex set I. he error unctons or each measurement that arse n the geometrc problems o nterest to us can be wrtten n the ollowng orm, A X b = () c X For example, the parameter vector n trangulaton problem corresponds to D space pont

3 [ X, Y, Z, ] X = and the re-projecton error the L norm ( d ( X )) ( d ( )) ) ( X ) = X () + o the ollowng dscrepancy d (X ) or smply d or the measurement u p X p X d = u, u, () p X p X where p s the -th row vector o 4 camera matrx P, and u [ ] = u, u. It s shown n [] that the error uncton (X ) s quas-convex. hat s, any o the γ -sublevel set C = { X ( X ) γ } (4) o s convex or γ R+ n the convex doman D = { X p X }. ote that the set C o a gven bound γ denes a second order cone due to Equaton. he bound γ s called the radus o cone and corresponds to the magntude o error n the mage space; we have an ellptc ds = d γ or γ around the measurement u. =, becomes the total error vector where s the number o measurements. A easble set F γ gven a γ s dened as the ntersecton o all the second order cones he vector [,..., ] { X } F γ = I γ (5) = { X γ }... I{ X γ } = I (6) We now have the easblty problem or the gven global error level γ : nd X (7) subject to γ, =,..., where X s any vector n F γ. ote that the bound γ s not a varable but a constant, and the easble set F γ s convex because t s the ntersecton o (ce-cream shape) convex cones o the same radus γ. Indeed, the easblty problem (7) has already been nvestgated and well-nown n the area o convex optmzaton; n our case, the soluton X can be obtaned by applyng second order cone programmng usng a solver such as CSDP or SeDuM [, 4]. he L norm o s dened to be the maxmum o s, whch s also a quas-convex uncton, and the L mnmzaton problem s to nd X such that the maxmum error o s s mnmzed: X {,,..., } mn max (8) hs mn-max problem s equvalent to mnmzng the global bound γ under the constrants γ : mnmze γ (9) subject to γ, =,..., he actual computaton o ths mnmzaton s done by checng whether or not the ntersecton F γ s nonempty and adjustng the bound γ. Fndng through teraton the smallest γ (and ts correspondng X opt, the parameter at the global optmum) that yelds non-empty easble set F γ nshes the optmzaton. hs s the man dea o bsecton method presented n Algorthm [, 6]. ote that ths s equvalent to repeatedly solvng the easblty problem 7, adjustng the bound γ. Let µ be the set o nput measurements. ow, let us choose a subset S or the optmzaton rom µ ; the subset S must contan at least mnmum number o data or parameter ttng - our or homography, or example. By applyng the bsecton algorthm (BA n bre) to the set S, we wll obtan maxmum resdual e and correspondng parameter X : ( e, X ) : = BA( S, U, L ); () where U and L s the ntal upper and lower bounds requred as nput to the bsecton algorthm. Practcally, L =, the mnmum possble error, and U = U const, a sucently large value. ow let us evaluate the qualty o the result o ths optmzaton. I the parameter X s globally optmal, that s, optmal or the whole data set µ, then we wll have e, where = ( X ), or all, we do not need any more optmzaton step. Even though actual probablty o ths case may be very low, t s

4 mportant to note that ths s not mpossble: Proposton here exsts a mnmum subset S µ that yelds the same L optmzaton result as the whole set µ does. Such a subset S could be ound by choosng the measurements o maxmum errors ater runnng the bsecton algorthm or the whole data set µ as was done n []. Fndng an S beore optmzaton seems not easy, practcally. However, we see that any subset that ncludes S yelds the same optmzaton result. Proposton here exst subsets S, =,..., K, K such that S S... S µ havng the same optmzaton result. In ths paper, we say that the set S µ s equvalent to µ t has the same optmzaton result e up to a computatonal resoluton; that s, e, =,..., () hereore, when X s ound to be not the global optmum n practce, then, nstead o ndng the subset S o mnmum number o data, we may try to nd an equvalent super-set S S that s equvalent to µ. hs can be done by ncludng a measurement (or a ew) rom µ \ S, runnng BA agan and testng the result. Here, we need a strategy to choose a measurement; the measurement o choce o ours s the one u * havng the maxmum error * = max ( X ). In addton, a derent par o upper and lower bounds, whose gap s tghter than the ntal par, can be ound rom the optmzaton result: L = e and U = max ( X ). hs s due to ollowng: Proposton he error e rom the L optmzaton or the set S = S U{ u * } must be n the range Proposton 4 he sequence non-decreasng sequence. e e * () e, =,..., K s a hs par o bounds wll yeld a reduced number o teratons nsde the bsecton algorthm. Proposed Algorthm ow, we presents our bsecton algorthm to compute the L optmzaton. *Input: µ,the set o measurements u, =,...,.. =. Choose ntal set S o measurements arbtrarly. U =, L = ; ; // ntal upper/lower bounds. Run the algorthm : ( e, X ) : = BA( S, U, L );. Compute the error or the rest o the measurements: : = RE( u, X ); or every u µ \ S. 4. e or every u µ \ S then stop; * 5. Fnd where * s the maxmum: e * or every u µ \ S. 6. Insert u to S and update the bounds: * + S = S U{ u } : * + + : = e *, L : U 7. goto step. = e ote that the number o data at Step should be sucent or the ttng problem, BA() represents the subroutne o bsecton algorthm, and RE() the subroutne o computng reprojecton error. Our method selects the measurement that s expected to ncrease the L error. At every teraton, the measurement o maxmum reprojecton error s chosen and joned to the sample set S or L estmaton. hs process s then repeated untl the set S becomes an optmal set, that s, equvalent to µ, havng the same optmzaton result. 4 LP wth L Reprojecton Error In the ormulaton o SOCP-based optmzaton, Eucldean dstance has been used n the lterature to measure the reprojecton error as gven n Equaton or n Equaton and Equaton or trangulaton example. Eucldean ( L ) dstance or the dscrepancy vector d gves a second ; 4

5 order cone wth respect to the parameter vector X. he easblty o the ntersecton o the second order cones s then checed to solve the problem, and hence an SOCP solver s needed. In place o Eucldean dstance, let us consder to use the maxmum o the two elements o d as the reprojecton error: max{ } = d, d. hen, the standard orm o n Equaton becomes A X b A X b = max, () c X c X ote that each o the error unctons s o convex-over-concave style; n each, the numerator s a non-negatve convex uncton (absolute value o the argument) and the denomnator s a postve ane (hence a concave) uncton. hs means that the error uncton gven n Equaton s also quas-convex. See [] or the quas-convexty o convex over concave unctons. Postveness o the denomnator s due to [4]. We note that when we are gven a postve bound γ, each uncton results n two lnear constrants n A X b c Fg. : Comparson o computaton tmes n X > γ c X, or n =, (4) Subsequently, the L norm o s now to nd a parameter n the ntersectons o 5 lnear nequaltes; we can solve ths optmzaton problem by usng a lnear programmng technque. Experments. We mplemented the proposed method wth LP n C or the experments o nter-mage homography. 5 Inter-Image Homography hs problem s to estmate the homography H when nler correspondng ponts u, u between two mages are gven. he error vector or a measurement s gven by Equaton. h u h u d = u, u, h u h u Here, we dd real experments. Image matches were ound usng KL[9]. We collected 6 data sets havng or more matches and tested our algorthm. Intal data set S had our measurements chosen randomly because our s the mnmum number o data or computng homography. Fgure shows the average o the computaton tme as the number o data ncreases. Orgnal method too.5 seconds when data ponts were gven, but t spent.5 seconds when 449 data ponts were provded. However, our method consumed only.94 seconds or the largest data set (=449) even though t ased.47 seconds o tme or the smallest data set (=). Fgure shows the evolutons o e and, the maxmum reprojecton error at the -th * Fg. : Evoluton o errors teraton. Green lne shows the evoluton o maxmum errors, and blue lne whch ncreased as e. teraton went on shows the evoluton o Eventually, e * when = 8, and the optmzaton nshed. Fgure s the result o mage sttchng usng 5

6 homography computed by our proposed algorthm. As we expected, the reduced computaton tme n Fgure was obtaned due to ndng and usng an equvalent data set, havng reduced number o measurements but yeldng the same optmzaton result. 6 Concluson Fg. : Image Sttchng Result he approach dscussed n ths paper provded an L norm mnmzaton method showng a reduced amount o computaton tme when the number o nput data was rather large compared to the number o parameters. We utlzed the act that the L norm mnmzaton could be done wth a subset o the nput measurements as long as the subset was equvalent to the nput data set. Each executon o bsecton method was gven not only a reduced number o data but also a par o upper and lower bounds havng a smaller gap. We also showed that LP could be adopted n place o SOCP by ntroducng L mage error. We beleve that our method s useul or large scale vson problems - thousands o mages and mllons o matchng ponts. Acnowledgment hs wor was supported by the Korea Scence and Engneerng Foundaton (KOSEF) grant unded by the Korea government (MOS) (o. R ) and also accomplshed as the result o the research project or culture contents technology development supported by KOCCA. Reerences [] B. Borchers. CSDP, A C lbrary or semdente programmng. Optmzaton Methods and Sotware, -:6 6, 999., 4 [] S. Boyd and L. Vandenberghe. Convex Optmzaton. Cambrdge Press, 4., 4, 5 [] R. Hartley and F. Schaaltzy. L mnmzaton n geometrc reconstructon problems. In Proc. IEEE Con. Computer Vson and Pattern Recognton, 4., [4] R. I. Hartley. he chralty. Int. Journal o Computer Vson, 6():4 6, [5] R. I. Hartley and A. Zsserman. Multple Vew Geometry n Computer Vson. Cambrdge Press, 4. [6] F. Kahl. Multple vew geometry and the L -norm. In Proc. Int. Con. on Computer Vson, pages 9, Bejng, Chna, 5., 4, 5 [7] Q. Ke and. Kanade. Quasconvex optmzaton or robust geometrc reconstructon. In Proc. Int. Con. on Computer Vson, Bejng, Chna, 5. [8] M. Lobo, L. Vandenberghe, S. Boyd, and H. Lebret. Applcatons o second-order cone programmng. Lnear Algebra and Its Applcatons, 84:9 8, 998. [9] [] D. st er. Automatc Dense reconstructon rom uncalbrated vdeo sequences. PhD thess, Royal Insttute o echnology KH, Sweden,. [] W. Press, S. euolsy, W. Vetterlng, and B. Flannery. umercal Recpes n C: he Art o Scentc Computng. Cambrdge Unversty Press, [] K. Sm and R. Hartley. Recoverng camera moton usng L mnmzaton. In Proc. IEEE Con. Computer Vson and Pattern Recognton, 6. [] K. Sm and R. Hartley. Removng outlers usng the L norm. In Proc. IEEE Con. Computer Vson and Pattern Recognton, 6., 4 [4] J. Sturm. Usng SeDuM., a Matlab toolbox or optmzaton over symmetrc cones. Optmzaton Methods and Sotware, -:65 65, 999., 4 Hyunjung, Lee: receved the M.S. degree n Meda echnology rom Sogang Unversty, Korea, n 5. As a Ph.D student, her research nterest ncludes optmzaton and D reconstructon. Yongdue, Seo: He s an assocate proessor o meda technology n Sogang Unversty. Hs research nterests nclude optmzaton, real-tme computer vson, camera sel-calbraton, and augmented realty. Rchard, Hartley: He s a proessor o Australan atonal Unversty, Canberra, Australa. He s one o the author o Multple Vew Geometry n computer vson. 6