RELATIVE CAMERA POSE ESTIMATION METHOD USING OPTIMIZATION ON THE MANIFOLD

Transcription

1 he Internatonal Archves of the Photogrammetry, Remote Sensng Satal Informaton Scences, Volume XLII-1/W1, 017 ISPRS Hannover Worsho: HRIGI 17 CMR 17 ISA 17 EuroCOW 17, 6 9 June 017, Hannover, Germany RELAIVE CAMERA POSE ESIMAION MEHOD USING OPIMIZAION ON HE MANIFOLD Chuanq. Cheng a, *, Xangyang. Hao a, Jansheng. L a, b a School of Navgaton Aerosace Engneerng, Informaton Engneerng Unversty, Zhengzhou , Chna b Insttute of Photogrammetry GeoInformaton, Lebnz Unverstät Hannover, Germany (legendq, xangyanghao004, lszhx)@16.com Commsson VI, ICWG I/IV KEY WORDS: Pose Estmaton, Otmzaton, Manfold, Le Grou/Algebra, Levenberg-Marquardt ABSRAC: o solve the roblem of relatve camera ose estmaton, a method usng otmzaton wth resect to the manfold s roosed. Frstly from maxmum-a-osteror (MAP) model to nonlnear least squares (NLS) model, the general state estmaton model usng otmzaton s derved. hen the camera ose estmaton model s aled to the general state estmaton model, whle the arameterzaton of rgd body transformaton s reresented by Le grou/algebra. he acoban of ont-ose model wth resect to Le grou/algebra s derved n detal thus the otmzaton model of rgd body transformaton s establshed. Exermental results show that comared wth the orgnal algorthms, the aroaches wth otmzaton can obtan hgher accuracy both n rotaton translaton, whle avodng the sngularty of Euler angle arameterzaton of rotaton. hus the roosed method can estmate relatve camera ose wth hgh accuracy robustness. 1. INRODUCION In general, there are three nds of relatve camera ose estmaton models: D-D, D-D, D-D, whereas the nd of D-D model s the most wdely used n hotogrammetry, ncremental structure-from-moton (SFM), vsual smultaneous localzaton mang (V-SLAM), augmented realty, autonomous navgaton so on. It can be descrbed as that how to determne the orentaton oston of a fully calbrated ersectve camera, gven n (n ) D onts n the world framewor ther corresondng D mage onts, whch s also nown as the ersectve-n-ont (PnP) roblem (Hartley, Rchard, 00). Consderng the mortance of PnP roblem, a large amount of wor has been done n the ast few decades. he PP roblem attracts a lot of researchers nterests, such as (L, 011) (Rec, M. Q., 014). Usually PP solutons are mlemented wth RANSAC outler reecton scheme. In ractce, there are often more than onts consderng the redundancy can generally mrove accuracy, most of recent wors on PnP roblem concentrate on the stuatons wth more than onts. Roughly, the state-of-the-art solutons on PnP roblem can be dvded nto two tyes the mult-stage method the drect mnmzaton method. ycally, the mult-stage methods frst estmate the onts coordnates n the camera framewor, transform the PnP (D-D) roblem nto D-D ose estmaton roblem. Also, the lnear methods usually have closed-form solutons. Latest rogress n lnear methods ncludes EPnP (Leett, V., 009), RPnP (L, S., 01), OPnP (Zheng, Y., M, 01). Leett, et al. (Leett, V., 009) exresses the n D onts as a weghted sum of four vrtual control onts, mang the PnP roblem reduce to estmate the coordnates of these control onts n the camera referental, whch reduces the comlexty to O(n). L, et al. (L, S., 01) onts out that due to underlyng lnearzaton scheme, EPnP erforms oor for slghtly redundant cases wth n = 4 or n = 5. hen, L, et al roose another non-teratve O(n) soluton whch retreves the otmum by solvng a seventh order olynomal. Zheng, et al (Zheng, Y., M, 01) ut forward a non-teratve O(n) soluton whch transforms the PnP roblem to an unconstraned otmzaton roblem solved by a Grobner bass solver. Moreover, the well-nown drect lnear transformaton (DL) s also a mult-stage method, because t frst estmates the roecton matrx extracts the camera ose. However due to gnorng the orthogonal constrant of rotaton matrx, ts accuracy s oor. he second tye of PnP solutons s drect mnmzaton methods. Its man dea s to mnmze a defned energy functon (or cost functon), ether n the mage sace or n the obect sace, whch contans all nonlnear constrants. here exsts some reresentatve drect mnmzaton methods. Lu et al. (Lu, C. P., 000) roose an orthogonal teraton method to mnmze the obect sace collnearty error, whle Garro et al. (Garro, V., 01) roose an alternatve mnmzaton method to mnmze the D sace geometrc error. Hesch, et al. (Hesch, J. A., 011) resent a drect least squares (DLS) method for comutng all solutons of the PnP roblem by solvng a system of three thrd-order olynomals. However, due to the Gayley reresentaton of rotaton, there are degeneraton cases. hen they rovde a remedy to conquer the degeneracy of the Gayley reresentaton by solvng DLS three tmes under dfferent rotated D onts, whereas the comutatonal tme s trled. o sum u, all the mentoned mult-stage methods are generally oor n accuracy, whle the drect mnmzaton methods suffer from the rs of gettng traed nto local mnmum. So n ractse, we often frstly acqure an ntal guess about the PnP * Corresondng author hs contrbuton has been eer-revewed. do: /srs-archves-xlii-1-w

2 he Internatonal Archves of the Photogrammetry, Remote Sensng Satal Informaton Scences, Volume XLII-1/W1, 017 ISPRS Hannover Worsho: HRIGI 17 CMR 17 ISA 17 EuroCOW 17, 6 9 June 017, Hannover, Germany soluton usng mult-stage methods such as EPnP, DL or RPnP, then we use otmzaton method such as Gauss- Newton or Levenberg-Marquardt scheme to generate an otmal result. However, n otmzaton, esecally n comuter vson robotcs, the correct treatment of angles consstently causes confuson. On one h, a mnmal arameterzaton s desred, but also sngulartes should be avoded. Interolaton of angles s not straghtforward, snce the grou of rotaton s only locally Eucldean. Probably the most elegant way to reresent rgd body transformaton s usng a Le grou/algebra reresentaton. hus we resent an aroach for relatve camera ose estmaton usng otmzaton method wth resect to the Le grou, whch can avod the sngularty of Euler angle arameterzaton of rotaton, mae the otmzaton method such as Gauss-Newton or Levenberg-Marquardt (Moré, J. J., 1978) more robust convenent.. MEHODS FOR SAE ESIMAION USING OPIMIZAION In ths secton, we gve a bref revew of state estmaton usng otmzaton. hs secton defnes the common notaton technology for the rest of the aer ntroduces dfferent tyes of otmzaton our method uses..1 Maxmum-a-osteror (MAP) estmaton Least Square Problems In general, we want to estmate a set of unnown varables gven a set of measurements f, where we now the lelhood functon ( f ). We estmate by comutng the assgnment of varables osteror ( f ): * that attans the maxmum of the * f f arg max ( ) arg max ( ) ( ) In case no ror nowledge s avalable, ( ) becomes a constant (unform dstrbuton) whch s nconsequental can be droed. hen MAP estmaton reduces to maxmum lelhood estmaton (MLE). Assumng that the measurements are ndeendent, roblem (1) factorzes nto: * f f arg max ( ) arg max ( ) In order to wrte () n a more exlct but stll wdely alcable form, assume that the measurement nose s a zeromean Gaussan nose wth nformaton matrx measurement lelhood n () becomes: 1 f, 1 f (1) (). hen, the 1 ( f ) ex( f fˆ ( ) ) () Snce maxmzng the osteror s the same as mnmzng the negatve log-osteror, or energy, the MAP estmate n () becomes: arg mn ( ) * arg mnlog( ( f)) 1 arg mn f fˆ ( ) 1 f, whch s a nonlnear least squares roblem.. Otmzaton Methods ( ) s smly a sum of squares, to mnmze t s called nonlnear least squares otmzaton. A common technque for nonlnear least squares otmzaton s the Gauss-Newton (GN) method. he Gauss-Newton method erforms teratvely, startng from a gven ntal guess 0 udates by the rule: ( 1) ( ) (4) (5) where at each ste the udate vector s found by solvng the normal equaton: Here, J ( J J ) J r (6) 1 1 f f r r f f ˆ( ) s the resdual error. A wdely used otmzaton method s a varant of GN called Levenberg-Marquardt (LM), whch alters the normal equaton as follows: ( J J dag( )) J r (7) f f f he arameter rotates the udate vector towards the drecton of the steeest descent. hus, f 0, ure GN s erformed, whereas f, gradent descent s used. In LM, the udate ste s erformed only f t can sgnfcantly reduce the resdual error. he arameter s self-adated n the LM method.. RELAIVE CAMERA POSE ESIMAION MODEL.1 he Camera Proecton Functon Camera Poses Ponts n the world usng the observaton functon: Here, the x R are maed to the camera mage zˆ(, x ) ro ( K x ) (8) x s homogeneous ont, s the rgd body transformaton whch conssts of the rotaton matrx R translaton vector t, K s the camera calbraton matrx (whch we assume s nown from ror calbraton) ro() s the D-D roecton functon: 1 ro( a ) : ( a1, a), a a R (9) he camera ose at a tme-ste s reresented as the rgd body transformaton. hs contrbuton has been eer-revewed. do: /srs-archves-xlii-1-w

3 he Internatonal Archves of the Photogrammetry, Remote Sensng Satal Informaton Scences, Volume XLII-1/W1, 017 ISPRS Hannover Worsho: HRIGI 17 CMR 17 ISA 17 EuroCOW 17, 6 9 June 017, Hannover, Germany. Pose Estmaton Model Gven a set of D onts x x whch are assocated wth D measurements z, to estmate the camera ose a tme-ste, we mnmze the followng energy functon usng LM algorthm: (10) 1 ( ) ˆ z z(, x ) 1 wth resect to the rgd body transformaton. We use the Huber cost functon as a robust ernel to guard aganst surous matches.. Pose Otmzaton wth resect to Le Grous he otmzaton methods resented n the revous secton are alcable for scalar felds whch are defned on Eucldean n vector saces. However, we want to mnmze the reroecton error wth resect to the rgd body transformaton, whch ncludes the rotaton ω ( 1,, ) n three dmensonal sace. ω can be any arameterzaton of rotaton n D (such as Euler angles or the rotaton vector). Performng a rotaton by then by ω s n general not equvalent to erformng a rotaton of ω+. Vector addton s smly not the rght oeraton to concatenate rotatons. hus, rotatons cannot be modelled as Eucldean vector sace, but as a Le grou...1 Le grou Le algebra A rgd body transformaton n can be exressed as an 4 4 matrx whch can be aled to homogeneous oston vectors (H. Strasdat., 010): R t 0 1 R, t R (11), wth SO() Here, SO() s the Le grou of rotaton matrces. he rgd body transformatons n form a smooth manfold therefore a Le grou, whch s called the Secal Eucldean Grou ( SE() ). he grou oerator s the matrx multlcaton. A mnmal reresentaton of ths transformaton s defned by the corresondng Le algebra se () whch s the tangent sace of SE() at the dentty. In, the algebra elements are 6-vectors ( ω, v ) : ω ( 1,, ) s the axs-angle reresentaton for rotaton, v s a rotated verson of the translaton t. Elements of the se () algebra can be maed to the SE() grou va the exonental mang ex SE() : ex SO() ( ) ex SE() (, ) : ω Vv R t ω v Here, sn( ) 1 cos( ) ex SO() ( ω) I ( ω) ( ω ). (1) Equaton (1) s the Rodrgues formula. (1) 1cos( ) sn( ) V= I ( ω) ( ) ω (14) Here, ω, () s an oerator whch mas a - ex vector to ts sew-symmetrc matrx. Snce SE() surectve, there s also an nverse mang log SE()... Pose-Pont ransformaton Jacoban Usng the chan rule, the artal dervatve of the resdual r z zˆ(, x) wth resect to s: r ( z zˆ (, x)) zˆ( y) ro(ex SE() ( ) x) yrxt y zˆ( y) y yrxt ro(ex SE() ( ) y) 0 0 yrxt zˆ( y) ro( q) ex SE() ( ) y yrxt qy y q y1 1 0 y f I ( y y y 0 1 y ) s 0 (15) We can get the udate vector from (7) n the tangent sace around dentty se () maed bac onto the manfold SE(), leadng to a modfed udate ste: 1 ex SE() ( ) (16) hus, we can use LM algorthm to solve the ose estmaton roblem. 4. EXPERIMENAL RESLULS In ths secton, we exermentally nvestgate the LM algorthm for camera ose otmzaton on the manfolds, comare the orgnal state-of-the-art solutons to PnP roblem wth ther corresondng otmzaton versons, ncludng the well-nown teratve aroach by Lu et al. (Lu, C. P., 000), denoted as LHM n short, the mult-stage method, ut forward by L et al. (L, S., 01), denoted as RPnP, OPnP method roosed by Zheng (Zheng, Y., 01). her otmzaton versons are denoted as LHM+LM, RPnP+LM OPnP+LM resectvely. Also the drect mnmzaton based method, DLS+++ (Hesch, J. A., 011), s ncluded. he source codes of LHM, RPnP, OPnP DLS++ are ublcly avalable on the nternet rovded n (Zheng, Y., 01). All the exerments are erformed n MALAB on a lato wth.4ghz CPU 8GB RAM. o acqure a quanttatve analyss, all the exerments are mlemented wth smulated data. We generate a vrtual ersectve camera, n D reference onts n the camera hs contrbuton has been eer-revewed. do: /srs-archves-xlii-1-w

4 he Internatonal Archves of the Photogrammetry, Remote Sensng Satal Informaton Scences, Volume XLII-1/W1, 017 ISPRS Hannover Worsho: HRIGI 17 CMR 17 ISA 17 EuroCOW 17, 6 9 June 017, Hannover, Germany framewor, whch are romly dstrbuted n a secfc range. All the smulated data arameter settngs are below n able 1. Parameters Settngs Focal length 800 xels Prncle ont (0 xels, 40 xels) Image soluton 640 xels 480 xels D ont x-range [-, ] D ont y-range [-, ] D ont z-range [4, 8] able 1. Parameter settngs for smulated data We rotate translate the D onts wth the ground-truth transformaton, ncludng the rotaton R translaton t. hen the absolute error n degrees between R the estmated rotaton R s measured. he rotaton error s defned as: err (deg ree) max acos( r, r ) 180 (17) where rot 1 r R resectvely. r are the -th column of R he translaton error s the relatve dfference between the estmated t, whch s defned as: err (%) ( t t t ) 100 (18) trans t Fgure. Medan rotaton error w.r.t. varyng number of onts 4.1 Varyng Number of Ponts wth Fxed Nose Level Frstly, we vary the number of onts n from 4 to 49, add zero-mean Gaussan nose wth fxed devaton ( xels) onto the roecton mages. At each n, 100 ndeendent tests are erformed. he average rotaton translaton error are resented n Fg. 1 to Fg. 4. Results show that all the solutons wth LM algorthm otmzed outerform ther orgnal versons, esecally the accuracy of RPnP s mroved effectvely, whch demonstrates the effectvty effcency of the roosed otmzaton strategy. We can fnd that the RPnP s not accurate enough, even n the resence of redundant corresondences, the maor reason les n ts underlyng aroxmaton schemes. Also, we can fnd that comared wth other methods, the LHM s not so accurate due to ts ossble local otmum. Moreover, wth the number of onts ncreasng, the accuracy of all solutons are all mroved effectvely. Fgure. Mean translaton error w.r.t. varyng number of onts Fgure 4. Medan translaton error w.r.t. varyng number of onts 4. Varyng Nose Levels wth Fxed Number of Ponts Fgure 1. Mean rotaton error w.r.t. varyng number of onts hen, we fx the number of onts n to be 10, add zeromean Gaussan nose wth varyng devaton levels (from 0.5 to 5 xels) onto the roecton mages. At each nose level, 100 ndeendent tests are erformed the average results are hs contrbuton has been eer-revewed. do: /srs-archves-xlii-1-w

5 he Internatonal Archves of the Photogrammetry, Remote Sensng Satal Informaton Scences, Volume XLII-1/W1, 017 ISPRS Hannover Worsho: HRIGI 17 CMR 17 ISA 17 EuroCOW 17, 6 9 June 017, Hannover, Germany reorted. he average rotaton translaton error are resented n Fg. 5 to Fg. 8. As shown n Fg. 5 to Fg. 8, the roosed otmzaton strategy s effcent effectve, as the accuracy of all solutons s mroved, esecally the RPnP method. Also, we can fnd that wth the nose ncreasng, the accuracy of all the method decreases. Fgure 8. Medan translaton error w.r.t. varyng nose levels 5. CONCLUSION Fgure 5. Mean rotaton error w.r.t. varyng nose levels We roose an aroach for relatve camera ose estmaton usng otmzaton method wth resect to the Le grou, whch can avod the sngularty of Euler angle arameterzaton of rotaton, mae the otmzaton method such as Gauss- Newton or Levenberg-Marquardt more robust convenent. Exermental results show that the roosed aroach outerform the orgnal method wthout otmzaton whch s caable of estmate relatve camera ose wth hgh accuracy, t can be wdely used n Photogrammetry. REFERENCES Garro, V., Croslla, F., & Fusello, A., 01. Solvng the n roblem wth ansotroc orthogonal rocrustes analyss. Proc. DIM/DPV, Hartley, Rchard, 00. Multle vew geometry n comuter vson. Cambrdge Unversty Press, nd edton. Fgure 6. Medan rotaton error w.r.t. varyng nose levels Hesch, J. A., & Roumelots, S. I., 011. A Drect Least- Squares (DLS) method for PnP. IEEE Internatonal Conference on Comuter Vson, ICCV 011, Barcelona, San, DBLP. H. Strasdat, J. M. M. Montel, A. J. Davson Scale drft-aware large scale monocular SLAM. n Robotcs: Scence Systems (RSS), Zaragoza, San. L, Xu, 011. A stable drect soluton of ersectve-three-ont roblem. Internatonal Journal of Pattern Recognton & Artfcal Intellgence, 5(5), L, S., Xu, C., & Xe, M, 01. A robust o(n) soluton to the ersectve-n-ont roblem. Pattern Analyss & Machne Intellgence IEEE ransactons on, 4(7), Fgure 7. Mean translaton error w.r.t. varyng nose levels Lu, C. P., Hager, G. D., & Molsness, E., 000. Fast globally convergent ose estmaton from vdeo mages. IEEE PAMI, (6), Leett, V., Moreno-Noguer, F., & Fua, P, 009. En: an accurate o(n) soluton to the n roblem. Internatonal Journal of Comuter Vson, 81(), hs contrbuton has been eer-revewed. do: /srs-archves-xlii-1-w

6 he Internatonal Archves of the Photogrammetry, Remote Sensng Satal Informaton Scences, Volume XLII-1/W1, 017 ISPRS Hannover Worsho: HRIGI 17 CMR 17 ISA 17 EuroCOW 17, 6 9 June 017, Hannover, Germany Moré, J. J he levenberg-marquardt algorthm: mlementaton theory. Lecture Notes n Mathematcs, 60, Rec, M. Q A fundamentally new vew of the ersectve three-ont ose roblem. Journal of Mathematcal Imagng Vson, 48(), Zheng, Y., Kuang, Y., Sugmoto, S., Åström, Kalle, & Outom, M, 01. Revstng the PnP Problem: A Fast, General Otmal Soluton. IEEE Internatonal Conference on Comuter Vson. IEEE, hs contrbuton has been eer-revewed. do: /srs-archves-xlii-1-w