CAMERA CALIBRATION BY REGISTRATION STEREO RECONSTRUCTION TO 3D MODEL Klečka Jan Docoral Degree Programme (1), FEEC BUT E-mail: xkleck01@sud.feec.vubr.cz Supervised by: Horák Karel E-mail: horak@feec.vubr.cz Absrac: Paper aims a unusual way o camera calibraion. The main idea is ha by regisraion of uncalibraed sereo reconsrucion o 3D model of he same scene is eliminaed ambiguiy of he reconsrucion. The reason for his is ha exac meric scene reconsrucion from image pair can be undersae as informaion equivalen o calibraion of he source camera pair. Described principles were verified by experimen on real daa and resuls are presened a he end of he paper. Keywords: EEICT, camera calibraion, sereo reconsrucion, daa regisraion 1. INTRODUCTION Camera calibraion is a process used o find a se of parameers for a mahemaical model of camera which hen represens relaion beween real world coordinae sysem and coordinae sysem of calibraed camera. Sandard way o deal wih his problem consiss basically from hree pars: Firsly is necessary o have an objec wih poins which are easy-o-find in a camera picure and simulaneously hese poins have well-defined relaive real world posiion. Secondly picure of he objec is need o be aken, wih he examined camera, so consequenly correspondences beween he camera coordinaes sysem and he real world coordinaes sysem can be obain from i. And finally a se of camera model parameers which fi hese correspondences is found by solving se of equaions. The mehod described by his paper essenially differs from he sandard way. A firs insead of he a priory defined objec is used only a 3D model of some reasonably srucured scene. To his model is hen regisered a sereo reconsrucion of capured scene, so a leas wo picures is needed. And a he end is ransformaion obained by regisraion process used o acquire projecion marices of boh inpu picures. An experimen on real daa has been performed o verify correcness of proposed algorihm. The 3D model for he experimen consis from muliple scans of 2D laser scanner SICK LMS 111 wih measuremen plane oriened verically and roaed around verical axis in 65 range wih 0.5 resoluion. Picures have been capured by hand held camera. Figure 1: Inpu daa of he experimen 455
2. UNCALIBRATED STEREO RECONSTRUCTION Every picure aken by a camera is a projecion of in general 3D scene in o a plane, so i s obvious ha due o his mechanism is one dimension (usually called deph) los. Sereo reconsrucion is process of recovering he informaion abou los dimension from wo picures of he same scene, aken from slighly differen viewpoin. Principle of his mehod is based on premise ha every poin in image can be presen as a ray in he 3D space. So if projecion of a 3D poin can be deeced in wo differen images hen is space posiion is found as an inersecion of heir respecive rays. However for a proper deerminaion of posiion and orienaion of a ray in space from image coordinaes i is necessary o have calibraed cameras. Bu, as shown in [1], even wihou knowledge of calibraion, resrains of epipolar geomery allows o obain some reconsrucion, neverheless only up o projecive ransformaion ambiguiy. This process consis of hree seps: Firsly fundamenal marix is need o be compued. Then i is necessary o find as much correspondence poins as possible usually for his sep is preferred o use dispariy map. Finally fundamenal marix is used o figure ou pair of projecion marices wich can be used o proecively ambiguous reconsrucion of every poin correspondence hrough riangulaion. 2.1. FUNDAMENTAL MATRIX Fundamenal marix is he algebraic represenaion of he epipolar geomery [1]. If x and x ' represen posiions of corresponding poins in homogenous coordinaes hen fundamenal marix F for hese wo images will saisfy equaion for every correspondence: T x' Fx 0 (1) Imporan properies of his marix are: F is rank wo marix wih seven degrees of freedom is scale invarian and de( F ) 0, any poin x in firs image define on second image so-call epipolar line, on which correspondence o x can be found, as l ' Fx, all epipolar lines cross each oher in one poin called epipole e (or e ' in second image). Compuing of fundamenal marix can be done by several ways. On experimenal daa has been applied following mehod: Firsly correspondence poin has been searched using SIFT [3] feaure deecor and descripor. Then ouliers in found correspondence has been removed by RANSAC algorihm which periodically compue Fˆ using minimal seven poin algorihm. And a las final F has been compued by linear opimizaion algorihm using all inliers followed by zeroing minimal singular value of linear crierion opimal marix. 2.2. DISPARITY MAP A dispariy map can be presened as resul of very dense correspondence search, usually so dense ha dispariy map has he same resoluion as source images. If x and x ' again represen posiions of corresponding poins, hen relaed poin in dispariy map can be described as D( x) x' x. Figure 2: Recified images (lef and middle), dispariy map (righ) 456
As menioned before fundamenal marix consrains a correspondence for any x o be found on epipolar line l ' and o make dispariy map compuaion and represenaion more simple i is usual o recify inpu images so heir epipolar lines will become parallel o each oher and o one of he axis (generally o he x axis). Then daa in dispariy map can be scalar because i represens difference only in one dimension. To compue dispariy map experimen daa has been recified and hen algorihm described in [4] has been used. Recified images and he grayscale represenaion of resuling dispariy map is presened in he Fig 2. 2.3. TRIANGULATION Triangulaion in sereo reconsrucion can be presened as a process of searching space poin X from relaive orienaion of wo rays which are respecive o correspondence pair of image poins x and x '. x PX x ' P' X (2) Because rue projecion marices are unknown in his case, i s insead used any canonical pair which fis o fundamenal marix, namely: I 0 P' e' P Where e' x is skew symmeric marix which saisfy e' x a e' a T x F e' v e' (3), v is arbirary vecor and nonzero scalar. The experimen has been done wih v 0 and 1. When projecion marices are defined, here is several way how o solve riangulaion problem in his ask. The experimen has been done using linear homogenous mehod: Where i T p is row of P and x, y x. xp yp x' p' y' p' p p p' p' 2 T X 0 (4) 3. DATA REGISTRATION A purpose of daa regisraion is o ransform wo daa ses in such way ha hey will become spaially consisen. In described algorihm has been his concep used o regisraion he uncalibraed sereo reconsrucion o 3D model of reconsruced scene because of he projecive ambiguiy is removed from such reconsrucion. For realizaion of his regisraion has been used concep of ICP (Ieraive Closes Poin) algorihm described in [2]. However due o he fac ha he ICP is a numerical mehod a guess of he soluion has o be done a firs. The iniial guess of searched projecion ransformaion H 0 has been go hrough several handpick reconsrucion o 3D model correspondences. Transformaion is hen derived from hese correspondences using homogenous leas square mehod. Then he found ransformaion is gradually geing precision by periodical appliance of following algorihm: Firsly he closes poin in 3D model is find for every poin of sereo reconsrucion. Secondly some of he wors (he mos disan) closes poins correspondences are rejeced as ouliers. Then from remaining correspondences is randomly picked subse, because i urned ou ha full se of correspondences inliers is usually oo large for effecive processing. Finally ransfor- 457
maion sep H s is calculaed from correspondence subse and is added o so far found ransformaion H i HsHi 1 where i is number of ieraion. In described experimen six correspondences has been handpick for iniial guess calculaion. For ouliers removal 25% of mos disan correspondence has been rejeced. And 5000 samples has been randomly picked for ransformaion sep calculaion from inliers. Figure 3: Iniial guess of regisraion (lef) and regisraion afer fifeen ieraions (righ) 4. RESULTS Afer sereo reconsrucion has been regisered o 3D model (in oher words rue sereo reconsrucion represened by any poin X HX has been found), i is possible o use found projecion ransformaion H o acquiring rue projecion marix pair P P ' from canonical pair P,P' H imax by solving following pair of equaions: Similar equaion can be derive for x P X PX P X HX P '. PH Accuracy of rue projecion marix P compuaion is presened on fig. 4 by exurising original 3D model by projecing is poins ino he firs image. 1, (5) Figure 4: 3D model exured by projecion ino firs image For obaining inernal and exernal camera parameers have been rue projecion marices decompose using QR decomposiion ino a form: P KR where K is upper riangular marix, R orhogonal marix of roaion ransformaion and is a ranslaion vecor. However because sandard QR decomposiion decompose marix ino produc of orhogonal marix and upper riangular marix (in his order) some modificaion has o be done. A firs le s inroduce rearrangemen funcion which swaps marix elemens posiion by following rule:, j n j mi a, b where i, j a is i elemen 458
of m n sized marix A on posiion i, j and b is elemen of marix B rea. Secondly P has been spli in o square marix P KR formed by is firs hree columns and vecor p K formed by is las column. Then he decomposiion has been realized as: QR Example of experimenal daa decomposiion: 2592 P K 0 0 r KR K K re R r qr re P R re Q (6) 48 2555 0 1 1448 x 70 712 1395, R y 20, 343 1 z 172 166 p K 5. CONCLUSION The described mehod shows iself o be able of acquiring camera projecion marices of sereo pair images. Useful applicaion of his algorihm can be found in fusion 3D and 2D daa. The weakes poin is now in regisraion sereo reconsrucion o 3D model. Especially correspondence deecion for he firs ransformaion guess proven o be difficul problem o be auomaed. Then also robusness and effeciveness of he ieraive par of algorihm aren saisfyingly high enough. Improvemen of regisraion par will be subjec of furher research. ACKNOWLEDGEMENT The compleion of his paper was made possible by gran No. FEKT-S-14-2429 - "The research of new conrol mehods, measuremen procedures and inelligen insrumens in auomaion", and he relaed financial assisance was provided from he inernal science fund of Brno Universiy of Technology and also wih assisance Compeence Cener realized by TACR (reg. number TE01020197). REFERENCES [1] Richard Harley, and Andrew Zisserman. Muliple view geomery in compuer vision, 2nd ed. Cambridge: Cambridge Universiy Press, 2003. ISBN 05-215-4051-8. [2] Paul J. Besl, and Neil D. McKay. (1992). "A mehod for regisraion of 3-D shapes," IEEE Transacions on Paern Analysis and Machine Inelligence [online]. vol. 14, issue 2, pp. 239-256. Available from: hp://ieeexplore.ieee.org.ezproxy.lib.vubr.cz/xpl/aricledeails.jsp?arnumber=121791 DOI: 10.1109/34.121791. [3] David G. Lowe. (2004). "Disincive image feaures from scale-invarian keypoins," Inernaional Journal of Compuer Vision [online]. vol. 60, issue 2, pp. 91-110. Available from: hp://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf DOI: 10.1023/B:VISI.0000029664.99615.94 [4] Heiko Hirschmüller. (2008). "Sereo processing by semiglobal maching and muual informaion," IEEE Transacions on Paern Analysis and Machine Inelligence [online]. vol. 30, issue 2, pp. 328-341. Available from: hp://ieeexplore.ieee.org.ezproxy.lib.vubr.cz/xpl/aricledeails.jsp?arnumber=4359315 DOI: 10.1109/TPAMI.2007.1166 459