Today: Calbraton What are the camera parameters? Where are the lght sources? What s the mappng from radance to pel color? Why Calbrate? Want to solve for D geometry Alternatve approach Solve for D shape wthout known cameras Structure from moton (unknown etrnscs) Self calbraton (unknown ntrnscs & etrnscs) Why bother pre-calbratng the camera? Smplfes the D reconstructon problem fewer parameters to solve for later on Improves accuracy Not too hard to do Elmnates certan ambgutes (scale of scene)
Applcatons D Modelng Match Move Images courtesy of Brett Allen ( Vson for Graphcs, wnter `) Image-Based Renderng Lght feld capture and renderng (Levoy & Hanrahan, 96) Camera Parameters So far we ve talked about: focal length prncpal (and nodal) pont radal dstorton CCD dmensons aperture There s also optcal center orentaton dgtzer parameters 2
Do we need all ths stuff? Usually smplfy to computable stuff Intrnscs: scale factor ( focal length ) aspect rato prncple pont radal dstorton Etrnscs optcal center camera orentaton How does ths relate to proecton matr? p MP Z Y X s sv su Proecton Models Orthographc l orthonorma and r r y z y t t M orthonormal and r r y z y t t f M Weak Perspectve M Affne [ ] t R M Perspectve M Proectve
4 The Proecton Matr Matr Proecton: MP p Z Y X s sv su M can be decomposed nto t R proect A c c v f u c fa t I R M proecton ntrnscs (A) orentaton poston Goal of Calbraton Learn mappng from D to 2D Can take dfferent forms: p MP Z Y X s sv su Proecton matr: Camera parameters: ),,,,, ( t R A f p Z Y X General mappng 2 R R
Calbraton: Basc Idea Place a known obect n the scene dentfy correspondence between mage and scene compute mappng from scene to mage Problem: must know geometry very accurately how to get ths nfo? Alternatve: Mult-plane calbraton Advantage Images courtesy Jean-Yves Bouguet, Intel Corp. Dsadvantages? Only requres a plane Don t have to know postons/orentatons Good code avalable onlne! Zhengyou Zhang s web ste: http://research.mcrosoft.com/~zhang/calb/ Intel s OpenCV lbrary: http://www.ntel.com/research/mrl/research/opencv/ Matlab verson by Jean-Yves Bouget: http://www.vson.caltech.edu/bouguet/calb_doc/nde.html 5
Alternatve: Mult-plane calbraton Images courtesy Jean-Yves Bouguet, Intel Corp. Need D -> 2D correspondence User provded (lots O clckng) User seeded (some clckng) Fully automatc? Chromaglyphs Courtesy of Bruce Culbertson, HP Labs http://www.hpl.hp.com/personal/bruce_culbertson/br98/chromagl.htm 6
Proector Calbraton A proector s the nverse of a camera has the same parameters, lght ust flows n reverse how to fgure out where the proector s? Basc dea. frst calbrate the camera wrt. proecton screen 2. now we can compute D coords of each proected pont. use standard camera calbraton routnes to fnd proector parameters snce we known D -> proector mappng Calbraton Approaches Possble approaches (not comprehensve!) Epermental desgn planar patterns non-planar grds Optmzaton technques drect lnear regresson non-lnear optmzaton Cues D -> 2D vanshng ponts specal camera motons» panorama sttchng» crcular camera movement Want accuracy ease of use usually a trade-off 7
Estmatng the Proecton Matr Place a known obect n the scene dentfy correspondence between mage and scene compute mappng from scene to mage Drect Lnear Calbraton 8
Drect Lnear Calbraton Can solve for m by lnear least squares What error functon are we mnmzng? Nonlnear estmaton Feature measurement equatons Mnmze mage-space error How to mnmze e(m)? Non-lnear regresson (least squares), Popular choce: Levenberg-Marquardt [Press 92] 9
Camera matr calbraton Advantages: very smple to formulate and solve can recover K [R t] from M usng RQ decomposton [Golub & VanLoan 96] Dsadvantages? doesn t model radal dstorton more unknowns than true degrees of freedom (sometmes) need a separate camera matr for each new vew Separate ntrnscs / etrnscs New feature measurement equatons features mages Use non-lnear mnmzaton e.g., Levenberg-Marquardt [Press 92] Standard technque n photogrammetry, computer vson, computer graphcs [Tsa 87] also estmates κ (freeware @ CMU) http://www.cs.cmu.edu/afs/cs/proect/cl/ftp/html/v-source.html [Zhang 99] estmates κ, κ 2, easer to use than Tsa code avalable from Zhang s web ste and n Intel s OpenCV http://research.mcrosoft.com/~zhang/calb/ http://www.ntel.com/research/mrl/research/opencv/ Matlab verson by Jean-Yves Bouget: http://www.vson.caltech.edu/bouguet/calb_doc/nde.html
Calbraton from (unknown) Planes What s the mage of a plane under perspectve? a homography ( proectve transformaton) preserves lnes, ncdence, concs H depends on camera parameters (A, R, t) where A fa c f uc v c [ r t] H A r 2 R [ r r2 r ] Gven homographes, can compute A, R, t Calbraton from Planes. Compute homography H for + planes Doesn t requre knowng D Does requre mappng between at least 4 ponts on plane and n mage (both epressed n 2D plane coordnates) 2. Solve for A, R, t from H, H 2, H plane f only f unknown 2 planes f (f,u c,v c ) unknown + planes for full K. Introduce radal dstorton model Solve for A, R, t, κ, κ 2 nonlnear optmzaton (usng Levenberg-Marquardt)