A Novel Camera Calibration Algorithm as Part of an HCI System: Experimental Procedure and Results

Transcription

1 A Novel Camera Calbraton Algorthm as Part of an HCI stem: Eermental Proedure Results K. auer, E. Yfants, M. Teruel, R. El-Khater Comuter Grahs Image Proessng Laborator hool of Comuter ene, Unverst of Nevada, Las Vegas Las Vegas, NV, UA 8954, UA ABTRACT Camera albraton s an ntal ste emloed n man omuter vson alatons for the estmaton of amera arameters. Along th mages of an arbtrar sene, these arameters allo for nferene of the sene s metr nformaton. Ths s a rmar reason for amera albraton s sgnfane to omuter vson. In ths aer, e resent a novel aroah to solvng the amera albraton roblem. The method as develoed as art of a Human Comuter Interaton (HCI) stem for the NAA Vrtual GloveBo (VGX) Proet. Our algorthm s based on the geometr roertes of ersetve roetons rovdes a losed form soluton for the amera arameters. Its aura s evaluated n the ontet of the NAA VGX, the results ndate that our algorthm aheves aura smlar to other albraton methods hh are haraterzed b greater omlet omutatonal ost. Beause of ts relablt de varet of otental alatons, e are onfdent that our albraton algorthm ll be of nterest to man. Keords: Camera Calbraton, Camera Modelng, Lens Dstorton, 3D Reonstruton. INTRODUCTION Camera albraton s a useful tool n the felds of omuter vson, omuter grahs, robot vson, et. In fat, the tasks ommon to these varous felds requre an understng of the 3D struture of a sene based on the nformaton etrated from a sequene of mages taken b to or more ameras. The roess of translatng the mages nto nformaton that an be used to reonstrut the sene observed nvolves several stes, one of the most mortant beng that of amera albraton. Calbratng the ameras s equvalent to ntalzng the sstem, gven that e obtan otmal values for the amera arameters. These arameters allo us to etrat metr nformaton from the sene thereb loate obets n sae usng the 3D struture reovered from the D mages. The albraton algorthm e ll resent n ths aer has been develoed as art of the researh undera at the Comuter Grahs Image Proessng Laborator at the Unverst of Nevada, Las Vegas. Ths researh roet s suorted b NAA, ts man goal s to rodue an effentl effetve NAA ae Grant/EPCoR: Develoment of a Natonall Comettve Program n Comuter Vson Tehnologes for Effetve Human-Comuter Interaton n Vrtual Envronments" alaton for Human- Comuter Interaton (HCI) that s, most mortantl, non-nvasve. Thus, e are develong a sstem n hh the ator s unhamered b an knd of eternal deve. efall, e ll determne the moton of the h solel from mages trakng the movement of the h here the subet ears, at most, a ar of ordnar gloves th olored markers at ke loatons. Ths ll allo us to gather the nformaton requred to aomlsh our task of resolvng the h s 3D oordnates, thereb estmatng ts ose moton. The struture of ths aer s as follos: eton ontans a bref desrton of our algorthm. The reader s ntrodued to our eermental tools roedures n eton 3. eton 4 rovdes the ratum eermental results. Fnall, e resent our onlusons n eton 5.. CAMERA CALIBRATION ALGORITHM We hose the Pnhole Camera Model for our roet. Ths model s based on the rnle of ollneart, here eah ont n the sae s roeted b a straght lne through the roeton enter nto the mage lane []. As mentoned above, amera albraton allos for the estmaton of all arameters neessar for obtanng the orresondene beteen mage oordnates (D) orld oordnates (3D) [3], [4], [5]. The arameters an be dvded nto to ategores: ) Intrns Parameters, hh rovde a lnk beteen the Pel Coordnate stem (PC) the Camera Coordnate stem (CC), ) Etrns Parameters, hh defne the loaton orentaton of the amera th reset to the World Coordnate stem (WC).. Estmaton of Internal Camera Parameters The nternal arameters to be estmated are the foal length f, the sale fators (n the dretons, resetvel), the radal dstorton oeffents k 0 k. In order to alulate the nternal arameters, e lae our albraton obet erendular to the amera, e set the orgn of the WC to be the ont roeted onto the enter of the mage, thus settng the rotaton matr R to a 80 degree - For more nformaton refer to htt://bovs.ar.nasa.gov/vslab/vg.htm 8 YTEMIC, CYBERNETIC AND INFORMATIC VOLUME 4 - NUMBER IN:

2 roll the translaton vetor t to a zero vetor of the form (0, 0, ) h. Gven that the rato beteen the length of a lne n the real orld the length of that lne n the mage remans onstant for all lnes (Fg. ), e an estmate the foal length as follos: refer to Fg. onsder the roeton of the to lnes 3 4, here the latter s rased to a knon heght z 3. ne uu 34 u3u4, e have that u - u f = () - h u4 - u3 f =. () - h z Also, sne t s knon that there s a onstant rato beteen f that both arameters have the same effet on the mage (salng), e an defne a Vrtual Camera n hh e set to one el er entmeter obtan a orresondng foal length f. We an thereb elmnate (), hh allos us to estmate f. from Eqs. () As e ust mentoned, f,, are determned b usng onl ontrol onts lng lose to the enter of the mage here the effets of radal dstorton are mnmal. Hoever, hen trng to arrve at a vable estmate of the dstorton oeffents, the ontrol onts to be onsdered must be unforml sattered throughout the entre mage. The nluson of lens dstorton s etremel mortant for the aura of our model sne lens dstorton has a sgnfant mat on the shae of the mage. The most ommonl onsdered form of dstorton s radal, hh s due to merfetons n the onstruton of the lens auses onts to be dslaed radall from the enter of the mage. It an be shon that the formulas modelng radal dstorton are gven b here δ δ 4 ( 0 + kr ) r k r r = 4 k0r ( + k r ) (3) []. Then, one e nlude Eq. (3) n our r = + model, e an solve for the dstorton oeffents k0 k va a Lnear Least quares (LL) Ft [8].. Estmaton of Eternal Camera Parameters In ths seton, e determne the orentaton of the amera th reset to the WC. Our goal s to estmate both the rotaton th reset to eah of the aes n the WC the total translaton. Intall assumng that the amera s onl rotated about the - as b an angle of θ, the rotaton matr s an -roll (Fg. ). Thus, from Fg. t follos that ED f - ED snθ = (4) a h λed = f - λed sn θ. (5) a h Thus, θ ma be eressed as Fg. : Fndng the foal length from to lnes at dfferent heghts To best estmate f,, gven that Eqs. () () do not model radal dstorton, e must onsder lnes that le entrel thn a regon nearb the enter of the mage sne ths area s least affeted b radal dstorton. θ = sn - h a ( λ -) ( λ + ). (6) Let us note that n man ameras, are not equal; as a result, the els have a retangular shae. The rato / s alled the aset rato ll be denoted b ρ [8]. IN: YTEMIC, CYBERNETIC AND INFORMATIC VOLUME 4 - NUMBER 9

3 z Fg. : Rotaton of the amera b θ degrees about the -as Refer to Fg., here T a n s the normal to the mage lane, n = (0, 0,) s the normal to the lane, π + θ s the angle beteen them. Therefore, f R ( ) s a rotaton matr about the -as, t s lear that C D B a b P θ θ E n n A θ D F nlude a rular attern, a grd attern, a resel measured bo (the NAA GloveBo). The softare tools onsst of a Regon eleton Tool (RT), a Dot Etraton Tool (DET), tools for rearrangng etrated onts. The rular attern (Fg. 3) s used to arr out the mlt albraton, as desrbed n eton.. It onssts of 80 evenl saed dots that form a rle of radus 0 n. Eah of the onsttuent dots s a rle of radus 0.04 n. The grd attern (Fg. 4) s the one used for fndng the nternal arameters of the amera (eton.) as ell as testng the fnal albraton. The grd onssts of 0.04 n.-radus dots that are dran n an 4 grd attern, th neghborng dots saed n. aart. The RT s neessar to selet the albraton attern n the mage to restrt the searh for dots to that regon. Ths tool orks b enlosng the regon of nterest thn a losed olgon, sefed b the user, so that the DET onl onsders onts that fall thn ths olgon. It s mortant to note that the olgon should enlose the entre albraton attern. n = R ( π + θ ) n. (7) Thus, e are able to fnd n for the ase n hh the amera s onl rotated about the -as. In general, hoever, the mage lane s not rotated solel about a sngle as. As a matter of fat, e ll usuall have a sequene of rotatons about the,, z aes. Hene, n order to fnd the angle beteen n n, e must frst dentf an as of rotaton n the lane. In order to fnd the as of rotaton, e rel on the fat that the mage of the as of rotaton remans unhanged as the amera s rotated. One the as s dentfed, e an use Eq. (6) to omute the angle of rotaton. To ths end, e onstruted a seal albraton obet that onssts of onts n a rular attern hh les entered at the orgn of the WC. We searh for an oosng ar of onts that are equdstant from the enter of the rle. ubsequentl, these onts are used to determne the as of rotaton, hh s n fat the lne onnetng them. Net, e use the lne erendular to the as th Eq. (6) to determne θ. One the angle of rotaton θ has been found, e an roeed th the onstruton of the orresondng rotaton matr, hh ll be denoted b R ( ). Ths allos us to fnd n u θ the oordnates of the enter of the mage lane, C. Hene, e an omute the equaton of the mage lane, hh enables us to onstrut the sought-after rotaton matr, = du dv n R [8]. 3. EXPERIMENTAL TOOL AND PROCEDURE 3. Eermental Tools The eermental tools develoed for ths roet fall nto to ategores: hsal tools softare tools. The hsal tools Fg. 3: Crular Calbraton Pattern One the regon has been seleted, the DET s used to auratel detet the el oordnates of the ontrol onts on the albraton atterns. To aomlsh ths, the mage s onverted to YIQ then segmented nto blak hte regons va a thresholdng. Fnall, the enter of mass of eah of the dot-reresentatve blak regons s omuted va a Deth Frst earh (DF) see. Fg. 4: Calbraton Grd Net, e utlze our ont orderng tools to rodue a meanngful arrangement of the ontrol onts. We have develoed to tools for ths urose: a tool for arrangng the onts from the rular attern nto lokse order, a tool for orderng the onts n the grd lne b lne. 0 YTEMIC, CYBERNETIC AND INFORMATIC VOLUME 4 - NUMBER IN:

4 ) The rle orderng tool orks b searatng the onts nto four regons n the manner shon n Fg. 5. Due to the fat that the mage s orgn s loated at ts to left orner, the onts n Regon are sorted b nreasng oordnate; n Regon the are sorted b dereasng ; n Regon 3 b dereasng, n Regon 4 b nreasng. Ths results n a lokse orderng of the onts. Fg. 5: Regons used for orderng the dots ) The tool for orderng the ontrol onts on the grd sorts the onts lne b lne from left to rght (Fg. 6) as follos: the four orners of the grd are the algorthm s nut. Let { bl, tl, br, tr } be the orners, here bl s the bottom left orner, tl s the to left orner, br s the bottom rght orner, tr s the to rght orner (see Fg. 6). The ntal ste s to fnd the head tal of ever lne. Ths s aomlshed b frst omutng the sloe s of the lne bl tl. For all remanng onts, e fnd the sloe of the lne equal to s, then usng br tr tl or bl ; f t s s onsdered a head of a lne. mlarl,, e an fnd all the tals of the lnes. One e have found heads tals, e sort eah set n nreasng order aordng to the -oordnate. The sortng s done n a manner smlar to that desrbed above. In fat, e onsder ever lne formed b onng the orresondng head tal. That s, e onsder ever lne l formed b heads[] tals[]. Let s be the sloe of ths lne. remanng onts Regon 3 tals[] equals s, then Regon 4 Regon Regon, f the sloe of the lne For all the heads[] or belongs to the lne l. Then, e sort all the onts n ever lne l n nreasng dstane from heads[]. The sequene of lnes l, l, l3..., l N gves us the desred orderng of the onts. Fg. 6: Arros to ndate sortng dreton 3. Eermental Proedure We begn b fndng the nternal amera arameters as desrbed n eton.. The amera s laed erendular to the grd as shon n Fg., an mage of the grd s atured. We etrat the D mage oordnates of ths grd usng the DET store them n the ordered set LP. Net, e rase the grd usng a resel measured heght fll the ordered set HP. The sets LP HP are rearranged usng the ont orderng tool desrbed n the revous seton then used th the soluton of Eqs. () () to omute the foal length f for the vrtual amera. In the net ste, e omute the radal dstorton oeffents b roetng all the onts n LP from the 3D WC to the D PC usng the values for f the heght of the amera h ust omuted. The atual mage oordnates of the onts are the ones that ere etrated from the mage usng the DET; these are used to fnd the dstorton at eah ont. Then, e ft a olnomal havng a form akn to that of Eq. (3) to the lot of dstortons usng LL, eldng the values of the radal dstorton oeffents k0 k. One e have obtaned the nternal amera arameters, e an roeed th the eternal albraton to determne the amera s rotaton translaton. To do so, e lae the amera at the desred loaton, then e oston the rular attern desrbed n eton 3. so that the enter of the rle aears at the enter of the mage. The dot etraton tool s used to etrat the IC oordnates of the dots store them n the ordered set CP; then, the rle orderng tool s used to order the onts n CP. We then roeed to determne the as of rotaton of the amera as follos: ever to onts on ooste sdes of the rle are ared together nto ars (, ), here s the ar number = (, r) = (, r ) are the mage oordnates of the to onts. For ever ar (, ), e omute the rato r =, here = (, r ) reresents the mage oordnates of the enter of the rle. The lne orresondng to the ar that gves r losest to one s taken to be the as of rotaton, hh e denote b a vetor v. We take the to rle onts that fall on the lne that s erendular to v v usng Eq. (6). omute the angle of rotaton θ about One e have θ, e an omute the IN: YTEMIC, CYBERNETIC AND INFORMATIC VOLUME 4 - NUMBER

5 rotaton matr of the amera, mentoned n eton. R 4. PRACTICUM = du dv n, as A set of knon ontrol onts as used to verf the orretness of the albraton. The set ontans 94 ontrol onts, ostoned ever n. n a b 4 grd attern. Frst, an mage of the ontrol onts s atured from eah albrated amera, the onts are roeted from the PC to WC usng the arameters omuted from the albraton roedure. The Euldean dstane beteen the omuted WC oordnates the atual knon orld oordnates s taken as the error measure. To tes of errors are onsdered, the albraton error the reonstruton error. For the albraton error, the onts are assumed to belong to a knon lane, sa the z = 0 lane, are roeted usng onl one amera. Before roetng the onts, e omensate for the radal dstorton, as mentoned n eton 3.. Net, the onts are roeted frst nto the CC then nto the WC usng the estmated eternal arameters. To omute the reonstruton error, mages from the to ameras are used n onunton to resolve all three oordnates smultaneousl va trangulaton. Table : Average Calbraton Errors ERROR (m) Tral No. X Y Z Magntude Cam Cam Cam Cam Cam Cam Cam Cam Cam Cam Cam Cam Cam Cam Cam * Cam * Cam * Cam * Cam * Cam * Average Errors * Tral arred out th the ameras loser together. Table : Average Reonstruton Errors ERROR (m) Tral No. X Y Z Magntude * * * Average Errors As shon b Tables, e obtan an error of about 3 mm n the 3D oordnate reonstruton. ome of the ossble soures of error n our albraton are measurement error, feature etraton, trangulaton error, other tes of lens dstortons. In fat, the feature etraton aura an be affeted b non-unform llumnaton, hh auses the deteted entrods of the ontrol onts to be slghtl dslaed from the atual enters. Regardng the lens dstorton, as mentoned n eton 3, our albraton model onl onsders radal lens dstorton. Hoever, onsderng other forms of dstorton suh as tangental thn rsm ma mrove the results of the albraton. Fnall, the trangulaton stage ontrbutes to the overall error due to the fat that the aura of trangulaton deends not onl on the baselne (dstane beteen the ameras) but also on the angle beteen the ras (Fg. 7). In fat, onts are less resel loalzed along the ra of roeton as the ras beome nreasngl arallel. In the NAA VGX, the setu requres that the ameras be lose to eah other; as a onsequene, the ras tend to be nearl arallel. Ths mats our error, hh an be seen n the reonstruton errors n Table. Fg. 7: The shaded regon reresents the reonstruton unertant regon, hh deends on the angle beteen the ras of roeton nreases as the ameras are laed loser together. YTEMIC, CYBERNETIC AND INFORMATIC VOLUME 4 - NUMBER IN:

6 5. CONCLUION We have resented a novel aroah to amera albraton based on the geometr roertes of the amera sstem. Our albraton algorthm has varous advantages dsadvantages n omarson to other albraton shemes. For nstane, t s omutatonall fast algorthmall straghtforard. In fat, our algorthm elds a losed form soluton thout relng on non-lnear otmzaton methods. Hoever, to searate roedures are needed for omutng the amera s ntrns etrns arameters. Yet, one the nternal amera arameters are omuted, t s ossble to reoston the amera thout reomutng them. As mentoned earler, the aura of our albraton method ould be further mroved b onsderng other forms of lens dstorton suh as tangental thn rsm dstorton. It ould also be ossble to use the arameters obtaned from our albraton method as an ntal guess to some of the non-lnear otmzaton methods used n the lterature [4], [5], [9]. Fnall, e ould lke to remark that the albraton method resented n ths aer as suessfull used to albrate the ameras n the NAA VGX. Usng the albrated ameras, e ere able to etrat the ostons dretons of the hs n the VGX along th the ostons dretons of eah ndvdual fnger th hgh aura [6]. Ths alloed us to reonstrut the orld oordnates of ke loatons on the user s h that ere fed nto our 3D h modelng module, eldng a rese realst vrtual modelng of the human h [7]. [9] J. Weng, P. Cohen M. Hernou, Camera Calbraton th Dstorton Models Aura Evaluaton, IEEE Transatons on Pattern Analss Mahne Intellgene, Vol. 4, No. 0, 99, REFERENCE [] O. Faugeras, Three-Dmensonal Comuter Vson: a Geometr Veont, Cambrdge, MA: MIT Press, 993. [] J. Weng, P. Cohen, M. Hernou, Camera Calbraton th Dstorton Models Aura Evaluaton, IEEE Trans. Pattern Analss Mahne Intellgene, Vol. 4, No. 0, 99, [3] M. B. Teruel, R.W. El-Khater, O. Kubushna E. A. Yfants, A Reve Paer on Camera Calbraton Tehnques, Tehnal Reort, NAA GloveBoX Proet, Comuter Grahs Image Proessng Laborator, hool of Comuter ene, Unverst of Nevada, Las Vegas, NV, 003. [4] O.D. Faugeras, G. Tosan, The Calbraton Problem for tereo, Pro. of IEEE Comuter Vson Pattern Reognton (CVPR), 986, [5] O.D. Faugeras, G. Tosan, Camera Calbraton for 3D Comuter Vson, Pro. Int'l Worksho Industral Alatons of Mahne Vson Mahne Intellgene, 987, [6] M.B. Teruel, O. Kubushna, R.W. El-Khater, E.A. Yfants, R. Bole, On Estmaton of 3D H Poston Usng tereo Vson," Pro. of the ICA 9 th Internatonal Conferene on Comuter ther Alatons, 004, [7] O. Kubushna, E.A. Yfants, L. E. Teruel, C. Hang, P. tubbedurd, G. Bebs, R. Bole, Interatve 3D H Model th Constrants, Pro. of the ICA th Int l Conferene on Intellgent Adatve stems oftare Engneerng, 003, [8] R. W. El-Khater, Camera Calbraton for the NAA Vrtual Glovebo Proet, M Thess, hool of Comuter ene, Unverst of Nevada, Las Vegas, et IN: YTEMIC, CYBERNETIC AND INFORMATIC VOLUME 4 - NUMBER 3