Calibration of an Articulated Camera System with Scale Factor Estimation

Transcription

1 Calbraton of an Artculated Camera System wth Scale Factor Estmaton CHEN Junzhou, Kn Hong WONG arxv:.47v [cs.cv] 7 Oct Abstract Multple Camera Systems (MCS) have been wdely used n many vson applcatons and attracted much attenton recently. There are two prncple types of MCS, one s the Rgd Multple Camera System (RMCS); the other s the Artculated Camera System (ACS). In a RMCS, the relatve poses (relatve -D poston and orentaton) between the cameras are nvarant. Whle, n an ACS, the cameras are artculated through movable jonts, the relatve pose between them may change. Therefore, through calbraton of an ACS we want to fnd not only the relatve poses between the cameras but also the postons of the jonts n the ACS. Although calbraton methods for RMCS have been extensvely developed durng the past decades, the studes of ACS calbraton are stll rare. In ths paper, we developed calbraton algorthms for the ACS usng a smple constrant: the jont s fxed relatve to the cameras connected wth t durng the transformatons of the ACS. When the transformatons of the cameras n an ACS can be estmated relatve to the same coordnate system, the postons of the jonts n the ACS can be calculated by solvng lnear equatons. However, n a non-overlappng vew ACS, only the ego-transformatons of the cameras and can be estmated. We proposed a two-steps method to deal wth ths problem. In both methods, the ACS s assumed to have performed general transformatons n a statc envronment. The effcency and robustness of the proposed methods are tested by smulaton and real experments. In the real experment, the ntrnsc and extrnsc parameters of the ACS are obtaned smultaneously by our calbraton procedure usng the same mage sequences, no extra data capturng step s requred. The correspondng trajectory s recovered and llustrated usng the calbraton results of the ACS. Snce the estmated translatons of dfferent cameras n an ACS may scaled by dfferent scale factors, a scale factor estmaton algorthm s also proposed. To our knowledge, we are the frst to study the calbraton of ACS. I. INTRODUCTION Calbraton of a Multple Camera System (MCS) s an essental step n many computer vson tasks such as SLAM (Smultaneous Localzaton and Map), survellance, stereo and metrology [4], [], [7], [9], [], [7]. Both the ntrnsc and extrnsc parameters of the MCS are requred to be estmated before the MCS can be used. The ntrnsc parameters [], [] descrbe the nternal camera geometrc and optcal characterstcs of each camera n the MCS. In a Rgd Multple Camera System (RMCS), the cameras are fxed to each other. The extrnsc parameters [5] of a RMCS descrbe the relatve pose (the relatve -D poston and orentaton, totally, sx degrees of freedom) between the cameras n the MCS. Calbraton methods of the ntrnsc parameters of a camera are well establshed [8], []. Calbraton methods for the extrnsc parameters of a RMCS are also wdely studed. For nstance, Maas proposed an automatc RMCS calbraton technque wth a movng reference bar whch can be seen by all cameras [5]. Antone and Teller developed an algorthm whch recovers the relatve poses of cameras by overlappng portons of the outdoor scene []. Baker and Alomonos presented RMCS calbraton methods usng calbraton objects such as a wand wth LEDs or a rgd board wth known patterns [], [4]. Dornaka proposed a stereo rg self-calbraton method by the monocular eppolar geometres and geometrc constrants of a movng RMCS, n whch only the feature correspondences between the monocular mages of each camera are requred [8]. In hand-eye calbraton, t s demonstrated that when a sensor s mounted on a movng robot hand, the relatonshp between the sensor coordnate system and hand coordnate system can be calculated by the moton nformaton of the hand and the sensor [9], [], [6]. One example of usng knematc nformaton of the cameras for RMCS s dscussed by Casp and Iran [6], they ndcated that f the cameras of a non-overlappng vew RMCS are close to each other and share a same projecton center, ther recorded mage sequences can be algned effectvely by the estmated transformatons nsde each mage sequence. However, n some types of MCS, the relatve poses between the cameras are not fxed, hence the calbraton methods for the RMCS cannot be used drectly. In Fgure, a novel applcaton of lmb pose estmaton by attachng cameras on the arms of a robot s shown. On each arm of the robot, two cameras are artculated to each other through the elbow jont of the arm. When the robot moves, the relatve pose between the cameras may change, whle, the coordnate of the elbow jont relatve to each camera attached on the correspondng arm s nvarant. In ths paper, such a type of MCS s named as Artculated Camera System (ACS). The jont of the elbow s named as the jont n the ACS. ACSs can be easly found n the real world, such as camera systems attached on human, robots and anmals. Before usng an ACS, t has to be calbrated. However, there are stll some unsolved problems: () In an ACS wth overlappng vew, tradtonal Correspondng Author J. Chen s wth the School of Informaton Scence & Technology Southwest Jaotong Unversty, Chna. E-mal address: jzchen@swjtu.edu.cn. K. H. Wong s wth the Department of Computer Scence and Engneerng, the Chnese Unversty of Hong Kong, Shatn, NT, Hong Kong. E-mal address: khwong@cse.cuhk.edu.hk. Ths work s supported by the Natonal Natural Scence Foundaton of Chna (No.64).

2 Fg. A ROBOT WITH FOUR CAMERAS ATTACHED ON IT, WHERE THE CAMERAS ARE ARTICULATED. calbraton methods cannot estmate the postons of the jonts n the ACS. () In a non-overlappng vew ACS, nether the postons of the jonts n the ACS nor the relatve poses between the cameras n the ACS can be estmated by tradtonal calbraton methods. These consderatons n mnd motvate us to develop the technologes n ths paper. The rest of ths paper are organzed as follows: Secton II and III analyss the constrants n a movng ACS. The correspondng calbraton methods are proposed. Secton V and VI evaluate the proposed method by smulaton and real experment. In secton VII, a bref concluson and the future plan are presented. II. CALIBRATION OF ACS WITH OVERLAPPING VIEWS Fg. AN ARTICULATED CAMERA SYSTEM WITH OVERLAPPING VIEWS Suppose two rgd objects are artculated at jont O and two cameras (camera A and B) are fxed on the two rgd objects respectvely (See Fgure ). Let C A be the coordnate system of camera A, C B the coordnate system of camera B. Suppose there are enough feature correspondences between the cameras so that the pose of C A and C B referrng to the same coordnate system C W can be estmated. Therefore, the relatve pose between C A and C B s known. We want to fnd the poston of O n the ACS. Let H AW and H BW be the Eucldean transformaton matrxes descrbe the C A and C B relatve to C W, so that for any pont P : [ ][ ] RAW T P A = H AW P W = AW P W () [ ][ ] RBW T P B = H BW P W = BW PW (), where R s the rotaton matrx, T s a vector, P W, P A and P B are the homogenous coordnates of the -D Pont P relatve to C W, C A and C B respectvely, P s a vector.

3 Accordng to equatons () and (): [ R T AW R T AW T AW P W = H AW P A = H BW P B () H AW P A H BW P B = (4) ][ ] [ PA R T BW R T BW T ][ ] BW PB = (5) R T AW P A R T BW P B = R T AWT AW R T BWT BW (6), where R T s the transpose of R. Suppose the ACS performed n transformatons. Let H AW and H BW be the Eucldean transformaton matrxes descrbe the C A and C B relatve to C W after the -th transformaton of the ACS. Accordng to equaton (6): (R AW )T PA (R BW )T PB = (R AW )T TAW (R BW )T TBW (7) Let Õ = [ Ō T A Ō T B ] T, where Ō A and ŌB are the coordnates of the jont O relatve to C A and C B respectvely. Equaton (7) can be rewrtten as: [ (R AW )T (R BW )T ] Õ = (R AW) T T AW (R BW) T T BW (8) Snce camera A and B are fxed on the artculated rgd objects, Õ s nvarant durng the transformaton of the ACS. The transformatons (R AW, R BW, T AW and T BW for [...n]) of the camera coordnate systems are calculated by the projected mage sequences. We propose that Õ can be estmated by a least squares method, when the ACS has moved to many dfferent postons and captured enough samples of R AW, R BW, T AW and T BW. The above dervaton shows that although the locaton of the jont OW n world coordnates s not constant, t equals (HAW ) O A or (HBW ) O B because the cameras can not move completely ndependent as they are connected wth a jont. The jont locaton can be calculated by the D subspace ntersecton of the camera transformaton matrces. III. CALIBRATION OF NON-OVERLAPPING VIEW ACS Fg. A NON-OVERLAPPING VIEW ARTICULATED CAMERA SYSTEM In many stuatons, there s no overlappng vew between the cameras n an ACS. And the lack of common features makes the calbraton method proposed n secton II become nvald (See Fgure ). Moreover, snce the relatve pose between the cameras n the ACS cannot be estmated by the overlappng vews, the calbraton of the relatve poses between the non-overlappng vew cameras s also requred. In ths secton, a calbraton method based on the ego-moton nformaton of the cameras n an ACS s dscussed. A. Recoverng the Poston of the Jont Relatve to the Cameras n the ACS Let CA nt and CB nt be the coordnate systems of camera A and B respectvely at the ntal state (tme t = ). Suppose the ACS performs n transformatons. Snce the coordnate of the jont O relatve to camera A s fxed durng the transformaton of the ACS. At tme t =, we have: O A = H AO A = [ R A T A ] O A (9)

4 4, where H A s the Eucldean transformaton matrx of camera A at tme relatve to Cnt A. R A and T A descrbe the orentaton and orgn of camera A at tme relatve to CA nt. Also O A s the coordnate of pont O at ntal state relatve to CA nt, and OA s the coordnate of pont O at tme relatve to Cnt A. If the poston of the jont O relatve to CA nt s fxed durng the transformatons of the ACS, we have: OA = O A, [,...,n]. For -th transformaton of the ACS, accordng to equaton (9): [ O A = H R A O A = A TA ] O A () (R A I)ŌA = T A () Let M A = [(R A I)T,(R A I)T,...,(R n A I)T ] T, TA = [(T A )T,(T A )T,...,(T n A )T ] T, we have: M A Ō A = T A () Snce the transformatons (R A and T A, [...n]) of camera A can be calculated by the projected mage sequence. We propose ŌA can be estmated by a least squares method. Smlarly, Ō B can also be estmated. Therefore, O A and O B are recovered. B. The Unqueness of the Jont Pose Estmaton If the dfferent segments of the artculated camera system (ACS) are connected by D rotatonal jonts (connected by pont rotatonal jonts) and the ACS can perform general transformatons, the soluton of the jont pose estmaton s unque: For the jont pose estmaton method usng specal moton (n secton III-A). Suppose the soluton of the jont pose estmaton s not unque, there must exst at least two dfferent D ponts Ō and Ō satsfy equaton (). We have: M A Ō = T A and M A Ō = T A. Therefore, any pont P = sō+( s)ō wll also satsfy equaton (), where s s an arbtrary scalar. Accordng to the defnton of P, P s the pont on the lne passng through the ponts Ō and Ō. Snce P satsfy equaton () represents that the poston of the pont P relatve to the camera n the ACS s nvarant durng the transformaton of the ACS, t means the dfferent segments of ACS are connected by the D rotatonal axs nstead of the D rotatonal jonts. The poston of the ponts on the D rotatonal axs relatve to the camera n the ACS s nvarant durng the transformaton of the ACS. However, t conflcts wth the assumpton. Smlarly, the unqueness of the jont pose estmaton method usng overlappng vews (n secton II) can also be verfed. C. Recoverng the Relatve Pose Between the Cameras of the Non-overlappng vew ACS Let H BA be the Eucldean transformaton matrx between C nt A P B = H BA P A = [ RBA T BA and C nt B, where P A and P B are the homogenous coordnate of Pont P relatve to C nt A, so that for any pont P : ] P A = H BA P A () and C nt B respectvely. The relatve pose ( R BA and T BA ) between C nt A and CB nt s defned as: R BA = R T BA (4) Let OB nvarant: T BA = R T BA T BA (5) be the coordnate of jont O at tme relatve to Cnt B. Snce the coordnate of the jont O relatve to camera B s Accordng to equatons (9) and (): O B = = = [ ] R B TB O B [ ][ ] R B TB RBA T BA O A [ R B R BA R B T ] BA +TB O A (6) O B = H BA O [ A ][ ] RBA T = BA R A TA O A [ RBA R = A R BATA +T ] BA O A (7)

5 5 Accordng to equatons (6) and (7): [ R B R BA R B T BA +TB ][ ŌA [ R B R BA Ō A +R B T BA +T B ] [ RBA R = A R BATA +T BA ] [ RBA R = AŌA +R BA TA +T BA ][ ŌA ] ] (8) (9) R BR BA Ō A +R BT BA R BA R AŌA R BA T A +T B T BA = () Snce ŌA can be estmated by the method dscussed n secton III-C, the R BA and T BA can be estmated by a least square method, when the ACS perform enough general motons. In our smulaton and real experment, the estmated R BA s refned by a method dscussed n []. Then the roll, ptch and yaw correspondng to the R BA are estmated accordng to the defnton of the rotaton matrx []. Let R BA = M(r,p,y), wherer p andy are the correspondng roll, ptch and yaw ofr BA,M s a functon from roll, ptch and yaw to the correspondng rotaton matrx. Then, the r, p, y, T BA and ŌA are optmzed by mnmzng the nonlnear error functon: n E(r,p,y,T BA,O A ) = (R BM(r,p,y)ŌA +R BT BA = M(r,p,y)R AŌA M(r,p,y)T A +T B T BA ) () usng a Levenberg-Marquardt method. Fnally, the R BA s recovered from the optmzed r, p and y. The relatve pose between the CA nt and CB nt s calculated by equatons (4) and (5). IV. DEALING WITH UNKNOWN SCALE FACTORS The non-overlappng vew ACS calbraton method dscussed above depends on the ego-moton nformaton of the cameras n the ACS. However, f the model of the scene s unknown, the estmated ego-translatons of the cameras may be scaled by dfferent unknown scale factors. These unknown scale factors must be consdered n the extrnsc calbraton process. A. Model Analyss Let T A and T B be the true ego-translaton of camera A and B n the world coordnate system, ˆTA and ˆT B be the estmated ego-translatons of camera A and B found by an SFM method, µ A and µ B be the correspondng unknown scale factors. So that: ˆT A = µ A T A () ˆT B = µ B T B () Let ˆŌ A be the pose of the jont relatve to C A calculated wth the estmated moton. Equaton () can be rewrtten as: Compare equaton (5) wth equaton (), we have: (R A I)ˆŌ A = ˆT A = µ A T A (4) ˆŌ (R A A I) = TA (5) µ A ˆŌ A = µ A Ō A (6) Let ˆR BA and ˆT BA be the extrnsc parameters calculated usng the estmated motons and jont pose. Equaton () can be rewrtten as: Accordng to equaton (), (), (6) and (7): R B ˆR BAˆŌ A +R B ˆT BA ˆR BA R AˆŌ A ˆR BAˆT A + ˆT B ˆT BA = (7) Let: R B ˆR BA µ A Ō A +R B ˆT BA ˆR BA R A µ AŌA ˆR BA µ A T A +µ BT B ˆT BA = (8) R µ A B ˆRBA Ō A +R B µ B ˆTBA µ A ˆRBA R AŌA µ A ˆRBA TA +TB ˆTBA = (9) µ B µ B µ B µ B µ A µ B ˆRBA = R BA ()

6 6 Equaton (9) can be rewrtten as: µ B ˆT BA = T BA () R B R BA Ō A +R B T BA R BA R AŌA R BA T A +T B T BA = () Snce the equatons () and () are exactly the same, we have: Therefore: R BA = R BA () T BA = T BA (4) ˆR BA = µ B R BA = φ BA R BA µ A (5) ˆT BA = µ B TBA = µ B T BA (6) Where φ BA = µb µ A. Equatons (5) and (6) show that the estmated rotaton matrx ˆR BA wll be scaled by the relatve scale factor (the rato of the scale factors of the cameras) and the estmated relatve translaton wll be scaled by the same scale factor of camera B. In the next secton, we wll dscuss the estmaton of the relatve scale factor. B. Rotaton Matrx and Relatve Scale Factor Estmaton Let R = φr+n, where R s a rotaton matrx and R T R = I, φ s an unknown scale factor, N s a unknown nose matrx. We want to recover R and φ from R. Accordng to the defnton, we have: R = φr+n = φ(r+ N ) = φm (7) φ Where M = R+ N φ. Let the sngular value decomposton of M be UDV T, where D = dag(σ,σ,σ ). As llustrated n appendx C of [], r can be approxmated by: Now, let the sngular value decomposton of r be Ũ DṼ T, snce R = φm, we have: Combne equatons (8), (9) and (4), the rotaton matrx r can be recovered by: R = UV T (8) Ũ = U (9) Ṽ = V (4) D = φd (4) R = ŨṼ T (4) When nose N s not sgnfcant, D I, the scale factor φ can be estmated by the followng approxmaton: trace( D) = trace(φd) trace(φi ) φ (4) φ trace( D) (44) In short, f we have enough samples of R A, ˆT A, R B and ˆT B we can fnd ÔA, ˆRBA and ˆT BA (see secton IV-A). Then usng the above formulas, n partcular, equaton (4) and (44), we can also fnd the real rotaton (R BA ) and the relatve scale factor φ BA. Let R BA = M(r,p,y), where r, p and y are the correspondng roll, ptch and yaw of R BA, M s a functon from roll, ptch and yaw to the correspondng rotaton matrx. In our smulaton and real experment, the estmated r, p, y, ˆTBA and φ BA can be optmzed by mnmzng the nonlnear error functon: n E(r,p,y,T BA,Q A ) = (φ BA RBM(r,p,y)ˆŌ A +RB ˆT BA φ BA M(r,p,y)RAˆŌ A φ BA M(r,p,y)TA + ˆT B ˆT BA ) (45) = usng a Levenberg-Marquardt method. If the pose of the jont s calbrated wth known scale factor (O A s known), the scale factor µ A can be estmated by equaton (6). The scale factor µ B can be calculated by µa φ BA. Fnally, the R BA s recovered from the optmzed r, p and y. The relatve pose between the CA nt and CB nt s calculated by equatons (4) and (5). Therefore, a non-overlappng vew ACS can also be calbrated usng scaled moton nformaton from each camera n t.

7 7 V. SIMULATION In ths secton, the proposed calbraton methods are evaluated wth synthetc transformaton data. A. Performance w.r.t. Nose n Transformaton Data Setup and Notatons: In each test, one ACS wth cameras and jont s generated randomly. In whch, O A meters, O B meters. The generated ACS performs random transformatons. Performance of the Calbraton Method for ACS wth Overlappng Vews: In the frst smulaton, the proposed algorthm s tested tmes. Zero mean Gaussan nose s added to the transformaton data of the cameras. The confguraton, nput and output of our smulaton system are lst as Table I. Snce we assume there are overlappng vews between the two cameras, the relatve pose between them can be estmated by many exstng methods as dscussed n secton I. Only the performance of jont pose estmaton s evaluated n our smulaton. The error of jont estmaton are computed by: Err = ŌA ˆŌ A ŌA + ŌB ˆŌ B ŌB (46), where ŌA s the ground truth, ˆŌ A s the estmated poston of jont O relatve to camera A. Smlarly, Ō B s the ground truth, ˆŌ B s the estmated poston of jont O relatve to camera B. The correspondng results are shown n Fgure 4. TABLE I CONFIGURATION, INPUT AND OUTPUT Confguraton No. of Cameras n the ACS No. of Jonts n the ACS Random transformatons per test (n) Number of tests Input ( =...n) Rotatons of cameras (R AW, R BW ) Translatons of cameras (T AW, T BW ) Zero Mean Gaussan nose: σ rot.4 and σ trans.meters Output Mean error of jont pose estmaton (see equaton (46)) STD error of jont pose estmaton (see equaton (46)) Mean Error of Jont Pose Estmaton STD Error of Jont Pose Estmaton Mean Error of Jont Pose Estmaton STD Error of Jont Pose Estmaton Fg. 4 MEAN AND STD ERROR OF JOINT POSE (O A ) ESTIMATION. (A) MEAN ERROR OF JOINT POSE ESTIMATION; (B) STD ERROR OF JOINT POSE. Performance of the Calbraton Method for Non-Overlappng Vews ACS: In the second smulaton, frstly, the pose of the jont s fxed relatve to CA nt durng the transformatons of the ACS. The pose of the jont relatve to the camera A (O A ) s calbrated by the transformatons of camera A. Smlarly, O B s calbrated. Then, the ACS performs several general transformatons (the jont s not needed to be fxed relatve to CA nt ), the relatve pose between the cameras are calbrated usng the estmated jont pose and the transformatons of the cameras. The confguraton, nput and output of the smulaton system are lsted as Table II. The error of jont pose, relatve rotaton, relatve translaton estmaton are calculated by equaton (46), (47) and (48) respectvely.

8 8 Fgure 5 shows the results of jont pose estmaton. Compare wth the calbraton method usng the overlappng vews, the calbraton method usng specal motons s more accurate. The mean and STD error of the relatve rotaton and translaton estmaton are presented n Fgure 6 and 7. The proposed algorthms are shown to be stable, when the zero mean Gaussan nose from to.4 s added to the roll, ptch and yaw of the rotaton data, and the zero mean Gaussan nose from to. meters s added to the translaton data. Err rot = roll roll + ptch ptch + yaw ŷaw (47) Err trans = T AB ˆT AB T AB (48) TABLE II CONFIGURATION, INPUT AND OUTPUT Confguraton No. of Cameras n the ACS No. of Jonts n the ACS Random transformatons per test (n) Number of tests Input ( =...n) Transformatons wth fxed jont pose: Rotatons of cameras (R A, R B ) Translatons of cameras (T A, T B ) General transformatons: Rotatons of cameras (R A, R B ) Translatons of cameras (T A, T B ) Zero Mean Gaussan nose: σ rot.4 and σ trans.meters Output Mean error of jont pose estmaton (see equaton (46)) STD error of jont pose estmaton (see equaton (46)) Mean error of relatve translaton estmaton (see equaton (48)) STD error of relatve translaton estmaton (see equaton (48)) Mean error of relatve rotaton estmaton (see equaton (47)) STD error of relatve rotaton estmaton (see equaton (47)) Mean Error of Jont Pose Estmaton STD Error of Jont Pose Estmaton x Mean Error of Jont Pose Estmaton STD Error of Jont Pose Estmaton Fg. 5 MEAN AND STD ERROR OF JOINT POSE (ÔA) ESTIMATION. (A) MEAN ERROR OF JOINT POSE ESTIMATION; (B) STD ERROR OF JOINT POSE ESTIMATION. Performance of the Calbraton Method for Non-Overlappng Vews ACS wth Unknown Scale Factors: The scale factors of the two cameras n each test are assumed to be unform dstrbuted n the range [.5,5]. Therefore, the relatve scale factor between the two cameras satsfes the unform dstrbuton n the range of [.,]. The jont pose of the ACS s generate randomly and estmated by the method descrbed n secton III-A. Other confguratons are the same as the second smulaton. The ÔA, R BA, ˆTBA and φ BA are estmated and optmzed as dscussed n secton IV. The error of jont pose, relatve rotaton, relatve translaton estmaton are calculated by equaton (46), (47) and (48) respectvely. The error of relatve

9 9 Mean Error of Relatve Rotaton Estmaton STD Error of Relatve Rotaton Estmaton Mean Error of Relatve Rotaton Estmaton (degree) STD Error of Relatve Rotaton Estmaton (degree) Fg. 6 MEAN AND STD ERROR OF RELATIVE ROTATION (R BA ) ESTIMATION. (A) MEAN ERROR OF RELATIVE ROTATION ESTIMATION; (B) STD ERROR OF RELATIVE ROTATION ESTIMATION. Mean Error of Relatve Translaton Estmaton STD Error of Relatve Translaton Estmaton Mean Error of Relatve Translaton Estmaton STD Error of Relatve Translaton Estmaton Fg. 7 MEAN AND STD ERROR OF RELATIVE TRANSLATION (ˆT BA ) ESTIMATION. (A) MEAN ERROR OF RELATIVE TRANSLATION ESTIMATION; (B) STD ERROR OF RELATIVE TRANSLATION ESTIMATION. scale factor estmaton s evaluated by ε φ = φ ˆφ φ. Where ˆφ s the estmated relatve scale factor, and φ s the ground truth. Fgure 9 and show the results of the relatve pose estmaton. Compared to fgure 6 and 7 the accuraces are smlar. Fgure shows the performance of the relatve scale factor estmaton. The accuracy of the relatve scale factor estmaton [( ε φ ) %] s no less than 98.5%, when the standard dervaton of the nose n ego-rotaton s less than and the standard dervaton of the nose n ego-translaton s less than. meters. VI. REAL EXPERIMENT In the real experments, an ACS wth two cameras (Cannon PowerShot G9) s set up as Fgure. The ntrnsc parameters of each camera n the ACS are calbrated by Bouguet s mplementaton ( Camera Calbraton Toolbox for Matlab ) of []. Snce the Bouguet s Toolbox can also estmate the pose nformaton of the camera, the transformatons of each camera are calculated usng the same mage sequence for the ntrnsc calbraton smultaneously. No addtonal mages nor manual nput s requred n the real experments. A. Calbraton of the Pose of the Jont n Each Camera By Overlappng Vews (Algorthm I): In the frst real experment, the two cameras n the ACS observe the same checker plane and record mages smultaneously. The two cameras are free to move durng the transformaton of the ACS. Two mage

10 Mean Error of Node Pose Estmaton STD Error of Node Pose Estmaton Mean Error of Node Pose Estmaton STD Error of Node Pose Estmaton Fg. 8 MEAN AND STD ERROR OF JOINT POSE WITH UNKNOWN SCALE FACTOR (ÔA). (A) MEAN ERROR OF JOINT POSE; (B) STD ERROR OF JOINT POSE. Mean Error of Relatve Rotaton Estmaton STD Error of Relatve Rotaton Estmaton Mean Error of Relatve Rotaton Estmaton (degree) STD Error of Relatve Rotaton Estmaton (degree) Fg. 9 MEAN AND STD ERROR OF RELATIVE ROTATION WITH UNKNOWN SCALE FACTOR(R BA ). (A) MEAN ERROR OF RELATIVE ROTATION; (B) STD ERROR OF RELATIVE ROTATION. sequences (Q and Q ) are recorded, each sequence conssts of 5 mages of sze 6 pxels. The estmated jont pose are lst n Table III as algorthm I. By Fxed-Jont Motons (Algorthm II): In the second real experment, the jont of the ACS s fxed relatve to the world coordnate system durng the transformaton of the ACS. The two cameras do not need to vew the same checker plane. And each camera records the mage sequence ndependently. Two mage sequences (Q and Q 4 ) are recorded, each sequence conssts of mages of sze 6 pxels. The camera pose of the frst mage s selected as the ntal pose to generate the transformaton sequence of each camera. The estmated jont pose are lst n Table III as algorthm II. The poses of the jont relatve to the two cameras n the ACS are also estmated manually for comparson purpose. Snce the camera pose of any mage n each mage sequence can be chosen as the ntal camera pose (see secton III-A), the proposed algorthm s also tested by choosng dfferent mages as the reference. The mean and standard dervaton of the correspondng calbraton results are presented n Table IV. B. Calbraton of Relatve Pose Between the Cameras n the Non-Overlappng Vew ACS (Algorthm III) In the thrd real experment, frstly, we use the non-overlappng vew ACS calbraton method to process the mage sequences Q and Q. The jont pose (ŌA) estmated by algorthm II s used as the nput for the relatve pose calbraton. Snce there are overlappng vews between Q and Q, we also calbrate the relatve pose between the two cameras by the feature correspondences for comparson. The calbraton result are lsted n Table V. After the jont pose relatve to each camera n the ACS and relatve pose between the cameras n the ACS are calbrated, the trajectory of the ACS s recovered (see Fgure ). The proposed calbraton method s also tested by non-overlappng vew mage sequences. Fgure, (c), (d) shows the confguraton of the non-overlappng vew ACS calbraton system n the real experment. Two mage sequences (Q 5 and

11 Mean Error of Relatve Translaton Estmaton STD Error of Relatve Translaton Estmaton Mean Error of Relatve Translaton Estmaton STD Error of Relatve Translaton Estmaton Fg. MEAN AND STD ERROR OF RELATIVE TRANSLATION WITH UNKNOWN SCALE FACTOR (ˆT BA ). (A) MEAN ERROR OF RELATIVE TRANSLATION; (B) STD ERROR OF RELATIVE TRANSLATION. Mean Error of Relatve Scale Factor Estmaton STD Error of Relatve Scale Factor Estmaton Mean Error of Relatve Scale Factor Estmaton STD Error of Relatve Scale Factor Estmaton Fg. MEAN AND STD ERROR OF RELATIVE SCALE FACTOR WITH UNKNOWN SCALE FACTOR (φ BA ). (A) MEAN ERROR OF RELATIVE SCALE FACTOR; (B) STD ERROR OF RELATIVE SCALE FACTOR. Q 6 ) are recorded, each sequence conssts of 7 mages of sze 6 pxels. There s no overlappng vew between Q 5 and Q 6. Fgure 4 shows some samples of the recorded mages. We also manually measured the relatve pose between the two cameras for comparson. Snce no feature correspondence can be used, we only get a rough estmaton by a ruler. The calbraton results are shown n Table VI. After the relatve pose between the cameras at the ntal state s estmated, the trajectory of the non-overlappng vew ACS s recovered (see Fgure 5). C. Calbraton of Relatve Pose Between the Cameras n the Non-Overlappng Vew ACS wth Unknown Scale Factors (Algorthm IV) The scale factor estmaton algorthm s evaluated n the fourth real experment. The estmated translatons from Q and Q are multpled by.8 and. respectvely. In ths case, f no nose exsts, the estmated relatve scale factor (φ BA ) should be 4. The estmated relatve scale factor (ˆφ BA ) n our experment was.899. Table VII lsts the correspondng results, n whch the estmated relatve translatons are dvded by., so that they can be easly compared wth the estmated relatve translatons n Table V. The experment showed that our algorthms can estmate the relatve scale factor and fnd the extrnsc parameters correctly. In order to test the stablty of the scale factor estmaton algorthm, the estmated translatons from Q 5 and Q 6 are multpled by.8 and. respectvely. tests are performed. In each test, mages are randomly selected as secton VI-B. The Mean and STD of the calbraton results s lsted n Table VIII. The results are good. VII. CONCLUSION In ths paper, an ACS calbraton method s developed. Both the smulaton and real experment show that the pose of the jont n an ACS can be estmated robustly. When there s no overlappng vew between the cameras n an ACS, the jont pose

12 TABLE III RESULTS OF JOINT POSE CALIBRATION I: the algorthm usng overlappng vews. (see secton VI-A) II: the algorthm usng fxed-jont motons. (see secton VI-A) M: manual measurement(ground truth). O A s the coordnate of the jont relatve to camera A, the same apples to O B. Algorthm Jont Pose (mm) X Y Z I O A O B II O A O B M O A ± 5± -4± O B -7± 5± -± TABLE IV MEAN AND STD OF THE JOINT POSE CALIBRATION ALGORITHM II USING DIFFERENT REFERENCE IMAGES O A s the coordnate of the jont relatve to camera A, the same apples to O B. Algorthm Jont Pose (mm) II X Y Z Mean O A O B STD O A O B and the relatve pose between the cameras can also be calculated. The trajectory of an ACS can be recovered after the ACS s calbrated. The proposed calbraton method requres only the mage sequences recorded by the cameras n the ACS. A scale factor estmaton algorthm s proposed to deal wth unknown scale factors n the estmated translaton nformaton of the cameras n an ACS. In the real experment, the ntrnsc and extrnsc parameters of the ACS are calbrated usng the same mage sequences smultaneously. Snce we stll cannot fnd any former study of the ACS calbraton n the lterature. We apologze for havng no comparson wth former ACS calbraton method. Our future plan may focus on usng an ACS attached on dfferent parts of human body to track the moton of the human. We foresee that f calbraton of artculated cameras become a smple routne, researchers wll fnd many novel and nterestng applcatons for such a camera system. REFERENCES [] M. Antone and S. Teller. Scalable extrnsc calbraton of omn-drectonal mage networks. Internatonal Journal of Computer Vson, 49():4 74,. [] P. Baker and Y. Alomonos. Complete calbraton of a mult-camera network. Proc. IEEE Workshop on Omndrectonal Vson, :4 4,. [] P. Baker, A. Ogale, and C. Fermuller. The Argus eye: a new magng system desgned to facltate robotc tasks of moton. Robotcs & Automaton Magazne, IEEE, (4): 8, 4. [4] P. T. Baker and Y. Alomonos. Calbraton of a multcamera network. Conference on Computer Vson and Pattern Recognton Workshop, 7:7,. [5] B. Caprle and V. Torre. Usng vanshng ponts for camera calbraton. Internatonal Journal of Computer Vson, 4():7 9, 99. [6] Y. Casp and M. Iran. Algnng Non-Overlappng Sequences. Internatonal Journal of Computer Vson, 48():9 5,. [7] S. Dockstader and A. Tekalp. Multple camera trackng of nteractng and occluded human moton. Proceedngs of the IEEE, 89():44 455,. [8] F. Dornaka. Self-calbraton of a stereo rg usng monocular eppolar geometres. Pattern Recognton, 4():76 79, 7. [9] Y. Furukawa and J. Ponce. Accurate camera calbraton from mult-vew stereo and bundle adjustment. Internatonal Journal of Computer Vson, 84():57 68, 9. [] R. R. Garca and A. Zakhor. Geometrc calbraton for a mult-camera-projector system. In WACV, pages ,. [] R. I. Hartley and A. Zsserman. Multple vew geometry n computer vson. Cambrdge Unversty Press, ISBN: 55458, second edton, 4. [] J. Hekkla and O. Slven. A four-step camera calbraton procedure wth mplct magecorrecton. Computer Vson and Pattern Recognton, 997. Proceedngs., 997 IEEE Computer Socety Conference on, pages 6, 997. [] R. Horaud and F. Dornaka. Hand-eye calbraton. Internatonal Journal of Robotcs Research, 4():95, 995. [4] M. Kaess and F. Dellaert. Vsual SLAM wth a Mult-Camera Rg. Techncal report, Georga Insttute of Technology, 6. [5] H. G. Maas. Image sequence based automatc mult-camera system calbraton technques. In Internatonal Archves of Photogrammetry and Remote Sensng, (B5):76 768, 998. [6] A. Malt. Hand-eye calbraton wth eppolar constrants: Applcaton to endoscopy. Robotcs and Autonomous Systems,. [7] E. Moras, A. Ferrera, S. A. Cunha, R. M. Barros, A. Rocha, and S. Goldensten. A multple camera methodology for automatc localzaton and trackng of futsal players. Pattern Recognton Letters,. [8] S. Shah and J. Aggarwal. Intrnsc parameter calbraton procedure for a (hgh-dstorton) fsh-eye lens camera wth dstorton model and accuracy estmaton*. Pattern Recognton, 9(): , 996. [9] R. Tsa and R. Lenz. A new technque for fully autonomous and effcent D robotcshand/eye calbraton. Robotcs and Automaton, IEEE Transactons on, 5():45 58, 989. [] Z. Zhang. A flexble new technque for camera calbraton. Techncal report, Techncal Report MSR-TR-98-7, Mcrosoft Research, 998. [] Z. Zhang. A flexble new technque for camera calbraton. IEEE Transactons on Pattern Analyss and Machne Intellgence, (): 4,.

13 TABLE V RESULT OF RELATIVE POSE CALIBRATION III: our method. (see secton VI-B) F: usng feature correspondences. Algorthm Relatve Rotaton (Degree) Roll Ptch Yaw III F Algorthm Relatve Translaton (mm) T x T y T z III F Fg. THE TRAJECTORY OF THE ACS RECOVERED FROM Q AND Q TABLE VI RESULT OF RELATIVE POSE CALIBRATION USING NON-OVERLAPPING VIEW IMAGE SEQUENCES III: our method. (see secton VI-B) M: manual measurement Algorthm Relatve Rotaton (Degree) Roll Ptch Yaw III M ± 5 9 ± 5 ± 5 Algorthm Relatve Translaton (mm) T x T y T z III M 9± ± 8 ± (c) (d) Fg. THE ACS WITH TWO CANNON POWERSHOT G9 USED IN THE REAL EXPERIMENT. (A) THE ACS USED IN THE REAL EXPERIMENT. (B) THE ACS AND TWO CHECKER PLANES. (C) IN THE FRONT OF THE ACS. (D) ON THE TOP OF THE ACS.

14 4 Img Img 6 Img Img 7 Images Recorded by Camera A Img Img 6 Img Img 7 Images Recorded by Camera B Fg. 4 IMAGES RECORDED BY THE ACS Fg. 5 THE TRAJECTORY OF THE ACS RECOVERED FROM Q 5 AND Q 6 TABLE VII RESULT OF RELATIVE POSE CALIBRATION WITH UNKNOWN SCALE FACTORS (.8 IN Q AND. IN Q ) IV: our scale factor estmaton method. (see secton VI-C) F: usng feature correspondences. Algorthm Relatve Rotaton (Degree) Roll Ptch Yaw IV F Algorthm Relatve Translaton (mm) T x T y T z IV F

15 5 TABLE VIII MEAN AND STD OF THE RELATIVE POSE CALIBRATION USING NON-OVERLAPPING VIEW IMAGE SEQUENCES WITH UNKNOWN SCALE FACTORS. (Q 5 AND Q 6 ) (SEE SECTION VI-C) Algorthm Relatve Rotaton (Degree) IV Roll Ptch Yaw Mean STD Algorthm Relatve Translaton (mm) IV T x T y T z Mean STD Algorthm Relatve Scale Factor IV φ BA Mean.95 STD.59