Calibration of an Articulated Camera System

Similar documents
Calibration of an Articulated Camera System

Calibration of an Articulated Camera System with Scale Factor Estimation

SLAM Summer School 2006 Practical 2: SLAM using Monocular Vision

What are the camera parameters? Where are the light sources? What is the mapping from radiance to pixel color? Want to solve for 3D geometry

Line-based Camera Movement Estimation by Using Parallel Lines in Omnidirectional Video

METRIC ALIGNMENT OF LASER RANGE SCANS AND CALIBRATED IMAGES USING LINEAR STRUCTURES

Structure from Motion

Angle-Independent 3D Reconstruction. Ji Zhang Mireille Boutin Daniel Aliaga

A Comparison and Evaluation of Three Different Pose Estimation Algorithms In Detecting Low Texture Manufactured Objects

New dynamic zoom calibration technique for a stereo-vision based multi-view 3D modeling system

A 3D Reconstruction System of Indoor Scenes with Rotating Platform

A Fast Visual Tracking Algorithm Based on Circle Pixels Matching

Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input

Resolving Ambiguity in Depth Extraction for Motion Capture using Genetic Algorithm

Improvement of Spatial Resolution Using BlockMatching Based Motion Estimation and Frame. Integration

Finding Intrinsic and Extrinsic Viewing Parameters from a Single Realist Painting

Range images. Range image registration. Examples of sampling patterns. Range images and range surfaces

An Optimal Algorithm for Prufer Codes *

EVALUATION OF RELATIVE POSE ESTIMATION METHODS FOR MULTI-CAMERA SETUPS

Alignment of Non-Overlapping Sequences

Model reconstruction and pose acquisition using extended Lowe s method

Computer Animation and Visualisation. Lecture 4. Rigging / Skinning

Real-time Motion Capture System Using One Video Camera Based on Color and Edge Distribution

Model-Based Bundle Adjustment to Face Modeling

Articulated Tree Structure from Motion A Matrix Factorisation Approach

The motion simulation of three-dof parallel manipulator based on VBAI and MATLAB Zhuo Zhen, Chaoying Liu* and Xueling Song

Simultaneous Object Pose and Velocity Computation Using a Single View from a Rolling Shutter Camera

R s s f. m y s. SPH3UW Unit 7.3 Spherical Concave Mirrors Page 1 of 12. Notes

3D Virtual Eyeglass Frames Modeling from Multiple Camera Image Data Based on the GFFD Deformation Method

3D Rigid Facial Motion Estimation from Disparity Maps

Calibrating a single camera. Odilon Redon, Cyclops, 1914

Parallelism for Nested Loops with Non-uniform and Flow Dependences

A high precision collaborative vision measurement of gear chamfering profile

A Binarization Algorithm specialized on Document Images and Photos

PROJECTIVE RECONSTRUCTION OF BUILDING SHAPE FROM SILHOUETTE IMAGES ACQUIRED FROM UNCALIBRATED CAMERAS

Quick error verification of portable coordinate measuring arm

Pose, Posture, Formation and Contortion in Kinematic Systems

Analysis of Continuous Beams in General

Analysis on the Workspace of Six-degrees-of-freedom Industrial Robot Based on AutoCAD

Planar Catadioptric Stereo: Multiple-View Geometry and Image-Based Camera Localization

DISTRIBUTED POSE AVERAGING IN CAMERA SENSOR NETWORKS USING CONSENSUS ON MANIFOLDS

Detection of an Object by using Principal Component Analysis

Positive Semi-definite Programming Localization in Wireless Sensor Networks

CS 534: Computer Vision Model Fitting

ScienceDirect. The Influence of Subpixel Corner Detection to Determine the Camera Displacement

A Robust Method for Estimating the Fundamental Matrix

Self-Calibration from Image Triplets. 1 Robotics Research Group, Department of Engineering Science, Oxford University, England

Secure and Fast Fingerprint Authentication on Smart Card

The Research of Ellipse Parameter Fitting Algorithm of Ultrasonic Imaging Logging in the Casing Hole

Large Motion Estimation for Omnidirectional Vision

ROBOT KINEMATICS. ME Robotics ME Robotics

UAV global pose estimation by matching forward-looking aerial images with satellite images

INVERSE DYNAMICS ANALYSIS AND SIMULATION OF A CLASS OF UNDER- CONSTRAINED CABLE-DRIVEN PARALLEL SYSTEM

Amnon Shashua Shai Avidan Michael Werman. The Hebrew University, objects.

APPLICATION OF AN AUGMENTED REALITY SYSTEM FOR DISASTER RELIEF

Computer Vision. Exercise Session 1. Institute of Visual Computing

Image Alignment CSC 767

Kinematics of pantograph masts

Improving Initial Estimations for Structure from Motion Methods

Corner-Based Image Alignment using Pyramid Structure with Gradient Vector Similarity

A New Feature of Uniformity of Image Texture Directions Coinciding with the Human Eyes Perception 1

Robust Recovery of Camera Rotation from Three Frames. B. Rousso S. Avidan A. Shashua y S. Peleg z. The Hebrew University of Jerusalem

MOTION PANORAMA CONSTRUCTION FROM STREAMING VIDEO FOR POWER- CONSTRAINED MOBILE MULTIMEDIA ENVIRONMENTS XUNYU PAN

New Extensions of the 3-Simplex for Exterior Orientation

3D Modeling Using Multi-View Images. Jinjin Li. A Thesis Presented in Partial Fulfillment of the Requirements for the Degree Master of Science

arxiv: v1 [cs.ro] 8 Jul 2016

A 3D Human Skeletonization Algorithm for a Single Monocular Camera Based on Spatial Temporal Discrete Shadow Integration

Correspondence-free Synchronization and Reconstruction in a Non-rigid Scene

Content Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers

DESIGN OF A HAPTIC DEVICE FOR EXCAVATOR EQUIPPED WITH CRUSHER

Six-axis Robot Manipulator Numerical Control Programming and Motion Simulation

MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION

Accounting for the Use of Different Length Scale Factors in x, y and z Directions

Human Skeleton Reconstruction for Optical Motion Capture

Proper Choice of Data Used for the Estimation of Datum Transformation Parameters

Learning the Kernel Parameters in Kernel Minimum Distance Classifier

High-Boost Mesh Filtering for 3-D Shape Enhancement

Aligning Non-Overlapping Sequences,

An Image Fusion Approach Based on Segmentation Region

Dynamic Camera Assignment and Handoff

An Application of the Dulmage-Mendelsohn Decomposition to Sparse Null Space Bases of Full Row Rank Matrices

Tsinghua University at TAC 2009: Summarizing Multi-documents by Information Distance

A NEW IMPLEMENTATION OF THE ICP ALGORITHM FOR 3D SURFACE REGISTRATION USING A COMPREHENSIVE LOOK UP MATRIX

An Adaptive Complementary Filter For Gyroscope/Vision Integrated Attitude Estimation

Inverse-Polar Ray Projection for Recovering Projective Transformations

A Revisit of Methods for Determining the Fundamental Matrix with Planes

A New Approach For the Ranking of Fuzzy Sets With Different Heights

Palmprint Feature Extraction Using 2-D Gabor Filters

Delayed Features Initialization for Inverse Depth Monocular SLAM

Invariant Shape Object Recognition Using B-Spline, Cardinal Spline, and Genetic Algorithm

A Background Subtraction for a Vision-based User Interface *

The Comparison of Calibration Method of Binocular Stereo Vision System Ke Zhang a *, Zhao Gao b

NUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS

X- Chart Using ANOM Approach

Human skeleton proportions from monocular data

3D vector computer graphics

MOTION BLUR ESTIMATION AT CORNERS

Recovering Camera Pose from Omni-directional Images

Towards A Human Robot Interaction Framework with Marker-less Augmented Reality and Visual SLAM

A Scalable Projective Bundle Adjustment Algorithm using the L Norm

Transcription:

Calbraton of an Artculated Camera System CHEN Junzhou and Kn Hong WONG Department of Computer Scence and Engneerng The Chnese Unversty of Hong Kong {jzchen, khwong}@cse.cuhk.edu.hk Abstract Multple Camera Systems (MCS) have been wdely used n many vson applcatons and attracted much attenton recently. There are two prncple types of MCS, one s the gd Multple Camera System (MCS); the other s the Artculated Camera System (ACS). In an MCS, the relatve poses (relatve -D poston and orentaton) between the cameras are nvarant. Whle, n an ACS, the cameras are artculated through movable jonts, the relatve pose between them may change. Therefore, through calbraton of an ACS we want to fnd not only the relatve poses between the cameras but also the postons of the jonts n the ACS. Although calbraton methods for MCS have been extensvely developed durng the past decades, the studes of ACS calbraton are stll rare. In ths paper, two ACS calbraton methods are proposed. The frst one uses the feature correspondences between the cameras n the ACS. The second one requres only the ego-moton nformaton of the cameras and can be used for the calbraton of the nonoverlappng vew ACS. In both methods, the ACS s assumed to have performed general transformatons n a statc envronment. The effcency and robustness of the proposed methods are tested by smulaton and real experments. In the real experment, the ntrnsc and extrnsc parameters of the ACS are calbrated usng the same mage sequences, no extra data capturng step s requred. The correspondng trajectory s recovered and llustrated usng the calbraton results of the ACS. To our knowledge, we are the frst to study the calbraton of ACS.. Introducton Calbraton of a Multple Camera System (MCS) s an essental step n many computer vson tasks such as SLAM (Smultaneous Localzaton and Map), survellance, stereo and metrology,, 7,. Both the ntrnsc and extrnsc parameters of the MCS are requred to be estmated before the MCS can be used. The ntrnsc parameters 9 descrbe the nternal camera geometrc and optcal character- Fgure. An obot wth Four Cameras Attached on It, Where the Cameras are Artculated. stcs of each camera n the MCS. In a gd Multple Camera System (MCS), the cameras are fxed to each other. The extrnsc parameters 5 of an MCS descrbe the relatve pose (the relatve -D poston and orentaton, totally, sx degrees of freedom) between the cameras n the MCS. Calbraton methods of the ntrnsc parameters of a camera are well establshed 5, 9. Calbraton methods for the extrnsc parameters of an MCS are also wdely studed. For nstance, Maas proposed an automatc MCS calbraton technque wth a movng reference bar whch can be seen by all cameras. Antone and Teller developed an algorthm whch recovers the relatve poses of cameras by overlappng portons of the outdoor scene. Baker and Alomonos presented MCS calbraton methods usng calbraton objects such as a wand wth LEDs or a rgd board wth known patterns, 4. Dornaka proposed a stereo rg selfcalbraton method by the monocular eppolar geometres and geometrc constrants of a movng MCS, n whch only the feature correspondences between the monocular mages of each camera are requred 8. In hand-eye calbraton, t s demonstrated that when a sensor s mounted on a movng robot hand, the relatonshp between the sensor coordnate system and hand coordnate system can be calculated by the moton nformaton of the hand and the sensor. One example of usng knematc nformaton of the cameras for MCS s dscussed by Casp and Iran

6, they ndcated that f the cameras of a non-overlappng vew MCS are close to each other and share a same projecton center, ther recorded mage sequences can be algned effectvely by the estmated transformatons nsde each mage sequence. However, n some types of MCS, the relatve poses between the cameras are not fxed, hence the calbraton methods for the MCS cannot be used drectly. In Fgure, a novel applcaton of lmb pose estmaton by attachng cameras on the arms of a robot s shown. On each arm of the robot, two cameras are artculated to each other through the elbow jont of the arm. When the robot moves, the relatve pose between the cameras may change, whle, the coordnate of the elbow jont refers to each camera attached on the correspondng arm s nvarant. In ths paper, such a type of MCS s named as Artculated Camera System (ACS). The jont of the elbow s named as the jont n the ACS. ACSs can be easly found n the real world, such as camera systems attached on human, robots and anmals. Before usng an ACS, t has to be calbrated. However, there are stll some unsolved problems: () In an ACS wth overlappng vew, tradtonal calbraton methods cannot estmate the postons of the jonts n the ACS. () In a nonoverlappng vew ACS, nether the postons of the jonts n the ACS nor the relatve poses between the cameras n the ACS can be estmated by tradtonal calbraton methods. These consderatons n mnd motvate us to develop the technologes n ths paper. The rest of ths paper are organzed as follows: Secton and analyss the constrants n a movng ACS. The correspondng calbraton methods are proposed. Secton 4 and 5 evaluate the proposed method by smulaton and real experment. In secton 6, a bref concluson and the future plan are presented.. Calbraton of ACS wth Overlappng Vews system of camera A, C B the coordnate system of camera B. Suppose there are enough feature correspondences between the cameras so that the pose of C A and C B referrng to the same coordnate system C W can be estmated. Therefore, the relatve pose between C A and C B s known. We want to fnd the poston of O n the ACS. Let H AW and H BW be the Eucldean transformaton matrxes descrbe the C A and C B refer to C W, so that for any pont P : P A = H AW P W = P B = H BW P W = AW T AW BW T BW P W () P W (), where s the rotaton matrx, T s vector, P W, P A and P B are the homogenous coordnates of the -D Pont P refer to C W, C A and C B respectvely. Accordng to equatons () and (): P W = H AW P A = H BW P B () H AW P A H BW P B = (4) T AW T AW T AW PA T BW T BW T BW PB = (5) T AW P A T BW P B = T AWT AW T BWT BW (6), where T s the transpose of. Suppose the ACS performed n transformatons, for the -th transformaton of the ACS, accordng to equaton (6): ( AW) T PA ( BW) T PB = ( AW) T TAW ( BW) T TBW (7) Let Õ = ŌA T ŌB T T, where Ō A and ŌB are the coordnates of the jont O refer to C A and C B respectvely. Equaton (7) can be rewrtten as: ( AW ) T ( BW )T Õ = ( AW) T T AW ( BW) T T BW (8) Fgure. An Artculated Camera System wth Overlappng Vews Suppose two rgd objects are artculated at jont O and two cameras (camera A and B) are fxed on the two rgd objects respectvely (See Fgure ). Let C A be the coordnate Snce camera A and B are fxed on the artculated rgd objects, Õ s nvarant durng the transformaton of the ACS. The transformatons ( AW, BW, T AW and T BW for... n) of the camera coordnate systems are calculated by the projected mage sequences. We propose that Õ can be estmated by a least squares method, when the ACS has moved to many dfferent postons and captured enough samples of AW, BW, T AW and T BW.

Let M A = ( A I)T, ( A I)T,..., ( n A I)T T, T A = (T A )T, (T A )T,..., (T n A )T T, we have: M A Ō A = T A () Snce the transformatons ( A and T A,...n) of camera A can be calculated by the projected mage sequence. We propose ŌA can be estmated by a least squares method. Smlarly, Ō B can also be estmated. Therefore, O A and O B are recovered. Fgure. A Non-overlappng Vew Artculated Camera System. Calbraton of Non-Overlappng Vew ACS In many stuatons, there s no overlappng vew between the cameras n an ACS. And the lack of common features makes the calbraton method proposed n secton become nvald (See Fgure ). Moreover, snce the relatve pose between the cameras n the ACS cannot be estmated by the overlappng vews, the calbraton of the relatve poses between the non-overlappng vew cameras s also requred. In ths secton, a calbraton method based on the ego-moton nformaton of the cameras n an ACS s dscussed... ecoverng the Poston of the Jont efers to the Cameras n the ACS Let CA nt and CB nt be the coordnate systems of camera A and B respectvely at the ntal state (tme t = ). Suppose the ACS performs n transformatons. Snce the coordnate of the jont O refers to camera A s fxed durng the transformaton of the ACS. At tme t =, we have: O A = H AO A = A T A O A (9), where H A s the Eucldean transformaton matrx of camera A at tme refers to CA nt. A and T A descrbe the orentaton and orgn of camera A at tme refer to CA nt. Also O A s the coordnate of pont O at ntal state refers to CA nt, and O A s the coordnate of pont O at tme refers to CA nt. If the poston of the jont O refers to CA nt s fxed durng the transformatons of the ACS, we have: OA = O A,,...,n. For -th transformaton of the ACS, accordng to equaton (9): O A = H A O A = A T A O A () ( A I)ŌA = T A ().. The Unqueness of the Jont Pose Estmaton If the dfferent segments of the artculated camera system (ACS) are connected by D rotatonal jonts (connected by pont rotatonal jonts) and the ACS can perform general transformatons, the soluton of the jont pose estmaton s unque: For the jont pose estmaton method usng specal moton (n secton.). Suppose the soluton of the jont pose estmaton s not unque, there must exst at lest two dfferent D ponts O and O satsfy equaton (). We have: M A O = T A and M A O = T A. Therefore, any pont P = so +( s)o wll also satsfy equaton (), where s s an arbtrary scalar. Accordng to the defnton of P, P s the pont on the lne passng through the ponts O and O. Snce P satsfy equaton () represents that the poston of the P refers to the camera n the ACS s nvarant durng the transformaton of the ACS, t means the dfferent segments of ACS are connected by the D rotatonal axs nstead of the D rotatonal jonts. The poston of the ponts on the D rotatonal axs refer to the camera n the ACS s nvarant durng the transformaton of the ACS. However, t conflcts wth the assumpton. Smlarly, the unqueness of the jont pose estmaton method usng overlappng vews (n secton ) can also be verfed... ecoverng the elatve Pose Between the Cameras of the Non-overlappng vew ACS Let H BA be the Eucldean transformaton matrx between CA nt and Cnt B, so that for any pont P : P B = H BA P A = BA T BA P A = H BA P A (), where P A and P B are the homogenous coordnate of Pont P refer to CA nt and Cnt B respectvely. The relatve pose ( BA and T BA ) between CA nt and CB nt s defned as: BA = T BA (4) T BA = T BA T BA (5)

Let OB be the coordnate of jont O at tme refer to CB nt. Snce the coordnate of the jont O refer to camera B s nvarant: OB = B TB O B = B TB BA T BA O A = B BA B T BA + TB O A (6) Accordng to equatons (9) and (): OB = H BAOA BA T = BA = A T A BA A BAT A + T BA Accordng to equatons (6) and (7): B BA B T BA + TB ŌA BA = A BATA + T BA ŌA B BA Ō A + B T BA + TB BA = AŌA + BA TA + T BA O A O A (7) (8) (9) B BA Ō A + BT BA BA AŌA BA T A + T B T BA = () Snce ŌA can be estmated by the method dscussed n secton., the BA and T BA can be estmated by a least square method, when the ACS perform enough general motons. In our smulaton and real experment, the estmated BA s refned by a method dscussed n 4. Then the roll, ptch and yaw correspondng to the BA are estmated accordng to the defnton of the rotaton matrx 9. Let BA = M(r, p, y), where r p and y are the correspondng roll, ptch and yaw of BA, M s a functon from roll, ptch and yaw to the correspondng rotaton matrx. Then, the r, p, y, T BA and ŌA are optmzed by mnmzng the nonlnear error functon: n E(r, p, y, T BA, O A ) = ( BM(r, p, y)ōa + BT BA = M(r, p, y) AŌA M(r, p, y)t A + T B T BA ) () usng a Levenberg-Marquardt method. Fnally, the BA s recovered from the optmzed r, p and y. The relatve pose between the CA nt and Cnt B s calculated by equatons (4) and (5). 4. Smulaton In ths secton, the proposed calbraton methods are evaluated wth synthetc transformaton data. 4.. Performance w.r.t. Nose n Transformaton Data Setup and Notatons: In each test, one ACS wth cameras and jont s generated randomly. In whch, O A meters, O B meters. The generated ACS performs random transformatons. Performance of the Calbraton Method for ACS wth Overlappng Vews: In the frst smulaton, the proposed algorthm s tested tmes. Zero mean Gaussan nose s added to the transformaton data of the cameras. The confguraton, nput and output of our smulaton system are lst as Table. Snce we assume there are overlappng vews between the two cameras, the relatve pose between them can be estmated by many exstng methods as dscussed n secton. Only the performance of jont pose estmaton s evaluated n our smulaton. The error of jont estmaton are computed by: Err = ŌA ˆŌ A ŌA + ŌB ˆŌ B ŌB (), where ŌA s the ground truth, ˆŌ A s the estmated poston of jont O refer to camera A. Smlarly, Ō B s the ground truth, ˆŌ B s the estmated poston of jont O refer to camera B. The correspondng results are shown n Fgure 4 and 5. Table. Confguraton, Input and Output Confguraton No. of Cameras n the ACS No. of Jonts n the ACS andom transformatons per test (n) Number of tests Input ( =...n) otatons of cameras ( AW, BW ) Translatons of cameras (TAW, T BW ) Zero Mean Gaussan nose: σ rot.4 and σ trans.meters Output Mean error of jont pose estmaton STD error of jont pose estmaton Performance of the Calbraton Method for Non- Overlappng Vews ACS: In the second smulaton, frstly, the pose of the jont s fxed refers to CA nt durng the transformatons of the ACS. The pose of the jont refers to the camera A (O A ) s calbrated by the transformatons of camera A. Smlarly, O B s calbrated. Then, the ACS performs several general transformatons (the jont s not needed to

Mean Error of Jont Pose Estmaton..5..5..5 otaton Nose (degree) Mean Error of Jont Pose Estmaton..4.6.8 Translaton Nose (meter) Fgure 4. Mean Error of Jont Poston Estmaton STD Error of Jont Pose Estmaton.5..5..5 otaton Nose (degree) STD Error of Jont Pose Estmaton..4.6.8 Translaton Nose (meter).. Table. Confguraton, Input and Output Confguraton No. of Cameras n the ACS No. of Jonts n the ACS andom transformatons per test (n) Number of tests Input ( =...n) Transformatons wth fxed jont pose: otatons of cameras ( A, B ) Translatons of cameras (TA, T B ) General transformatons: otatons of cameras ( A, B ) Translatons of cameras (TA, T B ) Zero Mean Gaussan nose: σ rot.4 and σ trans.meters Output Mean error of jont pose estmaton STD error of jont pose estmaton Mean error of relatve translaton estmaton STD error of relatve translaton estmaton Mean error of relatve rotaton estmaton STD error of relatve rotaton estmaton Mean Error of Jont Pose Estmaton Fgure 5. STD Error of Jont Poston Estmaton be fxed refer to CA nt ), the relatve pose between the cameras are calbrated usng the estmated jont pose and the transformatons of the cameras. The confguraton, nput and output of the smulaton system are lsted as Table. The error of jont pose, relatve rotaton, relatve translaton estmaton are calculated by equaton (), () and (4) respectvely. Fgure 6 and 7 show the results of jont pose estmaton. Compare wth the calbraton method usng the overlappng vews, the calbraton method usng specal motons s more accurate. The mean and STD error of the relatve rotaton and translaton estmaton are presented n Fgure 8, 9, and. The proposed algorthms are shown to be stable, when the zero mean Gaussan nose from to.4 s added to the roll, ptch and yaw of the rotaton data, and the zero mean Gaussan nose from to. meters s added to the translaton data. Err = roll roll + ptch ptch + yaw ŷaw () Mean Error of Jont Pose Estmaton..5..5 otaton Nose (degree)..4.6.8 Translaton Nose (meter) Fgure 6. Mean Error of Jont Poston Estmaton STD Error of Jont Pose Estmaton 8 6 4 x otaton Nose (degree) STD Error of Jont Pose Estmaton..4.6.8 Translaton Nose (meter).. Err = T AB ˆT AB T AB (4) Fgure 7. STD Error of Jont Poston Estmaton

Mean Error of elatve otaton Estmaton STD Error of elatve Translaton Estmaton Mean Error of elatve otaton Estmaton (degree).5.5 otaton Nose (degree)...8.6.4 Translaton Nose (meter) STD Error of elatve Translaton Estmaton.5..5 otaton Nose (degree)...8.6.4 Translaton Nose (meter) Fgure 8. Mean Error of elatve otaton Estmaton Fgure. STD Error of elatve Translaton Estmaton STD Error of elatve otaton Estmaton (degree).8.6.4. otaton Nose (degree) STD Error of elatve otaton Estmaton..4.6.8 Translaton Nose (meter) Fgure 9. STD Error of elatve otaton Estmaton Mean Error of elatve Translaton Estmaton.5..5..5 otaton Nose (degree) Mean Error of elatve Translaton Estmaton..4.6.8 Translaton Nose (meter) Fgure. Mean Error of elatve Translaton Estmaton 5. eal Experment In the real experments, an ACS wth two cameras (Cannon PowerShot G9) s set up as Fgure (a). The ntrnsc parameters of each camera n the ACS are calbrated by Bouguet s mplementaton ( Camera Calbraton Toolbox for Matlab ) of 5. Snce the Bouguet s Toolbox can also estmate the pose nformaton of the camera, the transformatons of each camera are calculated usng the same mage sequence for the ntrnsc calbraton smultaneously. No addtonal mages nor manual nput s requred n the real experments... 5.. Calbraton of the Pose of the Jont n Each Camera By Overlappng Vews (Algorthm I): In the frst real experment, the two cameras n the ACS observe the same checker plane and record mages smultaneously. The two cameras are free to move durng the transformaton of the ACS. Two mage sequences (Q and Q ) are recorded, each sequence conssts of 5 mages of sze 6. The estmated jont pose are lst n Table as algorthm I. By Fxed-Jont Motons (Algorthm II): In the second real experment, the jont of the ACS s fxed refers to the world coordnate system durng the transformaton of the ACS. The two cameras do not need to vew the same checker plane. And each camera records the mage sequence ndependently. Two mage sequences (Q and Q 4 ) are recorded, each sequence conssts of mages of sze 6. The camera pose of the frst mage s selected as the ntal pose to generate the transformaton sequence of each camera. The estmated jont pose are lst n Table as algorthm II. The poses of the jont refer to the two cameras n the ACS are also estmated manually for comparson purpose. Snce the camera pose of any mage n each mage sequence can be chosen as the ntal camera pose (see secton.), the proposed algorthm s also tested by choosng dfferent mages as the reference. The mean and standard dervaton of the correspondng calbraton results are presented n Table 4. 5.. Calbraton of elatve Pose Between the Cameras n the Non-Overlappng Vew ACS (Algorthm III) In the thrd real experment, frstly, we use the nonoverlappng vew ACS calbraton method to process the mage sequences Q and Q. The jont pose (Ō A ) estmated by algorthm II s used as the nput for the relatve pose calbraton. Snce there are overlappng vews between Q and Q, we also calbrate the relatve pose between the two cameras by the feature correspondences for compar-

Table. esults Of Jont Pose Calbraton I: the algorthm usng overlappng vews. II: the algorthm usng fxed-jont motons. M: manual measurement(ground truth). O A s the coordnate of the jont refers to camera A, the same apples to O B. Algorthm Jont Pose (mm) X Y Z I O A.8 5.7 -.47 O B -7.7 5.8 -.5 II O A 4.55 47.64-7.66 O B -65 54.4-5.48 M O A ± 5± -4± O B -7± 5± -± Table 4. Mean and STD of the Jont Pose Calbraton Algorthm II Usng Dfferent eference Images. (O A s the coordnate of the jont refers to camera A, the same apples to O B.) Algorthm Jont Pose (mm) II X Y Z Mean O A 5.44 47.9-9. O B -6.97 56. -9. STD O A.89.6. O B..67.58 son. The calbraton result are lsted n Table 5. After the jont pose refers to each camera n the ACS and relatve pose between the cameras n the ACS are calbrated, the trajectory of the ACS s recovered (see Fgure ). The proposed Table 5. esult of elatve Pose Calbraton III: our method. F: usng feature correspondences. Algorthm elatve otaton (Degree) oll Ptch Yaw III 7.758 -.66-8.9 F 7.5459 -.64-78.9854 Algorthm elatve Translaton (mm) T x T y T z III 95.48 -.4576 4.54 F 94.5-9.869 8.979 calbraton method s also tested by non-overlappng vew mage sequences. Fgure (b), (c), (d) shows the confguraton of the non-overlappng vew ACS calbraton system n the real experment. Two mage sequences (Q 5 and Q 6 ) are recorded, each sequence conssts of 7 mages of sze 6. There s no overlappng vew between Q 5 and Q 6. Fgure 4 shows some samples of the recorded mages. We also manually measured the relatve pose between the two cameras for comparson. Snce no feature correspondence can be used, we only get a rough estmaton by a ruler. The calbraton results are shown n Table 6. After the relatve pose between the cameras at the ntal state s estmated, the trajectory of the non-overlappng vew ACS s recovered (see Fgure 5). Fgure. The Trajectory of the ACS ecovered from Q and Q (a) (c) (d) Fgure. The ACS wth Two Cannon PowerShot G9 Used n the eal Experment. (a) The ACS Used n the eal Experment. (b) The ACS and two Checker Planes. (c) In the Front of the ACS. (d) On the Top of the ACS. (b) Img Img 6 Img Img 7 (a) Images ecorded by Camera A Img Img 6 Img Img 7 (b) Images ecorded by Camera B Fgure 4. Images ecorded by the ACS 6. Concluson In ths paper, an ACS calbraton method s developed. Both the smulaton and real experment show that the pose of the jont n an ACS can be estmated robustly. When there s no overlappng vew between the cameras n an ACS, the jont pose and the relatve pose between the cameras can also be calculated. The trajectory of an ACS can be recov-

Table 6. esult of elatve Pose Calbraton Usng Non- Overlappng Vew Image Sequences. (III: our method. M: manual measurement.) Algorthm elatve otaton (Degree) oll Ptch Yaw III.8 88.45.75 M ± 5 9 ± 5 ± 5 Algorthm elatve Translaton (mm) T x T y T z III 9. -7.87-9.8 M 9± ± 8 ± Fgure 5. The Trajectory of the ACS ecovered from Q 5 and Q 6 ered after the ACS s calbrated. The proposed calbraton method requres only the mage sequences recorded by the cameras n the ACS. In the real experment, the ntrnsc and extrnsc parameters of the ACS are calbrated usng the same mage sequences smultaneously. Our future plan may focus on usng an ACS attached on dfferent parts of human body to track the moton of the human. We foresee that f calbraton of artculated cameras become a smple routne, researchers wll fnd many novel and nterestng applcatons for such a camera system. Acknowledgement. We apprecate the revews for ther suggestons and menton of the relatve works. We would lke to thank Prof. Jaya JIA for dscusson, Ms. SHAO Lu n HKBU for checkng the Englsh, Mr. Gang LI, Hongnng DAI and other frends n CUHK for ther help. The research s supported by by a drect grant (code 55) from the Faculty of Engneerng, The Chnese Unversty of Hong Kong, Shatn, Hong Kong. eferences M. Antone and S. Teller. Scalable extrnsc calbraton of omn-drectonal mage networks. Internatonal Journal of Computer Vson, 49():4 74,. P. Baker and Y. Alomonos. Complete calbraton of a mult-camera network. Proc. IEEE Workshop on Omndrectonal Vson, :4 4,. P. Baker, A. Ogale, and C. Fermuller. The Argus eye: a new magng system desgned to facltate robotc tasks of moton. obotcs & Automaton Magazne, IEEE, (4): 8, 4. 4 P. T. Baker and Y. Alomonos. Calbraton of a multcamera network. Conference on Computer Vson and Pattern ecognton Workshop, 7:7,. 5 B. Caprle and V. Torre. Usng vanshng ponts for camera calbraton. Internatonal Journal of Computer Vson, 4():7 9, 99. 6 Y. Casp and M. Iran. Algnng Non-Overlappng Sequences. Internatonal Journal of Computer Vson, 48():9 5,. 7 S. Dockstader and A. Tekalp. Multple camera trackng of nteractng and occluded human moton. Proceedngs of the IEEE, 89():44 455,. 8 F. Dornaka. Self-calbraton of a stereo rg usng monocular eppolar geometres. Pattern ecognton, 4():76 79, 7. 9. I. Hartley and A. Zsserman. Multple vew geometry n computer vson. Cambrdge Unversty Press, ISBN: 55458, second edton, 4., 4. Horaud and F. Dornaka. Hand-eye calbraton. Internatonal Journal of obotcs esearch, 4():95, 995. M. Kaess and F. Dellaert. Vsual SLAM wth a Mult- Camera g. Techncal report, Georga Insttute of Technology, 6. T. Kanade, P. ander, and P. Narayanan. Vrtualzed realty: constructng vrtual worlds from real scenes. Multmeda, IEEE, 4():4 47, 997. H. G. Maas. Image sequence based automatc multcamera system calbraton technques. In Internatonal Archves of Photogrammetry and emote Sensng, (B5):76 768, 998. 4 Z. Zhang. A flexble new technque for camera calbraton. Techncal report, Techncal eport MS-T- 98-7, Mcrosoft esearch, 998. 4 5 Z. Zhang. A flexble new technque for camera calbraton. IEEE Transactons on Pattern Analyss and Machne Intellgence, (): 4,., 6