Large Motion Estimation for Omnidirectional Vision

Similar documents
SLAM Summer School 2006 Practical 2: SLAM using Monocular Vision

CS 534: Computer Vision Model Fitting

Improvement of Spatial Resolution Using BlockMatching Based Motion Estimation and Frame. Integration

Structure from Motion

A Binarization Algorithm specialized on Document Images and Photos

Subspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;

Parallelism for Nested Loops with Non-uniform and Flow Dependences

Analysis of Continuous Beams in General

Learning the Kernel Parameters in Kernel Minimum Distance Classifier

Line-based Camera Movement Estimation by Using Parallel Lines in Omnidirectional Video

Resolving Ambiguity in Depth Extraction for Motion Capture using Genetic Algorithm

Reducing Frame Rate for Object Tracking

A Fast Visual Tracking Algorithm Based on Circle Pixels Matching

MOTION BLUR ESTIMATION AT CORNERS

MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION

R s s f. m y s. SPH3UW Unit 7.3 Spherical Concave Mirrors Page 1 of 12. Notes

3D vector computer graphics

Range images. Range image registration. Examples of sampling patterns. Range images and range surfaces

Delayed Features Initialization for Inverse Depth Monocular SLAM

X- Chart Using ANOM Approach

Content Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers

Wishing you all a Total Quality New Year!

Support Vector Machines

A Robust Method for Estimating the Fundamental Matrix

Angle-Independent 3D Reconstruction. Ji Zhang Mireille Boutin Daniel Aliaga

Proper Choice of Data Used for the Estimation of Datum Transformation Parameters

High-Boost Mesh Filtering for 3-D Shape Enhancement

The Codesign Challenge

Mathematics 256 a course in differential equations for engineering students

Skew Angle Estimation and Correction of Hand Written, Textual and Large areas of Non-Textual Document Images: A Novel Approach

User Authentication Based On Behavioral Mouse Dynamics Biometrics

Distance Calculation from Single Optical Image

MOTION PANORAMA CONSTRUCTION FROM STREAMING VIDEO FOR POWER- CONSTRAINED MOBILE MULTIMEDIA ENVIRONMENTS XUNYU PAN

Lecture 4: Principal components

Face Recognition University at Buffalo CSE666 Lecture Slides Resources:

Outline. Type of Machine Learning. Examples of Application. Unsupervised Learning

Simultaneous Object Pose and Velocity Computation Using a Single View from a Rolling Shutter Camera

New dynamic zoom calibration technique for a stereo-vision based multi-view 3D modeling system

Video Object Tracking Based On Extended Active Shape Models With Color Information

An efficient method to build panoramic image mosaics

Kinematics of pantograph masts

Calibration of an Articulated Camera System

The Research of Ellipse Parameter Fitting Algorithm of Ultrasonic Imaging Logging in the Casing Hole

Fitting & Matching. Lecture 4 Prof. Bregler. Slides from: S. Lazebnik, S. Seitz, M. Pollefeys, A. Effros.

Calibration of an Articulated Camera System with Scale Factor Estimation

Lecture 5: Multilayer Perceptrons

A MOVING MESH APPROACH FOR SIMULATION BUDGET ALLOCATION ON CONTINUOUS DOMAINS

Positive Semi-definite Programming Localization in Wireless Sensor Networks

Dynamic wetting property investigation of AFM tips in micro/nanoscale

Tsinghua University at TAC 2009: Summarizing Multi-documents by Information Distance

Term Weighting Classification System Using the Chi-square Statistic for the Classification Subtask at NTCIR-6 Patent Retrieval Task

Environmental Mapping by Trinocular Vision for Self-Localization Using Monocular Vision

Hermite Splines in Lie Groups as Products of Geodesics

Model-Based Bundle Adjustment to Face Modeling

Real-time Motion Capture System Using One Video Camera Based on Color and Edge Distribution

Model reconstruction and pose acquisition using extended Lowe s method

Face Tracking Using Motion-Guided Dynamic Template Matching

Load Balancing for Hex-Cell Interconnection Network

A Unified Framework for Semantics and Feature Based Relevance Feedback in Image Retrieval Systems

An Improved Image Segmentation Algorithm Based on the Otsu Method

NUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS

A Comparison and Evaluation of Three Different Pose Estimation Algorithms In Detecting Low Texture Manufactured Objects

Calibration of an Articulated Camera System

An Image Fusion Approach Based on Segmentation Region

On Incremental and Robust Subspace Learning

Accounting for the Use of Different Length Scale Factors in x, y and z Directions

EVALUATION OF RELATIVE POSE ESTIMATION METHODS FOR MULTI-CAMERA SETUPS

Related-Mode Attacks on CTR Encryption Mode

Unsupervised Learning

PCA Based Gait Segmentation

Simulation: Solving Dynamic Models ABE 5646 Week 11 Chapter 2, Spring 2010

AP PHYSICS B 2008 SCORING GUIDELINES

VFH*: Local Obstacle Avoidance with Look-Ahead Verification

Human Skeleton Reconstruction for Optical Motion Capture

Articulated Tree Structure from Motion A Matrix Factorisation Approach

Edge Detection in Noisy Images Using the Support Vector Machines

Hierarchical clustering for gene expression data analysis

Fitting and Alignment

Problem Definitions and Evaluation Criteria for Computational Expensive Optimization

3D Rigid Facial Motion Estimation from Disparity Maps

Communication-Minimal Partitioning and Data Alignment for Af"ne Nested Loops

Assignment # 2. Farrukh Jabeen Algorithms 510 Assignment #2 Due Date: June 15, 2009.

S1 Note. Basis functions.

2x x l. Module 3: Element Properties Lecture 4: Lagrange and Serendipity Elements

Complex Numbers. Now we also saw that if a and b were both positive then ab = a b. For a second let s forget that restriction and do the following.

Vanishing Hull. Jinhui Hu, Suya You, Ulrich Neumann University of Southern California {jinhuihu,suyay,

Multi-view 3D Position Estimation of Sports Players

METRIC ALIGNMENT OF LASER RANGE SCANS AND CALIBRATED IMAGES USING LINEAR STRUCTURES

CORRELATION ICP ALGORITHM FOR POSE ESTIMATION BASED ON LOCAL AND GLOBAL FEATURES

Circuit Analysis I (ENGR 2405) Chapter 3 Method of Analysis Nodal(KCL) and Mesh(KVL)

Multi-stable Perception. Necker Cube

An Efficient Background Updating Scheme for Real-time Traffic Monitoring

Parallel matrix-vector multiplication

An Entropy-Based Approach to Integrated Information Needs Assessment

UAV global pose estimation by matching forward-looking aerial images with satellite images

From: AAAI-82 Proceedings. Copyright 1982, AAAI ( All rights reserved.

LEAST SQUARES. RANSAC. HOUGH TRANSFORM.

Genetic Tuning of Fuzzy Logic Controller for a Flexible-Link Manipulator

Determining the Optimal Bandwidth Based on Multi-criterion Fusion

PHOTOGRAMMETRIC ANALYSIS OF ASYNCHRONOUSLY ACQUIRED IMAGE SEQUENCES

Transcription:

Large Moton Estmaton for Omndrectonal Vson Jong Weon Lee, Suya You, and Ulrch Neumann Computer Scence Department Integrated Meda Systems Center Unversty of Southern Calforna Los Angeles, CA 98978, USA {jonlee suyay uneumann}@graphcs.usc.edu Abstract We present a method for estmatng large camera motons, ncludng combnatons of large rotatons and dsplacements. We know that translaton estmates are mproved as the dsplacements between two camera postons ncrease. However, there s lttle work on moton estmaton methods targeted at large motons snce t s dffcult to fnd correspondences between two planar perspectve mages under large motons. We can overcome the correspondence problem wth omndrectonal cameras, whch obtan a hemsphere or cylnder projecton of the envronment. We frst develop a moton estmaton method for smooth omndrectonal camera motons. Our method ncrementally mproves estmates from ncomplete nformaton provded by a sngle feature correspondence, usng an Implct Extended Kalman Flter (IEKF). hs general moton estmaton method produces stable estmates from smooth camera motons, but t s not drectly applcable to estmatng large motons. We adapt the moton estmaton method to large motons by usng a novel Recursve Rotaton Factorzaton (RRF) that removes the mage motons due to rotaton. Smulaton results show that the RRF large moton estmates are accurate and robust. INRODUCION Moton estmaton technques are dvded nto two categores, based on ther use of optcal flows or feature correspondences. Both technques are used to estmate motons from planar projecton mages that only offer a lmted feld of vew. Due to the lack of global nformaton, estmates from perspectve mages are very senstve to nose when the translaton drecton s outsde of the feld of vew. It s well known that estmaton methods have dffculty dstngushng between small pure translatons and small pure rotatons [3], whch often occur between mages obtaned by smooth camera motons. Fgure (a, b) shows examples of smlar moton flows for pure translaton and pure rotaton motons obtaned by a (c) (d) Fgure. Moton flow for planar perspectve and omndrectonal mages. ranslaton moton parallel to a scene wth a perspectve camera Rotaton moton around the upaxs wth a perspectve camera (c) ranslaton moton parallel to a scene wth an omndrectonal camera (d) Rotaton moton around the upaxs wth an omndrectonal camera planar projecton. he motons of Fgure are produced by a small translaton moton parallel to the mage plane and by a small rotaton moton around the axs perpendcular to the optcal axs. Recently developed omndrectonal cameras mage 36s horzontally and at least 9s vertcally. he omndrectonal mages contan global nformaton so moton estmaton technques can overcome the lmted felds of vew ntroduced by planar projectons. Omndrectonal mages easly dstngush between small translatons and rotatons, as llustrated n Fgure (c, d). Researchers have developed methods for estmatng motons usng the omndrectonal cameras. Gluckman and Nayar showed three algorthms, each developed for planar perspectve cameras, appled to projectons from omndrectonal mages [6]. Svoboda appled the 8pont algorthm [7] [8] to omndrectonal mages []. Many have observed that omndrectonal vson overcomes

problems assocated wth tradtonal perspectve cameras, often producng more accurate results. he Kalman flter s used for several 5DOF ( of freedom) moton estmaton methods usng mages from perspectve cameras. Gennery [5], Dckmanns [4], and Azarbayejan [] used Extended Kalman flters to recover the shape and moton parameters from mage sequences. Soatto developed the moton estmaton methods based on essental and subspace constrants usng the Implct Extended Kalman flter [9]. hey demonstrated that moton estmates were mproved by applyng Kalman flters to streams of mages. We develop a moton estmaton technque for omndrectonal mages. he technque estmates motons ncrementally usng an Implct Extended Kalman Flter (IEKF). Each ndvdual feature provdes partal nformaton about the camera moton. We ncrementally mprove moton estmates wth each feature as n the SCAA method [5], however the SCA method was developed for calbrated features and full 6DOFpose trackng. Our technque estmates 5DOF translaton and rotaton motons concurrently from uncalbrated features, based on the rgdty and the depth ndependent constrants orgnally ntroduced by Bruss and Horn [2] and derved for the sphercal moton feld by Gluckman and Nayar [6]. he man dfferences of our technques from prevously mentoned approaches are a combnaton of the recursve estmaton framework adapted from the Implct Extended Kalman Flter and the constrants used n moton estmatons. here are several advantages of our approach over prevously mentoned technques. Snce the flter ncrementally mproves the estmaton wth each ndvdual feature, t generates estmates more frequently and wth lower latency than other approaches. It also does not requre any specal case processng when there are not enough features to fully constran the moton estmate. Any number of features tracked between two consecutve mages wll refne the moton estmate to the possble. If feature motons are corrupted by ndependent nose, then ncorporatng them ndependently can offer mproved flterng over technques that use all tracked features concurrently. We can also reuse the same set of features to mprove estmates. hs property s desrable snce computng correspondences between two consecutve mages s often the major tmeconsumng part of a moton estmaton process. A major problem wth moton estmaton technques s that translaton drecton estmates are often unrelable. It s well known that translaton estmates can be mproved usng large dsplacements. However, most researchers have not been consdered mxtures of large dsplacements and rotatons. hs class of motons s referred to as large moton n ths paper for convenence. Note that large moton estmaton s rarely dscussed n the context of planar projecton cameras snce correspondences are easly lost under large motons. Also, we note that optcal flow methods are dffcult to apply because of the extensve mage dsplacements and parallax effects that occur wth large motons. Experence and reason show that omndrectonal magery allows 2Dfeature correspondence trackng over large motons due to the global vew of the envronment. Svoboda also mentons n hs thess that large translaton motons are estmated better usng omndrectonal rather than perspectve cameras []. Whle our estmaton method works well for small motons, problems arose when t was drectly appled to omndrectonal mages taken from a camera undergong large motons. Whle most rotaton estmates were accurate, the translaton estmates were unrelable. Poor translaton estmates were obtaned when large rotatons were present. Imagefeature motons caused by translatons are relatvely small compared to the mage motons caused by rotatons. We observed sgnfcant translaton drecton errors n the presence of large rotatons. o solve ths problem we employ a Recursve Rotaton Factorzaton (RRF) wth our smooth moton algorthm. he resultng large moton algorthm can generate accurate and robust estmates of camera moton between any two panoramc mages for whch correspondences are known. 5DOF MOION ESIMAION Algorthm An omndrectonal camera projects a 36 vew of the world to a sngle 2D mage. 3D world ponts can be mapped to ponts on the unt sphere, creatng a sphercal projecton. he mappng functon depends on the mrror and optcs used n the omndrectonal camera. Examples can be found n [6]. he pont P n the 3D world s represented as the pont p on the unt sphere usng the sphercal projecton p = P P. he moton of a 3D pont P relatve to a movng camera can be represented as the nstantaneous velocty of P translatng along an axs and a rotaton about axs Ω. P & = Ω P he moton feld equaton s derved usng the dervatve of the sphercal projecton (Eq. ) wth respect to tme and substtutng t nto Equaton 2. he derved velocty vector equaton at pont p s U P () ( p) = (( p) p ) Ω p. (3) (2)

Gluckman and Nayar remove a depth from the velocty vector and derve the depth ndependent constrant [6] ( p ( U + ( Ω ))) =. (4) p We estmate translaton and rotaton motons concurrently from a set of moton vectors U measured at ponts p. Measurements of both U and p are assumed to contan some error, whch we model as Gaussan, whte and zeromean nose. q = p + n q = U + n ranslaton velocty s represented as α=[θ,φ], a pont on the unt sphere: θ s the azmuth angle, and φ s the elevaton. Axs Ω s represented as three parameters, Ω x, Ω y, and Ω z. We derve a nonlnear mplct dynamc system n Equaton 7, based on the depth ndependent constrant and nose measurements. and Ω are unknown parameters n the above system, whch s subject to three constrants. Frst, (α) s the pont on the unt sphere. Second the parameters and Ω have to evolve n such a way that measurements U satsfy the "aposteror" dynamcs, ( ) D( p, U, Ω) α q = p + n = ( α ) D( q, q, Ω) n (8) = where ñ s a resdual nose from the measurement nose n. he last constrant s called "apror" dynamcs. and Ω are not change arbtrarly. hey have to follow the model descrbed by the applcaton. We use very smple statstcal model, the frst order random walk descrbed n Equaton 9 because we lack a mechancal model. f α f Ω ( α ) ( Ω) = α = Ω We derve a dscrete dynamc model for the unknown parameters by applyng the constrants n Equaton. α Ω G We use Equaton to derve equatons for the Extended Kalman Flter (EKF). he only dfference s the measurement equaton, whch s mplct n our approach. he equatons used for estmatng general motons usng the frst order random walk model follow. ~, ( t + ) = fα ( α( t) ) + nα ( t) ( t + ) = f ( Ω( t) ) + n ( t) Ω (9) ( α, q, q, Ω) = ( α ) D( q, q, Ω) n N (, Σ ), n Ω (5) (6) (7) () = n~ me update ("predcton") step where the state vector X =[α,ω]. Measurement update ("correcton") step where X P t X P A t ( t + t) = X ( t t) X ( ) = X ( + t) = P( t t) + Σ P( ) = P αω ( t + t + ) = X ( t + t) + L( t + ) G( α ( t + t), q ( t + ), q ( t + ), Ω( t + t) ( t + t + ) = B( t + ) P( t + t) B ( t + ) + L( t + ) D( t + ) R( t + ) D ( t + ) L ( t + ) ( + ) = C( t + ) P( t + t) C ( t + ) + D( t + ) R( t + ) D ( t + ) ( t + ) = P( t + t) C ( t + ) A ( t + ) ( + ) = I L( t + ) C( t + ) G ( t + ) = L B t C D t ( + ) α G = [ q, q ] and R s the covarance matrx of the measurement error. We estmate motons usng IEKF descrbed above. Most methods usng Kalman Flters collect all features found from mages at tme t and t+ before estmatng the moton. Our method, on the other hand, ncrementally estmates the moton from each ndvdual feature, as descrbed n the SCA method [5]. he SCA method was developed to use wth calbrated features measured at varyng tmes to compute a full 6DOF camerapose. Our technque estmates 5DOF translaton and rotaton motons concurrently from uncalbrated features that are observed n snapshot of tme. In both cases ndvdual features provde ncomplete nformaton about the camera moton. Our approach has some advantages over flter methods that use all detected features at the same tme. We generate estmates more frequently and wth lower latency than methods requrng all feature correspondences before updatng the estmate. We need no specal cases for varyng numbers of corresponded features. Motons are estmated for any number, ncludng one, of tracked features between two mages. Our method never fals for lack of constrants; ts accuracy degrades gracefully. If feature motons are corrupted by ndependent nose, then ncorporatng them ndependently can offer mproved flterng over other technques that use all tracked features concurrently.

Intal estmate for N IEKF X k+ & P k+ k = n? X n+ Indvdu al P k Intal estmate for IEKF X n+k+ & P k+ k = n? X 2n+ Although, each pont can be processed ndependently, our algorthm can concurrently apply any number of features. For example, mproved estmates are obtaned under some condtons by processng fve features concurrently. Moton estmates are usually smlar whether we use N moton updates wth one feature at a tme or one update wth N features concurrently f all features contan smlar nose. he dfference s the weghtng created by the covarance matrx P. he values of P usually decrease as the number of updates ncrease. If only a few features contan relatvely large nose (.e., 2D trackng outlers), we can mprove estmates by usng N features at the same tme. One drawback of usng multple features s ncreased executon tme. N updates usng one feature at a tme requres N x (67 multplcatons + 549 addtons). However, one update usng N features requres (5N 2 + 56N + 5) multplcatons + (4 N 2 + 35N + f k P k Intal estmate for IEKF f k X nm+k+ & P k+ k = n? DONE Fnal Fgure 2. 5DOF moton estmaton. X and P ndcate the state estmate and the error covarance respectvely. f s one feature from the set wth n features. m s the number of teratons used for the same set of features 8 6 4 2 5 5 5 5 5 2.5 2.5.5.5 5 5 (c) Fgure 3. he moton measured for each feature mproves the camera moton estmaton. angular error n the translaton drecton angular error n rotaton axes (c) angle dfference of rotaton angles 4) addtons. Most moton estmaton methods use all tracked features from two consecutve mages once. We can use the same set of features more than once to mprove estmates, as llustrated n Fgure 2. Each estmate depends on the pror state and the measurement qualty of an ndvdual feature. We mprove the moton estmate from the ntal state by usng all tracked features separately for the frst teraton. he result of the frst teraton s used as the ntal state for the second teraton where we agan use the same set of features. In processng each feature, only the ntal state dffers from the prevous teraton. Iteratng wth the same set of features produces mproved fnal moton estmates. Iteratons contnue untl the estmate converges. he number of teratons needed depends on the number of avalable tracked features. We typcally terate fve tmes wth 24 tracked features from two consecutve smoothmoton mages. hs teraton capablty s very mportant snce 2Dfeature trackng s one of the more computatonally expensve processes n moton estmaton systems. We can also generate vald estmates wth fewer tracked features usng the teraton procedure. For example, the estmate generated by two teratons of 5 features s very smlar to the estmate generated wth one teraton of features. Smulaton Results Smulaton results show the method s robust and adaptable to varyng nput data. Snce we use a Kalman flter, we need an ntal estmate. An ntal guess may be far from the correct moton. he ntal errors of translaton and rotatons are 9 or 45 s dependng on smulatons. Rotaton estmates are modeled by a rotaton axs and a rotaton angle about that axs. ranslaton estmate errors are shown as the angle between correct and estmated translaton drectons. he rotaton errors are shown as the angle between correct and estmated rotaton axes and the dfference between correct and estmated rotaton angles. When we estmate motons for streams of mages, we compare between the correct and estmated absolute camera orentaton for each mage. he absolute orentaton estmates are computed by ntegratng the rotaton estmates between all pror mages as well as the current mage par. he smulatons usng streams of 5 5 5 2 Num ber offram es De gre e 2 3 5 5 Num ber offram es Fgure 4. Moton estmates for general camera moton. angular error n translaton drectons angle dfference of rotaton angles.

mages show the accumulaton errors. Our smulatons are based on features dstrbuted n a 4m square room. For the frst smulaton, shown n Fgure 3, we show how each successve feature contrbutes to the overall estmate. hs smulaton also shows the method reusng the same features more than once to mprove estmates. features are generated between two consecutve mages and used twce to estmate motons. he appled moton s a 2 cm translaton and a 3 rotaton. Gaussan whte nose wth.3 standard devaton s added to the projected feature postons. For the second smulaton, we tracked 4 features between consecutve frames. he moton of the smulated camera s a 2cm/frame translaton and a 3s maxmum snusodal rotaton about the upaxs. We start the estmaton from a zero ntal estmate, zero rotaton angles, and about 9 s of error for each of the translaton drecton and rotaton axs. Gaussan whte nose wth.3 standard devaton s added to smulate trackng errors. he results are presented n Fgure 4 where we observe that only a few frames are needed to converge to an accurate moton estmate. he thrd smulaton shows that we can stll estmate motons even wth only one tracked feature between consecutve frames, under the assumpton that the moton s constant over the entre sequence. A sngle tracked feature s randomly selected for each mage par. he appled moton s a pure translaton of 2cm/frame. We added Gaussan whte nose wth.5 standard devaton. he ntal condton s almost 45s apart from the correct translaton drecton. Fgure 5 shows that the estmate slowly converges to the correct value. Clearly varyng motons cannot be estmated wth only one tracked feature snce t s severely underconstraned. However, 5 even a sngle tracked feature does provde useful mprovement n our approach. he last smulaton shows that our algorthm estmates rotatons correctly even wth very small translatons. Some methods estmate translatons frst and then rotatons. hese often produce errors for near pure rotatons snce the translaton estmates are error prone. he smulaton n Fgure 6 shows that ths case does not cause problems n our method even though we use the depth ndependent constrant, whch s strongly dependent on the translaton drecton. he moton for ths smulaton s a 3 rotaton about the upaxs of a camera wth a very small translaton. All of the presented results demonstrate the robust, stable, and flexble behavors of our moton estmaton algorthm. LARGE MOION ESIMAION Yes More teraton No Factorng out the orentaton. Orentaton estmate Intal Image Modfed mage Rotaton estmate DONE Fnal estmate 5DOF Moton estmaton Fxed Image ranslaton estmate Fgure 7. Large moton estmaton procedure 2 4 6 8 2 4 6 Num ber offram es (one feature for each fram e) Fgure 5. Wth only one tracked feature, useful moton estmates are stll obtaned. 5 5 5.3.8.3.2 5 5 Fgure 6. Moton estmaton of near pure rotaton. angular error n rotaton axes and angle dfference of rotaton angles Motvaton Moton estmaton technques all suffer from naccurate translaton estmates. As the scenetocamera dstance ncreases, the translaton estmates decrease n accuracy because the moton vectors caused by translatons shrnk. hese translaton naccuraces make moton estmaton technques dffcult to use n practcal systems. We know that translaton estmates are mproved as the lengths of moton vectors ncrease. We can obtan large moton vectors from large translaton motons and ths s one motvaton for nvestgatng effort nto large moton estmaton. Wthout any modfcatons, we appled our 5DOF moton estmaton algorthm to large motons. Relatvely pure translaton estmates produced accurate results. However, translatons mxed wth large rotatons, e.g. 9

s, produced sgnfcantly more error. For these moregeneral large moton cases, the magefeature motons due to translaton are short compared to the feature motons due to rotatons. herefore, we developed a largemoton estmaton algorthm based on our IEKF framework and a Recursve Rotaton Factorzaton (RRF). RRF LargeMoton Algorthm he RRF algorthm s smlar to the IEKF algorthm presented at the prevous sectons. We add an addtonal step to the teratons to acheve accurate translaton estmates for general mxtures of large translaton and rotaton motons. As stated prevously, large translaton motons are estmated accurately n the absence of rotaton. General motons contanng rotatons can produce accurate translaton estmates f we remove the feature motons due to rotaton. For sphercal projectons, feature motons due to rotaton are easly modeled and subtracted from the feature moton vectors. We model general large motons as R, and compute the nverse rotaton R and apply t to the projecton pror to estmatng the resdual (translaton) moton =RR. We recursvely estmate R to remove the rotaton moton between two omndrectonal mages, leavng only the translaton moton to estmate. Fgure 7 shows a flowchart of the RRF algorthm for large moton estmaton. he number of teraton requred for the RRF procedure depends on the rotaton magntude. Larger angles usually requre more teratons. Experments show that between 5 and 8 teratons may be requred to completely remove rotatons. (See Fgures 9 and 2 n the followng secton for examples.) We adaptvely determne the number of teratons by comparng the convergence rate to a threshold. As the number of teraton ncreases the RRF technque requres more executon tme. When tested on mage streams, the executon tme for each estmate can vary greatly, whch s not desrable. However, n Fgure 2 we show an example that solves ths problem by usng the prevous frame orentaton estmate as the ntal value of the current frame orentaton. Smulaton Results We present fve dfferent smulatons to show robustness and flexblty of the RRF method over dfferent condtons. Ffty 3D ponts are generated randomly n the envronment, a room of sze 4m 4m 3m. A subset of features s corresponded n two mages for large moton estmaton. hs number ranges between 5 to, dependng on the dstance between the two camera postons. We add Gaussan whte nose wth.5 to 3. standard devatons. For most smulatons we show results for rotaton angles up to 8 s to demonstrate the effectveness of the RRF approach even for extreme condtons. he frst smulaton shows two factors presented n earler sectons. We estmate large motons wth two dfferent dsplacements mxed wth a fxed rotaton of 8 s (Fgure 8). Gaussan whte nose wth.5 standard devaton s added to tracked features. Fgure 8 shows that both cases converge to the correct motons. Larger dsplacements converge faster than smaller dsplacements. he errors n the ntal teratons show how translaton estmates are nfluenced by resdual rotatons pror to ther complete removal. 5 3 2 5 Num ber ofteratons 5 2 Num ber of teratons 5 D egrees 2 Num ber ofteratons 2 6 4 Num ber ofteratons 5 Num ber of teratons 5 Num ber ofteratons Fgure 8. Moton estmaton wth varous translaton dsplacements wth 8 rotaton angle. Charts on the left column descrbe the angular error n translaton drectons, and charts on the rght column descrbe the angle dfference of rotaton angles. m 2 m 5 Num ber of teratons 8 4 4 8 Num ber ofteratons Fgure 9. Moton estmates wth varous rotaton angles wth the m translaton dsplacement. Charts on the left column descrbe the angular error n translaton drectons, and charts on the rght column descrbe the angle dfference of rotaton angles 45 9 s

Fgure 8 and Fgure 9 show how vared rotaton angles affect the translaton estmates. he largest rotaton of 8 s shown n Fgure 8. Rotaton angles n Fgure 9 are 45 and 9s. Gaussan whte nose wth.5 standard devaton s added to tracked features. he convergence rates are dfferent for each angle, but larger rotaton angles do not always converge more slowly than smaller angles. he thrd smulaton shows estmates wth varous nose levels. he same moton, a meter dsplacement wth a 8 rotaton, s appled wth two dfferent nose levels set to.3 and.6. Fgure shows that accuracy depends on the nose level, but the number of teratons needed to converge s largely unaffected. wo moton estmaton methods are compared n the fourth smulaton to demonstrate that the method has advantages over exstng omndrectonal moton estmaton methods. Optcal flow methods are dffcult to apply because of extensve mage dsplacements produced by large motons. Svobodas 8pont method s sutable for large moton estmatons but he dd not consder them. herefore, we mplemented an 8pont method based on Svobodas paper. Both our method and the 8pont method estmate pure translaton (m) and pure rotaton ( s along the upaxs) motons accurately. Rotatons of large motons (m dsplacement and ~ 9 rotaton along the upaxs) are estmated accurately by both methods. However, our method estmates translaton drectons more accurately n these cases. he accuracy mprovement gets even larger as nose ncreases as shown n Fgure. he standard devatons of the 8pont 5 5 Num ber ofteratons Num ber of teratons 6 3 3 6 9 5 5 5 Num ber ofteratons 5 Num ber ofteratons Fgure. Moton estmate wth varous nose levels ( meter dsplacement and 8 rotaton angle). Charts on the left column descrbe the angular error n translaton drectons, and charts on the rght column represent angle dfference of rotaton angles..3.6 nose method translaton estmates are bgger than our average translaton errors. he last example s dfferent from prevous cases. hs smulaton estmates motons over a sequence of mages. he camera moton s shown n Fgure 2 (d). A ndcates the start poston of the movng camera, and B ndcates a fxed mage from whch the large moton s estmated. he camera moves along the path shown n Fgure 2 (d), and varous rotatons are appled along the path, ncludng a snusod rotaton of 5 maxmum about the upaxs. In ths case, the pror rotaton estmate s used to ntalze the current estmaton. hs sgnfcantly reduces the number of teratons requred for convergence. We also ran ths smulaton wthout usng prevous rotaton estmates, and smlar estmates were obtaned. he man dfference s the number of teratons requred. By usng pror rotatons, only fve or fewer teratons are needed. Wthout usng pror rotatons, to 8 teratons are needed (Fgure 2c). CONCLUSION Research n moton estmaton manly targets small (smooth) motons because of the practcal lmtatons of planarprojecton magery. Recent developments n omndrectonal magng systems provde practcal means E rro r n E rror n E rror n 2.5 2.5.5 3 5 7 9 5 5 Rotaton angles n 3 5 7 9 4 3 2 Rotaton angles n 3 5 7 9 Rotaton angles n E rro r n E rror n E rror n.6.4.2 3 2 3 5 7 9 Rotaton angles n 3 5 7 9 9 6 3 Rotaton angles n 3 5 7 9 Rotaton angles n (c) Fgure. Moton estmaton wth varous nose levels ( m dsplacement and ~ 9 rotaton along the upaxs). Left and rght sde charts descrbe translaton and rotaton error respectvely..3.5 (c) 3. nose

Num ber ofteratons 5 2 9 6 3 5 2 9 6 3 9 6 3 5 5 Num ber of fram es 5 5 Num ber offram es 5 5.4 of vewng an entre envronment and therefore trackng features between wdely spaced cameras. hs drves the need for estmatng large camera motons. Svoboda consdered motons wth large dsplacement and no rotaton n hs thess. In ths paper, we consder general large motons composed of large dsplacements and rotatons. he smulaton results show that our large moton estmaton method generates accurate translaton and rotaton estmates for a wde range of condtons. 5 5 Number offram es.4.8.2.6.4.8.2.6 Num ber offram es 5 5.4 B A Num ber of fram es (c) (d) Fgure 2. Moton estmaton over the streams of mages. Left column of and descrbe the angular error n translaton drectons and rght column descrbe the angle dfference of rotaton angles. Use pror estmaton Adaptve moton estmaton (c) he number of teraton used for an adaptve moton estmaton for each frame (d) he moton path. A and B are two startng postons and A moves along the path. A and B are located on (m, m, ) and (m, m, ) respectvely. O s the center of the space, (,, ). O [3] K. Danlds and H.H. Nagel, he couplng of rotaton and translaton n moton estmaton of planar surfaces, n IEEE Conf. on Computer Vson and Pattern Recognton, New York, NY, June 993, pp. 8893. [4] E. D. Dckmanns and V. Graefe, "Dynamc monocular machne vson," Machne Vson and Applcatons,, 988, pp. 22324. [5] D. B. Gennery, "rackng known 3dmensonal object," Proc. of AAAI 2 nd Natonal Conference Artfcal Intellgence, Pttsburg, 982, pp. 37. [6] J. M. Gluckman and S. K. Nayar, "Egomoton wth omncameras," n Proc. of IEEE Internatonal Conference on Computer Vson, ICCV 98, Bombay, Inda. [7] R. I. Hartley, In defense of the 8pont algorthm, n 5 th Int. Conf. on Computer Vson, MI Cambrdge Massachusetts. 995, pp. 647. [8]. S. Huang and A. N. Netraval, Moton and structure from feature correspondences: a revew, Proc. of the IEEE, vol. 82, No. 2, Feb. 994, pp. 268. [9] S. Soatto and P. Perona, "Recursve 3D moton estmaton usng subspace constrants," Int. journal of Computer Vson, 997. []. Svoboda,. Pajdla and Vaclav Hlavac, "Moton Estmaton Usng Central Panoramc Cameras," n IEEE Conf. on Intellgent Vehcles, Stuttgart, Germany, October 998. []. Svoboda, "Central Panoramc Cameras Desgn, Geometry, Egomoton," Ph.D. hess, Czech echncal Unversty, ftp://cmp.felk.cvut.cz/pub/cmp/artcles/svoboda/ phdthess.ps.gz. [2] R. Y. sa and. S. Huang, Unqueness and estmaton of threedmensonal moton parameters of rgd objects wth curved surfaces, IEEE ransactons on Pattern Analyss and Machne Intellgence, vol. PAMI6, No., Jan. 984, pp. 327. [3] J. Weng,. S. Huang and N. Ahuja, Moton and structure from two perspectve vews: algorthms, error analyss, and error estmaton, IEEE ransactons on Pattern Analyss and Machne Intellgence, vol., No. 5, May 989, pp. 45476. [4] B. L. Yen and. S. Huang, Determnng 3D moton and structure of a rgd body usng the sphercal projecton, n Computer Vson, Graphcs, and Image Processng, vol. 2, 983, pp. 232. [5] Greg Welch and Gary Bshop, "SCAA: Incremental rackng wth Incomplete Informaton," SIGGRAPH 97, August 997. REFERENCE [] A. Azarbayejan, B. Horowtz, and A. Pentland, "Recursve Estmaton of Structure and Moton usng Relatve Orentaton Constrants." IEEE Conf. on Computer Vson and Pattern Recognton, Los Alamtos, CA, June 993, pp. 294299. [2] A. Bruss and B.K.P. Horn, Passve navgaton, Computer Vson, Graphcs, and Image Processng, vol. 2, 983, pp. 32.