Monocular Visual Odometry

Transcription

1 Elective in Robotics coordinator: Prof. Giuseppe Oriolo Monocular Visual Odometry (slides prepared by Luca Ricci)

2 Monocular vs. Stereo: eamples from Nature Predator Predators eyes face forward. The field of vision for each eye overlaps in the front to create binocular vision (stereo vision). Prey A prey animal has eyes that face sideways. Only a small area of overlap between the field of vision for each eye (monocular vision).

3 Monocular vs. Stereo: eamples from Nature Monocular vision salient features: Increased field of view (FOV) Limited depth perception Type of vision FOV (field of vision) Monocular vision Wide does not overlap Stereoscopic vision Narrow overlapping FOV by right and left eyes Estimation of distances Cannot estimate distances accurately Can estimate distances accurately

4 Monocular vs. Stereo: eamples from Nature Monocular vision salient features: Increased field of view (FOV) Limited depth perception The reasons why i won t ask you to close an eye Accomodation Familiar size Occlusion Motion paralla Teture gradient Fish-eye effect Perspective Aerial perspective

5 Monocular vs. Stereo: recovering depth In Nature Pigeons have monocular vision rather than binocular vision. Bobbing their heads generates paralla motion for depth perception. In Geometry There eists a precise geometric relation between the projections of the same point in different views In Computer Vision Parallel Tracking and Mapping (PTAM) an effective implementation of a monocular visual SLAM algorithm

6 In Geometry: Epipolar Geometry ( R T X X ) X RX T or R T

7 Frontal pinhole In Geometry: Pin hole Camera Model

8 In Geometry: The Epipolar Constraint Algebraic elimination of depth.. R T Tˆ TR ˆ Epipolar constraint T TR ˆ Essential matri 0 T Epipolar geometry entities E TR ˆ (O O X) Epipolar plane l l Epipolar lines O O Baseline e e Epipoles

9 In Geometry: The Essential Matri. Collect constraints from all points Essential matri salient features: A special 3 3 matri Apparently 8 dof (9 matri elements up to scale) Practically 5 dof (3 rotation translation up to scale) 3 (3) ˆ T SO R TR E How to estimate an essential matri from a pair of views (8 point algorithm 5 point algorithm) Need paralla motion T = 0 won t work!. Rewrite 0 S E T S e e e e e e e e e E ] [ T a ] [ T n a a ]... [

10 In Geometry: The Essential Matri A special 3 3 matri Essential matri salient features: TR ˆ (3) T Apparently 8 dof (9 matri elements up to scale) Practically 5 dof (3 rotation translation up to scale) Therefore E R SO How to estimate an essential matri from a pair of views (8 point algorithm 5 point algorithm) 3 Set of feature correspondences Essential matri estimation Camera pose

11 In Computer Vision: Monocular Visual Odometry vs Monocular Visual SLAM Monocular Visual Odometry Feature matching between image frame: Monocular Visual SLAM Feature matching between current image frames and a live map: Faster: works in costant time. Accumulated small errors will cause drifts. Cannot mantain a consistent scale from couples of frames (need 3 or more views). Motion singuralities (pure rotations do not constraint enough the motion). Slower: but accurate. Repeated observation of the same features ensures no drifts in trajectory estimate. Scale fied once set the map. Etra cost for epanding and maintaining the map. Method based on EKF are limited by the size of the map.

12 In Computer Vision: Parallel Tracking and Mapping (PTAM) Main features: Monocular visual SLAM algorithm Intended for small workspace AR (Augmented Reality) applications Mapping and Tracking are separated and run in two parallel threads Mapping is based on keyframes New points are initialized with an epipolar search No feature or map uncertainties model (bundle adjustment on a vast number of image features) Robust against partial camera occlusions (50 % of features available)

13 PTAM: what is a map A collection of M map points and N keyframes Map point : a 3D point in the world ( p j w = ( j w ; y j w ; z j w ; ) T ) Keyframe: a pyramid of greyscale 8bpp images (i.e ) and an associated camera-centred coordinate frame ( Ki )

14 PTAM: what is a map Map initialization:. Acquire the first keyframe

15 PTAM: what is a map Map initialization:. Acquire the first keyframe. Translate and rotate the camera while tracking the features (hyp. 0 cm translation)

16 PTAM: what is a map Map initialization:. Acquire the first keyframe. Translate and rotate the camera while tracking the features (hyp. 0 cm translation) 3. Acquire the second keyframe and build the map ( epipolar search )

17 PTAM: the tracker New camera frame real - time task! TRACKING THREAD Map available? TRUE Apply camera motion model Track the map Update camera motion model AssessTrackingQuality To Map Maker FALSE Get lost? TRUE Relocaliser FALSE Need New Keyframe? back to the top TRUE To Map Maker

18 PTAM: more about the tracker Map point search Fied - range image search around the point s predicted image location. Map point in original keyframe (first spotted). Fied range patch etracted from original image ( 8 8 ) 3. Affine warp of the patch based on motion model pose estimate 4. Projection on the current view based on the motion model pose estimate Eamples of affine warping 5. Confrontation with the current view (Sum of Squared Differences score)

19 PTAM: the map maker slow but accurate! MAPPING THREAD MapMaker::run( ) Add keyframe to the map Add map points via epipolar search TRUE New Keyframe? Locally converged? Globally converged? FALSE TRUE FALSE FALSE Local Bundle adjustment Set Global and Local convergence to FALSE TRUE Check map points and trash bad ones Global Bundle adjustment back to the top

20 PTAM: monocular SLAM on a quadrotor? Some basic facts: Hummingbird quadrotor equipped with a monocular camera facing downwards Multi-Robot-Integration Platform (MIP) implements quadrotor communication wireless camera sensor interface and some other useful stuff Passing camera frames from MIP to PTAM will do the trick... MIP WIRELESS CAMERA Camera frames PTAM MAPPING HUMMINGBIRD TRACKING Visual odometry

21 MIP: an overview A C++ software aimed to develop control and estimation robotics algorithms Good level of modularity Use of abstracted low-level interfaces Interface with 3D simulation environment (Player/Gazebo) MIP components Main Baselib main of the program. Here is created and launched the Scheduler basic library for general purpose and robotics functionalities e.g. IP communication pose class matri class Resources Algorithms Tasks class providing interface modules respect to the hardware or the MIP platform facilities e.g. camera quadrotor keyboard... class collection of robotics algorithms e.g. Visual odometry algorithm estimate (Kalman filtering) High level robot activities that must be eecute in parallel glueing algorithms and resources e.g. tracking deployment target navigation mutual localization entrapment eploration

22 MIP: an overview Configuration file Resource (options) Task (options) LOADER Resource quadrotor RESOURCES Resource M camera TASKS Task Run func. Ee. time Task N Run func. Ee. time SHEDULER Task run( ) Algorithm Visual SLAM Task run( ) Eecution cycle ALGORITHMS TaskN run( ) Algorithm K EKF

23 MIP: use with PTAM and quadrotor QUADROTOR (simulated) Sensor data (e.g. IMU) Camera frame CAMERA (simulated) Control CLOCHE KEYBOARD HIT ( pilot quadrotor via keyboard) UAV LOCALIZATION ( through PTAM) Camera frame Camera Pose VISUAL ODOMETRY (communicate with PTAM) Camera Pose Camera frame = TASKS = RESOURCES = ALGORITHMS PTAM

24 MIP: use with PTAM and quadrotor. Start 3D simulation environment (Player/Gazebo)

25 MIP: use with PTAM and quadrotor. Start 3D simulation environment (Player/Gazebo). Select the configuration file and run MIP

26 MIP: use with PTAM and quadrotor. Start 3D simulation environment (Player/Gazebo). Select the configuration file and run MIP 3. Start PTAM and navigate

27 References [ ] Georg Klein and David Murray Parallel Tracking and Mapping for Small AR Workspaces In Proc. International Symposium on Mied and Augmented Reality (ISMAR'07 Nara) 007 [ ] R. I. Hartley and A. Zisserman Multiple View Geometry in Computer Vision. Cambridge University Press second edition 004. [ 3 ] Y. Ma and S. Soatto A invitation to 3-D vision from images to geometric models Springer-Verlag New York 004. [ 4 ] J. Shi and C. Tomasi Good features to track In Proc. IEEE Intl. Conference on Computer Vision and Pattern Recognition (CVPR 94) pages IEEE Computer Society 994.