Stereo SLAM, PhD migliore@elet.polimi.it Department of Electronics and Information, Politecnico di Milano, Italy
What is a Stereo Camera? Slide n 2 Do you remember the pin-hole camera?
What is a Stereo Camera? Slide n 3 Two cameras that perceive the world - Each camera has a P matrix
What is a Stereo Camera? Slide n 4 Two cameras that perceive the world
What is a Stereo Camera? Slide n 5 Two cameras that perceive the world
What is a Stereo Camera? Slide n 6 Two cameras that perceive the world
What is a Stereo Camera? Slide n 7 Error modeling problem
Stereo SLAM (Paz et al. 2008) Slide n 8 The idea - Use the Unified Inverse Depth parametrization (Montiel et al. 2006) - Rectify images and initialize the point using
Stereo SLAM (Paz et al. 2008) Slide n 9 Measurement Equations
Stereo SLAM (Paz et al. 2008) Slide n 10 Measurement Equations
Stereo SLAM (Paz et al. 2008) Slide n 11
Classic EKF SLAM Slide n 12 PhD - migliore@elet.polimi.it
Classic EKF SLAM Slide n 12 Extended Kalman Filter Video Frame PhD - migliore@elet.polimi.it
Classic EKF SLAM Slide n 12 Extended Kalman Filter FD Feature Detection Feature Initialization Prediction Video Frame Update SLAM Filter PhD - migliore@elet.polimi.it
Classic EKF SLAM Slide n 12 Extended Kalman Filter FD Feature Detection Feature Initialization Prediction Video Frame Update SLAM Filter PhD - migliore@elet.polimi.it
Classic EKF SLAM Slide n 12 Extended Kalman Filter FD Feature Detection Feature Initialization Prediction Data Association Video Frame DA Update SLAM Filter PhD - migliore@elet.polimi.it
Classic EKF SLAM Slide n 12 Extended Kalman Filter FD Feature Detection Feature Initialization Prediction Data Association Video Frame DA Update SLAM Filter PhD - migliore@elet.polimi.it
Stereo SLAM (Paz et al. 2008) Slide n 13 Data Association Trouble
Stereo SLAM (Paz et al. 2008) Slide n 14 Data Association Trouble
Stereo SLAM (Paz et al. 2008) Slide n 15 Data Association Trouble
Compatibility Slide n 16
NN Data Association Slide n 17
NN Data Association Slide n 18
Joint Compatibility Slide n 19
JCBB Slide n 20
JCBB Slide n 21
Demo Time Slide n 22 Switch on Matlab
Stereo SLAM (Paz et al. 2008) Slide n 23 Joint Compatibility Branch & Bound Results
Stereo SLAM (Paz et al. 2008) Slide n 24 Results
Scaling problem Slide n 25
Scaling problem Slide n 26 O(n 2 )
Solution: local maps Slide n 27 Switch to matlab again
Stereo SLAM (Paz et al. 2008) Slide n 28 Results
Stereo SLAM (Paz et al. 2008) Slide n 29 Results
Stereo SLAM (Tomono 2009) Slide n 30 Results
Stereo SLAM (Tomono 2009) Slide n 30 Results
Stereo SLAM (Tomono 2009) Slide n 31 Results
Stereo SLAM (Tomono 2009) Slide n 32 Results
Inverse Scaling? Slide n 33 Is it possible to use the inverse scaling? Yes Results? Coming soon!!
Thanks for your attention Slide n 34 PhD - migliore@elet.polimi.it
Thanks for your attention Slide n 34 Questions PhD - migliore@elet.polimi.it
Omnidirectional SLAM, PhD migliore@elet.polimi.it Department of Electronics and Information, Politecnico di Milano, Italy
What is an Omni Camera? Slide n Omnidirectional sensors come in many varieties, but by definition must have a wide field-of-view. ~180º FOV ~360º FOV >180º FOV wide FOV dioptric cameras (e.g. fisheye) catadioptric cameras (e.g. cameras and mirror systems) polydioptric cameras (e.g. multiple overlapping cameras)
(Poly-)Dioptric solutions Slide n One to two fish-eye cameras or many synchornized cameras Pros: - High resolution per viewing angle Cons: - Bandwidth - Multiple cameras
(Poly-)Dioptric solutions Slide n One to two fish-eye cameras or many synchornized cameras Homebrewed polydioptric cameras are cheaper, but require calibrating and synchronizing; commercial designs tend to be expensive
Catadioptric solutions Slide n Usually single camera combined with convex mirror Pros: - Single image Cons: - Blind spots - Low resolution
Camera Models Slide n 40 Perspective camera Image plane (CCD) Single effective viewpoint
Camera Models Slide n 40 Perspective camera Image plane (CCD) Single effective viewpoint
Camera Models Slide n 40 Perspective camera Image plane (CCD) Single effective viewpoint
Camera Models Slide n 40 Perspective camera Image plane (CCD) Single effective viewpoint
Camera Models Slide n Catadioptric cameras
Camera Models Slide n Catadioptric cameras mirror
Camera Models Slide n Catadioptric cameras mirror perspective camera
Camera Models Slide n Catadioptric cameras mirror perspective camera
Camera Models Slide n Catadioptric cameras mirror perspective camera
Camera Models Slide n Catadioptric cameras mirror perspective camera
Camera Models Slide n Catadioptric cameras mirror perspective camera
Camera Models Slide n Central catadioptric cameras mirror camera
Camera Models Slide n Central catadioptric cameras mirror camera single effective viewpoint
Camera Models Slide n Central catadioptric cameras mirror (surface of revolution of a conic) camera single effective viewpoint
Types of central catadioptric cameras Slide n 43 F1 F2
Types of central catadioptric cameras Slide n 43 hyperbola + perspective camera F1 F2
Types of central catadioptric cameras Slide n 43 hyperbola + perspective camera parabola + orthographic lens F1 F1 F2
Types of central catadioptric cameras Slide n 43 hyperbola + perspective camera parabola + orthographic lens F1 F1 F2
Types of central catadioptric cameras Slide n 43 hyperbola + perspective camera parabola + orthographic lens F1 F1 F2
Types of central catadioptric cameras Slide n 43 hyperbola + perspective camera parabola + orthographic lens... F1 F1 F2
Other types of central cameras Slide n 44
Other types of central cameras Slide n 44
Why do we need calibration? Slide n 45 Z Y X p = v u
Why do we need calibration? Slide n 45 Calibration gives the relation between 2D & 3D Z For each pixel 3D vector emanating from the single viewpoint X Y p = u v
Why do we need calibration? Slide n 45 Calibration gives the relation between 2D & 3D Z For each pixel 3D vector emanating from the single viewpoint X Y p = u v
Why do we need calibration? Slide n 45 Calibration gives the relation between 2D & 3D Z For each pixel 3D vector emanating from the single viewpoint X Y p = u v
Why do we need calibration? Slide n 45 Calibration gives the relation between 2D & 3D Z For each pixel 3D vector emanating from the single viewpoint X Y p = u v
Why do we need calibration? Slide n 45 Calibration gives the relation between 2D & 3D Z For each pixel 3D vector emanating from the single viewpoint X Y p = u v
What? Slide n Z X Y u v
What? Slide n Center of the omnidirectional image Z X Y u v
What? Slide n Center of the omnidirectional image Camera focal length Z X Y v u Focal length
What? Slide n Center of the omnidirectional image Camera focal length Orientation and position between camera & mirror Z X Y R, T Focal length u v
What? Slide n Center of the omnidirectional image Camera focal length Orientation and position between camera & mirror Mirror shape Z X Y R, T Focal length u v
Assumptions Slide n Z X Y R, T u v Focal length
Assumptions Slide n 1. Mirror and camera axes are aligned => Z X Y R, T u v Focal length
Assumptions Slide n 1. Mirror and camera axes are aligned => Z X Y R, T u v Focal length
Assumptions Slide n 1. Mirror and camera axes are aligned => Z X Y 2. x-y mirror axes coincide with u-v camera axes => R, T u v Focal length
And how about non-central cameras? Slide n Reflected rays do not intersect in a point but are tangent to a caustic
And how about non-central cameras? Slide n Reflected rays do not intersect in a point but are tangent to a caustic
Visual Odometry (Scaramuzza et al. 2009) Slide n 49
Omni SFM (Lhuillier et al. 2008) Slide n 50
Omni SFM (Lhuillier et al. 2008) Slide n 51
Omni SFM (Lhuillier et al. 2008) Slide n 52
Thanks for your attention Slide n 53 PhD - migliore@elet.polimi.it
Thanks for your attention Slide n 53 Questions PhD - migliore@elet.polimi.it