CSE 527: Introduction to Computer Vision

Transcription

1 CSE 527: Introduction to Computer Vision Week 10 Class 2: Visual Odometry November 2nd, 2017

2 Today Visual Odometry Intro Algorithm SLAM

3 Visual Odometry Input Output Images, Video Camera trajectory, motion path (preferably in 3D) [YouTube]

4 Ways to Measure Motion GPS Odometry (wheels) Doesn t work indoors Inaccurate (triangulation from satellites) IMU (Accelerometer, Gyroscope) 2nd derivative 6DOF Drift WiFi / Bluetooth beacons Expensive Instrumented environments Problem on slippage terrain Drift

5 Visual Odometry Cameras as Motion Sensor Cheap: size, price, power draw Ubiquitous More information than just motion Passive sensor Needs processing power Follows nature Works in GPSdenied environments (underwater, sky, on Mars!) Camera is a bearing sensor: Measure object angle w.r.t optical axis (i.e. you can navigate by the stars!) Depth (range) cameras Stereo, SL, ToF Make odometry even easier

6 Visual Odometry Basic idea: Estimate 6DOF transformation (R, t) incrementally Use only static world features Need be robust to moving objects Detect, Extract, Track P Image plane p Can achieve this in several ways: p1 2D2D: Monocular, E, F matrices 3D3D: Stereo, range cameras, LIDAR) 2D3D: Reconstruction (SfM) Camera optical center 2 O1 O2 [R t]

7 Visual Odometry Algorithm 1. Get image Ik 2. Compute correspondences between Ik 1 and Ik (either feature matching or tracking). 3. Find correct correspondences and compute essential matrix E We saw: #2 in Weeks 3 and 4 #3 in Weeks 8 and 9 #4 in Weeks 8 and 9 #5 in Weeks 9 and 10 (last time) But #6?? 4. Decompose E into Rk and tk 5. Compute 3D model (triangulate points X). 6. Rescale tk according to relative scale r 7. k = k + 1

8 [Lazbnik]

9 [Lazbnik]

10 [Lazbnik]

11 [Lazbnik]

12 [Lazbnik]

13 [Lazbnik]

14 [Lazbnik]

15 [Lazbnik]

16 Even if we have a good K matrix (and get the E matrix) we can still only get similarityambiguous reconstruction... [Lazbnik]

17 Scale Ambiguity xrexlt = 0 2xRExLT = xr2exlt = 0 E1 can have one scale, and E2 a different scale 5xRExLT = xr5exlt = 0 What can we do?... Hint: What can we get with E and 2D features in both images? E1 E2

18 [Scaramuzza]

19 Questions?

20 Visual Odometry What other way do we know to sequentially find Rk+1, tk+1? Known 3D points Assume we already have Rk,tk and a set of 3D points we triangulated from them. Rk tk???

21 Visual Odometry What other way do we know to sequentially find Rk+1, tk+1? Known 3D points Assume we already have Rk,tk and a set of 3D points we triangulated from them. We can use the PnP algorithm! Use proxy 2D3D correspondences: Match 2D features from P3 to P2 Get estimated 3D points for features in P2 Use 3D points with proxy 2D points by association Solve robust PnP as usual Rk tk???

22 PTAM [Klein, Murray, 2007] [YouTube]

23 Simultaneous Localization and Mapping SLAM is basically VO with a few extra tricks to make it more complete: Loop closing Maintaining the space map Maintaining a pose graph (motion path) Modeling uncertainty The boundary between SLAM and VO is vague. The terms are used interchangeably. VO is usually used directly for Odometry Measure distance, speed Navigation SLAM has more applications: Mapping, charting a space (re)localizing within a known space SLAM is a more probabilistic approach to VO. [Lv]

24 [Seigwart]

25 [Seigwart]

26 [Seigwart]

27 [Seigwart]

28 [Seigwart]

29 Modeling Uncertainty MonoSLAM Extended Kalman Filter based (EKF) SLAM [Davison 2003] The state of the KF contains Camera position (6 DOF) All features positions (!!) Does not scale well (up to ~1000 features)

30 Loop Closure Determine if visited this place before connect the pose graph. Features: Points (2D, 3D) Patches (intensities) Lines (edges) Need to do matching search problem with a good prior (current location) [Strasdat]

31 Iterative Closest Point Algorithm: For each source point, Match the closest reference point Estimate the combination of rotation and translation (rejecting outlier points) Transform the source points using the obtained transformation. Iterate (reassociate the points, and so on). Key issues to get robust solution: Finding (choose neighbors) Matching (choose metric) Estimating (robust)

32 ICPbased Loop Closure [Fallon et al 2012]

33 ICPbased (volumetric) SLAM [Newcombe 2011]

34 Wrapup What have we learned today: Visual Odometry Intro Algorithm Ambiguity SLAM Intro ICP