Visual SLAM for small Unmanned Aerial Vehicles

Transcription

1 Visual SLAM for small Unmanned Aerial Vehicles Margarita Chli Autonomous Systems Lab, ETH Zurich

2 Simultaneous Localization And Mapping How can a body navigate in a previously unknown environment while constantly building and updating a map of its workspace using on board sensors only? One of the most challenging problems in probabilistic robotics Pure localization with a known map. SLAM: no a priori knowledge of the robot s workspace Mapping with known robot poses. SLAM: the robot poses have to be estimated along the way Robot localization using Satellite images [Senlet and Elgammal, ICRA 2012] Helicopter pose given by Leica tracker Video courtesy of S. Lynen 2

3 Integrated Components for Assisted Rescue and Unmanned Search operations IP running between , budget: 17 M, 24 partners Search-and-rescue combining robotics for land, sea and air ETHZ: map generation, people detection, from a UAV 3

4 4

5 How far are we from UAVs able to act as an aid in alpine search-and-rescue? Image source: 5

6 Small UAVs in SHERPA Sensor fusion for environment reconstruction & victim localization: IMU, visible light and thermal cameras, for robust SLAM Close collaboration with Club Alpino Italiano and the Swiss Institute for Avalanche Research 20m 6

7 AIRobots: Innovative aerial service robots for remote inspections by contact Aerial inspection of large industrial facilities Visual pre-inspection Inspection by contact (ultasonic probing) Reduce human involvement & outage periods Safety risks Costs ETH: onboard SLAM for navigation and control with custom-built sensor [Burri et al., CARPI 2012] 7

8 SLAM for small UAVs. Properties & Challenges Weight Lightweight & safe(r) easily deployable than larger aerial vehicles Limited payload (<500g): 10g need approx. 1W in hovering mode (rotor-wing) Limited computational power onboard Choose sensors with high information density Agility RW: highly agile (up to 8m/s) FW: max. 30m/s High-rate, real-time state estimation. The UAV cannot stop Fast, unstable dynamics Autonomy Low bandwidth/ unreliable data links onboard processing RW: Limited battery life (~10mins) FW: Solar powered (5-14hrs endurance) Platform dynamics RW: ROTOR-WING FW: FIXED-WING control speed 8

9 Single Camera SLAM Vision for SLAM Images = information-rich snapshots of a scene Cameras: compact + lightweight HW advances SLAM using a single camera: Hard but (e.g. cannot recover depth from 1 image) very applicable Image Courtesy of G. Klein 9

10 A glance over Monocular SLAM lit Can we track the motion of a camera while it is moving? Pick natural scene features to serve as landmarks (in most modern SLAM systems) Range sensing (laser/sonar): points, line segments, 3D planes, corners Vision: point features, lines, textured surfaces. Courtesy of A. Davison Key: features must be distinctive & recognizable from different viewpoints 10

11 MonoSLAM [Davison, ICCV 2003] Courtesy of A. Davison camera view internal SLAM map 11

12 Prominent Monocular SLAM systems MonoSLAM [Davison 2003 & Davison et al. 2007] PTAM [Klein, Murray 2007] Graph-SLAM [Eade, Drummond 2007] revolutionary in the Vision & Robotics communities, but not ready to leave the lab & perform everyday tasks 12

13 Challenges Fast motion Large scales Robustness Rich maps Low computation Sensor failures Handle larger amounts of data effectively Competing goals: PRECISION EFFICIENCY Key: agile manipulation of information 13

14 sfly: Swarm of micro flying robots aim: Fully autonomous small UAVs operating in unknown, cluttered environments, in a search-and-rescue scenario. 14

15 Enabling UAV navigation Task Autonomous UAV stabilization in GPS-denied environments Autonomous navigation and 3D mapping Long and sustained flights Approach Monocular visual-inertial navigation Downward-looking camera: bearing only measurements (monocular SLAM) IMU: Acceleration & angular velocity measurements Work with S. Weiss, M. Achtelik, S. Lynen and L. Kneip 15

16 Onboard visual SLAM (1 of 2) PTAM [Klein & Murray, ISMAR 2007] Keyframe-based SLAM for small, static scenes Tracking and mapping in separate threads Restrict no. keyframes visual odometry bigger drift Finest-scale features: most prone to outliers discarded from mapping but still crucial for tracking [Weiss et al., JFR 2013] 3D features camera pose pyramid levels Keyframe: typical outdoor scene 16

17 Onboard visual SLAM (2 of 2) [Weiss et al., JFR 2013] Tracking: Still track in finest pyramidal level Use AGAST corners Implementation ROS Package available Currently: ATOM 1.6GHz, 5 KFs: 20Hz Core2Duo 1.83 GHz, 15 KFs: 80 Hz (camera limitation), only using 1 core 17

18 Vision/IMU Controlled Flights [Achtelik et al., IROS 2012] 18

19 Vision-based UAV navigation First UAV system capable of vision-based flight in large real scenarios Framework used by NASA JPL, UPenn, MIT, TUM, What next? Extend capabilities for increased autonomy and deployability Follow-up directions: higher level tasks, exploit swarm and multi-robot behavior. Photo credits: Francois Pomerlaeu

20 BRISK: Binary Rotation Invariant Scalable Keypoints [ICCV 2011] Construct scale-space Image Pyramid Detect corners (FAST based) Assign scale to detected maxima Work with S. Leutenegger 20

21 BRISK: Binary Rotation Invariant Scalable Keypoints [ICCV 2011] BRIEF pattern for intensity pair samples generated randomly [Calonder et al., 2010] BRISK pattern: Used to access image values in a keypoint neighborhood Red circles: smoothing kernel applied. Scaled and rotated versions stored in a look-up table Pairwise intensity comparisons used for orientation assignment Binary Descriptor: a concatenation of pairwise comparison results BRISK sampling pattern 21

22 BRISK in action Precision-Recall: comparable to SIFT and SURF Detection and description ~10x faster than SURF Very fast matching using Hamming distance Open-source, BSD license Part of OpenCV 22

23 Tightly coupled visual-inertial SLAM (1 of 2) [RSS 2013] In-house developed sensor with hardware synchronized (stereo) camera & IMU Work on tight visual & inertial fusion: replace motion model with IMU constraints on the actual motion Vision-Only vs. Visual-Inertial Tight Fusion in Batch Optimization Many Landmarks Many Landmarks Pose Speed / biases Keypoint measurement IMU measurement t t 23

24 Tightly coupled visual-inertial SLAM (2 of 2) [RSS 2013] 24

25 Keyframe-Based Visual-Inertial SLAM Using Nonlinear Optimization [RSS 2013] S. Leutenegger, P. Furgale, V. Rabaud, M. Chli, K. Konolige, and R. Siegwart Robustness from tight coupling Also in difficult lighting conditions /motion blur Accuracy from non-linear optimization rather than filtering Combined reprojection error and IMU error cost function. Use of Keyframes to track changing dynamics: No drift in stand-still using marginalization of non-keyframe poses Real-time operation Fast keypoint matching using BRISKbased stereo processing Building reconstruction 25

26 Path planning for UAVs using RRBT [ICRA 2013] RRBT: Rapidly-exploring Random Belief Trees [Bry et al., ICRA 2011] Sample nominal poses to form candidate trajectories predicting future state distributions Plan the motion resulting to the smallest increase in uncertainty at the goal Can avoid obstacles and featureless regions (or degenerate configurations) Simulation: path planned is slightly longer than the direct path to excite necessary states and error covariance for better convergence origin destination Work with M. Achtelik 26

27 Uncertainty-aware UAV Path Planning Work with M. Achtelik 27

28 Uncertainty-aware UAV Path Planning b&w: original map and obtained path color: map following simulated featureless region & path Work with M. Achtelik 28

29 Conclusion Visual SLAM: has come a long way: from handheld to vision-stabilised flights of UAVs key to spatial awareness of robots bridges the gap between Computer Vision and Robotics Still work to be done before robots are ready for real missions Potential for great impact in search-and-rescue missions Main challenge: increase applicability & ease deployability Robustness to real environments and safety to users Fast camera dynamics (e.g. in aggressive maneuvers), dynamic scenes, sensor outage Path planning and obstacle avoidance Richer maps Semantic reasoning Multi-robot collaboration 29

30 Photo credits: Francois Pomerlaeu 30