Introduction to Mobile Robotics

Transcription

1 Introduction to Mobile Robotics Gaussian Processes Wolfram Burgard Cyrill Stachniss Giorgio Grisetti Maren Bennewitz Christian Plagemann SS08, University of Freiburg, Department for Computer Science

2 Announcement Sommercampus 2008 will feature a hands-on course on Gaussian processes Topics: Understanding and applying GPs Pre-requisites: Programming, basic maths Tutor: Sebastian Mischke Duration: 4 sessions Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 2

3 Overview The regression problem Gaussian process models Learning GPs Applications Summary Some figures were taken from Carl E. Rasmussen: NIPS 2006 Tutorial Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 3

4 The regression problem Given n observed points how can we recover the dependency to predict new points? Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 4

5 The regression problem Given n observed points Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 5

6 The regression problem Solution 1: Parametric models Linear Quadratic Higher order polynomials Learning: optimizing the parameters Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 6

7 The regression problem Solution 1: Parametric models Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 7

8 The regression problem Solution 1: Parametric models Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 8

9 The regression problem Solution 2: Non-parametric models Radial Basis Functions Neural Networks Splines, Support Vector Machines Histograms, Learning: finding the model structure and optimize the parameters Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 9

10 The regression problem Given n observed points Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 10

11 The regression problem Solution 3: Express the model directly in terms of the data points Idea: Any finite set of values sampled from has a joint Gaussian distribution with a covariance matrix given by Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 11

12 Gaussian process models Then, the n+1 dimensional vector which includes the new target that is to be predicted, comes from an n+1 dimensional normal distribution. The predictive distribution for this new target is a 1-dimensional Gaussian. Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 12

13 The regression problem Given n observed points Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 13

14 Gaussian process models Example Given the n observed points and the squared exponential covariance function with and a noise level Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 14

15 Gaussian process models Example Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 15

16 Learning GPs The squared exponential covariance function: index/input distance amplitude characteristic lengthscale noise level The parameters are easy to interpret! Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 16

17 Learning GPs Example: low noise Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 17

18 Learning GPs Example: medium noise Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 18

19 Learning GPs Example: high noise Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 19

20 Learning GPs Example: small lengthscale Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 20

21 Learning GPs Example: large lengthscale Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 21

22 Learning GPs Covariance function specifies the prior prior posterior Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 22

23 Gaussian process models Recall, the n+1 dimensional vector which includes the new target that is to be predicted, comes from an n+1 dimensional normal distribution. The predictive distribution for this new target is a 1-dimensional Gaussian. Why? Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 23

24 The Gaussian Distribution Recall the 2-dimensional joint Gaussian: The conditionals and the marginals are also Gaussians Figure taken from Carl E. Rasmussen: NIPS 2006 Tutorialc Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 24

25 The Gaussian Distribution Simple bivariate example: Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 25

26 The Gaussian Distribution Simple bivariate example: conditional joint marginal Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 26

27 The Gaussian Distribution Marginalization: Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 27

28 The Gaussian Distribution The conditional: Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 28

29 The Gaussian Distribution Slightly more complicated in the general case: The conditionals and the marginals are also Gaussians Figure taken from Carl E. Rasmussen: NIPS 2006 Tutorialc Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 29

30 The Gaussian Distribution Conditioning the joint Gaussian in general With zero mean: Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 30

31 Gaussian process models Recall the GP assumption Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 31

32 Gaussian process models Mean and variance of the predictive distribution have the simple form with Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 32

33 Learning GPs Learning a Gaussian process means choosing a covariance function finding its parameters and the noise level How / to what objective? Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 33

34 Learning GPs The hyperparameters can be found by maximizing the likelihood of the training data e.g., using gradient methods Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 34

35 Learning GPs Or, for a fully Bayesian treatment, by integrating over the hyperparameters using their priors This integral can be approximated numerically using Markov chain sampling. Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 35

36 Learning GPs Objective: high data likelihood data fit complexity penalty const Due to the Gaussian assumption, GPs have Occam s razor built in. Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 36

37 Occam s razor use the simplest explanation for the data Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 37

38 Too long Lengthscale just right Too short

39 Understanding Gaussian processes GP mean prediction can be seen as weighted summation over the data weights Thus, for every mean prediction, there exist an equivalent kernel the produces the same result But: hard to compute Purpose: Visualization / understanding Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 39

40 The GP Equivalent Kernel For different lengthscales Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 40

41 The GP Equivalent Kernel At different locations Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 41

42 Classification Task: Predict discrete (e.g. binary) target values Learn the class probabilies from observed cases Problem: Noise is not Gaussian Approach: Introduce latent real-valued variables for each (discrete) target such that Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 42

43 Classification Problem: Integration over latent variables intractable Approximations necessary Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 43

44 Advanced Topics / Extensions Classification / non-gaussian noise Sparse GPs: Approximations for large data sets Heteroscedastic GPs: Modeling nonconstant noise Nonstationary GPs: Modeling varying smoothness (lengthscales) Mixtures of GPs Uncertain inputs Kernels for non-vectorial inputs Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 44

45 Further Reading Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 45

46 Applications 1. Learning sampling models for DBNs 2. Body scheme learning for manipulation 3. Learning to control an aerial blimp 4. Monocular range sensing Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 46

47 Applications (1) Learning sampling models for dynamic Bayesian networks (DBNs) Joint work with Dieter Fox and Wolfram Burgard Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 47

48 Learning sampling models A mobile robot collides with obstacles Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 48

49 Learning sampling models A mobile robot collides with obstacles Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 49

50 Learning sampling models A mobile robot collides with obstacles Recursive state estimation problem failure event (binary) failure parameters world pose command measurement Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 50

51 Bayesian Filtering We approximate the posterior using a Particle Filter Sample the failure mode Sample the failure parameters Sample the robot state Weight the samples using the current observation Problem: Low frequency of failure events Related work: [de Freitas et al. 2003] Look-ahead particle filter [Thrun et al. 2001] Risk sensitive particle filter Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 51

52 Data-driven Proposals Idea: Learn proposal distributions and utilizing the latest observation. Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 52

53 Gaussian Process Proposals The learning task is for for where is a feature vector computed from. We apply Gaussian processes for this learning task, since they are applicable to regression and classification and provide predictive distributions. Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 53

54 Gaussian Process Proposals Idea of Gaussian process regression Any finite set of values sampled from has a joint Gaussian distribution with a covariance matrix given by Thus, Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 54

55 Learning the Proposals Learning from many collisions without destroying the robot. Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 55

56 Learning the Proposals Resulting training set: hard classification problem without clear class boundaries Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 56

57 Learning the Proposals Learned collision probabilities using Gaussian processes for classification Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 57

58 Learning the Proposals Learned predictor for collision contact points using Gaussian process regression Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 58

59 Experimental Results Experiments with a mobile robot colliding with not directly observable objects Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 59

60 Experimental Results Comparison of an optimized standard particle filter (Std PF) with our approach (GP-PF) Detection rate False positives 0 Std PF 150 particles Std PF 950 particles GP-PF 50 particles GP-PF 300 particles The GP-PF detected nearly all collisions with a low rate of false alarms Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 60

61 Applications (2) Body Scheme Learning for Robotic Manipulation Joint work with Jürgen Sturm and Wolfram Burgard Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 61

62 Body Scheme Learning Existing robot models are typically specified (geometrically) in advance and the parameters are calibrated manually Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 62

63 Problem Description Think Bootstrap, monitor, and maintain internal representation of body Sense 6D Poses Act Joint angles Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 63

64 Body Scheme Factorization Idea: Factorize the model We represent the kinematic chain as a Bayesian network Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 64

65 Bootstrapping Learning the model from scratch consists of two steps: 1. Learning the local models (conditional density functions) 2. Finding the network/body structure Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 65

66 Learning the Local Models Using Gaussian process regression Learn 1D 6D transformation function for each (action, marker, marker) triple Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 66

67 Finding the Network Structure Select the most likely network topology Corresponding to the minimum spanning tree Maximizing the data likelihood Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 67

68 Model Selection 7-DOF example Fully connected BN Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 68

69 Model Selection 7-DOF example Fully connected BN Selected minimal spanning tree Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 69

70 Experiments Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 70

71 Applications (3) Learning to Control an Aerial Blimp Joint work with Axel Rottmann and Wolfram Burgard Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 71

72 Learning to Control a Blimp Indoor Blimp (1.8m length, 0.9m diameter) 1 horizontal propeller 2 tiltable propellers for height control and in-plane movement Total lift: 420 grams Total weight: 310 grams (envelope, gondola, fins, propellers, cables) Remaining payload for hardware: 110 grams Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 72

73 Goal Altitude Control using Reinforcement Learning Learning from interaction with the environment Learning online from scratch environment state reward agent action Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 73

74 Altitude Control with RL Problem description: States: Actions: engine speed in percent Reward: height goal-height Height and vertical velocity are Kalman filtered estimates using the downward-facing sonar sensor Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 74

75 Altitude Control with RL Our approach Monte Carlo: Learn expected long-term reward of state-action pairs Learn online while the blimp is in operation On-policy: Continuous learning and planning Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 75

76 Learning the Q-function Use GPs to approximate the Q-function Points Targets are given by the expected long-term reward Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 76

77 Learning procedure We learn from a sequence of state-action pairs For each state-action pair in the sequence: generate an episode until add it to the training set of the GP adapt the parameters of the covariance function Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 77

78 Typical Optimal Policy Solving the MDP with known dynamics Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 78

79 Typical Optimal Policy Solving the MDP with known dynamics best action distance to goal (m) velocity (m/s 2 ) Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 79

80 Learning in Simulation Comparison with standard Monte-Carlo RL Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 80

81 Experiments Learning on the real blimp Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 81

82 Experiments Online learning vs. manually tuned controller Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 82

83 Applications (4) Monocular Range Sensing Joint work with Jürgen Hess, Felix Endres, Cyrill Stachniss and Wolfram Burgard Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 83

84 Monocular Range Sensing Can we learn range from single, monocular camera images? Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 84

85 Training Setup Mobile robot + laser range finder Omni-directional monocular camera Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 85

86 Training Setup DFKI Saarbrücken University of Freiburg Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 86

87 Learning Range from Vision Associate (polar) pixel columns with ranges Extract features Associate with ranges Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 87

88 Pre-processing Warp images into a panoramic view 120 pixels per column Transform to HSV -> 420 dimensions Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 88

89 Visual Features We evaluated two types of features 1. No human engineering: Principle components analysis (PCA) on raw input 2. Use of domain specific knowledge: Edge features that shall correspond to floor boundaries Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 89

90 Experiments 9 8 Typical 180 scan Ground Truth Distances (Laser) Predicted means (FeatureGP) Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 90

91 Experiments Prediction errors (RMSE on test set) Saarbrücken Freiburg Typical 180 scan Ground Truth Distances (Laser) Predicted means (FeatureGP) Laws3+Canny Laws3+Canny+LMD Laws5 Laws5+LMD Feature-GP Feature-GP+GBP Laws3+Canny Laws3+Canny+LMD Laws5 Laws5+LMD Feature-GP Feature-GP+GBP 5 4 RMSE 3 RMSE Image Number Image Number Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 91

92 Experiments Prediction errors (RMSE on test set) Saarbrücken Freiburg Edge-A1 + Link function 1.70 m 2.86 m Edge-A2 + Link function 2.01 m 2.08 m Edge-B1 + Link function 1.74 m 2.87 m Edge-B2 + Link function 2.06 m 2.59 m Feature-GP 1.04 m 1.04 m Feature-GP + GBP 1.03 m 0.94 m PCA-GP 1.24 m 1.40 m PCA-GP + GBP 1.22 m 1.41 m Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 92

93 Mapping Results Laser-based Vision-based Saarbrücken: Freiburg: Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 93

94 Mapping Results Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 94

95 More Applications Time-series forecasting Visualization of high-dimensional data Learning in Geo-Statistics Localization in cellular networks Terrain regression Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 95

96 Summary GPs are a flexible and practical approach to Bayesian regression. Prior knowledge is encoded in a human understandable way. Learned models can be interpreted. Efficiency mainly depends on the number of training points. Real-world problem sizes require sparse approximations. Mobile Robotics, SS08, Universität Freiburg, Gaussian Processes 96