08 An Introduction to Dense Continuous Robotic Mapping

Transcription

1 NAVARCH/EECS 568, ROB Winter An Introduction to Dense Continuous Robotic Mapping Maani Ghaffari March 14, 2018

2 Previously: Occupancy Grid Maps Pose SLAM graph and its associated dense occupancy grid map

3 Extension to 3D Occupancy Maps: OctoMap An Efficient Probabilistic 3D Mapping Framework Based on Octrees: Freiburg campus (left) and Freiburg building 079 (right) 3

4 Semantic OctoMap Given a semantically labeled point cloud, compute a semantic class label per voxel: Stanford 2D-3D-Semantics Dataset (left) and NYU Depth Dataset V2 (right) 4

5 Semantic OctoMap + Conditional Random Fields (CRFs) Conditional Random Fields-based octree map with a voxel resolution of 0.1m: Multi-class CRF-based OctoMap; Courtesy D. Lang, Semantic 3D Octree Maps based on Conditional Random Fields 5

6 Semantic OctoMap + Conditional Random Fields (CRFs) By using CRFs on top of octree maps we can account for local correlation between voxels in the map, relaxing the fundamental assumption of marginalized inference per voxel. Multi-class CRF-based OctoMap; Courtesy: D. Lang, Semantic 3D Octree Maps based on Conditional Random Fields 6

7 Q. Where do we get segmented point clouds!? Labeled (Segmented) Point Cloud 2D image segmentation and transferring to 3D; for example see: Fully Convolutional Networks for Semantic Segmentation, or SegNet. Directly labeling point clouds in 3D; for example see: PointNet. Active area, probability more works will appear as we talk... Maybe in a few years we can buy a camera that has an embedded AI for producing such high-level measurements! 7

8 Labeled (Segmented) Point Cloud Segmented point clouds can be seen as high-level measurements or prior over a local area of the semantic map: Finding a good prior can be hard; Informative prior = better posterior or more efficient inference! Many advances in deep learning can be used for better initialization and prior over the state of desired random variables. 8

9 Motivation Can we build a model of the environment that takes into account structural and semantic correlation though joint inference; is continuous and queries can be made at any resolution; can deal with sparse measurements; and is computationally tractable. 9

10 Continuous Robotic Mapping Continuous mapping techniques are based on regression analysis and classification methods (supervised learning): heavily studied in statistics and machine learning: K. P. Murphy, Machine Learning: A Probabilistic Perspective. C. Bishop, Pattern Recognition and Machine Learning. C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning. B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. supervised learning problems have been studied for more than a century in statistics; we can apply many of the available methods with minor modifications to robotic mapping problem. 10

11 Continuous Robotic Mapping Supervised learning can be divided into regression and classification problems: regression is concerned with the prediction of continuous quantities; classification assigns an input pattern to an output class where outputs are discrete class labels. 11

12 Bayesian vs. Non-Bayesian Inference approach: Bayesian which is preferred as it gives the posterior distribution as the output. Non-Bayesian relies on point-estimation where often no notion of uncertainty is available at prediction. 12

13 Continuous Robotic Mapping We can use binary models for occupancy mapping and multi-class models for semantic mapping. Some of the available options: Kernelized Bayesian logistic regression; Relevance Vector Machine (RVM): Bayesian, sparse, fast. Gaussian Processes (GPs): fully Bayesian, exact inference in regression if Gaussian likelihood used, slower with cubic time complexity. 13

14 Today: Gaussian Processes An Introduction to Gaussian Processes 14

15 Gaussian Processes What is a Gaussian Process (GP)? GPs are nonparametric Bayesian regression techniques that employ statistical inference to learn dependencies between points in a data set. 15

16 Gaussian Processes What is a Gaussian Process (GP)? GPs are nonparametric Bayesian regression techniques that employ statistical inference to learn dependencies between points in a data set. Definition (Gaussian process) A Gaussian process is a collection of random variables, any finite number of which have a joint Gaussian distribution. 15

17 Gaussian Processes What is a Gaussian Process? It s a generalization of the multi-variate normal distribution. GP places distributions over functions rather than vectors. It s a non-parametric kernel method. 16

18 Output, f(x) The functions drawn at random from the GP prior. 1.5 Gaussian Processes 1 prior Input, x 17

19 Output, f(x) Gaussian Processes The functions drawn at random from the GP predictive posterior posterior Input, x 18

20 Gaussian Processes Observations are noisy due to the sensor noise: y = f(x) + ɛ y where ɛ y N (0, σ 2 n) The joint distribution of the observed target values, y, and the function values (the latent variable), f, at the query points: [ [ ] y K(X, X) + σ 2 N (0, n I n K(X, X ) ) f ] K(X, X) K(X, X ) 19

21 Gaussian Processes The joint distribution of the observed target values, y, and the function values (the latent variable), f, at the query points: [ [ ] y K(X, X) + σ 2 N (0, n I n K(X, X ) ) f ] K(X, X) K(X, X ) k(, ): covariance function. x : query/test point. y: target (observations). X: d n design matrix of aggregated input vectors (training points). σ 2 n: sensor noise variance. I n : identity matrix of size n. f : latent variable. 20

22 Gaussian Processes The predictive conditional distribution for a single query point f X, y, x N (E[f ], V[f ]): E[f ] = k(x, x ) T [K(X, X) + σ 2 ni n ] 1 y V[f ] = k(x, x ) k(x, x ) T [K(X, X) + σ 2 ni n ] 1 k(x, x ) 21

23 covariance, k(r) Examples of covariance functions using unit length-scale. Gaussian Processes Squared Exponential Matern 8 = 5/2 Sparse distance, r 22

24 Gaussian Processes From left, Squared Exponential, Matérn ν = 5/2, and Sparse covariance functions. 23

25 Gaussian Processes: Model Selection Hyperparameters: free parameters of mean, covariance, and likelihood functions. Learning hyperparameters θ: minimize the negative log of the marginal likelihood function. log p(y X, θ) = 1 2 yt (K(X, X) + σ 2 ni n ) 1 y 1 2 log K(X, X) + σ2 ni n n log 2π 2 24

26 Gaussian Processes: Model Selection log p(y X, θ) = 1 2 yt (K(X, X) + σ 2 ni n ) 1 y 1 2 log K(X, X) + σ2 ni n n log 2π yt (K(X, X) + σni 2 n ) 1 y : data-fit. 1 2 log K(X, X) + σ2 ni n : penalizes the model complexity. n 2 log 2π: constant. 25

27 Gaussian Processes Classification (GPC) Supervised classification: the problem of learning input-output mappings from a training dataset for discrete outputs (class labels). Binary GP Classification: define class labels as y {±1}. 26

28 Gaussian Processes Classification In GPC, the inference is performed in two steps. first computing the predictive distribution of the latent variable corresponding to a query case, f X, y, x N (E[f ], V[f ]). then a probabilistic prediction, p(y = +1 X, y, x ), using a sigmoid function, σ( ), that assigns class labels with a probability that increases monotonically with the latent. 27

29 A Direct Application Example: Semantic Mapping Gaussian Processes Semantic Map (GPSM) 28

30 Features of GPSM Gaussian Processes Semantic Map uses raw pixelated semantic measurements (class labels) in the map inference process. generalizes the traditional occupied and unoccupied class assignment, i.e. binary to multi-class classification. infers the structural and semantic correlation from measurements rather than resorting to assumptions; therefore, can infer missing labels and deal with sparse measurements. 29

31 Features of GPSM is continuous, and queries can be made at any desired locations; therefore, the map can be inferred with any resolution. is agnostic to the input dimensions and can handle an arbitrary number of non-spatial dimensions. 30

32 Problem Statement Problem (Gaussian processes semantic map) Given a point cloud measurement that is (possibly partially) assigned with noisy semantic class labels, infer a semantic map representation of the point cloud as a Gaussian process. 31

33 Problem Formulation Map: n m -tuple random variable (M [1],..., M [nm] ) such that m [i] N (µ [i], v [i] ), i {1: n m }. 32

34 Problem Formulation Map: n m -tuple random variable (M [1],..., M [nm] ) such that m [i] N (µ [i], v [i] ), i {1: n m }. Spatial coordinates of the map: X R 3. 32

35 Problem Formulation Map: n m -tuple random variable (M [1],..., M [nm] ) such that m [i] N (µ [i], v [i] ), i {1: n m }. Spatial coordinates of the map: X R 3. Semantic class labels: C = {c [j] } nc j=1. 32

36 Problem Formulation Map: n m -tuple random variable (M [1],..., M [nm] ) such that m [i] N (µ [i], v [i] ), i {1: n m }. Spatial coordinates of the map: X R 3. Semantic class labels: C = {c [j] } nc j=1. Set of possible measurements: Z X C. 32

37 Problem Formulation Map: n m -tuple random variable (M [1],..., M [nm] ) such that m [i] N (µ [i], v [i] ), i {1: n m }. Spatial coordinates of the map: X R 3. Semantic class labels: C = {c [j] } nc j=1. Set of possible measurements: Z X C. Observation: n z -tuple random variable (Z [1],..., Z [nz] ) such that z [k] Z, k {1: n z }, z [k] = (x [k], y [k] ), x [k] X, and y [k] C. 32

38 Problem Formulation Training set: D = {(x [i], y [i] )} nt i=1, D Z. Target vector: y = vec(y [1],..., y [nt] ). 33

39 Problem Formulation Given observations Z = z, we wish to estimate p(m = m Z = z). Infer the map as a Gaussian process by defining the process as the function y : Z M: y(x) GP(f m (x), k(x, x )) For any map point, m [i] = y(x [i] ) N (µ [i], v [i] ). 34

40 Problem Formulation We use one-vs.-rest multi-class classification, i.e. n c binary GPC. Once the mean and variance of the latent f are available, predict the averaged predictive probability of the class c [j] : p(c [j] = +1 D, x ) = σ(u)n (u E[f ], V[f ])du 35

41 Two-dimensional Toy Example A two-dimensional toy example of GP multi-class classification using a synthetic dataset. 36

42 Results: Sparse and Missing Measurements Effects Image (top left), ground truth (top right), Semantic OctoMap (bottom left), GPSM (bottom right) 37

43 Results: Mapping under Noisy and Misclassified Labels Image (top left), ground truth (top right), Semantic OctoMap (bottom left), GPSM (bottom right) 38

44 Computational Complexity Analysis GPSM using Fully Independent Training Conditional (FITC) approximation for large-scale inference scales as O(n t n 2 u) where n u is the number of inducing points and n u n t. Semantic OctoMap uses octree data structure and scales as O(n p log n n ), where n p and n n the number of points in the point cloud and the total number of nodes, respectively. 39

45 Potential Extensions Hierarchical structure for geometry and semantics; Topological hierarchy; Integration of object detection or instant segmentation together with scene flow estimation and SLAM for managing dynamic environments; Map fusion in continuous or discrete form depending on the application;... Extending the input space using deep kernel learning; Sparse kernels; 40