CS8803: Statistical Techniques in Robotics Byron Boots. Predicting With Hilbert Space Embeddings

Transcription

1 CS8803: Statistical Techniques in Robotics Byron Boots Predicting With Hilbert Space Embeddings 1

2 HSE: Gram/Kernel Matrices bc YX = 1 N bc XX = 1 N NX (y i )'(x i ) > = 1 N Y > X 2 R 1 1 i=1 NX i=1 '(x i )'(x i ) > = 1 N X > X 2 R 1 1 µ x = '(x) 2 R 1 1 µ Y = 1 N NX i=1 (y i ) 2 R 1 1 Would like to calculate: µ Y x = b C YX b C 1 XX µ x 2 R 1 1 CS8803: STR Fast Approximate Kernel Methods 2

3 HSE: Gram/Kernel Matrices µ Y x = b C YX b C 1 XX µ x ˆµ Y x = Y > X X > X + I 1 '(x) (Woodbury) Matrix Inversion Lemma = Y ( > X X + NI) 1 > X'(x) = Y (G XX + NI) 1 G XX (:,i) where G XX = > X X 2 R N N G XX (:,i)= > X'(x i ) 2 R N 1 CS8803: STR Fast Approximate Kernel Methods 3

4 Predicting via Kernel Regression E[f(Y ) x] =hf,µ Y x i E[f(Y ) x] = b f(y ), (Y ) b C 1 YY b C YX b C 1 XX µ x b b C CS8803: STR Fast Approximate Kernel Methods 4

5 A. B. x 10 x10 3 Accuracy (%) Avg. Prediction Err. 6 Avg. Prediction Err. Slot Car Slot Car 8 Hilbert Space Embeddings of Hidde Mean 7 advantage the other learned models by taking of the fact that Example: Slotcar Position Estimation Last x 10 6 A. B. 8 over 250 trials) with different estimated prediction (averaged 90 LDS 3 5 HMM Mean 7 85 (LE kernel). The LE kernel finds the best representation of th IMU RR-HMM Last 6 2 4LDS 80 Embedded HMM state IMU space (Gaussian RBF kernel), and the bottom g space. The top graph shows the Kalman filter state space ( models. Red line indicates true 2-d 2 1 position of the car over racetrack (bottom).0 (B) A comparison of training data embed unit 30 (IMU) (A)6 30 Figure 3: Slot car with inertial measurement Racetrack Prediction0 Horizon Racetrack Prediction Ho Figure 4. Slot car inertial measurement data. (A) The slot racetrack Figure 5 E[f (Y ) x] = hf, µy x i car platform and 4. the Slot IMU (top) and the racetrack (botfigure car inertial measurement data man vs. Racetrack tom). (B) Squared error for prediction with different estiembedde car platform and the IMU (top) and the r 0 mated models and baselines. different to learn a model of the noisy IMU data, and, after CS8803: STR Fast Approximate Kernel Methods Non-hu time we 5 MR oitacol rac tnereffid tom). (B) Squared error for prediction with this data while the slot car circled the trackeigenmap controlled Laplacian 11 seco mated models and baselines. by a constant policy. The goal of this experiment was 40

6 Predicting via Kernel Regression E[f(Y ) x] =hf,µ Y x i E[f(Y ) x] = b f(y ), (Y ) C b 1 YYC YXC b 1 XX µ x b C b f(y ) > > > Y ( Y Y + I) 1 Y X X > X + I 1 '(x) f(y )( > Y Y + I) 1 > Y Y ( > X X + I) 1 > X'(x) f(y )(G XX + NI) 1 G XX (:,i) CS8803: STR Fast Approximate Kernel Methods 6

7 A. B. x 10 x10 3 Accuracy (%) Avg. Prediction Err. 6 Avg. Prediction Err. Slot Car Slot Car 8 Hilbert Space Embeddings of Hidde Mean 7 advantage the other learned models by taking of the fact that Example: Slotcar Position Estimation Last x 10 6 A. B. 8 over 250 trials) with different estimated prediction (averaged 90 LDS 3 5 HMM Mean 7 85 (LE kernel). The LE kernel finds the best representation of th IMU RR-HMM Last 6 2 4LDS 80 Embedded HMM state IMU space (Gaussian RBF kernel), and the bottom g space. The top graph shows the Kalman filter state space ( models. Red line indicates true 2-d 2 1 position of the car over racetrack (bottom).0 (B) A comparison of training data embed unit 30 (IMU) (A)6 30 Figure 3: Slot car with inertial measurement Racetrack Prediction0 Horizon Racetrack Prediction Ho Figure 4. Slot car inertial measurement data. (A) The slot racetrack Figure 5 E[f (Y ) x] = hf, µy x i car platform and 4. the Slot IMU (top) and the racetrack (botfigure car inertial measurement data man vs. Racetrack tom). (B) Squared error for prediction with different estiembedde car platform and the IMU (top) and the r 0 mated models and baselines. different to learn a model of the noisy IMU data, and, after CS8803: STR Fast Approximate Kernel Methods Non-hu time we 7 MR oitacol rac tnereffid tom). (B) Squared error for prediction with this data while the slot car circled the trackeigenmap controlled Laplacian 11 seco mated models and baselines. by a constant policy. The goal of this experiment was 40

8 Example: Depth Camera and Manipulator Control Input (7-DOF) Kinect Sensor (1,228,800-dimensional observations) CS8803: STR Fast Approximate Kernel Methods 8

9 Example: Depth Camera and Manipulator Control Input (7-DOF) Kinect Sensor (1,228,800-dimensional observations) The Challenge: learn a full generative model CS8803: STR Fast Approximate Kernel Methods 8

10 Example: Depth Camera and Manipulator Control Input (7-DOF) Kinect Sensor (1,228,800-dimensional observations) continuous actions and observations highly nonlinear dynamics no prior knowledge of physics, kinematics, or geometry non-gaussian noise CS8803: STR Fast Approximate Kernel Methods 8

11 Example: Depth Camera and Manipulator Control Input (7-DOF) Kinect Sensor (1,228,800-dimensional observations) Training Data: high-dimensional observations 7-DOF continuous control 15 minutes (~60 GBs) of motor babbling Test Data: test set of arm motor babbling (completely new trajectories) CS8803: STR Fast Approximate Kernel Methods 8

12 Example: Depth Camera and Manipulator Filtering (tracking): Prediction (4 seconds): True Observation E[o t a 1:t,o 1:t 1 ] E[o t+120 a 1:t+120,o 1:t 1 ] o t+120 CS8803: STR Fast Approximate Kernel Methods 9

13 CS8803: Statistical Techniques in Robotics Byron Boots Fast Approximate Kernel Methods CS8803: STR Fast Approximate Kernel Methods 10

14 Kernel Methods are Expensive y Y (K + I) 1 K(:,x) Inversion is expensive For n data points, computing the posterior mean is O(n 3 ) storing the kernel is O(n 2 ) Can use approximation to reduce the computation and storage costs CS8803: STR Fast Approximate Kernel Methods 11

15 Methods for Approximating Kernel Machines Factorize the Kernel Matrix Random Features (Random or Greedy) Subset of Data CS8803: STR Fast Approximate Kernel Methods 12

16 Factorization for Approximating Kernel Machines y Y (K + I) 1 K(:,x) y Y (K + 1 I) 1 K(:,x) Y (R XR > X + 1 I) 1 R XR(x) > YR > X(R X R > X + 2 I) 1 R(x) Matrix Inversion Lemma Result: linear regression in finite feature space CS8803: STR Fast Approximate Kernel Methods 13

17 Factorization for Approximating Kernel Machines How do we factorize K? Lots of options in linear algebra: thin SVD, incomplete Cholesky, etc. For n data points, all the decompositions are approx O(n 3 ) However: incremental / approximate versions can be much faster incremental SVD: O(nd 3 ) incomplete Cholesky: O(nd 2 ) CS8803: STR Fast Approximate Kernel Methods 14

18 Factorization for Approximating Kernel Machines Factorization-based approaches essentially approximate an infinite dimensional feature space with a small number of basis vectors Is there a simpler, faster way to choose the basis vectors? CS8803: STR Fast Approximate Kernel Methods 15

19 Methods for Approximating Kernel Machines Factorize the Kernel Matrix Random Features (Random or Greedy) Subset of Data CS8803: STR Fast Approximate Kernel Methods 16

20 Random Features (Rahimi and Recht) Random features use randomly chosen basis vector to approximate the feature space What are the basis vectors? What type of randomness to use? CS8803: STR Fast Approximate Kernel Methods 17

21 Shift Invariant Kernels Kernel value only depends on the difference between two data points!(",#) =!(" #) =!(Δ) A shift invariant kernel!(δ) is the Fourier transformation of a non-negative measure Eg. CS8803: STR Fast Approximate Kernel Methods 18

22 Random Fourier Features What basis should we use? $ %& (" #) can be replaced by cos(&(" #)) since both!(" #) and ((&) are real functions cos(&(" #)) =cos(&") cos(&#) +sin(&") sin(&#) For each &, use feature [cos(&"), sin(&") ] What type of randomness? Randomly draw & from ((&) Eg. Gaussian RBF kernel, drawn from Gaussian CS8803: STR Fast Approximate Kernel Methods 19

23 Random Fourier Features Random'features'usually'need'more'feature'dimensions'than' factoriza4on'approaches'to'achieve'the'same'approxima4on'accuracy CS8803: STR Fast Approximate Kernel Methods 20

24 Methods for Approximating Kernel Machines Factorize the Kernel Matrix Random Features (Random or Greedy) Subsets of Data CS8803: STR Fast Approximate Kernel Methods 21

25 Nystrom s Method Use sub-block of the kernel matrix to approximate the entire kernel matrix G W C G C W + CT CS8803: STR Fast Approximate Kernel Methods 22

26 Nystrom s Method Use sub-block of the kernel matrix to approximate the entire kernel matrix G CS8803: STR Fast Approximate Kernel Methods 23

27 Summary y Y (K + I) 1 K(:,x) y Y (K + 1 I) 1 K(:,x) Y (R XR > X + 1 I) 1 R XR(x) > YR > X(R X R > X + 2 I) 1 R(x) Matrix Inversion Lemma Result: linear regression in finite feature space CS8803: STR Fast Approximate Kernel Methods 24