TerraSwarm. A Machine Learning and Op0miza0on Toolkit for the Swarm. Ilge Akkaya, Shuhei Emoto, Edward A. Lee. University of California, Berkeley

Transcription

1 TerraSwarm A Machine Learning and Op0miza0on Toolkit for the Swarm Ilge Akkaya, Shuhei Emoto, Edward A. Lee University of California, Berkeley TerraSwarm Tools Telecon 17 November 2014 Sponsored by the TerraSwarm Research Center, one of six centers administered by the STARnet phase of the Focus Center Research Program (FCRP) a Semiconductor Research Corpora@on program sponsored by MARCO and DARPA.

2 Overview 1. Mo0va0on 2. Overview of Current ML Toolkit Capabili0es 3. Case Study: Coopera0ve Robot Localiza0on and Control State Es0ma0on: Par0cle Filtering Path Planning: Informa0on Based Methods for Robot Trajectory Op0miza0on Actor- oriented Design for State Space Dynamics and Measurements 4. Future Direc0ons &Conclusions 2

3 Mo0va0on ML technology in programming languages: MATLAB, Python, Octave, Julia, R... And in the form of toolkits: GMTK, StreamLab, SHOGUN, Weka,... The state- of- the- art tools tradi0onally interact with data and present no na0ve way of incorpora0ng system aspects Goal: to make the ML aspects a na0ve part of the system design by Exploi0ng component- level interac0ons in the swarm Restoring the system level roots of machine learning methodologies by providing the right interfaces between machine learning tools and CPS design aspects. 3

4 Mo0va0on We present an actor- oriented machine learning toolkit that focuses on Applica0ons of ML Algorithms to streaming data Enabling ML techniques to be na0vely integrated into system design Context- aware parameteriza0on of a rich set of ML algorithms Library of easy- to- use tools for developers who are not ML experts Enhancing programmability of swarmlets 4

5 g(x t,u t,t) Inference for Streaming Data Goal : Inference on data that is evolving x 1 x t-1 x t x t+1 x T f(x t,u t,t) z 1 z 1 t-1 z 1 z t 1 t+1 z 1 T observations z 2 z 2 t-1 z 2 t z 2 t+1 z 2 T 5

6 g(x t,u t,t) The Machine Learning Toolkit in Ptolemy II Machine Learning: 1. Hidden Markov Models (HMM) 2. Gaussian Mixture Models (GMM) Parameter Es0ma0on Classifica0on x 1 x t-1 x t x t+1 x T f(x t,u t,t) z 1 z 1 t-1 z 1 z t 1 t+1 z 1 T observations z 2 z 2 t-1 z 2 t z 2 t+1 z 2 T 6

7 The Machine Learning Toolkit in Ptolemy II State Es0ma0on: Par0cle Filtering 7

8 The Machine Learning Toolkit in Ptolemy II Op0miza0on: CompositeOp0mizer: An actor- oriented gradient- descent solver 8

9 Applica0on: Swarmlets for Coopera0ve Robot Control Problem Defini0on: A team of robots, tracking/pursuing a target. Model: State Space Model of target dynamics Observa@ons: Robot sensor measurements (generally nonlinear func@ons of target posi@on + noise) Tasks: Target State Es@ma@on Robot Path Planning: Mul@ple Objec@ves Collision/Obstacle Avoidance, Pursuit, SLAM, Fast Localiza@on, Minimal Uncertainty, 9

10 TerraSwarm Research Center 10 11/17/14 Coopera0ve Robot Control : Challenges Coopera@on between robots Complex measurement/noise models Range Measurements ( e.g., RSSI) Bearings Measurement (e.g., Cameras) Nonlinear robot dynamics Unknown Environment

11 Coopera0ve Robot Localiza0on: State Space Models t = apple xt y t x 0 Uniform([ 100, 100]) y 0 Uniform([ 100, 100]) z t = apple z1 z 2 Target state ( posi0on ) Range Measurements z i t = kr i t t k +! t, i =1, 2! t N (0, 2 ), 2 =5.0 apple 0 t+1 = t + t, t N ( 0, Measurement model apple ) Target state dynamics 11

12 Algorithm Workflow 1. Robots make independent range measurements 2. A centralized (or local) coopera0ve state es0ma0on algorithm es0mates target posi0on given measurements 3. Robot trajectories are op0mized w.r.t. some objec0ve func0on based on the es0mated target posi0on 4. Robots move according to the planned path 12

13 g(x t,u t,t) Target State Es0ma0on x 1 x t-1 x t x t+1 x T f(x t,u t,t) z 1 z 1 t-1 z 1 z t 1 t+1 z 1 T observations z 2 z 2 t-1 z 2 t z 2 t+1 z 2 T Given z t, t=1, T: noisy measurements of a target state x t, Es0mate p(x T z 1:T ): Posterior density of the target state Par0cle filtering is a popular Bayesian Filtering technique to solve this problem: Provides a density es0mate of x T as a par0cle set 13

14 The Par0cle Filter Introducing the par0cle filter: Sequen0al Monte Carlo methods as a general family A Bayesian filter that performs maximum- likelihood state es0ma0on for state- space models with nonlinear dynamics and non- Gaussian noise, in the general case A stochas0c (and oben becer performing) alterna0ve of the Kalman filter ( which is only op0mal for the linear Gaussian case) 14

15 Par0cle Filter: Opera0on Establish a prior belief of the state, represented as a set of par0cles Each par0cle is a candidate state, which is the intruder posi0on in this par0cular applica0on 15

16 Par0cle Filter: Opera0on Make a measurement 16

17 Par0cle Filter: Assigning Weights Assign weights to each par0cle according to how well it explains the measurement ( subject to a measurement model and noise specifica0on) 17

18 Par0cle Filter: Assigning Weights The par0cle weights ( under Gaussian noise) would look like the following: 18

19 Par0cle Filter: Resampling The resul0ng set of par0cles would look like: 19

20 Par0cle Filter: Propaga0on Propagate resul0ng par0cles according to dynamics model TerraSwarm Research Center 11/17/14 20

21 Par0cle Filtering with Range Sensors x weights x Collaborative Particle Output particles target robot1 robot2 state x estimate 10 5 z1 z2 y 0-5 x Robot 1 Robot x x 21

22 Two- Observer Par0cle Filter Measurement Input State Space Model t = apple xt y t x 0 Uniform([ 100, 100]) y 0 Uniform([ 100, 100]) z t = apple z1 z 2 z it = kr it t k +! t, i =1, 2! t N (0, 2 ), 2 =5.0 apple 0 t+1 = t + t, t N ( 0, apple ) 22

23 Path Planning One candidate metric to be used for online trajectory op0miza0on: Informa0on based methods: Mutual Informa0on A par0cle set is a good probabilis0c measure of the uncertainty in a state variable Size of par0cle set can be used to tune approxima0on bounds Op0miza0on Goal: Maximize Mutual Informa0on between measurements and par0cle set: Locate intruder as precisely as possible, with fewest steps Can equivalently be formulated as: Minimize uncertainty in es0mated intruder loca0on 23

24 An Actor- oriented Op0mizer Consider the general constrained problem of the form: Currently supports: COBYLA, a gradient- descent constrained op@miza@on solver 24

25 Cost Func0ons for Path Planning: Mutual Informa0on Op0miza0on Goal: Maximize Mutual Informa0on between future measurements and predicted par0cle set: Locate intruder as precisely as possible, with fewest steps This can equivalently be formulated as: Minimizing the uncertainty in es0mated intruder loca0on. One- step op0mal trajectories: 25

26 Coopera0ve Target Localiza0on: Models 26

27 Demo: MI Maximiza0on 27

28 Demo: Direct Pursuit 28

29 Demo: Hybrid Approach - 1 Follower 29

30 Bridging Actor- Oriented Modeling and ML Algorithms Goal: ML Algorithms that are aware of the system models Methodology: Implement measurement models and system dynamics as decorator actors in the system model Easy to share, consistent models of underlying system models Scalable and unambiguous ML algorithm design for non- experts 30

31 Shared State Space Models for Model Predic0ve Control 31

32 Measurement Models and Dynamics as Decorators 32

33 Measurement Models and Dynamics as Decorators 33

34 Target Localiza0on: Adding a new Sensor 34

35 Demo: Predic0on 35

36 ML and Op0miza0on: Swarmlets PILOT App: State Estimation and Control 2 service discovery 1 TerraSwarm Research Center 11/17/14 36

37 Conclusions Presented an actor- oriented machine learning toolkit that is designed for ML and Op0miza0on applica0ons on streaming data Enhancing programmability of swarmlets Actor libraries for common state- space dynamics and sensor models 37

38 Looking Ahead Enhancing ML capabili0es: Discrete Solvers (Mixed) Integer Programming Tool e.g., GMTK Developing Swarmlets: Providing Services to TerraSwarm Developers More case studies Anomaly sensor fusion 38

39 Demos: Available in Ptolemy II hcp://chess.eecs.berkeley.edu/ptexternal/ 39

40 Thank You! Ques0ons? Comments? 40