Decision making for autonomous naviga2on. Anoop Aroor Advisor: Susan Epstein CUNY Graduate Center, Computer science

Transcription

1 Decision making for autonomous naviga2on Anoop Aroor Advisor: Susan Epstein CUNY Graduate Center, Computer science

2 Overview Naviga2on and Mobile robots Decision- making techniques for naviga2on Building and upda2ng models Using models to make decisions Human naviga2on Challenges in Autonomous naviga2on

3 Autonomous naviga2on Autonomous Naviga2on: Ability to reach a given loca2on effec2vely without human interven2on. Many applica2ons: Driverless cars Warehouse Automa2on Search and rescue

4 Problem statement Given a mobile robot situated in an environment and a naviga2on task, how should the robot issue commands to the actuator to complete the task? Robot: Single vs Team Global Sensor (GPS) vs Local Onboard Sensor Environment: Known (Warehouse) vs Unknown (Home) Sta2c vs Dynamic Determinis2c vs Non determinis2c Task: Exploratory vs Target based

5 Example 1 (Warehouse Automa2on) Amazon order fulfillment centers uses robots to improve delivery. hyps:// v=utba9yvzbjm Features: Mul2ple robots Uses global sensors Dynamic goals (Orders are dynamic) Known, sta2c environment

6 Example 2 (Indoor Service Robots) irobot s roomba robot vacuums floors in homes. hyps:// kre Features: Single robot Uses local sensors Sta2c goals (Clean the house) Dynamic, unknown environment (People moving around) Connected (Maybe offloads computa2on to the cloud)

7 Decision making Autonomous robot naviga2on can be seen as a sequen2al decision making problem Op2mal decision making depends on techniques to Build and update models of the world Use these models to make decisions

8 Build and Update models Knowledge : Environment (Maps, Robots, Obstacles) State of the decision- making robot (Posi2on, Orienta2on) Goals (Target ordering) or U2lity func2on (Rewards and Penal2es) Parameters to control/configure the robots. How to acquire knowledge: Hardcoded by the designer Build using sensor inputs

9 Methods to acquire knowledge Type Mapping Posi2on and orienta2on of robot Target ordering Parameter es2ma2on Methods SLAM SLAM, GPS Auc2ons or other op2miza2on methods Reinforcement Learning, Gene2c Algorithms

10 Simultaneous Localiza2on and mapping (SLAM) Problem : How to es2mate the robot s loca2on and the structure of the environment when sensors and actuators are noisy. Solu2on: Model the sensor / actuator noise probabilis2cally Use Bayesian sta2s2cs to improve es2mates of loca2on of the landmarks and posi2on of the robot. The loca2on of the landmarks are models as a markov chain Drawbacks: Computa2onally intensive Useful only if the landmarks are sta2c

11 Decision making depends on the nature of the domain Domain Determinis2c Non- determinis2c Sta2c Graph based search (atomic states) Classical Planning (factored states) Sampling based approaches MDP, POMDP (atomic states) Factored POMDP (factored states) Monte Carlo sampling based approaches Dynamic Agent centered search (local plans) Hybrid architectures (abstract plans) Reac2ve architectures (no plan)

12 Sta2c, Determinis2c domain Knowledge Determinis2c (Discrete world model) Non- determinis2c (Probabilis2c world model) Sta2c Graph based search (atomic states) Classical Planning (factored states) Sampling based approaches MDP, POMDP (atomic states) Factored POMDP (factored states) Monte Carlo sampling based approaches Dynamic Agent centered search (local plans) Hybrid architectures (abstract plans) Reac2ve architectures (no plan)

13 Graph- based search (atomic state) A* Use heuris2cs to decide which node should be explored first; d(s) = f(s) + h(s) Admissible heuris2cs guarantee op2mality IDA* Variant of itera2ve deepening (ID) search. Instead of using depth of the tree as cutoff, heuris2c path length is used as cutoff Combines the memory efficiency of ID, with heuris2c search of A*

14 Classical planning (factored states) STRIPS Uses first order logic to represent state, ac2ons and goals. Extract the differences between current state and goal state. Iden2fy operators to reduce the difference. Solve the sub- problem of producing a state where the operator is applicable. ABSTRIPS (Hierarchal problem solving) Abstract the problem by ignoring operator precondi2ons If abstract sub- problem cannot be solved, then the more concrete versions will never be solved. Factored states allows the crea2on of hierarchies

15 Other classical planning approaches Plan space planning Each node in the search space is a par2al order plan. Models state and end states as dummy ac2on, so that the ini2al plan is state - > finish Refine the plan by adding operators to resolve unfulfilled precondi2ons and post condi2ons in the plan. Why does it work? If an abstract solu2on solves a problem, then stop the search process.

16 Other classical planning approaches GraphPlan based planning Uses a data structure called planning graph to create a non linear planner Planning graph is an abstract representa2on of the state space. Search of solu2on of size n before searching for plan of size n+1 Why does it work? Abstract sub- problems with no solu2ons saves computa2on

17 Sampling based planning Rapidly exploring Random Trees

18 Sampling based planning Probabilis2c RoadMaps

19 Knowledge Determinis2c (Discrete world model) Non- determinis2c (Probabilis2c world model) Sta2c Graph based search (atomic states) Classical Planning (factored states) Sampling based approaches MDP, POMDP (atomic states) Factored POMDP (factored states) Monte Carlo sampling based approaches Dynamic Agent centered search (local plans) Hybrid architectures (abstract plans) Reac2ve architectures (no plan)

20 Par2ally observable MDPs Decision making can be modeled as Markov decision process (MDP) when ac2ons are non- determinis2c and goals are defined by a reward func2on. Solving MDPs results in a policy which defines the right ac2on to be taken in any given state. POMDPs assumes that the robot does not know its state accurately. Model the state of a robot as a probability distribu2on called belief state. POMDPs are solved by conver2ng the problem into a Belief MDP problem where the state is a belief state.

21 Monte Carlo Sampling Belief states can be represented using samples draw from the distribu2on. To compute the posterior belief state aoer execu2on of an ac2on. Compute the posterior distribu2on of each of the samples Integrate the posterior of the samples to get the es2mate of the real posterior distribu2on

22 Knowledge Determinis2c (Discrete world model) Non- determinis2c (Probabilis2c world model) Sta2c Graph based search (atomic states) Classical Planning (factored states) Sampling based approaches MDP, POMDP (atomic states) Factored POMDP (factored states) Monte Carlo sampling based approaches Dynamic Agent centered search (local plans) Hybrid architectures (abstract plans) Reac2ve architectures (no plan)

23 Agent Centered Search Strategy to handle dynamic environments Create locally op2mal plans Execute un2l failure Re- plan Incremental search D*, MPGAA* Online POMDPs

24 MPGAA* Run A* algorithm with the current knowledge of the obstacles Execute the A* plan When the A* plan fails, rerun the A* search with the following modifica2ons to improve performance: Rerun could use improved heuris2cs, derived from the first A* run. Rerun could construct part of the new plan from the old A* plans.

25 Online POMDPs Offline POMDPs specify, prior to the execu2on, the best ac2on for all possible situa2on. Online POMDPs find the best ac2on for the current state by solving a small horizon POMDP. This allows POMDPs to work in domains where the model is changing.

26 Knowledge Determinis2c (Discrete world model) Non- determinis2c (Probabilis2c world model) Sta2c Graph based search (atomic states) Classical Planning (factored states) Sampling based approaches MDP, POMDP (atomic states) Factored POMDP (factored states) Monte Carlo sampling based approaches Dynamic Agent centered search (local plans) Hybrid architectures (abstract plans) Reac2ve architectures (no plan)

27 Reac2ve Architectures Strategy Don t plan Decisions are made from rules on sensor inputs Dynamic environments don t mayer as long as the sensors are accurate Examples: Subsump2on, AuRA Drawbacks: Subop2mal behavior

28 Knowledge Determinis2c (Discrete world model) Non- determinis2c (Probabilis2c world model) Sta2c Graph based search (atomic states) Classical Planning (factored states) Sampling based approaches MDP, POMDP (atomic states) Factored POMDP (factored states) Monte Carlo sampling based approaches Dynamic Agent centered search (local plans) Hybrid architectures (abstract plans) Reac2ve architectures (no plan)

29 Hybrid Architectures Strategy 1: Build long term abstract plans Handle uncertainty like obstacle avoidance through reac2ve behavior, or short term plans Examples: 3T, Minerva Strategy 2: Run long term planners and short term reac2ve behaviors in parallel. Combine the outputs using a vo2ng mechanism Examples: DAMN

30 Human Naviga2on Mul2ple representa2ons (but no metric maps) Routes, Local Graphs. Mul2ple wayfinding techniques Search, Retrace.

31 Challenges in autonomous naviga2on Par2al target ordering Mul2 robot sepng Human interac2on Learning the sta2c parts of the models Learning to plan in dynamic domains

32 Thank you