Classical Single and Multi-Agent Planning: An Introductory Tutorial. Ronen I. Brafman Computer Science Department Ben-Gurion University

Size: px

Start display at page:

Download "Classical Single and Multi-Agent Planning: An Introductory Tutorial. Ronen I. Brafman Computer Science Department Ben-Gurion University"

Meagan Woods
5 years ago
Views:

1 Classical Single and Multi-Agent Planning: An Introductory Tutorial Ronen I. Brafman Computer Science Department Ben-Gurion University

2 Part I: Classical Single-Agent Planning

3 What Is AI Planning! A sub-field of AI concerned with modelling and answering the following problem: how to enable an artificial agent to autonomously choose good behaviours.! Original motivation: autonomous robots that achieve goals without being provided with an explicit program! R2D2, Get me some coffee! Instead of telling the agent what to do, we want to tell it what to achieve, and it will figure out what to do on its own.! Studies: models, languages, and algorithms for describing and solving different types of sequential decision problems! Many different applications in support of autonomy and task automation

4 Some AI Planning Applications

5 Focuses on Generic Methods (Domain Independence)! In principle, one could write special planners for different settings! Such planners can be very efficient, but:! A lot of effort to develop! Requires much expertise But! AI planning focuses on generic algorithms and language! Domain knowledge can be provided to planners via constraints that are also expressed in a general language 5

6 NASA s Experience Galileo's Jupiter or Cassini's Saturn missions:! $1G budget! Ground crew of Mars micro-rover Sojourne exploited AI/autonomy:! $100M budget! Much smaller ground team! Sojourne was designed to operate for two month, but lasted much longer Soujourne s level of autonomy is still low compared to what we can achieve!!

7 NASA s Experience Space exploration should be:! Low cost & rapid development: use off-the-shelf algorithms/ systems as much as possible! Low cost control: autonomous operations for long period of times! Smart algorithms can optimize success probability given tight deadlines and resource constraints Main Conclusion: low cost, off-the-shelf components for autonomous operation can lead to significant cost savings

8 AI Planning: Components Input:! World model:! Variables describing the state of the world! Initial/Current state of the world! Action model describes how the agent s ability to influence the world! Performance criteria: goal states, good states, bad states etc. Output:! Good behaviour

9 Rich Set of Models! Uncertainty! Stochastic actions! Partial observability and sensing! Temporal actions! Concurrency/interactions! Multi-agent! 9

10 Our Focus: Classical AI Planning! Single agent! Actions are deterministic and instantaneous! Initial state is fully known! The goal is to make some set propositions true! I.e. Reach some goal states! Behaviour = plan: a sequence of actions that, when applied in the initial state lead to one of the goal states

11 Classical Planning: An Informal Example Logistics Planning! World model:! Variables describing the location of trucks, planes, and packages! Initial state: current location of each object! Action model! Actions include load/unload packages and move vehicles! Model describes how each action impacts value of state variables! Goal: A state in which the packages are in the destination Output:! A sequences of actions move, load, unload, fly, etc. possibly concurrent, that cause the packages to end up in their destinations

12 Key Notions! Model: describes the relevant aspects of the world! Should be intuitive! May be very large and difficult to work with! Language: Tool for describing the world implicitly! Should be easy to describe intuitive/natural models! Interpretation: maps between language and model! Queries: questions about the model we wish to answer! Algorithms: compute answers to queries

13 Key Elements of Classical Planning! Model: Transition system! Set of states and transitions between them! Each transition has a label the action that brings it about! Each action has a cost (1 by default)! Initial state + Goal states! Language and Interpretation: We ll see! Query: Find a plan = sequence of actions/labels that takes us from the initial state to a goal state! Algorithms: path finding algorithms?

14 Example: Blocks World! States: possible configurations of 3 blocks on a table!actions: move a block from one block, or from the table, on top of another

15 Classical Planning Model = Automaton! States of the world = automaton state! Actions = letters/edge labels! Initial state = Initial state! Goal states = Absorbing state! Plan = a word in the language of the automata Simple solution: shortest path algorithm on automata graph

16 The Curse of Dimensionality Blocks States Even for a relatively small number of objects we cannot realistically construct, store, or compute with the explicit state transition graph

17 What Can We Do?! Obviously, we cannot construct the entire statetransition diagram, nor maintain it in memory, nor run algorithms on this explicit structure! We need to describing the model implicitly! We need techniques for generating path to the goal that explore only a small fragment of the model 17

18 Language: STRIPS [Fikes & Nilsson, 71]! A concise language for describing classical AI planning problems! States described via a set P of propositions! Any assignment to P is a possible world! Initial state: Simply one such assignment! Goal condition: partial assignment over P! Any state satisfying it is a goal state

19 STRIPS! Actions described by two lists! Preconditions: list of propositions! Must be true for the action to be applied in a state! Effects: list of literals! Become true following the action s application! Propositions not mentioned in the effects list remain unchanged

Example 1: BlocksWorld! State variables: on(r,b),on(b,r),on(r,g) on(g,r),on(g,b),on(b,g),on(r,table), on(g,table),on(b,table),clear(b),clear(r), clear(g),clear(table)! Actions: Move(X,Y,Z)!

20 Example 1: BlocksWorld! State variables: on(r,b),on(b,r),on(r,g) on(g,r),on(g,b),on(b,g),on(r,table), on(g,table),on(b,table),clear(b),clear(r), clear(g),clear(table)! Actions: Move(X,Y,Z)! Preconditions: on(x,y),clear(x), Z!=Table, clear(z)! Effects: -on(x,y),on(x,z),-clear(z),clear(y)! Actions: Move(X,Y,Table)! Preconditions: on(x,y),clear(x)! Effects: -on(x,y),on(x,z),clear(y) 20

21 Example 2: The Logistics Domain

22 Example: The Logistics Domain! Every states of the world describes the location of packages, trucks and airplanes! Initial state:! package p1 is in the post-office in Chania, p2 is in Hotel Halepa, and p3 is in Beer-Sheva! Truck in Heraklion airport, and a truck in Beer-Sheva! Airplane in Heraklion airport! Goal: p1 and p2 in Beer-Sheva, p3 in Hotel Halepa! Actions: we can load and unload packages into/from a truck or a plane, we can move trucks between locations in the same country, and we can fly planes between airports

23 Logistics Domain Example! Propositions: at(p1,chaniapo), at(p1,halepa), at(p1,heraklionap), at(p1, TLV), at(p1, Beer-Sheva), in(p1,crete-truck), in(p1, isra-track), in(p1, plane), at(cretetruch, ChaniaPO),! Initial state (propositions assigned true): at(p1,chaniapo), at(p2,halepa), at(p3,bs), at(crete-truck, HeraklionAP), at(isra-truck, BS), at(plane, HeraklionAP)! Goal: at(p1,bs) & at(p2,bs) & at(p3, Halepa)

24 Logistics Domain Example Actions:! Load(package,vehicle,location)! Preconditions: at(package,location), at(vehicle,location)! Effects: -at(package, location), in(package, vehicle)! Unload -- similar! Move(vehicle,loc1,loc2)! Preconditions: at(vehicle, loc1)! Effects: -at(vehicle,loc1), at(vehicle, loc2)

25 Applying an Action! To apply an action at a state:! First, check if the preconditions hold! Next, update the value of the variables in the effect! All other variables state the same 25

26 Action Application: Example State: at(p1,chaniapo), at(p2,halepa), at(p3,bs), at(crete-truck, HeraklionAP), at(isra-truck, BS), at(plane, HeraklionAP)! Apply Load(p1,crete-truck,Halepa)! Preconditions: at(p1,halepa), at(crete-truck,halepa) not satisfied! Action is inapplicable 26

27 Action Application: Example State: at(p1,chaniapo), at(p2,halepa), at(p3,bs), at(crete-truck, HeraklionAP), at(isra-truck, BS), at(plane, HeraklionAP)! Apply Move(crete-truck,HeraklionAP,Halepa)! Preconditions: at(crete-truch,heraklionap) satisfied! Effect: at(crete-truck,halepa), -at(crete-truck,heraklionap)! New state: at(p1,chaniapo), at(p2,halepa), at(p3,bs), at(crete-truck, HeraklionAP), at(crete-truck, Halepa), at(isra-truck, BS), at(plane, HeraklionAP) 27

28 Planning Is Not Easy! STRIPS planning is hard:! PSPACE-Complete [Bylander, 1994]! NP-complete for practical purpose! Tractability?! (i.e., plans that are not too long)! In special cases with special structure under severe restrictions (e.g single effect + more restrictions)! Or under strong decomposability properties (2 nd part)

29 Planning in Practice! Many interesting and very nice algorithms! Partial order planning! Graphplan! Regression! Planning via Satisfiability! State-of-the-art: planning via heuristic forward search! Key: all planning algorithms require search! Best heuristics exist for forward search

30 Planning via Heuristic Forward Search! Start with the initial state! Expand the most promising state! Until a goal state is found! Most promising state: varies with the search algorithm, but generally speaking the state with the best heuristic value

31 Forward Search Space A B C {on&a&b,on&b&c,clear&a,on&table&c}" Move&to&table&a&b" Ini8al"State" {on&table&a,on&b&c,clear&a,clear&b,on&table&c}" Move&from&table&a&b" Move&to&table&b&c" Move&b&c&a" Goal"Condi8ons" {on&a&b,on&b&c,clear&a,on&table&c}" {on&table&a,on&table&b,"on&table&c,clear&a,clear&b,clear&c}" {on&table&a,on&b&a,on&table&c,clear&b,clear&c}" B A C 1"

32 Two Key Issues! How to compute the heuristic value! Key idea: solve a simple problem and use the solution to inform your choices in the original problem! We ll see two methods there are many others! How to use the heuristic value to improve search! There are many heuristic search algorithms with different properties I assume you know about search! A*, Best-first, hill-climbing, Weighted A*,

33 Heuristics! Heuristic functions should be fast to compute! Computed for each search state! Heuristic functions should be as accurate as possible! To find optimal plans we need admissible heuristics! Admissible: optimistic ; does not overestimate cost

34 Two Methods! Pattern Database Heuristic!! Landmark heuristic

35 Pattern Databases! Before search:! Project original problem to a subset of variables! Generate complete transition system! Compute shortest path from each state to the goal and store it! During search! With each state, associate the distance from its projection to the (projected) goal as its heuristic value

36 Pattern database heuristics ALR ARL LLR RRL ALL ARR LRR LLL RRR RLL BLL BRR LRL RLR BRL BLR Logistics problem with one package, two trucks, two locations: state variable package: {L, R, A, B} state variable truck A: {L, R} state variable truck B: {L, R} 36

37 Pattern database heuristics Abstraction induced by π {package} : ALR ARL LLR RRL ALL ARR LRR LLL RRR RLL BLL BRR LRL RLR BRL BLR h {package} (LRR) = 2 37

38 Pattern database heuristics Abstraction induced by π {package,truck A} : ALR ARL LLR RRL ALL ARR LRR LLL RRR RLL BLL BRR LRL RLR BRL BLR h {package,truck A} (LRR) = 2 38

39 39 h {Package,Truck A} h {Package} State 2 2 LRR 2 2 LLR 2 2 LLL 2 2 LRL 2 1 ALR 2 1 ALL 1 1 ARL 1 1 ARR 1 1 BLR 1 1 BLL 1 1 BRL 1 1 BRR 0 0 RRR 0 0 RLR 0 0 RLL 0 0 RRL π { } π {, }

40 {Package,Truck A} Best first search with h LRR Iter. Fringe LLR LRL 0 1 LRR(2) LLR(2),LRL(2) 2 ALR(2),LLL(2),LRL(2) ALR LLL 3 4 ARR(1),ALL(2), LLL(2),LRL(2) RRR(0),ALL(2), LLL(2),LRL(2) ARR ALL RRR Tie breaks with the leftmost state in the fringe 40

41 Landmarks! Fact Landmark: a fact that must be true at some state during every plan! Action Landmark: an action that must appear in every valid plan! Disjunctive Action Landmark: a set of actions, one of which must appear in every valid plan! Cost of Disjunctive Action Landmark: the cost of its cheapest action 41

42 Landmark Heuristic! An admissible heuristic! State of the art for optimal planning Basic idea:! Compute a set of disjunctive action landmarks, using the relaxed planing problem! Combine their cost intelligently

43 Relaxed Planning! Relaxed planning: ignore negative effects of an action! Anything achieved cannot be destroyed! Many heuristics operate on the relaxed planning problem:! Original Load(package,vehicle,location)! Preconditions: at(package,location), at(vehicle,location)! Effects: -at(package, location), in(package, vehicle)! Relaxed Load(package,vehicle,location)! Preconditions: at(package,location), at(vehicle,location)! Effects: in(package, vehicle)

44 Landmarks: Example! Remove negative effects (relax )! Propositions: i,a,b,c,g! Initial state {i}. Goal condition {g}! Relaxed Actions:! o 1 [3]: i " a,b! o 2 [4] : i " a,c! o 3 [5] : i " b,c! o 4 [0]: a,b,c " g! An optimal plan:!o 1 o 2 o 4! Cost:!3+4+0 = 7! Landmarks:!W={o 4 } (cost 0)!X={o 1, o 2 } (cost 3)!Y={o 1, o 3 } (cost 3)!Z={o 2, o 3 } (cost 4)!Also: {o 1, o 2, o 3 } (cost 3)

45 Landmarks as Heuristics! Every landmark cost is a lower bound on plan cost (why?)! What if we have a couple of landmarks?! Their max cost is still admissible (= 4)! Sum may not be admissible (=10)!We can do better with hitting sets! W={o 4 } (cost 0) X={o 1, o 2 } (cost 3) Y={o 1, o 3 } (cost 3) Z={o 2, o 3 } (cost 4)

46 Hitting Sets! Hitting set: a set of actions, one of which appears in every landmark! o 1, o 2, o 4 or o 2, o 3, o 4! Hitting set cost: sum of actions costs! Cost (o 1, o 2, o 4 )=7!Cost (o 1, o 2, o 3 )=8! Cost of minimum hitting set is always admissible! (= 7) why?! W={o 4 } (cost 0) X={o 1, o 2 } (cost 3) Y={o 1, o 3 } (cost 3) Z={o 2, o 3 } (cost 4) Cost(o 1 )=3 Cost(o 2 )=4 Cost(o 3 )=5 Cost(o 4 )=0

47 Computing Landmarks 1. Relax the original planning problem! Ignore negative effects 2. Select one precondition for every action 3. Construct the justification graph Nodes = propositions Edges: from chosen precondition to every effect 4. Any cut in this graph is a disjunctive action landmark

48 Pattern database heuristics Example: justification graph Example pcf D: D(o 1 )=D(o 2 )=D(o 3 )=i, D(o 4 )=b o 1 a o 1 [3] : i a, b o 2 [4] : i a, c o 3 [5] : i b, c o 4 [0] : a, b, c g i o 2 o 1 o 3 o 2 b o 4 g o 3 c 48

49 Pattern database heuristics Example: cuts of a justification graph Example Landmark W = {o 4 } (cost 0) o 1 a o 1 [3] : i a, b o 2 [4] : i a, c o 3 [5] : i b, c o 4 [0] : a, b, c g i o 2 o 1 o 3 o 2 b o 4 g o 3 c 49

50 Pattern database heuristics Example: cuts of a justification graph Example Landmark X = {o 1, o 2 } (cost 3) o 1 a o 2 o4 o 1 [3] : i a, b o 2 [4] : i a, c o 3 [5] : i b, c o 4 [0] : a, b, c g i o 1 o 3 o 2 b g o 3 c 50

51 Pattern database heuristics Example: cuts of a justification graph Example Landmark Y = {o 1, o 3 } (cost 3) o 1 a o 1 [3] : i a, b o 2 [4] : i a, c o 3 [5] : i b, c o 4 [0] : a, b, c g i o 2 o 1 o 3 o 2 b o 4 g o 3 c 51

52 Pattern database heuristics Example: cuts of a justification graph Example Landmark Z = {o 2, o 3 } (cost 4) o 1 a o 1 [3] : i a, b o 2 [4] : i a, c o 3 [5] : i b, c o 4 [0] : a, b, c g i o 2 o 1 o 3 o 2 b o 4 g o 3 c 52

53 Beyond Classical Planning! Temporal planning: actions that have duration! Planning under uncertainty:! Unknown initial state, non-deterministic actions, no observations (conformant planning)! Unknown initial state, non-deterministic actions, observations (contingent planning)! Stochastic actions, full observability (Markov Decision Processes! Stochastic actions, partial observability (Partially observable Markov Decision Processes)

54 Summary! Planning is an exciting area of research with many applications! The key to successful planning is the ability to generate a good heuristic function! Heuristics are typically generated by analyzing a simpler problem! Other important techniques are! Domains analysis attempt to detect and exploit properties of a domains, such as symmetry! Pruning techniques methods that allow us to ignore certain branches in the tree, due to symmetry and other reasons

Foundations of Artificial Intelligence

Foundations of Artificial Intelligence 37. Automated Planning: Abstraction and Pattern Databases Martin Wehrle Universität Basel May 13, 2016 Planning Heuristics We consider three basic ideas for general