CMU-Q Lecture 6: Planning Graph GRAPHPLAN. Teacher: Gianni A. Di Caro

Transcription

1 CMU-Q Lecture 6: Planning Graph GRAPHPLAN Teacher: Gianni A. Di Caro

2 PLANNING GRAPHS Graph-based data structure representing a polynomial-size/time approximation of the exponential search tree Can be used to automatically produce good heuristic estimates (for A*) Can be used to search for a solution using the GRAPHPLAN algorithm 2

3 PLANNING GRAPHS No-Op Persistent action) Level S 2 Level O 2 Leveled graph: vertices organized into levels/stages, with edges only between levels Condition Precondition Operation Postcondition Two types of vertices on alternating levels: o Conditions o Operations Two types of edges: Precondition: from condition to operation Postcondition: from operation to condition 3

4 GENERIC PLANNING GRAPH Condition S 0 contains all the conditions that hold in initial state 4

5 GENERIC PLANNING GRAPH: OPERATION LEVEL Level S 0 Level O 0 Precondition Condition Add operation to level O i if its preconditions appear in level S i 5

6 GENERIC PLANNING GRAPH: CONDITION LEVEL Precondition Level S 1 No-Op Condition (Persistent action) Add condition to level S i if it is the postcondition of an operation (it is in ADD or DELETE lists) in level O i 1 Add p if in ADD list Add p if in DEL list Keep a previous condition (persistence, no-op action) 6

7 CONDITIONS MONOTONICALLY INCREASE p No-Op O 1 p p No-Op O 1 No-Op p p O 2 q No-Op q No-Op q r No-Op r No-Op r #Conditions (always carried forward by no-ops) 7

8 GENERIC PLANNING GRAPH No-Op (Persistent action) Level S 2 Level O 2 Condition Precondition Operation Postcondition Repeat 8

9 GENERIC PLANNING GRAPH Idea: S i contains all conditions that could hold at stage i (of the real plan) based on past actions; O i contains all operations that could have their preconditions satisfied at stage i (of the real plan) No ordering among the operations is assumed at each stage, they could be executed in parallel The level j at which a condition first appears is an estimate of how difficult is to achieve that condition It can optimistically estimate how many steps it takes to reach a goal g (or sub-goal g i ) from the initial state admissible heuristic! 9

10 OPERATIONS MONOTONICALLY INCREASE p No-Op O 1 p p No-Op O 1 No-Op p p O 2 q No-Op q No-Op q r No-Op r No-Op r #Operations (as a result of conditions monotonic increase, that keep previous preconditions hold and set new preconditions true) 10

11 MUTUAL EXCLUSION LINKS As it is the graph would be too optimistic! The graph data structure also records conflicts between actions or conditions: two operations or conditions are mutually exclusive (mutex) if no valid plan can contain both at the same time A bit more formally: o Two operations are mutex if either their preconditions or postconditions are mutex (inconsistent effects, competing needs, interference) o Two conditions are mutex if one is the negation of the other, or all action pairs that achieve them are mutex (inconsistent support) 11

12 A RUNNING EXAMPLE Have cake and eat cake too problem 12

13 A RUNNING EXAMPLE Only Eat(Cake) is applicable 13

14 A RUNNING EXAMPLE 14

15 MUTEX CASES Inconsistent postconditions (for two operations): the effect of one operation negates the effect of the other Note: no-hop = persistence precondition = condition itself Inconsistent Postconditions B B Inconsistent postconditions Eat(Cake) and no-op Have(Cake) Eaten(Cake) and Eat(Cake) Inconsistent postconditions 15

16 MUTEX CASES Interference (between two operations): a postcondition of one operation negates a precondition of other B B Interference Eat(Cake) and no-op Eaten(Cake) Interference Eat(Cake) and no-op Have(Cake) 16

17 MUTEX CASES Interference Inconsistent postconditions Negation of each other Interference Inconsistent post 17

18 MUTEX CASES Competing needs (for two ops): a precondition of one operation is mutex with a precondition of the other because: o they are the negate of each other, or o they have inconsistent support Bake(Cake) [PRE: Have(Cake)] and Eat(Cake) [PRE: not Have(Cake)] Mutually negated no-op actions B B Competing Needs Competing needs (negation) Competing needs (negation) 18

19 MUTEX CASES Inconsistent support (for two conditions): each possible pair of operations that achieve the two conditions is mutex. B Have(Cake) and Eaten(Cake), are mutex in S 1 but not in S 2 because they can be achieved by Bake(Cake) Inconsistent and Eaten(Cake) Support Inconsistent support No inconsistent support: Bake(Cake) + no-op Eaten(Cake) C 19

20 A CONSTRUCTIVE PROCESS, NO CHOICES The process of constructing the planning graph does not require to choose among the actions No combinatorial search is involved! At each O level, identify applicable actions, including the persistent ones At each S level identify all conditions that could result from the actions at the previous level Check for mutexes and add mutex link relations Check for level-off 20

21 #MUTEXES MONOTONICALLY DECREASE Intuitive proof: o If two operations or conditions are a mutex at a given level, they will be also mutex for all previous levels (because of less conditions or actions to use to possibly avoid their conflicts) o At each level we should also think of all operations that cannot be executed and all actions that do not hold: these are in mutex with everything o By adding them over time the total # of mutexs decreases 21

22 CONDITION MUTEX RELATIONSHIPS MONOTONICALLY DECREASE p No-Op p O 1 q No-Op q r No-Op r #Conditions mutex (new operations appear that can resolve inconsistent support conflicts) 22

23 OPERATION MUTEXES MONOTONICALLY DECREASE Proof idea: o Different behavior for the different types of mutexes o Inconsistent postconditions and interference mutex are properties of the operations themselves hold at every level, we can t remove them o Competing needs depend on conditions at level S i : if two conditions are mutex because they are the negation of each other, then the mutex will hold at every level and so for the operations. Instead, if the conditions are mutex because of inconsistent support, by induction the monotonic increasing in #operations monotonically implies decrease in #mutexes 23

24 LEVELING OFF As a corollary of the previous properties, we see that the planning graph levels off o Consecutive levels become identical Proof: o Limit in the # of construction steps o Upper bound on #operations and #conditions based on problem definition o Lower bound of 0 on #mutexes 24

25 PLANNING GRAPH COMPLEXITY The graph is polynomial in the size of the planning problem P(L,A) (literals, actions) oeach S i has at most L nodes and L 2 mutex links oeach O i has at most A+L nodes, (A+L) 2 mutex links, and 2(AL+L) pre- and post-condition links oa graph with n levels has a size of O(n(A+L) 2 ) o The time to build the graph has the same complexity 25

26 PLANNING GRAPH RELAXED PROBLEM The relaxed problem provides an optimistic estimate of the number of steps to reach each sub-goal g i Heuristics! Level cost of sub-goal (literal) g i = level where g i first appears Level cost for g is an admissible heuristic estimate but it can be quite bad because level number of actions If any sub-goals g i fail to appear in the final level of the graph The problem is unsolvable ( LB) If all sub-goals appear Possibly there is a plan that achieves g, conditional that no mutexes exist 26

27 HEURISTICS FROM PLANNING GRAPHS To estimate the cost of a conjunction of sub-goals for using it as a heuristic in informed search: o Max level: max{level cost of any sub-goal} (clearly admissible) o Level sum: sum of level costs for all sub-goals o Set level: level at which all sub-goals appear without any pair of them being mutex Which is admissible? 1. Level sum 2. Set level 3. Both 4. Neither 27

28 THE GRAPHPLAN ALGORITHM 1. Grow the planning graph until all sub-goals are reachable (appear as conditions) and not mutex (If planning graph levels off first Fail) 2. Call EXTRACT-SOLUTION() on current planning graph (regression backward) 3. If none found, EXPAND-GRAPH by adding another one level to the planning graph and try again to extract the solution, starting backward from the new last level (new actions have been added and therefore they can be used for the regression) 28

29 THE GRAPHPLAN ALGORITHM function GRAPHPLAN(problem) return solution or failure graph INITIAL-PLANNING-GRAPH (problem) goals CONJUNCTS(problem.GOAL) nogoods an empty hash table for t = 0 to do if goals all non-mutex in S t of graph then solution EXTRACT-SOLUTION (graph, goals, NUMLEVELS(graph ), nogoods) if solution failure then return solution if graph and nogoods have both leveled off then return failure graph EXPAND-GRAPH (graph, problem) 29

30 THE GRAPHPLAN ALGORITHM function GRAPHPLAN(problem) return solution or failure graph INITIAL-PLANNING-GRAPH (problem) goals CONJUNCTS(problem.GOAL) nogoods an empty hash table for t = 0 to do if goals all non-mutex in S t of graph then solution EXTRACT-SOLUTION (graph, goals, NUMLEVELS(graph ), nogoods) if solution failure then return solution if graph and nogoods have both leveled off then return failure graph EXPAND-GRAPH (graph, problem) Each time EXTRACT-SOLUTION() fails to find a solution for a subset of goals at a given level, pair (level, subgoals) is stored in a map (hash table) as a new nogood entry During a search, if the same pair is being tried out again, the search can be aborted right away without searching again When a call to EXTRACT-SOLUTION() fails to find a solution, there must be at least one subset of goals that were no achievable, and marked as nogood. The hope is that by expanding the graph, in the next run, there ll be fewer nogoods If same nogood pairs are generated during the search, nogoods have leveled off 30

31 EXTRACT-SOLUTION Backward search where each state corresponds to an S level of the planning graph and a set of unsatisfied sub-goals Initial state is the last level of the planning graph, along with the goals of the planning problem Actions available at level S i are to select any conflict-free subset of operations in A i 1 whose effects cover the goals in the state Resulting state has level S i 1 and its goals are the preconditions for selected actions Goal is to reach a state at level S 0 31

32 EXTRACT-SOLUTION ILLUSTRATED Blue circles are sub-goals Different options to achieve sub-goals in terms of operations Plan: [2 ops] + [1 op] Plan: [1 op] + [2 ops] 32

33 EXTRACT-SOLUTION IN PRACTICE 1. If the current planning graph has n levels, start from the last state level n (set i = n), that includes all sub-goals not in mutex 2. At A i 1 list all minimal subsets of actions not in mutex that cover all sub-goals in S i 3. Select one of the subsets based on some criterion / heuristic 4. The preconditions of the actions in the selected subset define the new search state as a subset of the conditions in S i 1 5. Iterate 2-4 until the initial state is reached Output the solution path 6. If at any A i 1 regression fails because its not possible to cover the sub-goals in S i with a subset of actions not in mutex with each other, backtrack to the last action level with untried action subsets and try out a different subset (as in depth-first) 7. In case of repeated failures, the search will eventually move back to A n 1 where, again, a different subset can be tried out, if no subsets are remaining, the extract-solution() function fails S 2 : A 1 Subset 1 S 2 : A 1 Subset 2 It is useless to try out multiple times the same action at the same level (hash function to check) The outlined process proceeds as a depth-first, meaning that it can find a solution but not necessarily the optimal one (the one with minimal number of actions) Optimal search is needed, with the transition costs being defined by the number of actions in each subset 33

34 HEURISTICS FOR BACKWARD SEARCH EXTRACT-SOLUTION is still a difficult search problem, and the search space can be exponential in the worst-case! Heuristic should be used How to choose among the actions? One general approach: Greedy, based on the level cost of the literals in the set of sub-goals o Pick first the sub-goal with the highest level cost o Prefer actions with easier preconditions to achieve the selected sub-goal o Easier preconditions = Sum (or max) of the level costs of its preconditions is smallest 34

35 FLAT TIRE PROBLEM Not a safe neighborhood, they will steal the good tire from trunk! Russel and Norvig, Example: The spare tire problem 35

36 BACKWARD SEARCH goals actions states (conditions) 36

37 BACKWARD SEARCH, STEPS EXPLAINED Starting from S 0, that does not include the goal s literals, EXPAND-GRAPH is called twice. The literals from the goal are then present in the S 2, and EXPAND-SOLUTION is called Backward search starts at state S 2 with goal At(Spare, Axle) The only action choice for achieving the goal is PutOn(Spare, Axle) The regression brings to a search state S 1 with goals At(Spare,Ground), At(Flat,Axle) At(Spare,Ground) can only be achieved by Remove(Spare,Trunk) At(Flat,Axle) achieved either by Remove(Flat,Axle) or LeaveOvernight() LeaveOvernight() is mutex with Remove(Spare, Trunk) because of interference choose Remove(Spare, Trunk) and Remove(Flat, Axle) New search state at S 0 with the goals At(Spare,Trunk), At(Flat,Axle), that are both in the initial conditions Solution found! Plan: Remove(Spare, Trunk) and Remove(Flat, Axle) in level A 0, PutOn(Spare, Axle) in A 1. 37

38 GRAPHPLAN PROPERTIES Observation: The size of the t-level planning graph and the time to create it are polynomial in t, #operations, #conditions Theorem: GRAPHPLAN returns a plan if one exists, and returns failure if one does not exists The time to find the optimal plan by EXTRACT-SOLUTION can still be exponential in the worst-case (which is consistent with the fact that planning is PSPACE-complete) If only one feasible solution is to be found, then backward search can stop at the first feasible plan, if found The solution is in the form of a partially ordered plan: if at stage t the plan includes n actions, then they can be executed in any desired order, since they are not in conflict with each other 38

39 SUMMARY ON THE USE OF PLANNING GRAPHS Data structure: Polynomial-time construction providing a relaxed representation of the planning search tree GRAPHPLAN: to return a plan if one exists, or return failure if one does not exists. The solution is in the form of a partially ordered plan Extract domain-independent heuristics: e.g., Set level for A* Fast-forwarding planner: 1. consider a relaxed version of the original problem (e.g., by removing action preconditions) 2. solve the relaxed version using GRAPHPLAN 3. Use the length/cost of the solution provided by GRAPHPLAN as a heuristic value for A 39

40 SUMMARY ON CLASSICAL (AUTOMATED) PLANNING Concepts: o Planning, Propositional Logic representation (STRIPS, PDDL) o Problems with finding effective admissible heuristics o Planning graphs o Set level heuristic o P-SPACE complexity Algorithms: o GRAPHPLAN (BACKWARD SEARCH ON PLANNING GRAPH) o PLANNING GRAPHS + A* o FAST-FORWARDING PLANNERS o FORWARD, BACKWARD SEARCH Big ideas: o Planning is search, but admits domain-independent heuristics 40