Creating Admissible Heuristic Functions: The General Relaxation Principle and Delete Relaxation

Transcription

1 Creating Admissible Heuristic Functions: The General Relaxation Principle and Delete Relaxation

2 Relaxed Planning Problems 74 A classical planning problem P has a set of solutions Solutions(P) = { π : π is an executable sequence of actions leading from to a state in } Suppose that: P is a classical planning problem P is another classical planning problem Solutions(P) Solutions(P ) If a plan achieves the goal in P, the same plan also achieves the goal in P But P can have additional solutions! Then P is a relaxation of P The constraints on what constitutes a solution have been relaxed

3 Relaxation Example 1 75 Example 1: Adding new actions All old solutions still valid, but new solutions may exist Modifies the state transition system (states/actions) This particular example: shorter solution exists on(b,a) on(a,c) Goal

4 Relaxation Example 2 76 Example 2: Adding goal states All old solutions still valid, but new solutions may exist Retains the same states and transitions This particular example: Optimal solution retains the same length Goal on(b,a) on(a,c) or on(c,b) Goal on(b,a) on(a,c) or on(c,b)

5 Relaxation 77 Definition of relaxation: Change the problem so that all old solutions remain valid, new solutions become possible! An optimal solution for the relaxed problem can never be more expensive than an original optimal solution You ve seen relaxations adapted to the 8-puzzle What can you do independently of the planning domain used?

6 Delete Relaxation (1) 78 One domain-independent technique: delete relaxation Assume a classical problem with: Positive preconditions Positive goals Both negative and positive effects Relaxation: remove all negative effects (all "delete effects")! before transformation: after transformation:

7 No physical meaning! Delete Relaxation (2): Example 79 State space: Original problem State space: delete-relaxed problem Notice: Transitions are not added but moved! Any action applicable (or goal satisfied) after unstack(a,c) in the original problem is applicable (satisfied) after unstack(a,c) here :strips preconds/goals are positive!

8 Delete Relaxation (3): Heuristic 80 So given any action sequence: If it is executable in P, it is executable in delete-relaxed P If it results in a goal state in P, it results in a goal state in del-relaxed P This is a relaxation! Easy to apply mechanically Remove all negative effects ( the delete effects are relaxed/removed ) If only this relaxation is applied: Gives us the optimal delete relaxation heuristic, h+(n) h+(n) = the cost of an optimal solution to a delete-relaxed problem starting in node n Need a heuristic value for every expanded state calculating h+(n) still takes too much time (requires planning)!

9 Important Issues 81 Important: You cannot use a relaxed problem as a heuristic. What would that mean? You use the cost of an optimal solution to the relaxed problem as a heuristic. Solving the relaxed problem can result in a more expensive solution inadmissible! You have to solve it optimally to get the admissibility guarantee. You don t just solve the relaxed problem once. Every time you reach a new state and want to calculate a heuristic, you have to solve the relaxed problem of getting from that state to the goal.

10 The h 1 Heuristic

11 The h 1 Heuristic 83 We need a faster heuristic! How about relaxing h+ further? ) (optimal delete relaxation): Remove delete effects, find a long plan achieving all goals Find a separate plan for each goal fact, in a way that ignores delete effects Take the maximum of their costs Size: Grows quickly with plan length Much easier/faster, since search trees tend to be wide!

12 The h 1 Heuristic: Example 84 Cheaper! We discover this is expensive

13 The h 1 Heuristic: Important Property 85 Each goal considered separately! This is why it is fast! No need to consider interactions no combinatorial explosion Each precondition considered separately! Can generalize, consider all pairs of goals/preconds Yields the h 2 heuristic, more powerful, still reasonably fast Each precondition considered separately!

14 Search Strategies for Admissible Heuristics 86 Given an admissible heuristic, we often use A* (not always) Guarantees optimal plans, just like Dijkstra Comparison: Let len(p) be the number of actions in plan p Let h(p) be an admissible heuristic (applied to the final state of plan p) Blocks world, blocks initially on the table, goal is a -block tower 399 blocks to pick up and stack Dijkstra considers plans where len(p) <= 798 A* considers plans where len(p) + h(p) <= 798 len(p)=400 h(p)=398 Explored by Dijkstra, not by A* len(p)=401 h(p)=398 len(p)=401 h(p)=397

15 Admissible Heuristics 87 Admissible heuristics: Must never overestimate To guarantee this, they often underestimate (by a large margin)! Example: h1 -- could have hundreds of goals but we can only count the most expensive one! Achieving clear(d) seems exactly as expensive as achieving all of the goals above Admissible heuristics help, but we d prefer to cut off search even earlier!

16 Non-admissible Heuristics for Non-Optimal Forward State Space Planning

17 Optimal and Satisficing Planning 89 Do we need admissibility? If we must guarantee optimal plans: Need optimal search strategies such as A* Must use admissible heuristics If we are satisfied with a high-quality plan: Satisficing planning Find a plan that is sufficiently good, sufficiently quickly Can use non-admissible heuristics which can often be more informative (give us more help in finding reasonably good alternatives) Generally much faster Handles more complex domains, larger problem instances

18 h add : Use sum instead of maximum 90 Cheaper! Turns out to be expensive

19 The h add Heuristic: Properties 91 The h add heuristic: Can under- and over-estimate: Not admissible! The same action can be counted twice Can miss the opportunity to achieve two goals with one action but we could have used a single teardownalltowers action! But: Retains much more information Example: Every unsatisfied goal contributes to the heuristic value + = 5

20 Search Algorithms for h add 92 Which search algorithm to use? A* expands many nodes (expensive), and for h add, there is still no optimality guarantee! HSP planner, and some others, use Hill Climbing Successors: States created by applying an action to the current state Value: Calculated as h add (maximize h add minimize +h add ) Only considers children of the current node tries to move quickly towards low heuristic values doesn t examine all nodes with len(p)+f(p) <= some value

21 Local Optima in Planning Problem: 93 Can end up in a local maximum (where h() is locally minimal) This is not good enough : The goal is not satisfied! Possible solution: Restart search from another expanded state

22 Local Optima: Restarting Init 94 Local Optimum on(b,a) on(a,c) Goal Restart in any of the gray nodes

23 Search Algorithms for h add 95 Late s: State-space planning too simple to be efficient! Most planners used very elaborate and complex search methods Planning competition : HSP (hill climbing + h add + tweaks) solved more problems than most other planners Now, heuristic forward search is the predominant approach!

24

25 Backward Search 97 Classical Planning: Find a path in a finite graph initial Where would we have to be before? What about this direction: Backward, goal initial? goal goal goal What could the previous action be? If we want to be here

26 Simple Backward Search (1) 98 Simple case: A single, completely defined goal state! This must have been true before The goal is not already achieved was executed This must have been true before was executed

27 Simple Backward Search (2) 99 Simple case: A single, completely defined goal state! Relevant actions: Now these states are goals

28 Simple Backward Search (3) 100 Analysis But we don t know how to reach them, or even if it is possible! If we reach one of these states, we know how to continue to the goal! We still need guidance: Heuristics, or something else

29

30 Reachable States 102 At every step, there is a set of states we can reach Example: Blocks World with blocks A/B Step 0: Step 1: Step 2: Step 3: s0 s1, s2 s0, s3, s4 (not s1 or s2!)

31 Reachable States (2) 103 Example: Blocks World, 3 blocks Step 0: s0 Step 1: s1, s2, s3 Step 2: s0, s4, s5, s10, s11, s16, s17 Would take a lot of time to calculate!

32 Possibly Reachable States (1) 104 We can quickly estimate the sets of reachable states Contains all truly reachable states, and some others (exact calculations are too costly!) Goal not satisfied Goal not satisfied in any state Goal not satisfied in any state Goal satisfied in The goal might be achievable using actions!

33 Possibly Reachable States (2) 105 We can use this to shrink the backward search space The goal seems to be reachable here, but we can t be sure (overestimating!)

34 Possibly Reachable States (3) 106 Backward search can then verify what is truly reachable In a much smaller search space than plain backward search The goal was not reachable here! Also not reachable! These are all of the relevant actions none is applicable at time 2!

35 Possibly Reachable States (4) Extend one time step and try again A larger number of states may be truly reachable then 107 Time 4 Continue backward search (complete search procedure with backtracking, etc) Use possibly reachable states to prune the search tree smaller search space

36 Planning graph! GraphPlan: Planning Graph 108 This is the idea behind GraphPlan! Also keeps track of possibly executable actions Proposition levels Which facts might possibly be true in each step? Details are in the book! Initial state 10 possibly executable actions 47 possibly executable actions 1200 possibly executable actions Action levels Which actions may possibly be executed in each step?

37 GraphPlan and Heuristics 109 When GraphPlan was introduced: Very successful, surprisingly fast Current use of planning graphs: The basis of very successful (non-admissible) heuristics, more informative than h add! Calculate heuristic value for state s: Apply delete relaxation Create planning graph starting in state s Backward search finds a solution to the relaxed problem No delete effects smaller graph, simpler calculations No delete effects smaller graph, no backtracking Cost of solution = heuristic value!

38

39 Plan-Space Search 111 Just because we are looking for a path in this graph initial goal goal goal doesn t mean we have to use it as a search space! What if a search step could insert actions anywhere in a plan?

40 initaction goalaction Partial-Order Planning 112 Partial-order planning starts with: A fake initial action, whose effects are the facts of the initial state A fake goal action, whose preconditions are the conjunctive goal facts clear(a) ontable(a) clear(b) ontable(b) clear(a) on(a,b) on(b,c) clear(c) on(c,d) ontable(d) handempty A set of flaws! First type: Open goals to be achieved on(c,d) ontable(d) handempty An ordering relation between these actions

41 initaction goalaction Resolving Open Goals To resolve an open goal: Alternative 1: Add a causal link from an action that already achieves it clear(a) ontable(a) clear(b) ontable(b) clear(a) on(a,b) on(b,c) clear(c) on(c,d) ontable(d) handempty on(c,d) ontable(d) handempty

42 initaction goalaction stack(a,b) Resolving Open Goals To resolve an open goal: holding(a) clear(b) holding(a) clear(b) Alternative 2: Add a new action with the open goal as an effect clear(a) on(a,b) clear(a) ontable(a) on(a,b) clear(b) ontable(b) handempty clear(a) on(b,c) clear(c) on(c,d) on(c,d) ontable(d) ontable(d) handempty handempty

43 pickup(b) stack(b,c) initaction goalaction stack(a,b) Partial Orders 115 Some pairs of actions can remain unordered holding(a) holding(a) clear(b) clear(b) clear(a) on(a,b) clear(a) ontable(a) on(a,b) clear(b) handempty on(b,c) ontable(b) clear(a) clear(c) on(c,d) ontable(d) handempty handempty clear(b) ontable(b) holding(b) clear(c) on(b,c) on(c,d) ontable(d) handempty handempty holding(b) holding(b) handempty clear(b) clear(c) clear(b)

44 pickup(b) stack(b,c) initaction goalaction stack(a,b) Threats 116 Second flaw type: A threat to be resolved initaction supports clear(b) for stack(a,b) there s a causal link pickup(b) deletes clear(b), and may occur between initaction and stack(a,b) We can t be certain that clear(b) still holds when stack(a,b) starts! holding(a) holding(a) clear(b) clear(b) clear(a) on(a,b) clear(a) ontable(a) on(a,b) clear(b) handempty on(b,c) ontable(b) clear(a) clear(c) on(c,d) ontable(d) handempty handempty clear(b) ontable(b) holding(b) clear(c) on(b,c) on(c,d) ontable(d) handempty handempty holding(b) holding(b) handempty clear(b) clear(c) clear(b)

45 Resolving Threats How to make sure that holds when starts? Alternative : The action that disturbs the precondition is placed after the action that has the precondition Only possible if the resulting partial order is consistent (acyclic)!

46 Resolving Threats Alternative 2: The action that disturbs the precondition is placed before the action that supports the precondition Only possible if the resulting partial order is consistent not in this case!

47 Heuristics 119 Again, this is not sufficient in itself Heuristics are required But it has advantages No need to commit to order

48

49 Example Questions 121 Example questions: Describe: Relaxation Delete relaxation The optimal relaxation heuristic The different kinds of flaws that can occur in a partially ordered plan Why: Don t planners use Dijkstra s algorithm to search the state space? Can the h_1 heuristic be calculated more quickly than the optimal delete relaxation heuristic h+? Is the number of open goals an admissible heuristic? Why / why not? Describe one important difference between backward and forward search Given a domain, problem and initial state: Expand the forward state space by two levels Expand the backward state space by two levels

50 TDDD48 Automated Planning (1) 122 Formal basis for planning Alternative representations of planning problems Simple and complex state transition systems Deeper discussions about all of these topics Completely different principles for heuristics Landmarks Pattern databases Alternative search spaces Planning by conversion to boolean satisfiability problems Extended expressivity Planning with time and concurrency Planning with probabilistic actions (Markov Decision Processes) Planning with non-classical goals

51 TDDD48 Automated Planning (2) Example: concurrent actions with duration 123 Duration dependent on parameters, current state Conditions / effects at the start or end of the action Implemented by quite a few planners affects algorithms and heuristics!

52 TDDD48 Automated Planning (3) 124 Planning with domain knowledge Using what you know: Temporal control rules Breaking down a task into smaller parts: Hierarchical Task Networks The travel task has a method called air-travel travel(x,y) air-travel(x,y) Task Method buy-ticket (airport(x), airport(y)) travel (x, airport(x)) fly(airport(x), airport(y)) travel (airport(y), y) Alternative types of planning Path planning And so on