Searching becomes much more demanding:

Transcription

1 Searching becomes much more demanding: If S has n nodes, then its B can have up to 2 n 1. If S is infinite, then even B 0 and other nodes of B can be infinite too. Generating successors(b) could involve going through all the s B one by one. We can try to address such demands with logic: Represent each belief set B as a logical formula φ B which describes exactly the states s B. E.g. for the vacuum-cleaner R we would represent B 0 as True B 1 as R is in the rightmost location B 2 as R is in the rightmost location. the rightmost location is clean We can often describe an infinite B with a finite φ B by mentioning only what is certain in B and leaving out all that is not. Suppose that we have described the necessary preconditions for doing an action a as another logical formula ψ a. Then action a can be chosen in B if and only if φ B implies ψ a and this is a logical inference task. But it is less clear whether the description φ B for the outcome B of the action B a B should be formed within... logic because this + allows the actions a to be represented declaratively as separate rules used by the search algorithm needs an expressive enough logic with more complicated inference or in the algorithm computing successors(b) because then the actions a are described as program code instead of separate rules. These belief state spaces B can also be used in stochastic state spaces S, where the same action a can have several outcomes, like s success a s a s failure. However, B does not include the frequencies or probabilities of the different outcomes each outcome is considered equally likely. This complicates learning about the environment E and adapting to it. One might call such a B nondeterministic instead of stochastic. The field of belief revision is studied in both AI (e.g. in the theory of Truth Maintenance Systems (TMS)) and Philosophy (e.g. as formal epistemology). There we distinguish between two major kinds of changes to R s beliefs: 44

2 Update when R s environment E changes. Here R s own actions a lead to such updates. Revision where E stays the same, but R s beliefs about it change. E.g. the user (or another robot...) says to the vacuum-cleaner I was at the other location, and I didn t see any dirt there. In general, R can have preferences among its beliefs: I believe this more strongly than that. Hence if I must change my beliefs, I am more willing to stop believing that and keep believing this, if possible. These flat belief sets B do not include such preferences. An important research question is the relation between such preferences and probabilities. This course and its book (Russell and Norvig, 2003, Chapter 10.8) touch these interesting but intricate issues only briefly. If the agent R can observe the outcome of its actions a afterwards, then we move from searching into planning, where the solution sought is no longer just a sequence of actions a B 1 a 0 2 a B1 3 a B3 last Bgoal but a tree- (or even graph-)like structure such as perform action a 1 if it succeeded then perform action a 2 if it succeeded then... else try something else than a 2 else try something else than a Informed Search Informed search is guided (either only or partially) by the following information: Evaluation function f(node) meaning that nodes with smallest values look more promising. So we... add to our search tree nodes a new field initialized with node.evaluation f(node) use tree- or graph-based UCS wrt. this new field node.evaluation as our general algorithmic idea. Heuristic function h(node.state) is a key component in most f. We design it to be... an approximation of the unknown function h (s) = pathcost from s into its nearest goal node based on what we know about this search space S easy to compute 45

3 Bucharest Craiova Dobreta Eforie Fagaras Giurgiu Hirsova Iasi Lugoj Mehadia Neamt Oradea Pitesti Urziceni Vaslui Figure 24: Straight-line distances to Bucharest (Russell and Norvig, 2003, Figure 4.1) fairly accurate so that it can guide the search efficiently. E.g. in the Romanian road problem a natural heuristic is the geographical straightline distance h SLD (town) to Bucharest as the crow flies ( linnuntietä in Finnish) in Figure 24. (Our agent R might measure these distances from its map with a ruler.) The greedy best-first search (GBFS) algorithm always minimizes this h as much as possible: its f(node) = h(node.state). In the Romanian road problem, this means always expanding the geographically closest town to Bucharest available in the current fringe, as shown in Figure 25. GBFS is in general similar to DFS: Not optimal because it can e.g. choose a long spiral around the goal. Not complete because it can even get stuck expanding an infinite hyperbolic spiral around the goal. Time and space complexity is in theory similar to UCS, Eq. (5) practice very much dependent on S and h it might be lucky and find a goal fast, or not The A algorithm Another natural evaluation function is f(node) = g(node) + h(node.state) (7) 46

4 (a) The initial state (b) After expanding (c) After expanding Fagaras Oradea (d) After expanding Fagaras Fagaras Oradea Bucharest Figure 25: Greedy best-first search example. (Russell and Norvig, 2003, Figure 4.2) 47

5 where this g(node) = the pathcost from the root into this node is the cost of getting into this node, and then h estimates the additional cost that would be needed get from this node into the nearest goal. Together the estimate the cost of the solution we get if we expand this branch further. In code: node.evaluation node.pathcost +h(node.state). UCS with this node.evaluation is called the A search algorithm. It is the most well-known heuristic search algorithm. Figure 26 applies it to the Romanian road problem with h SLD. Another view to A is that consist of two parts: f(node) = UCS which proceeds cautiously to be optimal UCS part {}}{ g(node) + h(node.state) }{{} GBFS part GBFS which can find a goal fast but this is not guaranteed. A is optimal if h satisfies an extra condition: kind of A kind of h tree-search as in Figure 15 admissible as in Section graph-search as in Figure 22 consistent as in Section Intuitively, then the GBFS part cannot fool the UCS part into accepting a quick but suboptimal solution. A will first expand all nodes such that and then some nodes such that f(node) < the pathcost C of the optimal solution (8) f(node) = C (9) before expanding and reporting a goal node from its fringe. A is even optimally efficient in the following sense: Let A be any other search algorithm, which also works by expanding search paths from the root (and not according to some radically different principles). If A is also optimal, then it too must expand all the inner nodes of Eq. (8). Then A can only be faster in finding a goal among the boundary nodes of Eq. (9). However, A inherits also the big memory requirements from UCS. 48

6 (a) The initial state (b) After expanding 366= = = = (c) After expanding 447= = Fagaras Oradea 646= = = = (d) After expanding 447= = Fagaras 646= = Oradea 671= Craiova Pitesti 526= = = (e) After expanding Fagaras 447= = = Fagaras Oradea 671= Bucharest Craiova Pitesti 591= = = = = (f) After expanding Pitesti 447= = = Fagaras Oradea 671= Bucharest Craiova Pitesti 591= = = = Bucharest Craiova 418= = = Figure 26: A search example. (Russell and Norvig, 2003, Figure 4.3) 49

7 3.2.2 Admissibility The tree-search A algorithm is optimal, if h is admissible: for all states s in the search space S. That is, if h is always optimistic about how good s is. E.g. the most optimistic heuristic gives UCS, which is optimal. h(s) h (s) (10) h(s) = 0 E.g. the heuristic h SLD is admissible in the Romanian route problem. The general reasoning goes as follows: Let G be a suboptimal goal node in the fringe: because f(g ) = g(g ) + h(g ) > C g(g ) > C since G is suboptimal h(g ) = 0 since G.State is a goal and h is optimistic. Let n be any node in an optimal solution path: because h is optimistic. f(n) = g(n) + h(n) C Hence every such n must have been expanded before G, and therefore treesearch A must have reported an optimal solution before it would expand G Consistency Optimism is not enough to guarantee the optimality of the graph-search A algorithm: Heuristic h might be more optimistic about an initial part B of a suboptimal solution than the initial part B of an optimal solution. Then the algorithm might expand B before B......and expand B with some node whose node.state would also be needed in B......but it is no longer available, because node.state becomes closed first. To avoid this, we must make sure that when we are expanding a node, we are also expanding the optimal path to its node.state. This could be ensured by redesigning the algorithm so that 50

8 each state s of S appears in at most one node this unique node is updated whenever we find a better path into s. The result would be similar to Dijkstra s shortest path algorithm in TRA. Or we can redesign the heuristic h to be consistent or monotonic instead: for all actions n a m in S. Eq. (11) is a triangle inequality in S: h(n) stepcost(n, a,m) + h(m) (11) Consider any triangle with corners X, Y and Z in ordinary planar geometry. There the length of the edge directly from X to Z the length of the edge from X to Y + the length of the edge from Y to Z. Now choose X = n, Y = m and Z = the closest goal state from both n and m. Eq. (11) continues to hold even if n has some even closer goal state than Z. Now the general reasoning can be developed as follows: When h is consistent, then the f-values along any path are nondecreasing: n a m f(n) = g(n) + h(n) from which by Eq. (11) =g(m.state) {}}{ g(n) + stepcost(n.state, a, m.state) +h(m.state). (12) }{{} =f(m) Then branch B has no time to steal any node from branch B. Hence A draws concentric contours of equal f-value in the state space S as shown in Figure 27. More accurate heuristics h tend to stretch these contours from circles around the start state s 0 into ovals reaching towards the nearest goal state. E.g. h SLD is also a consistent heuristic for the Romanian route problem. If a heuristic h is consistent, then h must also be admissible. The converse does not quite hold: There are heuristics h which are admissible but not consistent......but they are mostly artificially constructed counterexamples. Admissible natural heuristics are in most cases consistent as well. 51

9 O Z N A I 380 S 400 F V T R L P M U H D C 420 G B E Figure 27: Map of Romania with some f-contours. (Russell and Norvig, 2003, Figure 4.4) The Pathmax Equation If heuristic h happens to be admissible but not consistent, then A can use the pathmax equation which corrects this on the fly. The idea is to program A so that the next node generated always satisfies Eq. (12): next.evaluation max(next.parentnode.evaluation, Then we have guaranteed that next.pathcost +h(next.state)). (13) next.evaluation next.parentnode.evaluation. That is, if h(next.state) threatens to be too optimistic, then A dampens its enthusiasm internally. The downside is that A also dampens the ability of h to guide the search On Memory-Bounded Heuristic Search The simplest way to reduce the memory requirements of A is to apply the iterative deepening idea from Section The resulting algorithm IDA is like ILS, but returns the node.evaluation instead, when the node.pathcost exceeds the current limit. Its behaviour is also similar in general, but a good heuristic h can sometimes help a lot. 52

10 1 create the root node as before 2 if RBFS(root, + ) returns some Solution 3 then return the same Solution 4 else return NoSolution RBFS(node, limit) : 1 if goaltest(node.state) 2 then return this Solution 3 expand this node into all its next nodes as before using the pathmax Eq. (13) 4 if this node did not expand into any next nodes 5 then return WasExceeded + 6 else best the next node whose next.evaluation is smallest of all 7 while best.evaluation limit 8 do second the second smallest next.evaluation value 9 if RBFS(best, min(limit, second)) returns WasExceeded l 10 then best.evaluation l 11 best the next node whose next.evaluation is smallest of all 12 else return the same Solution 13 return WasExceeded best.evaluation Figure 28: Recursive Best-First Search (Russell and Norvig, 2003, Figure 4.5). Recursive Best-First Search (RBFS) in Figure 28 is better, because it remembers more than just the current f-value limit: It remembers also the best alternative computed f-value along the path from the root into this node in the current search tree T as its limit parameter. Backtracking from this node can then continue rapidly from this best alternative. This is a simple form of intelligent backtracking studied in AI search. The actual best alternative subtrees of T are discarded and regenerated as needed to save memory, as shown in Figure 29. These best alternatives can be remembered by updating the node.evaluation fields as the search progresses. Some properties of RBFS: Optimal if h is admissible. RBFS adds consistency on the fly with the pathmax Eq. (13). Time complexity depends on how often RBFS changes its mind and backtracks from one branch into another and maybe back again. Each change of mind regenerates previously discarded nodes. Space complexity is the same as for DFS. When RBFS backtracks, it gives away the memory allocated for the disappearing node. 53

11 (a) After expanding,, Fagaras Oradea Craiova Pitesti (b) After unwinding back to and expanding Fagaras Fagaras Oradea Bucharest 450 (c) After switching back to and expanding Pitesti Fagaras Oradea Craiova Pitesti Bucharest Craiova Figure 29: RBFS example with the limit parameter boxed and updated values crossed. (Russell and Norvig, 2003, Figure 4.6) 54

12 If there is enough memory, it might be more efficient to leave this node in memory instead, so that RBFS would not have to regenerate it, if RBFS changes its mind later. Hence RBFS does not utilize all the available memory as fully as possible. Simplified Memory-bounded A (SMA ) fixes that: it......uses a preallocated pool[0...m] of nodes, where the constant M = the amount of memory we want to give....always chooses the next leaf to expand = the best according to the following comparison....compares nodes lexicographically wrt. the following fields: 1. node.evaluation as before, which is initialized with the pathmax Eq. (13) updated as in RBFS when a leaf is discarded. 2. inverse node.depth that is, deeper is better so that SMA uses its pool to store even the longest paths that fit. 3. inverse node.creationtime so that a recently created node is not discarded too soon to be expanded....discards an old node only when the next node must be stored in an already full pool. This old node = the worst according to the preceding comparison among all leaves those nodes who have no successors currently in the pool. Note that a node becomes a leaf again when all its successors have been discarded. Then it can be either expanded again or discarded too. Figure 30 shows an example. The SMA algorithm sketched here and in Figure 31 is... complete if its pool can hold the shortest solution: if M d + b where these extra b locations are needed for expanding one node at a time. Even these b extra locations are not needed, if the successors(node.state) can be computed incrementally one by one instead. optimal in the given memory, because it returns the best solution which fits into its pool. practical in many but not all situations, where A spends space and RBFS spends time in considering many different solution candidates. 55

13 Figure 30: SMA example with M = 2. (Russell and Norvig, 1995, Figure 4.11) 56

14 Figure 31: SMA code sketch. (Russell and Norvig, 1995, Figure 4.12) 57

15 3.2.5 On Inventing and Analyzing Heuristics Let us consider heuristics h using the 8-puzzle in Figure 9 as the example. This problem has roughly d 22 for a randomly generated start state e.g. here we need 26 moves to get to the goal (Russell and Norvig, 2003, Figure 4.7) b 3. One systematic way to invent a heuristic h is by 1. relaxing the original problem statement by forgetting some of its rules 2. setting h(r) = the cost of an optimal solution to the instance r of this relaxed problem. This h is then admissible for the original problem: If P is an optimal solution to the original problem, then P is also one solution to the relaxed problem which in general has some other optimal solution Q. A good h is such that pathcost(q) is not much smaller than pathcost(p), so that Q is a good clue about P. Intuitively, the search is then responsible for reintroducing these forgotten rules, so that it returns P instead of Q. E.g. h SLD was obtained by relaxing the rule cars cannot fly in the Romanian route problem. E.g. the 8-puzzle rules are of the form a tile can move from square A into square B if 1. A is (horizontally or vertically) adjacent to B and 2. B is blank. If we forget just rule 2, then we get the Manhattan (city-block) distance heuristic 8 h 2 (s) = how far is tile number t from its correct square in s t=1 where these distances are computed as horizontal and vertical steps. E.g. here in Figure 9 h 2 (Start State) = = 18. If we forget both rules 1 and 2 instead, then we get the misplaced tiles heuristic h 1 (s) = how many of the tiles in s are in a wrong square which gives here in Figure 9 How can we choose between h 1 and h 2? h 1 (Start State) = 8. 58

16 Figure 32: Comparing h 1 and h 2. (Russell and Norvig, 2003, Figure 4.8) An easy answer is that we can take them both: Since both h 1 and h 2 are admissible, then so is h 3 (s) = max(h 1 (s), h 2 (s)) (14) and this h 3 always uses the less optimistic and therefore more precise of the two for any given s. A more detailed answer is that is this case h 2 dominates h 1 : h 2 (s) h 1 (s) for all s. Hence Eq. (14) uses h 2 instead of h 1, and therefore we can always use h 2 and forget h 1 altogether. Domination translates directly to performance as shown in the left half of Figure 32. But in general we might have developed several admissible heuristics, none of which dominates any of the others. The we can combine them together with max by Eq. (14). One numeric measure for the goodness of a heuristic h is its effective branching factor b : Suppose that A generated N more nodes before reporting the goal on some test input s 0. Then b = the branching factor for a full tree with N + 1 nodes and height goal.depth the uninformed BFS finding the same goal. 59

17 This b is the (approximate) solution of N + 1 = goal.depth l=0 (b ) l = (b ) goal.depth b 1 This b is not a whole number, but the average number of children a search tree node has an intuitive measure of the remaining uncertainty after h has been used to guide the search (compared to b). A smaller b means a better heuristic h as can be seen from the right side of Figure