S A E RC R H C I H NG N G IN N S T S A T T A E E G R G A R PH P S

Transcription

1 LECTURE 2 SEARCHING IN STATE GRAPHS

2 Introduction Idea: Problem Solving as Search Basic formalism as State-Space Graph Graph explored by Tree Search Different algorithms to explore the graph Slides mainly taken from [RusselNorvig2010], cf. chapter 3 for more details (few) [Poole Mackworth2010], cf. chapter 3.1 to 3.6

3 2 types of search Uninformed or blind search no clue whether one node is better than any other search blindly and exhaustively taking into account only the path already explored depth-first search, breadth-first search, uniform-cost search Informed search Try to take the goal into account Need heuristic function to approximate distance to the goal guided by heuristic, but heuristic might be wrong (greedy) best-first search, A* algorithm

4 Example: Romania

5 Example: Romania On holiday in Romania; currently in Arad. Flight leaves tomorrow from Bucharest Formulate goal: be in Bucharest Formulate problem: states: various cities actions: drive between cities Find solution: sequence of cities, e.g., Arad, Sibiu, Fagaras, Bucharest

6 State Space Problem Formulation A problemis defined by four items: 1. set of states, e.g., "at some city 2. initial state, e.g., "at Arad 3. actions or successor function S(x) = set of action state pairs, e.g., S(Arad) = {<Arad Zerind, Zerind>, } or simply S(x) = successor states, e.g., S(Arad) = {Zerind, Sibiu, Timisoara } 4. goal test, can be explicit, by giving a list of nodes: e.g., x = "at Bucharest" implicit, by Boolean function: e.g., Checkmate(x) option:path cost(additive) e.g., sum of distances, number of actions executed, etc A solutionis a sequence of actions leading from the initial state to a goal

7 Another Example: The 8-puzzle states? actions? goal test? path cost?

8 Another Example: The 8-puzzle states? locations of tiles, e.g. (7,2,4,5,Ø,6,8,3,1) actions? move blank left, right, up, down goal test? goal state is given: (Ø,1,2,3,4,5,6,7,8) path cost? 1 per move [Note: optimal solution of n-puzzle family is NP-hard]

9 From Problems to Graphs a (directed) graph is a couple (N,A) where Nis a set of nodes Aof ordered pairs (couples) of nodes from Acalled arcs. a pathfrom node sto node gis a sequence of nodes n 0, n 1,..., n k s.t. s=n 0, g=n k, and n i-1,n i A A cycleis a nonempty path such that the end node is the same as the start node arcs can be labeled by a costfunction : A R+ For a path p = n 0, n 1,..., n k, the cost of path pis cost(p) = cost( n 0,n 1 ) cost( n k-1,n k ) Unlabeled arcs n,n A, cost ( n,n ) = 1

10 Graph defined by a problem Nodes:set of nodes generated by iteratively applying actions from starting node N = { n / k N, n=s K (n₀)}, for initial state n₀ Arcs: defined by successor function (i.e. actions) A = { (n,s(n)) / n N} Graph is big! e.g., for 8-puzzle problem: 9! = 362,880 states Graph is implicit Graph can be infinite

11 Tree search algorithms Basic idea: simulated exploration of state space by generating successors of already-explored states (a.k.a. expanding states)

12 [Poole Mackworth 2010]

13 Search strategies A search strategyis defined by the order of node expansion Strategies are evaluated along the following dimensions: completeness: does it always find a solution if one exists? time complexity: number of nodes generated space complexity: maximum number of nodes in memory optimality: does it always find a least-cost solution? Time and space complexity are measured in terms of b: maximum branching factor of the search tree d: depth of the least-cost solution m: maximum depth of the state space (may be ) l: the depth limit (for Depth-limited complexity) C*: the cost of the optimal solution (for Uniform-cost complexity) ε: minimum step cost, a positive constant (for Uniform-cost complexity)

14 Depth-First Search Strategy: expand deepest unexpanded node Implementation: put new successor nodes in front of frontier i.e. select last element added to the frontier i.e. treat frontier as a stack, i.e. LIFO

15 [Poole Mackworth 2010]

16 Properties of depth-first search Complete?No: fails in infinite-depth state-spaces, and state-spaces with loops But can be modified to avoid repeated states along path complete in finite spaces Time?O(b m ): terrible if mis much larger than d but if solutions are dense, may be much faster than breadth-first Space? O(bm), i.e., linear space! Optimal? No

17 Breadth-First Search Strategy: expand shallowest unexpanded node Implementation: put new successor nodes at end of frontier i.e. select first element added to the frontier i.e. treat frontier as a queue, i.e. FIFO

18 [Poole Mackworth 2010]

19 Properties of breadth-first search Complete? Yes (if b is finite) Time?1+b+b 2 +b 3 + +b d + b(b d -1) = O(b d+1 ) Space?O(b d+1 )(keeps every node in memory) Optimal?Yes (if cost = 1 per step) Space is the bigger problem (more than time)

20 Uniform-cost search Breadth-first is only optimal if path cost is a non-decreasing function of depth: for all nodes n d-1 at depth d-1 and n d at depth d cost( n 0,..., n d ) cost( n 0,..., n d-1 ) e.g., constant step cost, as in the 8-puzzle (cost 1 per move) Can we guarantee optimality for any positive step cost? Uniform-cost Search: Strategy: expand node with smallest path cost g(n). Implementation: use a priority queue, i.e., new successors are merged into the queue sorted by g(n) Russel Norvig 2010]

21 Uniform-Cost (lower-first) search Complete? Yes, if step cost ε (i.e. strictly positive) Time?number of nodes with g cost of optimal solution, O(b ceiling(c*/ ε) )where c * is the cost of the optimal solution Space?number of nodes with g cost of optimal solution, O(b ceiling(c*/ ε) ) Optimal?Yes nodes expanded in increasing order of g(n)

22 Depth-Limited Search = depth-first search with depth limit l, i.e., nodes at depth lhave no successors

23 Iterative Deepening Search

24 Iterative deepening search l =0

25 Iterative deepening search l =1

26 Iterative deepening search l =2

27 Iterative deepening search l =3

28 Properties of iterative deepening Complete? Yes search Time?(d+1)b 0 + d b 1 + (d-1)b b d = O(b d ) Space? O(bd) Optimal?Yes, if step cost = 1

29 Comparison IDS / DLS Number of nodes generated in a depth-limited search to depth dwith branching factor b: N DLS = b 0 + b 1 + b b d-2 + b d-1 + b d Number of nodes generated in an iterative deepening search to depth d with branching factor b: N IDS = (d+1)b 0 + d b 1 + (d-1)b b d-2 +2b d-1 + 1b d Example: For b = 10, d = 5, N DLS = , , ,000 = 111,111 N IDS = , , ,000 = 123,456 Overhead = (123, ,111)/111,111 = 11%

30 Summary of algorithms

31 Repeated states Failure to detect repeated states can turn a linear problem into an exponential one!

32 S Repeated States: Cycles B B S C C State Space Graph search C S B S Example of a Search Tree never generate a state generated before must keep track of all possible states (uses a lot of memory) e.g., 8-puzzle problem, we have 9! = 362,880 states approximation for DFS/DLS: only avoid states in its (limited) memory: avoid looping paths. Graph search optimal for BFS and UCS, not for DFS. optimal but memory inefficient

33 Graph search

34 Heuristic Search Take advantage of knowledge of the goal Some nodes might be closer to the goal than others Guide the search by some heuristic commonsense rule(s) intended to increase the probability of solving rules of thumb Heuristic function h(n): N R+ Estimate of cost of going fromnode nto the goal Example: for route finding, use air-distance for h Algorithms: (Greedy) Best-First, A*

35 Example: Romania with step costs

36 Greedy Best-First Search Evaluation function f(n) = h(n) (heuristic) estimate of cost from nto goal e.g., h SLD (n)= straight-line distance from nto Bucharest Greedy best-first search expands the node that appearsto be closest to goal

37 Greedy best-first search example

38 Greedy best-first search example

39 Greedy best-first search example

40 Greedy best-first search example

41 Greedy best-first search example But in fact going via Rimnicu Vilcea is shorter!!

42 Properties of Greedy Best-First search Complete?No can get stuck in loops Time?O(b m ), but a good heuristic can give dramatic improvement Space?O(b m ) --keeps all nodes in memory Optimal? No

43 A * search Idea: avoid expanding paths that are already expensive Evaluation function f(n) = g(n) + h(n) g(n) = cost so far to reach n h(n)= estimated cost from nto goal f(n) = estimated total cost of path through nto goal

44 A * search example

45 A * search example

46 A * search example

47 A * search example

48 A * search example

49 A * search example

50 Admissible heuristics A heuristic h(n)is admissibleif for every node n, h(n) h * (n), where h * (n)is the true cost to reach the goal state from n. An admissible heuristic never overestimatesthe cost to reach the goal, i.e., it is optimistic Example: h SLD (n) (never overestimates the actual road distance) Theorem: If h(n) is admissible, A * using TREE-SEARCHis optimal

51 Optimality of A * (proof) Suppose some suboptimal goal G 2 has been generated and is in the fringe. Let nbe an unexpanded node in the fringe such that n is on a shortest path to an optimal goal G. f(g 2 ) = g(g 2 ) since h(g 2 ) = 0 g(g 2 ) > g(g) since G 2 is suboptimal f(g) = g(g) since h(g) = 0 f(g 2 ) > f(g) from above

52 Optimality of A * (proof) (cont.) Suppose some suboptimal goal G 2 has been generated and is in the fringe. Let nbe an unexpanded node in the fringe such that n is on a shortest path to an optimal goal G. f(g 2 ) > f(g) from above h(n) h*(n) since h is admissible g(n) + h(n) g(n) + h * (n) f(n) f(g) Hence f(g 2 ) > f(n), and A * will never select G 2 for expansion [RusselNorvig2010]

53 Admissible heuristics E.g., for the 8-puzzle: h 1 (n) = number of misplaced tiles h 2 (n) = total Manhattan distance (i.e., number of squares from desired location of each tile) h 1 (S) =? h 2 (S) =?

54 Admissible heuristics E.g., for the 8-puzzle: h 1 (n) = number of misplaced tiles h 2 (n) = total Manhattan distance (i.e., number of squares from desired location of each tile) h 1 (S) = 8 h 2 (S) = = 18

55 Dominance If h 2 (n) h 1 (n)for all n(both admissible) then h 2 dominatesh 1 h 2 is better for search in the previous example (8-puzzle) example h 2 dominates h 1 Typical search costs (average number of nodes expanded): d=12 IDS = 3,644,035 nodes A * (h 1 ) = 227 nodes A * (h 2 ) = 73 nodes d=24 IDS = too many nodes A * (h 1 ) = 39,135 nodes A * (h 2 ) = 1,641 nodes [RusselNorvig2010]

56 Relaxed problems A problem with fewer restrictions on the actions is called a relaxed problem The cost of an optimal solution to a relaxed problem is an admissible heuristic for the original problem If the rules of the 8-puzzle are relaxed so that a tile can move anywhere, then h 1 (n) gives the shortest solution If the rules are relaxed so that a tile can move to any adjacent square,then h 2 (n) gives the shortest solution

57 Effective branching factor Effective branching factor b* Is the branching factor that a uniform tree of depth dwould have in order to contain N+1nodes. N +1=1+ b *+(b*) (b*) d Measure is fairly constant for sufficiently hard problems. Can thus provide a good guide to the heuristic s overall usefulness

58 Effectiveness of different heuristics Results averaged over random instances of the 8-puzzle

59 Consistent heuristics A heuristic is consistent if for every node n, every successor n'of n generated by any action a, h(n) cost(n,a,n') + h(n') It s the triangle inequality!

60 Consistent heuristics A heuristic is consistentif for every node n, every successor n'of n generated by any action a, h(n) cost(n,a,n') + h(n') If his consistent, we have f(n') = g(n') + h(n') = g(n) + cost(n,a,n') + h(n') g(n) + h(n) = f(n) i.e., f(n) is non-decreasing along any path (hence: monotone).

61 Consistent heuristics A heuristic is consistentif for every node n, every successor n'of n generated by any action a, h(n) cost(n,a,n') + h(n') If his consistent, we have f(n') = g(n') + h(n') = g(n) + cost(n,a,n') + h(n') g(n) + h(n) = f(n) i.e., f(n) is non-decreasing along any path (hence: monotone). Theorem: If h(n) is consistent, A* using GRAPH-SEARCH is optimal keeps all checked nodes in memory to avoid repeated states

62 Optimality of A * A * expands nodes in order of increasing f value Gradually adds "f-contours" of nodes Contour ihas all nodes with f=f i, where f i < f i+1

63 Properties of A* Complete?Yes (unless there are infinitely many nodes with f f(g) ) Time? Exponential Space? Keeps all nodes in memory Optimal? Yes A* expands all nodes with f(n)< c*, some nodes with f(n) = c*, but no nodes with f(n)> c*, [RusselNorvig2010]

64 Simple Memory Bounded A* This is like A*, but when memory is full we delete the worst node (largest f-value). we remember the best descendent in the branch we delete. If there is a tie (equal f-values) we delete the oldest nodes first. simple-mba* finds the optimal reachablesolution given the memory constraint. Time can still be exponential. A Solution is not reachable if a single path from root to goal does not fit into memory