INTRODUCTION TO HEURISTIC SEARCH

Transcription

1 INTRODUCTION TO HEURISTIC SEARCH

2 What is heuristic search? Given a problem in which we must make a series of decisions, determine the sequence of decisions which provably optimizes some criterion.

3 What is NOT heuristic search? Any algorithm which cannot produce a provably correct (globally optimal) solution Greedy hill climbing Simulated annealing Genetic algorithms Gradient descent EM...

4 Overview Basics Brute force search Depth-first search Breadth-first search Heuristic search Heuristic function Best-first search (A*) Other details Conclusions

5 What is a state space search problem? State: a node in a graph Start state: where we begin our search Goal state(s): where we want to go Operators: how we move from one node to the next Typical problem: What is the minimum cost sequence of operators to move from start to a goal? What is the shortest path from start to a goal? Common variant: does there exist a path from the start to any goal?

6 Dijkstra s algorithm State: Cities in North America Start state: New York Goal state: Hollywood Operators: taking a bus, cost: bus ticket

7 Dijkstra s algorithm State: Cities in North America Start state: New York Goal state: Hollywood Operators: taking a bus, cost: bus ticket

8 Dijkstra s algorithm State: Cities in North America Start state: New York Goal state: Hollywood Operators: taking a bus, cost: bus ticket

9 Sliding tile, the most studied puzzle

10 What is an implicit state space? Observation: some state spaces are huge Problem: we cannot store all nodes Solution: define start and goals, but implicitly define all other states via operators Operator moveblankright(state) copy(state, newstate) updatecol(newstate, blank, col(state, blank)+1) updatecol(newstate, toright(state, blank), col(state, blank))

11 Standard terminology Expand. apply operators to a state (aka, node) Generate. create a state by expanding an existing one Successor. state created when expanding a node Duplicate. generate a state more than once OPEN. a sorted list of nodes which have been generated but not expanded CLOSED. a list of nodes which have been expanded

12 Standard notation: successors distance from start to node n (known) distance from start to predecessor n cost to move from node n to successor n

13 Another simple problem State: a subset of {X 0, X 1,..., X n } Start state: {} Goal state: {X 0, X 1,..., X n } Operators: Add one new X, i c(n, n ) is given as input

14 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

15 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

16 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

17 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

18 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

19 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

20 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

21 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

22 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

23 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

24 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

25 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

26 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

27 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

28 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

29 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

30 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

31 Depth-first search (DFS) Intuition: expand a node, then all of its successors, then all of its successors, etc. Solution reconstruction: pop back up the call stack Memory-efficient Easy to implement Infinite branches Not naively optimal Duplicate expansion!

32 Breadth-first search (BFS) Intuition: expand all of start s successor, then all of their successors, then all of their successors, etc. Solution reconstruction: store back pointers that can be retraced Some duplicate detection No depth problems Memory (for DD)

33 Breadth-first search (BFS) Intuition: expand all of start s successor, then all of their successors, then all of their successors, etc. Solution reconstruction: store back pointers that can be retraced Some duplicate detection No depth problems Memory (for DD)

34 Breadth-first search (BFS) Intuition: expand all of start s successor, then all of their successors, then all of their successors, etc. Solution reconstruction: store back pointers that can be retraced Some duplicate detection No depth problems Memory (for DD)

35 Breadth-first search (BFS) Intuition: expand all of start s successor, then all of their successors, then all of their successors, etc. Solution reconstruction: store back pointers that can be retraced Some duplicate detection No depth problems Memory (for DD)

36 Breadth-first search (BFS) Intuition: expand all of start s successor, then all of their successors, then all of their successors, etc. Solution reconstruction: store back pointers that can be retraced Some duplicate detection No depth problems Memory (for DD)

37 Breadth-first search (BFS) Intuition: expand all of start s successor, then all of their successors, then all of their successors, etc. Solution reconstruction: store back pointers that can be retraced Some duplicate detection No depth problems Memory (for DD)

38 Breadth-first search (BFS) Intuition: expand all of start s successor, then all of their successors, then all of their successors, etc. Solution reconstruction: store back pointers that can be retraced Some duplicate detection No depth problems Memory (for DD)

39 Breadth-first search (BFS) Intuition: expand all of start s successor, then all of their successors, then all of their successors, etc. Solution reconstruction: store back pointers that can be retraced Some duplicate detection No depth problems Memory (for DD)

40 Breadth-first search (BFS) Intuition: expand all of start s successor, then all of their successors, then all of their successors, etc. Solution reconstruction: store back pointers that can be retraced Some duplicate detection No depth problems Memory (for DD)

41 Breadth-first search (BFS) Intuition: expand all of start s successor, then all of their successors, then all of their successors, etc. Solution reconstruction: store back pointers that can be retraced Some duplicate detection No depth problems Memory (for DD)

42 Breadth-first search (BFS) Intuition: expand all of start s successor, then all of their successors, then all of their successors, etc. Solution reconstruction: store back pointers that can be retraced Some duplicate detection No depth problems Memory (for DD)

43 Heuristic functions A heuristic function estimates the distance from a state to the (closest) goal state. We often create heuristics by relaxing the problem.

44 Heuristic functions A heuristic function estimates the distance from a state to the (closest) goal state. We often create heuristics by relaxing the problem. Sliding tile puzzle heuristic: Manhattan distance h( ) =

45 Standard notation: node priority (known) distance from start to n heuristic estimate of the distance from n to goal estimate of the cost of a path from start to goal through n

46 Admissible heuristic functions An admissible heuristic function always underestimates the distance from a state to the goal. They are sometimes called optimistic. Theorem: The Manhattan distance heuristic function is admissible for the sliding tile puzzle.

47 Consistent heuristic functions A consistent heuristic function is admissible and satisfies that. They are sometimes called monotonic because f costs cannot decrease on a path from start to goal.

48 Depth-first branch and bound (DFBnB) Intuition: perform DFS, but calculate f(n) for each node. If f(n) is worse than some bound, prune. We have found a solution with cost 34. (over DFS): ignores parts of the search space worse than current known solution

49 Depth-first branch and bound (DFBnB) Intuition: perform DFS, but calculate f(n) for each node. If f(n) is worse than some bound, prune. We have found a solution with cost 34. We continue to search. (over DFS): ignores parts of the search space worse than current known solution

50 Depth-first branch and bound (DFBnB) Intuition: perform DFS, but calculate f(n) for each node. If f(n) is worse than some bound, prune. We have found a solution with cost 34. We continue to search. The lower bound of the node is 34. (over DFS): ignores parts of the search space worse than current known solution

51 Depth-first branch and bound (DFBnB) Intuition: perform DFS, but calculate f(n) for each node. If f(n) is worse than some bound, prune. We have found a solution with cost 34. We continue to search. The lower bound of the node is 34. Since it cannot be optimal, prune. (over DFS): ignores parts of the search space worse than current known solution

52 Best-first search (A*) Intuition: expand nodes in best-first order according to priority until expanding the goal Solution reconstruction: store back pointers that can be retraced Duplicate detection Ignores states with worse f than goal Memory (for DD) Priority queue

53 Best-first search (A*) Intuition: expand nodes in best-first order according to priority until expanding the goal Solution reconstruction: store back pointers that can be retraced Duplicate detection Ignores states with worse f than goal Memory (for DD) Priority queue

54 Best-first search (A*) Intuition: expand nodes in best-first order according to priority until expanding the goal Solution reconstruction: store back pointers that can be retraced Duplicate detection Ignores states with worse f than goal Memory (for DD) Priority queue

55 Best-first search (A*) Intuition: expand nodes in best-first order according to priority until expanding the goal Solution reconstruction: store back pointers that can be retraced Duplicate detection Ignores states with worse f than goal Memory (for DD) Priority queue

56 Best-first search (A*) Intuition: expand nodes in best-first order according to priority until expanding the goal Solution reconstruction: store back pointers that can be retraced Duplicate detection Ignores states with worse f than goal Memory (for DD) Priority queue

57 Best-first search (A*) Intuition: expand nodes in best-first order according to priority until expanding the goal Solution reconstruction: store back pointers that can be retraced Duplicate detection Ignores states with worse f than goal Memory (for DD) Priority queue

58 Best-first search (A*) Intuition: expand nodes in best-first order according to priority until expanding the goal Solution reconstruction: store back pointers that can be retraced Duplicate detection Ignores states with worse f than goal Memory (for DD) Priority queue

59 A* Theoretical properties Given an admissible heuristic h... Upon selecting goal for expansion, the shortest path has been found. Given h, no state space search strategy can prove optimality and expand fewer nodes than A*. If h is also consistent, no intermediate node will be re-expanded.

60 Terminology and data structures The function of the OPEN list is to determine the next node to expand. DFS: stack BFS: queue (naively) or hash table A*: priority queue (can also require hash table) The function of the CLOSED list is to detect nodes that have already been generated. BFS: hash table, possibly one layer at a time A*: hash table

61 Other search problems Constraint-satisfaction problems We specify conditions that goal must satisfy, but there could be some parts of the state we do not care about. Games against nature Operators are not deterministic. Under typical assumptions, these are called Markov decision processes. Multi-player games Other agents apply operators which we cannot control.

62 Conclusions Heuristic search algorithms use a heuristic function to guide a search from a start state to a goal state. A variety of search strategies can leverage the heuristic function in different ways. Depth-first Breadth-first Best-first (A*) A* has several important theoretical properties.