Problem Solving: Informed Search

Transcription

1 Problem Solving: Informed Search References Russell and Norvig, Artificial Intelligence: A modern approach, 2nd ed. Prentice Hall, 2003 (Chapters 1,2, and 4) Nilsson, Artificial intelligence: A New synthesis. McGraw Hill, 2001

2 Outline Best first search - Greedy search - A* search - Dijkstra's algorithm More on best first search - Dominance - Relaxed problems Iterative improvement - Hill climbing - Simulated annealing

3 Informed (heuristic) Search or blind uninformed non-adversary search informed adversary or heuristic

4 heuristic search Informed (heuristic) Search Informed search strategies, other than the information available in the problem definition, use other information, such as heuristics to guide search towards a good solution

5 Heuristic search: Best first search heuristic search: best first search Best first search - Greedy search - A* search - Dijkstra's algorithm More on best first search - Dominance - Relaxed problems Iterative improvement - Hill climbing - Simulated annealing

6 heuristic search: best first search Recall the general tree search algorithm defun TREE-SEARCH (problem, fringe) returns solution state INITIAL-STATE(problem) fringe.insert(make-node(state)) loop do if EMPTY?(fringe) then return *FAILURE* node fringe.remove() if problem.goal-test?(state(node)) then return SOLUTION(node) children problem.expand(node) for each node in children do fringe.insert(node) The search strategy is defined by the order of node expansion!

7 heuristic search: best first search Recall the general tree search algorithm defun EXPAND(problem, node) returns successors successors {} --- set of NODES, initially empty for each action, state in problem.successors(state(node)) do child new NODE, child.parent node child.action action, child.state state step-cost problem.step-cost(state(node), action, state) child.cost COST(node) + step-cost child.depth DEPTH(node) + 1 successors.add(child) return successors

8 heuristic search: best first search Best-first search Idea: For each node use an evaluation function, able to estimate the goodness of the node The goodness of a node says how promising is the node towards the solution Expand the best unexpanded node Implementation: fringe is a queue sorted in decreasing order of goodness Special cases: Greedy search A* search

9 Romania with step costs in kms heuristic search: best first search Straight line distance (SLD) to Bucharest

10 Best first search: Greedy search Best first search - Greedy search - A* search - Dijkstra's algorithm More on best first search - Dominance - Relaxed problems Iterative improvement - Hill climbing - Simulated annealing heuristic search: best first search: greedy search

11 heuristic search: best first search: greedy search Greedy search Idea: greedy search expands the node that appears to be closest to the goal Evaluation function f(n) = h(n) The cost for attaining the current node is not taken into account Only an estimation about how far (in terms of cost) is the node from the goal is considered (e.g. h SLD (n) = straightline distance from the current node n to Bucharest)

12 heuristic search: best first search: greedy search Greedy search Implementation: The intended behavior can be ensured by using an ordered queue in which nodes are inserted according to the value returned by the function f(n) = h(n) defun GREEDY-SEARCH(problem) returns solution return TREE-SEARCH ( problem, ordered-queue )

13 Greedy search: An example heuristic search: best first search: greedy search

14 Greedy search: An example heuristic search: best first search: greedy search

15 Greedy search: An example heuristic search: best first search: greedy search

16 heuristic search: best first search: greedy search Greedy search Completeness: NO it can get stuck in loops, e.g. if Oradea is a goal, loop Lasi Neamt Lasi Neamt It is complete only under the assumption that state space is finite and that we check for repeated states Time complexity: O(b m ) --- recall that m = max depth of the search space A good heuristic can improve it tremendously Space complexity: O(b m ) Optimality: No --- keeps all nodes in memory

17 Best first search: A* search Best first search - Greedy search - A* search - Dijkstra's algorithm More on best first search - Dominance - Relaxed problems Iterative improvement - Hill climbing - Simulated annealing heuristic search: best first search: A-star search

18 heuristic search: best first search: A-star search A* search Idea: avoid expanding paths that are already expensive Evaluation function f(n) = g(n) + h(n) g(n) actual cost spent so far to reach n (from the initial state) h(n) estimated cost to reach the goal (from the current node n) In practice, uniform-cost search and greedy search are put together to give rise to the most powerful algorithm in this category

19 heuristic search: best first search: A-star search A* search A* search uses an admissible heuristic h(n) In other words, 0 h(n) h*(n), where h*(n) is the true cost from n to the goal (e.g. h SLD (n) never overestimates the actual driving distance) By definition, h(g)=0 for any state G that satisfies the goal

20 heuristic search: best first search: A-star search A* search Implementation: The intended behavior can be ensured by using an ordered queue in which nodes are inserted according to the function f(x) = g(x) + h(x) defun A-STAR-SEARCH(problem) returns solution return TREE-SEARCH ( problem, ordered-queue )

21 A* search: An example heuristic search: best first search: A-star search

22 A* search: An example heuristic search: best first search: A-star search

23 A* search: An example heuristic search: best first search: A-star search

24 A* search: An example heuristic search: best first search: A-star search

25 A* search: An example heuristic search: best first search: A-star search

26 A* search: Optimality Theorem: A* search is optimal Proof: heuristic search: best first search: A-star search Suppose some suboptimal node G2, which embeds a state that satisfies the goal, has been generated and is in the queue Let n be an unexpanded node on a shortest path to a node G that embeds an optimal state f(g2) = g(g2) + h(g2) = g(g2) since h(g2)=0 g(g) < g(g2) since G2 is suboptimal; thus g(g) < f(g2) f(n) = g(n) + h(n) g(g) since h is admissible A* will never choose G2 for expansion since f(n) < f(g2)

27 heuristic search: best first search: A-star search A* search: Optimality (another way) Lemma: A* expands nodes in increasing order of f value It gradually adds f-contours of nodes Contour k contains all nodes with f = f k with f k < f k+1 If C* is an optimal-path cost: A* expands all nodes with f(n) < C* A* expands some nodes with f(n) = C* A* does not expand nodes with f(n) > C*

28 A* search: Optimality (another way) f-contours of nodes... (an example) heuristic search: best first search: A-star search

29 Proof of lemma: Consistency A heuristic h is consistent if h(n) c(n,a,n') + h(n') If h is consistent, then f(n') f(n) f(n') = g(n') + h(n') f(n') = g(n) + c(n,a,n') + h(n') g(n) + h(n) f(n') = g(n) + c(n,a,n') + h(n') f(n) Hence, f(n) is non-decreasing along any path (cf. triangle inequality) heuristic search: best first search: A-star search Let us assume that the function devised to evaluate the cost of a step (i.e STEP-COST) has a corresponding function c : node x action x node

30 A* search: Properties Completeness: Yes, unless there is an infinite number of nodes with f f(g) Time complexity: O(b d ) --- d = depth of the closest optimal solution A good heuristic (i.e. h(x) very close to h*(x)) can improve it very much Space complexity: O(b d ) Optimality: Yes heuristic search: best first search: A-star search --- keeps all nodes in memory --- it cannot expand f k+1 until f k is finished

31 Best first search Best first search - Greedy search - A* search - Dijkstra's algorithm More on best first search - Dominance - Relaxed problems Iterative improvement - Hill climbing - Simulated annealing heuristic search: best first search: Djikstra's algorithm

32 Dijkstra a algorithm Given a vertex v, what is the length of the shortest path from v to every vertex v' in the graph? A greedy algorithm: choose node that minimized distance from initial state (i,e, use g instead of h) You must know the entire search space to use Dijkstra's algorithm heuristic search: best first search: Djikstra's algorithm

33 A* vs. Dijkstra's Algorithm heuristic search: best first search: Djikstra's algorithm Dijkstra s algorithm is a degenerate case of A*, where h(n) = 0 - A* finds the best path to a particular v - A* will typically use less memory

34 A* vs. Dijkstra's Algorithm heuristic search: best first search: Djikstra's algorithm Dijkstra's is like a puddle of water flooding outwards on a flat floor, whereas A* is like the same puddle expanding on a bumpy and graded floor toward a drain (the target node) at the lowest point in the floor

35 A* vs. Dijkstra's Algorithm Instead of spreading out evenly on all sides, the water seeks the path of least resistance, only trying new paths when something gets in its way The heuristic function is what provides the grade of the hypothetical floor. heuristic search: best first search: Djikstra's algorithm

36 More on best first search Best first search - Greedy search - A* search - Dijkstra's algorithm More on best first search - Dominance - Relaxed problems Iterative improvement - Hill climbing - Simulated annealing heuristic search: best first search: more on...

37 Admissible heuristics: Dominance If h 2 (n) h 1 (n) for all n, and both are admissible, then h 2 dominates h 1 and is better for searching For instance, when d=14, typical search costs are: IDS = nodes A*(h1) = 539 nodes A*(h2) = 113 nodes when d=24, typical search costs are: IDS approx nodes A*(h1) = nodes A*(h2) = nodes Relaxed problems heuristic search: best first search: more on...

38 Admissible heuristics (8-puzzle) Examples of admissible heuristics for the 8- puzzle problem: h 1 (n) = number of misplaced tiles heuristic search: best first search: more on... h 2 (n) = total Manhattan distance (for all tiles, sum the number of squares from tile location to desired location)

39 Admissible heuristics: Relaxation Best first search - Greedy search - A* search - Dijkstra's algorithm More on best first search - Admissible heuristics - Relaxed problems Iterative improvement - Hill climbing - Simulated annealing heuristic search: best first search: more on...

40 heuristic search: best first search: more on... Admissible heuristics: Relaxation One way to obtain admissible heuristics is: Simplify the problem by relaxing some of its constraints Find an exact solution for the relaxed problem (easier than the initial problem) Use the cost of the exact solution for the relaxed problem as the heuristic value Key: the optimal solution cost to the relaxed problem cannot be greater than that of the real problem

41 Relaxed problems: 8-puzzle Relax 8-puzzle rules so that a tile can move to any square heuristic search: best first search: more on... Under this hypothesis, h 1 (n) (number of misplaced tiles) gives the shortest solution Relax 8-puzzle rules so that a tile can move to any adjacent square (even if it is full) Under this hypothesis, h 2 (n) (Manhattan distance) gives the shortest solution

42 More relaxed problems (TSP) heuristic search: best first search: more on... Traveling Salesman Problem (TSP) = find the shortest tour that visits all cities exactly once Relaxation: minimum spanning tree can be computed in O(n 2 ) and it is a lower bound on shortest (open) tour

43 heuristic search: best first search: more on... More relaxed problems (n-queens) n-queens = place n queens on an n x n board such that no two queens are placed on the same row, column or diagonal Strategy: start with any queen distribution on the board. Move a queen at a time to reduce number of conflicting pairs

44 Iterative improvement Best first search - Greedy search - A* search - Dijkstra's algorithm More on best first search - Dominance - Relaxed problems Iterative improvement - Hill climbing - Simulated annealing heuristic search: iterative improvement

45 Iterative improvement heuristic search: iterative improvement In many optimization problems the path to the solution is irrelevant. The goal state itself is the solution (e.g. TSP and n-queen problems) All problems in which a configuration or schedule that meets some constraints must be found satisfy this description In such kind of problems, the state space represents the set of complete configurations and iterative improvement of the solution can be adopted as a strategy to search for a solution

46 Iterative improvement Strategy: Keep a single current state and try to improve it Repair the solution until it meets all constraints Space complexity: constant heuristic search: iterative improvement

47 Iterative improvement (TSP) heuristic search: iterative improvement Start with any complete tour. Perform pairwise exchanges The sequence of steps does not matter. We want the goal state (tour)

48 heuristic search: iterative improvement Iterative improvement (n-queens) Start with any queen distribution on the board Move a queen at a time to reduce the number of conflicting pairs

49 Iterative improvement: Hill climbing Best first search - Greedy search - A* search - Dijkstra's algorithm More on best first search - Dominance - Relaxed problems Iterative improvement - Hill climbing - Simulated annealing heuristic search: iterative improvement

50 heuristic search: iterative improvement: hill climbing Gradient ascent/descent: hill-climbing defun HILL-CLIMBING (problem) returns local-maximum current, neighbor: node current MAKE-NODE(INITIAL-STATE(problem)) loop do neighbor [a max-valued successor of current-node] if VALUE(neighbor) VALUE(current) then return STATE(current) current neighbor

51 heuristic search: iterative improvement: hill climbing Gradient ascent/descent: hill-climbing Can get stuck on local maxima, depending or initial state Value Global maximum Local maximum If space is continuous: problems choosing step size, slow convergence States

52 Simulated Annealing Best first search - Greedy search - A* search - Dijkstra's algorithm More on best first search - Dominance - Relaxed problems Iterative improvement - Hill climbing - Simulated annealing heuristic search: iterative improvement: simulated annealing

53 Simulated annealing heuristic search: iterative improvement: simulated annealing Idea: escape local maxima allowing some bad moves, but gradually reducing their frequency and size

54 Simulated annealing Similar to the annealing process in metalurgics: drop temperature gradually allows crystalline structure reach a minimal energy state If T decreases slowly enough, always reaches the best state heuristic search: iterative improvement: simulated annealing Widely used in VLSI design, flight scheduling, production scheduling, and other large optimization problems

55 Simulated annealing heuristic search: iterative improvement: simulated annealing Each point s of the search space is compared to a state of some physical system, and the function E(s) to be minimized is interpreted as the internal energy of the system in that state The goal is to bring the system, from an arbitrary initial state, to a state with the minimum energy

56 Simulated annealing heuristic search: iterative improvement: simulated annealing At each step, the SA heuristic considers some neighbours of the current state s, and probabilistically decides between moving the system to state s' or staying put in state s The probabilities are chosen so that the system ultimately tends to move to states of lower energy Typically this step is repeated until the system reaches a state which is good enough for the application, or until a given computation budget has been exhausted

57 Simulated annealing heuristic search: iterative improvement: simulated annealing defun SIM-ANNEALING(problem,schedule) returns goal-state current, next: node, T: Temperature current MAKE-NODE ( INITIAL-STATE(problem) ) k 1 loop do T schedule[k] if T = 0 then return STATE(current) next [a randomly selected successor of current] EVAL(next) EVAL(current) if > 0 then current next --- only with probability e /T else current next k k+1 Temperature: controls the probability of downward steps (varies according to schedule)