Solving Problems by Searching Artificial Intelligence Santa Clara University 2016
Problem Solving Agents Problem Solving Agents Use atomic representation of states Planning agents Use factored or structured states Uninformed search algorithms Algorithms only use state space Informed search algorithms Algorithms use some guidance where to find solution
Problem Solving Agents Problem formulation What actions and state to consider Goal formulation What is a desirable state
Problem Solving Agents First set of assumptions Environment is observable Agent knows current state Environment is discrete In any state, there are only a finite number of actions to be taken (and states to be reached) Environment is known Agent knows which states are reached by what action Environment is deterministic Each action has exactly one outcome
Problem solving agents Search: Looking for a sequence of actions that lead to the (a) goal Execution: Follow the sequence of actions Formulate - Search - Execute Notice: agent does not react to precepts during the execute phase Open loop system: loop between agent and environment is broken
Problem Solving Agents Well-defined problem 1. Initial state 2. Descriptions of actions 3. Transition model: Which action leads to what state 4. Goal test 5. Path cost
Recurring Sample Problem: Navigating Rumania 71 Oradea Neamt 118 75 Zerind 140 Timisoara 151 99 Fagaras 80 Rimnicu Vilcea 87 Iasi 92 Vaslui 111 Lugoj 97 Pitesti 211 142 70 75 Drobeta Mehadia 120 146 101 85 138 Bucharest 90 Craiova Giurgiu 98 Urziceni Hirsova 86 Eforie
A typical toy problem L R L R S S R R L R L R L L S S S S R L R L S S
Another toy problem 7 2 4 1 2 5 6 3 4 5 8 3 1 6 7 8 Start State Goal State
Another toy problem
A simple real world problem Find a flight from A to B (with stop overs)
Uninformed Searching Uninformed searching Go through the search space looking for a state that fulfills the goal condition Search space represented as a graph of nodes Expand nodes (create all the neighbors of the current node)
Uniformed Searching State examples: 8-puzzle State given by a matrix with entries 0-8 0 represents the blank space Note: for half of the initial states, there is no solution _goal_state = [[1,2,3], [4,5,6], https://gist.github.com/flatline/838202 [7,8,0]]
Uninformed Searching n - queens problem States are given by the position of j queens Each queen needs to be in a different column State given by an array (Python list) of j numbers Avoid generating states that are not legal def init (self, parent, move): self.parent = parent if parent: self.lista = parent.lista[:] else: self.lista = [] self.lista.append(move) def str (self): return "State "+str(self.lista) def test(self): #test whether any queen can take another one for col in range(0,len(self.lista)): row = self.lista[col] for j in range(len(self.lista)): if j == col: continue if self.lista[j] == row+j-col or self.lista[j] == row-j+col: return False return True
Uninformed Searching When generating nodes, can revisit the same node Repeated state Loopy path Search tree is then infinite, even though state space is finite Assuming going from one state to another incurs costs Assuming costs add up Then loopy paths never are cost-optimal
Uninformed Searching (a) (b) (c) Black: explored set discovered and expanded White: frontier discovered but not yet expanded Gray: unknown set not yet discovered
Example 71 Oradea Neamt Breadth first search: حفظنا اهلل منه Generate all accessible nodes in order until finding goal state 75 118 Zerind 140 Timisoara 111 70 75 Drobeta 151 99 Fagaras 80 Rimnicu Vilcea Lugoj Mehadia 120 Pitesti 211 97 146 101 85 138 Bucharest 90 Craiova Giurgiu 87 Iasi 92 142 98 Urziceni Vaslui Hirsova 86 Eforie
Uninformed Searching Breadth first search Finds guaranteed a goal state if there is a path to the goal state Expand initial state, then the states expanded from it, then the states expanded from them, def bfs(instance): states = fifo(instance.initial) while states: current = states.pop(0) else: for state in current.neighbours(): if state.notmarked() and state.feasible(): if state.isgoal(): return state state.markvisited() states.append(state)
Uninformed Search BFS: Will find closest goal node Assume a branching factor of b Number of unexplored nodes reachable from a node during the breadth-first search Assume nearest goal state is at distance d from the initial state BFS creates b + b 2 + b 3 +...+ b d = O(b d ) nodes
Uninformed Search BFS: Memory is stressed: too many nodes kept in fifo queue Time needs depends on the depth of the search
Uninformed Search Uniform cost search: Assume that all state transitions incur a cost g(node) = cost to reach node from initial node Expand the node n with the lowest path cost g(n) Apply goal test only when the node is expanded
Uninformed Search def uniformcostsearch(instance): frontier = priorityqueue(instance.initial, 0) explored = [] while frontier: current = frontier.pop() #get lowest cost node if current.isgoal(): print(current) #print out inverse path to goal for node in current.neighbours(): node.parent = current pathcost = current.cost + current.actioncost(node) if node in frontier and node.cost > pathcost: node.cost = pathcost #duplicate, update node.parent = current elif node not in frontier and node not in explored: frontier.insert(node) explored.append(current)
Group Quiz Apply Uniform Cost Search to go from to Bucharest 71 Oradea Neamt 118 75 Zerind 140 Timisoara 151 99 Fagaras 80 Rimnicu Vilcea 87 Iasi 92 Vaslui 111 Lugoj 97 Pitesti 211 142 70 75 Drobeta Mehadia 120 146 101 85 138 Bucharest 90 Craiova Giurgiu 98 Urziceni Hirsova 86 Eforie
Uninformed Search Depth First Search: Use stack instead of fifo in organizing the frontier def dfs(instance): states = lifo(instance.initial) while states: current = states.pop(0) else: for state in current.neighbours(): if state.notmarked() and state.feasible(): if state.isgoal(): return state state.markvisited() states.append(state)
Uninformed Search DFS will find a solution more quickly if they are lots of solutions at a certain depth. n-queens problem: n bfs dfs 4 14 8 5 43 5 6 148 31 7 511 9 8 1964 113 9 8041 41 10 34814 102 11 164245 52 12 841988 261 13 4601177 111 14 26992956 1899 15
Uninformed Search A A A B C B C B C D E F G H I J K L M N O D E F G H I J K L M N O D E F G H I J K L M N O A A A B C B C C D E F G H I J K L M N O D E F G I J K L M N O E F G J K L M N O A A C B C C E F G E F G F G J K L M N O K L M N O L M N O A A A C C C DFS on a F G L M N O F G L M N O F G M N O binary tree
Uninformed Search DFS works well if there are goal states at a limited depth but horribly if there are not Depth limited search If you don t find a goal, break off after reaching a maximum depth of d How do we determine d? Sometimes, the problem can tell us Iterative deepening depth first search
Uninformed Search Iterative deepening depth first search Do limited depth dfs Increase limit, if no solution found Why is this a good idea? We recreate parts of the previous search tree Repeated work is not a big portion of the total work done db +(d 1)b 2 +(d 2)b 3 +...+2b d 1 +1b d = O(b d )
Uninformed Search Bidirectional Search Basic idea: Search from the initial state and from the goal state Success if you can connect Complexity O(bˆd/2) Start Goal
Informed Searches Use additional information Rumanian road map: Straight-line distance to goal 8-puzzle Sum of the Manhattan distance of current location of a piece to goal location of a piece n-queen problem Need to redefine states
Informed Search Tree search Expand nodes without regard whether expanded nodes have been visited Graph search Expand nodes but disregard explored nodes
Informed Search def treesearch(instance): frontier = [ instance.initial ] while frontier: current = frontier.pop() if current.isgoal(): return current else: for node in current.neighbor() node.parent = current frontier.push(node)
Informed Search def graphsearch(instance): explored = [] frontier = [instance.initial] while frontier: current = frontier.pop() if current.isgoal(): print( current ) explored.push(current) for node in current.neighbor(): if node not in explored and node not in frontier: node.parent = current frontier.push(node)
Informed Search Best-first search Do graph search Use a priority queue Have an evaluation function for each node
Informed Search Greedy first search: Expand nodes according to the best node Example: Use straight-line distance to the goal Bucharest Craiova Drobeta Eforie Fagaras Giurgiu Hirsova Iasi Lugoj 366 0 160 242 161 176 77 151 226 244 Mehadia Neamt Oradea Pitesti Rimnicu Vilcea Timisoara Urziceni Vaslui Zerind 241 234 380 100 193 253 329 80 199 374
Informed Search (a) The initial state (b) After expanding 366 Timisoara Zerind 253 329 374 (c) After expanding Timisoara 329 Zerind 374 Fagaras Oradea Rimnicu Vilcea 366 176 380 193 (d) After expanding Fagaras Timisoara Zerind 329 374 Fagaras Oradea Rimnicu Vilcea 366 380 193 Bucharest 253 0
Informed Search Greedy best first search has difficulties Going from Iasi to Fagaras First expand Neamt This puts Iasi back into the frontier Can never consider Vaslui, because Vaslui is farther from Fagaras according to the heuristics 118 75 Oradea 71 Neamt Zerind 151 140 99 Fagaras 80 Timisoara Rimnicu Vilcea 111 Lugoj 70 Mehadia 75 Drobeta 120 Pitesti 211 97 146 101 85 138 Bucharest 90 Craiova Giurgiu 87 Iasi 92 142 98 Urziceni Vaslui Hirsova 86 Eforie
Informed Search Greedy best-first search Combine with graph search algorithm Do not consider repeated states Version is complete for finite trees
Informed Search A* search Evaluation function f for node n: g(n) cost of getting to node n h(n) heuristic for getting from node n to goal f(n) =g(n)+h(n)
Informed Search A* search When will A* be optimal? Heuristic needs to be admissible Never overestimate the costs to reach the goal from the current node Heuristic needs to be consistent (when using graph search) Estimated costs of reaching goal from n is less than or equal to the costs of going from n to successor n plus estimated costs of reaching the goal from n. 8n, 8n 0,n 0 successor of n h(n) apple costs(n, n 0 )+h(n 0 )
Informed Search Tree-version of A* is optimal if heuristic is admissible Graph version of A* is optimal if heuristic is consistent Clue: f-costs are nondecreasing on any path generated Can "prune" even neighbors of initial node
(a) The initial state (b) After expanding 366=0+366 393=140+253 Timisoara 447=118+329 Zerind 449=75+374 (c) After expanding Timisoara Zerind 447=118+329 449=75+374 Fagaras Oradea Rimnicu Vilcea 646=280+366 415=239+176 671=291+380 413=220+193 (d) After expanding Rimnicu Vilcea Timisoara Zerind 447=118+329 449=75+374 Fagaras Oradea 646=280+366 415=239+176 671=291+380 Rimnicu Vilcea Craiova Pitesti 526=366+160 417=317+100 553=300+253 (e) After expanding Fagaras Timisoara Zerind 447=118+329 449=75+374 Fagaras Oradea Rimnicu Vilcea 646=280+366 671=291+380 Bucharest Craiova Pitesti 591=338+253 450=450+0 526=366+160 417=317+100 553=300+253 (f) After expanding Pitesti Timisoara Zerind 447=118+329 449=75+374 Fagaras Oradea Rimnicu Vilcea 646=280+366 671=291+380 Bucharest Craiova Pitesti 591=338+253 450=450+0 526=366+160 553=300+253 Bucharest Craiova Rimnicu Vilcea 418=418+0 615=455+160 607=414+193
Informed Search A* expands nodes according to contours O Z N A T 380 S 400 R L F P I V D M C 420 G B U H E
Informed Search A* works well If there are many sub-optimal paths that are not quite a goal, then A* will explore many unnecessary spaces
Informed Search A* problems A* needs to maintain big arrays it never throws away a node runs out of memory before it runs out of user s patience
Informed Search Memory bounded A* variants Iterative deepening Cut-off consideration of nodes with f-value larger than a certain value If not successful, adjust cut-off point Can be difficult if f-values are floats
Informed Search Recursive best-first search Uses only linear space Uses an f-limit value to keep track of the best alternative path available
Informed Search RBFS Search f_limit on top heuristic on bottom (a) After expanding,, and Rimnicu Vilcea 447 415 Fagaras Oradea Rimnicu Vilcea 646 415 671 413 447 393 (b) After unwinding back to and expanding Fagaras 393 417 Fagaras Oradea 646 415 671 Rimnicu Vilcea 413 417 Craiova Pitesti 526 417 553 366 Timisoara Zerind 447 449 366 Timisoara 447 Zerind 449 591 Bucharest 450 (c) After switching back to Rimnicu Vilcea and expanding Pitesti 366 447 393 Timisoara 447 Zerind 449 447 Fagaras Oradea Rimnicu Vilcea 646 415 450 671 417 447 Craiova Pitesti 526 417 553 Bucharest Craiova Rimnicu Vilcea 418 615 607
Informed Search RBFS Search (a) Expand on best path unless the current best leaf (Pitesti) is worse than the best alternative value (Fagaras) Don t forget to update the value for Rimicu Vilcea (a) After expanding,, and Rimnicu Vilcea 591 447 415 Fagaras Oradea Rimnicu Vilcea 646 415 671 413 447 417 Fagaras Oradea 646 415 671 Bucharest 450 393 (b) After unwinding back to and expanding Fagaras 393 Rimnicu Vilcea 413 417 (c) After switching back to Rimnicu Vilcea and expanding Pitesti Craiova Pitesti 526 417 553 366 Timisoara Zerind 447 449 366 Timisoara 447 366 Zerind 449 447 393 Timisoara 447 Zerind 449 447 Fagaras Oradea Rimnicu Vilcea 646 415 450 671 417 447 Craiova Pitesti 526 417 553 Bucharest Craiova Rimnicu Vilcea 418 615 607
Informed Search RBFS Search (b) Unwind to This puts the best leaf value of the forgotten subtree in Rimnicu Vilcea. Expand Fagaras with best leaf value of 450 (a) After expanding,, and Rimnicu Vilcea 447 415 Fagaras Oradea Rimnicu Vilcea 646 415 671 413 447 393 (b) After unwinding back to and expanding Fagaras 393 417 Fagaras Oradea 646 415 671 Rimnicu Vilcea 413 417 Craiova Pitesti 526 417 553 366 Timisoara Zerind 447 449 366 Timisoara 447 Zerind 449 591 Bucharest 450 This is worse than 415 for Rimnicu Vilcea (c) After switching back to Rimnicu Vilcea and expanding Pitesti 447 393 Fagaras Oradea 646 415 450 671 447 Rimnicu Vilcea 417 366 447 Craiova Pitesti 526 417 553 Timisoara Zerind 447 449 Bucharest Craiova Rimnicu Vilcea 418 615 607
Informed Search RBFS Search (c) Back up to after updating Fagaras Expand again RV Then expand Pitesti This time, we reach Bucharest, because the best alternative value (Timisoara) is worse (a) After expanding,, and Rimnicu Vilcea 591 447 415 Fagaras Oradea Rimnicu Vilcea 646 415 671 413 447 417 Fagaras Oradea 646 415 671 Bucharest 450 447 393 (b) After unwinding back to and expanding Fagaras 393 393 Fagaras Oradea 646 415 450 671 Rimnicu Vilcea 413 417 (c) After switching back to Rimnicu Vilcea and expanding Pitesti Craiova Pitesti 526 417 553 447 Rimnicu Vilcea 417 366 447 Craiova Pitesti 526 417 553 Timisoara Zerind 447 449 366 Timisoara 447 366 Zerind 449 Timisoara Zerind 447 449 Bucharest Craiova Rimnicu Vilcea 418 615 607
Informed Search RBFS generates less nodes than IDA* In a sense, IDA* and RBFS throw away too much memory
Informed Search Memory bounded A* Simplified memory bounded A*
Informed Search Simplified memory bounded A* Do A* until algorithm runs out of memory Need to free a node Frees the node with the highest value of f After backing up the f-value to its parent If all the descendants of n are forgotten SMA* does not know where to go from n But SMA* knows whether n is worth while considering
Heuristics Example 8-puzzle Branching factor about 3 Depth about 22 Exhaustive tree search gives 3**22 ~3*10**10 moves Graph search has 9!/2 possible states Example 15-puzzle Graph search has 16!/2 = 10,461,394,944,000 states
Heuristics Heuristic may not overstate the true costs of moving to the goal state 1. Use number of misplaced tiles 2. Use the sum of the Manhattan distances of tiles in the actual state to the tiles in the goal state 7 2 4 1 2 5 6 3 4 5 8 3 1 6 7 8 Start State Goal State
Heuristics 8-puzzle Second choice works better
Heuristics Relaxed search problems Instead of humans inventing heuristics Computer could generate heuristics by relaxing the rules of the game 8 puzzle: Allow tiles move to any position Second heuristic gives the costs to move to the goal in the relaxed version Automatic generation of heuristics Formulate the rules of the game Then relax automatically the rules Try out heuristics
Heuristics Pattern databases: Generate admissible heuristics from subproblems 2 4 1 2 5 6 3 54 6 8 3 1 7 8 Start State Goal State Heuristic: Costs of moving 1,2,3,4 to a goal state
Heuristics Pattern databases: Use various heuristics 1,2,3,4 5,6,7,8 Pattern database stores the exact solution for a number of subproblems Disjoint pattern database Solve 1, 2, 3, 4 subproblem only counting the moves of tiles 1, 2, 3, or 4 Solve 5, 6, 7, 8 subproblem only contain the moves of tiles 5, 6, 7, or 8 Heuristic is the sum of both values