CMU-Q Lecture 2: Search problems Uninformed search. Teacher: Gianni A. Di Caro

Transcription

1 CMU-Q Lecture 2: Search problems Uninformed search Teacher: Gianni A. Di Caro

2 RECAP: ACT RATIONALLY Think like people Think rationally Agent Sensors? Actuators Percepts Actions Environment Act like people Act rationally Techniques for rational decisionmaking in machines Action a Abstraction Problem State A State B Formal Model Problem Class 2

3 RECAP: TASK ENVIRONMENT~PROBLEM Representation (States, Transitions) Agent Sensors? Actuators Percepts Actions Environment Agent behavior: Agent function that maps percept sequences to an action Task environment (~Problem): Environment + Actuators + Sensors + Performance Different task environments require different techniques for problem representation, agent design, and rational decision-making 3

4 4

5 (MODEL-BASED) AUTOMATED PLANNING Task environment: Fully Observable Deterministic Known Sequential Discrete Single agent Objective: The agent has well defined goals that aims to reach at the minimum cost, starting from a defined start state Output: A plan that takes from the start state to a goal state at the minimum cost Plan: A sequence of decisions to be executed from start state in open-loop Problem representation: A graph, where nodes are problem states and arcs are feasible actions (state transitions) 5

6 GOAL-BASED AGENT Has goal(s), that describe situations that are desirable The agent uses goal information to select between possible actions in the current state by accounting for their effect in the future toward the achievement of goals Search: the process of looking for a sequence of actions that reaches the goal (optimizing performance) 6

7 SEARCH / DETERMINISTIC PLANNING A search problem is defined by: ü States (configurations of the environment) ü Start state and goal state(s) ü Actions available to the agent in each state ü Transition model: the state resulting from doing action a in state s (Successor function) ü Cost model: step costs c(s, a, s & ) 0, path costs (additive) What are the states? Find one (the optimal) sequence of actions (path) from start to a goal state 7

8 PATH SEARCH ON GRAPHS State space of the problem: Set of all states reachable from initial state by any sequence of actions Determined by: initial state + actions + transition model Ø The state space structure can be represented as a directed graph where the nodes are states and edges between nodes are actions1 1 Minimum cost path finding on graphs 8

9 EXAMPLE: PANCAKE FLIPPING s t Finding the minimum number of flips is NP-hard 1 1 Number of states? 9

10 EXAMPLE: 8-PUZZLE s t States Locations of tiles 8-puzzle: 181,440 states 15-puzzle: 1.3 trillion states 24-puzzle: states Actions Move blank left, right, up, down NP-Complete! Path cost 1 per move 10

11 EXAMPLE: PATHFINDING Initial state Arad Actions Go from one city to another Transition model If you go from city A to city B, you end up in city B Goal state Bucharest Path cost Sum of edge costs (traveling times, Km) 11

12 EXAMPLE: TOURING PROBLEMS Visit every city at least once, starting and ending in Bucharest How the state space looks like? Traveling salesperson problem (TSP): Visit each city exactly once and find the tour of minimum cost (costs may be not symmetric ) # states for a symmetric n-cities TSP: 12

13 EXAMPLE: TOURING 16,892 cities PROBLEMS 71,009 cities Trivial to find one feasible solution, (NP-) hard to find the best one 13

14 SOLUTION APPROACHES FOR SEARCH Meta-Algorithms: incremental node generation Ø Tree search Ø Graph search Uninformed ü Breadth-First Search (BFS) ü Bidirectional BFS Ø Uniform-Cost Search (UCS) Informed Greedy Best-First Search v A* Ø Theta* Depth-First Search (DFS) Depth-Limited DFS Ø Iterative Deepening Search (IDS) 14

15 TREE SEARCH o Let s begin at the start node and expand it: make the set of all its possible successor states (i.e., resulting from feasible actions) o Include the set resulting from the node expansion into a set of unexpanded nodes, that is called the frontier set Successor state Starting state Action Frontier set o At each step, pick a node from the frontier to expand o Keep going until reach a goal state o Try to expand as few nodes as possible, or move along the path of minimum cost Goal state Search tree 15

16 TREE SEARCH function TREE-SEARCH(problem, strategy) set of frontier nodes containing the start state of problem loop if there are no frontier nodes then return failure else choose a frontier node for expansion using strategy if the node contains a goal state then return the corresponding solution else expand the node and add the resulting nodes to the set of frontier nodes end loop 16

17 TREE SEARCH EXAMPLE 17

18 TREE SEARCH EXAMPLE 18

19 TREE SEARCH EXAMPLE Strategy Frontier 19

20 TREE SEARCH Frontier nodes Explored nodes 20

21 TREE SEARCH ISSUES Tree search can expand the same states again and again Algorithms that forget their history are doomed to repeat it! 21

22 ANOTHER CYCLING EXAMPLE Frontier/open set 1 {m} 2 {h,l,r,n} 3 {l,r,n,c,g,i,m} 4 {l,r,n,c,i,m,b,f,h} 5 {l,r,c,i,m,b,f,h,o,s} 6 {l,r,c,m,b,f,h,o,s,d,n,j} 7 Cycle! c 4 b c g 2 h 1 m 3 g i m f h i 5 l r n 6 i m o 7 d h n j s a f k p u b g l q v c h m r w Goal state a can be reached from start state m in multiple ways. The same intermediate state c can be encountered following different paths from m d i n s x e j o t y 22

23 STATES VS. NODES State s: an admissible configuration of the world Node n: a bookkeeping data structure used to represent the search tree o Nodes are on specific search paths, as defined by PARENT pointers, states are not o Two different nodes on the tree can contain the same state s, if s was generated via two different search paths Right PARENT cost-to come start STATE Node ACTION = Right PATH-COST =

24 STATES VS. NODES A Node s data structure contains: n.state: the state s to which n corresponds to n.parent: the node in the search tree that generated node n n.action: the action that was applied to the parent to generate n n.path-cost: the cost g(n) of the path from the initial node to n as indicated by the parent pointers (cost-to-come) start Right PARENT cost-to come STATE Node ACTION = Right PATH-COST = 6 Same state, different nodes 24

25 GRAPH SEARCH function GRAPH-SEARCH(problem, strategy) set of frontier nodes containing the start state of problem loop if there are no frontier nodes then return failure else choose a frontier node for expansion using strategy, and add it to the explored set if the node contains a goal state then return the corresponding solution else expand the node and add the resulting nodes to the set of frontier nodes, only if not in the frontier or explored set 25

26 GRAPH SEARCH ILLUSTRATED Each node is associated to a different state, the search tree grows on the state graph! Separation property: Every path from initial state to an unexplored state has to pass through the frontier. The frontier separates explored vs. unexplored regions 26

27 NO REPEATED STATES IN GRAPH SEARCH a f k p u b g l q v c h m r w d i n s x e j o t y 1 m 2 5 h l r n c 3 6 g i m i m o s 4 7 b f h i d h n j a c 8 g c e i 9 b 10 d h a c g Path to goal from start: (a b c d i n m) Cost: 6 Additional data structure to hold explored/closed set + a map function to search it (e.g., hash function) 27

28 MEASURING PERFORMANCE How do we will score the performance of different approaches for finding a solution? Completeness Optimality Time Space Guaranteed to find a solution when there is one? Finds the best solution? How long does it take to find a solution? How much memory needed to perform the search? 28

29 MEASURING PERFORMANCE o In AI scenarios, state graphs are usually HUGE, such that they are given implicitly, trough the initial state, actions and the transition model (the configuration resulting from applying action a to state s). o Graph nodes are generated lazily during the search process based on node expansion choices A B E C D Explicit graph description Tree search generation example: A {B,C} B {A,C} C {A,B,D} B {A,C,D} D {A,C,E} 29

30 MEASURING PERFORMANCE It may be not always useful/possible to reason on V and E to quantify performance, but rather other quantities are considered Branching factor b, the maximum number of successors of any node Depth d, the depth (# of steps along path from root) of shallowest goal Max length, of any path from start in the state space # nodes generated during search ~ Time Max # of nodes stored in memory during search ~ Space 30

31 UNINFORMED VS. INFORMED Based on either tree or graph search (or both), two main classes of search algorithms exist, which mainly differentiate from each other because of the strategy used to expand nodes : Uninformed Can only generate successors, sum up what has happened so far, and distinguish goals from non-goals Informed Strategies that know whether one non-goal is more promising than another 31

32 UNINFORMED VS. INFORMED SEARCH Strategy How desirable is to be in a certain intermediate state for the sake of (effectively) reaching a goal state from start Uninformed: Generation order, estimated cost-to-come Informed: Generation order, estimated cost-to-come, estimated cost-to-go Heuristic information Start ~ Cost-to-come (Pessimistic) ~ Cost-to-go (Optimistic) Goal 32

33 UNINFORMED SEARCH Uninformed Can only generate successors, use cost-to-come, and distinguish goals from non-goals 33

34 BREADTH-FIRST SEARCH (BFS) Strategy: Expand shallowest unexpanded node All nodes at a given level of the search tree are expanded before any other node is expanded. Horizontal, level-by-level search Can be implemented by using a FIFO queue for the frontier set Goal test applied when a node is generated 34

35 BFS FOR PATH FINDING 35

36 BFS GRAPH SEARCH FOR 8-PUZZLE (One) Goal found after 26 node expansions and 47 goal checks 36

37 BFS PROPERTIES Algorithm BFS Complete? Optimal? Time Space ~Yes Not really Θ b 5 Θ b 5 Complete: Yes if a goal node is a finite depth. It s true also in infinite graphs. In both cases, the branching factor b must be finite Optimality: Only if costs are a non-decreasing function of the depth (e.g., all actions have the same cost). In general, the shallowest goal (the first generated) is not necessarily the optimal one As it is, not useful for finding Shortest Paths

38 BFS PROPERTIES Algorithm BFS Complete? Optimal? Time Space ~Yes Not really Θ b 5 Θ b 5 Time complexity: Imagine each node has exactly b successors, and solution is at depth d, then BFS will generate 5 8:; b 8 = Θ b 5 nodes. b=2 d=7 Space complexity: For graph search BFS, Θ b 5 nodes are in frontier and Θ b 5<; in explored set ( tree search would not save much, since only the explored nodes could be avoided). E.g., for b=10, d=12 Θ(10 12 ) nodes. If 1 node requires 1 kb kb! 38

39 BIDIRECTIONAL SEARCH Idea: Possibly improve the running time of BFS by running two simultaneous searches, forward from the initial state and backward from the goal What is the worst-case running time of BIDIRECTIONAL SEARCH? 1. Θ(b d) 2. Θ((b/2) 5 ) 3. Θ(b 5/C ) 4. Θ(b 5 ) 39

40 BIDIRECTIONAL SEARCH Θ(b 5/C ) + Θ(b 5/C F ) 2 b 5 Θ(b 5 ) For b=10, d=6, each BFS generates up to depth d=3 2,220 nodes vs. 1,111,110: big memory save! Time complexity: Θ(b 5/C ) running the two searches in parallel, or Θ(b 5 ) if sequential alternating Issues: Asymmetric costs Unidirectional moves Repeated check for frontier intersection (additional constant time with hashing) Existence of multiple goals Abstract goal definition 40

41 BID-BFS: PATH FINDING EXAMPLE 41

42 UNIFORM-COST SEARCH (UCS) Strategy: Expand frontier node with lowest (estimated) path cost g n from the starting node, the cost-to-come Can be implemented by using a priority queue (PQ) ordered by g(n) for the frontier set Other changes from BFS: o o o Goal test applied when node is selected for expansion, not when included in the frontier (the first time a node is selected it can be on a sub-optimal path) If a successor is already in the frontier set, its path cost needs to be updated if lower than the previously computed one g(x): (Pessimistic) Estimate. Real cost-to-come BFS would stop here! G g(g) = 310 g(g) =

43 DIJKSTRA S ALGORITHM (1959) Shortest paths from a node s to all all other nodes Input: Graph G=(V,E) ( x s) dist[x] = + # dist[x] g(x) ~ distance from s (start) dist[s] = 0 C = # Closed set ~ Expanded nodes set O = V # Open set, ordered by dist[] ~ Our frontier set while O do u = extract_min(o) # expand node u C = C {u} # If u == goal: exit, since sp s u has been found foreach vertex v Successor(u) do dist[v] = min {dist[v], dist[u] + step_cost(u,v)} # Relaxation end do end do Time complexity: O( E + V log V ) using a Fibonacci heap for min queue 43

44 UNIFORM-COST SEARCH Algorithm UCS Complete? Optimal? Time Space Sorta Yes Θ b ;J K /M Θ b ;J K /M Completeness: If the cost of every step exceeds ε > 0 (and b is finite) there is only a finite number of expansions before the total path cost gets equal to the path cost of goal state. Instead, it could get into an infinite loop for ε = 0 or for costs infinitely decreasing On this branch, step-costs go as 2 -d 44

45 UNIFORM-COST SEARCH Algorithm UCS Complete? Optimal? Time Space Sorta Yes Θ b ;J K /M Θ b ;J K /M Optimality: (1) When a node n is selected for expansion, the shortest path to it has been found; otherwise another node m would be in the frontier (separation property) such that the path from start to n through m is shorter. But in this case m should have been selected before n for expansion (Contradiction!) (2) Because steps are non negative, paths never get shorter as nodes are added. UCS expands nodes (goals) in order of optimality and this is true also for the goal nodes 45

46 UNIFORM-COST SEARCH Algorithm UCS Complete? Optimal? Time Space Sorta Yes Θ b ;J K /M Θ b ;J K /M Optimality: Given completeness, the previous result ensures that when a goal node is selected for expansion the optimal path to it has been found Time complexity: If C is the cost of the optimal solution and ε is a lower bound on the step cost, the worst-case depth of the search tree is 1 + C /ε The complexity is Θ(b 5J; ) when step costs are uniform (in this case BFS is preferred) 46

47 DEPTH-FIRST SEARCH Strategy: Expand deepest unexpanded node Vertical diving + backtracking when reach a leaf Can be implemented by using a stack for the frontier (LIFO) Recursive implementation is also common (not use stack) 47

48 DFS FOR 8-PUZZLE Goal found after 19 node expansions and 32 goal checks 48

49 DFS FOR 8-PUZZLE Stack R 20L L

50 DEPTH-FIRST SEARCH Algorithm DFS Complete? Optimal? Time Space Finite Y No Θ b R Θ b m In a finite state space, which version of DFS is complete? 1. TREE SEARCH 2. GRAPH SEARCH 3. Both 4. Neither 50

51 DEPTH-FIRST SEARCH Graph search avoid loops, and in a finite space will eventually expand every node à Complete At almost zero cost, a check can be added to Tree search to avoid looping on the current path from start to the current node It makes DFS Tree search complete in finite space but doesn t avoid proliferation of redundant paths In infinite spaces, both versions fail for completeness if one diving branch is infinite Completeness: Clearly not in general x.. 51

52 DEPTH-FIRST SEARCH Algorithm DFS Complete? Optimal? Time Space In finite spaces No Θ b R Θ b m Optimality: Nope, if goal is on the branch being explored it will be reported as goal but there could be a better path Time complexity: Graph search is bounded by the size of the state space, it can generate up to the total number of states. Tree search is O b R, where m is the maximum depth of any node, can be much larger than state space! Space complexity: DFS tree search needs to store only a single path from the root to a leaf (m worst-case), along with unexpanded sibling nodes for each node on the path (b per node). Graph search is equivalent to BFS 52

53 DEPTH-FIRST BACKTRACKING SEARCH Recursive approach Only one successor is generated at a time When no further expansions are possible the search backtracks to the latest partially expanded node Each partially expanded node remembers which successor to generate next O(m) memory is needed It saves memory (and time) Particularly effective when successors are obtained by state modifications (CSPs) 53

54 DFS VS. BFS Algorithm DFS Algorithm BFS Complete? Optimal? Time Space In finite spaces No Θ b R Θ b m Complete? Optimal? Time Space Yes if Not really Θ b 5 Θ b 5 d, b finite Example: b=10, d=16=m BFS: nodes, 10 exabytes of memory if 1 node = 1 Kbytes DFS: = 160 nodes ~ 156 KB Depth-first tree search is the workhorse of many areas of AI 54

55 DFS VS. BFS Algorithm DFS Algorithm BFS Complete? Optimal? Time Space In finite spaces No Θ b R Θ b m Complete? Optimal? Time Space Yes if Not really Θ b 5 Θ b 5 d, b finite Performance depends on the structure of the search tree If goal is not far from root (shallow) BFS If the tree is deep (large m) and goal nodes are rare BFS If the branching factor b is large DFS If goal nodes are deep and frequent DFS If the search tree is infinite and b is finite BFS If the search tree is very deep DFS + limits for search depth If the search space is very large and depth is unknown IDS 55

56 DEPTH-LIMITED DF SEARCH Strategy: Expand deepest unexpanded node until a limit Run DFS with given depth limit l Additional source of incompleteness when l < d DFS is a special case of depth-limited for l = How to set l? Algorithm DL-DFS Complete? Optimal? Time Space If l > d No Θ b R Θ b m 56

57 ITERATIVE DEEPENING SEARCH (IDS) Limit = 0 A A Limit = 1 A A A A B C B C B C B C Strategy: Expand deepest unexpanded node until a depth limit l, and iteratively increase the limit l Limit = 2 Limit = 3 B A D E F G B A D E F G A C C A B C D E F G A B C D E F G A A B C D E F G A B C D E F G A A B C D E F G A B C D E F G A Run DFS with depth limit l = 1,2, until a goal is found B D E F G H I J K L M N O B A D E F G C C B C D E F G H I J K L M N O A B C D E F G B C D E F G H I J K L M N O A B C D E F G B C D E F G H I J K L M N O A B C D E F G Combines the best properties of BFS and DFS H I J K L M N O B A D E F G H I J K L M N O C H I J K L M N O A B C D E F G H I J K L M N O H I J K L M N O A B C D E F G H I J K L M N O H I J K L M N O A B C D E F G H I J K L M N O 57

58 ITERATIVE DEEPENING SEARCH Algorithm IDS Complete? Optimal? Time Space Yes No Θ b 5 Θ b d Completeness: Yes, for the same reason BFS is complete Optimal: Suffers from the same shortcoming of DFS Time complexity: Seems wasteful but most of the nodes are at the bottom level d, such that asymptotically is like BFS. For a fixed branching factor b, nodes at first level are generated d times, those at second level, d-1 times, and so on: d b + d 1 b C b 5 = Θ b 5 Space complexity: Enjoys the same properties as DFS 58

59 SUMMARY OF UNINFORMED Algorithm BFS UCS DFS IDS ALGORITHMS Complete? Optimal? Time Space Yes if d,b finite Not really Θ b 5 Θ b 5 Sorta Yes Θ b ;J K /M Θ b ;J K /M In finite spaces No Θ b R Θ b m Same as BFS No Θ b 5 Θ b d 59

60 SUMMARY (Model based, Deterministic) Automated planning, search problems Tree search and graph search meta-algorithms Performance metrics: completeness, optimality, time and space Uninformed search algorithms: BSF (problems with memory space) Bidirectional BFS (save in memory) UCS (~BFS for finding cost-optimal solutions) DFS (Tree search version has very low space requirements) Depth-limited DFS (avoid infinite diving) IDS (progressively increase depth limits) (Structure of search tree + Resources) vs. Algorithm 60