Luke. Leia 2. Han Leia

Transcription

1 Reading SE : ata Structures raph lgorithms raph Search hapter.,.,. Lecture raph T raphs are a formalism for representing relationships between objects a graph is represented as = (V, E) Han V is a set of vertices E is a set of edges Leia Luke V = {Han, Leia, Luke} operations include: E = {(Luke, Leia), (Han, Leia), iterating over vertices (Leia, Han)} iterating over edges iterating over vertices adjacent to a specific vertex asking whether an edge exists connected two vertices raphs In Practice q Web graph Vertices are web pages Edge from u to v is a link to v appears on u q all graph of a computer program Vertices are functions Edge from u to v is u s v q Task graph for a work flow Vertices are tasks Edge from u to v if u must be completed before v begins raph Representation : djacency Matrix raph Representation : djacency List V x V array in which an element (u, v) is true if and only if there is an edge from u to v Han Luke Han Han Luke Leia Leia Runtime: Luke iterate over vertices O( v ) iterate ever edges O( v ) Leia iterate edges adj. to vertex O( v ) edge exists? O() Space required: O( v ) V -ary list (array) in which each entry stores a list (linked list) of all adjacent vertices Han Luke Han Leia Luke Runtime: iterate over vertices O(v) iterate ever edges O( e ) Leia iterate edges adj. to vertex O(d) (d is number of adj. vertices) edge exists? O(d) Space required: O( v + e )

2 Terminology Weighted raphs q In directed graphs, edges have a specific direction q In undirected graphs, edges are two-way q Vertices u and v are adjacent if (u, v) E q sparse graph has O( V ) edges (upper bound) q dense graph has Ω( V ) edges (lower bound) q complete graph has an edge between every pair of vertices q n undirected graph is connected if there is a path between any two vertices Each edge has an associated weight or cost. linton Kingston Bainbridge Mukilteo Edmonds Seattle Bremerton 8 Paths and ycles path is a list of vertices {v, v,, v n } such that (v i, v i+ ) E for all i < n. cycle is a path that begins and ends at the same node. hicago Seattle Path Length and ost Path length: the number of edges in the path Path cost: the sum of the costs of each edge Seattle. hicago Salt Lake ity. Salt Lake ity.. San Francisco allas p = {Seattle, Salt Lake ity, hicago, allas, San Francisco, Seattle} San Francisco allas length(p) = cost(p) =. Trees as raphs irected cyclic raphs (s) Every tree is a graph with some restrictions: the tree is directed there are no cycles (directed or undirected) there is a directed path from the root to every node B E F H s are directed graphs with no cycles. add() access() main() Trees s raphs program graph mult() read()

3 Topological Sort iven a directed graph, = (V, E), output all the vertices in V such that no vertex is output before any other vertex with an edge to it. to Topological Sort Label each vertex s in-degree Initialize a queue to contain all in-degree zero vertices While there are vertices remaining in the queue Remove a vertex v with in-degree of zero and output it Reduce the in-degree of all vertices adjacent to v Put any of these with new in-degree zero on the queue Runtime: O( v + e ) to Queue = Output = to Queue = Output = to Queue = Output = to Queue = Output = 8

4 to Queue = Output = to Queue = Output = Exercise to Queue = Output = esign the algorithm to initialize the in-degree array. ssume the adjacency list representation. raph Search eneral raph Search lgorithm Many problems in computer science correspond to searching for a path in a graph, given a start node and goal criteria Route planning Mapquest Packet-switching VLSI layout -degrees of Kevin Bacon Program synthesis Speech recognition We ll discuss these last two later Open some data structure (e.g., stack, queue, heap) riteria some method for removing an element from Open Search( Start, oal_test, riteria) insert(start, Open); repeat if (empty(open)) then return fail; select Node from Open using riteria; Mark Node as visited; if (oal_test(node)) then return Node;

5 Open Stack riteria Pop epth-first raph Search FS( Start, oal_test) push(start, Open); repeat if (empty(open)) then return fail; Node := pop(open); Mark Node as visited; if (oal_test(node)) then return Node; Open Queue Breadth-First raph Search riteria equeue (FIFO) BFS( Start, oal_test) enqueue(start, Open); repeat if (empty(open)) then return fail; Node := dequeue(open); Mark Node as visited; if (oal_test(node)) then return Node; for each hild of node do if (hild not already visited) then enqueue(hild, Open); end BFS - FS - QUEUE = STK = 8 Two Models Planning. Standard Model: raph given explicitly with n vertices and e edges. q Search is O(n + e) time in adjacency list representation. I Model: raph generated on the fly. q Time for search need not visit every vertex. huge graph may be implicitly specified by rules for generating it on-the-fly Blocks world: vertex = relative positions of all blocks edge = robot arm stacks one block stack(blue,table) stack(blue,red) stack(green,red) stack(green,blue) stack(green,blue)

6 I omparison: FS versus BFS epth-first search oes not always find shortest paths Must be careful to mark visited vertices, or you could go into an infinite loop if there is a cycle Breadth-first search lways finds shortest paths optimal solutions Marking visited nodes can improve efficiency, but even without doing so search is guaranteed to terminate FS Space Requirements ssume: Longest path in graph is length d Highest number of out-edges is k FS stack grows at most to size dk For k=, d=, size is Is BFS always preferable? BFS Space Requirements onclusion ssume istance from start to a goal is d Highest number of out edges is k BFS Queue could grow to size k d For k=, d=, size is,,,,, In the I Model, FS is hugely more memory efficient, if we can limit the maximum path length to some fixed d. If we knew the distance from the start to the goal in advance, we can just not add any children to stack after level d But what if we don t know d in advance? Recursive epth-first Search Finding onnected omponents FS(v: vertex) mark v; for each vertex w adjacent to v do if w is unmarked then FS(w) Note: the recursion has the same effect as a stack For each vertex v do mark[v]= ; :=. For each vertex v do if mark[v] = then dfs(v); := +; dfs(v: vertex) mark[v] := ; for each vertex w adjacent to v do if mark[w] = then dfs(w) ll those vertices with the same mark are in the same connected component

7 Saving the Path For undirected graphs, each edge appears twice on the adjacency lists. mark Our pseudocode returns the goal node found, but not the path to it How can we remember the path? dd a field to each node, that points to the previous node along the path Follow pointers from goal back to start to recover path 8 (Unweighted raph) Seattle Seattle Salt Lake ity Salt Lake ity San Francisco allas San Francisco allas (Unweighted raph) raph Search, Saving Path Seattle Salt Lake ity San Francisco allas Search( Start, oal_test, riteria) insert(start, Open); repeat if (empty(open)) then return fail; select Node from Open using riteria; Mark Node as visited; if (oal_test(node)) then return Node; for each hild of node do if (hild not already visited) then hild.previous := Node; Insert( hild, Open ); end

8 Shortest Path for Weighted raphs Edsger Wybe ijkstra (-) iven a graph = (V, E) with edge costs c(e), and a vertex s V, find the shortest (lowest cost) path from s to every vertex in V ssume: only positive edge costs q Invented concepts of structured programming, synchronization, weakest precondition, and "semaphores" for controlling computer processes. The Oxford English ictionary cites his use of the words "vector" and "stack" in a computing context. q Believed programming should be taught without computers q Turing ward q In their capacity as a tool, computers will be but a ripple on the surface of our culture. In their capacity as intellectual challenge, they are without precedent in the cultural history of mankind. ijkstra s lgorithm for Single Source Shortest Path Pseudocode for ijkstra Similar to breadth-first search, but uses a heap instead of a queue: lways select (expand) the vertex that has a lowest-cost path to the start vertex orrectly handles the case where the lowest-cost (shortest) path to a vertex is not the one with fewest edges Initialize the cost of each node to s.cost := insert(s,,heap); While (! empty(heap)) n := deletemin(heap); For each edge e=(n,a) do if (n.cost + e.cost < a.cost) then a.cost = n.cost + e.cost; a.previous = n; if (a is in the heap) then decreasekey(a, a.cost, heap) else insert(a, a.cost, heap) end end Important Features ijkstra s lgorithm in ction Once a vertex is removed from the heap, the cost of the shortest path to that node is known While a vertex is still in the heap, another shorter path to it might still be found The shortest path itself can found by following the backward pointers stored in node.previous B E B E F H 8 8

9 ijkstra s lgorithm in ction ijkstra s lgorithm in ction B E x B E F H B E x B x E F H ijkstra s lgorithm in ction ijkstra s lgorithm in ction B E x B x x E F H B E x B x x x E F x H ijkstra s lgorithm in ction ijkstra s lgorithm in ction B 8 E x B x x x E F x 8 H x B 8 E x B x x x E F x x 8 H x

10 ijkstra s lgorithm in ction B 8 E x B x x x E x F x x 8 H x ata Structures for ijkstra s lgorithm V times: Select the unknown node with the lowest cost E times: a s cost = min(a s old cost, ) runtime: O( E log V ) findmin/deletemin decreasekey or insert O(log V ) O(log V ) Problem: Large raphs q It is expensive to find optimal paths in large graphs, using BFS or ijkstra s algorithm (for weighted graphs) q How can we search large graphs efficiently by using commonsense about which direction looks most promising? nd St nd St st St th St th ve S th ve 8 th ve th ve th ve th ve th ve rd ve nd ve Plan a route from th & th to rd & st 8 Best-First Search nd St nd St st St th St S rd ve th ve th ve th ve th ve 8 th ve th ve th ve Plan a route from th & th to rd & st nd ve The Manhattan distance ( x+ y) is an estimate of the distance to the goal It is a search heuristic q Best-First Search Order nodes in priority to minimize estimated distance to the goal q ompare: BFS / ijkstra Order nodes in priority to minimize distance from the start

11 Best-First Search Obstacles Open Heap (priority queue) riteria Smallest key (highest priority) h(n) heuristic estimate of distance from n to closest goal Best_First_Search( Start, oal_test) insert(start, h(start), heap); repeat if (empty(heap)) then return fail; Node := deletemin(heap); Mark Node as visited; nd St st St th St Best-FS eventually will expand vertex to get back on the right track th ve S th ve 8 th ve th ve th ve th ve th ve rd ve nd ve Non-Optimality of Best-First Improving Best-First nd St nd St st St th St th ve S Path found by Best-first th ve 8 th ve th ve th ve Shortest Path th ve th ve rd ve nd ve q Best-first is often tremendously faster than BFS/ijkstra, but might stop with a non-optimal solution q How can it be modified to be (almost) as fast, but guaranteed to find optimal solutions? q * - Hart, Nilsson, Raphael 8 One of the first significant algorithms developed in I Widely used in many applications * Optimality of * Exactly like Best-first search, but using a different criteria for the priority queue: minimize (distance from start) + (estimated distance to goal) priority f(n) = g(n) + h(n) f(n) = priority of a node g(n) = true distance from start h(n) = heuristic distance to goal Suppose the estimated distance is always less than or equal to the true distance to the goal heuristic is a lower bound Then: when the goal is removed from the priority queue, we are guaranteed to have found a shortest path!

12 * in ction pplications of *: Planning nd St nd St st St th St S h=+ h=+ huge graph may be implicitly specified by rules for generating it on-the-fly Blocks world: vertex = relative positions of all blocks edge = robot arm stacks one block stack(blue,table) stack(green,blue) stack(blue,red) H=+ th ve th ve 8 th ve th ve th ve th ve th ve rd ve nd ve stack(green,red) stack(green,blue) 8 Blocks World Blocks world: distance = number of stacks to perform heuristic lower bound = number of blocks out of place pplication of *: Speech Recognition (Simplified) Problem: System hears a sequence of words It is unsure about what it heard For each word, it has a set of possible guesses E.g.: Word is one of { hi, high, I } What is the most likely sentence it heard? # out of place =, true distance to goal = Speech Recognition as Shortest Path W W W onvert to a shortest-path problem: Utterance is a layered Begins with a special dummy start node Next: layer of nodes for each word position, one node for each word choice Edges between every node in layer i to every node in layer i+ ost of an edge is smaller if the pair of words frequently occur together in real speech + Techniy: - log probability of co-occurrence Finally: a dummy end node Find shortest path from start to end node W W W W W W W

13 Summary: raph Search epth First Little memory required Might find non-optimal path Breadth First Much memory required lways finds optimal path ijskstra s Short Path lgorithm Like BFS for weighted graphs Best First an visit fewer nodes Might find non-optimal path * an visit fewer nodes than BFS or ijkstra Optimal if heuristic estimate is a lower-bound