Problem: Large Graphs

Transcription

1 S : ata Structures raph lgorithms raph Search Lecture Problem: Large raphs It is expensive to find optimal paths in large graphs, using S or ijkstra s algorithm (for weighted graphs) How can we search large graphs efficiently by using commonsense about which direction looks most promising? xample xample 5 nd St 5 nd St 5 nd St 5 nd St 5 st St 5 st St 5 th St S 5 th St S nd ve rd ve th ve 5 th ve th ve 7 th ve 8 th ve 9 th ve th ve nd ve rd ve th ve 5 th ve th ve 7 th ve 8 th ve 9 th ve th ve Plan a route from 9 th & 5 th to rd & 5 st Plan a route from 9 th & 5 th to rd & 5 st est irst Search est irst Search The Manhattan distance (Δ x+ Δ y) is an estimate of the distance to the goal It is a search heuristic est irst Search Order nodes in priority to minimize estimated distance to the goal ompare: S / ijkstra Order nodes in priority to minimize distance from the start Open Heap (priority queue) riteria Smallest key (highest priority) h(n) heuristic estimate of distance from n to closest goal est_irst_search( Start, oal_test) insert(start, h(start), heap); repeat if (empty(heap)) then return fail; Node := deletemin(heap); if (oal_test(node)) then return Node; for each hild of node do if (hild not already visited) then insert(hild, h(hild),heap); end Mark Node as visited; end 5

2 Obstacles est S eventually will expand vertex to get back on the right track 5 nd St 5 nd St Non Optimality of est irst Path found by est-first 5 nd St S 5 st St S 5 st St 5 th St 5 th St th ve 9 th ve 8 th ve 7 th ve th ve 5 th ve th ve rd ve nd ve th ve 9 th ve 8 th ve 7 th ve th ve Shortest Path 5 th ve th ve rd ve nd ve 7 8 Improving est irst est first is often tremendously faster than S/ijkstra, but might stop with a non optimal solution How can it be modified to be (almost) as fast, but guaranteed to find optimal solutions? * Hart, Nilsson, Raphael 98 One of the first significant algorithms developed in I Widely used in many applications * xactly like est first search, but using a different criteria for the priority queue: minimize (distance from start) + (estimated distance to goal) priority f(n) = g(n) + h(n) f(n) = priority of a node g(n) = true distance from start h(n) = heuristic distance to goal 9 Optimality of * * in ction Suppose the estimated distance is always less than or equal to the true distance to the goal heuristic is a lower bound 5 nd St 5 nd St 5 st St S h=+ h=7+ Then: when the goal is removed from the priority queue, we are guaranteed to have found a shortest path! 5 th St H=+7 th ve 9 th ve 8 th ve 7 th ve th ve 5 th ve th ve rd ve nd ve

3 pplication of *: Speech Recognition (Simplified) Problem: System hears a sequence of words It is unsure about what it heard or each word, it has a set of possible guesses.g.: Word is one of { hi, high, I } What is the most likely sentence it heard? Speech Recognition as Shortest Path onvert to a shortest path problem: Utterance is a layered egins with a special dummy start node Next: layer of nodes for each word position, one node for each word choice dges between every node in layer i to every node in layer i+ ost of an edge is smaller if the pair of words frequently occur together in real speech Technically: log probability of co occurrence inally: a dummy end node ind shortest path from start to end node W W W Summary: raph Search W W W W W W W epth irst Little memory required Might find non optimal path readth irst Much memory required lways finds optimal path Iterative epth irst Search Repeated depth first searches, little memory required ijskstra s Short Path lgorithm Like S for weighted graphs est irst an visit fewer nodes Might find non optimal path * an visit fewer nodes than S or ijkstra Optimal if heuristic estimate is a lower bound 5 ynamic Programming lgorithmic technique that systematically records the answers to sub problems in a table and re uses those recorded results (rather than re computing them). loyd Warshall for (int k = ; k =< V; k++) for (int i = ; i =< V; i++) for (int j = ; j =< V; j++) if ( ( M[i][k]+ M[k][j] ) < M[i][j] ) M[i][j] = M[i][k]+ M[k][j] Simple xample: alculating the Nth ibonacci number. ib(n) = ib(n ) + ib(n ) Invariant: fter the kth iteration, the matrix includes the shortest paths for all pairs of vertices (i,j) containing only vertices..k as intermediate vertices 7 8

4 Initial state of the matrix: a b c d e a b - - c d e M[i][j] = min(m[i][j], M[i][k]+ M[k][j]) - a d b e - c 9 loyd-warshall - for ll-pairs shortest path a b c d e a - b c d e a d b e inal Matrix ontents - c Network lows S : ata Structures Network low iven a weighted, directed graph =(V,) Treat the edge weights as capacities How much can we flow through the graph? 7 H 9 5 I Network flow: definitions Network flow: definitions efine special source s and sink t vertices efine a flow as a function on edges: apacity: f(v,w) <= c(v,w) onservation: f ( u, v) = for all u v V except source, sink Value of a flow: f = f ( s, v) v Saturated edge: when f(v,w) = c(v,w) apacity: you can t overload an edge onservation: low entering any vertex must equal flow leaving that vertex We want to maximize the value of a flow, subject to the above constraints

5 Network lows iven a weighted, directed graph =(V,) Treat the edge weights as capacities How much can we flow through the graph? s H t 5 ood Idea that oesn t Work Start flow at While there s room for more flow, push more flow across the network! While there s some path from s to t, none of whose edges are saturated Push more flow along the path until some edge is saturated alled an augmenting path How do we know there s still room? xample () onstruct a residual graph: Same vertices dge weights are the leftover capacity on the edges If there is a path s t at all, then there is still room Initial graph no flow low / apacity 7 8 Include the residual capacities xample () xample () ugment along by unit (which saturates ) / / / / / / / / / / / / / / / / low / apacity Residual apacity 9 low / apacity Residual apacity

6 / / low / apacity Residual apacity xample () ugment along (which saturates and ) / / / / / / Now what? There s more capacity in the network but there s no more augmenting paths Network flow: definitions Network flow: definitions efine special source s and sink t vertices efine a flow as a function on edges: apacity: f(v,w) <= c(v,w) Skew symmetry: f(v,w) = f(w,v) onservation: f ( u, v) = for all u v V except source, sink Value of a flow: f = f ( s, v) v Saturated edge: when f(v,w) = c(v,w) apacity: you can t overload an edge Skew symmetry: sending f from u v implies you re sending f, or you could return f from v u onservation: low entering any vertex must equal flow leaving that vertex We want to maximize the value of a flow, subject to the above constraints Main idea: ord ulkerson method Start flow at While there s room for more flow, push more flow across the network! While there s some path from s to t, none of whose edges are saturated Push more flow along the path until some edge is saturated alled an augmenting path How do we know there s still room? onstruct a residual graph: Same vertices dge weights are the leftover capacity on the edges dd extra edges for backwards capacity too! If there is a path s t at all, then there is still room 5

7 xample (5) dd the backwards edges, to show we can undo some flow xample () ugment along (which saturates and, and empties ) / / low / apacity Residual apacity ackwards flow / / / / / / 7 / / low / apacity Residual apacity ackwards flow / / / / / / 8 xample (7) How should we pick paths? inal, maximum flow / / / Two very good heuristics (dmonds Karp): Pick the largest capacity path available Otherwise, you ll just come back to it later so may as well pick it up now / low / apacity Residual apacity ackwards flow / / / / 9 Pick the shortest augmenting path available or a good example why on t Mess this One Up Running time? / / / / / ach augmenting path can t get shorter and it can t always stay the same length So we have at most O() augmenting paths to compute for each possible length, and there are only O(V) possible lengths. ach path takes O() time to compute ugment along, then, then, then Total time = O( V) Should just augment along, and, and be finished

8 Network lows What about multiple sources? Network lows reate a single source, with infinite capacity edges connected to sources Same idea for multiple sinks s 7 H s 7 H s 9 5 t s! 9 s 5 t One more definition on flows We can talk about the flow from a set of vertices to another set, instead of just from one vertex to another: f ( X, Y ) = f ( x, y) x X y Y Should be clear that f(x,x) = So the only thing that counts is flow between the two sets Network cuts Intuitively, a cut separates a graph into two disconnected pieces ormally, a cut is a pair of sets (S, T), such that V = S T S T = {} and S and T are connected subgraphs of 5 Minimum cuts Min ut xample (8) If we cut into (S, T), where S contains the source s and T contains the sink t, S T Of all the cuts (S, T) we could find, what is the smallest (max) flow f(s, T) we will find? 7 apacity of cut = 5 8

9 oincidence? raphut NO! Max flow always equals Min cut Why? If there is a cut with capacity equal to the flow, then we have a maxflow: We can t have a flow that s bigger than the capacity cutting the graph! So any cut puts a bound on the maxflow, and if we have an equality, then we must have a maximum flow. If we have a maxflow, then there are no augmenting paths left Or else we could augment the flow along that path, which would yield a higher total flow. If there are no augmenting paths, we have a cut of capacity equal to the maxflow Pick a cut (S,T) where S contains all vertices reachable in the residual graph from s, and T is everything else. Then every edge from S to T must be saturated (or else there would be a path in the residual graph). So c(s,t) = f(s,t) = f(s,t) = f and we re done oding S : ata Structures ictionaries for ata ompression oes not use statistical knowledge of data. ncoder: s the input is processed develop a dictionary and transmit the index of strings found in the dictionary. ecoder: s the code is processed reconstruct the dictionary to invert the process of encoding. xamples: LZW, LZ77, Sequitur, pplications: Unix ompress, gzip, I 5 5 LZW ncoding lgorithm LZW ncoding xample () Repeat find the longest match w in the dictionary output the index of w put wa in the dictionary where a was the unmatched symbol b a b a b a b a b a 5 5

10 LZW ncoding xample () LZW ncoding xample () b ab a b a b a b a b a b ab ba a b a b a b a b a 55 5 LZW ncoding xample () LZW ncoding xample (5) b ab ba aba a b a b a b a b a b ab ba aba 5 abab a b a b a b a b a LZW ncoding xample () b ab ba aba 5 abab a b a b a b a b a LZW ecoding lgorithm mulate the encoder in building the dictionary. ecoder is slightly behind the encoder. initialize dictionary; decode first index to w; put w? in dictionary; repeat decode the first symbol s of the index; complete the previous dictionary entry with s; finish decoding the remainder of the index; put w? in the dictionary where w was just decoded; 59

11 LZW ecoding xample () LZW ecoding xample (a) b a? a b ab a b LZW ecoding xample (b) LZW ecoding xample (a) b ab b? a b b ab ba a b a LZW ecoding xample (b) LZW ecoding xample (a) b ab ba ab? a b ab b ab ba aba a b ab a 5

12 LZW ecoding xample (b) LZW ecoding xample (5a) b ab ba aba 5 aba? a b ab aba b ab ba aba 5 abab a b ab aba b 7 8 LZW ecoding xample (5b) LZW ecoding xample (a) b ab ba aba 5 abab ba? a b ab aba ba b ab ba aba 5 abab bab a b ab aba ba b 9 7 LZW ecoding xample (b) ecoding xercise b ab ba aba 5 abab bab 7 bab? a b ab aba ba bab ase b c d r

13 ounded Size ounded Size n bits of index allows a dictionary of size n oubtful that long entries in the dictionary will be useful. Strategies when the dictionary reaches its limit.. on t add more, just use what is there.. Throw it away and start a new dictionary.. ouble the dictionary, adding one more bit to indices.. Throw out the least recently visited entry to make room for the new entry. Notes on LZW xtremely effective when there are repeated patterns in the data that are widely spread. Negative: reates entries in the dictionary that may never be used. pplications: Unix compress, I, V. bis modem standard 7 7 LZ77 Ziv and Lempel, 977 is implicit Use the string coded so far as a dictionary. iven that x x...x n has been coded we want to code x n+ x n+...x n+k for the largest k possible. Solution If x n+ x n+...x n+k is a substring of x x...x n then x n+ x n+...x n+k can be coded by <j,k> where j is the beginning of the match. xample ababababa babababababababab... coded ababababa babababa babababab... <,8> 75 7 Solution Problem Solution What if there is no match at all in the dictionary? ababababa cabababababababab... coded Solution. Send tuples <j,k,x> where If k = then x is the unmatched symbol If k > then the match starts at j and is k long and the unmatched symbol is x. If x n+ x n+...x n+k is a substring of x x...x n and x n+ x n+... x n+k x n+k+ is not then x n+ x n+...x n+k x n+k+ can be coded by <j,k, x n+k+ > where j is the beginning of the match. xamples ababababa cabababababababab... ababababa c ababababab ababab... <,,c> <,9,b> 77 78

14 Solution xample a bababababababababababab... <,,a> a b ababababababababababab... <,,b> a b aba bababababababababab... <,,a> a b aba babab ababababababab... <,,b> a b aba babab abababababa bab... <,,a> Surprise ode! a bababababababababababab$ <,,a> a b ababababababababababab$ <,,b> a b ababababababababababab$ <,,$> 79 8 Surprise ecoding <,,a><,,b><,,$> <,,a> a <,,b> b <,,$> a <,,$> b <,,$> a <,9,$> b... <,,$> b <,,$> $ Surprise ecoding <,,a><,,b><,,$> <,,a> a <,,b> b <,,$> a <,,$> b <,,$> a <,9,$> b... <,,$> b <,,$> $ 8 8 Solution The matching string can include part of itself! If x n+ x n+...x n+k is a substring of x x...x n x n+ x n+...x n+k that begins at j < n and x n+ x n+... x n+k x n+k+ is not then x n+ x n+...x n+k x n+k+ can be coded by <j,k, x n+k+ > ounded uffer Sliding Window We want the triples <j,k,x> to be of bounded size. To achieve this we use bounded buffers. Search buffer of size s is the symbols x n s+...x n j is then the offset into the buffer. Look ahead buffer of size t is the symbols x n+...x n+t Match pointer can start in search buffer and go into the look ahead buffer but no farther. match pointer Sliding window aaaabababaaab$ search buffer coded uncoded text pointer look-ahead buffer uncoded tuple <,5,a> 8 8