22 Elementary Graph Algorithms. There are two standard ways to represent a

Transcription

1 VI Graph Algorithms Elementary Graph Algorithms Minimum Spanning Trees Single-Source Shortest Paths All-Pairs Shortest Paths 22 Elementary Graph Algorithms There are two standard ways to represent a graph =(, ): as a collection of adjacency lists or as an adjacency matrix Either way applies to both directed and undirected graphs The adjacency-list representation provides a compact way to represent sparse graphs those for which MAT AADS, Fall Nov

2 23 Minimum Spanning Trees Electronic circuit designs often need to make the pins of several components electrically equivalent by wiring them together To interconnect a set of pins, we can use an arrangement of 1wires, each connecting two pins Of all such arrangements, the one that uses the least amount of wire is usually the most desirable MAT AADS, Fall Nov Model this wiring problem with a connected, undirected graph =,, where is the set of pins, is the set of possible interconnections between pairs of pins, and for each edge (,, we have a weight (, ) specifying the cost (amount of wire) to connect and Find an acyclic subset that connects all of the vertices and whose total weight is minimized = (, ) (, MAT AADS, Fall Nov

3 Since is acyclic and connects all of the vertices, it must form a tree, which we call a spanning tree since it spans the graph We call the problem of determining the tree the minimum-spanning-tree problem (MST) MAT AADS, Fall Nov We examine two algorithms for solving the MST problem: Kruskal s and Prim s algorithms We can easily make each of them run in time ( lg ) using ordinary binary heaps By using Fibonacci heaps, Prim s algorithm runs in time ( + lg ), which improves over the binary-heap implementation if The two algorithms are greedy algorithms Greedy strategy does not generally guarantee nding globally optimal solutions to problems For the MST problem we can prove that greedy strategies do yield a tree with minimum weight MAT AADS, Fall Nov

4 23.1 Growing a minimum spanning tree Let us have a connected, undirected graph =, with a weight function :, and we wish to nd a MST for The following generic method grows the MST one edge at a time The generic method manages a set of edges, maintaining the following loop invariant: Prior to each iteration, is a subset of some minimum spanning tree MAT AADS, Fall Nov At each step, we determine an edge (, ) that we can add to without violating this invariant, in the sense that (, ) is also a subset of a MST We call such an edge a safe edge for GENERIC-MST(, ) 1. ; 2. while does not form a spanning tree 3. nd an edge (, ) that is safe for 4. (, ) 5. return MAT AADS, Fall Nov

5 Initialization: After line 1, the set trivially satis es the loop invariant Maintenance: The loop in lines 2 4 maintains the invariant by adding only safe edges Termination: All edges added to are in a MST, and so the set returned in line 5 must be a MST The tricky part is nding a safe edge in line 3 One must exist, since the invariant dictates that there is a spanning tree such that Within the while loop body, must be a proper subset of, and there must be an edge (, s.t. (, ) and (, ) is safe for MAT AADS, Fall Nov Acut (, ) of an undirected graph =, is a partition of An edge (, crosses the cut if one of its endpoints is in and the other is in We say that a cut respects a set of edges if no edge in crosses the cut Alight edge crossing a cut has the minimum weight of any edge crossing the cut Note that there can be ties More generally, an edge is a light edge satisfying a given property if its weight is the minimum of any edge satisfying the property MAT AADS, Fall Nov

6 MAT AADS, Fall Nov Theorem 23.1: Let =, be a connected, undirected graph with a real-valued weight function on. Let be a subset of that is included in some MST for, let (, ) be any cut of that respects, and let (, ) be a light edge crossing (, ). Then, edge (, ) is safe for. MAT AADS, Fall Nov

7 Theorem 23.1 gives us a better understanding of the workings of the GENERIC-MST method on a connected graph =, As the method proceeds, the set is always acyclic; otherwise, a MST including would contain a cycle, which is a contradiction At any point in the execution, the graph =, is a forest, and each of the connected components of is a tree Some of the trees may contain just one vertex, as is the case, e.g., when the method begins: is empty and the forest contains trees, one for each vertex MAT AADS, Fall Nov Moreover, any safe edge (, ) for connects distinct components of, since (, ) must be acyclic The while loop in lines 2 4 of GENERIC-MST executes 1times because it nds one of the 1edges of a minimum spanning tree in each iteration Initially, when, there are trees in, and each iteration reduces that number by 1 When the forest contains only a single tree, the method terminates MAT AADS, Fall Nov

8 Corollary 23.2: Let =, be a connected, undirected graph with a real-valued weight function defined on. Let be a subset of that is included in some MST for, and let =, be a connected component (tree) in the forest =,. If (, ) is a light edge connecting to some other component in, then (, ) is safe for. Proof: The cut (, ) respects, and (, ) is a light edge for this cut. Therefore, (, ) is safe for. MAT AADS, Fall Nov The algorithms of Kruskal and Prim These algorithms use a speci c rule to determine a safe edge in line 3 of GENERIC-MST In Kruskal s algorithm, the set is a forest whose vertices are all those of the given graph The safe edge added to is always a least-weight edge in the graph that connects two distinct components In Prim s algorithm, the set forms a single tree The safe edge added to is always a least-weight edge connecting the tree to a vertex not in the tree MAT AADS, Fall Nov

9 Kruskal s algorithm Find a safe edge to add to the growing forest by nding, of all the edges that connect any two trees in the forest, an edge (, ) of least weight Let and denote the two trees that are connected by (, ) Since (, ) must be a light edge connecting to some other tree, Corollary 23.2 implies that (, ) is a safe edge for This is a greedy algorithm because at each step it adds an edge of least possible weight MAT AADS, Fall Nov MST-KRUSKAL(, ) for each vertex. 3. MAKE-SET( ) 4. sort the edges of. into nondecreasing order by weight 5. for each edge (,., taken in nondecreasing order by weight 6. if FIND-SET( FIND-SET( ) 7. (, ) 8. UNION(, ) 9. return MAT AADS, Fall Nov

10 FIND-SET( ) returns a representative element from the set that contains Determine whether and belong to the same tree by testing FIND-SET = FIND-SET( ) UNION procedure combines trees Lines 1 3 initialize the set to the empty set and create trees, one containing each vertex The for loop in lines 5 8 examines edges in order of weight, from lowest to highest MAT AADS, Fall Nov The loop checks, for each edge (, ), whether the endpoints and belong to the same tree If they do, then the edge (, ) cannot be added to the forest without creating a cycle, and the edge is discarded Otherwise, the two vertices belong to different trees In this case, line 7 adds the edge (, ) to, and line 8 merges the vertices in the two trees MAT AADS, Fall Nov

11 MAT AADS, Fall Nov MAT AADS, Fall Nov

12 The running time depends on how we implement the disjoint-set data structure Initializing the set (line 1) takes (1) time, and the time to sort the edges (line 4) is ( lg ) The for loop (lines 5 8) performs ( ) FIND-SET and UNION operations on the disjoint-set forest With the MAKE-SET operations, these take a total of ( + )time is the very slowly growing funct We assume that is connected, so have 1, and so the disjoint-set operations take ( ) time Moreover, since = (lg ) = (lg ), the total running time of Kruskal s algorithm is ( lg ) Observing that <, we have lg = (lg ), and the running time of Kruskal s algorithm is ( lg ) MAT AADS, Fall Nov Prim s algorithm Prim s algorithm has the property that the edges in the set always form a single tree We start from an arbitrary root vertex and grow until the tree spans all the vertices in Each step adds to a light edge that connects to an isolated vertex (no edge of is incident) By Corollary 23.2, this rule adds only edges that are safe for and eventually forms a MST Greedy: each step adds to the tree an edge that contributes the min amount to the tree s weight MAT AADS, Fall Nov

13 MAT AADS, Fall Nov MAT AADS, Fall Nov

14 To implement Prim s algorithm ef ciently, we need a fast way to select a new edge to add to the tree formed by the edges in In the pseudocode below, the connected graph and the root of the MST to be grown are inputs to the algorithm All vertices that are not in the tree reside in a min-priority queue based on a key attribute For each vertex, the attribute. is the minimum weight of any edge connecting to a vertex in the tree; by convention. if there is no such edge MAT AADS, Fall Nov Attribute. names the parent of in the tree The algorithm implicitly maintains the set from GENERIC-MST as =,. When the algorithm terminates, the min-priority queue is empty; the minimum spanning tree for is thus =,. MAT AADS, Fall Nov

15 MST-PRIM(,, ) 1. for each NIL while 7. EXTRACT-MIN( ) 8. for each. [ ] 9. if and, < (, ) MAT AADS, Fall Nov The algorithm maintains the following three-part loop invariant: Prior to each iteration of the while loop of lines 6 11, 1. =,. 2. The vertices already placed into the minimum spanning tree are those in 3. For all vertices, if. NIL, then. and. is the weight of a light edge,. connecting to some vertex already placed into the MST MAT AADS, Fall Nov

16 Line 7 identi es a vertex incident on a light edge that crosses the cut (, ) Removing from the set adds it to the set of vertices in the tree, thus adding (,. ) to The for loop of lines 8 11 updates the and attributes of every vertex adjacent to but not in the tree, thereby maintaining the third part of the loop invariant MAT AADS, Fall Nov The total time for Prim s algorithm is ( lg + lg ) = ( lg ), which is asympt. the same as for Kruskal s algorithm We can improve the asymptotic running time of Prim s algorithm by using Fibonacci heaps If a Fibonacci heap holds elements, an EXTRACT-MIN operation takes (lg ) amortized time and a DECREASE-KEY operation (to implement line 11) takes (1) amortized time Therefore, by using a Fibonacci heap for the min-priority queue, the running time of Prim s algorithm improves to ( + lg ) MAT AADS, Fall Nov

17 24 Single-Source Shortest Paths In a shortest-paths problem, we are given a weighted, directed graph =,, with weight function : mapping edges to real-valued weights The weight ( ) of path =,,, is the sum of the weights of its constituent edges: = (, ) MAT AADS, Fall Nov We de ne the shortest-path weight (, ) from to by, = min if there is a path from to otherwise A shortest path from vertex to vertex is then de ned as any path with weight (, ) MAT AADS, Fall Nov

18 Optimal substructure of a shortest path Shortest-paths algorithms typically rely on the property that a shortest path between two vertices contains other shortest paths within it Recall that optimal substructure is one of the key indicators that dynamic programming and the greedy method might apply The following lemma states the optimalsubstructure property of shortest paths more precisely MAT AADS, Fall Nov Lemma 24.1 (Subpaths of shortest paths are shortest paths) Given a weighted, directed graph =,, with weight function :, let =,,, be a shortest path from vertex to vertex and, for any and such that, let =,,, be the subpath from vertex to. Then, is a shortest path from to. MAT AADS, Fall Nov

19 Proof: If we decompose path into then we have that = + +. Now, assume that there is a path weight <. Then, from to with is a path from to whose weight + + <, which contradicts the assumption that is a shortest path from to. MAT AADS, Fall Nov Negative-weight edges If the graph =, contains no negativeweight cycles reachable from the source, then for all, the shortest-path weight, is well de ned, even if it has a negative value If contains a negative-weight cycle reachable from, shortest-path weights aren t well de ned No path from to a vertex on the cycle can be a shortest path we always nd a path with lower weight by following the proposed shortest path and then traversing the negative-weight cycle MAT AADS, Fall Nov

20 If there is a negative-weight cycle on some path from to, we de ne, MAT AADS, Fall Nov Cycles A shortest path cannot either contain a positiveweight cycle, since removing the cycle from the path produces a path with the same source and destination vertices and a lower path weight If =,,, is a path and =,,, is a positive-weight cycle on this path ( = and ( ) >0), then the path =,,,,,, has weight = < ( ), and so cannot be a shortest path from to MAT AADS, Fall Nov

21 If there is a shortest path from a source vertex to a destination vertex that contains a 0-weight cycle, then there is another shortest path from to without this cycle We can repeatedly remove 0-weight cycles from the path until the shortest path is cycle-free Therefore, we can assume that when we are nding shortest paths, they have no cycles, i.e., they are simple paths Any acyclic path in =, contains at most distinct vertices, and at most 1edges Thus, we can restrict our attention to shortest paths of at most 1 edges MAT AADS, Fall Nov Relaxation The attribute. is an upper bound on the weight of a shortest path from source to We call. a shortest-path estimate The following ( )-time procedure initializes the shortest-path estimates and predecessors ( ): INITIALIZE-SINGLE-SOURCE(, ) 1. for each vertex NIL MAT AADS, Fall Nov

22 The process of relaxing an edge (, ) consists of testing whether we can improve the shortest path to found so far by going through and, if so, updating. and. A relaxation step may decrease the value of the shortest-path estimate. and update s predecessor attribute. The following code performs a relaxation step on edge (, ) in (1) time: RELAX(,, ) 1. if. >. + (, ) (, ) 3.. MAT AADS, Fall Nov MAT AADS, Fall Nov

23 Properties of shortest paths and relaxation To prove the following algorithms correct, we appeal to several properties of shortest paths and relaxation: Triangle inequality (Lemma 24.10) For any edge (,, we have,, +, Upper-bound property (Lemma 24.11) We always have., for all vertices, and once. achieves the value,, it never changes MAT AADS, Fall Nov No-path property (Corollary 24.12) If there is no path from to, then we always have. =, Convergence property (Lemma 24.14) If is a shortest path in for some,, and if. = (, ) at any time prior to relaxing edge (, ), then. = (, ) at all times afterward MAT AADS, Fall Nov

24 Path-relaxation property (Lemma 24.15) If =,,, is a shortest path from = to, and we relax the edges of in the order (, ),(, ),,(, ), then. = (, ). This property holds regardless of any other relaxation steps that occur, even if they are intermixed with relaxations of the edges of Predecessor-subgraph property (Lemma 24.17) Once. = (, ) for all, the predecessor subgraph is a shortest-paths tree rooted at MAT AADS, Fall Nov The Bellman-Ford algorithm Solves the single-source shortest-paths problem in the case in which weights may be negative Given a graph =, with source and weight function, it returns a boolean value indicating whether or not there is a negativeweight cycle that is reachable from the source If there is such a cycle, the algorithm indicates that no solution exists If there is no such cycle, the algorithm produces the shortest paths and their weights MAT AADS, Fall Nov

25 Relax edges, progressively decreasing. until we achieve the actual shortest-path weight (, ) BELLMAN-FORD(,, ) 1. INITIALIZE-SINGLE-SOURCE(, ) 2. for =1to for each edge,. 4. RELAX,, 5. for each edge,. 6. if. >. + (, ) 7. return FALSE 8. return TRUE MAT AADS, Fall Nov MAT AADS, Fall Nov

26 The algorithm makes 1 passes over the edges of the graph Each pass is one iteration of the for loop of lines 2 4 and consists of relaxing each edge of the graph once After making 1passes, lines 5 8 check for a negative-weight cycle and return the appropriate boolean value The Bellman-Ford algorithm runs in time ( ), the initialization in line 1 takes ( ) time, each of the 1passes over the edges in lines 2 4 takes ( ) time, and the for loop of lines 5 7 takes ( ) time MAT AADS, Fall Nov Lemma 24.2: Let =, be a weighted, directed graph with source and weight function :, and assume that contains no negative-weight cycles that are reachable from. Then, after the 1iterations of the for loop of lines 2 4 of BELLMAN-FORD, we have. = (, ) for all vertices that are reachable from. Proof: We prove the lemma by appealing to the path-relaxation property. MAT AADS, Fall Nov

27 Consider any vertex that is reachable from, and let =,,,, where = and =, be any shortest path from to. Because shortest paths are simple, has at most 1edges, and so 1. Each of the 1iterations of the for loop of lines 2 4 relaxes all edges. Among the edges relaxed in the th iteration, for =1,2,,, is (, ). By the path-relaxation property, therefore,. =. =, = (, ). MAT AADS, Fall Nov Corollary 24.3: Let =, be a weighted, directed graph with source and weight function :, and assume that contains no negative-weight cycles that are reachable from. Then, for each vertex, there is a path from to if and only if BELLMAN-FORD terminates with. when it is run on. MAT AADS, Fall Nov

28 Theorem 24.4 (Correctness of the B-F algorithm) Let BELLMAN-FORD be run on a weighted, directed graph =(, ) with source and weight function :. If contains no negative-weight cycles that are reachable from, then the algorithm returns TRUE, we have. = (, ) for all vertices, and the predecessor subgraph is a shortest-paths tree rooted at. If does contain a negative-weight cycle reachable from, then the algorithm returns FALSE. MAT AADS, Fall Nov Dijkstra s algorithm Dijkstra s algorithm solves the single-source shortest-paths problem on a weighted, directed graph =(, )for the case in which all edge weights are nonnegative Therefore, we assume that (, 0for each edge (, With a good implementation, the running time of Dijkstra s algorithm is lower than that of the Bellman-Ford algorithm MAT AADS, Fall Nov

29 Dijkstra s algorithm maintains a set of vertices whose nal shortest-path weights from the source have already been determined The algorithm repeatedly selects the vertex with the minimum shortest-path estimate, adds to, and relaxes all edges leaving The following implementation uses a minpriority queue of vertices, keyed by their values MAT AADS, Fall Nov DIJKSTRA(,, ) 1. INITIALIZE-SINGLE-SOURCE(, ) while 5. EXTRACT-MIN( ) for each vertex. [ ] 8. RELAX (,, ) MAT AADS, Fall Nov

30 Shaded edges indicate predecessor values Black vertices are in the set MAT AADS, Fall Nov Maintain the invariant that = at the start of each iteration of the while loop of lines 4 8 Line 3 initializes the min-priority queue to contain all the vertices in ; since ; at that time, the invariant is true after line 3 Each time through the while loop, line 5 extracts a vertex from = and line 6 adds it to set, thereby maintaining the invariant The rst time through this loop, = Vertex, therefore, has the smallest shortestpath estimate of any vertex in MAT AADS, Fall Nov

31 Then, lines 7 8 relax each edge (, ) leaving, thus updating the estimate. and the predecessor. if we can improve the shortest path to found so far by going through Observe that the algorithm never inserts vertices into after line 3 and that each vertex is extracted from and added to exactly once, so that the while loop iterates exactly times Because Dijkstra s algorithm always chooses the lightest or closest vertex in to add to set, we say that it uses a greedy strategy MAT AADS, Fall Nov Theorem 24.6 (Correctness of Dijkstra s algorithm) Dijkstra s algorithm, run on a weighted, directed graph =(, )with non-negative weight function and source, terminates with. = (, ) for all vertices. Proof: We use the following loop invariant: At the start of each iteration of the while loop of lines 4 8,. = (, ) for each vertex. It suf ces to show for each vertex, we have. = (, ) at the time when is added to set. Once. = (, ), we rely on the upper-bound property to show that it holds thereafter. MAT AADS, Fall Nov

32 Initialization: Initially, =, and so the invariant is trivially true. Maintenance: We wish to show that in each iteration,. = (, ) for the vertex added to set. For the purpose of contradiction, let be the rst vertex for which. (, ) when it is added to set. Focus on the situation at the beginning of the iteration of the while loop in which is added to and derive the contradiction that. = (, ) at that time by examining a shortest path from to. We must have because is the rst vertex added to set and. =, = 0 at that time. MAT AADS, Fall Nov Because, we also have that just before is added to. There must be some path from to, for otherwise. =, by the nopath property, which would violate our assumption that. (, ). Because there is at least one path, there is a shortest path from to. Prior to adding to, path connects a vertex in ( ) to a vertex in ( ). Consider the rst vertex along such that, and let be s predecessor along. Decompose path into. (Either path or may have no edges.) MAT AADS, Fall Nov

33 MAT AADS, Fall Nov We claim that. =, when is added to. To prove this claim, observe that. Then, because we chose as the rst vertex for which. (, ) when it is added to, we had. =, when was added to. Edge (, ) was relaxed at that time, and the claim follows from the convergence property. We can now obtain a contradiction to prove that. = (, ). Because appears before on a shortest path from to and all edge weights are non-negative (notably those on path ), we have,,, and thus. =,,.. MAT AADS, Fall Nov

34 But because both vertices and were in when was chosen in line 5, we have... Thus, the two inequalities above are in fact equalities, giving. =, =, =.. Consequently,. =,, which contradicts our choice of. We conclude that. =, when is added to, and that this equality is maintained at all times thereafter. Termination: At termination, which, along with our invariant that =, implies that =. Thus,. =, for all vertices. MAT AADS, Fall Nov