CS 0 Advanced Data Structures and Algorithms Weighted Graphs July 0, 07 Tong Wang UMass Boston CS 0 July 0, 07 /
Weighted Graphs Each edge has a weight (cost) Edge-weighted graphs Mostly we consider only positive weights Minimum spanning trees Shortest paths Tong Wang UMass Boston CS 0 July 0, 07 /
Minimum Spanning Tree Given a connected undirected graph, we often want the cheapest way to connect all the nodes This is a minimum cost spanning tree A spanning tree is a tree that contains all the nodes in the graph The total cost is just the sum of the edge costs The minimum spanning tree is the spanning tree whose total cost is minimal among all spanning trees The kind of tree involved here has no particular root node, and such trees are called free trees, because they are not tied down by a root Tong Wang UMass Boston CS 0 July 0, 07 /
MST Example Point set in the plane Geometric spanning trees The middle one is the minimum spanning tree The one on the right has shortest paths to the central node Tong Wang UMass Boston CS 0 July 0, 07 /
MST Example CLRS. Tong Wang UMass Boston CS 0 July 0, 07 /
Prim s Algorithm for MST A greedy algorithm: Decide what to do next by selecting the immediately best option, without regard to the global structure Pseudocode Choose an arbitrary vertex s as the first tree node Add s to PrimTree while there are still nontree nodes Choose the edge of minimum weight between a tree node and a nontree node, (u, v) Add (u, v) to TreeEdge and v to PrimTree Tong Wang UMass Boston CS 0 July 0, 07 6 /
Prim s Algorithm for MST Choose an arbitrary node s Initialize tree T = { s } Initialize E = { } Repeat until all nodes are in T: Choose an edge (u, v) with minimum weight such that u is in T and v is not Add v to T, add (u, v) to E Output (T, E) Tong Wang UMass Boston CS 0 July 0, 07 7 /
Correctness of Prim s Algorithm Prim s alogrithm creates a spanning tree No cycle can be introduced by adding edges between tree and nontree vertices Why does the tree has the minimal weight? Proof by contradiction Assume Prim s algorithm does not return an MST Then there is an edge (x, y) that does not belong to the MST Instead, (x, y) is replaced by a different edge (v, ) To maintain the property of spanning, when we remove an edge, we must add another edge The cost of (x, y), however, is not greater than the cost of (v, ) Thus Prim s tree is still minimal after all Tong Wang UMass Boston CS 0 July 0, 07 8 /
Correctness of Prim s Algorithm If we remove (x, y), a different edge must be added In the new spanning tree, let (v, ) be the first edge along the path from x to y that is not in the tree generated by Prim s algorithm Cost(x, y) Cost(v, ) Substitute (x, y) for (v, ) back into the tree Repeat until the tree by Prim s algorithm is restored Tong Wang UMass Boston CS 0 July 0, 07 9 /
Naive Implementation of Prim s Algorithm Prim s algorithm grows the MST in stages, one vertex at a time Thus n iterations During each iteration, search all edges and find the minimum weight edge with one vertex in the tree and the other vertex not in the tree Thus each iteration takes O(m) time O(m n) total time Tong Wang UMass Boston CS 0 July 0, 07 0 /
Better Implementation of Prim s Algorithm Maintain an array Dist that stores the distances from nontree nodes to the nearest tree nodes Initialize Dist to MAXINT When vertex u is added to the tree, loop through its adjacency list Consider edge (u, v) If v is already a tree node, ignore it If v is not a tree node, and if Cost(u, v) < Dist[v] Dist[v] = Cost(u, v) parent[v] = u After updating Dist, find the node with the minimum distance to be the next tree node O(n ) total time Further improvement: Use a priority queue, so that the node with the minimum distance can be found in O(log n) time, for a total time of O(m log n) Tong Wang UMass Boston CS 0 July 0, 07 /
Kruskal s algorithm This algorithm works like building the Huffman code tree Start with all the nodes separate and then join them incrementally, using a greedy approach that turns out to give the optimal solution Set up a partition, a set of sets of nodes, starting with one node in each set Then find the minimal edge to join two sets, and use that as a tree edge, and join the sets Each set in the partition is a connected component Tong Wang UMass Boston CS 0 July 0, 07 /
Example v v 8 v 0 6 v v v v v Tong Wang UMass Boston CS 0 July 0, 07 /
Example v v v v v v v v v v v v v v v Tong Wang UMass Boston CS 0 July 0, 07 / v
Example v v v v v v v 6 v v v v v v v 7 v 8 Tong Wang UMass Boston CS 0 July 0, 07 / v
Running Example The component numbers for the first few steps: Node V 0 V V V V V V 6 Set 0 6 V 0 -V 0 0 6 ( joins 0) V -V 6 0 0 (6 joins ) V 0 -V 0 0 0 V -V 0 0 0 0 and so on Edges are used in cost order If an edge has both to and from vertices with the same component number, it is skipped The resulting tree is defined by the edges used Tong Wang UMass Boston CS 0 July 0, 07 6 /
Correctness of Kruskal s Algorithm The algorithm adds n edges without creating a cycle Thus it must create a spanning tree Why does the tree has the minimal weight? Proof by contradiction Assume Kruskal s algorithm does not return an MST There is a real MST T min Then there is an edge (x, y) in the tree by Kruskal s that is not in T min Insert (x, y) in T min and create a cycle with the path P from x to y When Kruskal s adds the edge (x, y), x and y are in different components That is, one of the edges in path P is not there yet; let it be (v, ) This means Cost(x, y) Cost(v, ) Tong Wang UMass Boston CS 0 July 0, 07 7 /
Correctness of Kruskal s Algorithm If we add (x, y) to T min, a cycle is formed The edge (v, ) is added after (x, y) Thus Cost(x, y) Cost(v, ) By replacing (v, ) with (x, y), we would not increase the cost Repeat until T min is transformed into the tree by Kruskal s Tong Wang UMass Boston CS 0 July 0, 07 8 /
Kruskal s Algorithm Note that several edges are left unprocessed in the edge list These are the high-cost edges we want to avoid using Pseudocode: Put the edges in a priority queue ordered by edge weight Initialize the tree edge set T to the empty set Repeat n times Get next edge (u, v) from PQ If component[u]!= component[v] Add (u, v) to T Merge component[u] and component[v] In each iteration, test the connectivity of two trees plus an edge This can be done with BFS or DFS on a sparse graph with at most n edges and n vertices. Thus each iteration takes O(n) time O(m n) total time If we can determine the connectivity of two trees plus an edge in O(log n) time, then the total time for Kruskal s algorithm is O(m log n) Tong Wang UMass Boston CS 0 July 0, 07 9 /
Data Structures for Kruskal s Algorithm Need two operations SameComponent(u, v): returns true or false MergeComponent(C, C): returns the merged component Best data structure: Union-Find Tong Wang UMass Boston CS 0 July 0, 07 0 /
The Union-Find Data Structure Let n be the number of elements in a set A partition divides the elements into a collection of disjoint subsets Examples: SCC, vertex coloring, vertices in Kruskal s algorithm Each subset in the partition is represented by one of its elements The operation Find(u) returns the representative of the subset that u belongs to The operation Union(u, v) joins the subsets containing u and v into a new subset Tong Wang UMass Boston CS 0 July 0, 07 /
The Union-Find Data Structure vertex 6 7 parent rank 0 0 0 0 size find(x) if (parent(x) == x) return x else return find(parent(x)) Each subset is represented by a backwards tree, where each child node has a pointer to its parent The rank of a tree is (without path compression) its height Initially, each vertex of the graph is in its own tree, and its parent pointer points to itself Find(i): Follow the parent pointers upwards until reaching a node that points to itself; return the label of this node as the label of the subset Union(i, j): Use the root of the higher-ranked tree as the root of the union it becomes the parent of the root of the lower-ranked tree Tong Wang UMass Boston CS 0 July 0, 07 /
Runtime Analysis of Union-Find Runtime of Union and Find depends on the heights of the trees When two trees of rank r are merged, the rank of the resulting tree is r + This is the only way to increase the rank How high can the trees become? The smallest tree of rank 0 has node The smallest tree of rank has nodes Merging a rank-0 tree to a rank- tree is still a rank- tree Merging two rank- trees gives a rank- tree The smallest tree of rank has nodes The smallest tree of rank k has k nodes by induction The maximum height is O(log n) O(log n) time for union or find Tong Wang UMass Boston CS 0 July 0, 07 /
Union-Find with Path Compression Path compression find(x) if (parent(x)!= x) parent(x) = find(parent(x)) return parent(x) Amortized runtime per operation of union-find with path compression is O() Tong Wang UMass Boston CS 0 July 0, 07 /
Back to Kruskal s Algorithm Initialize all vertices as singleton trees for Union-Find Keep all edges in a priority queue Loop through the edges out of PQ (at most m iterations) For edge (u, v), use Union-Find to determine whether u and v are in separate trees O(log n) time If u and v are in the same tree, move on to the next edge If not, join the two trees Total runtime O(m log n) Tong Wang UMass Boston CS 0 July 0, 07 /
Unweighted Single-Source Shortest-Path Problem Find the shortest path (measured by number of edges) from a designated vertex s to every vertex A special case of the weighted shortest-path problem by giving weight to all edges This problem has an efficient solution: BFS The problem becomes more complicated when (different) weights are allowed Tong Wang UMass Boston CS 0 July 0, 07 6 /
Positive-Weighted, Single-Source, Shortest-Path Problem Many times the edges have weights indicating that the relations between nodes are not always equal Example: distances between cities In this case BFS does not work Goal: Find the shortest path (measured by total cost) from a designated vertex s to every vertex in a weighted graph Weights (costs) are assigned to edges All edge costs are positive Tong Wang UMass Boston CS 0 July 0, 07 7 /
Dijkstra s Algorithm A greedy technique that works well Let weight(u, v) be the cost for edge (u, v) If the shortest path from s to t passes through x, then the path from s to x must be shortest as well Starting from vertex s, we set up an array to hold all the running min costs D i from s to vertex i Being greedy, we visit a new node v with the least cost from s that has not been visited yet Then we update the min cost D i by passing through v D i = min{d i, D v + weight(v, i)} Repeat by visiting the next node with the lowest cost D i that has not been visited Tong Wang UMass Boston CS 0 July 0, 07 8 /
Running Example, Starting From V 0 0 v v 8 v 0 6 v 0 v v 8 v 0 6 v 0 9 v v 8 v 0 6 v 0 9 v v 8 v 0 6 v Tong Wang UMass Boston CS 0 July 0, 07 9 /
Running Example, Starting From V 0 0 9 v v 8 v 0 6 v 0 8 v 6 v 8 v 0 6 v 0 6 v 7 v 8 v 0 6 v 0 6 v 8 v 8 v 0 6 v Tong Wang UMass Boston CS 0 July 0, 07 0 /
Path with Minimum Weights from V 0 0 v 8 v visiting D 0 D D D D D D 6 start 0 V 0 V 9 V 9 V 9 V 8 V 6 6 V final: 0 6 v 0 6 v Tong Wang UMass Boston CS 0 July 0, 07 /
Pseudocode for Dijkstra s Algorithm Input: a directed graph G and vertex s D i : an array of distances from s to each node Initialize D i = For each edge (s, i) in the adjacency list of s Set D i = weight(s, i) Initialize tree T = { s } Repeat until all nodes are in T Choose a node v not in T that minimizes D v Add v to T For each edge (v, w) such that w T D w = min(d w, D v + weight(v, w)) Tong Wang UMass Boston CS 0 July 0, 07 /
Prim and Dijkstra Are Very Similar Prim s Algorithm: Choose an arbitrary node s Initialize tree T = { s } Initialize E = { } Repeat until all nodes are in T: Choose an edge (u, v) with minimum weight such that u is in T and v is not Add v to T, add (u, v) to E Output (T, E) Dijkstra s Algorithm: Input: a directed graph G and source node s D i : an array of distances from s to each node Initialize D i = For each edge (s, i) in the adjacency list of s Set D i = weight(s, i) Initialize tree T = { s } Repeat until all nodes are in T Choose a node v not in T that minimizes D v Add v to T For each edge (v, w) such that w T D w = min(d w, D v + weight(v, w)) Tong Wang UMass Boston CS 0 July 0, 07 /
Correctness of Dijkstra The argument of the correctness of Dijkstra s algorithm is similar to that of Prim s algorithm By contradiction Assume there is a shorter path An edge (u, v) in Dijkstra s tree is absent in the shorter path Add (u, v) and get a cycle One of the edges in the cycle is not there when (u, v) is added by Dijkstra (u, v) is no worse than this edge Tong Wang UMass Boston CS 0 July 0, 07 /