Lecture 10. Elementary Graph Algorithm Minimum Spanning Trees

Similar documents
Graphs. Graph G = (V, E) Types of graphs E = O( V 2 ) V = set of vertices E = set of edges (V V)

Elementary Graph Algorithms

Design and Analysis of Algorithms

Minimum Spanning Trees Ch 23 Traversing graphs

Graph Representation

Representations of Graphs

Unit 2: Algorithmic Graph Theory

Chapter 22. Elementary Graph Algorithms

Part VI Graph algorithms. Chapter 22 Elementary Graph Algorithms Chapter 23 Minimum Spanning Trees Chapter 24 Single-source Shortest Paths

Graphs. Graphs. Representation of Graphs. CSE 680 Prof. Roger Crawfis. Two standard ways. Graph G = (V, E)» V = set of vertices

Graph Algorithms: Chapters Part 1: Introductory graph concepts

Review: Graph Theory and Representation

Graph representation

CS583 Lecture 09. Jana Kosecka. Graph Algorithms Topological Sort Strongly Connected Component Minimum Spanning Tree

COT 6405 Introduction to Theory of Algorithms

DFS & STRONGLY CONNECTED COMPONENTS

Graph: representation and traversal

Graph Search. Adnan Aziz

Introduction to Algorithms. Lecture 11

Elementary Graph Algorithms. Ref: Chapter 22 of the text by Cormen et al. Representing a graph:

Graph Algorithms. Definition

Minimum Spanning Trees

Outlines: Graphs Part-2

CHAPTER 23: ELEMENTARY GRAPH ALGORITHMS Representations of graphs

CSI 604 Elementary Graph Algorithms

Computer Science & Engineering 423/823 Design and Analysis of Algorithms

Basic Graph Algorithms

Computer Science & Engineering 423/823 Design and Analysis of Algorithms

Minimum Spanning Trees

Minimum Spanning Trees

CS Elementary Graph Algorithms

CS Elementary Graph Algorithms

Graph implementations :

Tutorial. Question There are no forward edges. 4. For each back edge u, v, we have 0 d[v] d[u].

Tree. number of vertices. Connected Graph. CSE 680 Prof. Roger Crawfis

CS 341: Algorithms. Douglas R. Stinson. David R. Cheriton School of Computer Science University of Waterloo. February 26, 2019

Basic Graph Algorithms (CLRS B.4-B.5, )

Undirected Graphs. DSA - lecture 6 - T.U.Cluj-Napoca - M. Joldos 1

Taking Stock. IE170: Algorithms in Systems Engineering: Lecture 16. Graph Search Algorithms. Recall BFS

Lecture Notes for Chapter 23: Minimum Spanning Trees

Graph: representation and traversal

Minimum Spanning Trees My T. UF

COP 4531 Complexity & Analysis of Data Structures & Algorithms

Data Structures. Elementary Graph Algorithms BFS, DFS & Topological Sort

Minimum Spanning Trees

Week 11: Minimum Spanning trees

Partha Sarathi Manal

Introduction to Algorithms

Greedy Algorithms. At each step in the algorithm, one of several choices can be made.

Graph Theory. Many problems are mapped to graphs. Problems. traffic VLSI circuits social network communication networks web pages relationship

CS473-Algorithms I. Lecture 14-A. Graph Searching: Breadth-First Search. Cevdet Aykanat - Bilkent University Computer Engineering Department

Introduction to Algorithms

Graph Algorithms. Chapter 22. CPTR 430 Algorithms Graph Algorithms 1

Taking Stock. IE170: Algorithms in Systems Engineering: Lecture 17. Depth-First Search. DFS (Initialize and Go) Last Time Depth-First Search

Taking Stock. Last Time Flows. This Time Review! 1 Characterize the structure of an optimal solution

Announcements Problem Set 5 is out (today)!

Graphs. CSE 2320 Algorithms and Data Structures Alexandra Stefan and Vassilis Athitsos University of Texas at Arlington

CS60020: Foundations of Algorithm Design and Machine Learning. Sourangshu Bhattacharya

Week 12: Minimum Spanning trees and Shortest Paths

Week 5. 1 Analysing BFS. 2 Depth-first search. 3 Analysing DFS. 4 Dags and topological sorting. 5 Detecting cycles. CS 270 Algorithms.

CS 561, Lecture 9. Jared Saia University of New Mexico

Trees and Graphs Shabsi Walfish NYU - Fundamental Algorithms Summer 2006

CS473-Algorithms I. Lecture 13-A. Graphs. Cevdet Aykanat - Bilkent University Computer Engineering Department

Practical Session No. 12 Graphs, BFS, DFS, Topological sort

CS 220: Discrete Structures and their Applications. graphs zybooks chapter 10

1 Start with each vertex being its own component. 2 Merge two components into one by choosing the light edge

Minimum Spanning Trees Ch 23 Traversing graphs

CS 270 Algorithms. Oliver Kullmann. Breadth-first search. Analysing BFS. Depth-first. search. Analysing DFS. Dags and topological sorting.

W4231: Analysis of Algorithms

Minimum-Spanning-Tree problem. Minimum Spanning Trees (Forests) Minimum-Spanning-Tree problem

Data Structures and Algorithms. Werner Nutt

Minimum Spanning Trees and Prim s Algorithm

Data Structures and Algorithms. Chapter 7. Graphs

CS 270 Algorithms. Oliver Kullmann. Analysing BFS. Depth-first search. Analysing DFS. Dags and topological sorting.

CSCE 750, Fall 2002 Notes 6 Page Graph Problems ffl explore all nodes (breadth first and depth first) ffl find the shortest path from a given s

Computer Science & Engineering 423/823 Design and Analysis of Algorithms

Introduction to Algorithms

Proof: if not f[u] < d[v], then u still grey while v is being visited. DFS visit(v) will then terminate before DFS visit(u).

Unit 5F: Layout Compaction

Unit 3: Layout Compaction

Shortest Path Routing Communications networks as graphs Graph terminology Breadth-first search in a graph Properties of breadth-first search

22 Elementary Graph Algorithms. There are two standard ways to represent a

Homework Assignment #3 Graph

Graph Theory. ICT Theory Excerpt from various sources by Robert Pergl

Solutions to relevant spring 2000 exam problems

22 Elementary Graph Algorithms. There are two standard ways to represent a

Outline. Graphs. Divide and Conquer.

Jana Kosecka. Red-Black Trees Graph Algorithms. Many slides here are based on E. Demaine, D. Luebke slides

Topic 12: Elementary Graph Algorithms

Graph Algorithms. Andreas Klappenecker. [based on slides by Prof. Welch]

22.1 Representations of graphs

Elementary Graph Algorithms

Sample Solutions to Homework #4

CISC 320 Introduction to Algorithms Fall Lecture 15 Depth First Search

Suggested Study Strategy

Algorithm Design and Analysis

2 A Template for Minimum Spanning Tree Algorithms

Practical Session No. 12 Graphs, BFS, DFS Topological Sort

Announcements. HW3 is graded. Average is 81%

Minimum Spanning Tree

Transcription:

Lecture 10. Elementary Graph Algorithm Minimum Spanning Trees T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to Algorithms, 3rd Edition, MIT Press, 2009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright 2000-2018 Networking Laboratory

Graphs Graph G = (V, E) V is set of vertices E is set (family) of edges (V V) edge or link or arc Types of graphs Undirected: edge (u, v) = (v, u) for all v, (v, v) E (No self loops) Directed: (u, v) is edge from u to v, denoted as u v self loops are allowed Weighted: each edge has an associated weight, given by a weight function w : E R Dense: E V 2 Sparse: E << V 2 E = O( V 2 ) Algorithms Networking Laboratory 2/122

Graphs If (u, v) E, then vertex v is adjacent to vertex u Adjacency relationship is: Symmetric if G is undirected Not necessarily so if G is directed If G is connected: There is a path between every pair of vertices E V 1 Furthermore, if E = V 1, then G is a tree Other definitions in Appendix B (B.4 and B.5) as needed On pages 1080-1093 Algorithms Networking Laboratory 3/122

Eulerian and Hamiltonian Graphs A graph containing paths that include every edge exactly once and end at the initial vertex is called an Eulerian graph A graph containing paths that include every vertex exactly once and end at the initial vertex is called a Hamiltonian graph P Q R Hamiltonian but not Eulerian T S Algorithms Networking Laboratory 4/122

Trees and Planar Graphs A connected graph with only one path between each pair of vertices is called a tree A tree can also be defined as a connected graph containing no cycles (figure 1) A graph that can be redrawn without crossings is called a planar graph (figures 2 and 3) P Q P Q R R T S T S Fig. 1 Fig. 2 Fig. 3 Algorithms Networking Laboratory 5/122

Other Definitions Two digraphs are isomorphic if there is an isomorphism between their underlying graphs that preserves the ordering of the vertices in each arc See figures 4 and 5 : not isomorphic See figures 6 and 7 : isomorphic Fig. 4 Fig. 6 Fig. 5 Fig. 7 Algorithms Networking Laboratory 6/122

Practice Problems Which of the following pairs are isomorphic graphs: Algorithms Networking Laboratory 7/122

Connected and Strongly Connected A digraph D is connected if it cannot be expressed as the union of two digraphs This is equivalent to saying that the underlying graph of D is a connected graph D is strongly connected if, for any two vertices v and w of D, there is a path from v to w Every strongly connected digraph is connected, but not all connected digraphs are strongly connected See figure 4 Fig. 4 Algorithms Networking Laboratory 8/122

Connected and Strongly Connected We define a graph G to be orientable if each edge of G can be directed so that the resulting digraph is strongly connected See figures 8 and 9 Any Eulerian graph is orientable Fig. 8 Fig. 9 Algorithms Networking Laboratory 9/122

Adjacency Two vertices v and w are adjacent if there is an edge (v,w) joining them, and the vertices v and w are then incident with such an edge Two distinct edges e and f are adjacent if they have a vertex in common v w e f adjacent vertices adjacent edges Algorithms Networking Laboratory 10/122

Representation of Graphs Two standard ways Adjacency lists a b a b d c b a c c d c d d a a c b Adjacency matrix 1 2 a c b d 3 4 1 2 3 4 1 0 1 1 1 2 1 0 1 0 3 1 1 0 1 4 1 0 1 0 Algorithms Networking Laboratory 11/122

Representation of Graphs Adjacency matrix A Incidence matrix M Algorithms Networking Laboratory 12/122

Algorithms Networking Laboratory 13/122 Adjacency Lists Consists of an array Adj of V lists One list per vertex For u V, Adj[u] consists of all vertices adjacent to u a d c b a b c d b c d d c a d c b a b c d b a d d c c a b a c If weighted, store weights also in adjacency lists

Degree The degree of a vertex v is the number of edges incident with v, and is written deg(v) A loop at v contributes 2 to the degree of v A vertex of degree 0 is an isolated vertex A vertex of degree 1 is an end-vertex Remember the handshaking lemma and its corollary u z v w i Algorithms Networking Laboratory 14/122

Handshaking Lemma Handshaking Lemma If several people shake hands, then the total number of hands shaken must be even In any graph the sum of all the vertex degrees is an even number Corollary in fact, twice the number of edges In any graph the number of vertices of odd degree is even Algorithms Networking Laboratory 15/122

Handshaking Dilemma The out-degree of a vertex v of G is the number of arcs of the form (v,w), and is denoted by outdeg(v) The in-degree of a vertex v of G is the number of arcs of the form (w,v), and is denoted by indeg(v) The sum of the out-degrees of all the vertices of G is equal to the sum of their in-degrees We call this result the handshaking dilemma A source of G is a vertex with in-degree 0 A sink of G is a vertex with out-degree 0 Algorithms Networking Laboratory 16/122

Storage Requirement For directed graphs Sum of lengths of all adj. lists is outdeg(v) = E v V Total storage: (V+E) For undirected graphs Sum of lengths of all adj. lists is deg(v) = 2 E v V Total storage: (V+E) Algorithms Networking Laboratory 17/122

Pros and Cons: Adj List Pros Space-efficient, when a graph is sparse Can be modified to support many graph variants Cons Determining if an edge (u,v) E is not efficient Have to search in u s adjacency list, (deg(u)) time (V) in the worst case Algorithms Networking Laboratory 18/122

Adjacency Matrix V V matrix A Number vertices from 1 to V in some arbitrary manner A is then given by: A[ i, j] a ij 1 0 if ( i, j) E otherwise 1 2 a c b d 3 4 1 2 3 4 1 0 1 1 1 2 0 0 1 0 3 0 0 0 1 4 0 0 0 0 1 2 a c b d 3 4 1 2 3 4 1 0 1 1 1 2 1 0 1 0 3 1 1 0 1 4 1 0 1 0 A = A T for undirected graphs Algorithms Networking Laboratory 19/122

Space and Time Space: (V 2 ) Not memory efficient for large graphs Time: to list all vertices adjacent to u (V) Time: to determine if (u, v) E (1) Can store weights instead of bits for weighted graph Algorithms Networking Laboratory 20/122

Practice Problems Draw the graph whose adjacency matrix is given as follows: Draw the graph whose incidence matrix is given as follows: Algorithms Networking Laboratory 21/122

Graph-Searching Algorithms Searching a graph Systematically follow the edges of a graph to visit the vertices of the graph Used to discover the structure of a graph Standard graph-searching algorithms Breadth-First Search (BFS) Depth-First Search (DFS) Algorithms Networking Laboratory 22/122

Breadth-First Search Input Graph G = (V, E), either directed or undirected, and source vertex s V Output d[v] = distance (smallest number of edges, or shortest path) from s to v, for all v V d[v] = if v is not reachable from s [v] = u such that (u, v) is last edge on shortest path s u is v s predecessor Builds Breadth-First Tree with root s that contains all reachable vertices v Algorithms Networking Laboratory 23/122

Breadth-First Search Expands the frontier between discovered and undiscovered vertices uniformly across the breadth of the frontier A vertex is discovered the first time it is encountered during the search A vertex is finished if all vertices adjacent to it have been discovered Colors the vertices to keep track of progress White Undiscovered Gray Discovered but not finished Black Finished Colors are required only to reason about the algorithm. Can be implemented without colors. Algorithms Networking Laboratory 24/122

Breadth-First Search 2 3 3 1 1 S 1 2 2 2 S 2 2 3 S 3 3 Finished Discovered Undiscovered Algorithms Networking Laboratory 25/122

Pseudo Code BFS(G,s) 1. for each vertex u in V[G] {s} 2 do color[u] white 3 d[u] 4 [u] nil white: undiscovered gray: discovered black: finished 5 color[s] gray 6 d[s] 0 Q: a queue of discovered vertices 7 [s] nil color[v]: color of v 8 Q d[v]: distance from s to v 9 enqueue(q,s) [u]: predecessor of v 10 while Q 11 do u dequeue(q) 12 for each v in Adj[u] 13 do if color[v] = white 14 then color[v] gray 15 d[v] d[u] + 1 16 [v] u 17 enqueue(q,v) 18 color[u] black Algorithms Networking Laboratory 26/122

BFS Example r s t u 0 v w x y Q: s 0 Algorithms Networking Laboratory 27/122

BFS Example r s t u 1 0 1 v w x y Q: w r 1 1 Algorithms Networking Laboratory 28/122

BFS Example r s t u 1 0 2 1 2 v w x y Q: r t x 1 2 2 Algorithms Networking Laboratory 29/122

BFS Example r s t u 1 0 2 2 1 2 v w x y Q: t x v 2 2 2 Algorithms Networking Laboratory 30/122

BFS Example r s t u 1 0 2 3 2 1 2 v w x y Q: x v u 2 2 3 Algorithms Networking Laboratory 31/122

BFS Example r s t u 1 0 2 3 2 1 2 3 v w x y Q: v u y 2 3 3 Algorithms Networking Laboratory 32/122

BFS Example r s t u 1 0 2 3 2 1 2 3 v w x y Q: u y 3 3 Algorithms Networking Laboratory 33/122

BFS Example r s t u 1 0 2 3 2 1 2 3 v w x y Q: y 3 Algorithms Networking Laboratory 34/122

BFS Example r s t u 1 0 2 3 2 1 2 3 v w x y Q: Algorithms Networking Laboratory 35/122

BFS Example r s t u 1 0 2 3 2 1 2 3 v w x y BF Tree Algorithms Networking Laboratory 36/122

Analysis of BFS Initialization takes O(V) Traversal Loop After initialization, each vertex is enqueued and dequeued at most once, and each operation takes O(1). So, total time for queuing is O(V) The adjacency list of each vertex is scanned at most once. The sum of lengths of all adjacency lists is (E) Summing up over all vertices total running time of BFS is O(V+E), linear in the size of the adjacency list representation of graph Algorithms Networking Laboratory 37/122

Breadth-First Trees For a graph G = (V, E) with source s, the predecessor subgraph of G is G = (V, E ) where V ={v V : [v] NIL} {s} E ={ ( [v],v) E : v V - {s} } The predecessor subgraph G is a breadth-first tree if: V consists of the vertices reachable from s and for all v V, there is a unique simple path from s to v in G that is also a shortest path from s to v in G. The edges in E are called tree edges E = V - 1 Algorithms Networking Laboratory 38/122

Depth-First Search Explore edges out of the most recently discovered vertex v When all edges of v have been explored, backtrack to explore other edges leaving the vertex from which v was discovered (its predecessor) Search as deep as possible first Continue until all vertices reachable from the original source are discovered If any undiscovered vertices remain, then one of them is chosen as a new source and search is repeated from that source Algorithms Networking Laboratory 39/122

Depth-First Search Input G = (V, E), directed or undirected. No source vertex given! Output 2 timestamps on each vertex. Integers between 1 and 2 V. d[v] = discovery time (v turns from white to gray) f [v] = finishing time (v turns from gray to black) [v] : predecessor of v = u, such that v was discovered during the scan of u s adjacency list. Uses the same coloring scheme for vertices as BFS White Undiscovered Gray Discovered but not finished Black Finished Algorithms Networking Laboratory 40/122

Pseudo Code DFS(G) 1. for each vertex u V[G] 2. do color[u] white 3. [u] NIL 4. time 0 5. for each vertex u V[G] 6. do if color[u] = white 7. then DFS-Visit(u) Uses a global timestamp time DFS-Visit(u) 1. color[u] GRAY White vertex u has been discovered 2. time time + 1 3. d[u] time 4. for each v Adj[u] 5. doif color[v] = WHITE 6. then [v] u 7. DFS-Visit(v) 8. color[u] BLACK Blacken u; it is finished. 9. f[u] time time + 1 white: undiscovered gray: discovered black: finished Algorithms Networking Laboratory 41/122

DFS Example 1/ u v w x y z Algorithms Networking Laboratory 42/122

DFS Example u v w 1/ 2/ x y z Algorithms Networking Laboratory 43/122

DFS Example u v w 1/ 2/ 3/ x y z Algorithms Networking Laboratory 44/122

DFS Example 1/ u v w 2/ 4/ 3/ x y z Algorithms Networking Laboratory 45/122

DFS Example u v w 1/ 2/ B 4/ 3/ x y z Algorithms Networking Laboratory 46/122

DFS Example u v w 1/ 2/ B 4/5 3/ x y z Algorithms Networking Laboratory 47/122

DFS Example u v w 1/ 2/ B 4/5 3/6 x y z Algorithms Networking Laboratory 48/122

DFS Example u v w 1/ 2/7 B 4/5 3/6 x y z Algorithms Networking Laboratory 49/122

DFS Example 1/ u v w 2/7 F B 4/5 3/6 x y z Algorithms Networking Laboratory 50/122

DFS Example u v w 1/8 2/7 F B 4/5 3/6 x y z Algorithms Networking Laboratory 51/122

DFS Example u v w 1/8 2/7 9/ F B 4/5 3/6 x y z Algorithms Networking Laboratory 52/122

DFS Example u v w 1/8 2/7 9/ F B C 4/5 3/6 x y z Algorithms Networking Laboratory 53/122

DFS Example u v w 1/8 2/7 9/ F B C 4/5 3/6 10/ x y z Algorithms Networking Laboratory 54/122

DFS Example u v w 1/8 2/7 9/ F B C 4/5 3/6 10/ x y z B Algorithms Networking Laboratory 55/122

DFS Example u v w 1/8 2/7 9/ F B C 4/5 3/6 10/11 x y z B Algorithms Networking Laboratory 56/122

DFS Example u v w 1/8 2/7 9/12 F B C 4/5 3/6 10/11 x y z B Algorithms Networking Laboratory 57/122

Analysis of DFS Loops on lines 1-2 & 5-7 take (V) time, excluding time to execute DFS-Visit DFS-Visit is called once for each white vertex v V when it s painted gray the first time. Lines 3-6 of DFS-Visit is executed Adj[v] times. The total cost of executing DFS-Visit is v V Adj[v] = (E) Total running time of DFS is (V+E) Algorithms Networking Laboratory 58/122

Depth-First Trees Predecessor subgraph defined slightly different from that of BFS The predecessor subgraph of DFS is G = (V, E ) where E ={( [v],v) : v V and [v] NIL}. How does it differ from that of BFS? The predecessor subgraph G forms a depth-first forest composed of several depth-first trees. The edges in E are called tree edges. Algorithms Networking Laboratory 59/122

Classification of Edges Tree edge: in the depth-first forest. Found by exploring (u, v) Back edge: (u, v), where u is a descendant of v (in the depth-first tree) Forward edge: (u, v), where v is a descendant of u, but not a tree edge Cross edge: any other edge Can go between vertices in same depth-first tree or in different depthfirst trees Algorithms Networking Laboratory 60/122

Practice Problems Perform a depth-first search on the following graph starting at A. Break all ties by picking the vertices in alphabetical order (i.e A before Z). Algorithms Networking Laboratory 61/122

Topological Sort A topological sort of a Directed Acyclic Graph (DAG) is a linear order of all its vertices s.t. if G contains an edg e (u, v), then u appears before v in the ordering If the graph is not acyclic, then no linear ordering is possible. A topological sort can be viewed as an ordering of its vertices along a horizontal line so that all directed edges go from left to right (V+E) Algorithms Networking Laboratory 62/122

Topological Sort Algorithms Networking Laboratory 63/122

Minimum Spanning Tree A town has a set of houses and a set of roads A road connects 2 and only 2 houses A road connecting houses u and v has a repair cost w(u, v) Goal: Repair enough (and no more) roads such that everyone stays connected: can reach every house from all other houses, and total repair cost is minimum Algorithms Networking Laboratory 64/122

Minimum Spanning Tree Model as a graph: Undirected graph G = (V, E) Weight w(u, v) on each edge (u, v) E Find T E such that T connects all vertices (T is a spanning tree) w(t ) = ( u, v) T w( u, v) is minimized Algorithms Networking Laboratory 65/122

Minimum Spanning Tree A spanning tree whose weight is minimum over all spanning trees is called a Minimum Spanning Tree, or MST Example: In this example, there is more than one MST Replace edge (b,c) by (a,h) Get a different spanning tree with the same weight Algorithms Networking Laboratory 66/122

Minimum Spanning Tree Which edges form the Minimum Spanning Tree (MST) of the below graph? 14 A 6 4 5 9 H B C 10 15 G E D 3 F 8 2 Algorithms Networking Laboratory 67/122

Minimum Spanning Tree MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees Let T be an MST of G with an edge (u,v) in the middle Removing (u,v) partitions T into two trees T 1 and T 2 Claim: T 1 is an MST of G 1 = (V 1,E 1 ), and T 2 is an MST of G 2 = (V 2,E 2 ) ( Do V 1 and V 2 share vertices? Why? ) Proof: w(t) = w(u,v) + w(t 1 ) + w(t 2 ) (There can t be a better tree than T 1 or T 2, or T would be suboptimal) Algorithms Networking Laboratory 68/122

Growing An MST Some properties of an MST: It has V -1 edges It has no cycles It might not be unique Building up the solution We will build a set A of edges Initially, A has no edges As we add edges to A, maintain a loop invariant: Loop invariant: A is a subset of some MST Add only edges that maintain the invariant If A is a subset of some MST, an edge (u, v) is safe for A if and only if A {(u, v)} is also a subset of some MST So we will add only safe edges Algorithms Networking Laboratory 69/122

Growing An MST Generic MST algorithm Use the loop invariant to show that this generic algorithm works Initialization The empty set trivially satisfies the loop invariant Maintenance Since we add only safe edges, A remains a subset of some MST Termination All edges added to A are in an MST, so when we stop, A is a spanning tree that is also an MST Algorithms Networking Laboratory 70/122

Growing An MST Finding a safe edge Intuitiveness: Let S V be any set of vertices that includes h but not g (so that g is in V - S). In any MST, there has to be one edge (at least) that connects S with V - S. Why not choose the edge with minimum weight? Some definitions: Let S V and A E A cut (S, V - S) is a partition of vertices into disjoint sets S and V - S Edge (u, v) E crosses cut (S, V - S) if one endpoint is in S and the other is in V - S A cut respects A if and only if no edge in A crosses the cut An edge is a light edge crossing a cut if and only if its weight is minimum over all edges crossing the cut Algorithms Networking Laboratory 71/122

Growing An MST cut, crosses, respects, light edge Algorithms Networking Laboratory 72/122

Growing An MST Theorem Let T be MST of G, and let A T be subtree of T Let (u,v) be min-weight edge connecting A to V-A Then (u,v) T Algorithms Networking Laboratory 73/122

Disjoint-Set Union Problem Want a data structure to support disjoint sets Collection of disjoint sets S = {S i }, S i S j = Need to support following operations: MakeSet(x): S = S U {{x}} Union(S i, S j ): S = S - {S i, S j } U {S i U S j } FindSet(x): return S i S such that x S i Before discussing implementation details, we look at example application: MSTs Algorithms Networking Laboratory 74/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 75/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 76/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1? Algorithms Networking Laboratory 77/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 78/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2? 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 79/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 80/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5? 21 13 9 1 Algorithms Networking Laboratory 81/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 82/122

Kruskal s Algorithm Kruskal() { T = ; 8? for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 83/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 84/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9? 1 Algorithms Networking Laboratory 85/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 86/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13? 9 1 Algorithms Networking Laboratory 87/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 88/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14? 17 25 5 21 13 9 1 Algorithms Networking Laboratory 89/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 90/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17? 25 5 21 13 9 1 Algorithms Networking Laboratory 91/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19? sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 92/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 14 17 25 5 21? 13 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 1 Algorithms Networking Laboratory 93/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25? 5 21 13 9 1 Algorithms Networking Laboratory 94/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 14 17 25 5 21 13 9 1 Algorithms Networking Laboratory 95/122

Kruskal s Algorithm Kruskal() { T = ; 8 for each v V MakeSet(v); Run the algorithm: 2 19 14 17 25 5 21 13 sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 9 1 Algorithms Networking Laboratory 96/122

Kruskal s Algorithm Algorithms Networking Laboratory 97/122

Kruskal s Algorithm Algorithms Networking Laboratory 98/122

Kruskal s Algorithm Algorithms Networking Laboratory 99/122

Kruskal s Algorithm Kruskal() { } T = ; for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T U {{u,v}}; What will affect the running time? 1 Sort O(V) MakeSet() calls O(E) FindSet() calls O(V) Union() calls (Exactly how many Union()s?) Union(FindSet(u), FindSet(v)); Algorithms Networking Laboratory 100/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; 14 key[v] = w(u,v); 6 4 5 3 10 8 2 9 15 Run on example graph Algorithms Networking Laboratory 101/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 14 10 2 3 8 9 15 Run on example graph Algorithms Networking Laboratory 102/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) r u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 14 10 2 0 3 8 Pick a start vertex r 9 15 Algorithms Networking Laboratory 103/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 14 10 2 0 3 8 9 15 Red vertices have been removed from Q Algorithms Networking Laboratory 104/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 14 10 2 0 3 3 8 9 15 Blue arrows indicate parent pointers Algorithms Networking Laboratory 105/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 14 14 10 2 0 3 3 8 9 15 Algorithms Networking Laboratory 106/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 14 14 10 2 0 3 u 3 8 9 15 Algorithms Networking Laboratory 107/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 14 14 10 2 0 8 3 u 3 8 9 15 Algorithms Networking Laboratory 108/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 10 14 10 2 0 8 3 u 3 8 9 15 Algorithms Networking Laboratory 109/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 10 14 10 2 0 8 3 3 8 u 9 15 Algorithms Networking Laboratory 110/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 10 2 14 10 2 0 8 3 3 8 u 9 15 Algorithms Networking Laboratory 111/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 10 2 14 10 2 0 8 15 3 3 8 u 9 15 Algorithms Networking Laboratory 112/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 10 2 14 10 2 0 8 15 3 3 8 u 9 15 Algorithms Networking Laboratory 113/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 6 4 5 10 2 9 14 10 2 0 8 15 3 3 8 u 9 15 Algorithms Networking Laboratory 114/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 5 10 2 9 14 10 2 0 8 15 3 3 8 u 9 15 Algorithms Networking Laboratory 115/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 5 5 2 9 14 10 2 0 8 15 3 3 8 u 9 15 Algorithms Networking Laboratory 116/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 5 5 2 9 14 10 2 0 8 15 3 3 8 u 9 15 Algorithms Networking Laboratory 117/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] u if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 5 5 2 9 14 10 2 0 8 15 3 3 8 9 15 Algorithms Networking Laboratory 118/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 5 5 2 9 14 10 2 0 8 15 3 3 8 9 15 u Algorithms Networking Laboratory 119/122

Prim s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 4 6 4 5 5 2 9 14 10 2 0 8 15 3 3 8 9 15 u Algorithms Networking Laboratory 120/122

Prim s Algorithm Algorithms Networking Laboratory 121/122

Prim s Algorithm Algorithms Networking Laboratory 122/122