CS 491 CAP Intro to Competitive Algorithmic Programming Lecture 6 Basic Graph Algorithms Uttam Thakore University of Illinois at Urbana-Champaign September 30, 2015
Updates ICPC Regionals teams will be announced by early next week If you have a team preference, please let me know by e-mail by the weekend CS 491 CAP Intro to Competitive Algorithmic Programming 2
Outline Minimum Spanning Tree Kruskal s Algorithm Disjoint Set Data Structure Prim s Algorithm Shortest Path Dijkstra s Algorithm Bellman-Ford Algorithm Floyd-Warshall Algorithm Strongly Connected Components Tarjan s Algorithm CS 491 CAP Intro to Competitive Algorithmic Programming 3
Minimum Spanning Tree CS 491 CAP Intro to Competitive Algorithmic Programming 4
Problem Definition Given a weighted, undirected graph: G = V, E Find a subset of E that connects all vertices in V into a tree (i.e., no cycles) with the minimum total edge weight Figure taken from Wikipedia CS 491 CAP Intro to Competitive Algorithmic Programming 5
Definitions: Cut: Any partition of a graph G into two sets S and T Notation: C = (S, T) Cut-set: All edges that have one endpoint in S and the other in T Also the edges that must be deleted to partition the graph S T S = {A, B, D, E} T = {C, F} Cut-set in red G Figure taken from Wikipedia CS 491 CAP Intro to Competitive Algorithmic Programming 6
Cut Property of MSTs For any cut C of a graph, if e is the unique minimum weight edge in the cut-set of C, then e belongs to all MSTs of the graph Proof: Consider a MST T Assume by contradiction that e is not in T There must exist another edge e T from S to V S, which has weight at least as large as e Adding e and removing e results in a spanning tree T that is at least as good as T Since T is an MST, T must also be an MST So e is part of some MST! CS 491 CAP Intro to Competitive Algorithmic Programming 7
Minimum-cost Edge Property of MSTs If there is a unique minimum cost edge e of a graph, it must belong to all MSTs of the graph Proof: Assume e was not included in the MST Then adding e would create a cycle Removing any of the edges e in the cycle formed would yield an MST of smaller weight Thus, e must be in the MST! G Figure taken from Wikipedia CS 491 CAP Intro to Competitive Algorithmic Programming 8
Kruskal s Algorithm Cut property implies: The edge with the smallest weight is in an MST Each edge with subsequently larger weight is also part of the MST, provided that it doesn t create a cycle This is Kruskal s algorithm! CS 491 CAP Intro to Competitive Algorithmic Programming 9
Kruskal s Algorithm Pseudocode: Sort the edges in increasing order of weight Iterate through each edge e in order, until size of the MST = V 1 If e connects two different connected components, then add e to the MST and merge the two connected components (using disjoint set data structure) Otherwise, ignore e and move on Runtime complexity: O( E log V ) (because of sorting) CS 491 CAP Intro to Competitive Algorithmic Programming 10
Kruskal s Algorithm: Illustration Figure taken from Wikipedia CS 491 CAP Intro to Competitive Algorithmic Programming 11
Kruskal s Algorithm: Illustration Figure taken from Wikipedia CS 491 CAP Intro to Competitive Algorithmic Programming 12
Kruskal s Algorithm: Illustration Figure taken from Wikipedia CS 491 CAP Intro to Competitive Algorithmic Programming 13
Kruskal s Algorithm: Illustration Figure taken from Wikipedia CS 491 CAP Intro to Competitive Algorithmic Programming 14
Kruskal s Algorithm: Illustration Figure taken from Wikipedia CS 491 CAP Intro to Competitive Algorithmic Programming 15
Kruskal s Algorithm: Illustration Figure taken from Wikipedia CS 491 CAP Intro to Competitive Algorithmic Programming 16
Disjoint Set Data Structure Also called Union-Find data structure Given a set of items, want a data structure that supports three operations: Create a singleton set from an item Merge two sets into a single set (union operation) Given an item, determine which set it belongs to (find operation) In our context, want to be able to merge two MSTs and find whether a new edge connects two MSTs CS 491 CAP Intro to Competitive Algorithmic Programming 17
Disjoint Set Data Structure Each group will form a rooted tree structure Maintain a vector P[v], which stores the parent of v Initially P v 1 to indicate that each node is a root by itself Find(u): Finds the root of the group that u belongs to If P u = 1: return u return Find(P[u]) Union(u, v): (merge two groups that u and v belong to) u Find u, v Find(v) If u = v : return (two items belong to the same group, no merge needed) Otherwise, set P[u ] v (basically attach one tree to the other tree) CS 491 CAP Intro to Competitive Algorithmic Programming 18
Disjoint Set Data Structure Problem? Can be too slow! The Union operations can, in the worst case, create a tree with depth linear in the number of items This in turn can make Find operation very slow Two heuristics to take care of this issue: Union by rank Attach the tree with smaller height to the root of another tree in Union operation Path compression Whenever Find is called, for each item visited along the way to the root, directly attach the item to the root) These two heuristics make the operations constant (practically) CS 491 CAP Intro to Competitive Algorithmic Programming 19
Kruskal & Disjoint Set DS Checking if an edge connects two different connected components can be done with Find operation in ~O(1) Merging two connected components as a single connected component can be done with Union operation in O(1) So the part after sorting is effectively O( E ) CS 491 CAP Intro to Competitive Algorithmic Programming 20
Prim s Algorithm Maintain a set S that starts out with a single node Find the smallest weighted edge e = (u, v) in the cut-set for C = (S, V S) Add e to the MST and add v to S Repeat until S = V Difference from Kruskal s: Instead of growing multiple sets, we grow a single supernode S CS 491 CAP Intro to Competitive Algorithmic Programming 21
Prim s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/09-mst.html CS 491 CAP Intro to Competitive Algorithmic Programming 22
Prim s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/09-mst.html CS 491 CAP Intro to Competitive Algorithmic Programming 23
Prim s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/09-mst.html CS 491 CAP Intro to Competitive Algorithmic Programming 24
Prim s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/09-mst.html CS 491 CAP Intro to Competitive Algorithmic Programming 25
Prim s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/09-mst.html CS 491 CAP Intro to Competitive Algorithmic Programming 26
Prim s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/09-mst.html CS 491 CAP Intro to Competitive Algorithmic Programming 27
Prim s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/09-mst.html CS 491 CAP Intro to Competitive Algorithmic Programming 28
Prim s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/09-mst.html CS 491 CAP Intro to Competitive Algorithmic Programming 29
Prim s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/09-mst.html CS 491 CAP Intro to Competitive Algorithmic Programming 30
Prim s Algorithm Initialize S {s}, D[v] weight(e(s, v)) for every v. If e(s, v) E, D[v] Repeat until S = V: Find v S with smallest D[v] Use linear search, or use priority queue for better performance Add v to S, add D[v] to the total weight of the MST For each edge neighboring edges of v, e(v, w) Update D[w] min(d[w] weight(v, w)) Runtime complexity: O( E log V ) CS 491 CAP Intro to Competitive Algorithmic Programming 31
Shortest Path CS 491 CAP Intro to Competitive Algorithmic Programming 32
Problem Definition Given a (undirected or directed) weighted graph G = V, E, find a path between two given nodes u and v that minimizes the total weight of the path Sometimes the edge weights can be negative Sometimes we want to compute shortest paths from u to all other nodes Sometimes we want to compute all-pair shortest paths CS 491 CAP Intro to Competitive Algorithmic Programming 33
Dijkstra s Algorithm Can solve simplest form: Single-Source Shortest Path (SSSP) with no negative weights Greedy algorithm Works very similarly to Prim s Algorithm CS 491 CAP Intro to Competitive Algorithmic Programming 34
Dijkstra s Algorithm Pseudocode: Maintain a set S that stores nodes for which the shortest path has already been determined Maintain a vector D[v] to store the shortest distance estimate from s Initially, S s, D[v] weight(s, v) If e(s, v) E, D[v] Repeat until S = V Find v S with the smallest D[v], and add it to S Use priority queue for better performance For each edge v u with weight w, D[u] min(d[u], D[v] + w) Note: If you are asked to find the actual sequence of nodes in the path, store that in D[v] in addition to the distance Running time: O( E log V ) CS 491 CAP Intro to Competitive Algorithmic Programming 35
Dijkstra s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/10-graphalgo.html CS 491 CAP Intro to Competitive Algorithmic Programming 36
Dijkstra s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/10-graphalgo.html CS 491 CAP Intro to Competitive Algorithmic Programming 37
Dijkstra s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/10-graphalgo.html CS 491 CAP Intro to Competitive Algorithmic Programming 38
Dijkstra s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/10-graphalgo.html CS 491 CAP Intro to Competitive Algorithmic Programming 39
Dijkstra s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/10-graphalgo.html CS 491 CAP Intro to Competitive Algorithmic Programming 40
Dijkstra s Algorithm: Illustration Figure from https://www.cs.indiana.edu/~achauhan/teaching/ B403/LectureNotes/10-graphalgo.html CS 491 CAP Intro to Competitive Algorithmic Programming 41
Bellman-Ford Algorithm Unlike Dijkstra s, can handle negative edge weights Can detect negative-weight cycles Very easy to code CS 491 CAP Intro to Competitive Algorithmic Programming 42
Bellman-Ford Algorithm Initialize D[s] 0 and D[v] for all v s For k = 1.. V 1: For each edge u v of weight w: D v min(d[v], D[u] + w) For each edge u v of weight w: If D v > D u + w: The graph contains a negative weight cycle Runtime complexity: O( E V ) CS 491 CAP Intro to Competitive Algorithmic Programming 43
Bellman-Ford Algorithm Why does it work? A shortest path can have at most V 1 edges Any longer path must contain a cycle At the kth iteration, all shortest paths using k or less edges are computed After V 1 iterations, all distances must be final That is, for every edge u v of weight w, D v D[u] + w holds Unless there is a negative weight cycle This is how negative weight cycle detection works CS 491 CAP Intro to Competitive Algorithmic Programming 44
Floyd-Warshall Algorithm Computes All-Pairs Shortest Paths (APSP) Like Bellman-Ford, can handle negative edge weights Store answers in table D[i][j] D[i][j] stores the total weight of the shortest path from vertex i to j VERY easy to code: only 4 lines! CS 491 CAP Intro to Competitive Algorithmic Programming 45
Floyd-Warshall Algorithm Pseudocode: Initialize matrix D with weights from the given graph ( if there is no edge) for k = 1.. V: for i = 1.. V: for j = 1.. V: D[i][j] min(d i j, D[i][k] + D[k][j]) Runtime complexity: O( V 3 ) CS 491 CAP Intro to Competitive Algorithmic Programming 46
Floyd-Warshall Algorithm Why does it work? Dynamic Programming D[k][i][j]: weight of shortest path from i to j using vertices numbered k as the intermediate nodes The recurrence relation then is D[k][i][j] = min(d[k 1][i][j], D[k 1][i][k] + D[k 1][k][j]) This holds because we have two possible choices: either use k as the intermediate node, or don t Turns out the first dimension is not necessary, can just overwrite Can be extended to find the actual shortest path sequence: see link CS 491 CAP Intro to Competitive Algorithmic Programming 47
Strongly Connected Components CS 491 CAP Intro to Competitive Algorithmic Programming 48
Problem Definition A directed graph is strongly connected if there is a path between each pair of vertices in the graph Given a directed graph G = V, E, partition the graph into subgraphs (components) that are each maximally strongly connected Maximal: no additional edges or vertices from G can be included in the subgraph without breaking the strongly connected property Figure taken from Wikipedia CS 491 CAP Intro to Competitive Algorithmic Programming 49
Algorithms Commonly used algorithms: Kosaraju s Algorithm Tarjan s Algorithm Both algorithms use DFS to traverse the graph and determine whether a set of nodes forms a strongly connected component CS 491 CAP Intro to Competitive Algorithmic Programming 50
Kosaraju s Algorithm Intuition: If a set of nodes is part of an SCC traversing the graph in one direction, it should be part of the same SCC traversing in the reverse direction Pseudocode: Mark all nodes as unvisited and initialize a stack S For each vertex u of the graph, perform Visit(u) For each element of S (popping fro the stack), perform Assign(u, u) Visit(u): If u is unvisited: Mark u as visited For each edge u v, Visit(v) Push u to the top of stack S Assign(u, root): If u has not been assigned to a component: Assign u to the component root For each edge u v, Assign(v, root) CS 491 CAP Intro to Competitive Algorithmic Programming 51
Kosaraju s Algorithm Runtime complexity: O( V + E ) Uses two DFSes one in the forward direction, and one in the reverse direction Can also be implemented using BFS CS 491 CAP Intro to Competitive Algorithmic Programming 52
Tarjan s Algorithm Can we do this in one DFS? Yes! This is Tarjan s Algorithm See the Competitive Programming book for details CS 491 CAP Intro to Competitive Algorithmic Programming 53
Questions? CS 491 CAP Intro to Competitive Algorithmic Programming 54