Randomized Graph Algorithms

Randomized Graph Algorithms Vasileios-Orestis Papadigenopoulos School of Electrical and Computer Engineering - NTUA papadigenopoulos orestis@yahoocom July 22, 2014 Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 1 / 37

Overview 1 The Min-Cut problem The problem The s-t min-cut problem Min-cut and s-t min-cut problem The Contract algorithm The Contraction algorithm The FastCut algorithm 2 Minimum Spanning Trees The problem Deterministic approaches Bor uvka s algorithm Heavy Edges and MST Verification Random Sampling for MSTs The linear-time MST algorithm Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 2 / 37

The Min-Cut problem Overview 1 The Min-Cut problem The problem The s-t min-cut problem Min-cut and s-t min-cut problem The Contract algorithm The Contraction algorithm The FastCut algorithm 2 Minimum Spanning Trees The problem Deterministic approaches Bor uvka s algorithm Heavy Edges and MST Verification Random Sampling for MSTs The linear-time MST algorithm Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 3 / 37

The Min-Cut problem The problem Min-cut Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 4 / 37

The Min-Cut problem The problem The problem In graph theory, a minimum cut of a graph is a cut (a partition of the vertices of a graph into two disjoint subsets that are joined by at least one edge) whose cut set has the smallest number of edges wikipedia Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 5 / 37

The problem The Min-Cut problem The problem Definition A cut in G is a partition of the vertices V = (C, C) into two non-empty sets Definition Ṭhe value or size of a cut C is the number of edges crossing the cut Definition The min-cut problem is that of finding a cut with minimum size in an input graph G Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 6 / 37

The Min-Cut problem The problem Also Multiple edges Connected graph assumption or else trivially solved with finding connected components Weighted graphs generalization Assuming non-negative edges, or else the problem is NP-complete The max-cut is a special case of it Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 7 / 37

The Min-Cut problem The s-t min-cut problem The s-t min-cut problem Definition The s-t min-cut problem is the min-cut problem with the restriction that two distinguished vertices s and t have the property s C and t C Lemma The value of a maximum s-t flow in a network equals the value of a s-t min-cut Maximum flow duality The s-t min-cut problem can be solved via the maximum flow problem in O(mn log (n 2 /m)) Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 8 / 37

The Min-Cut problem Min-cut and s-t min-cut problem Deterministic approach of min-cut via s-t min-cut Lemma The min-cut problem for a graph G can be solved via polynomial number of invocations of an s-t algorithm applied to the same graph Deterministic complexity It can be shown that n-1 flow computations (s-t min-cut calculations) suffice Since the best deterministic maximum flow algorithm runs in O(mn log (n 2 /m)), this approach will require Ω(mn 2 ) Fortunately, the n-1 maximum flow computations can be implemented in time proportional to the cost of a single maximum flow computation So we can compute a min-cut in time O(mn log (n 2 /m)) Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 9 / 37

The Contract algorithm The Min-Cut problem The Contract algorithm Definition Given an edge (x,y) in a multigraph G(V,E), a contraction of the edge (x,y) corresponds to replacing the vertices x and y by a new vertex z, and for each u / {x, y} replacing the edge (x,u) and (y,u) by the edge (z,u) Any multiple edges created are to be retained The graph obtained by this contraction is denoted by G/(x,y) Definition Given a collection of edges F E, the contracted graph is denoted by G/F The vertex set and the edge set of the graph G/F are denoted by V/F and E/F Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 10 / 37

The Min-Cut problem The Contract algorithm The Contract algorithm Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 11 / 37

The Min-Cut problem The Contract algorithm - Analysis The Contract algorithm Lemma It is possible to keep track of the correspondence between the original graph and the contracted one in O(n) time per contraction Lemma The effect of contracting the edges in F is independent of the order of contraction Lemma The selection of an edge (x,y) uniformly at random from the set of all edges in the graph G can be done in O(n) Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 12 / 37

The Min-Cut problem The Contract algorithm - Analysis The Contract algorithm Theorem The Contract algorithm can be implemented to run in O(n 2 ) time on any n-vertex multigraph G Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 13 / 37

The Min-Cut problem The Contraction algorithm The Contraction algorithm Lemma A cut in C is produced as output by the algorithm Contract iff none of the edges crossing this cut is contracted by the algorithm Let K be the a min-cut in G with size of k Lemma In a n-vertex multigraph G with min-cut value k, no vertex has a degree smaller than k Further, the total number of edges in the graph satisfies m nk/2 Lemma Given an edge (x,y) in a graph G, the min-cut value in G/(x,y) is at least as large as the min-cut value in G Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 14 / 37

The Min-Cut problem The Contraction algorithm The Contraction algorithm The number of vertices in H decreases by one during each iteration of Contract algorithm At the i-th iteration, there are n i = n i + 1 vertices in H Since the number of edges in each step is at least n i k/2 (handshaking lemma), the probability that any edge of K is contracted is at most 2/n i So: Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 15 / 37

The Min-Cut problem The Contraction algorithm The Contraction algorithm Theorem Any specific min-cut K is output by the algorithm Contract with probability Ω(n 2 ) The algorithm The above theorem gives as a Monte Carlo algorithm running in O(n 4 log n), by repeating the algorithm contract O(n 2 log n) times Lemma Suppose the algorithm Contract is terminated when the number of vertices remaining in the contracted graph is exactly t Then any specific min-cut K survives in the resulting contracted graph with probability at least: ( t ) ( 2 / n ) ( 2 = Ω( t ) 2) n Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 16 / 37

The Min-Cut problem The FastCut algorithm The FastCut algorithm At this point, we will modify the implementation of the contraction algorithm to reduce its running time to O(n 2 log n) As the complexity of our general module algorithm contract cannot be less than O(n 2 ), the only way to succeed it is by increasing the probability that a min-cut is produced by the algorithm contract The crucial insight is to use two independent invocations of the contract algorithm itself on the contracted graph with n/ 2 vertices Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 17 / 37

The Min-Cut problem The FastCut algorithm The FastCut algorithm Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 18 / 37

The Min-Cut problem The FastCut algorithm - Analysis The FastCut algorithm Theorem Ạlgorithm FastCut runs in O(n 2 log n) time and O(n 2 ) space Theorem Algorithm FastCut succeeds in finding a min-cut with probability Ω(1/ log n) Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 19 / 37

Minimum Spanning Trees Overview 1 The Min-Cut problem The problem The s-t min-cut problem Min-cut and s-t min-cut problem The Contract algorithm The Contraction algorithm The FastCut algorithm 2 Minimum Spanning Trees The problem Deterministic approaches Bor uvka s algorithm Heavy Edges and MST Verification Random Sampling for MSTs The linear-time MST algorithm Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 20 / 37

Minimum Spanning Trees The problem Minimum Spanning Trees Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 21 / 37

Minimum Spanning Trees The problem The problem Given a connected, undirected graph, a spanning tree of that graph is a subgraph that is a tree and connects all the vertices together A single graph can have many different spanning trees We can also assign a weight to each edge, which is a number representing how unfavourable it is, and use this to assign a weight to a spanning tree by computing the sum of the weights of the edges in that spanning tree A minimum spanning tree (MST) or minimum weight spanning tree is then a spanning tree with weight less than or equal to the weight of every other spanning tree wikipedia Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 22 / 37

Minimum Spanning Trees The problem Also Minimum Spanning Forest Distinct edge weights assumption The exact weight is irrelevant in the unit-cost RAM model Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 23 / 37

Minimum Spanning Trees Deterministic approaches Deterministic approaches Prim s algorithm, greedy with Fibonacci heap O(E + V log V ) Kruskal s algorithm, greedy with union-find O(E log E) Bor uvka s algorithm, O(E log V ) Best-known algorithm runs in O(E log β(e, V )), where β(e, V ) = min {i log (i) V E/V } Unfortunately, the algorithm has a large constant due to its complicated data structures Not applicable in practice Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 24 / 37

Minimum Spanning Trees Bor uvka s algorithm Bor uvka s algorithm At this point, we are going to present the idea behind Bor uvka s algorithm, as it will be useful for the randomized algorithm we are going to create Sequential implementation in O(m log n): Data: A graph G Result: The MST of G initialization; in each phase in time O(m) with two passes of the edges; 1st pass: find the less-weighted edge of each component Consecutive find, the edges of the same component are noted; 2nd pass: include the less-weighted edges in the MST and update the components using union ; Algorithm 1: Bor uvka s algorithm Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 25 / 37

Bor uvka s algorithm Minimum Spanning Trees Bor uvka s algorithm The reasoning of the algorithm is based on the following lemmas: Lemma Let v V be any vertex in G The MST for G must contain the edge (v,w) that is the minimum-weight edge incident on v Lemma Given a graph G with n vertices and m edges, a Bor uvka phase can be implemented in time O(n + m) Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 26 / 37

Bor uvka s algorithm Minimum Spanning Trees Bor uvka s algorithm Lemma The set of edges marked for contraction during a Bor uvka phase induces a forest in G Lemma Let G be the graph obtained from G after a Bor uvka phase The MST of G is the union of the edges marked for contraction with the edges in the MST of G Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 27 / 37

Bor uvka s algorithm Minimum Spanning Trees Bor uvka s algorithm Conclusions: Reduction Bor uvka s algorithm reduces the MST problem in a n-vertex graph with m edges to the MST problem in an (n/2)-vertex graph with at most m edges Bor uvka phase complexity Ịn each Bor uvka phase the time required for the reduction is O(n + m) Bor uvka s running time It follows that the Bor uvka s algorithm worst-case running time is O(m log n) Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 28 / 37

Minimum Spanning Trees Heavy Edges and MST Verification Heavy Edges and MST Verification Now, we are going to touch the subject of MST-Verification problem complexity: The problem Ṭake as input a tree T in a graph G Check if T is an MST of G Definition Fix a forest F in G and consider any pair of vertices u, v V Let w F (u,v) denote the maximum weight of an edge on the (unique) path P(u,v) in F We set w F (u,v)= when u and v are disconnected in G Definition An edge (u, v) E is said to be F-heavy if w(u,v)> w F (u,v) The edge (u,v) is said to be F-light if w(u,v) w F (u,v) Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 29 / 37

Minimum Spanning Trees Heavy Edges and MST Verification Lemma Ạll edges in a forest F must be F-light Heavy Edges and MST Verification Lemma Let F be any forest in G If an edge (u,v) is F-heavy, then it does not lie in the MST for G Existence of MST-verification algorithms There exist linear-time verification algorithms that can also identify all F-heavy and F-light edges with respect to any forest F Theorem Given a graph G and a forest F, all F-heavy edges in G can be identified in time O(n + m) Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 30 / 37

Minimum Spanning Trees Random Sampling for MSTs Random Sampling for MSTs In the MST algorithm we are constructing, the use of randomization is in the use of random sampling to identify and eliminate edges that are guaranteed not to belong to the MST Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 31 / 37

Minimum Spanning Trees Random Sampling for MSTs Random Sampling for MSTs Let G a graph Definition We define as G(p) the random graph obtained by independently including each edge of G in G(p) with probability p Obviously, G(p) is expected to have n vertices and mp edges There is no guarantee, although, that G(p) is connected Let F be the minimum spanning forest for G(p) Then, for reasonably large values of p, F should be a very good approximation to the MST of G We remind the lemma suggesting that F-heavy edges does not lie in the MST for G So we expect very few edges in G to be F-light Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 32 / 37

Minimum Spanning Trees Random Sampling for MSTs Random Sampling for MSTs Lemma Let F be the minimum spanning forest in the random graph G(p) obtained by independently including each edge of G with probability p Then the number of F-light edges in G is stochastically dominated by a random variable X that has a negative binomial distribution with parameters n and p In particular, the expected number of F-light edges in G is at most n/p Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 33 / 37

Minimum Spanning Trees The linear-time MST algorithm The expected linear-time MST algorithm Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 34 / 37

Minimum Spanning Trees The linear-time MST algorithm Properties Correctness In each recursion phase the Bor uvka step includes edges that are in the MST The verification step excludes only F-heavy edges of G that cannot be part of the MST Complexity Recursive relation T (n, m) T (n/8, m/2) + T (n/8, n/4) + c(n + m) leading to expected O(n + m) Worst case analysis Worst runtime is equivalent to the runtime of Bor uvka s algorithm ( Las Vegas algorithm) Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 35 / 37

References References Rajeev Motwani & Prabhakar Raghavan Randomized Algorithms Karger David, (1993) Global Min-cuts in RNC and Other Ramifications of a Simple Mincut Algorithm Karger, David R; Klein, Philip N; Tarjan, Robert E (1995) A randomized linear-time algorithm to find minimum spanning trees Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein Introduction to Algorithms VK Balakrishnan Schaum s Outline of Theory and Problems of Combinatorics Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 36 / 37

References End Vasileios-Orestis Papadigenopoulos (NTUA) Randomized Graph Algorithms July 22, 2014 37 / 37