Introduction to Algorithms

Transcription

1 Introduction to Algorithms, Lecture 1 /1/200 Introduction to Algorithms.04J/1.401J LECTURE 11 Graphs, MST, Greedy, Prim Graph representation Minimum spanning trees Greedy algorithms hallmarks. Greedy choice property Prim s algorithm Prof. Manolis Kellis /1/200 L1.1 Graphs (review) Definition. A directed graph (digraph) G = (V, E) is an ordered pair consisting of a set V of vertices (singular: vertex), a set E V V of edges. In an undirected graph G = (V, E), the edge set E consists of unordered pairs of vertices. In either case, we have E = O(V 2 ). Moreover, if G is connected, then E V 1, which implies that lg E = Θ(lg V). (Review CLRS, Appendix B.) /1/200 L1.2 Adjacency-matrix representation The adjacency matrix of a graph G = (V, E), where V = {1, 2,, n}, is the matrix A[1.. n, 1.. n] given by 1 if (i, j) E, A[i, j] = 0 if (i, j) E A Θ(V 2 ) storage dense representation. /1/200 L1. Adjacency-list representation An adjacency list of a vertex v V is the list Adj[v] of vertices adjacent to v. Adj[1] = {2, } Adj[2] = {} Adj[] = {} 44 Adj[4] = {} For undirected graphs, Adj[v] = degree(v). For digraphs, Adj[v] = out-degree(v). Handshaking Lemma: v V degree(v) = 2 E for undirected graphs adjacency lists use Θ(V + E) storage a sparse representation (exercise B.4-1). /1/200 L1.4 Minimum spanning trees Minimum spanning trees Our first problem on graphs Input: A connected, undirected graph G = (V, E) with weight function w : E R. For simplicity, assume that all edge weights are distinct. (CLRS covers the general case.) Output: A spanning tree T a tree that connects all vertices of minimum weight: w( T ) = w( u, v). ( u, v) T /1/200 L1. /1/200 L1. Leiserson, Demaine, Kellis 1

2 Introduction to Algorithms, Lecture 1 /1/200 Example of MST Example of MST /1/200 L1. The Challenge: There are exponentially many paths through the graph. Finding the minimum weight using /1/200 exhaustive search (brute force): Θ(2 V ) L1. Finding an efficient algorithm Two key algorithm design techniques: * Greedy algorithms * Dynamic Programming Hallmarks of these techniques /1/200 L1. First understand nature of the problem. It shows key properties allow us to do better: Overlapping subproblems: exponentially many paths, due to heavy reuse of small number of edges. Optimal substructure: can construct global optimal solution by combining optimal solns to subproblems. Greedy choice property: moreover, can achieve glob optimal solution by series of locally optimal choices. Prove that these properties hold. Then give an algorithm that exploits them to efficiently solve the MST problem. /1/200 L1. Hallmarks of optimization problems: Greedy vs. Dyn Prog An optimal solution to a problem (instance) contains optimal solutions to subproblems. MST T: A recursive solution contains a small number of distinct subproblems repeated many times.. Greedy choice property Locally optimal choices lead to globally optimal solution Greedy algorithsm 1&2 only: Dynamic Programming /1/200 L1.11 An optimal solution to a problem (instance) contains optimal solutions to subproblems. /1/200 L1. Leiserson, Demaine, Kellis 2

3 Introduction to Algorithms, Lecture 1 /1/200 MST T: u Remove any edge (u, v) T. v MST T: u Remove any edge (u, v) T. v /1/200 L1.1 /1/200 L1. MST T: u T 2 T 1 v Remove any edge (u, v) T. Then, T is partitioned into two subtrees T 1 and T 2. Theorem. The subtree T 1 is an MST of G 1 = (V 1, E 1 ), the subgraph of G induced by the vertices of T 1 : V 1 = vertices of T 1, E 1 = { (x, y) E : x, y V 1 }. Similarly for T 2. /1/200 L1. proof Proof technique: Cut and paste argument. By contradiction: Show that if a better solution existed for a subpart of the optimal solution, then we could get a better optimal solution contradiction. Theorem. Subtree T1 is an MST of G1 = (V1, E1). Proof: If not MST, some T1 exists w/ lower weight w(t) = w(u, v) + w(t 1 ) + w(t 2 ). If T 1 were a lower-weight spanning tree than T 1 for G 1, then T = {(u, v)} T 1 T 2 would be a lower-weight spanning tree than T for G. /1/200 L1.1 Small space of subproblems, highly overlapping Bottom-up approach can exploit that. Greedy choice property. Greedy choice property (yes: Greedy, no: DP) Locally optimal choices lead to globally optimal solution A recursive solution contains a small number of distinct subproblems repeated many times. /1/200 L1.1 Theorem. Let T be the MST of G = (V, E), and let A V. Suppose that (u, v) E is the least-weight edge connecting A to V A. Then, (u, v) T. /1/200 L1.1 Leiserson, Demaine, Kellis

4 Introduction to Algorithms, Lecture 1 /1/200. Greedy choice property Proof. Suppose (u, v) T. Cut and paste. T : u u (u, v) = least-weight edge connecting A to V A Consider the unique simple path from u to v in T. Let (u,v ) be the first edge on this path that connects a vertex in A to a vertex in V A. By construction, w(u,v )>w(u,v), since (u,v) is the least-weight edge. Thus, replacing (u,v ) with (u, v) in T leads to T, also a spanning tree with lower weight. Hence contradiction, as T is a MST. v v Solve MST problem using greedy algorithm Several algorithms are possible, depending on the greedy choice made /1/200 Note on confusing notation: T contains (u,v ) and T contains (u,v). L1.1 /1/200 L1.20 Example of MST /1/200 L1.21 Prim s algorithm IDEA: Maintain V A as a priority queue Q. Key each vertex in Q with the weight of the leastweight edge connecting it to a vertex in A. Q V key[v] for all v V key[s] 0 for some arbitrary s V while Q do u EXTRACT-MIN(Q) for each v dj[u] do if v Q and w(u, v) < key[v] then key[v] w(u, v) DECREASE-KEY π[v] u At the end, {(v, π[v])} forms the MST. /1/200 L /1/200 L1.2 /1/200 L1.24 Leiserson, Demaine, Kellis 4

5 Introduction to Algorithms, Lecture 1 /1/ /1/200 L1.2 /1/200 L /1/200 L1.2 /1/200 L /1/200 L1.2 /1/200 L1.0 Leiserson, Demaine, Kellis

6 Introduction to Algorithms, Lecture 1 /1/ /1/200 L1.1 /1/200 L /1/200 L1. /1/200 L V times Analysis of Prim Θ(V) total degree(u) times Q V key[v] for all v V key[s] 0 for some arbitrary s V while Q do u EXTRACT-MIN(Q) for each v dj[u] do if v Q and w(u, v) < key[v] then key[v] w(u, v) π[v] u Handshaking Lemma Θ(E) implicit DECREASE-KEY s. Time = Θ(V) T EXTRACT-MIN + Θ(E) T DECREASE-KEY /1/200 L1. /1/200 L1. Leiserson, Demaine, Kellis

7 Introduction to Algorithms, Lecture 1 /1/200 Analysis of Prim (continued) Kruskal s Algorithm Time = Θ(V) T EXTRACT-MIN + Θ(E) T DECREASE-KEY Q T EXTRACT-MIN T DECREASE-KEY Total array O(V) O(1) O(V 2 ) binary heap O(lg V) O(lg V) O(E lg V) Fibonacci heap O(lg V) amortized O(1) amortized O(E + V lg V) worst case /1/200 L1. Greedy choice: extend by lowest-weight edge. Forest. MAKE-SET, FIND-SET, UNION. Use DISJOINT-SET data structure O(E lg V). /1/200 L1. MST algorithms Summary Prim s algorithm: Single tree extended. Least-weight edge A:V-A. Running time = O(E + V lg V). Kruskal s algorithm: Forests merged. Least-weight non-cycle edge. Running time = O(E lg V). Best to date: Karger, Klein, and Tarjan [1]. Randomized algorithm. O(V + E) expected time. /1/200 L1. Graphs, MST, Greedy, Prim Graph representation Minimum spanning trees Greedy algorithms hallmarks Optimal substructure Overlapping subproblems Greedy choice property Prim s algorithm / Kruskal s algorithm /1/200 L1.40 Leiserson, Demaine, Kellis