CS : Data Structures

Size: px

Start display at page:

Download "CS : Data Structures"

Samson Gardner
6 years ago
Views:

1 CS : Data Structures Michael Schatz Dec 7, 2016 Lecture 38: Union-Find

Assignment 10: Due Monday Dec 5 @ 10pm Remember: javac Xlint:all &

2 Assignment 10: Due Monday Dec 10pm Remember: javac Xlint:all & checkstyle *.java & JUnit Solutions should be independently written!

3 Part 1: Dijkstra s Algorithm aka Shortest Path Revisited

Dijkstra s Correctness X Y S U V Assume that Dikjstra s algorithm has correctly found the shortest path to the first N items, including from S to X and U (but not yet Y or V) It now decides the next

4 Dijkstra s Correctness X Y S U V Assume that Dikjstra s algorithm has correctly found the shortest path to the first N items, including from S to X and U (but not yet Y or V) It now decides the next closest node to visit is v, using the edge from u Could there be some other shorter path to v? No: cost(s->x->v) must be greater than or equal to cost(s->u->v) or it would have already revised the cost of v to go through X No: cost(s->x->y) must be greater than or equal to cost(s->u->v) (and therefore cost(s->x->y->v) must be even greater) or it would be visiting node Y next

5 Dijkstra s Pseudocode dijkstra(graph, start): distance = {} for v in graph.vertices: distance[v] = infinity distance[start] = 0 unvisited = graph.vertices unvisited.remove(start) current = start while!unvisted.empty() for e in current.outgoing: v = e.tovertex if v in unvisited: d = distance[current] + e.weight if d < distance[v]: distance[v] = d unvisited.remove(current) current = unvisited.findsmallestdistance() if distance[current] == infinity: break Running time? Explore every edge once, Visit every node once At every node, scan the unvisited nodes to find next smallest distance O( E + V 2 ) Can you do better? Greedy choice In case >1 connected components

6 Dijkstra with Priority Queue dijkstra(graph, start): distance = {} for v in graph.vertices: if v == start distance[v] = 0 else distance[v] = infinity Q.add_with_priority(v, distance[v]) while!q.empty() cur = Q.extract_min() if distance[current] == infinity: break for e in current.outgoing: v = e.tovertex if v in Q: d = distance[current] + e.weight if d < distance[v]: distance[v] = d Q.decrease_priority(v,d) Normally it takes O(n) time to find an element in a heap, but use an integer index (or something) to directly reference its position A min-priority queue that supports extract_min(), add_with_priority(), and decrease_priority() Now Dikstra will run in O( E lg V ) Notice we have to do work on every edge

7 Min-priority queue public interface PriorityQueue<T extends Comparable<T>> { void insert(t t); void remove() throws EmptyQueueException; T top() throwsemptyqueueexception; boolean empty(); } public interface MinPriorityQueue<K, P extends Comparable<T>> { void addwithpriority(k key, P priority); void decreasepriority(k key, P newp) throws InvalidItem; T extractmin() throws EmptyQueueException; boolean empty(); }

8 Min-priority queue public interface MinPriorityQueue<K, P extends Comparable<T>> { void addwithpriority(k key, P priority); void decreasepriority(k key, P newp) throws InvalidItem; T extractmin() throws EmptyQueueException; boolean empty(); } MPQ heap Mike/0 / \ Peter/7 Kelly/1 / \ / Katherine/9 Sydney/7 James/2 keys James Katherine Kelly Mike Peter Sydney

9 Min-priority queue public interface MinPriorityQueue<K, P extends Comparable<T>> { void addwithpriority(k key, P priority); void decreasepriority(k key, P newp) throws InvalidItem; T extractmin() throws EmptyQueueException; boolean empty(); } How to implement? MPQ heap keys Mike/0 / \ Peter/7 Kelly/1 / \ / Katherine/9 Sydney/7 James/2 James Katherine Kelly Mike Peter Sydney (1) If keys are integers in the range 0 to n, make an array of references (2) Use a HashMap from keys to hash (3) Give client a reference, store in graph node

10 Fibonnaci Heap Special form of heaps specifically designed to allow fast updates to values Set of heap-ordered tree (value(n) < value(n.children)) Maintain pointer to overall minimum element Set of marked nodes used to track heights of certain trees Insert Findmin Deletemin Decreasekey Merge Binary Heap O(1) O(lg n) O(lg n) O(lg n) O(n) Fibonnaci Heap O(1) O(lg n) O(1) O(1) O(1) Reduces Dijkstra s run time to O( E + V lg V )

11 Part 2: Minimum Spanning Trees

12 Long Distance Calling Supposes it costs different amounts of money to send data between cities A through F. Find the least expensive set of connections so that anyone can send data to anyone else. Given an undirected graph with weights on the edges, find the subset of edges that (a) connects all of the vertices of the graph and (b) has minimum total costs This subset of edges is called the minimum spanning tree (MST) Removing an edge from MST disconnects the graph, adding one forms a cycle

13 Dijkstra s!= MST 100 X S Y Dijkstra s will build the tree S->X, S->Y (tree visits every node with shortest paths from S to every other node) but the MST is S->X, X->Y (tree visits every node and minimizes the sum of the edges)

14 Prim s Algorithm 1. Pick an arbitrary starting vertex and add it to set T. Add all of the other vertices to set R Every vertex will be included eventually, so doesn t matter where you start T={A} R={B,C,D,E,F}

15 Prim s Algorithm 1. Pick an arbitrary starting vertex and add it to set T. Add all of the other vertices to set R Every vertex will be included eventually, so doesn t matter where you start T={A} R={B,C,D,E,F} 2. While R is not empty, pick an edge from T to R with minimum cost A-B: 4 <- pick me A-C: 7 A-D: 8 A-F: 10

eventually, so doesn t matter where you start T={A} R={B,C,D,E,F} 2.

16 Prim s Algorithm 1. Pick an arbitrary starting vertex and add it to set T. Add all of the other vertices to set R Every vertex will be included eventually, so doesn t matter where you start T={A} R={B,C,D,E,F} 2. While R is not empty, pick an edge from T to R with minimum cost A-B: 4 <- pick me A-C: 7 A-D: 8 A-F: 10

17 Prim s Algorithm 3. Repeat! A-B

18 Prim s Algorithm 3. Repeat! A-B A-C

19 Prim s Algorithm 3. Repeat! A-B A-C C-D

20 Prim s Algorithm 3. Repeat! A-B A-C C-D C-F

21 Prim s Algorithm 3. Repeat! A-B A-C C-D C-F C-E By making a set of simple local choices, it finds the overall best solution The greedy algorithm is optimal J

22 Prim s Algorithm Prim s Algorithm Sketch 1. Initialize a tree with a single vertex, chosen arbitrarily from the graph. 2. Grow the tree by one edge: of the edges that connect the tree to vertices not yet in the tree, find the minimum-weight edge, and add it to the tree. 3. Repeat step 2 (until all vertices are in the tree).

23 Prim s Algorithm Prim s Pseudo-code 1. For each vertex v, Compute C[v] (the cheapest cost of a connection to v from the MST) and an edge E[v] (the edge providing that cheapest connection). Initialize C[v] to and E[v] to a special flag value indicating that there is no edge connecting v to the MST 2. Initialize an empty forest F (set of trees) and a set Q of vertices that have not yet been included in F (initially, all vertices). 3. Repeat the following steps until Q is empty: 1. Find and remove a vertex v from Q with the minimum value of C[v] 2. Add v to F and, if E[v] is not the special flag value, also add E[v] to F 3. Loop over the edges vw connecting v to other vertices w. For each such edge, if w still belongs to Q and vw has smaller weight than C[w], perform the following steps: 1. Set C[w] to the cost of edge vw 2. Set E[w] to point to edge vw. 4. Return F How fast is the naïve version? O( V 2 ) What data structures do we need to make it fast? HEAP

24 Prim s Algorithm Faster Prim s Pseudo-code 1. Add the cost of all of the edges to a heap (or sort the edge costs) 2. Repeatedly pick the next smallest edge (u,v) 1. If u or v is not already in the MST, add the edge uv to the MST How fast is this version O( E lg E ) Fastest Prim s Pseudo-code 1. Add all of the vertices to a min-priority-queue prioritized by the min edge cost to be added to the MST. Initialize (key=v, dist=, edge=<>) 2. Repeatedly pick the next closest vertex v from the MPQ 1. Add v to the MST using the recorded edge 2. For each edge (v, u) 1. If cost(v,u) < MPQ(u) 1. MPQ.decreasePriority(u, cost(v,u), (v,u)) How fast is this version Using Fibonacci Heap O( E lg V ) O( E + V lg V )

25 Teaser: Kruskal s Algorithm Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree

26 Traveling Salesman Problem What if we wanted to compute the minimum weight path visiting every node once? ABDCA: = 14 ACDBA: = 14* ABCDA: = 11 ADCBA: = 11* ACBDA: = 7 ADBCA: = 7 * 3 A C B D 2

27 Greedy Search Greedy Search cur=graph.randnode() A 11 8 B while (!done) next=cur.getnextclosest() Greedy: ABDCA = = 73 Optimal: ACBDA = = 38 C 50 D Greedy finds the global optimum only when 1. Greedy Choice: Local is correct without reconsideration 2. Optimal Substructure: Problem can be split into subproblems Optimal Greedy: Dijkstra s, Prim s, Cookie Monster

28 TSP Complexity No fast solution Knowing optimal tour through n cities doesn't seem to help much for n+1 cities 3 A 1 4 B 2 1 [How many possible tours for n cities?] C 5 D Extensive searching is the only provably correct algorithm A 4 B Brute Force: O(n!) ~20 cities max 20! = 2.4 x C 5 D 2 E 30

29 Branch-and-Bound Abort on suboptimal solutions as soon as possible ADBECA = = 10 ABDE = > 10 ADE = 1+30 > 10 AED = 1+30 > 10 Performance Heuristic Always gives the optimal answer Doesn't always help performance, but often does Current TSP record holder: 85,900 cities 85900! = C A [When not?] 4 E B D

30 TSP and NP-complete TSP is one of many extremely hard problems of the class NP-complete Extensive searching is the only way to find an exact solution Often have to settle for approx. solution WARNING: Many natural problems are in this class Find a tour the visits every node once Find the smallest set of vertices covering the edges Find the largest clique in the graph Find the highest mutual information encoding scheme Find the best set of moves in tetris

31 Part 3: Kruskal s Algorithm and Union Find

32 Kruskal s Algorithm B C E 9 14 D A 10 F Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree

33 Kruskal s Algorithm B C E 9 14 D A 10 F Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree

34 Kruskal s Algorithm B C A E F 9 14 D Tinting is just to make it easier to look at Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree

35 Kruskal s Algorithm B C E 9 14 D A 10 F Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree

Kruskal s Algorithm B 12 4 7 C 3 4 2 8 E 9 14 D A 10 F Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2.

36 Kruskal s Algorithm B C E 9 14 D A 10 F Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree

37 Kruskal s Algorithm B C E 9 14 D A 10 F Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree

38 Kruskal s Algorithm B 4 C E A F 10 D Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree

39 Kruskal s Algorithm B 4 C E A F 10 D Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree

40 Kruskal s Algorithm 4 C E B A F 10 D Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree

41 Kruskal s Algorithm C 4 E B D 4 A F Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph is a separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree

42 Kruskal s Algorithm C 4 E B D 4 A F Running time: Kruskal s Algorithm Sketch 1. Create a forest F (a set of trees), where each vertex in the graph Easy: is a O( V ) separate tree 2. Create a set S containing all the edges in the graph 3. while S is nonempty and F is not yet spanning Easy: O( E lg E ) 1. remove an edge with minimum weight from S 2. if the removed edge connects two different trees then add it to the forest F, combining two trees into a single tree Hmm???

Disjoint Sets C E C E B D B D A F A F Disjoint Subset: every element is assigned to a single subset Problem: Determine which elements are part of the same subset

43 Disjoint Sets C E C E B D B D A F A F Disjoint Subset: every element is assigned to a single subset Problem: Determine which elements are part of the same subset as sets are dynamically joined together Two main methods: Find: Are elements x and y part of the same set? Union: Merge together sets containing elements x and y

44 Quick Find

45 Implementation Find is quick O(1), but each union is O(N) time Kruskal s ultimately merges all N items: O(N 2 ) Can we do any faster?

46 Implementation Find is quick O(1), but each union is O(N) time Kruskal s ultimately merges all N items: O(N 2 ) Can we do any faster?

47 Quick Find Forest

48 Quick Union

49 Quick Union Why not just change id[q]? Why set it to p s root instead of p?

50 Quick Union Implementation

51 Quick Union Example What is the worst case for Quick Union?

52 Quick-Union Analysis

53 Improved Quick-Union: Weighting

54 Improved Quick-Union: Weighting

55 Weighted Quick-Union Implementation

56 Weighted Quick-Union

57 Weighted Quick-Union Hooray! Tree stays pretty flat! How tall can the tree grow? Uniting a short tree and a tall tree doesn t increase the height of the tall tree Height of tree only grows when two trees of equal height are united, thereby doubling the number of elements How many rounds of doubling can we do until we include all N elements? lg N rounds => O(lg N) max height J

58 Weighted Quick-Union Analysis Should we stop trying here? Usually very happy with lg(n), but here we can do better!

59 Improvement 2: Path Compression

60 Path Compression Implementation

61 Weighted Path Compression Example

62 Weighted Path Compression Analysis Much bigger than #atoms in the universe

63 Kruskal s Algorithm C 4 E B D 4 A F Running time: Kruskal s Algorithm Sketch 1. Using Create Weighted-Path a forest F (a Compression set of trees), Quick where Union each vertex in the graph is a (aka separate Union-Find) tree each find or union in O(lg* V ) time. Easy: O( V ) 2. Total Create time a is set O( E lg E S containing + Elg* V ) all the => edges O( E lg E ) in the graph 3. while S is nonempty and F is not yet spanning Easy: O( E lg E ) If the 1. edges remove are an already edge with in sorted minimum order weight (or use from a S linear 2. time if the sorting removed algorithm edge connects such as counting two different sort), trees then add it to the reduce forest total time F, combining to O( E lg* V ) two trees into a single tree Hmm???

64 Other Applications: Connected Components

65 Other Applications: Connected Components How would you use Union-Find to solve this? Assign every dot to a set, then merge sets that have an edge between them

66 Other Applications: Connected Components 63 Connected Components

67 Never underestimate the power of the LOG STAR!!!!

68 Next Steps 1. Reflect on the magic and power of the logstar! 2. Final Exam: Sunday Dec 9am

69 Welcome to CS Questions?

Greedy Algorithms. At each step in the algorithm, one of several choices can be made.

Greedy Algorithms. At each step in the algorithm, one of several choices can be made. Greedy Algorithms At each step in the algorithm, one of several choices can be made. Greedy Strategy: make the choice that is the best at the moment. After making a choice, we are left with one subproblem