Autumn Minimum Spanning Tree Disjoint Union / Find Traveling Salesman Problem

Transcription

1 Autumn Minimum Spanning Tree Disjoint Union / Find Traveling Salesman Problem

2 Input: Undirected Graph G = (V,E) and a cost function C from E to the reals. C(e) is the cost of edge e. Output: A spanning tree T with minimum total cost. That is: T that minimizes C ( T ) = C( e) e T 2

3

4 Cost = = 0

5 Boruvka 926 Kruskal 956 Prim 957 also by Jarnik 90 Karger, Klein, Tarjan 995 Randomized linear time algorithm Probably not practical, but very interesting 5

6 G = (V,E) with costs C. G connected. Let (V,A) be a subgraph of G that is contained in a minimum spanning tree. Let U be a set such that no edge in A has one end in U and one end in V-U. Let C({u,v}) minimal and u in U and v in V-U. Let A be A with {u,v} added. Then (V,A ) is contained in a minimum spanning tree. 6

7 A C({u,v}) is minimal U u v V-U A 7

8 U T C({u,v}} is minimal C({u,v}) < C({x,y}) x u v V-U y 8

9 U T C(T ) = C(T) + C({u,v}) - C({x,y}) C(T ) < C(T) x u v V-U y A T is also a minimum spanning tree 9

10 Sort the edges by increasing cost; Initialize A to be empty; For each edge e chosen in increasing order do if adding e does not form a cycle then add e to A Invariant: A is always contained in some minimum spanning tree 0

11 {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5} 0 2 2

12 {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5}

13 {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5} 0 2 2

14 {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5} 0 2 2

15 {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5}

16 {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5}

17 {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5}

18 {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5}

19 {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5}

20 {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5}

21 Sorted edge list {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5} Disjoint Union / Find Union(a,b) - union the disjoint sets named by a and b Find(a) returns the name of the set containing a 2

22 2 Find(5) = 7 Find() = {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5}

23 2 Find() = Find(6) = {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5}

24 Union(,7) {7,} {2,} {7,5} {5,6} {5,} {,6} {2,7} {2,} {,} {,5}

25 Sort the edges by increasing cost; Initialize A to be empty; for each edge {i,j} chosen in increasing order do u := Find(i); v := Find(j); if not(u = v) then add {i,j} to A; Union(u,v); 25

26 Initial state Intermediate state

27 Find(i) - follow pointer to root and return the root. Union(i,j) - assuming i and j roots, point i to j. 7 Union(,7)

28 Weighted Union Always point the smaller tree to the root of the larger tree 2 7 W-Union(,7)

29 On a Find operation point all the nodes on the search path directly to the root Find()

30 up weight

31 PC-Find(i : index) r := i; while not(up[r] = 0) do r := up[r] k := up[i]; while not(k = r) do up[i] := r; i := k; k := up[k] return(r) end{find} W-Union(i,j : index) // i and j are roots wi := weight[i]; wj := weight[j]; if wi < wj then up[i] := j; weight[j] := wi + wj; else up[j] :=i; weight[i] := wi +wj; end{w-union}

32 Worst case time complexity for a W-Union is O() and for a PC-Find is O(log n). Time complexity for m operations on n elements is O(m log* n) where log* n is a very slow growing function. Essentially constant time per operation! Using ranked union gives an even better bound theoretically. 2

33 The time complexity of PC-Find is O(log n). An up tree formed by W-Union of height h has at least 2 h nodes. Inductive Proof. T T h+ T2 h Weight(T2) > 2 h (ind. hyp.) Weight(T) > Weight(T2) > 2 h Weight(T) > 2 h +2 h =2 h+

34 n/2 Weighted Unions n/ Weighted Unions

35 After n - = n/2 + n/ + + Weighted Unions If there are n = 2 k nodes then there are k pointers on the longest path to root. Find 5

36 For disjoint union / find with weighted union and path compression. average time per operation is essentially a constant. worst case time for a PC-Find is O(log n). An individual operation can be costly, but over time the average cost per operation is not. 6

37 Sort the edges by increasing cost; Initialize A to be empty; for each edge {i,j} chosen in increasing order do u := PC-Find(i); v := PC-Find(j); if not(u = v) then add {i,j} to A; W-Union(u,v); 7

38 Let G have n vertices and m edges. Sort the edges - O(m log m). Traverse the sorted edge list doing PC- Finds and W-Unions - O(m log* n) Total time is O(m log m). 8

39 We maintain a single tree. For each vertex not in the tree maintain the smallest edge to a vertex in the tree

40

41

42

43

44

45

46 Adjacency Lists - we need to look at all the edges from a newly added vertex. Array for the best edges in or to the tree to cost

47 Priority queue for all edges to the tree (blue edges). Insert, delete-min, delete (e.g. binary heap) to cost to cost

48 n vertices and m edges. Priority queue O(log n) per operation. O(m) priority queue operations. An edge is visited when a vertex incident to it joins the tree. Time complexity is O(m log n). Storage complexity is O(m). 8

49 Kruskal Simple Requires sorting of edges O(m log m) Prim More complicated Perhaps better with dense graphs - O(m log n) Can be improved to O(m + n log n) using a Fibonacci Heap 9

50 . Show that if the edge costs are distinct then the minimum spanning tree is unique. 2. Define the a bottleneck spanning tree as a spanning tree with minimum maximum edge cost. Show that a minimum spanning tree is a bottleneck spanning tree. 50

51 Input: Undirected Graph G = (V,E) and a cost function C from E to the reals. C(e) is the cost of edge e. Output: A cycle that visits each vertex exactly once and is minimum total cost. 5

52

53 Cost = = 5

54 Hamiltonian Cycle Is there a cycle that visits each vertex exactly once Ignores costs Triangle inequality constraint C(u,v) < C(u,x) + C(x,v) Euclidean Traveling Salesman Vertices are points on the plane and the cost is the Euclidian distance between them Implies triangle inequality 5

55 Telescope planning Route planning coin pickup mail delivery Circuit board drilling Multispectral image compression 55

56 Old well-studied problem Example of an NP-hard problem These problems are very hard to solve exactly No polynomial time algorithms known to exist Interesting and effective approximation algorithms Good practical algorithms Simple algorithms with provable approximation bounds 56

57 a d Euclidean distance n(n-)/2 edges b e c f g h 57

58 a d b e c f g h 58

59 a d b e c f g h Marking Order = a, b, c, d, e, f, h, g 59

60 a d b e c f g h Marking Order = a, b, c, d, e, f, h, g 60

61 Time and Storage Time O(n 2 log n) with Kruskal or Prim Storage O(n) because edges can be computed as needed Quality of Solution H C(H) < 2 C(H*) where H* is an optimal tour This is a 2-approximation algorithm Same approximation bound applies to case of triangle inequality 6

62 Setup T minimum spanning tree W the depth-first walk of T H the tour computed by the algorithms H* an optimal tour 62

63 a b d e C(W) = 2 C(T) C(H) < C(W) triangle inequality c f g h Depth-first walk = a,b,c,b,a,d,e,f,h,f,e,g,e,d,a Marking order = a,b,c, d,e,f,h, g 6

64 . C(W) = 2 C(T) 2. C(H) < C(W), triangle inquality. C(H) < 2 C(T), last two lines. C(T) < C(H*), minus an edge H* is a spanning tree 5. C(H) < 2 C(H*), last two lines 6

65 Branch-and-Bound n < 25? Linear Programming n < 00 Cutting Plane Methods for Euclidian case n < 5,000 with concord see 65

66 /2 approximation algorithm of Christofedes Polynomial approximation scheme for Euclidian TSP by Aurora (998), Mitchell (999) To get within (+ε) of optimal can be done in time polynomial in /ε and n. These are not practicalm 66

67 Local Search Lin-Kernighan method Simulated Annealing Genetic Algorithms Neural Networks 67

68 Start with an initial solution that is usually easy to find, but is not necessarily good. Repeatedly modify the current solution to a better nearby one. Until no nearby one is better. 68

69 x u 2-opt(x,y,u,v) x u y y v v 69

70 Lin-Kernighan (97) Find an initial tour T. For every pair of distinct edges (x,y), (u,v) in T if C(x,u) + C(y,v) < C(x,y) + C(u,v) then T := T {(x,y),(u,v)} union {(x,u),(y,v)} exit for loop and go to Return T 70

71 Euclidian case a d b e c f g h 7

72 Euclidian case a d b e c f g h 72

73 Euclidian case a d b e c f g h 7

74 Euclidian case a d b e c f g h 7

75 Euclidian case a d b e c f g h 75

76 Euclidian case a d b e c f g h 76

77 Euclidian case a d b e c f g h 77

78 Empirical O(n 2.2 ) time Finds optimal in most examples < 00 points Excellent Implementations Can easily handle hundreds of thousands of points 78

79 Local search can lead to a local minimum in the solution space, not necessarily a global minimum. Solution Surface Local minimum Global minimum 79