CS 4407 Algorithms. Lecture 8: Circumventing Intractability, using Approximation and other Techniques

Size: px

Start display at page:

Download "CS 4407 Algorithms. Lecture 8: Circumventing Intractability, using Approximation and other Techniques"

Julius Underwood
5 years ago
Views:

1 CS 4407 Algorithms Lecture 8: Circumventing Intractability, using Approximation and other Techniques Prof. Gregory Provan Department of Computer Science University College Cork CS

2 Lecture Outline Introduction to Approximation Algorithms Motivation Approximation Algorithm definitions Examples Vertex cover Traveling Salesman Summary

3 Motivation By now we ve seen many NP-Complete problems. We conjecture none of them has polynomial time algorithm.

4 Motivation Is this a dead-end? Should we give up altogether?

5 Motivation Or maybe we can settle for good approximation algorithms?

6 Today s Learning Objectives Objectives: To formalize the notion of approximation. To demonstrate several such algorithms. Overview: Optimization and Approximation Vertex Cover, Traveling Salesman

7 Optimization Many of the problems we ve encountered so far are really optimization problems. I.e - the task can be naturally rephrased as finding a maximal/minimal solution. For example: finding a maximal clique in a graph.

8 Approximation An algorithm that returns an answer C which is close to the optimal solution C* is called an approximation algorithm. Closeness is usually measured by the ratio bound ρ(n) the algorithm produces. Which is a function that satisfies, for any input size n, max{c/c*,c*/c} ρ ρ(n).

9 Approximation Algorithms Approximation algorithms: provide sub-optimal answers that are still reasonable Approximation algorithms tend to use greedy approaches In some cases, these algorithms rely on heuristics to help search through the possible answers for the best one (which is not really the best one, but might be good enough) A heuristic is generally defined as a rule of thumb, some basic idea applied to problem solving Often, heuristics are used in Artificial Intelligence to reduce an intractable search to something tractable

10 Definitions A feasible solution is an object of the right type but not necessarily an optimal one Example: a feasible solution to graph coloring is one that colors all vertices such that no two adjacent vertices have the same color, but such a solution may not have the minimum number of colors Example: a feasible solution to bin packing will fill items into bins such that no bin exceeds 1 in size, but many not provide the minimum number of bins Value function returns the value of the parameter for the feasible solution that is used to determine optimality (number of colors, number of bins, etc) Optimum value the value returned by the value function when tested on the optimal solution (or as provided by the optimal algorithm) We judge an approximation algorithm by the value provided by the value function compared to the optimum value This is a value >= 1 (where 1 means as good as the optimal algorithm)

11 Vertex cover Examples Real-world motivation Approximation algorithm Traveling salesman Real-world motivation Approximation algorithm

12 Network Power Say you have a network, with links between some components Each link requires power supply, hence, you need to supply power to a set of nodes that cover all links Obviously, you d like to connect the smallest number of nodes

13 VERTEX-COVER Instance: an undirected graph G=(V,E). Problem: find a set C V of minimal size s.t. for any (u,v) E, either u C or v C. Example:

14 Minimum VC NP-hard Proof: It is enough to show the decision problem below is NP-Complete: Instance: an undirected graph G=(V,E) and a number k. Problem: to decide if there exists a set V V of size k s.t for any (u,v) E, u V or v V. This follows immediately from the following observation.

15 Minimum VC NP-hard Observation: Let G=(V,E) be an undirected graph. The complement V\C of a vertex-cover C is an independent-set of G. Proof: Two vertices outside a vertex-cover cannot be connected by an edge.

16 VC - Approximation Algorithm C φ E E while E φ do let (u,v) be an arbitrary edge of E C C {u,v} remove from E every edge incident to either u or v. return C.

17 Demo

18 Polynomial Time O(n 2 ) C φ E E O(n 2 ) while E φ do let (u,v) be an arbitrary edge of E C C {u,v} remove from E every edge incident to either u or v return C O(1) O(n)

19 Correctness The set of vertices our algorithm returns is clearly a vertex-cover, since we iterate until every edge is covered.

20 Vertex-cover problem (Example 2) Near Optimal size=6 Optimal Size=3 20

21 How Good an Approximation is it? Observe the set of edges our algorithm chooses no common vertices! any VC contains 1 in each our VC contains both, hence at most twice as large

22 The vertex-cover problem: Bounds Optima l This is a polynomial-time 2-aproximation algorithm. (Why?) Because: Selected Vertices APPROX-VERTEX-COVER is O(V+E) C* A C = 2 A C 2 C* Selected Edges 22

23 Example 2: Traveling Salesman The Mission: A Tour Around the World

24 The Problem: Traveling Costs Money 1795$

25 Introduction Objectives: To explore the Traveling Salesman Problem. Overview: TSP: Formal definition & Examples TSP is NP-hard Approximation algorithm for special cases Inapproximability result

26 TSP Instance: a complete weighted undirected graph G=(V,E) (all weights are non-negative). Problem: to find a Hamiltonian cycle of minimal cost

27 Polynomial Algorithm for TSP? What about the greedy strategy: At any point, choose the closest vertex not explored yet?

28 The Greedy $trategy Fails

29 The Greedy $trategy Fails

30 TSP is NP-hard The corresponding decision problem: Instance: a complete weighted undirected graph G=(V,E) and a number k. Problem: to find a Hamiltonian cycle whose cost is at most k.

31 TSP is NP-hard Theorem: HAM-CYCLE p TSP. Proof: By the straightforward efficient reduction illustrated below: verify! k= V HAM-CYCLE TSP

32 What Next? We ll show an approximation algorithm for TSP, with approximation factor 2 for cost functions that satisfy a certain property.

33 The Triangle Inequality Definition: We ll say the cost function c satisfies the triangle inequality, if u,v,w V : c(u,v)+c(v,w) c(u,w) u v w

34 Approximation Algorithm 1. Grow a Minimum Spanning Tree (MST) for G. 2. Return the cycle resulting from a preorder walk on that tree.

35 Demonstration and Analysis The cost of a minimal Hamiltonian cycle the cost of a MST

36 Demonstration and Analysis The cost of a preorder walk is twice the cost of the tree

37 Demonstration and Analysis Due to the triangle inequality, the Hamiltonian cycle is not worse.

38 The Bottom Line optimal HAM cycle preorder MST = ½ walk ½ our HAM cycle

39 Example a d e T a d T a d e e b f g b f g b f g c h c h c h T a d e a d optimal solution e b f g b f g c h H c h H*

40 Formal Algorithm The TSP with triangle inequality APPROX_TSP_TOUR( G,c) 1 Select a vertex r V [ G ] to be a root vertex 2 grow a MST T for G from root r using MST_PRIM(G,c,r) 3 Let L be the list of vertices visited in a preorder walk of T. 4 return the ha miltonian cycle H that visit the vertices in the order L.

41 Formal Proof Theorem APPROX_TSP_TOUR is an approximation algorithm with ratio bound of 2 for TSP with triangular inequality. Proof. c ( T ) c ( H *) c( W ) = 2c( T ) 2c( H *) c( H ) c( W ) c( H ) 2 c( H *)

42 What About the General Case? We can show TSP cannot be approximated within any constant factor ρ 1

43 Formal Proof Sketch of TSP Bounds The general TSP Theorem 35.3 If N P P and ρ 1, there is no polynomial time approximation algorithm with ratio bound ρ for some general TSP. Proof. Let G = ( V, E ) be an instance of HC. Let G ' = ( V, E' ) be the complete graph of V. Set c( u, v ) = 1 ρ V + 1 if ( u, v ) E otherw ise

44 Summary As it turns out, we can sometimes find efficient approximation algorithms for NPhard problems. We ve seen two such algorithms: for VERTEX-COVER (factor 2) for TSP (factor 2). In some cases (TSP) finding close approximations is NP-hard

COMP 355 Advanced Algorithms Approximation Algorithms: VC and TSP Chapter 11 (KT) Section (CLRS)

COMP 355 Advanced Algorithms Approximation Algorithms: VC and TSP Chapter 11 (KT) Section 35.1-35.2(CLRS) 1 Coping with NP-Completeness Brute-force search: This is usually only a viable option for small