There are severe limits on our ability to compute near-optimal tours It is NP-complete to decide whether a given undirected =(,)has a Hamiltonian cycle An approximation algorithm for the TSP can be used to solve the Hamiltonian cycle problem: Given a graph =(,), form an input to the TSP by setting, for each pair,, the cost =1 if (,, and equal to +2otherwise If there is a Hamiltonian cycle in, then there is a tour of cost, and otherwise each tour costs at least + 1 MAT-72606 ApprAl, Spring 2017 22-Mar-17 114 If there were to exist a 2-approximation algorithm for the TSP, then we could use it to distinguish graphs with Hamiltonian cycles from those without any Run the approximation algorithm on the new TSP input, and if the tour computed has cost at most, then there exists a Hamiltonian cycle in, and otherwise there does not Setting the cost for the non-edges to be +2 has a similarly inflated consequence, and we obtain an input to the TSP of polynomial size provided that, for example, = (2 ) MAT-72606 ApprAl, Spring 2017 22-Mar-17 115 1
Theorem 2.9: For any > 1, there does not exist an -approximation algorithm for the traveling salesman problem on cities, provided NP. In fact, the existence of an (2 )-approximation algorithm for the TSP would similarly imply that P = NP. This however is not the end of the story A natural assumption to make about the input to the TSP is to restrict attention to those inputs that are metric; that is, for each triple,, we have that the triangle inequality holds + MAT-72606 ApprAl, Spring 2017 22-Mar-17 116 This rules out the construction used in the reduction for the HC problem above; the nonedges can be given cost at most 2 for the triangle inequality to hold, and this is too small to yield a nontrivial nonapproximability result A natural greedy heuristic to consider for the TSP; this is often referred to as the nearest addition algorithm Find the two closest cities, say, and, and start by building a tour on that pair of cities consisting of going from to and then back to again This is the first iteration MAT-72606 ApprAl, Spring 2017 22-Mar-17 117 2
In each subsequent iteration, we extend the tour on the current subset by including one additional city, until we include all cities In each iteration, we find a pair of cities and for which the cost is minimum; let be the city that follows in the current tour on We add to, and insert into the current tour between and MAT-72606 ApprAl, Spring 2017 22-Mar-17 118 The crux of the analysis is the relationship of this algorithm to Prim s algorithm for the minimum spanning tree (MST) in an undirected graph A spanning tree of a connected graph =(,) is a minimal subset of edges such that each pair of nodes in is connected by a path using edges only in A MST is a spanning tree for which the total edge cost is minimized Prim s algorithm computes a MST by iteratively constructing a set along with a tree, starting with ={}for some (arbitrarily chosen) node and = (, )with = MAT-72606 ApprAl, Spring 2017 22-Mar-17 119 3
MAT-72606 ApprAl, Spring 2017 22-Mar-17 120 In each iteration, it determines the edge (, ) such that and is of minimum cost, and adds the edge (, ) to Clearly, this is the same sequence of vertex pairs identified by the nearest addition algorithm Furthermore, there is another important relationship between MST problem and TSP Lemma 2.10: For any input to the traveling salesman problem, the cost of the optimal tour is at least the cost of the minimum spanning tree on the same input MAT-72606 ApprAl, Spring 2017 22-Mar-17 121 4
Theorem 2.11: The nearest addition algorithm for the metric TSP is a 2-approximation algorithm. Proof: Let,,, = 1,, be the subsets identified at the end of each iteration of the nearest addition algorithm (where =), and let =,,,,,,, where, is the edge identified in iteration 1(with, = 3,,). We also know that 1,,, is a MST for the original input, when viewed as a complete undirected graph with edge costs. Thus, if OPT is the optimal value for the TSP input, then OPT MAT-72606 ApprAl, Spring 2017 22-Mar-17 122 The cost of the tour on the first two nodes and is exactly 2. Consider an iteration in which a city is inserted between cities and. How much does the length of the tour increase? An easy calculation gives +. By the triangle inequality, we have that + or, equivalently,. Hence, the increase in cost in this iteration is at most + =2. Thus, overall, we know that the final tour has cost at most 2 2OPT, and the theorem is proved. MAT-72606 ApprAl, Spring 2017 22-Mar-17 123 5
A graph is said to be Eulerian if there exists a permutation of its edges of the form (, ),,,,,,(, ) We will call this permutation a traversal of the edges, since it allows us to visit every edge exactly once A graph is Eulerian if and only if it is connected and each node has even degree (the number of edges with as one of its endpoints) Furthermore, if a graph is Eulerian, one can easily construct the required traversal of the edges, given the graph MAT-72606 ApprAl, Spring 2017 22-Mar-17 124 To find a good tour for a TSP input, suppose that we first compute a MST (e.g., by Prim s alg.) Suppose that we then replace each edge by two copies of itself The resulting (multi)graph has cost at most 2OPT and is Eulerian We can construct a tour of the cities from the Eulerian traversal of the edges, (, ),,,,,,(, ) Consider the sequence of nodes,,,,, and remove all but the first occurrence of each city in this sequence MAT-72606 ApprAl, Spring 2017 22-Mar-17 125 6
MAT-72606 ApprAl, Spring 2017 22-Mar-17 126 This yields a tour containing each city exactly once (assuming we then return to at the end) To bound the length of this tour, consider two consecutive cities in this tour, and Cities,, have already been visited earlier in the tour By the triangle inequality,, can be upper bounded by the total cost of the edges traversed in the Eulerian traversal between and, i.e., the total cost of the edges (, ),,(, ) In total, the cost of the tour is at most the total cost of all of the edges in the Eulerian graph, which is at most 2OPT MAT-72606 ApprAl, Spring 2017 22-Mar-17 127 7
Theorem 2.12: The double-tree algorithm for the metric traveling salesman problem is a 2- approximation algorithm. The message of the analysis of the double-tree algorithm is also quite useful If we can efficiently construct an Eulerian subgraph of the complete input graph, for which the total edge cost is at most times the optimal value of the TSP input, then we have derived an -approximation algorithm as well This strategy can be carried out to yield a 3/2- approximation algorithm MAT-72606 ApprAl, Spring 2017 22-Mar-17 128 Consider the output from the MST computation This graph is not Eulerian, since any tree must have nodes of degree one, but it is possible that not many nodes have odd degree is the set of odd-degree nodes in the MST The sum of node degrees must be even, since each edge in the graph contributes 2 to this total The total degree of the even-degree nodes must also be even, but then the total degree of the odd-degree nodes must also be even I.e., we must have an even number of odddegree nodes; = 2 for some positive MAT-72606 ApprAl, Spring 2017 22-Mar-17 129 8
Suppose that we pair up the nodes in : (, ),,,,(, ) Such a collection of edges that contain each node in exactly once is called a perfect matching of Classic result of combinatorial optimization: given a complete graph (on an even number of nodes) with edge costs, it is possible to compute the perfect matching of minimum total cost in polynomial time Given the MST, we identify the set of odddegree nodes with even cardinality, and then compute a minimum-cost perfect matching on MAT-72606 ApprAl, Spring 2017 22-Mar-17 130 MAT-72606 ApprAl, Spring 2017 22-Mar-17 131 9
Add this set of edges to our MST to construct an Eulerian graph on our original set of cities connected since the MST is connected has even degree, since we added a new edge incident to each node of odd degree in the MST As in the double-tree algorithm, we can shortcut this graph to produce a tour of no greater cost This is known as Christofides algorithm Theorem 2.13: Christofides algorithm for the metric traveling salesman problem is a 3/2- approximation algorithm. MAT-72606 ApprAl, Spring 2017 22-Mar-17 132 Proof: Show that the edges in the Eulerian graph produced by the algorithm have total cost at most. We know that the MST edges have total cost at most OPT. So we need only show that the perfect matching on has cost at most OPT/2. First observe that there is a tour on just the nodes in of total cost at most OPT. This again uses the shortcutting argument. Start with the optimal tour on the entire set of cities, and if for two cities and, the optimal tour between and contains only cities that are not in, then include edge (, ) in the tour on. MAT-72606 ApprAl, Spring 2017 22-Mar-17 133 10
Each edge in the tour corresponds to disjoint paths in the original tour, and hence by the triangle inequality, the total length of the tour on is no more than the length of the original tour. Now consider this shortcut tour on the node set. Color these edges red and blue, alternating colors. This partitions the edges into two sets; each of these is a perfect matching on the node set. In total, these two edge sets have cost OPT. Thus, the cheaper of these two sets has cost OPT/2. Hence, there is a perfect matching on of cost OPT/2. Therefore, the algorithm must find a matching of cost at most OPT/2. MAT-72606 ApprAl, Spring 2017 22-Mar-17 134 No better approximation algorithm for the metric TSP is known Substantially better algorithms might yet be found, since the strongest negative result is Theorem 2.14: Unless P = NP, for any constant < 1.0045, no -approximation algorithm for the metric TSP exists. It is possible to obtain a PTAS in the case that cities correspond to points in the Euclidean plane and the cost of traveling between two cities is equal to the Euclidean distance between the two points MAT-72606 ApprAl, Spring 2017 22-Mar-17 135 11
3. Rounding data and dynamic programming The knapsack problem Scheduling jobs on identical parallel machines The bin-packing problem Dynamic programming is a standard technique in algorithm design an optimal solution for a problem is built up from optimal solutions for a number of subproblems, normally stored in a table or multidimensional array Approximation algorithms can be designed using dynamic programming in a variety of ways, many of which involve rounding the input data in some way MAT-72606 ApprAl, Spring 2017 22-Mar-17 137 12
3.1 The knapsack problem We are given a set of items = {1,,}, where each item has a value and a size All sizes and values are positive integers The knapsack has a positive integer capacity The goal is to find a subset of items that maximizes the value of items in the knapsack subject to the constraint that the total size of these items is no more than the capacity; that is, MAT-72606 ApprAl, Spring 2017 22-Mar-17 138 We consider only items that could actually fit in the knapsack, so that for each We can use dynamic programming to find the optimal solution to the knapsack problem We maintain an array entry () for = 1,, Each entry () is a list of pairs (, ) A pair (, ) in the list of entry () indicates that there is a set from the first items that uses space exactly and has value exactly I.e., there exists a set {1,,}s.t. = and = MAT-72606 ApprAl, Spring 2017 22-Mar-17 139 13
Each list keeps track of only the most efficient pairs To do this, we need the notion of one pair dominating another one: (, ) dominates another pair (, )if and ; that is, the solution indicated by the pair (, )uses no more space than (, ), but has at least as much value Domination is a transitive property: if (, ) dominates (, ) which dominates (, ), then (, ) also dominates (, ) We will ensure that in any list, no pair dominates another one MAT-72606 ApprAl, Spring 2017 22-Mar-17 140 This means that we can assume each list () is of the form (, ),,(, )with < < < and < < Since the sizes of the items are integers, there are at most +1pairs in each list Furthermore, if we let = be the maximum possible value for the knapsack, then there can be at most +1pairs in the list Finally, we ensure that for each feasible set {1,,}(with ), the list () contains some pair (, ) that dominates, MAT-72606 ApprAl, Spring 2017 22-Mar-17 141 14
Alg. 3.1 is the dynamic program that constructs the lists () and solves the knapsack problem We start out with (1) = {(0,0), (, )} For each = 2,,, we do the following We first set ( ( 1), and for each (, ( 1), we also add the pair ( +, + ) to the list () if + We finally remove from () all dominated pairs by sorting the list with respect to their space component, MAT-72606 ApprAl, Spring 2017 22-Mar-17 142 retaining the best value for each space total possible, and removing any larger space total that does not have a corresponding larger value One way to view this process is to generate two lists, ( 1) and the one augmented by (, ), and then perform a type of merging of these two lists We return the pair (, ) from () of maximum value as our solution MAT-72606 ApprAl, Spring 2017 22-Mar-17 143 15
ALGORITHM 3.1: A dynamic programming algorithm for the knapsack problem 1. (1) {(0,0), (, )} 2. for 2to do 3. ( ( 1) 4. for each (, ( 1) do 5. if + then 6. Add ( +, + ) to () 7. Remove dominated pairs from () 8. return max (,)() MAT-72606 ApprAl, Spring 2017 22-Mar-17 144 Theorem 3.1: Algorithm 3.1 correctly computes the optimal value of the knapsack problem. Proof: By induction on we prove that () contains all non-dominated pairs corresponding to feasible sets {1,,}. Certainly this is true in the base case by setting (1) to {(0,0), (, )}. Suppose it is true for ( 1). Let {1,,}, and let = and =. We claim that there is some pair (, ()such that and. MAT-72606 ApprAl, Spring 2017 22-Mar-17 145 16
First, suppose that. Then the claim follows by the induction hypothesis and by the fact that we initially set () to ( 1) and removed dominated pairs. Now suppose. Then for = {}, by the induction hypothesis, there is some (, ( 1) that dominates,,, so that and. Then the algorithm will add the pair ( +, + ) to (), where + and +. Thus, there will be some pair (, ()that dominates (, ). MAT-72606 ApprAl, Spring 2017 22-Mar-17 146 Algorithm 3.1 takes (min(, ))time This is not a polynomial-time algorithm, since we assume that all input numbers are encoded in binary; thus, the size of the input number is essentially log, and so the running time () is exponential in the size of the input number, not polynomial If we were to assume that the input is given in unary, then () would be a polynomial in the size of the input It is sometimes useful to make this distinction between problems MAT-72606 ApprAl, Spring 2017 22-Mar-17 147 17
Definition 3.2: An algorithm for a problem is said to be pseudopolynomial if its running time is polynomial in the size of the input when the numeric part of the input is encoded in unary. If the maximum possible value were some polynomial in, then the running time would indeed be a polynomial in the input size We now show how to get a PTAS for the knapsack problem by rounding the values of the items so that is indeed a polynomial in The rounding induces some loss of precision in the value of a solution, but this does not affect the final value by too much MAT-72606 ApprAl, Spring 2017 22-Mar-17 148 Recall the definition of an approximation scheme Definition 3.3: A polynomial-time approximation scheme (PTAS) is a family of algorithms { }, where there is an algorithm for each >0, such that is a (1+)-approximation algorithm (for minimization problems) or a (1)-approximation algorithm (for maximization problems). The running time of the algorithm is allowed to depend arbitrarily on 1/: this dependence could be exponential in 1/, or worse We often focus attention on algorithms for which we can give a good bound of the dependence of the running time of on 1/ MAT-72606 ApprAl, Spring 2017 22-Mar-17 149 18
Definition 3.4: A fully polynomial-time approximation scheme (FPAS, FPTAS) is an approximation scheme such that the running time of is bounded by a polynomial in 1/. We can now give a FPTAS for knapsack Let us measure value in (integer) multiples of (we set below), and convert each value by rounding down to the nearest multiple of Precisely, we set = for each item We then run the Algorithm 3.1 on the items with sizes and values, and output the optimal solution for the rounded data as a near-optimal solution for the true data MAT-72606 ApprAl, Spring 2017 22-Mar-17 150 The main idea: Show that the accuracy we lose in rounding is not so great, and yet the rounding enables us to have the algorithm run in polynomial time Let us first do a rough estimate; if we used values = instead of, then each value is inaccurate by at most, and so each feasible solution has its value changed by at most We want the error introduced to be times a lower bound on the optimal value (and so be sure that the true relative error is at most ) Let be the maximum value of an item; that is, = max MAT-72606 ApprAl, Spring 2017 22-Mar-17 151 19
is a lower bound on OPT, since one can pack the most valuable item in the knapsack by itself Thus, it makes sense to set so that = or, in other words, to set = / Note that with the modified values, = = = / Thus, the running time of the algorithm is (min(, )) = ( /) and is bounded by a polynomial in 1/ The algorithm returns a solution whose value is at least (1 ) times that of an optimal solution MAT-72606 ApprAl, Spring 2017 22-Mar-17 152 ALGORITHM 3.2: An approximation scheme for the knapsack problem 1. max 2. / 3. for all 4. Run Algorithm 3.1 for knapsack instance with values MAT-72606 ApprAl, Spring 2017 22-Mar-17 153 20
Theorem 3.5: Algorithm 3.2 is a FPTAS for the knapsack problem. Proof: We need to show that the algorithm returns a solution whose value is at least (1) times the value of an optimal solution. Let be the set of items returned by the algorithm. Let be an optimal set of items. Certainly OPT (put the most valuable item in a knapsack by itself). Furthermore, by the definition of, ( + 1), so that. Applying the definitions of the rounded data, along with the fact that is an optimal solution for the values, we can derive the following inequalities: MAT-72606 ApprAl, Spring 2017 22-Mar-17 154 = OPT OPT = 1 OPT. MAT-72606 ApprAl, Spring 2017 22-Mar-17 155 21
3.2 Scheduling jobs on identical parallel machines Earlier we saw that by first sorting the jobs in order of non-increasing processing requirement, and then using a list scheduling rule, we find a schedule of length guaranteed to be at most 4/3 times the optimum This result contains the seeds of a PTAS For any given value of >1, we give an algorithm that runs in polynomial time and finds a solution of objective function value at most times the optimal value MAT-72606 ApprAl, Spring 2017 22-Mar-17 156 As earlier, let the processing requirement of job be, = 1,,, and let denote the length of a given schedule with job completion times, =1,,; the optimal value is denoted Each processing requirement is positive integer In the analysis of the list scheduling rule its error can be upper bounded by the processing requirement of the last job to complete MAT-72606 ApprAl, Spring 2017 22-Mar-17 157 22
The 4/3-approximation was based on this fact, combined with the observation that when each job s processing requirement is more than /3, this natural greedy-type algorithm actually finds the optimal solution We present an approximation scheme for this problem based on a similar principle Focus on a specified subset of the longest jobs, and compute the optimal schedule for that subset Then we extend that partial schedule by using list scheduling on the remaining jobs There is a trade-off between the number of long jobs and the quality of the solution found MAT-72606 ApprAl, Spring 2017 22-Mar-17 158 Let be a fixed positive integer; we will derive a family of algorithms, and focus on among them We partition the job set into two parts: the long jobs and the short jobs, where a job is considered short if This implies that there are at most long jobs Enumerate all schedules for the long jobs, and choose one with the minimum makespan Extend this schedule by using list scheduling for the short jobs I.e., given an arbitrary order of the short jobs, schedule these jobs in order, always assigning the next job to the machine currently least loaded MAT-72606 ApprAl, Spring 2017 22-Mar-17 159 23
Consider the running time of algorithm To specify a schedule for the long jobs, we simply indicate to which of the machines each long job is assigned; thus, there are at most distinct assignments If we focus on the special case of this problem in which the number of machines is a constant (say, 100, 1,000, or even 1,000,000), then this number is also a constant, not depending on the size of the input Thus, we can check each schedule, and determine the optimal length schedule in polynomial time in this special case MAT-72606 ApprAl, Spring 2017 22-Mar-17 160 As in the analysis of the local search algorithm earlier, we focus on the last job to finish Recall that we derived the equality that + The validity of this inequality relied only on the fact that each machine is busy up until the time that job starts To analyze the algorithm that starts by finding the optimal schedule for the long jobs, we distinguish now between two cases MAT-72606 ApprAl, Spring 2017 22-Mar-17 161 24
If the last job to finish (in the entire schedule), job, is a short job, then this job was scheduled by the list scheduling rule, and it follows that the previous inequality holds Since job is short, and hence (), it also follows that + 1+ 1 1+1 If is a long job, then the schedule delivered is optimal, since its makespan equals the length of the optimal schedule for just the long jobs, which is clearly no more than for the entire input MAT-72606 ApprAl, Spring 2017 22-Mar-17 162 The algorithm can easily be implemented to run in polynomial time (treating as a constant) Theorem 3.6: The family of algorithms is a polynomial-time approximation scheme for the problem of minimizing the makespan on any constant number of identical parallel machines. A significant limitation is that the number of machines needs to be a constant It is not too hard to extend these techniques to obtain a PTAS even if the number of machines is allowed to be an input parameter MAT-72606 ApprAl, Spring 2017 22-Mar-17 163 25
We didn t really need the schedule for the long jobs to be optimal We used the optimality of the schedule only when the last job to finish was a long job If we had found a schedule for the long jobs that had makespan at most 1+ times the optimal value, then that would have been sufficient We will see how to obtain this near-optimal schedule for long jobs by rounding input sizes and dynamic programming, as previously on the knapsack problem MAT-72606 ApprAl, Spring 2017 22-Mar-17 164 Let us first set a target length for the schedule As before, we also fix a positive integer ; we will design a family of algorithms { } where either proves that no schedule of length exists, or else finds a schedule of length 1+ Later we will show how such a family of algorithms also implies the existence of a polynomial-time approximation scheme We can assume that otherwise no feasible schedule exists, since MAT-72606 ApprAl, Spring 2017 22-Mar-17 165 26
The algorithm is quite simple Partition the jobs into long and short, but require that > for to be long We round down the processing requirement of each long job to its nearest multiple of / We will determine in polynomial time whether or not there is a feasible schedule for these rounded long jobs that completes within time If yes, we interpret it as a schedule for the long jobs with their original processing requirements If not, we conclude that no feasible schedule of length exists for the original input MAT-72606 ApprAl, Spring 2017 22-Mar-17 166 Finally, we extend this schedule to include the short jobs by using the list scheduling algorithm We need to prove that the algorithm always produces a schedule of length at most 1+ whenever there exists a schedule of length at most When the original input has a schedule of length, then so does the reduced input consisting only of the rounded long jobs (which is why we rounded down the processing requirements); in this case, the algorithm does compute a schedule for the original input MAT-72606 ApprAl, Spring 2017 22-Mar-17 167 27
Suppose that a schedule is found It starts with a schedule of length at most for the rounded long jobs Let be the set of jobs assigned by this schedule to one machine Since each job in is long, and hence has rounded size at least /, it follows that Furthermore, for each job, the difference between its true processing requirement and its rounded one is at most / (because rounded value is a multiple of / ) + = 1 + 1 MAT-72606 ApprAl, Spring 2017 22-Mar-17 168 Now consider the effect of assigning the short jobs: each job, in turn, is assigned to a machine for which the current load is smallest Since, also Since the average load assigned to a machine is less than, there must exist a machine that is currently assigned jobs of total processing requirement less than So, when we choose the machine that currently has the lightest load, and then add job, this machine s new load is at most + < + = 1+ 1 MAT-72606 ApprAl, Spring 2017 22-Mar-17 169 28
Hence, the schedule produced by list scheduling will also be of length at most 1+1 To complete, we must still show that we can use dynamic programming to decide if there is a schedule of length for the rounded long jobs Clearly if there is a rounded long job of size greater than, then there is no such schedule Otherwise, describe an input by a -dimensional vector, where the th component specifies the # of long jobs of rounded size equal to /, = 1,, We know that for <, there are no such jobs, since that would imply that their original processing requirement was less than /, and hence not long MAT-72606 ApprAl, Spring 2017 22-Mar-17 170 So there are at most distinct inputs a polynomial number! How many distinct ways are there to feasibly assign long jobs to one machine? Each rounded long job still has processing time at least / at most jobs are assigned to one machine Again, an assignment to one machine can be described by a -dimensional vector, where again the th component specifies the number of long jobs of rounded size equal to / that are assigned to that machine MAT-72606 ApprAl, Spring 2017 22-Mar-17 171 29
Consider the vector (,,, ); we call it a machine configuration if / Let be the set of all machine configurations Note that there are at most ( + 1) distinct configurations, since each machine must process a number of rounded long jobs that is in the set {0,1,,} Since is fixed, this means that there are a constant number of configurations MAT-72606 ApprAl, Spring 2017 22-Mar-17 172 Let OPT(,, ) be the min # of machines sufficient to schedule this arbitrary input This value is given by the following recurrence (assign some jobs to one machine, and then using as few machines as possible for the rest): OPT,, =1+ min (,,, OPT,, View as a table with a polynomial number of entries (one for each possible input type) To compute each entry, find the minimum over a constant number of previously computed values The desired schedule exists exactly when the corresponding optimal value is at most MAT-72606 ApprAl, Spring 2017 22-Mar-17 173 30
Finally, one can convert the family of algorithms { } into a PTAS Theorem 3.7: There is a PTAS for the problem of minimizing the makespan on an input number of identical parallel machines. Note that since we consider ( + 1) configurations and 1/, the running time in the worst case is exponential in (1/ ) Thus, in this case, we did not obtain a FPTAS (in contrast to the knapsack problem) MAT-72606 ApprAl, Spring 2017 22-Mar-17 174 This is for a fundamental reason This scheduling problem is strongly NPcomplete Even if we require that the processing times be restricted to values at most (), a polynomial function of the number of jobs, this special case is still NP-complete If a FPTAS exists for this problem, it could be used to solve this special case in polynomial time, which would imply that P = NP MAT-72606 ApprAl, Spring 2017 22-Mar-17 175 31