Algorithms and Experimental Study for the Traveling Salesman Problem of Second Order Gerold Jäger joint work with Paul Molitor University Halle-Wittenberg, Germany August 22, 2008
Overview 1 Introduction Traveling Salesman Problem of Second Order Applications Related Problems Complexity Results 2 Heuristics for the Traveling Salesman Problem of Second Order Cheapest-Insert Algorithm Nearest-Neighbor Algorithm Two-Directional-Nearest-Neighbor Algorithm General-Assignment-Patching Algorithm k-opt Algorithm
Overview 3 Exact Algorithms for the Traveling Salesman Problem of Second Order Branch-and-Bound Algorithm Integer-Programming Algorithm 4 Experimental Comparison of Algorithms Experiments Experimental Comparison of Heuristics Experimental Comparison of Exact Algorithms
Introduction Traveling Salesman Problem of Second Order Traveling Salesman Problem of Second Order (2-TSP) Input: Output: Complete directed graph with n vertices v 1,..., v n. Three-dimensional cost matrix C = (c ijk ) R n,n,n with costs c ijk of a sequence of vertices (v i, v j, v k ). Complete tour T = (v i1,..., v in, v i1 ), Tour costs c(t ) = c in 1,i n,i 1 + c in,i 1,i 2 + n 2 l=1 c i l,i l+1,i l+2 are minimum.
Introduction Applications Bioinformatics: Models for transcription factor binding sites in gene regulation. Two models are the Permuted Markow (PM) Model and the Permuted Variable Length Markow (PVLM) Model. Solving order 1 of the PM/PVLM Model requires the solution of an ATSP instance. Solving order 2 of the PM/PVLM Model requires the solution of a 2-TSP instance.
Introduction Related Problems Asymmetric Traveling Salesman Problem (ATSP) Input: Output: Complete directed graph with n vertices v 1,..., v n. Cost matrix C = (c ij ) R n,n with costs c ij for an arc (v i, v j ). Complete tour T = (v i1,..., v in, v i1 ), Tour costs c(t ) = c in,i 1 + n 1 l=1 c i l,i l+1 are minimum.
Introduction Related Problems Assignment Problem (AP) Input: Output: Complete directed graph with n vertices v 1,..., v n. Cost matrix C = (c ij ) R n,n with costs c ij for an arc (v i, v j ). Set of cycles Z 1,..., Z k, where each vertex is visited exactly once. Sum of costs of all cycles c(t ) = c(z 1 ) + + c(z k ) is minimum, where cycle Z j = (v 1,..., v t ) has costs c(z j ) = c it,i 1 + t 1 l=1 c i l,i l+1.
Introduction Related Problems Assignment Problem of Second Order (2-AP) Input: Output: Complete directed graph with n vertices v 1,..., v n. Three-dimensional cost matrix C = (c ijk ) R n,n,n with costs c ijk of a sequence of vertices (v i, v j, v k ). Set of cycles Z 1,..., Z k, where each vertex is visited exactly once. Sum of costs of all cycles c(t ) = c(z 1 ) + + c(z k ) is minimum, where cycle Z j = (v 1,..., v t ) has costs c(z j ) = c it 1,i t,i 1 + c it,i 1,i 2 + t 2 l=1 c i l,i l+1,i l+2.
Introduction Complexity Results ATSP is NP-hard. [Karp, 1972] 2-TSP is NP-hard. Reduction from ATSP. AP is solvable in O(n 3 ). Hungarian Method [Kuhn, 1955] 2-AP is NP-hard. [Fischer, Lau, 2008]
Heuristics for the Traveling Salesman Problem of Second Order Eight heuristics are described in the paper. In the following five of them are presented.
Heuristics for the Traveling Salesman Problem of Second Order Cheapest-Insert Algorithm Generalization of an ATSP Algorithm of [Rosenkrantz, Stearns, Lewis, 1977]. Start with a good arc e = (v, w), and consider the subtour T := (v, w, v). Insert a good new vertex in T at a good place of the tour T. Repeat the last step, until T contains all vertices.
Heuristics for the Traveling Salesman Problem of Second Order Nearest-Neighbor Algorithm Generalization of an ATSP Algorithm of [Rosenkrantz, Stearns, Lewis, 1977]. Start with a good arc e = (v, w), and consider the path P = (v, w). Add a good new vertex to the tail of P, until P contains all vertices. Repeat the last step, until P contains all vertices. Close the path P, and receive a tour T.
Heuristics for the Traveling Salesman Problem of Second Order Two-Directional-Nearest-Neighbor Algorithm Generalization of the Nearest-Neighbor Algorithm. Vertices are added in both directions. At each step the direction is added in such a way that the difference between the second-best and the best vertex in each direction is larger, not from the best vertex over both directions.
Heuristics for the Traveling Salesman Problem of Second Order General-Assignment-Patching Algorithm Generalization of an ATSP Algorithm of [Karp, Steele, 1985]. Start with a good set of cycles Z 1,..., Z k, such that each vertex is visited exactly once. Which set is chosen? The set of cycles of a 2-AP solution is the set with the smallest costs. Problem: 2-AP is NP-hard. Idea: Compute a lower bound for the solution of the 2-AP instance by the solution of a corresponding AP instance. Solvable in O(n 3 ).
Heuristics for the Traveling Salesman Problem of Second Order General-Assignment-Patching Algorithm Choose two cycles Z 1 and Z 2 with the most vertices. Replace two arcs (v 1, w 1 ) Z 1 and (v 2, w 2 ) Z 2 by (v 1, w 2 ) and (v 2, w 1 ).
Heuristics for the Traveling Salesman Problem of Second Order General-Assignment-Patching Algorithm The arcs are chosen in such a way, that (v 1, w 1 ), (v 2, w 2 ) is as bad as possible and (v 1, w 2 ), (v 2, w 1 ) is as good as possible. Receive from Z 1 and Z 2 a new cycle Z 3. Repeat these steps, until you have only one cycle, which is a tour.
Heuristics for the Traveling Salesman Problem of Second Order k-opt Algorithm Generalization of an ATSP Algorithm of [Lin, Kernighan, 1973]. All previously presented algorithms are construction heuristics, i.e., they construct a tour. The k-opt Algorithm is an improvement heuristic, i.e., it starts with a tour (received by a construction heuristic) and improves it. A k-opt step changes a given tour T in such a way, that k tour arcs are replaced by k non-tour arcs and the set of arcs after the change is still a tour. The k-opt Algorithm applies tour improving k-opt steps with r k to a starting tour as long as such steps exist.
Exact Algorithms for the Traveling Salesman Problem of Second Order Branch-and-Bound Algorithm Compute with a heuristic a good upper bound for the instance. Go through all (n 1)! possible tours in lexicographical order. If an improvement occurs, the current optimum tour and the upper bound are updated. Go through all subpaths, first through subpaths with smaller number of vertices. Contract the subpath to a vertex. Compute the corresponding approximated 2-AP solution to the subpath. This solutions gives a local lower bound.
Exact Algorithms for the Traveling Salesman Problem of Second Order Branch-and-Bound Algorithm If the local lower bound of a subpath is not smaller than the current upper bound, all tours and subpaths, starting with this subpath, can be omitted from consideration.
Exact Algorithms for the Traveling Salesman Problem of Second Order Integer-Programming Algorithm 2-AP can be solved by the following IP model: n n n min c ijk x ijk (1) i=1,i j,k j=1,j k k=1 x ijk {0; 1} 1 i, j, k n, i j, i k, j k (2) n n x ijk = 1 1 k n (3) i=1,i j,k j=1,j k n n i=1,i j,k k=1,k j n n j=1,j i,k k=1,k j x ijk = 1 1 j n (4) x ijk = 1 1 i n (5)
Exact Algorithms for the Traveling Salesman Problem of Second Order Integer-Programming Algorithm n k=1,k i,k j x ijk = k=1,k i,k j x kij 1 i j n (6) For solving not only the 2-AP, but also the 2-TSP, all possible subtours have to be forbidden. For the subtour (v s1,..., v st, v s1 ) the following inequality has to be added: t 2 x st 1,s t,s 1 + x st,s 1,s 2 + x si,s i+1,s i+2 t 1 (7) i=1 Unfortunately an exponential number of such subtours exists.
Exact Algorithms for the Traveling Salesman Problem of Second Order Integer-Programming Algorithm Therefore at each step only the necessary subtour inequalities are added: 1 Define IP by the conditions (1), (2), (3), (4), (5), (6). 2 Solve IP. 3 Receive a solution with k cycles Z 1,..., Z k and cycle Z = (v s1,..., v st, v s1 ) with the minimum number of vertices t. 4 IF k = 1 5 THEN TERMINATE with solution Z. 6 ELSE Add condition (7) for cycle Z to IP. 7 GOTO 2. If an IP solver (as CPLEX) is started with a good upper bound (received by a heuristic), the solver can be accelerated.
Experimental Comparison of Algorithms Experiments We compare all basic heuristics and all basic heuristics followed by a 5-OPT step in quality (upper bound) and time. We compare the upper bound of all basic and OPT heuristics with the optima. We compare all exact algorithms in time. All experiments were done for two classes: real and random instances.
Experimental Comparison of Algorithms Experimental Comparison of Heuristics The times of the OPT versions are much larger than for the basic versions. The times of all basic versions are similar. The times of all OPT versions are similar. The quality of the OPT versions is much better than for the basic versions. The quality results for the random and real instances completely differ: For the real instances, the OPT Cheapest-Insert Algorithm and the OPT General Assignment Patching Algorithm are the best algorithms. For the random instances, the OPT Two-Directional- Nearest-Neighbor Algorithm is the best algorithm.
Experimental Comparison of Algorithms Experimental Comparison of Heuristics Real instances: for 44 of 45 instances at least one (of seven) OPT heuristics finds the optimum. Random instances: the upper bounds are more far away from the optimum.
Experimental Comparison of Algorithms Experimental Comparison of Exact Algorithms Random instances can be solved faster than real instances. Random instances and real instances of smaller dimensions: BnB Algorithm faster. Real instances of larger dimensions (21 and 26 ): Only IP-Algorithm successful.
Experimental Comparison of Algorithms Experimental Comparison of Exact Algorithms Thanks for your attention!