A Theoretical Framework to Solve the TSPs as Classification Problems and Shortest Hamiltonian Path Problems

Transcription

1 American Journal of Intelligent Systems 2014, 4(1): 1-8 DOI: /j.ajis A Theoretical Framework to Solve the TSPs as Classification Problems and Shortest Hamiltonian Path Problems Akihiko Kawashima 1,*, Yasuo Sugai 2 1 Graduate School of Engineering, Nagoya University, Nagoya, Japan 2 Graduate School of Engineering, Chiba University, Chiba, Japan Abstract In the study of Traveling Salesman Problem (TSP), a main theme is the reduction of computational cost. This reduction needs the intelligence which is heuristics or the theoretical knowledge such as necessary conditions for optimality. If TSP can be manipulated as classification problems, the analysis of the classifier, which can solve efficiently these classification problems, provides the required intelligence. In fact, we can replace the TSP with a classification problem and some Shortest Hamiltonian Path Problems (SHPPs) which also can be replaced with these sub problems in the same way. Hence, the TSPs are equivalent to the classification problems. In this paper, we propose the theoretical framework to solve the TSPs as classification problems and SHPPs. We show some especial conditions in the SHPPs as sub problems converted from the TSPs, and the specification for the framework in future works. Keywords Traveling salesman problem, Shortest Hamiltonian path problem, Classifier, Optimality, Machine learning 1. Introduction Many methods have been proposed in Traveling Salesman Problem (TSP)[1-3]. Concorde[4] is well known as a complete solver and had calculated the optima for all of the TSPLIB instances[5] which are the benchmark set for TSP. Generally, the computational cost increases dramatically with increasing the number of cities. Therefore, approximate methods are necessary in engineering or industrial domains. LKH proposed by Keld Helsgaun[6],[7] is the best approximate method[8] improved from Lin-Kernighan method[9]. This method obtained the best tour of the TSPLIB instance pla85900, and this solution was proved as the optimal tour in 2009[10]. Concorde and LKH are well known as effective implementations which support parallel computing although the setting of many parameters is needed for each instance. The intelligence, which affects the priority in search space to obtain the optimal or sub-optimal solution, could improve almost all of the existing methods at the point of calculation cost[11]. This intelligence is the theoretical knowledge such as the necessary conditions for optimality or the experience which have been taken in solving many instances. By the progress in calculation cost, the existing * Corresponding author: a_kawashima@nuem.nagoya-u.ac.jp (Akihiko Kawashima) Published online at Copyright 2014 Scientific & Academic Publishing. All Rights Reserved methods are able to solve more large size instances of TSP. In resent research, a measure of difficulty for 2-opt exchanges in improvement of solutions is pointed out as the important features that make TSP instances hard or easy to be approximated by 2-opt exchanges[12]. Nevertheless it is only useful for methods with 2-opt and the feature is not based on the optimality. In our study, we consider that machine learning systems can be applied to the TSPs and acquire the intelligence what features are useful for leaning through the study of machine learning for the TSPs, if the TSP instances can be managed as classification problems. In fact, with the necessary conditions for optimality, we have achieved the replacement of the TSPs in Euclidean plane with a classification problem and some shortest Hamiltonian path problems (SHPPs) which also can be replaced with a classification problem and some SHPPs in same way. Consequently, the TSPs in Euclidean plane can be replaced with only the classification problems. In addition to, the learning data for classifiers in machine learning can be generated with the optimal tours of which the instances have been solved completely such as TSPLIB[13]. In this paper, we propose a theoretical framework to solve the TSPs in Euclidean plane as a classification problem and some SHPPs. Since the intelligence for the classification problems is useful for the TSPs whether the machine learning systems are applied to, we show the specification for the framework in not only machine learning systems but also the other several approaches.

2 2 Akihiko Kawashima et al.: A Theoretical Framework to Solve the TSPs as Classification Problems and Shortest Hamiltonian Path Problems 2. Traveling Salesman Problem 2.1. Description of TSP The TSP as a NP-hard problem is to obtain a minimum cost tour, where a tour is a cycle that each city of the city set V given in an instance is visited exactly once, and this constraint of tours is expressed as S. A tour X in S is expressed as follow, X = ( v1,, vi,, vn, v1). (1) Each element v i is the i-th visiting city in a tour X, and N is the number of cities given in an instance. The total cost E(X) of X is shown as follow, N EX ( ) = dv ( i, v i + 1), (2) i where d(v i,v i+1 ) is the cost of the edge e(v i,v i+1 ) between the v i and the v i+1 (i=1,,n, v N+1 =v 1 ). The optimal tour X * is shown as follow, * X = arg min EX ( ). (3) X S In Euclidean plane, the edge costs are defined as follow, 2 2 i i 1 vi vi+ 1 vi vi+ 1 dv (, v + ) = ( x x ) + ( y y ), (4) where x vi and y vi are the coordinates of a city v i Necessary Conditions for Optimality Two necessary conditions that a tour is optimal in the Euclidean plane TSPs were proved; Flood[14] had proven that the optimal tour does not intersect itself, and Gonzales[15] had proven that the visiting order of the cities on convex hull in the optimal tour is the same as the order in which these cities appear on convex hull. Figure 1 shows the case of the instance berlin52 as an example for the second necessary condition for optimality. Figure 1. An example for the second necessary condition. The numbers in this figure are the city labels on convex hull. The order which the cities are visited on the convex hull (broken line) in the optimal tour (solid line) is , and the order which the cities appear in the convex hull is also Both orders are equal to each other 2.3. Description of Shortest Hamiltonian Path Problem A SHPP is to search a minimum cost path between the starting city v s and the terminating city v t visiting for each city v in the given city set V exactly once, and this constraint for the paths is expressed as S. A SHPP can be solved as a standard TSP with the special edge cost between the v s and the v t [2]. A solution path P vs,vt in S from the v s to the v t is expressed as follow, P = ( v,, v,, v ) ( v = v, v = v ).(5) v s, v t 1 i N 1 s N t 3. Theory of Proposed Framework 3.1. Alternative Description of TSP Because the second necessary condition leads the optimal visiting order of the cities on convex hull, the optimal tour can be divided into some paths at the cities on convex hull with optimality. Such a city on convex hull C is called a terminal t. When the number of terminals is m, the number of paths is also m. The optimal tour X * can be replaced with the ordered terminals t j in C (j=1,,m) as follow, * X = ( t1,, tj,, tm,, t1). (6) Hence, to obtain a tour we must decide all paths from t j to t j+1 (j=1,,m, t m+1 =t 1 ). Then, at first, we should classify the cities v not in C into m subsets V j (j=1,,m) which construct sub problems as SHPPs through this classification problem as a sub problem. A subset V j includes the terminals t j and t j+1. Secondly, we should solve each SHPP as sub problems to obtain the required paths P tj,tj+1 from the starting city t j to the terminating city t j+1 in V tj (j=1,,m, j m+1 =j 1 ). The V tj and P tj,tj+1 are shown as follows, Vt = { tj,, t j + 1), (7) j P = ( t,, t ), (8) t j, t j+ 1 j j + 1 where * is a meta character as a wild card to be replaced with classified cities. That is, a TSP in Euclidean plane can be replaced with a classification problem and m SHPPs. The equation (2) can be represented as follow, m EX ( ) = EP ( t, P ). j t (9) j + 1 j Then, a relation holds as follow, m min EX ( ) min EP ( j, P j + 1). (10) j The equal sign holds when the classifications of the cities not on convex hull are optimal Replaced SHPPs with Special Conditions The SHPPs which are converted from the TSPs have the

3 American Journal of Intelligent Systems 2014, 4(1): two special conditions; (1) the starting city v s and the terminating city v t are neighbouring on convex hull C, and (2) the cities are distributed only in one side of the straight line connecting the v s and the v t. And it is clear that the terminals t j in the optimal path are also visited with the appearing order on the convex hull C. That is, the necessary conditions for optimality can be applied to the SHPPs in the same way for the TSPs. Furthermore, a SHPP also can be replaced with a classification problem and m SHPPs. Figure 2 shows the optimal path and the convex hull of a SHPP as a sub problem of the instance berlin52 in TSPLIB. This figure indicates that the second necessary condition is also kept in the SHPPs. the TSPLIB instance berlin52 when the instance is replaced only with the classification problems. The variable d is the depth in tree structure described in the next section. After the calculation of convex hull, the cities not on convex hull are classified into some subsets as SHPPs. And the convex hulls for each subset are also calculated. Finally, a solution tour is constructed through the process. Figure 2. A SHPP as a sub problem of berlin52 in TSPLIB. Solid lines are the optimal path from the starting city 2 to the terminating city 14 in this sub problem, and broken lines are the convex hull of this city set As stated above, the SHPPs on our framework have the special condition in contrast with the standard SHPPs. The starting city v s and the terminating city v t are inevitably v s =t 1 and v t =t m with the m terminals on the convex hull C. According to this condition, the SHPPs seem to be easier than the original TSPs Our Framework for the TSPs in Euclidean Plane As shown in Figure 3, while repeating the replacements for each SHPP until the sub problems include only two cities, after all, the TSPs in Euclidean plane can be replaced only with the classification problems. Figure 4. The optimal classifications as the convex hulls of berlin52 in TSPLIB. The solid lines are the edges of convex hulls, and the broken lines are the edges replaced with some paths as solutions of SHPPs 3.4. Tree Structure as a Tour As our framework has the hierarchical process, the constructed tours through our procedure can be expressed in tree structure. In addition to, the tours, which have no intersect, are also expressed in tree structure by reverse calculation with our framework. For example, Figure 5 shows the tree structure of the optimal tour for the instance berlin52 in TSPLIB. The depth of the root node is d=0. This tree has maximal depth d=5 and all of the internal nodes as SHPPs corresponds the convex hulls in Figure 4 for each depth. In the tree structures, a tour is constructed by the leaf nodes with following in depth-first search. 4. Specifications for Our Framework Figure 3. The proposed framework for the TSPs in Euclidean plane Thus, the substance in our framework is the classifier for the cities v i not in C, and the quality of solutions depends on the error in the classifications. Alternatively, depending on the number or the distribution of the cities in the SHPPs, we should directly solve the SHPPs without the replacement. For example, Figure 4 shows the optimal classifications of 4.1. Implementation for Construction Methods For the usage as construction methods, our framework needs only the setting of the classifier which the cities not on convex hull are classified into sub sets. We implement a pseudo code for construction methods on our framework in Figure 6 and describe the variables and the explanations of the pseudo code in this following. N: the number of the cities in an instance.

4 4 Akihiko Kawashima et al.: A Theoretical Framework to Solve the TSPs as Classification Problems and Shortest Hamiltonian Path Problems V[]: the array for the city set, each element includes members x, y, label and key. V[i].x: the coordinate x of a city V[i]. V[i].y: the coordinate y of a city V[i]. V[i].label: the label of a city V[i]. V[i].key: be used for the classification in sorting. C[]: the array for the sub problems, each elements include members tail, head, size, depth, and parent. C[i].tail: the starting index in V[]. C[i].head: the terminating index in V[]. C[i].size: the number of cities for each sub problem. C[i].depth: the depth in tree structure. C[i].parent: the parent problem for each sub problem. num: the number of the sub problems. start: the label of the starting city. m: the number of the terminals on convex hull. a: the number of generated sub problems. p and q: the temporal variables for the range in V[]. i, j and k: the iterators for loops. Figure 5. The tree structure of the optimal tour of berlin52 in TSPLIB. Open circles are internal nodes in tree structuzre. The root node in the left side means a TSP and the other open circles mean SHPPs. Solid circles are leaf nodes that mean SHPPs which have only two cities. The leaf nodes are equal to the edges constructing a tour 0. Lines 0-9: initialize the valuables, where the first problem C[1] is used for the original TSP. 1. Lines 11-15: read the information of a problem C[i]. 2. Line 16: check the number of cities of C[i]. If the number is two, do not need calculate for the sub problem C[i]. 3. Line 17: call the function for calculating the convex hull of the C[i]. This function returns the ordered terminates t j (j=1,,m) in appearing on convex hull in V[p..p+m-1]. We have implemented the function with Graham scan[16] which is suited to obtain the appearing order of the cities on convex hull. 4. Line 18: adjust the order of the terminals so that the starting city is at V[p] and the terminating city is at V[p+m-1]. With the variable start which records the label of the starting city, we can obtain the correct order of the terminals. 5. Lines 19-22: regulate a; the a is equal to m in the depth d=0, or equal to (m-1) in the depth greater than zero, because these sub problem do not use the path connected directly from v s to v t in the replaced SHPPs. 6. Lines 23-25: terminals t j are labelled with j. 7. Lines 26-28: the cities v not in C are classify into a subsets by labelling with the return number of the function for classification. The return number is the label of the generated subsets for the sub problems. 8. Line 29: the cities V[p..q] are sorted with their key in ascending order. After this sorting, the cities are gathered for each sub problem. 9. Lines 30-31: update the information of the generated sub problems C[num+1..num+a] by reading the V[p..q].key, where tail, head, size, depth and parent are calculated. 10. Line 32: update the num incremented by a. 11. Line 33: increment the iterator i. 12. Lines 16-34: this procedure is looping while the iterator i is less than and equal to num. In this following, we describe the time and space complexity of the procedures working on our framework. At first, we define variables; V i is a city set which has n i cities, m i is the number of the cities on convex hull C i, and g(n i ) is the computational cost in classification, where i is the iterator in the procedure. Then, we estimate the calculation cost for a loop in the next items, 1. Read the problem information: w Check the number of cities: w Calculate the convex hull: w 3 n i log n i. 4. Adjust the order of the terminals: w 4 m i. 5. Label the terminals: w 5 m i 6. Classify the cities not on convex hull: w 6 g(n i ) 7. Sort the cities: w 7 n i log n i. 8. Update the cluster information: w 8 m i. Hence, the calculation cost f(n i ) for the i-th loop is expressed as follow, f( ni) = w1+ w2 + wn 3 ilog ni + wm 4 i + wm 5 i + wg 6 ( ni) + wn 7 ilog ni + wm 8 i, (11) = wgn 6 ( i) + ( w3+ w7) nilog ni + ( w4 + w5 + w8) mi + ( w1+ w2), where w i is the suitable coefficients in computation for each procedure. If the cost g(n i ) is equal to or less than n i log n i,

5 American Journal of Intelligent Systems 2014, 4(1): the f(n i ) is expressed with a coefficient w as follow, f ( n ) wn log n. (12) i i i Otherwise, if the cost g(n i ) is greater than n i log n i, the f(n i ) is represented as follow, f ( n ) wg( n ). (13) i Then, by summation of (12) or (13) for all i, all of the calculation costs f(n) is represented as follow, i= 1 Replacement-Procedure (V, C, N) i H f( N) = w gn ( ), (14) 1: C[].size 0 2: C[1].tail 1 3: C[1].head N 4: C[1].size N 5: C[1].depth 0 6: C[1].parent NIL 7: V[N+1] V[0] 8: num 1 9: i 1 10: while i num do 11: p C[i].tail 12: q C[i].head 13: n C[i].size 14: d C[i].depth 15: start V[p].label 16: if n 2 then 17: m Convex-Hull (V, p, q) 18: Adjust the order V[p..p+m-1] correctly. 19: a m 20: if d 1 then 21: a a-1 22: end if 23: for j 1 to m do 24: key[p+j-1] num + j 25: end for 26: for j m+1 to n do 27: key[p+j-1] Classify(V, p, j, C, num, a) 28: end for 29: Sorting (V, p, q) 30: Update C[num+1..num+a] 31: num num + a 32: end if 33: i i+1 34:end while Figure 6. A pseudo code for constructing methods on our framework where H is the total number of the clusters which excludes the two cities SHPPs as leaf nodes in tree structure because these problems do not need to be computed. The relation between H and all of the clusters M is shown as follow, i M = H + N. (15) To estimate the range of H, with the tree structure for TSP tours in our framework, we consider the influence when the sub sets, which the cities not on convex hull are classified into, are changed. These changes could be substituted for the moving of leaf nodes in the tree structure. At first, it is clear that each internal node inevitably has two or more child nodes except the root node which inevitably has three or more child nodes, because a TSP must have at least three cities for constructing a tour and also a SHPP must have at least two cities for constructing a path. In Figure 7, the movement of a leaf node u from the state (a) to the state (a) leads taking an increase of internal node at w with the shift of a leaf node from the place w to the place w, while the movement of a leaf node v leads taking an decrease of internal node with the shift of the leaf node from the place v to the place v. Figure 7. The numerical changes of internal nodes If the internal node at the v has over three children, the movement of a leaf node take only an increase of internal node without a decrease. From the state (a) to the state (a), the movement of the internal node u take only a decrease of internal node. Consequently, When the root node has all of the leaf nodes, the number of internal nodes is minimum as H=1, and when the root node has three child nodes and all of the internal nodes has two child nodes, which a child node is an internal node and the other is a leaf node, the number of internal nodes is maximum as H=N-2. Therefore, the range of H is shown as follow, 1 H N 2. (16) And the calculation cost f(n) is represented with big O notation as follow,

6 6 Akihiko Kawashima et al.: A Theoretical Framework to Solve the TSPs as Classification Problems and Shortest Hamiltonian Path Problems ON ( log N) f( N) ONgN ( ( )). (17) In addition, since the space needs O(N) without the procedure in classification, the space complexity is O(N) when the classification system requires the space which equal to or less than O(N). If the calculation costs for each classifications follow the equation (12), the time complexity is O(Nlog 2 N) and the space complexity is O(N). These orders appear when the optimal classifications as learning data in machine learning are generated with the optimal tour[12] For Improvement Methods After the running through our procedure, the solution tour could be improving with reclassifications of cities. We describe this idea with Figure 8 in this following. 5. Results and Discussion 5.1. As a Construction Method with a Simple Classifier When we apply the simple function to the classifier, our framework will operate similar to classical methods. For example, we consider that the following equation is applied to the classifier, arg min{ dv ( i, tj) + dv ( i, tj+ 1) dt ( j, tj+ 1)}. (18) j This equation means the increasing cost replacing the edge e(t j, t j+1 ) with the two edges e(v i, t j ) and e(v i, t j+1 ) which incident to the city v i not in C. In this case, the time complexity is O(Nlog 2 N) and the space complexity is O(N). The solutions with this classifier have percent error in the EUC_2D instances of TSPLIB Generated SHPP Benchmarks The learning data for machine learning can be generated with the optimal tours of which the instances had been solved completely such as the EUC_2D TSPLIB instances. Accordingly, the optimal classifications can be regarded as the benchmark instances of the SHPPs. In Figure 9, we show that how many instances we can generate as the SHPP benchmarks with the TSPLIB instances. The x-axis indicates the number of cities as SHPP instances in log scale, and the y-axis indicates the frequency of the SHPP problems in log scale. There are many instances of the SHPPs enough to study the classifier on our framework. Figure 8. An image for improvement procedure on our framework If a city or some cities are reclassified from a subset to another subset, the recalculations are needed for the two sub trees which are associated with this movement. The root nodes of these sub trees having the same parent at the same depth in tree structure. The tour improvement should be performed in such way while comparing the tour costs. Since the improving process is the recalculations for some sub trees, our framework could be applied to the methods which have the procedure depends on tree structure such as Genetic Programming (GP)[17],[18]. Insofar as we know, however, there is any method with GP for the TSPs though the many constraints will be required. If we achieve the construction of an effective method with GP, new intelligence may be given for the TSP For Parallel Computing Because each sub problem independents of the other sub problems in the same level of tree structure, for the candidate patterns of classifications or for each sub problem, we could apply parallel computing to our framework easily though the implementation needs the experience in those technique. Figure 9. The distribution of the number which the TSPLIB instances can generate the SHPP instances However, there is only slight knowledge and experience in constructing of machine learning system for the classifications. Thus, we should begin the training with small size SHPPs through trial and error. At the beginning, as the first goal in our study, we aim that a classifier trained in machine learning achieves the optimal classification in small size instances of the SHPPs About the Optimal Classification Although we can construct a classifier in machine learning systems only with the geometric information such

7 American Journal of Intelligent Systems 2014, 4(1): as the coordinates of cities, however, through generating the learning data for the TSPLIB instances with the optimal tours, the optimal classifications provide us the knowledge that the convex hulls generated as the optimal classifications are frequently overlapping each other in geometry. These overlaps appear in the instances which have over 100 cities. Figure 10 shows an example of the overlapping in optimal classification for the instance d15112 of TSPLIB. These convex hulls are at the depth d=1 in tree structure. There are 21 internal nodes as convex hulls and 2 leaf nodes as edges. This matter exhibits that only the simple geometric information such as the equation (18) cannot generate the optimal classifications, and that the topological information such as a minimum spanning tree will be essential to realize the optimal classifications. Consequently, for the classifiers in machine learning, we must use quantity of many various features not only the geometric information but also the topological information through trial and error. in the same way for the TSPs, which leads that the TSPs are replaced with only classification problems. If the classifications are optimal, the solution is also optimal. The performance of the proposed framework depends on the applied classifier in quality and complexity. We conjecture easily that the classification problems are very difficult to be solved with the simple classifiers, because the optimal classifications are overlapping in the instances which have over 100 cities. Therefore, the main theme of the proposed framework is the study of the classifiers. We showed the specifications as several approaches with the framework for the TSPs. And with the TSPLIB instance, we can generate many SHPP instances of which the optimal solutions can be obtained. Through the study of the classifiers for the SHPPs from small size instances to large size instances, we expect the achievement to acquire the effective intelligence for the TSPs with our framework in future work. REFERENCES [1] E. L. Lawler, J. K. Lenstra, A. H. G. Rinnooy Kan and D. B. Shmoys, Ed, "THE TRAVELING SALESMAN PROBLEM", John Wiley & Sons, [2] Gerhard Reinelt, "The Traveling Salesman: Computational Solutions for TSP Applications", Berlin Heidelberg, Springer-Varlag, [3] G. Gutin and A. P. Punnen, Ed, "The Traveling Salesman Problem and Its Variations", Kluwer Academic Publishers, [4] Concorde, Online Available: sp/concorde/index.html. [5] TSPLIB95, Online Available: g.de/software/tsplib95/index.html. [6] K. Helsgaun, "An Effective Implementation of the Lin-Kernighan Traveling Salesman Heuristic," European Journal of Operational Research 126(1), pp , Figure 10. The overlapping of the convex hulls at the depth d=1 in the optimal classifications for the TSPLIB instance d Conclusions We have proposed the theoretical framework to solve the TSPs in Euclidean plane as a classification problem and some SHPPs. The necessary conditions or optimality can replace the TSPs with a classification problem and some SHPPs. The replaced SHPPs as sub problems have the two special conditions; (1) the starting city and the terminating city are neighbouring on convex hull and (2) the cities are distributed in one side of the straight line connecting the starting city and the terminating city. The SHPPs also can be replaced with a classification problem and some SHPPs [7] LKH2, Online Available: arch/lkh/index.html. [8] DIMACS TSP Challenge, Online Available: tgers.edu/challenges/tsp/ [9] S. Lin and B. W. Kernighan, "An effective heuristic algorithm for the traveling-salesman problem," Operations Research 21, pp , [10] D. Applegate, R. Bixby, V. Chvatal, W. Cook, D. Espinoza, M. Goycoolea and K. Helsgaun, "Certification of an optimal TSP tour through cities," Operations Research Letters 37, pp.11-15, [11] S. Eilon, C. D. T. Watson-Gandy and N. Christofides, "Distribution management: Mathematical modelling and practical analysis," London: Griffin, pp , [12] O. Mersmann, B. Bischi, J. Bossek, H. Trautmann, M. Wagner and F. Neumann, "Local Search and the Traveling

8 8 Akihiko Kawashima et al.: A Theoretical Framework to Solve the TSPs as Classification Problems and Shortest Hamiltonian Path Problems Salesman Problem: A Feature-Based Characterization of Problem Hardness," Learning and Intelligent Optimization, pp , [13] A. Kawashima and Y. Sugai, "A Generator of Learning Data for the TSPs Based on the Visiting Order of the Cities on Convex Hull," IEEE Multi-Conference on System and Control, FrC04-1, pp , [14] M. Flood, "The traveling salesman problem," Operations Research 4, pp.61-75, [15] R. H. Gonzales, "Solution to the traveling salesman problem by dynamic programming on the hypercube," Technical Reports No. 18, Operations Research Center, M.I.T., [16] R. L. Graham, "An Efficient Algorithm for Determining the Convex Hull of a Finite Planar Set," Information Processing Letters, vol.1, pp , [17] John R. Koza, "Hierarchical genetic algorithms operating on populations of computer programs," Proceedings of the Eleventh International Joint Conference on Artificial Intelligence IJCAI-89, pp , [18] J. R. Koza, "Genetic Programming: A Paradigm for Genetically Breeding Populations of Computer Programs to Solve Problems," Stanford University Computer Science Department technical report, STAN-CS , June 1990.