Combining Two Local Searches with Crossover: An Efficient Hybrid Algorithm for the Traveling Salesman Problem Weichen Liu, Thomas Weise, Yuezhong Wu and Qi Qi University of Science and Technology of Chine (USTC) and Institute of Applied Optimization (IAO) of Hefei University 2017-07-17 [2]
2 / 31 Outline 1 Research Content 2 Local Search Algorithm Lin-Kernighan Algorithm Ejection Chain Method Multi-Neighborhood Search 3 Crossover Operator Heuristic Crossover Operator Order Based Crossover Operator 4 Hybrid Algorithms for the TSP LS-LS-X Hybrids Hybrid Global Search with Local Search 5 Conclusion
3 / 31 Outline 1 Research Content 2 Local Search Algorithm 3 Crossover Operator 4 Hybrid Algorithms for the TSP 5 Conclusion
3 / 31 Traveling Salesman Problem Traveling Salesman Problem (TSP): Given a collection of n cities and the travel distance between them, solving a TSP means to find the shortest round-trip tour through all cities and back to the starting point.
4 / 31 Traveling Salesman Problem Given: A cost matrix D = (D i,j ), where D i,j is the cost of traveling from city i to j. Target: Find a permutation t of the integers from 1 to n minimizing the sum D t[1],t[2] +D t[2],t[3] + +D t[n],t[1]. In this paper, we focus on symmetric TSPs, where D i,j = D j,i holds. Prominent N P-hard problem in Combinatorial Optimization.
5 / 31 Experimentation Environment TSP Suite: A holistic benchmark environment for algorithms solving the TSP written in Java. It offers integrated support for implementing, testing, benchmarking and comparing algorithms. Benchmark: TSPLIB contains 110 symmetric TSP instances whose city scale is range from 14 to 85900.
Outline 1 Research Content 2 Local Search Algorithm Lin-Kernighan Algorithm Ejection Chain Method Multi-Neighborhood Search 3 Crossover Operator 4 Hybrid Algorithms for the TSP 5 Conclusion 6 / 31
Outline 1 Research Content 2 Local Search Algorithm Lin-Kernighan Algorithm Ejection Chain Method Multi-Neighborhood Search 3 Crossover Operator 4 Hybrid Algorithms for the TSP 5 Conclusion 6 / 31
6 / 31 Lin-Kernighan Algorithm The LK10 is an improved LK heuristic algorithm introduced in [5]. T6 T7 T9 T8 T4 T5 T2 T0 T1 T3
Outline 1 Research Content 2 Local Search Algorithm Lin-Kernighan Algorithm Ejection Chain Method Multi-Neighborhood Search 3 Crossover Operator 4 Hybrid Algorithms for the TSP 5 Conclusion 7 / 31
Ejection Chain Method FSM** is an improved Ejection Chain Method [1, 3]. It iteratively improves a stem-and-cycle reference structure (S&C) by applying two rules. t t t t q r r r r j s1 s2 s1 s2 s1 s2 s1 s2 q1 j q2 (a) S&C (b) Trial (c) Rule 1 (d) Rule 2 7 / 31
Outline 1 Research Content 2 Local Search Algorithm Lin-Kernighan Algorithm Ejection Chain Method Multi-Neighborhood Search 3 Crossover Operator 4 Hybrid Algorithms for the TSP 5 Conclusion 8 / 31
8 / 31 Multi-Neighborhood Search Multi-Neighborhood Search (MNS) is an efficient local search algorithm introduced in [4]. Rules Reversing Left-Rotation Right-Rotation Swap Improve Solution
9 / 31 Outline 1 Research Content 2 Local Search Algorithm 3 Crossover Operator Heuristic Crossover Operator Order Based Crossover Operator 4 Hybrid Algorithms for the TSP 5 Conclusion
9 / 31 Outline 1 Research Content 2 Local Search Algorithm 3 Crossover Operator Heuristic Crossover Operator Order Based Crossover Operator 4 Hybrid Algorithms for the TSP 5 Conclusion
Heuristic Crossover Operator 1 It first selects a random city as the current (starting) city of the offspring tour. 2 Second, it considers the four (directed) edges incident to the current city. Over these edges, a probability distribution is defined based on their cost. The probability associated with an edge incident to a previously visited city is equal to zero. 3 An edge is selected based on this distribution. If none of the parental edges leads to an unvisited city, a random edge is selected. 4 The step 2 and 3 are repeated until a complete tour has been constructed. 9 / 31
10 / 31 Outline 1 Research Content 2 Local Search Algorithm 3 Crossover Operator Heuristic Crossover Operator Order Based Crossover Operator 4 Hybrid Algorithms for the TSP 5 Conclusion
10 / 31 Order Based Crossover Operator The Order Based Crossover Operator (OX2) selects (at random) several positions in a parent tour and the order of the cities in the selected positions of this parent is imposed on the other parent. Parent 1 Parent 2 (1 2 3 4 5 6 7 8) (2 4 6 8 7 5 3 1) (1 2 3 * * * 7 8) (* 4 6 * * 5 * *) (1 2 3 * * * 7 8) (1 2 3 4 6 5 7 8)
11 / 31 Outline 1 Research Content 2 Local Search Algorithm 3 Crossover Operator 4 Hybrid Algorithms for the TSP LS-LS-X Hybrids Hybrid Global Search with Local Search 5 Conclusion
11 / 31 Outline 1 Research Content 2 Local Search Algorithm 3 Crossover Operator 4 Hybrid Algorithms for the TSP LS-LS-X Hybrids Hybrid Global Search with Local Search 5 Conclusion
11 / 31 Motivation Different local search algorithms have different features, use different data structures, and different search moves neighborhoods. Combining two different local searches means combining their different strengths. Crossover helps the search to escape from local optima, while retaining good building blocks.
LS-LS-X Hybrids Begin LS One Improved Y LS Two N End N Improved Y Crossover 12 / 31
13 / 31 Performance Measure and Time Normalized Runtime (NT): measured runtime normalized with machine- and problem-depending performance factor. Empirical cumulative distribution function (ECDF) returns the fraction of runs that have reached a given goal error F t (normally, F t = 0) for a time measure such as NT or FE. It is plotted over the runtime. The earlier and the higher the ECDF rises, the better is the algorithm.
LS algorithms Performance FSM**, LK10 and MNS algorithms 1.0 0.9 ECDF 0.8 0.7 0.6 0.5 0.4 0.3 0.2 FSM** LK10 MNS 0.1 0 2 4 6 log 10(NT) 8 (e) ECDF for NT and F t =0 14 / 31
LS-LS Hybrids Performance LS-LS hybrids FSM**-LK10 LK10-MNS 1.0 0.9 0.8 ECDF 0.7 0.6 0.5 0.4 0.3 0.2 0.1 FSM**-LK10 LK10-MNS FSM** LK10 MNS 0 2 4 6 log 10(NT) 8 (f) ECDF for NT and F t =0 15 / 31
LS-LS-X Hybrids Performance LS-LS-X hybrids LK10-MNS-HX FSM**-LK10-OX2 1.0 0.9 ECDF 0.8 0.7 0.6 0.5 0.4 0.3 0.2 LK10-MNS-HX FSM**-LK10-OX2 FSM**-LK10 LK10-MNS 0.1 0 2 4 6 log 10(NT) 8 (g) ECDF for NT and F t =0 16 / 31
LS-LS-X Hybrids Performance Different LS-LS has different suitable Crossover Operator 1.0 0.9 ECDF 0.8 0.7 0.6 0.5 0.4 0.3 0.2 LK10-MNS-HX FSM**-LK10-OX2 FSM**-LK10-HX LK10-MNS-OX2 0.1 0 2 4 6 log 10(NT) 8 (h) ECDF for NT and F t =0 17 / 31
LS-LS-X Hybrids All tested LS-LS-X hybrids 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 ECDF LK10-MNS-HX FSM**-LK10-OX2 FSM**-LK10-HX LK10-MNS-OX2 FSM**-LK10 LK10-MNS FSM** LK10 MNS 0.1 0 2 4 6 log 10(NT) 8 (i) ECDF for NT and F t =0 18 / 31
LS-LS-X Hybrids City Scale from 128 to 255 and 256 to 511: 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 ECDF LK10-MNS-HX FSM**-LK10-OX2 FSM**-LK10 LK10-MNS 2 3 4 5 6 7 log 10(NT) 8 (j) F t =0 for 128 n 255 1.0 ECDF 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 2 3 4 5 6 7 log 10(NT) 8 (k) F t =0 for 256 n 511 19 / 31
20 / 31 LS-LS-X Hybrids City Scale from 128 to 255 and 256 to 511: 1.0 ECDF 0.9 0.8 LK10-MNS-HX 0.7 FSM**-LK10-OX2 FSM**-LK10-HX 0.6 LK10-MNS-OX2 0.5 FSM**-LK10 0.4 LK10-MNS 0.3 FSM** LK10 0.2 MNS 0.1 2 3 4 5 6 7 log 10(NT) 8 (l) F t =0 for 128 n 255 1.0 ECDF 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 2 3 4 5 6 7 log 10(NT) 8 (m) F t =0 for 256 n 511
21 / 31 LS-LS-X Hybrids LS-LS-X Experiment Result LK10-MNS-HX (rank 1), LK10-MNS-MPX (2), FSM**-LK10-OX2 (3), FSM**-LK10-CX (4.5), LK10-MNS-OX2 (4.5), FSM**-LK10-HX (6), FSM**-LK10-MPX (7), FSM**-LK10 (8.5), LK10-MNS (8.5), LK10-MNS-CX (10). : LS-LS-X hybrid algorithms ranking from best to worst. The different algorithm types LS-LS hybrid and LS-LS-X hybrid are highlighted.
22 / 31 Outline 1 Research Content 2 Local Search Algorithm 3 Crossover Operator 4 Hybrid Algorithms for the TSP LS-LS-X Hybrids Hybrid Global Search with Local Search 5 Conclusion
22 / 31 Local Search (LS) Feature Fast convergence, but may get trapped by local optima All Solutions Neighbors
Hybrid Evolutionary Algorithm with LS Begin Mutation Crossover LS LS LS LS N Stop Y End 23 / 31
Hybrid Evolutionary Algorithm with LS EA-LS-LS-X experiment result: 1.0 ECDF 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 MA(16+64)-LK10-MNS-HX MA(16+64)-FSM**-LK10-HX MA(16+64)-FSM**-LK10 MA(16+64)-LK10-MNS 0.1 0 1.5 log 10(NT) 2.5 3.5 4.5 5.5 6.5 7.5 8.5 (a) ECDF for NT and F t =0 24 / 31
Hybrid Evolutionary Algorithm with LS EA-LS-LS-X experiment result: 1.0 ECDF 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 MA(16+64)-LK10-MNS-HX MA(16+64)-FSM**-LK10-HX MA(16+64)-FSM**-LK10 MA(16+64)-LK10-MNS-OX2 MA(16+64)-LK10-MNS MA(16+64)-LK10 MA(16+64)-FSM**-LK10-OX2 MA(16+64)-FSM** MA(16+64)-MNS 0.1 0 1.5 log 10(NT) 2.5 3.5 4.5 5.5 6.5 7.5 8.5 (b) ECDF for NT and F t =0 25 / 31
Hybrid Evolutionary Algorithm with LS EA-LS-LS-X experiment result: MA(2+8)-LK10-MNS-HX (rank 1), MA(2+4)-LK10-MNS-HX (2), MA(16+64)-LK10-MNS-HX (3), MA(16+64)-LK10-MNS (4), MA(16+64)-LK10-MNS-OX2 (5), MA(16+64)-FSM**-LK10-HX (6), MA(16+64)-FSM**-LK10 (7), MA(16+64)-LK10 (8), MA(2+8)-FSM**-LK10-OX2 (9), MA(2+4)-FSM**-LK10-OX2 (10), MA(2+8)-FSM**-LK10-HX (11), MA(2+4)-LK10-MNS-OX2 (12), MA(2+8)-LK10-MNS-OX2 (13), MA(16+64)-FSM**-LK10-OX2 (14), MA(2+8)-LK10-MNS (15), MA(2+4)-FSM**-LK10-HX (16), MA(2+4)-LK10-MNS (17), MA(2+8)-FSM**-LK10 (18), MA(16+64)-FSM** (19), MA(2+4)-LK10 (20), MA(2+8)-LK10 (21), MA(2+4)-FSM**-LK10 (22), MA(16+64)-MNS (23), MA(2+8)-FSM** (24), MA(2+4)-FSM** (25), MA(2+8)-MNS (26), MA(2+4)-MNS (27). : EA hybrid algorithms ranking from best to worst. 26 / 31
27 / 31 Outline 1 Research Content 2 Local Search Algorithm 3 Crossover Operator 4 Hybrid Algorithms for the TSP 5 Conclusion
27 / 31 Conclusion The new LS-LS-X hybrids are better than their pure LS algorithm and LS-LS hybrid components. The new EC-LS-LS-X hybrids outperform the LS-LS-X algorithms as well as EC-LS and EC-LS-LS hybrids. MA(2+4)-LK10-MNS-HX becomes the new most powerful hybrid EA algorithm in the huge collection of algorithms and experimental results of the popular TSP Suite. Different LS-LS hybrids have different suitable crossover operators.
28 / 31 References Weichen Liu, Thomas Weise, Yuezhong Wu, and Raymond Chiong. Hybrid ejection chain methods for the traveling salesman problem. In Bio-Inspired Computing-Theories and Applications, pages 268 282. Springer, 2015. Weichen Liu, Thomas Weise, Yuezhong Wu, and Qi Qi. Combining two local searches with crossover: An efficient hybrid algorithm for the traveling salesman problem. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 17), July 15 19, 2017, Berlin, Germany, New York, NY, USA. ACM Press. Weichen Liu, Thomas Weise, Yuezhong Wu, Dan Xu, and Raymond Chiong. An improved ejection chain method and its hybrid versions for solving the traveling salesman problem. Journal of Computational and Theoretical Nanoscience, 13(6):3601 3610, 2016. Thomas Weise, Raymond Chiong, Ke Tang, Jörg Lässig, Shigeyoshi Tsutsui, Wenxiang Chen, Zbigniew Michalewicz, and Xin Yao. Benchmarking optimization algorithms: An open source framework for the traveling salesman problem. IEEE Computational Intelligence Magazine (CIM), 9(3):40 52, August 2014. Yuezhong Wu, Thomas Weise, and Weichen Liu. Hybridizing different local search algorithms with each other and evolutionary computation: Better performance on the traveling salesman problem. In Proceedings of the 18th Genetic and Evolutionary Computation Conference (GECCO 16). ACM Press: New York, NY, USA, July20 24, 2016.
Thanks! 29 / 31
: Distribution of Symmetric Instances in TSPLIB 30 / 31 Appendix TSPLIB contains 110 symmetric TSP instances whose city scale is range from 14 to 85900. Instance Scale Number of instances 0 2 4 1 1 2 4 2 5 1 8 2 5 2 6 1 8 2 6 2 7 1 17 2 7 2 8 1 20 2 8 2 9 1 11 2 9 2 10 1 13 2 10 2 11 1 15 2 11 2 12 1 6 2 12 2 13 1 4 2 13 2 14 1 4 2 14 2 15 1 1 2 15 2 16 1 1 2 16 2 17 1 1
31 / 31 Global Search VS Local Search Global Search VS Local Search: Objective Function L G S C P Search Space (a) Global Search VS Local Search