Massively Parallel Approximation Algorithms for the Traveling Salesman Problem

Transcription

1 Massively Parallel Approximation Algorithms for the Traveling Salesman Problem Vaibhav Gandhi May 14, 2015 Abstract This paper introduces the reader to massively parallel approximation algorithms which compute an optimal tour close to the best tour for a traveling salesman going through a set of cities. The algorithms mentioned in this paper take advantage of parallel computing. The paper compares and analyzes the results of an exhaustive search against the massively parallel approximation algorithms. 1 Introduction The traveling salesman problem (TSP) describes a salesman who needs to travel through a list of given cities exactly once by taking the shortest possible tour. The problem is NP-Complete since there are no known polynomial time algorithms to solve it. The main motivation to solve this problem is that the applications of Traveling Salesman are immense. It can be used by banks to find an optimal route for updating cash in ATMs, by delivery companies to chalk out the best possible route between the delivery centers and save up on fuel and man hours. It can be used outside of delivery based applications too. For example it can be used for drilling holes in circuits. The easiest way to solve the traveling salesman problem would be by using brute force search which is also called as exhaustive search. The algorithm involves going through each possible ordered permutation of cities and finding out the cheapest route. The running time for this approach is O(n!) which is the factorial of the number of cities. One can easily say that this kind of solution is not feasible even for computing the route between 20 cities. This is a serious limitation. Recent advances by vendors like Amazon, Google and Microsoft have made cloud services easily accessible to the public. One can easily harness the power of the cloud to solve huge problems at a fraction of the cost in parallel by making use of multiple cores on multiple nodes. Traveling salesman is one such problem that can be modified for such infrastructure. 1

2 This paper proposes two Massively Parallel Approximation algorithms which can be used to obtain approximate solutions close to the best solutions for the traveling salesman problem. The paper is broken down as follows: Section 2 provides a brief overview of the literature that was reviewed and that inspired the design of these algorithms. Section 3 provides description of the algorithms. Section 4 discusses the results of the experiments. Section 5 contains an analysis of the results and Section 6 concludes the paper with final comments and future plans in this area. 2 Related Work The following papers were reviewed and used a basis for designing the algorithms proposed in this paper. 1. A parallel Tabu search algorithm for large traveling salesman problems [2]: Fiechter, the author, proposes the use of Tabu Search for global optimization. The algorithm starts of by generating a tour and then making modifications to it iteratively till we get the possible tour. The methods mentioned in the paper are adapted for parallel computing and used to find tours for up to 100,000 cities. 2. Parallel Heuristics for TSP on MapReduce [3]: The authors of this paper propose an adapted version of Neighbourhood Search and Tabu Search that uses MapReduce to compute tours in parallel. The results of these 2 algorithms are compared against a sequential algorithm and the authors conclude that the algorithms are not well suited for MapReduce. 3. Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem[1]: The authors of this paper are intrigued by the way ants move around and they adapt this knowledge to solve the traveling salesman problem. 4. New Parallel Randomized Algorithms for the Traveling Salesman Problem [5]: The authors demonstrate a new method based on randomized optimization which partitions feasible regions into sub regions and manipulates the regions based on random sampling. The techniques described further on in this paper take inspiration from the literature reviewed but differs a lot from it. Concepts like Tabu search and Randomized optimizations were modified for these techniques. The central idea of both of the techniques is to start with a tour and work on it to reach local minima whereas some of the papers reviewed usually worked by computing a tour. 3 Algorithm Description The algorithms mentioned below makes use of a seed to generate a graph of cities in a 1000 x 1000 area. The seed is used to initialize the pseudo random 2

3 number generator. The random number generator randomly assigns each city an x and y coordinate in this space. The distance between any two cities is the Euclidean Distance which is the length of the line segment connecting the two cities. The cost of the tour or length of the route is computed by calculating the Euclidean Distance between subsequent cities. 3.1 Exhaustive Branch and Bound Search Exhaustive search goes through each possible tour and finds the shortest possible route. For the purpose of this research paper, a modified version of exhaustive search was implemented to work on a multi-core computer. Each core would compute a different tour and the results from all the cores were combined to find the shortest tour. Over all for a quad core computer, each core would compute (n!)/4 routes where n is the number of cities. Despite executing the algorithm in parallel, the running time of the program was in exponential time. To overcome this, the exhaustive algorithm was modified to make use of branch and bound pruning. A starting city is known, which is the root node. The nodes of the tree are different cities. A branch is a route from root to the leaf node. Cities do not repeat in a branch. Every time a branch is computed, a cost which is the length of the tour, is obtained. This cost is maintained globally. Every new branch computed is checked against the lowest cost of previous branches. This is called bounding. Any branch that exceeds the bounds is pruned, i.e. the route is going to lead to a lengthier tour and thus is not worth computing. This results in fewer computations without affecting the quality of the solution. 3.2 Approximation Algorithms The algorithms mentioned below start by generating a tour and processing it to find the most optimal approximate tour Massively Parallel Stochastic Search Stochastic Search is a modified Local Search. The problem of getting stuck in local minima in local search leads to not so optimal results. Stochastic Search randomizes the initialization step which leads to random solutions. The stochastic algorithm randomly generates an initial tour using a combination of user submitted seed and the iteration number. The cost of this tour is computed. The cities in this tour are randomly swapped and resulting in new routes and after each swapping the cost is recomputed. This step is repeated till the cost of the current route is better than the cost of the previous route. The point where there is no improvement in the cost is the local minima. The steps listed above are repeated for a number of iterations and the lowest local minima from these iterations is the lowest cost route using this approach. 3

4 The program was modified to work on a multi-core computer. For a quad core computer, each core would compute n/4 routes where n is the number of iterations Massively Parallel Random Search The Random Search algorithm randomly generates an initial tour using a combination of user submitted seed and the iteration number. The cost of this tour is computed. This is repeated for a number of iterations and the lowest cost from these iterations is the lowest cost route using this approach. 4 Results A 16 core parallel computer was used for all the experiments mentioned in this paper. Different systems were not used so as to compare the time complexity of both algorithms as well as the superiority of the results. Parallel Java 2, a library developed by Prof. Alan Kaminsky [4] was used for handling all the parallel communication, segmentation of iterations and management of task between cores and aggregation of the results from the different cores. The results obtained from the Exhaustive Search are used as a benchmark for the Approximation Algorithms. Shown below in Table 1 is a copy of the results for the exhaustive search for seed 456. Table 2 and Table 3 contain a snapshot of the data from Stochastic Search and Random Search for the same seed. Cities Time (ms) Cost Table 1: Exhaustive Cost values for Seed : 456 All the remaining graphical plots of the results are presented in Appendix A. The data for other seeds can be found in an accompanying cd drive. 4

5 Table 2: Stochastic and Random data for Seed : 456 5

6 Table 3: Continuation of Stochastic and Random data for Seed : 456 6

7 Figure 1: Plot of Cost Comparison for 10 iterations for Seed : 456 Stochastic Search performs way better than Random Search as compared to Exhaustive Search in terms of cost as expected for 10 iterations Figure 2: Plot of Time Comparison for 10 iterations for Seed : 456 Time required for Stochastic Search and Random Search is negligible (approx 10ms) for 10 iterations as compared to Exhaustive Search. 7

8 Figure 3: Plot of Cost Comparison for iterations for Seed : 456 Stochastic Search performs close to Exhaustive Search in terms of cost but Random Search is not good enough for iterations. Figure 4: Plot of Time Comparison for iterations for Seed : 456 Time required for Stochastic Search and Random Search is negligible (approx 100ms) for iterations as compared to Exhaustive Search. 8

9 Figure 5: Plot of Percentage Cost Difference for 10 cities for Seed : 456 The percentage cost difference of Stochastic or Random Search vs Exhaustive Search constantly decreases till it eventually reaches 0. Figure 6: Plot of Percentage Cost Difference for 18 cities for Seed : 456 The percentage cost difference of Stochastic or Random Search vs Exhaustive Search constantly decreases till it reaches a threshold after which there is no improvement. 9

10 Figure 7: Plot of Percentage Cost Difference for Stochastic Search for Seed : 456 Stochastic Search approaches cost obtained from Exhaustive Search. Figure 8: Plot of Percentage Cost Difference for Random Search for Seed : 456 For more number of cities, Random search performs poorly. 10

11 5 Analysis Massively Parallel Stochastic Search works much better than Massively Parallel Random Search as compared to the results from Branch and Bound Exhaustive Search. Stochastic Search comes to within 5% of the cost of the Exhaustive Search but in 0.05% time. Stochastic Search was expected to perform better than the Random Search and it did as shown in the results. This is because Stochastic Search improves on the randomly generated tour in each iteration to reach a better local minima as compared to the Random Search. As cities increase, Stochastic Search as well as Random Search require more number of iterations to reach the solution closest to the tour computed from Exhaustive Search. This is because the number of possibilities increase exponentially for each new city (O(n!) number of possibility where n is the number of cities). Both Random Search and Stochastic Search require only a fraction of the time (0.05%) of Exhaustive Search despite the number of iterations because the algorithms do not compute the possibilities but rather process a randomly generated tour at each iteration which is very fast. 6 Future Work As observed from the results, the Stochastic Search is able to evaluate the same route as Exhaustive Search or in most cases computes a route with cost approaching less than 5% cost difference. This is achieved at a fraction of the time of Exhaustive search. Random Search on the other hand does not get such good results. This was expected from the start. In future, I want to try different approaches that would lead to better results in terms of cost and time. I also want to try the current algorithms on real data sourced from one of the maps to visualize the results. References [1] M. Dorigo and L. M. Gambardella. Ant colony system: a cooperative learning approach to the traveling salesman problem. Evolutionary Computation, IEEE Transactions on, 1(1):53 66, [2] C.-N. Fiechter. A parallel tabu search algorithm for large traveling salesman problems. Discrete Applied Mathematics, 51(3): , [3] S. Jain and M. Mallozzi. Parallel heuristics for tsp on mapreduce. Brown University, [4] A. Kaminsky. Parallel java 2 library, [5] L. Shi, S. Ólafsson, and N. Sun. New parallel randomized algorithms for the traveling salesman problem. Computers & Operations Research, 26(4): ,

12 Appendix A : Graphical Plots Figure 9: Plot of Cost Comparison for 10 iterations for Seed :

13 Figure 10: Plot of Time Comparison for 10 iterations for Seed : Figure 11: Plot of Cost Comparison for iterations for Seed :

14 Figure 12: Plot of Time Comparison for iterations for Seed : Figure 13: Plot of Percentage Cost Difference for 10 cities for Seed :

15 Figure 14: Plot of Percentage Cost Difference for 18 cities for Seed : Figure 15: Plot of Percentage Cost Difference for Stochastic Search vs Exhaustive Search for Seed :

16 Figure 16: Plot of Percentage Cost Difference for Random Search vs Exhaustive Search for Seed :

17 Figure 17: Plot of Cost Comparison for 10 iterations for Seed :

18 Figure 18: Plot of Time Comparison for 10 iterations for Seed : Figure 19: Plot of Cost Comparison for iterations for Seed :

19 Figure 20: Plot of Time Comparison for iterations for Seed : Figure 21: Plot of Percentage Cost Difference for 10 cities for Seed :

20 Figure 22: Plot of Percentage Cost Difference for 18 cities for Seed : Figure 23: Plot of Percentage Cost Difference for Stochastic Search vs Exhaustive Search for Seed :

21 Figure 24: Plot of Percentage Cost Difference for Random Search vs Exhaustive Search for Seed :

22 Figure 25: Plot of Cost Comparison for 10 iterations for Seed :

23 Figure 26: Plot of Time Comparison for 10 iterations for Seed : Figure 27: Plot of Cost Comparison for iterations for Seed :

24 Figure 28: Plot of Time Comparison for iterations for Seed : Figure 29: Plot of Percentage Cost Difference for 10 cities for Seed :

25 Figure 30: Plot of Percentage Cost Difference for 18 cities for Seed : Figure 31: Plot of Percentage Cost Difference for Stochastic Search vs Exhaustive Search for Seed :

26 Figure 32: Plot of Percentage Cost Difference for Random Search vs Exhaustive Search for Seed :

27 Figure 33: Plot of Cost Comparison for 10 iterations for Seed :

28 Figure 34: Plot of Time Comparison for 10 iterations for Seed : 9876 Figure 35: Plot of Cost Comparison for iterations for Seed :

29 Figure 36: Plot of Time Comparison for iterations for Seed : 9876 Figure 37: Plot of Percentage Cost Difference for 10 cities for Seed :

30 Figure 38: Plot of Percentage Cost Difference for 18 cities for Seed : 9876 Figure 39: Plot of Percentage Cost Difference for Stochastic Search vs Exhaustive Search for Seed :

31 Figure 40: Plot of Percentage Cost Difference for Random Search vs Exhaustive Search for Seed :

32 Figure 41: Plot of Cost Comparison for 10 iterations for Seed :

33 Figure 42: Plot of Time Comparison for 10 iterations for Seed : 1053 Figure 43: Plot of Cost Comparison for iterations for Seed :

34 Figure 44: Plot of Time Comparison for iterations for Seed : 1053 Figure 45: Plot of Percentage Cost Difference for 10 cities for Seed :

35 Figure 46: Plot of Percentage Cost Difference for 18 cities for Seed : 1053 Figure 47: Plot of Percentage Cost Difference for Stochastic Search vs Exhaustive Search for Seed :

36 Figure 48: Plot of Percentage Cost Difference for Random Search vs Exhaustive Search for Seed :

37 Figure 49: Plot of Cost Comparison for 10 iterations for Seed :

38 Figure 50: Plot of Time Comparison for 10 iterations for Seed : 2179 Figure 51: Plot of Cost Comparison for iterations for Seed :

39 Figure 52: Plot of Time Comparison for iterations for Seed : 2179 Figure 53: Plot of Percentage Cost Difference for 10 cities for Seed :

40 Figure 54: Plot of Percentage Cost Difference for 18 cities for Seed : 2179 Figure 55: Plot of Percentage Cost Difference for Stochastic Search vs Exhaustive Search for Seed :

41 Figure 56: Plot of Percentage Cost Difference for Random Search vs Exhaustive Search for Seed :

42 Figure 57: Plot of Cost Comparison for 10 iterations for Seed :

43 Figure 58: Plot of Time Comparison for 10 iterations for Seed : 6630 Figure 59: Plot of Cost Comparison for iterations for Seed :

44 Figure 60: Plot of Time Comparison for iterations for Seed : 6630 Figure 61: Plot of Percentage Cost Difference for 10 cities for Seed :

45 Figure 62: Plot of Percentage Cost Difference for 18 cities for Seed : 6630 Figure 63: Plot of Percentage Cost Difference for Stochastic Search vs Exhaustive Search for Seed :

46 Figure 64: Plot of Percentage Cost Difference for Random Search vs Exhaustive Search for Seed :

47 Figure 65: Plot of Cost Comparison for 10 iterations for Seed :

48 Figure 66: Plot of Time Comparison for 10 iterations for Seed : 5545 Figure 67: Plot of Cost Comparison for iterations for Seed :

49 Figure 68: Plot of Time Comparison for iterations for Seed : 5545 Figure 69: Plot of Percentage Cost Difference for 10 cities for Seed :

50 Figure 70: Plot of Percentage Cost Difference for 18 cities for Seed : 5545 Figure 71: Plot of Percentage Cost Difference for Stochastic Search vs Exhaustive Search for Seed :

51 Figure 72: Plot of Percentage Cost Difference for Random Search vs Exhaustive Search for Seed :

52 Figure 73: Plot of Cost Comparison for 10 iterations for Seed :

53 Figure 74: Plot of Time Comparison for 10 iterations for Seed : 1078 Figure 75: Plot of Cost Comparison for iterations for Seed :

54 Figure 76: Plot of Time Comparison for iterations for Seed : 1078 Figure 77: Plot of Percentage Cost Difference for 10 cities for Seed :

55 Figure 78: Plot of Percentage Cost Difference for 18 cities for Seed : 1078 Figure 79: Plot of Percentage Cost Difference for Stochastic Search vs Exhaustive Search for Seed :

56 Figure 80: Plot of Percentage Cost Difference for Random Search vs Exhaustive Search for Seed :