A CLUSTERING HEURISTIC BY EFFECTIVE NEAREST NEIGHBOR SELECTION

Size: px

Start display at page:

Download "A CLUSTERING HEURISTIC BY EFFECTIVE NEAREST NEIGHBOR SELECTION"

Olivia Eaton
5 years ago
Views:

1 A CLUSTERING HEURISTIC BY EFFECTIVE NEAREST NEIGHBOR SELECTION Mahmuda Naznin, Paul Juell, Kendall E. Nygard and Karl Altenburg Department of Computer Science and Operations Research North Dakota State University Fargo, ND 58105, USA {Mahmuda.Naznin, Paul.Juell, Kendall.Nygard, Abstarct We introduce a clustering heuristic based on an effective nearest neighbor search procedure by reducing the TSP (Traveling Saleman Problem) tour length. This technique contributes to the improvement of the TSP solutions and also provides a technique of finding cluster center in a large domain where finding the center point of data clusters is a difficult problem as the huge volume of data is not sorted in real time domain. In our approach, we find as the number of data points grows the our proposed heusristic reduces the tour length. Time and distance tradeoff has been analyzed also from the simulation result. We start with an approach that to find the group of data closed enough we need to calculate centroid. A good center point is that which contributes to both local and global optima. The TSP tour can get better result due to this issue. Therefore, super point idea emerges with the centroid calculation issue. By finding the suitable nearest neighbor of a super point contributes to the local optimal distance. To avoid foothills stuck, super points are recalculated or updated to obtain global optima. We define the merged point as a Superpoint by calculating the centroid of the two points. Such as, if p(x1, y1), p(xn, yn) are points defined in the domain, the Superpoint S is defined by ((x1+.+xn)/n, (y1+.+yn)/n). The distance between any two points p(x1, y1) and q(x2, y2) can be defined as Euclidian distance d. The point n is defined as the Nearest Neighbor of point m where, d(m, n) is the minimum distance of all the distances between m and the other points. The objective is to find a heuristic for the next neighbor of the TSP tour. The agent can start any point ordering. The method is based on greedy TSP, where two closed data points are merged to a super data point which is basically the centroid of the visited points. The method searches the nearest neighbor of this Superpoint. If the nearest neighbor of the Superpoint does not contribute to the shortest path in TSP tour, selecting another nearest neighbor of the last visited point contributes to the shorter tour length. In this case the nearest neighbor of Superpoint is discarded. The whole process is being repeated till all data points are visited. We develop the simulation in Java SDK 1.5. For different data density our heuristic shows more effective than greedy NN in the TSP tour.

2 1 Introduction Finding an optimal Traveling Salesman Tour is a well known research problem which is known as TSP. The tour is defined as a problem by finding the shortest distance tour of a number of cities where cities will be visited once and the starting city and the returning city will be the same for a Salesman who is traveling those cities. The city can be chosen randomly. This quantity is referred to as the tour length, since it is the length of the tour a salesman would make when visiting the cities in the order specified by the permutation, returning at the end to the initial city. The distance is symmetric in the TSP. Figure 1 shows a typical NN TSP tour. In Figure 1 the tour sequence follows the node as: , where 2 is the nearest neighbor of 1, 3 is the nearest neighbor of 2, so on. The Traveling Salesman Problem is an NP hard problem and so any algorithm for finding an optimal tour must have a worst-case running time that grows faster than any polynomial [8], [9]. This leaves researchers with two alternatives: either to look for heuristics that merely finds near-optimal tours, or attempt to develop optimization algorithms that work well on real-world, rather than the worst-case instances. Figure 1. A Traveling Salesman tour. The Traveling Salesman Problem has many applications, from VLSI chip fabrication to genomics [13], [15]. The problem of finding a Hamiltonian cycle or path in a graph is a special case of the traveling salesman problem [3], where each pair of vertices with an edge between them has distance 1, while non edge vertex pairs are separated by distance infinity. Closely related is the problem of finding the longest path or cycle in a graph, which occasionally arises in pattern recognition problems [8], [9]. The minimum spanning tree (MST) of a graph defines the cheapest subset of edges that keeps the graph in a cluster of connected component [8], [9]. The traveling salesman problem (TSP) is one of the most difficult problems that occur in different types of clustering algorithms [7], [11], [13], [15]. The mapping from a TSP instance to a clustering problem instance is straightforward and is shown by Lawer et al. 1

3 [7]. Each object in a cluster is considered as a city and the dissimilarity between two objects is transformed to the distance between the corresponding cities. The TSP tour, which must have the minimum distance among all the cities to complete the tour, is similar to an optimal rearrangement of the objects with the minimum dissimilarity in a cluster. Thus, TSP is the same problem as finding an optimal permutation, except that TSP finds a cycle through the cities and rearrangement clustering finds a path. The remainder of the paper is organized as follows: Section 2 provides some background review on Traveling Salesman Problem (TSP). Section 3 describes the problem domain and our approach. Section 4 provides the analysis of the results of the implementation of the heuristic. Section 5 gives the conclusion and finally Section 6 gives some future work outlines. 2 Related Work In this section we discuss some well known heuristics and search algorithms for the Traveling Salesman Problem. The most natural heuristic for the TSP is the famous Nearest Neighbor algorithm (NN). In this algorithm one mimics the traveler whose rule of thumb is always to go next to the nearest as-yet-unvisited location. There can be any ordering of the cities, with the initial city c i chosen arbitrarily and in general c k chosen to be the city so that, distance d(c i, c k ) is the minimum distance of the distances between c i and any other point. The corresponding tour traverses the cities in any order, returning to c 1 after visiting city c N when c 1 is the initial city from where he tour was started and Nth is the last city of N number of cities. The running time for NN as described is O(N 2 ) [8], [9]. Some authors use the name Greedy for Nearest Neighbor. There are some modifications of generating intermediate partial tours typically by this heuristic which is called the multi-fragment heuristic by Bentley [4]. His greedy heuristic can be implemented to run in time O(N 2 logn) and is thus, somewhat slower than NN. But the worst examples known for Greedy only make the ratio grow as (log N)/(3 log(log N)) which is shown by Frieze et al. [19]. The Clarke-Wright heuristic is derived from a more general vehicle routing algorithm by Clarke and Wright [17]. In terms of the TSP, it is started as a pseudo-tour in which an arbitrarily chosen city is the hub and the salesman returns to the hub after each visit to another city. In other words, it can be considered as a multi-graph in which every nonhub vertex is connected by two edges to the hub. For each pair of non-hub cities, let the savings be the amount by which the tour would be shortened if the salesman went directly from one city to the other, bypassing the hub. It is analogous to the Greedy algorithm. The traveler can go through the non-hub city pairs in non-increasing order of savings, performing the bypass so long as it does not create a cycle of non-hub vertices or cause a non-hub vertex to become adjacent to more than two other non-hub vertices. The construction process terminates when only two non-hub cities remain connected to the hub, in which case we have a true tour. As with Greedy, this algorithm can be implemented to run in time O(N 2 logn) [17]. 2

4 The previous three algorithms all have worst-case ratios that grow with N even when the triangle inequality holds. As observed by Rosenkrantz, Stearns, and Lewis [20], there are at least three simple polynomial-time tour generation heuristics, Double Minimum Spanning Tree, Nearest Insertion, and Nearest Addition which have worst-case ratio 2 under the triangle inequality. The algorithm of Christofides [2] has a worst-case ratio of just 3/2 assuming the triangle inequality. This bound is tight, even for Euclidean distance: it was proved by Nemhauser et al. [19]. A modification of the Christofides algorithm with the same worst-case guarantee and an O(N 2.5 ) running time can be obtained by using a scaling based matching algorithm and halting once the matching is guaranteed to be no longer than O (1/N) times optimal proved by Gabow, Kaplan and Tarjan [21]. There are some improvements of TSP heuristics by applying different techniques of local search. Among simple local search algorithms, the most famous are 2-Opt and 3-Opt. The 2-Opt algorithm was first proposed by Croes [28]. This move deletes two edges, thus breaking the tour into two paths, and then reconnects those paths in the other possible way. In 3-Opt algorithm it is shown by Lin et al. [29] that, the exchange replaces up to three edges of the current tour. It is not surprising that a wide variety of tabu search algorithms have been proposed for the TSP improvement by Glover [16], Rossier, Troyon, and Liebling [22], Malek, Guruswamy, and Pandya [23], Knox [24]. All these algorithms use 2-Opt exchanges as their basic moves, but they differ as to the nature of the tabu lists and the implementation of aspiration levels. The tabu mechanisms used by Troyon et al. [22] and Malek, Guruswamy, and Pandya [23] are based on the endpoints of the changed edges rather than the edges themselves, but they appear to be similar in flavor. The invention of simulated annealing actually preceded that of tabu search. Like tabu search, simulated annealing allows uphill moves. However, whereas tabu search in essence only makes uphill moves when it is stuck in local optima, simulated annealing can make uphill moves at any time. Moreover, simulated annealing relies heavily on randomization, whereas tabu search in its basic form chooses its next move in a strictly deterministic fashion except possibly when there is a tie for the best non-tabu neighbor. This was originally proposed by Kirkpatrick et al. [25] and Cerny [26]. There is another application of a neural net approach to the TSP solution was due to Hopfield and Tank [27]. Their approach was based on the integer programming formulation of the TSP. 3 Our NN Approach 3.1 Preliminaries and Definitions The Superpoint heuristic method is based on the greedy TSP, where it merges two closed data points to a super data point. We define the merged point as a superpoint by 3

5 calculating the centroid of the two points. If p(x1, y1),, p(xn, yn) are points defined in the domain, Superpoint S is defined by ((x1+.+xn)/n, (y1+.+yn)/n). The distance between any two points, p(x1, y1) and q(x2, y2), can be defined as Euclidian distance, d. Point n is defined as the nearest neighbor of point m, where d(m, n) is the minimum distance of all the distances between m and the other points. 3.2 Superpoint Heuristic for TSP In this section, we discuss the Superpoint TSP heuristic method and the implementation steps. Data points are taken as input. The main objective is to find a heuristic rule for the next neighbor of the TSP tour. The agent can start at any point. The method is based on the greedy TSP but sequentially calculates a super data point which is basically the centroid of the already visited points. The method searches for the nearest neighbor of this superpoint. If the nearest neighbor of the superpoint is closer than the nearest neighbor of the last visited point in the tour, the tour expands to the point nearest to the superpoint. If, however, the nearest neighbor of the last visited node is closer, the tour expands in the usual nearest neighbor way. The whole process is being repeated until all of the data points are visited. Figure 2 shows a tour built with Superpoint TSP heuristic method. Suppose after visiting node 3, the centroid of nodes 1, 2, and 3 has a nearest neighbor which is node 5. For node 3, the nearest neighbor is node 4. The superpoint (centroid of nodes 1, 2, and 3) nearest neighbor is closer than node 4. So, the next node to be visited is node 5. Then from node 5, the tour starts again. The superpoint is saved for the next centroid calculation. Figure 2. The tour by the Superpoint TSP heuristic. 4

6 The tour sequence is now node The thick dotted lines represent the nearest neighbor selection from the superpoint. The pseudo code of the Superpoint TSP heuristic method is given as follows: Input: List of data points I = {1,.., N}. Output: Distance, time to complete the tour, Superpoint. Start: Superpoint of visited points S = Ø Step 1. Start with a random point, i, where 1 i N. Step 2. Mark i as visited node. Find j, the nearest neighbor of i; mark j also as visited node; and set i j. Step 3. While the tour is not finished Step 4. Find the nearest neighbor of the point i which is NN j. Step 5. Find superpoint S by calculating the centroid of i and the visited nodes. Step 6. Calculate the nearest neighbor, NN s, of S. Step 7. If the distance between i and NN j is less than the distance between S and NN s, then i = NN j else, i = NN s End if Go to Step 3. End While End 4 Simulation Results In this section, we include some of the results after running the heuristic. First, we describe the test bed. The code is written in the environment of Java SDK 1.5. We use a machine with an Intel Pentium 4 processor, 512MB RAM, and a clock speed of 2.40 GHz. The cities to visit are implemented by a doubly linked list. The list is created dynamically so that there is no fixed storage required. For the experiments, we run the program where the data points are generated randomly with a uniform distribution. We use the random number generator of Java SDK 1.5. The number of data points is ranged 1,000 to 20,000. We generate the test cases. Suppose that, for 1,000 points run we generate 1,000 random points in (1,000 x 1,000) square unit area; 1,000 points in (2,000 x 2,000) area; etc. The points are all integers such as (2, 15), (7, 8), etc. Point (2, 5) means that the x co-ordinate of the point is 2 and the y co-ordinate of the point is 15. We mapped the area on square grids. Each grid size is 1x1 square units. Therefore, a (2,000 x 2,000) area has 4,000,000 available grid points. The test problem points are generated on grid locations. Different data densities are tested in this procedure. Figure 3 shows the corresponding line graphs for the average distance calculated by the Superpoint TSP and the NN TSP tour. In this figure, we can see that the average tour 5

7 length of the Superpoint TSP tour is consistently less then the average tour length of the NN TSP tour. As the number of data points increases, the tour length of the Superpoint TSP decreases compared to the typical NN TSP method. The tour length increases as the number of data points increases for both the NN TSP and the Superpoint TSP heuristic procedures. Figure 4 shows the corresponding line graphs for the average time required by the Superpoint TSP and the NN TSP heuristic. Avg. distance for TSP vs. SuperTSP SuperTSP TSP distance #of points Figure 3. Mean distance calculated by the Superpoint TSP and the NN TSP heuristics. Avg. time for TSP and SuperTSP SuperTSP TSP time # of points Figure 4. Mean time required by the Superpoint TSP and the NN TSP heuristics. 6

8 Figure 5 shows the line graph representing the average deviation between the average distances calculated by the Superpoint TSP and the NN TSP heuristic. In the figure, we can see that the average time deviation increases as the number of data points increases. Figure 6 shows the line graph representing the average deviation between the average time required by the Superpoint TSP and the NN TSP heuristic. Avg. deviation of distance between TSP and SuperTSP 7000 Avg. deviation Avg. distance deviation # of points Figure 5. Mean deviation between the distances calculated by the Superpoint TSP and the NN TSP heuristics. Avg. deviation of time for TSP and SuperTSP Avg. deviation avg. time deviation (ms) # of points Figure 6. Mean deviation between the time required by the Superpoint TSP and the NN TSP heuristics. 7

9 Figure 7 shows the line graph represnting the percentage change in the average length of the tours calculated by the Superpoint TSP and TSP heuristic. In the figure, we can see the tendency of average distance reduction as the number of the data points increases. The tour length is reduced to 4% for the number of data sets ranging from 1,000 to 19,000. % reduction of tour distance 4.5 % reduction of tour distance % reduction of distance # of points Figure 7. Change in average tour distance calculated by the Superpoint TSP and the NN TSP heuristics. 5 Conclusion In this paper we introduce a clustering heuristic based on an effective nearest neighbor search procedure by reducing the TSP (Traveling Saleman Problem) tour length. This technique contributes to the improvement of the TSP solutions and also provides a technique of finding cluster center in a large domain where finding the center point of data clusters is a difficult problem as the huge volume of data is not sorted in real time domain. In our approach, we find as the number of data points grows the our proposed heusristic reduces the tour length. Time and distance tradeoff has been analyzed also from the simulation result. We find a good center point is that which contributes to global optima. References [1] D. Applegate, R. Bixby, V. Chavatal, W. Cook, On The Solution of Traveling Salesman Problems, Documenta Mathematica Journal der Deutschen Mathematiker- Vereinigung, International Congress of Mathematicians, pp ,

10 [2] N. Christofides, Worst-case Analysis of A New Heuristic For The Traveling Salesman Problem, Report Number 388, Graduate School of Industrial Administration, Carnegie Mellon University, [3] M. Chrobak, T. Szymacha, A. Krawczyk, A Data Structure Useful For Finding Hamiltonian cycles, Theoretical Computer Science, vol. 71, pp , [4] J. L. Bentley, Experiments on Traveling Salesman Heuristics, First Annual ACM- SIAM Symposium on Discrete Algorithms, pp , [5] J. Mittenthal, C. E. Noon, An Insert/Delete Heuristic For The Traveling Salesman Subset-tour Problem With One Additional Constraint, Journal of The Operational Research Society, vol. 43, no. 3, , [6] G. A. P. Kindervater, J. K. Lenstra, D. Shmoys, The Parallel Complexity of TSP Heuristics, Journal of Algorithms, vol. 10, pp , [7] E. L. Lawler, J. K. Lenstra, A. H. G. Rinnooy Kan, D. B. Shmoys, The Traveling Salesman Problem: A Guided Tour of Combinatorial Optimization, John Wiley and Sons, New York [8] T. Coremen, C. E. Leiserson, R. L. Rivest, Introduction to Algorithms, Prentice Hall, MIT Press, Cambrige, MA, [9] M. D. Berg, M. V. Kreveld, M. Overmars, O. Schwarzkopf, Computational Geometry, Algorithms and Applications, Springer-Verlag, New York, [10] S. Climer, W. Zhang, Rearrangement Clustering: Pitfalls, Remedies, and Applications, Journal of Machine Learning Research, vol. 1, no.1, January, [11] M. Furukawa, M. Watanabe, Y. Matsumura, Local Clustering Organization (LCO): Solving A Large-Scale TSP, Journal of Robotics and Mechatronics, vol. 17, no. 5, pp , [12] K. Rose, E. Guurewitz, G.C. Fox, Constrained Clustering As An Optimization Method, IEEE Transaction on Pattern Analysis and Machine Learning, vol. 15, pp , August [13] T. Kawai, Y. Yokoi, Y. Miura, T. Tabata, T. Nagasu, K. Aoshima, A Novel Clustering Algorithm With Map Energy Minimization, Genome Informatics, vol. 13, pp , [14] D. Eppstein, Fast Hierarchical Clustering and Other Applications of Dynamic Closest Pairs, Proceedings of SODA, [15]cc.ee.ntu.edu.tw/~cchen/course/simulation/CAD/unit5C.pdf. [16] F. Glover, Tabu Search For The p-median Problem, University of Colorado, Technical Report, [17] G. Clark, J. W. Wright, Scheduling of Vehicles From A Central Depot to A Number of Delivery Points, Operations Research, vol. 12, pp , [18] A. Frieze, G. B. Sorkin, The Probabilistic Relationship Between The Assignment Problem And Asymmetric Traveling Salesman Problems, Proceedings of SODA, ACM Publishers, , [19] M. L. Fisher, G. L. Nemhauser, L. A. Wolsey, An Analysis of Approximations For Finding a Maximum Weight Hamiltonian circuit. Operations Research, vol. 27, no. 4, pp , [20] D. J. Rosenkrantz, R. E. Stearns, P. M. Lewis II, System Level Concurrency Control For Distributed Database Systems, ACM Trans. on Database Systems, vol. 3, no. 2, pp , June

11 [21] H. N. Gabow, H. Kaplan, R.E. Tarjan, Unique Maximum Matching Algorithms, Journal of Algorithms, vol. 40, pp , [22] Y. Rossier, M. Troyon, T.M. Liebling, Probabilistic Exchange Algorithms and Euclidean Traveling Salesman Problems, OR Spectrum, vol. 8, pp. 151, [23] M. Malek, M Guruswamy, M. Pandya, H. Owens, Serial and Parallel Simulated Annealing and Tabu Search Algorithms for the Traveling Salesman Problem, Annals of Operations Research, vol. 21, pp.59-84, [24] J. Knox, Tabu Search Performance on the Symmetric Traveling Salesman Problem, Computers and Operations Research, vol. 21, pp , [25] S. Kirkpatrick, D. Gellatt, M. Vecchi, Optimization By Simulated Annealing, Computer Science, vol. 220, pp , [26] V. Cerny, Quantum computers and intractable (NP -complete) Computing problems, Phys. Review, vol. 48, no. 1, pp , [27] J. J. Hopfield and D. W. Tank, Neural Computation of Decisions in Optimization Problems, Biological Cybernetics, vol. 52, pp , [28] G. A. Croes, A Method For Solving Traveling Salesman Problem, Operations Research, vol. 6, pp , [29] S. Lin, B. Kernighan, An Effective Heuristic Algorithm For The TSP, Operations Research, vol. 10

Two new variants of Christofides heuristic for the Static TSP and a computational study of a nearest neighbor approach for the Dynamic TSP

Two new variants of Christofides heuristic for the Static TSP and a computational study of a nearest neighbor approach for the Dynamic TSP Orlis Christos Kartsiotis George Samaras Nikolaos Margaritis Konstantinos