A Clustering Approach to the Bounded Diameter Minimum Spanning Tree Problem Using Ants Tyler Derr Thesis Adviser: Dr. Thang N. Bui Department of Math & Computer Science Penn State Harrisburg Spring 2015 Outline 1 BDMST Problem 2 AB-BDST Algorithm 3 Results 4 Conclusion
Outline 1 BDMST Problem 2 AB-BDST Algorithm 3 Results 4 Conclusion Spanning Tree A spanning tree of a graph G is a subgraph of G that is a tree having the same set of vertices as G.
Bounded Diameter Spanning Tree A bounded diameter spanning tree with a diameter constraint d is a spanning tree in which the longest path in the tree is at most d edges. Spanning Complete tree Graph with aofdiameter 8 vertices of 2 Bounded Diameter Spanning Tree A bounded diameter spanning tree with a diameter constraint d is a spanning tree in which the longest path in the tree is at most d edges. 2 1 u 3 4 6 5 v diameter = 2 diameter = 6
The Problem Bounded Diameter Minimum Spanning Tree Problem (BDMST) Input: An undirected complete graph G = (V, E) with non-negative cost on the edges and an integer d 2 representing the diameter constraint. Output: A spanning tree T of G that has a diameter d and is of minimal cost. Where the cost of T is the sum of the costs on all the edges of T. BDMST Complexity and Applications N P-hard Approximating within a logarithmic factor of optimal is also N P-hard Easy if d = 2, 3, or n 1, where n = V Applications Telecommunication and ad-hoc wireless networks Data compression for information retrieval 8 Distributed mutual exclusion algorithms
Current Heuristics for BDMST Variable Neighborhood Search (VNS) Greedy Randomized Adaptive Search Procedue (GRASP) Hierarchical Clustering Learning Automata Ant Colony Optimization (ACO) Genetic Algorithms (GAs) Algorithm categories containing most of the best results Most robust across graph types and sizes Comparison of ACO and ABO Ant Colony Optimization (ACO) Ants work individually and sequentially Ants are used to build a solution to the problem Ants require global information Ant-Based Optimization (ABO) Ants are used collectively as a colony in parallel Ants are used to reduce the search space of the problem Ants require only local information Can be easily adapted to parallel and distributed algorithms
Outline 1 BDMST Problem 2 AB-BDST Algorithm 3 Results 4 Conclusion The Ant-Based Algorithm Idea: A Graph Clusters BDST Optimized Center inof each of the Cluster vertices BDST Graph Cluster
The Ant-Based Algorithm AB-BDST Algorithm(G = (V, E), d) Repeat Species one ants explore the graph Partition vertices into clusters Construct a BDST in each cluster Species two ants explore the graph Construct a BDST T for G Local Optimize T Keep T if it is the best found so far Post processing Until some stopping criteria are met Return the best tree found Phase One Phase Two Optimization AB-BDST Algorithm(G = (V, E), d) Repeat Species one ants explore the graph Partition vertices into clusters Construct a BDST in each cluster Species two ants explore the graph Construct a BDST T for G Local Optimize T Keep T if it is the best found so far Post processing Until some stopping criteria are met Return the best tree found Phase One Phase Two Optimization
Initial Pheromone Each edge in the graph is assigned a pheromone value Lower cost edges are given higher pheromone than higher cost edges Ant Species One Place one ant on each vertex of the graph Phase One: Exploration AB-BDST Algorithm(G = (V, E), d) Repeat Species one ants explore the graph Partition vertices into clusters Construct a BDST in each cluster Species two ants explore the graph Construct a BDST T for G Local Optimize T Keep T if it is the best found so far Post processing Until some stopping criteria are met Return the best tree found Phase One Phase Two Optimization
Phase One: Exploration Idea: Use ants to discover edges that are of low cost that can be used to cluster the vertices and create spanning trees Each ant moves fixed number of steps laying down pheromone on edges they traverse. Ants select an edges to traverse using proportional selection An edge is chosen at random, but an edge having a higher level of pheromone (i.e., lower cost) has a higher probability of being selected. Each ant keeps a tabu list that prevents it from going back to a recently visited vertex. Periodically some of the pheromone on each edge is evaporated Phase One: Cluster Vertices AB-BDST Algorithm(G = (V, E), d) Repeat Species one ants explore the graph Partition vertices into clusters Construct a BDST in each cluster Species two ants explore the graph Construct a BDST T for G Local Optimize T Keep T if it is the best found so far Post processing Until some stopping criteria are met Return the best tree found Phase One Phase Two Optimization
Phase One: Cluster Vertices Idea: Use high pheromone edges to partition vertices into clusters. Clusters are constructed one at time. Each cluster is grown as follows: Select a high pheromone edge whose endpoints do not belong to any cluster. Vertices are added one at a time. A vertex is selected to join a cluster if it is connected to the cluster through a high pheromone edge. Stop when the cluster has grown too large or there are no edges having high enough pheromone to connect. Once a stopping condition is met for a cluster, another cluster is started and expanded using the above steps. Phase One: Merging Clusters Idea: Merge clusters having only one or two vertices into a neighboring cluster All clusters having only one of two vertices are merged into their nearest neighboring cluster Nearest neighboring clusters are based on edges cost, not pheromone.
Phase One: Construct a BDST in Each Cluster AB-BDST Algorithm(G = (V, E), d) Repeat Species one ants explore the graph Partition vertices into clusters Construct a BDST in each cluster Species two ants explore the graph Construct a BDST T for G Local Optimize T Keep T if it is the best found so far Post processing Until some stopping criteria are met Return the best tree found Phase One Phase Two Optimization Phase One: Construct a BDST in Each Cluster Idea: Construct a low cost height constrained spanning tree in each cluster using edges with high pheromone level where the height constraint h = d 2 1 Cluster BDSTs are constructed ensuring the height constraint is not violated For each cluster: Select a vertex from the highest pheromone edge Using random proportional selection based on edge cost expand the subtree by a single vertex until complete Find two or three centers of the constructed cluster BDST
Phase Two: Exploration AB-BDST Algorithm(G = (V, E), d) Repeat Species one ants explore the graph Partition vertices into clusters Construct a BDST in each cluster Species two ants explore the graph Construct a BDST T for G Local Optimize T Keep T if it is the best found so far Post processing Until some stopping criteria are met Return the best tree found Phase One Phase Two Optimization Phase Two: Exploration Idea: Use ants to discover edges that are of low cost that connect the cluster BDSTs together. Create a graph G = (V, E ) where, V is the set of all centers from each of the cluster BDSTs. E is then the set of edges connecting the vertices in V, but only the edges between clusters (i.e., not between centers of the same cluster BDST). Reinitialize the pheromone levels on the edges in E. Place one ant of species two on each of the vertices in V. The ants of species two move in the same way the first species does.
Phase Two: Build a BDST T for G AB-BDST Algorithm(G = (V, E), d) Repeat Species one ants explore the graph Partition vertices into clusters Construct a BDST in each cluster Species two ants explore the graph Construct a BDST T for G Local Optimize T Keep T if it is the best found so far Post processing Until some stopping criteria are met Return the best tree found Phase One Phase Two Optimization Phase Two: Build a BDST T for G Idea: Construct a low cost height constrained spanning tree T for G using the high pheromone edges from species two ants to connect the cluster subtrees. star cost for cluster c = Low star cost c Clusters, c c cost(c.center, c.center) High star cost c c
Phase Two: Build a BDST T for G Idea: Construct a low cost height constrained spanning tree T for G using the high pheromone edges from species two ants to connect the cluster subtrees. Select the first cluster BDST to be added to T based on the lowest star cost. Use random proportional selection based on edge cost to expand T one cluster BDST at a time until complete. After the cluster BDSTs are in T, attempt to rearrange the cluster BDSTs. Note: The height constraint is always maintained during construction. Cluster BDSTs are only connected to T with edges in E. Local Optimization AB-BDST Algorithm(G = (V, E), d) Repeat Species one ants explore the graph Partition vertices into clusters Construct a BDST in each cluster Species two ants explore the graph Construct a BDST T for G Local Optimize T Keep T if it is the best found so far Post processing Until some stopping criteria are met Return the best tree found Phase One Phase Two Optimization
Local Optimization Idea: Attempt to improve T using local optimization techniques. Repeat T RepeatedAncestorSwap(T ) T InsertAndDelete(T ) T EdgeDeletionAndSubtreeReconnection(T ) Until no improvement is found in any method Each local opt is internally looped until no improvement can be found. The height constraint is always maintained throughout all three methods. Local Opt: Repeated Ancestor Swap Idea: Reduce the height of the tree and lower cost of T. Before After
Local Opt: InsertAndDelete Idea: Introduce lower cost edges into our tree effectively lowering the tree cost. Before Transition After Local Opt: EdgeDeletionAndSubtreeReconnection Idea: Greedily take any improvements the other methods were unable to find Before Transition After
Always Keep the Best Tree AB-BDST Algorithm(G = (V, E), d) Repeat Species one ants explore the graph Partition vertices into clusters Construct a BDST in each cluster Species two ants explore the graph Construct a BDST T for G Local Optimize T Keep T if it is the best found so far Post processing Until some stopping criteria are met Return the best tree found Phase One Phase Two Optimization Post Processing AB-BDST Algorithm(G = (V, E), d) Repeat Species one ants explore the graph Partition vertices into clusters Construct a BDST in each cluster Species two ants explore the graph Construct a BDST T for G Local Optimize T Keep T if it is the best found so far Post processing Until some stopping criteria are met Return the best tree found Phase One Phase Two Optimization
Post Processing Idea: Prepare for next iteration Emphasize the edges from T best by increasing their pheromone by an enhancement factor Increase the pheromone enhancement factor Decrease the evaporation factor About half the ants are randomly assigned to new vertices After many iterations without finding a new best tree, the edges of T best have their pheromone levels lowered to 5% to 10% of their current values Stopping Criteria AB-BDST Algorithm(G = (V, E), d) Repeat Species one ants explore the graph Partition vertices into clusters Construct a BDST in each cluster Species two ants explore the graph Construct a BDST T for G Local Optimize T Keep T if it is the best found so far Post processing Until some stopping criteria are met Return the best tree found Phase One Phase Two Optimization
Stopping Criteria AB-BDST terminates after 2500 iterations or after 1500 iterations without improvement on T best An Example Euclidean Graph n = 100, d = 10 Euclidean Graph
Outline 1 BDMST Problem 2 AB-BDST Algorithm 3 Results 4 Conclusion Implementation Details Implemented in C++ Testing machine: Intel Core i5-3570 CPU at 3.4 GHz with 32 GB of RAM running the Ubuntu 12.04 OS 50 runs for each instance
135 Euclidean Graphs 120 non-euclidean Graphs Euclidean non-euclidean n 70 100 250 500 1000 100 250 500 1000 d 7 10 15 20 25 10 15 20 25 # of instances 15 30 30 30 30 30 30 30 30 Current Benchmark Instance #s Only first 5 instances in each group are used. Euclidean Results The best known solutions for these 25 benchmark graphs come from 8 different methods AB-BDST did not find any best known solutions On average AB-BDST was 2.27% away from the best known solution Example approximate runtimes of AB-BDST: n = 100 & d = 10: 15 seconds n = 500 & d = 20: 10 minutes n = 1000 & d = 25: 50 minutes Runtimes of two previous heuristics: n = 500 & d = 20 using a Pentium 4 2.8 Ghz CPU ACO: 65 minutes VNS: 62 minutes
Non-Euclidean Results The previous best known solutions for these 20 benchmark graphs were found from only 2 methods AB-BDST found better best known solutions for 14 of the 20 graphs and tied another best known solution Better average solution for 15 of the 20 (50 runs per instance) Lower standard deviation on all 20 graphs On average the solutions found by AB-BDST showed a 0.71% improvement over previous best solutions Example approximate runtimes of AB-BDST: n = 100 & d = 10: 12 seconds n = 1000 & d = 25: 32 minutes Discussion AB-BDST performed well on the non-euclidean graphs AB-BDST still needs some improvement to obtain the best solutions in Euclidean graphs The ants have proven to be advantageous in this problem because AB-BDST showed improvement over the following: On average 238% lower cost solutions than using a pure greedy construction On average 17% lower cost solutions than using a randomized greedy construction Note: Both of the methods used the delete and reconnect local optimization method
Outline 1 BDMST Problem 2 AB-BDST Algorithm 3 Results 4 Conclusion Conclustion Ant-based algorithm for the BDMST Problem Uses two species of ants Clustering approach to break down the BDMST problem into sub-problems Uses pheromone to cluster vertices and construct BDSTs Local optimization methods are used to improve solution quality Performs well against existing algorithms Future Work Combining the cluster BDSTs in more of a Kruskal s approach Incorporate the ideas of path relinking Constructing a distributed version of the algorithm focused on improving solution quality
Thank You