Ant Algorithms. Simulated Ant Colonies for Optimization Problems. Daniel Bauer July 6, 2006

Transcription

1 Simulated Ant Colonies for Optimization Problems July 6, 2006

2 Topics 1 Real Ant Colonies Behaviour of Real Ants Pheromones 2 3

3 Behaviour of Real Ants Pheromones Introduction Observation: Ants living in colonies take the shortest route e.g. from the anthill to a source of food This way commonly known ant highways emerge But Ants are blind Ants have very limited communication capabilities How can complex behaviour emerge?

4 Behaviour of Real Ants Pheromones Behaviour of Real Ants Anthill Sugar

5 Behaviour of Real Ants Pheromones Double Bridging Experiments Deneubourg, Gross et al. 1989, cm anthill 30 cm sugar anthill sugar

6 Behaviour of Real Ants Pheromones Pheromones Communication regarding path is performed by use of pheromone trails: Isolated ants move randomly (exploring the environment) A moving ant lays down portions of pheromone (chemical marker substance) and leaves a trail Other ants discovering pheromone follow the strongest trail more probably and reinforces the trail with it s own pheromone The more ants follow a trail, the more attractive it becomes to follow

7 Behaviour of Real Ants Pheromones Pheromones Simple model: An ant follows a path with a probability proportional to the amount of pheromone on it.

8 Behaviour of Real Ants Pheromones So far... Individual ants are blind and dumb Ants complex behaviour emerges from social interaction Communication is performed solely by pheromone trails Ant colonies are able to solve certain problems How can we exploit this mechanism in computation?

9 Real Ant Colonies Idea: Use simulated ant colonies to find shortest path in graphs. source (anthill) destination (source of food) Simulated ants live in a descrete time environment They move from one node to an adjacent one in one step They leave a pheromone trail on the arc dependent on the path costs. (This is problematic, why?)

10 Path Memory Real Ant Colonies source (anthill) destination (source of food) Problem: Cycles may occur Solution: Ants have memory to store their path

11 Backward Movement and Pheromone evaporation Path memory also allows the ant to return to the source on the same way Idea: Ants find forward solutions probabilistically without putting down pheromone If a solution was found (the ant reaches the destination) they deterministically move backward along their path While in backward mode they leave pheromone on arcs depending on the value of the solution We want pheromones to evaporate after some time This Algorithm is called

12 - Terminology Real Ant Colonies Graph: G = (N, A) each arc (i, j) A is associated with it s artificial pheromone trail τ ij τ ij is proportional to the utility of using the arc (i, j) to build 1 a good solution. Choose update: C k = P pathcosts for ant k Neighbourhood of ant k in node i: Ni k all nodes which are adjacent to i but not the one where the ant came from Pheromone evaporation parameter ρ (0, 1]

13 - Algorithm Real Ant Colonies Initialize pheromone trails (i, j) Aτ ij 1 In each turn: for each ant k: If k is in forward mode: Go from current node i to j with probability pij k τ = P ij if j Ni k else pij k = 0 l N k i τ il Store arc used and it s costs to k s path memory Switch to backward mode if destination is reached, eliminate cycles from path memory If k is in backward mode: Go back one step deterministically (look up previous used arc in the path memory) Update pheromone on the path for an ant k: τ ij τ ij + C k Switch to forward mode if source is reached. Calculate pheromone evaporation: (i, j) A : τ ij τ ij (1 ρ) τ ij until convergence

14 - Example Real Ant Colonies applied to the double bridging experiment ants 15 ants 10 ants ants ants 30 ants

15 Performance of Convergence cannot be guaranteed for all problems Performance of the Algorithms is highly dependent on Parameters Only for a great number of ants a solution is found quickly in more complex problems Pheromone evaporation is important to avoid stagnation behaviour There are polynomial time algorithms which find shortest path more efficiently can be seen as an didactic example, not as an applicable algorithm.

16 Combinatorical Optimization Problems Main idea: From a set of entities find a subset that optimizes a cost function while meeting constraints. Examples: graph coloring job scheduling knapsack problem travelling salesman CO-Problems tend to be N P-hard (all of the above problems are N P-Complete) Most of the time we can just approximate optimal solutions

17 Heuristics and Metaheuristics Heuristics are empirical or rule-of-thumb informations used to guide searches for solutions Example: always prefer the shortest arc when searching a shortest path Metaheuristics are heuristics that can be applied in a range of problems ACO can be seen as a metaheuristic for optimization problems other examples: local search, simulated annealing

18 (AS) Real Ant Colonies (AS) (Dorigo et al. 1996) was one of the first ACO algorithms It can be seen as a generalization/extension of It was first applied to the Traveling Salesman Problem

19 Traveling Salesman Problem Given a set of n cities In TSP a solution is any circular tour including all cities find the shortest circular tour formalization: completely connected graph with annotated path costs (distance) G=(N,A) 3 Idea: Let ants generate tours, then update pheromone trails dependent on length of tour

20 - Start Cities and Number of Ants In the completely connected graph representing a TSP there is no unique source Ants can start from an arbitrary city Ants in city i: b i Total number of ants: m= n i=1 b i

21 cont. Real Ant Colonies Ants perform cycle detection on-line Path memory is called tabu list: tabu k Allowed successor cities: A = N\tabu k All ants move simultaniously After each cycle of the algorithm the shortest path is remembered

22 - Move Probability Ants in the AS are not totally blind They use the shortest arc distance as a heuristic probability is calculated by integrating weightet pheromone trail and heuristic heuristic arc value of arc (i, j) A: η ij = 1/d ij Probability for ant k of choosing arc (i, j): if j A k j : pk ij = τij α ηβ ij P l A k l τil α η β il else pij k = 0 α and β are parameters to be choosen.

23 - Pheromone Update and Evaporation Pheromone is not initialized with 1, but with a small constant c If the ants have completed a tour each (a cycle is finished) the pheromone is updated. The update process is performed for all ants simultaniously Evaporation is included in the update process Update for all arcs (i, j) A: τ ij ρ τ ij + τ ij with τ ij = k ants τ ij k and pheromone laid on arc (i,j) if visited by ant k (else τ ij = 0): τij k Q = P pathcosts of ant k

24 - Algorithm Initialize pheromone trails (i, j) Aτ ij c Place the m ants on the n nodes, write start city of ant k to tabu k In each cycle: for each ant k find a route: repeat Go from current node i to j with probability pij k Add city to tabu k until tabu k = n (route was completed) for all ants k calculate total route costs. for all ants k and all arcs (i, j) calculate τij k Calculate pheromone change τ ij for all arcs (i,j) Perform pheromone update (i, j): τ ij ρ τij + τ ij move all ants k to their starting city (from tabu k (n) to tabu k (1) remember shortest route empty tabu lists until convergence (all ants take same route or user defined no. of cycles) Output shortest route

25 - Parameters Performance of the Ant System depends on the choice of parameters Number of ants: m Initial pheromone trail: c Weighting of shortest path heuristic β and pheromone α Evaporation rate: ρ A constant related to the quantity of pheromone placed: Q

26 - Experimental Results Best values for ρ, α and β were found in experiments on a 30 city TSP (maximal 5000 cycles were performed): α β ρ development of best tour length:

27 - Experimental Results cont. Same values were applied to a simple 10 city problem.

28 vs. other heuristics Comparision of performance of different general purpose heuristics in the 30 city problem. Performance compared to other TSP-tailored heuristics:

29 Structure of the ACO Metaheuristic All ACO Algorithms share the same structure: ConstructAntSolutions UpdatePheromones DemonActions (optional) (e.g. remember shortest path...)

30 How to apply ACO to other optimization problems Since TSP is N P-hard any N P Problem can be reduced to it theoretically In practice it might be far easier to find a way to apply ACO metaheuristics directly to a certain problem We might even find a general procedure how to apply ACO to any CO-Problem

31 Combinatorical Optimization Problems again All Combinatorical Optimization Problems share the same structure: From a set of entities find a subset that optimizes a cost/value function while meeting constraints. General aproach: Generate subsets by finding path through a graph (called construction graph) If a solution was found (what we call a solution depends on the problem) evaluate it Then update pheromone trails

32 Multiple Knapsack Problem Given a knapsack with a number m of resource constraints j (e.g. room, weight...) with capacity a j and a set of items i I each with a value v i and recource requirements r j i Find a subset S of I that maximizes i S v i subject to the constraints j : i S r ij a j Application of ACO by Leguizamon and Michalewicz (1999)

33 Multiple Knapsack Problem - Graph Each item is a node in the totally connected Graph G=(I,A) Item nodes are annotated their value and their resource requirements While building solutions ants are only allowed to proceed to nodes which do not violate the constraints. If no allowed successor node was found the ant has found a solution

34 Multiple Knapsack Problem - Pheromone and Heuristics Pheromone Trails are associated with items, not with arcs. τ i = q i S v i (weighting constant q (0, 1]) Heuristics: items with high value and low requirements should be prefered average tightness of resource constraints r i = 1 m m a j j=1 r ij Heuristic η i = v i r i

35 Are Simulated Ants Agents? Ants presented so far do not correspond to our definition of an agent Simulated ants are not intelligent at all Simulated ants are not self interested The environment is far to simple to be called environment

36 Routing in Computer Networks Ants are processes traveling between machines They might also have to perform other tasks (get offers, buy items...) They are self interested, they want to find the shortest path If in backward mode they leave pheromone on each machine as a hint for ants with the same destination. The environment they live in is highly complex (and changes dynamically) they can be called mobile agents (which inherit properties of ants)

37 Parallel Processing Real Ant Colonies Ants can move sequentially (in a loop) or syncroniously Ants have process character ACO algorithms can be parallelized fine grained parallelization: Individual ants have their own processor coarse grained parallelization: Parts of the population have their own processor

38 Advantages of ACO Algorithms ACO is robust and versatile It can be applied to a number of optimization problems It can be applied to different versions of the problem Application can be done in a straightforward way Algorithms shows good performance compared to other metaheuristics The population based approach can be exploitet in parallel implementations

39 Behaviour of natural populations can be an inspiration for new algorithms Algorithms based on artificial ant colonies can solve shortest path problems or provide a meta heuristic in a range of optimization problems Their performance depends on the choice of several parameters

40 References M. Dorigo et al., : Optimization by a colony of cooperating agents, IEEE Transactions on Systems, Man, and CyberneticsPart B, Vol.26, No.1, 1996, pp.1-13 M. Dorigo, T. Sttzle. Ant Colony Optimization. MIT Press, Cambridge, M. Dorigo, Solve Difficult Optimization Problems, 2001 ( meta/newsite/downloads/dorigo- ECAL2001.pdf)