METAHEURISTIC Jacques A. Ferland Department of Informatique and Recherche Opérationnelle Université de Montréal ferland@iro.umontreal.ca March 2015
Overview Heuristic Constructive Techniques: Generate a good solution Local (Neighborhood) Search Methods (LSM): Iterative procedure improving a solution Population-based Methods: Population of solutions evolving to mimic a natural process
Population-based Methods Genetic Algorithm (GA) Hybrid GA-LSM Genetic Programming (GP) Adaptive Genetic Programming (AGP) Scatter Search (SS) Ant Colony Algorithm (ACA) Particle Swarm Optimiser (PSO)
Genetic Algorithm (GA) Population based algorithm At each generation three different operators are first applied to generate a set of new (offspring) solutions using the N solutions of the current population P: selection operator: selecting from the current population parent-solutions that reproduce themselves crossover (reproduction) operator: producing offspring-solutions from each pair of parent-solutions mutation operator: modifying (improving) individual offspring-solution A fourth operator (culling operator) is applied to determine a new population of size N by selecting among the solutions of the current population and the offspring-solutions according to some strategy
Two variants of GA At each generation of the Classical genetic algorithm: - N parent solutions are selected and paired two by two - A crossover operator is applied to each pair of parent-solutions according to some probability to generate two offspring-solutions. Otherwise the two parent-solutions become their own offspring-solutions - A mutation operator is applied according to some probability to each offspring-solution. - The population of the next generation includes the offspring-solutions At each generation of the Steady-state population genetic algorithm: - An even number of parent-solutions are selected and paired two by two - A crossover operator is applied to each pair of parent-solutions to generate two offspring-solutions. - A mutation operator is applied according to some probability to each offspring-solution. -The population of the next generation includes the N best solutions among the current population and the offspringsolutions
Problem used to illustrate General problem min x X f ( x) Assignment type problem: Assignment of resources j to activities i min f m j= 1 ( x) s.t. x = 1 i = 1,, n ij x = 0 or 1 i = 1,, n ; j = 1,, m ij
Encoding the solution The phenotype form of the solution x єr p is encoded (represented) as a genotype form vector z єr n (or chromozome) where m may be different from n. j For example in the assignment type problem: let x be the following solution: for each 1 i n, x ij(i) = 1 x ij = 0 for all other j Assigning resources to activities i min f ( x) m Subject to x = 1 i = 1,, n j= 1 ij x = 0 or 1 i = 1,, n ij j = 1,, m x єr nxm can be encoded as z єr n where z i = j(i) i = 1, 2,, n i.e., z i is the index of the resource j(i) assigned to activity i
Genetic Algorithm (GA) Population based algorithm At each generation three different operators are first applied to generate a set of new (offspring) solutions using the N solutions of the current population P: selection operator: selecting from the current population parent-solutions that reproduce themselves crossover (reproduction) operator: producing offspring-solutions from each pair of parent-solutions mutation operator: modifying (improving) individual offspring-solution A fourth operator (culling operator) is applied to determine a new population of size N by selecting among the solutions of the current population and the offspring-solutions according to some strategy
Selection operator This operator is used to select an even number (2, or 4, or, or N) of parent-solutions. Each parent-solution is selected from the current population according to some strategy or selection operator. Note that the same solution can be selected more than once. The parent-solutions are paired two by two to reproduce themselves. Selection operators: Random selection operator Proportional (or roulette whell) selection operator Tournament selection operator Diversity preserving selection operator
Random selection operator Select randomly each parent-solution from the current (entire) population Properties: Very straightforward Promotes diversity of the population generated
Proportional (Roulette whell) selection operator Each parent-solution is selected as follows: i) Consider any ordering of the solutions z 1, z 2,, z N in P ii) Select a random number α in the interval iii) Let τ be the smallest index such that iv) Then z τ is selected [0, 1 k N ( 1 / f( z k ) )] 1 k τ (1 / f( z k ) ) α 1 / f( z 1 ) 1 / f( z 2 ) 1 / f( z 3 ) 1 / f( z N ) α τ The chance of selecting z k increases with its fittness 1 / f( z k ) For the problem min f (x) where x is encoded as z 1/f (z) measures the fittness of the solution z
Tournament selection operator Each parent-solution is selected as the best solution in a subset of randomly chosen solutions in P: i) Select randomly N solutions one by one from P (i.e., the same solution can be selected more than once) to generate the subset P ii) Let z be the best solution in the subset P : z = argmin zєp f(z) iii) Then z is selected as a parent-solution
Elitist selection The main drawback of using elitist selection operator like Roulette whell and Tournament selection operators is premature converge of the algorithm to a population of almost identical solutions far from being optimal. Other selection operators have been proposed where the degree of elitism is in some sense proportional to the diversity of the population.
Genetic Algorithm (GA) Population based algorithm At each generation three different operators are first applied to generate a set of new (offspring) solutions using the N solutions of the current population P: selection operator: selecting from the current population parent-solutions that reproduce themselves crossover (reproduction) operator: producing offspring-solutions from each pair of parent-solutions mutation operator: modifying (improving) individual offspring-solution A fourth operator (culling operator) is applied to determine a new population of size N by selecting among the solutions of the current population and the offspring-solutions according to some strategy
Crossover (recombination) operators Crossover operator is used to generate new solutions including interesting components contained in different solutions of the current population. The objective is to guide the search toward promissing regions of the feasible domain X while maintaining some level of diversity in the population. Pairs of parent-solutions are combined to generate offspringsolutions according to different crossover (recombination) operators.
One point crossover The one point crossover generates two offspring-solutions from the two parent-solutions z 1 = [ z 11, z 21,, z n1 ] z 2 = [ z 12, z 22,, z n2 ] as follows: i) Select randomly a position (index) ρ, 1 ρ n. ii) Then the offspring-solutions are specified as follows: oz 1 = [ z 11, z 21,, z ρ1, z ρ+12,, z n2 ] oz 2 = [ z 12, z 22,, z ρ2, z ρ+11,, z n1 ] Hence the first ρ components of offspring oz 1 (offspring oz 2 ) are the corresponding ones of parent 1 (parent 2), and the rest of the components are the corresponding ones of parent 2 (parent 1)
Two points crossover The two points crossover generates two offspring-solutions from the two parent-solutions z 1 = [ z 11, z 21,, z n1 ] z 2 = [ z 12, z 22,, z n2 ] as follows: i) Select randomly two positions (indices) µ,ν, 1 µ ν n. ii) Then the offspring-soltions are specified as follows: oz 1 = [ z 11,, z µ-11, z µ2,, z ν2, z ν+11,, z n1 ] oz 2 = [ z 12,, z µ-12, z µ1,, z ν1, z ν+12,, z n2 ] Hence the offspring oz 1 (offspring oz 2 ) has components µ, µ+1,, ν of parent 2 (parent 1), and the rest of the components are the corresponding ones of parent 1 (parent 2)
Uniform crossover The uniform crossover requires a vector of bits (0 or 1) of dimension n to generate two offspring-solutions from the two parent-solutions z 1 = [ z 11, z 21,, z n1 ], z 2 = [ z 12, z 22,, z n2 ] : i) Generate randomly a vector of n bits, for example [0, 1, 1, 0,, 1, 0] ii) Then the offspring-solutions are specified as follows: parent 1: [ z 11, z 21, z 31, z 41,, z n-11, z n1 ] parent 2: [ z 12, z 22, z 32, z 42,, z n-12, z n2 ] Vector of bits: [ 0, 1, 1, 0,, 1, 0 ] Offspring oz 1 : [ z 11, z 22, z 32, z 41,, z n-12, z n1 ] Offspring oz 2 : [ z 12, z 21, z 31, z 42,, z n-11, z n2 ]
Uniform crossover The uniform crossover requires a vector of bits (0 or 1) of dimension m to generate two offspring-solutions from the two parent-solutions z 1 = [ z 11, z 21,, z n1 ], z 2 = [ z 12, z 22,, z n2 ] : i) Generate randomly a vector of bits, for example [0, 1, 1, 0,, 1, 0] ii) Then the offspring-solutions are specified as follows: parent 1: [ z 11, z 21, z 31, z 41,, z n-11, z n1 ] parent 2: [ z 12, z 22, z 32, z 42,, z n-12, z n2 ] Vector of bits: [ 0, 1, 1, 0,, 1, 0 ] Offspring oz 1 : [ z 11, z 22, z 32, z 41,, z n-12, z n1 ] Offspring oz 2 : [ z 12, z 21, z 31, z 42,, z n-11, z n2 ] Hence the i th component of oz 1 (oz 2 ) is the i th component of parent 1 (parent 2) if the i th component of the vector of bits is 0, otherwise, it is equal to the i th component of parent 2 (parent 1)
( z ) Let d z, the Hamming distance between z and z : ( ) d z, z = number of components of z and z that are different. Path relinking 1 ( z ) Denote by N the set of feasible solution obtained 1 by modifying slightly z. The path relinking procedure generates a sequence of feasible solutions along a path linking two 1 solutions x x 2 z and z i Path relinking procedure: 1. ps = z and ps = z 1 2 ( ( 1 ) ( 2 )) 2. Use the roulette whell selection operator to select z N p s, p s If f ( z) < f ( z ) and f ( z) < f ( z ), stop with z 2 3. Otherwise ps = ps, and ps = z ( ) Let also 4. If d ps, ps = 0, then stop with z; otherwise repeat 2 { } ( 1, 2 ) = ( 1 ) : (, 2 ) < ( 1, 2 ) N ps ps z N ps d z ps d ps ps
1. ps = z and ps = z 1 2 If ( ) < ( ) and ( ) < ( ), stop with 2 ps ps ps z 4. If d( ps, ps ) 0, then ( ( ) ( )) 2. Use the roulette whell selection operator to select z N p s, p s f z f z f z f z z 3. Otherwise =, and = = stop with z; otherwise repeat 2 1 z 2 z
1. ps = z and ps = z 1 2 If ( ) < ( ) and ( ) < ( ), stop with 2 ps ps ps z 4. If d( ps, ps ) 0, then ( ( ) ( )) 2. Use the roulette whell selection operator to select z N p s, p s f z f z f z f z z 3. Otherwise =, and = = stop with z; otherwise repeat 2 1 ps = 1 z 2 ps = 2 z
Let also ( 1, 2 ) = { ( 1 ) : (, 2 ) < ( 1, 2 )} N ps ps z N ps d z ps d ps ps 1. ps = z and ps = z 1 2 If ( ) < ( ) and ( ) < ( ), stop with 2 ps ps ps z 4. If d( ps, ps ) 0, then ( ( ) ( )) 2. Use the roulette whell selection operator to select z N p s, p s f z f z f z f z z 3. Otherwise =, and = = stop with z; otherwise repeat 2 1 ps = 1 z 2 ps = 2 z
1. ps = z and ps = z 1 2 If ( ) < ( ) and ( ) < ( ), stop with 2 ps ps ps z 4. If d( ps, ps ) 0, then ( ( ) ( )) 2. Use the roulette whell selection operator to select z N p s, p s f z f z f z f z z 3. Otherwise =, and = = stop with z; otherwise repeat 2 1 ps = 1 z z 2 ps = 2 z
1. ps = z and ps = z 1 2 If ( ) < ( ) and ( ) < ( ), stop with 2 ps ps ps z 4. If d( ps, ps ) 0, then ( ( ) ( )) 2. Use the roulette whell selection operator to select z N p s, p s f z f z f z f z z 3. Otherwise =, and = = stop with z; otherwise repeat 2 1 z 2 ps 1 ps = 2 z
Let also ( 1, 2 ) = { ( 1 ) : (, 2 ) < ( 1, 2 )} N ps ps z N ps d z ps d ps ps 1. ps = z and ps = z 1 2 If ( ) < ( ) and ( ) < ( ), stop with 2 ps ps ps z 4. If d( ps, ps ) 0, then ( ( ) ( )) 2. Use the roulette whell selection operator to select z N p s, p s f z f z f z f z z 3. Otherwise =, and = = stop with z; otherwise repeat 2 1 z 2 ps 1 ps = 2 z
1. ps = z and ps = z 1 2 If ( ) < ( ) and ( ) < ( ), stop with 2 ps ps ps z 4. If d( ps, ps ) 0, then ( ( ) ( )) 2. Use the roulette whell selection operator to select z N p s, p s f z f z f z f z z 3. Otherwise =, and = = stop with z; otherwise repeat 2 1 z 2 ps 1 ps = 2 z z
1. ps = z and ps = z 1 2 If ( ) < ( ) and ( ) < ( ), stop with 2 ps ps ps z 4. If d( ps, ps ) 0, then ( ( ) ( )) 2. Use the roulette whell selection operator to select z N p s, p s f z f z f z f z z 3. Otherwise =, and = = stop with z; otherwise repeat 2 1 z 1 ps 2 z 2 ps
Let also ( 1, 2 ) = { ( 1 ) : (, 2 ) < ( 1, 2 )} N ps ps z N ps d z ps d ps ps 1. ps = z and ps = z 1 2 If ( ) < ( ) and ( ) < ( ), stop with 2 ps ps ps z 4. If d( ps, ps ) 0, then ( ( ) ( )) 2. Use the roulette whell selection operator to select z N p s, p s f z f z f z f z z 3. Otherwise =, and = = stop with z; otherwise repeat 2 1 z 1 ps 2 z 2 ps
1. ps = z and ps = z 1 2 If ( ) < ( ) and ( ) < ( ), stop with 2 ps ps ps z 4. If d( ps, ps ) 0, then ( ( ) ( )) 2. Use the roulette whell selection operator to select z N p s, p s f z f z f z f z z 3. Otherwise =, and = = stop with z; otherwise repeat 2 1 z 1 ps z 2 z 2 ps
Ad hoc crossover operator The preceding crossover operators are sometimes too general to be efficient. Hence, whenever possible, we should rely on the structure of the problem to specify ad hoc problem dependent crossover operator in order to improve the efficiency of the algorithm.
Recovery procedure Furthermore, whenever the structure of the problem is such that the offspring-solutions are not necessarily feasible, then an auxiliary procedure is required to recover feasibility. Such a procedure is used to transform the offspring-solution into a feasible solution in its neighborhood.
Genetic Algorithm (GA) Population based algorithm At each generation three different operators are first applied to generate a set of new (offspring) solutions using the N solutions of the current population P: selection operator: selecting from the current population parent-solutions that reproduce themselves crossover (reproduction) operator: producing offspring-solutions from each pair of parent-solutions mutation operator: modifying (improving) individual offspring-solution A fourth operator (culling operator) is applied to determine a new population of size N by selecting among the solutions of the current population and the offspring-solutions according to some strategy
Mutation operator Mutation operator is an individual process to modify offspring-solutions In traditional variants of Genetic Algorithm the mutation operator is used to modify arbitrarely each componenet z i with a small probability: For i = 1 to n Generate a random numberβ [0, 1] If β < βmax then select randomly a new value for z i where βmax is small enough in order to modify z i with a small probability Mutation operator simulates random events perturbating the natural evolution process Mutation operator not essential, but the randomness that it introduces in the process, promotes diversity in the current population and may prevent premature convergence to a bad local minimum
Hybrid GA - LSM
Population based algorithm Genetic Hybrid Algorithm GA - LSM(GA) At each generation three different operators are first applied to generate a set of new (offspring) solutions using the N solutions of the current population P: selection operator: selecting from the current population parent-solutions that reproduce themselves crossover (reproduction) operator: producing offspring-solutions from each pair of parent-solutions mutation operator: modifying improve each (improving) offspring-solution individual using offspring-solution a LSM A fourth operator (culling operator) is applied to determine a new population of size N by selecting among the solutions of the current population and the offspring-solutions according to some strategy
Hybrid GA Methods - LSM This is a good strategy since it is well known that in general, Genetic Algorithms GA(and population based algorithms in general) are very time consuming and generate worse solutions than LSM Strength of hybrid methods comes from combining complementary search strategy to take advantage of their respective strength. For instance, - Intensify the search in a promissing region with the LSM - Diversify the search through the selection operator, crossover operator of the GA
Genetic Programming ( GP )
Genetic Programming Algorithm (GA) GP ( ) Population based algorithm i At each generation, use the solutions in the current population P to generate At each offspring generation - solutions three different to create operators the population are first of applied the next to generate generation a : set of new (offspring) solutions using the N solutions of the current Repeat population the following P: process until N offspring-solutions are generated: Select ran domly a generation strategy gs GS : selection operator: selecting from the current population parent-solutions Select parent-solu that reproduce tions in P themselves for applying the generation strategy crossover gs (reproduction) GS : operator: producing offspring-solutions from each Generate offspring-solutions pair parent-solutions to be included in the intermediary mutation population operator: P modifying (improving) individual offspring-solution A fourth operator (culling ( operator) rator is ) applied is applied to to determine a new a new population of size Nby by selecting among solutions the solutions in P P of the according current to some population strateg and y the offspring-solutions according to some strategy
Genetic Programming Algorithm (GA) GP ( ) Population based algorithm i At each generation, use the solutions in the current population P to generate At each offspring generation - solutions three different to create operators the population are first of applied the next to generate generation a : set of new (offspring) solutions using the N solutions of the current Repeat population the following P: process until N offspring-solutions are generated: Select ran domly a generation strategy gs GS : selection operator: selecting from the current population parent-solutions Select parent-solu that reproduce tions in P themselves for applying the generation strategy crossover gs (reproduction) GS : operator: producing offspring-solutions from each Generate offspring-solutions pair parent-solutions to be included in the intermediary mutation population operator: P modifying (improving) individual offspring-solution A fourth operator ((culling operator) ) is is applied to to determine a new a new population of size N by selecting solutions among the in Psolutions P according of the to current population some strategand y the offspring-solutions according to some strategy
Generation strategies GS i i i i i Transfer some of the best parent-solutions of P in P Some iterations of a local search method on a parent-solution Crossover on a pair of parent-solutions Crossover on a pair of parent-solutions + mutation on offsring-solutions Mutation on a parent-solution Descent method Tabu Search Simulated annealing Large Multiple-Neighborhood Search Adaptive Large neighborhood Seach One point crossover Two points crossover Uniform crossover Ad hoc crossover Path relinking
Genetic Programming Algorithm (GA) GP ( ) Population based algorithm i At each generation, use the solutions in the current population P to generate At each offspring generation - solutions three different to create operators the population are first of applied the next to generate generation a : set of new (offspring) solutions using the N solutions of the current Repeat population the following P: process until N offspring-solutions are generated: Select ran domly a generation strategy gs GS : selection operator: selecting from the current population parent-solutions Select parent-solu that reproduce tions in P themselves for applying the generation strategy crossover gs (reproduction) GS : operator: producing offspring-solutions from each Generate offspring-solutions pair parent-solutions to be included in the intermediary mutation population operator: P modifying (improving) individual offspring-solution A fourth operator ((culling operator) ) is is applied to to determine a new a new population of size N by selecting solutions among the in Psolutions P according of the to current population some strategand y the offspring-solutions according to some strategy
Parent - solution selection Selection operators (like in GA): Random selection operator Proportional (or roulette whell) selection operator Tournament selection operator Diversity preserving selection operator
Genetic Programming Algorithm (GA) GP ( ) Population based algorithm i At each generation, use the solutions in the current population P to generate At each offspring generation - solutions three different to create operators the population are first of applied the next to generate generation a : set of new (offspring) solutions using the N solutions of the current Repeat population the following P: process until N offspring-solutions are generated: Select ran domly a generation strategy gs GS : selection operator: selecting from the current population parent-solutions Select parent-solu that reproduce tions in P themselves for applying the generation strategy crossover gs (reproduction) GS : operator: producing offspring-solutions from each Generate offspring-solutions pair parent-solutions to be included in the intermediary mutation population operator: P modifying (improving) individual offspring-solution A fourth operator ((culling operator) ) is is applied to to determine a new a new population of size N by selecting solutions among the in Psolutions P according of the to current population some strategand y the offspring-solutions according to some strategy
Generation strategies GS i i i i i Transfer some of the best parent-solutions of P in P Some iterations of a local search method on a parent-solution Crossover on a pair of parent-solutions Crossover on a pair of parent-solutions + mutation on offsring-solutions Mutation on a parent-solution Descent method One point crossover Tabu Search Genetic Algorithm ( GA Two ) points crossover Simulated annealing Uniform crossover Large Multiple-Neighborhood Search Ad hoc crossover Adaptive Large neighborhood Seach Path relinking
Adaptive Genetic Programming AGP ( )
Adaptive Genetic Programming Algorithm ming (GA) AGP (( )) Population based algorithm i At each generation, use the solutions in the current population P to generate At each offspring generation - solutions three different to create operators the population are first of applied the next to generate generation a P : set of new (offspring) solutions using the N solutions of the current Repeat population the following P: process until N offspring-solutions are generated: Select randomly a generation strategy y gs GS : according to its score π selection operator: selecting from the current population parent-solutions gs : Select parent-solu that reproduce tions in P themselves for Adaptive applying layer the : generation strategy crossover gs (reproduction) GS : operator: producing offspring-solutions π from each j pair parent-solutions Probabilistic choices on to specify Generate offspring-solutions to be included in the intermediary π i mutation population operator: P modifying (improving) individual offspring-solution the intervals in the roulette wheel selection A fourth operator ((culling operator) ) is is applied to to determine a new a new population of size N by selecting solutions among the in Psolutions P according of the to current population some strategand y the offspring-solutions according to some strategy
Adaptive Genetic Programming Algorithm ming (GA) AGP (( )) Population based algorithm i At each generation, use the solutions in the current population P to generate At each offspring generation - solutions three different to create operators the population are first of applied the next to generate generation a P : set of new (offspring) solutions using the N solutions of the current Repeat population the following P: process until N offspring-solutions are generated: Select randomly a generation strategy y gs GS : according to its score π selection operator: selecting from the current population parent-solutions gs : Select parent-solu that reproduce tions in P themselves for applying the generation strategy crossover gs (reproduction) GS : operator: producing offspring-solutions Score: from each Generate offspring-solutions pair parent-solutions to be included Past in performance the intermediary to contribute mutation population operator: P modifying (improving) individual during offspring-solution the solution ocess A fourth operator ((culling operator) ) is is applied to to determine a new a new population of size N by selecting solutions among the in Psolutions P according of the to current population some strategand y the offspring-solutions according to some strategy
Adaptive Genetic Programming Algorithm i (GA) AGP (( )) Population based algorithm i At each generation, use the solutions in the current population P to generate At each offspring generation - solutions three different to create operators the population are first of applied the next to generate generation a P : set of new (offspring) solutions using the N solutions of the current Repeat population the following P: process until N offspring-solutions are generated: Score update: Select randomly a generation strategy y gs GS : according to its score π selection operator: selecting from the current population parent-solutions gs : New best solution Select parent-solu that reproduce tions in P themselves for applying the generation strategy Better solution crossover gs than (reproduction) GSparent-solutions : operator: producing offspring-solutions from each Not previously Generate visited solutions offspring-solutions pair parent-solutions to be included in the intermediary Worse mutation solution population than operator: paren P t-solut modifying ions (improving) individual offspring-solution Update the score π gs A fourth operator ((culling operator) ) is is applied Sc to ore to determine update pr a ocess: new a new population of size N by selecting solutions among the in Psolutions π P according of the to i: the score obtained current at population some strategand y the offspring-solutions according the to current some iter strategy ation Then ( ) π : = ρπ + 1 ρ π i i i