k b n g c i g m p k d g

Transcription

1 Optimization Using a Host-Parasite Model with Variable-Size Distributed Populations Bjorn Olsson Department of Computer Science University of Skovde Box 08, S- 8, Sweden bjorne@ida.his.se Abstract This paper presents a model of coevolution between two variable-size, spatially distributed populations, evolving in an environment with a ow of resources. The two populations have a host-parasite relationship where the host species is dependent on the uptake of resources from the environment for their reproduction. The parasites seek to \infect" host organisms and parasitize on their resources to produce parasite ospring. We show how the approach can be used for optimization tasks by using instances of the problem task to determine the outcome of each interaction between a host and a parasite organism. As an initial test of the model, we apply it to the problem of designing sorting networks for several problem sizes: 6, 7, 8, and 9-input. I. Introduction We attempt to model the coevolution of host and parasite species, where both species have variable size populations which evolve in a two-dimensional \world". The \world" contains resources which are absorbed by host organisms and used in their reproduction. Parasite organisms attempt to \infect" hosts and make use of the hosts' resources to produce parasite ospring. Parasites are dependent on hosts, since they have no means of their own to absorb resources directly from the environment. We show that this model can be used for optimization by assigning to one species the task of evolving optimized solutions and to the other the task of evolving test cases. This idea was explored successfully by Hillis in [], which also concerned optimization using a model of host-parasite coevolution. Our model is dierent in a number of ways: We let populations evolve in an environment with explicit resources, we let interaction between hosts and parasites occur in a less controlled manner, and we use variable size populations, allowing parasites to \kill" the hosts they successfully infect. Our model is also dierent in that the use of explicit tness functions has been abandoned altogether. There are a number of models in the Articial Life community of distributed agents evolving in an environment with resources. Examples include the Echo [], LEE [6] and Avida [] models. We are unaware, however, of any work on adapting such models to perform formally dened optimization tasks. In this paper, we attempt to show empirically that this type of Articial Life model can be used In: \Proceedings of the 996 IEEE International Conference on Evolutionary Computation", Nagoya, 996. for optimization purposes. II. Distributed Host-Parasite Coevolution A. Overview The environment consists of a two-dimensional grid with wrap-around. On this grid we evolve two populations of \organisms" - a host population and a parasite population. Host organisms are movable, whereas parasites have no ability to move independently. Each host organism tries to make a one-step movement in a randomly chosen direction for each time step of the simulation. The movement succeeds if the adjacent grid cell in the chosen direction is empty, otherwise the host just stays in its present grid cell until the next time step. In addition to the environment there is a store of \resources", in the form of a number of \resource particles". Resources are poured into the environment at a particular rate and can be absorbed from the environment by host organisms. A host needs to have absorbed a certain number of resource particles before it can reproduce. When reproducing, the host makes a mutated copy of itself and, in so doing, spends a number of resource particles which are returned to the store. Parasites have no ability to take up resources independently. Instead, parasites are able to \infect" hosts and use up the hosts' resources to produce parasite ospring. Whenever an organism dies, all resource particles it contains are returned to the store. Distribution of particles from the store back to the environment are regulated so that a distribution event always results in the same amount (normally just a single particle) being distributed to every cell on the grid. Since reproduction takes place whenever an organism has access to the required resources, population sizes will vary during the simulation. A number of parameters dealing with resources are used to control the nature of the population dynamics. The total amount of available resources, together with the amounts required for reproduction, directly inuence the environment's carrying capacity. As another example, the amount of resources required for reproduction of a host organism and a parasite organism, respectively, inuence the population dynamics. It may be possible to use this model for optimization

2 purposes by assigning to the host population the task of evolving solutions to an optimization problem. In the example application described later in this paper, we have assigned to the host population the task of evolving sorting networks. We let each host encode a candidate sorting network and each parasite encode one or more test cases, i.e. input sequences. Each time a parasite attempts to infect a host we determine the outcome by testing the sorting network encoded by the host on the test case encoded by the parasite. The probability that the attempt to infect the host succeeds is determined by counting the number of misplaced integers in the sequence when it has been passed through the sorting network. B. Algorithm The execution of a simulation consists of the following steps:. Create an initial population of hosts consisting of H init individuals and place these in grid cells chosen at random with the restriction that no cell may contain more than one host. We have used H init = ; Create an initial population of parasites consisting of P init individuals and place these in grid cells chosen at random with the restriction that no cell may contain more than one parasite. We have used P init = ; Fill up the central store of resources C by letting C = R, where R denotes the total available number of resource units. We have used R = 0 6 for a grid of 0,000 cells.. Repeat steps to until at least one of the following is true: (a) the optimal sorting network for the given task has been found, (b) the maximum number of iterations have been executed, (c) host population size = 0, (d) parasite population size = 0.. Let each host absorb all resources from the grid cell which it currently occupies. 6. Let each host make one attempt to move to the adjacent grid cell in a randomly chosen direction. The move succeeds if the chosen cell is not occupied by another host, and fails otherwise. 7. For each parasite p i that is occupying the same grid cell as an uninfected host h, let p i make one attempt at \infecting" h. The probability that the attempt succeeds, P Inf (\probability of infection") is determined by executing interaction between host and parasite according to the problem-specic interaction function. This function should build a candidate solution to the problem from the host genome and a set of test cases from the parasite genome. It should return a value for P Inf, based on how well the candidate solution performs on the test cases. 8. For each parasite, p, which is currently infecting a host h, let p reproduce if h contains a resource amount Rp. Parasite reproduction results in the creation of O p mutated copies of p, the death of p itself, and the death of h. For each ospring, a random grid cell within a distance of 0 cells from h and p is chosen. The ospring is placed in the chosen cell unless the cell already contains a parasite. If no free cell has been found within a xed number of trials, the ospring dies. We have used Rp = 0 and O p =. 9. Let each host \die" with a probability of P d H. A host that \dies" is removed from the population and all the resources it contains are returned to C. If the host is currently infected by a parasite, the parasite also \dies". We have used P d H = 0:0. 0. Let each parasite \die" with a probability of P d P. A parasite that \dies" is just removed from the population (it does not contain any resources). We have used P d P = 0:0.. If C >= worldsize, where worldsize denotes the number of cells in the grid, add one unit of resource to each grid cell and then set C = C? worldsize. We have used worldsize = 0; For each host h which is not infected with a parasite, let h reproduce if it contains a resource amount Rh. Host reproduction results in a single mutated copy of h, and in splitting of the resources of h so that it keeps half of the amount for itself and donates the other half to the ospring. A random grid cell adjacent to h's cell is chosen and the ospring is placed in the chosen cell unless it already contains a host. If none of the cells adjacent to h's cell is unoccupied, the ospring dies. We have used Rh = 0. III. Example Application In this section, we rst introduce the example problem we have used. We then describe the problem-specic aspects added to the model in order to apply it to the example problem, i.e. the representation used for genetic encoding of candidate solutions and the interaction between hosts and parasites. A. Sorting Networks A sorting network is a sorting algorithm consisting of a xed sequence of comparison-swap operations. Such algorithms are called homogeneous [] or oblivious [0] since the sequence of comparisons is exactly the same for all inputs of a given size. A sorting network for input-sequences of length n can be represented graphically (see gure ) as n horizontal lines. Along the horizontal lines, from left to right, appears a number of vertical lines, each one connecting two horizontal ones. Each vertical line represents

3 a comparator, i.e. a comparison of the two numbers from the two horizontal lines which it connects. If the upper number is larger than the lower one, their positions are switched. Any sequence of n numbers which are input at the left end of the network should be sorted in increasing order after having passed through to the right end. The number of steps in the sorting procedure can be reduced if we allow adjacent non-overlapping comparisons to be executed in parallel. A typical optimization problem in connection with sorting networks is the design of a minimum-length n-input network, for a given value of n. By length we mean the number of comparators, whereas depth refers to the number of steps after parallelization. The example network in gure, which was evolved in one of our simulations, has length = 9 and depth =. appears in gure. Since the P set of possible comparators n? on the n-input problem is i= i, we have used genetic codes of arity,, 8, and 6 for, respectively, the 6, 7, 8, and 9-input problems. The length of the host chromosome is equal to the number of comparators used in the networks which we are trying to evolve, i.e. for the 8-input task, for example, we use host chromosomes with 9 loci. We have left as future work evaluation of possible advantages of the use of, for example, binary coding to exploit lower order schemas or building blocks []. 6 k b n g c i g m p k d g 6 Fig.. Each character in the host chromosome (top) represents a comparator. The probability of the parasite (left) \infecting" the host is a function of the number of misplaced integers in the output sequence (right). Parasite chromosomes may encode a number of input sequences. Fig.. Evolved 8-input sorting network using 9 comparators. The problem of designing a minimum-length sorting network for a given n becomes a formidable task already for relatively small values of n, due to the large search space. Consider, for example, the task of designing a 8- input network P of length 9. For each of the 9 comparators we have 7 i= i = 8 ways of connecting two of the input-lines. This gives a search space of 8 9 ' 0 7 networks. The optimal length of sorting networks is only known for n 8, whereas for n > 8 new current upper bounds are still being found. An example is the recent work of [], in which a new best network was found for n =. The problem of evolving minimum-length sorting networks can be approached in dierent ways, as discussed in []. We have chosen to use the lenghts of the currently known upper bounds (as listed in []) and try to evolve valid sorting networks of these lengths. We have chosen to let evolution work on xed length networks, searching for the network of given length with highest sorting accuracy. If the accuracy of the network is perfect, then it is a valid sorting network of the given length. This is contrary to the approach in [] and [], which allow variable-length networks during the optimization process. B. Representations and Genetic Operators Host chromosomes are sequences of characters, where each character represents a given comparator. An example Mutation of a host ospring consists of replacing the character in a single locus on the chromosome with another, randomly chosen character. We have chosen to let each reproduced chromosome be subjected to m point mutations, where m = 0 and m = each have a 0. probability and m > a 0. probability. For m >, we draw a random value with uniform distribution in the range ; ::; l, where l = chromosome length. Thereafter, m distinct loci are point mutated. This allows mutation to occasionally aect a large number of loci simultaneously - something that is very unlikely to occur in the standard approach where the probability of more than a few simultaneous mutations on the same chromosome is negligible. The genetic coding for parasite chromosomes uses the integers through n. In the initial parasite population, each parasite chromosome contains a single permutation of the integer sequence ; ::; n. Parasite mutation consists of exchanging the positions of two integers in the encoded sequence. In the simulations discussed below, we used a parasite mutation probability of 0. per locus. This means that during reproduction each locus had a 0. probability of having its integer value moved to another (random) locus and replaced by the integer value taken from the recieving locus. In addition to mutations, we use a form of recombination where genetic material is passed from one organism to another. We hesitate to call the recombination operator we use \crossover" for two reasons: Firstly, it is a one-

4 way process where genetic material from one organism is incorporated into the genome of another - rather than the mixing of genetic material from two organisms. Secondly, the recombination does not take place in conjunction with reproduction. We implement recombination as follows: At certain intervals (here every 00th time step), each host organism has a particular probability (here 0.0) of looking in its vicinity for another host. If it nds another host, it proceeds to copy a portion of its own chromosome. A random locus is chosen and either the portion preceding or following after the chosen locus is copied. The copied section is then introduced into the recipient host, replacing its previous chromosome segment in the corresponding loci sequence. This form of recombination is more similar to the conjugation [] which takes place between bacteria, than to the crossover of higher organisms. Given the chosen representation, we can design a function for the interaction between host and parasite individuals, as required in step 7 of the algorithmic description of the model. We design the function so that, given a parasite attempting to infect a host, we let the sorting network encoded by the host sort all the input-sequences encoded by the parasite. Each output sequence from the network is then examined and the total number of errors, N rw rong, counted. An error is a pair of integers n N ; n N+ in an integer sequence n ; ::; n nr inputs such that n N > n N+. We calculate the probability that the infection attempt succeeds as P Inf =, so that 0 P NrW rong n? Inf if the parasite genome encodes a single input sequence. During a simulation it would be impractical to use exhaustive testing on all input sequences as a tness function due to the large number of possible sequences. In this work we altogether abandon the idea of using tness functions and let the reproductive success of a host be implicitly determined by how well it survives interaction with the parasites it comes across. However, in order to gather data about the simulation we use a small random set of input sequences as a screening test applied to all host ospring and then only exhaustive testing on those that pass the screening. We thus get an estimate of the average sorting accuracy from the results of the screening tests and an estimate of the best found solution sofar from the exhaustive tests. Also, the exhaustive testing on those that pass the screening guarantees that if there is a valid sorting network in the population, then we will detect it. Note, however, that these tests are used for gathering statistics only. \Fitness" is determined implicitely by random interaction between hosts and parasites, where higher quality hosts have a lower probability of becoming infected. C. Comparison with Other Approaches Hillis [] focused on the problem of evolving 6-input sorting networks and, in the host-parasite version of his simulations, was able to evolve a 6-input sorting network of length 6, which is just one comparison more than the best known network for that problem size. Initial populations, in Hillis' work, were seeded with comparators which appear in many of the known 6-input sorting networks. Hillis used a Genetic Algorithm with the population distributed over a grid. In the non-coevolutionary version, tness of each network was determined by testing its sorting accuracy on a number of test cases, of which some were xed and some randomly generated. Only the best scoring half of the population was then allowed to contribute genetic material to the next generation, with crossovers taking place between networks located near each other on the grid. In the coevolutionary version, a population of parasites were introduced - each parasite coding for a set of 0 to 0 test cases. Fitness for a sorting network in a given cell on the grid was then determined by testing it on the sequences encoded by the parasite located in the same grid cell. Fitness for the parasite was determined by the number of test cases which the sorting network failed to sort. Selection and reproduction was, for both populations, carried out in the same way as in the non-coevolutionary version. One of the main dierences between our approach and that of Hillis is that we have altogether abandoned the familiar Genetic Algorithm concept of explicitly determining the tness of each individual and then using a selection mechanism to pick out high-tness individuals for reproduction. In our approach, a host organism which encounters a parasite runs a risk of being \infected" and \killed" by the parasite, unless it sorts the parasite's input sequence(s) correctly. Hosts representing better sorting networks will therefore on the average live longer and produce more ospring, although their tness is never evaluated explicitly. We rely instead on endogenous tness evaluation [8] [7]. This form of selection also results in variations in population size during our simulations. In initial time steps of our simulations this leads to the pattern of oscillations in population sizes typical of predator-prey coevolution [9]. Population sizes usually become stable towards the end of a simulation, however, as is exemplied in gure, although one may hypothesize that there are local oscillations on the grid also when the global population sizes appear stable. We have left as future work an evaluation of the impact of population size variations on the evolutionary process. D. Results Table shows results from 0 simulation runs each for the 6, 7, and 8-input problems. The results on the 9-input problem are from simulation runs only. The length, in number of comparators, of the sorting networks we tried to evolve were those listed in [] as the best currently known; opt. is the total number of permutations of the input se-

5 quence (optimum n = n!); avg. best is the number and percentage of permutations sorted correctly by the best evolved network, averaged over the 0 (or ) runs; nr. o.f. stands for number of runs in which an optimal sorting network (i.e. a valid sorting network of the given length) was found; time is the average number of time steps needed and nr. sol. the average number of candidate solutions evaluated in those simulations in which the optimum was found. Simulations with n = 6 were allowed to run for a maximum of,000 time steps, and this gure was doubled for each increment of n, except for n = 9, where a maximum of 00,000 time steps was allowed. Figure shows data from one of the simulations on the 7-input problem. There are initial oscillations in population sizes, which even out to relatively stable population sizes of,000 to,000 later in the run. After,96 time steps and evaluation of ' : 0 candidate solutions, a valid 7-input sorting network of length 6 is found. As table shows, an optimum (a valid sorting network of the given length) was found in each of the 0 runs on the 6-input problem. The 8-input problem was the hardest to solve, with only in 0 runs nding a valid sorting network. n length 6 9 opt. 70,00 0,0 6,880 avg 70,06 6,70 8,67 best 00% 99.7% 9.0% 98.8% nr o.f. 0 9 nr runs time,706 6,78 9,77 nr sol (0 ) Table Results for n = 6; ::; 9. IV. Conclusions and Future Work We have introduced a model of coevolving hosts and parasites, evolving under spatial distribution in an environment with a ow of resources. We have shown empirically that such models, although usually designed for ecological modeling purposes, can be applied to a formally dened optimization task. Much remains to be done on how the characteristics of this model can be applied to other problems. A goal of future analysis is to understand the eects that the population dynamics caused by coevolutionary relationships have on the evolutionary dynamics. Population dynamics has been studied extensively using dierential-equation based models such as Lotka-Volterra and variants thereof [9]. Much less is understood of how patterns of change in population sizes aect the pattern nr. prop. correct Population size Parasites Hosts time Best sofar Sorting accuracy Average time Fig.. Example simulation using n = 7. Note that 'best sofar' refers to the best sorting network of those which have passed a screening test and then been tested on all permutations of the input sequence, whereas 'average' is based on the results of the screening tests only. Thus, the average accuracy may occasionally exceed that of the 'best sofar'. of evolutionary/adaptive change in the species. There are few, if any, examples of evolutionary algorithms with variable population sizes being used for optimization tasks. How population dynamics can be exploited to improve the capacity for optimization is yet to be investigated. References [] Chris Adami and C. Titus Brown. Evolutionary learning in the d articial life system "avida". In Articial Life, 99. [] D. Hillis. Co-evolving parasites improve simulated evolution as an optimization procedure. In Articial Life II, 99. [] John H. Holland. Adaptation in Natural and Articial Systems: An Introductory Analysis with Applications to Biology, Control, and Articial Intelligence. MIT Press, nd edition, 99. [] Hugues Juille. Incremental co-evolution of organisms: A new approach for optimization and discovery of strategies. In Advances in Articial Life: Third European Conference on Arti- cial Life, 99. [] Donald E. Knuth. The Art of Computer Programming: Volume - Sorting and Searching. Addison Wesley, 97. [6] F. Menczer and R. K. Belew. Latent energy environments. In Adaptive Individuals in Evolving Populations: Models and Algorithms, 99. [7] Filippo Menczer and Richard K. Belew. Latent energy environments. To appear in: Belew and Mitchell (eds) Adapting Individuals in Evolving Populations, Addison Wesley, 99. [8] Melanie Mitchell and Stephanie Forrest. Genetic algorithms in articial life. Articial Life, (), 99. [9] James D. Murray. Mathematical Biology. Springer-Verlag, nd edition, 99. [0] Ian Parberry. A computer-assisted optimal depth lower bound for nine-input sorting networks. Mathematical Systems Theory, :0{6, 99. [] James D. Watson, Nancy H. Hopkins, Jerey W. Roberts, Joan Argetsinger Steitz, and Alan M. Weiner. Molecular Biology of the Gene, volume. Benjamin Cummings, edition, 987.