1 Lab + Hwk 5: Particle Swarm Optimization

Size: px

Start display at page:

Download "1 Lab + Hwk 5: Particle Swarm Optimization"

Octavia Shelton
6 years ago
Views:

1 1 Lab + Hwk 5: Particle Swarm Optimization This laboratory requires the following equipment: C programming tools (gcc, make). Webots simulation software. Webots User Guide Webots Reference Manual. The laboratory duration is about 3 hours. Homework will be due on the 7 th day after your lab session, at 12 noon. Please format your solution as a PDF file with the name [name]_lab[#].pdf, where [name] is your account user name and [#] is the number of the lab+homework assignment, and upload it as the Report submission onto the student section of the course webpage. All additional material (movies, code, and any other relevant files) should be archived as a zip file with the name [name]_lab[#].tgz (hint: tar cvzf [name]_lab[#].tgz [directory]) and uploaded as the Additional Material submission onto the student section of the course webpage. Every misnamed file will result in a 1 point penalty to your assignment grade. Keep in mind that no late solutions will be accepted unless supported by health reasons (and a note from the doctor). If there are extreme circumstances which will prevent you from attending a particular lab session, it may be possible to make arrangements with the TAs before the lab in question takes place. The more advance notice you can give in these situations, the more likely it is that we will be able to work something out. 1.1 Grading In the following text you will find several exercises and questions. The notation S x means that the question can be solved using only additional simulation; you are not required to write or produce anything for these questions. The notation Q x means that the question can be answered theoretically, without any simulation; your answers to these questions must be submitted in your Report. The length of answers should be approximately two sentences unless otherwise noted. The notation I x means that the problem has to be solved by implementing a piece of code and performing a simulation; the code written for these questions must be submitted in your Additional Material. Please use this notation in your answer file! The number of points for each exercise is given between parentheses. The combined total number of points for the laboratory and homework exercises is Optimization In many instances, we want to find either the minimum or maximum value of some function. If the function in question is complex or even unknown, it is impossible to JP; TL, Lab+Hwk 5: Particle Swarm Optimization 1

2 accomplish this task analytically. Furthermore, if the function parameter space is very large, a systematic search of this space is often too computationally expensive. Therefore, a number of techniques have been developed to find near-optimal solutions (i.e. local minima and maxima), such as genetic algorithms and particle swarm optimization. 1.3 The Particle Swarm To find minima/maxima, repeated evaluations of the function in question must be done at different points. A good optimization algorithm will use the information obtained from these evaluations to choose the locations of future evaluations, and eventually to decide where the minimum/maximum is. In Particle Swarm Optimization, a set of particles is initialized with a random position for each particle. Each particle is also assigned a random velocity in the search space. Particles are then flown through the search space, moving at their respective velocities. Optimization is achieved by each particle remembering at what position they achieved the best evaluation of the function, or the particle s personal best. Particles also remember the best achieving position of their neighborhood. The neighborhood for some particle A is a group of particles to which A belongs. This group can be topological (whatever particles are closest) or fixed throughout the algorithm. It can consist of either a subset of the particle swarm (local neighborhood), or the entire swarm (global neighborhood). The neighborhood best of a neighborhood is the position that yielded the best evaluation by any particle in the neighborhood. At each generation of the algorithm, the velocity of each particle is updated, using a randomized attraction to both the particle best and the neighborhood best. This allows particles to move towards areas of the search space that have yielded good results, and in doing so, discover nearby better results. In this way, the particles will eventually converge on possibly global optima. The idea for this type of behavior took inspiration from the ways birds act while flying in a flock. 2 Lab: Using PSO 2.1 PSO on the Sphere Function The Sphere function is defined as follows: f ( x) = n 2 x i i= 1 We want to find the set of x i that give the value closest to 0 (given by x i = 0 for all x i ). The values for x i are initially randomly distributed between and This function is often used as a simple test for optimization functions; although it may seem trivial to a human observer, it can be difficult for an optimization algorithm to figure out that the best solution is all zeroes. We will use PSO with a swarm size of 30 and the neighborhood of a particle given by the three nearest particles on each side (e.g. particle 8 has neighborhood {5, 6, 7, 8, 9, 10, 11}) with particle 29 being next to particle 0. We will optimize the Sphere Comment [AM1]: I.e. mention neighborhood index = 7 JP; TL, Lab+Hwk 5: Particle Swarm Optimization 2

3 function with n = 10, and, therefore, ten components in the vector representing each particle. In other words, a particle is a set of 10 numbers, and the closer to 0 all the numbers become, the better the performance. Copy the pso directory from the student section of the course webpage into your home directory. Compile the PSO program using make. You can execute the program with./main <number of iterations>. S: Run the program with 10 iterations, 100 iterations, 1000 iterations, and iterations. At the end of each trial, both the achieved value and the found solution for 10 runs will be printed along with the average value. Lower values mean better the solutions. Q 1 (5): What do you observe? S: Try the simulation with iterations. Q 2 (5): How many iterations do you think will be necessary to get the exact answer? Why? S: The definition VMAX in main.c controls the maximum velocity that particles can achieve in the simulation. Using 100 iterations, try varying VMAX from 0.5 to 4.0. Q 3 (5): What do you observe? S: Try varying VMAX from to Q 4 (5): What happens now? Why? It can be very inconvenient to have to tune VMAX to give the optimal performance for a simulation. A feature that was developed for PSO soon after its invention was an inertia coefficient. This was a coefficient typically less than 1 which is multiplied with the velocity at each iteration, in order to naturally slow particles down without imposing a hard velocity threshold. S: Uncomment line 77 of pso.c (it will be marked in the code), set VMAX to 4.0, and run the program again. Try increasing the number of iterations. Q 5 (5): What do you observe now? 2.2 PSO for Unsupervised Learning We ve established that PSO can do a good job of optimizing simple mathematical functions. The next step is to test it in a more demanding environment, in particular in the presence of noisy functions. In Evolution of Homing Navigation in a Real Mobile Robot (Floreano and Mondada, 1996), a genetic algorithm was used to train a robot to move through a maze and avoid obstacles. We want to use Webots to evolve robotic controllers for the mobile robots, comparing the performance of GA and PSO. For the experiments in this lab, we use the e-puck robot instead of the Khepera that was used in the paper. The behavior we wish to achieve on the robots is obstacle avoidance, the same as in the Floreano and Mondada paper. Copy the ga_obs_sup, pso_obs_sup and obs_con controllers and the ga_obs and pso_obs worlds from the student section of the course webpage. Compile all controllers. These worlds are simple walled arenas with no obstacles and twenty e-puck robots. We use a two neuron, single-layer, neural networks to control each robot, with proximity sensors and the JP; TL, Lab+Hwk 5: Particle Swarm Optimization 3

4 relative position center of mass as inputs and the motor speed as outputs. There are recursive and lateral connections from the outputs of both neurons back into each other. The controller we evolve is the set of weights for these neurons (8 proximity sensors + 2 recursive/lateral connections + 1 threshold = 11 weights for each neuron => 22 weights total). The fitness of a controller is measured with the equation: F = V ( 1 v)(1 i), V, v, i [0,1] where V is a measure of the average rotation speed of the two wheels over the trial, v is a measure of the average difference in wheel speed of the two wheels over the trial, and i is the average activation value of the proximity sensor with the highest activity. Each of these terms in the fitness function encourage the robot to go fast, go straight, and not stay close to walls, respectively. The default time for a single evaluation of fitness is ~30 s. The genetic algorithm we re using is similar to that used in 1996 by Floreano and Mondada. It selects the best performing half of the population as potential parents. Out of these, parents are selected using a Roulette Wheel scheme. Crossover occurs with probability 0.2 and mutation occurs with probability 0.15 to create children to replenish the missing half of the population Fast Heterogeneous Learning Using PSO or GA on a single robot, it can take a long time to evolve highperforming controllers. This is because the robot will have to test every member of the population/swarm one after another for each iteration of the algorithm. For example, doing 100 iterations of PSO with 20 particles with each evaluation taking 1 minute means we need 2000 minutes or over 36 hours to complete! One way to speed up this process is to use multiple robots doing heterogeneous learning. In heterogeneous learning, each robot is using a different controller than the others. By using multiple robots, we can distribute the particles among the robots and evaluate their performance in parallel, thus saving a lot of time. Going back to our previous example, if we have 20 robots each evaluating a different particle, it will only take 1 minute to evaluate 20 particles, and the total evolution time goes down to about 1.7 hours. We will now test evolution of obstacle avoidance behavior using heterogeneous learning with GA and PSO. In the follow questions, you will be asked to run simulations that may take several minutes to complete. In order to save time, you may want to look ahead to the next question to see if it is something which you can work on while you wait. S: Load the ga_obs world. Run the world to simulate the evolution. You will need to use fast mode to simulate quickly enough, but you may want to observe how the robots are behaving occasionally (pay attention to whether they are near the beginning or end of an evolutionary run). The simulation will do 10 separate evolutionary runs and print the final fitness for each, as well as the average fitness at the end. When the evolution is finished, observe the performance of the robots (they will be running the best found controller). JP; TL, Lab+Hwk 5: Particle Swarm Optimization 4

5 Q 6 (5): What kind of fitness values do you get? S: Now load the pso_obs world, and run the simulations again. Q 7 (5): How does the performance compare? We now switch to using the modified version of our genetic algorithm specifically designed for noisy fitness function evaluation (see Zhang, 2003). At each iteration, the performance of the potential parent set is reevaluated, and the new fitness value is combined with previous values to get a combined estimate. Q 8 (5): Why might this modification help to improve performance? S: In the ga.c file, set the NOISY definition from 0 to 1. This enables the modification. Run the simulations again. Q 9 (5): Record the final fitness values. How do the new values compare? The modification for GA for handling noisy fitness function evaluations was adapted for PSO in Particle Swarm Optimization for Unsupervised Robotic Learning (Pugh, 2005). In this, at each iteration, the previous best positions of the particles are re-evaluated, and the new fitness value combined with previous ones. S: In the pso.c file, set the NOISY definition from 0 to 1. This enables the modification. Run the simulations again. Q 10 (5): Record the final fitness values. How do the new fitness values compare? Q 11 (5): In both modifications, the number of iterations run by the algorithm is divided by two. Why does this make sense? JP; TL, Lab+Hwk 5: Particle Swarm Optimization 5

6 3 Hwk: Analyzing PSO Depending on the function being optimized, different parameter settings will give better results in PSO. We will now explore the influence of parameters on a more complex numerical function, the Rastrigin function, given by: n 2 [ x i 10cos( 2 x i ) + 10] f ( x) = π i= 1 This function contains many local optima which could cause premature convergence of particles. S: Go back to the pso directory. Rerun optimization of the Sphere function (with the inertia coefficient) setting the number of particles to 3, 5, 10, 15, 30, and 60. For each number of particles n, set the number of iterations to 3000/n (i.e. 1000, 600, 300, 200, 100, 50; this will keep the computational time approximately constant). You can specify the number of particles from the command line with./main <number of iterations> <number of particles>. Q 12 (5): What performances did you get for each number of particles? Which number is best? S: In pso.c, change the definition of FUNCTION from 1 to 2. This selects the Rastrigin function. Try running the simulation again while varying the number of particles. Q 13 (5): What performances did you get now for each number of particles? Which number is best? Q 14 (10): How do the results for the Sphere and Rastrigin functions compare? Why is this happening? You may need up to five lines to fully explain this. Next, we will explore when it s useful to use a global neighborhood, where all the particles are attracted to the single best found position, and when it s useful to use a local neighborhood, when different groups of particles are attracted to different positions. S: On the Sphere function, rerun the optimization setting the definition of NB (number of neighbors on each side) to 1 (local neighborhood) and 15 (global neighborhood) with 15 particles and 100 iterations. Q 15 (5): What performances did you get? Which neighborhood type is best? S: Now set the number of iterations to 1000, and run the same experiments on the Rastrigin function. Q 16 (5): What performances did you get now? Which neighborhood type is best? Q 17 (5): For evolving aggregation in Webots, robots can cluster well either by moving forwards towards the group or by moving backwards towards the group. However, robots are better able to navigate to a more tight cluster going forwards, as they have more proximity sensors to see what they are approaching, and will therefore give a slightly better fitness overall. Would a global or local neighborhood be preferable here? Q 18 (10): What if you want to optimize the Sphere function, but with Gaussian noise added. Would this affect your neighborhood choice? Why? JP; TL, Lab+Hwk 5: Particle Swarm Optimization 6

7 4 References Floreano D. and Mondada F., "Evolution of Homing Navigation in a Real Mobile Robot", IEEE Trans. on System, Man, and Cybernetics: Part B, 26(3): , Zhang Y., Martinoli A., and Antonsson E. K. Evolutionary Design of a Collective Sensory System, In H. Lipson, E. K. Antonsson, and J. R. Koza, editors, Proc. of the 2003 AAAI Spring Symposium on Computational Synthesis, March 2003, Stanford, CA, pp AAAI Press Pugh, J., Zhang, Y., and Martinoli, A. Particle Swarm Optimization for Unsupervised Robotic Learning, In Proc. of the IEEE Swarm Intelligence Symposium 2005, Pasadena, CA. JP; TL, Lab+Hwk 5: Particle Swarm Optimization 7

1 Lab + Hwk 5: Particle Swarm Optimization

1 Lab + Hwk 5: Particle Swarm Optimization This laboratory requires the following equipment: C programming tools (gcc, make), already installed in GR B001 Webots simulation software Webots User Guide Webots