CS206: Evolutionary Robotics

Transcription

1 CS206: Evolutionary Robotics Programming Assignment 10 of 10 Description: In this final assignment, you will connect the Python application you developed in assignments 1 through 3 with the robot simulator you have created in assignments 4 through 9. When you are done, the two applications will work together as follows: The Python hill climber will start up the robot simulator and send it a collection of random synaptic weights. The robot simulator will read in these weights, and use them to set the synaptic weights of the artificial neural network you created in the last assignment. The robot simulator will run for 1000 time steps, and then record the final position of the robot s main body. It will save this value to a file and then exit. The Python hill climber will read in this value and treat it as the fitness of the synaptic weights it just sent. It will create a copy of these weights, and mutate them slightly. It will then start up the robot simulator again, send it this new set of weights, and so on. Let us get started. 1. Create an empty directory called Assignment Copy your Assignment3.py file from assignment 3 into this directory and rename it Assignment10.py. 3. Recall that the Python code implements a hill climber, a simple evolutionary algorithm that optimizes a neural network to exhibit some desired behavior. Comment out each line of the main function so that when you run the code it starts and then immediately terminates: #numgenes = 10; #numgenerations = 1000; #fits = Evolve(numGenes, ); 4. Recall that in the function Evolve, a matrix of synaptic weights was evolved. We now need to evolve a 4 8 matrix of synaptic weights: one synaptic to connect each of the four sensors to each of the eight motors. So replace the numgenes=10 line with numsensors = 4; nummotors = 8; #numgenerations = 1000; 1

2 5. Now uncomment the Evolve function, and change the parameters sent to it: fits = Evolve(numSensors, nummotors, ) 6. Comment out all of the lines in the Evolve function so that your code still runs yet does not do anything (yet). 7. Now uncomment and modify the first four lines in Evolve accordingly: these lines create a vector for saving the best fitness as evolution proceeds; they create a random matrix of synaptic weights; and they evaluate the fitness of those weights: fits = Matrix_Create(1,numGenerations); parent = Matrix_Create(numSensors,numMotors); parent = Matrix_Randomize(parent)*2-1; parentfit = Fitness3_Get(parent); 8. The new function Fitness3 Get starts up the robot simulator and sends the matrix of weights to it; it waits for the robot simulator to terminate (after the simulator has saved out the fitness of those weights); reads in the fitness value; and deletes the files that allow the two programs to communicate with each other. def Fitness3_Get(synapses): weightsfilename = weights.dat ; fitfilename = fit.dat ; Send_Synapse_Weights_ToFile(synapses,weightsFileName); Simulate_Robot(); Wait_For_Fitness_File(fitFileName); fitness = Fitness_Collect_From_File(fitFileName); Delete_File(weightsFileName); Delete_File(fitFileName); return( fitness ); 9. Comment out all of the lines below Send Synapse Weights ToFile, and then implement only this function. This function should save all 32 weights to file weights.dat in the same directory as the python and robot simulator executables. Given that the rest of the Python code is commented out, when you run your code it should save this file and then quit. 2

3 10. Let us turn now to your robot simulator code. Make a copy of Assignment 9, and name the copy Assignment 10. Make sure that the simulator executable is compiled into your new Assignment10 directory. 11. We need to ensure that the robot simulator, when initiated, runs for 1000 time steps and then terminates. To do so, we need to define a new variable int timestep, set it to zero in initphysics, and then increment it during each pass through clientmoveanddisplay. (Make sure also that the application always starts unpaused now.) 12. Once 1000 timesteps have elapsed, the application should terminate: void RagdollDemo::clientMoveAndDisplay() { timestep++; if ( timestep==1000 ) exit(0); } 13. Run the executable several times. You should see the robot move for 1000 time steps and then see the simulation terminate. 14. You should have code in initphysics that sets the elements of matrix weights to random values. Modify this code so that instead the elements are set to the values stored in weights.dat. 15. Now we need to capture the final position of the robot s main body. So modify the code in clientmoveanddisplay as follows (you will need to make use of Bullet s getcenter OfMassPosition function): void RagdollDemo::clientMoveAndDisplay() { if ( timestep==1000 ) { Save_Position(body[0]); exit(0); } } Within the function, capture the (x,y,z) position of the main body, and print the z-component of this position to the screen. This component reports how much the robot moves toward the viewer (negative values) and how much it moves into the screen (positive values). 16. You may need to pause execution after you print the value, because the application will terminate immediately after printing the value. (The following will pause execution until the user presses Enter.) 3

4 printf char ch = getchar(); 17. Within Save Position, write this value to the file fits.dat immediately after printing it. Verify that the number in the file and the number printed to the screen are the same. 18. Now run the executable several times. Note that it is reading in the same set of weights each time, so the robot s behavior (and its final position) should always be the same. You may see that the behavior is different from one execution to the next. This is because it is sometimes difficult to get the physics engine to behave deterministically, even though in theory it should do so at all times. 19. If you do see that the final positions differ from one run to the next, the culprit often lies in the detection and resolution of contacts. This sometimes results in the touch sensors only being updated once every few time steps, and sometimes this updating is not deterministic. You can fix this by replacing m_dynamicsworld->stepsimulation(ms / f); in clientmoveanddisplay with the following: while ( (touches[2]==0) && (touches[4]==0) && (touches[6]==0) && (touches[8]==0) ) m_dynamicsworld->stepsimulation(ms / f); (This assumes that the third, fifth, seventh and ninth body parts are the four lower legs respectively.) This new code continuously updates the motion of the robot until at least one of the touch sensors fires. 20. If you add this code and you are still seeing different final positions from one run to the next ensure that (1) you are reading in the same 32 synaptic weights in initphysics each time; (2) you are running the simulation for exactly 1000 time steps each time; and (3) all eight motors receive the same desired velocity each time enableangularmotor is called. If you are still observing non-deterministic behavior, contact the instructor. 21. Remove the line that prints the main body s final position and the line that pauses the simulator so that the simulator now runs and then terminates without any user intervention. 22. Now return to your Python code. Immediately after Fitness3 Get sends the weight matrix to a file, it must start the robot simulator. You can use the os package which allows system commands to be executed from within Python code: 4

5 import os; def Fitness3_Get(): [send weight matrix to file weights.dat] # If you are using a Windows machine os.system( App_RagdollDemo_vs2010.exe ); # If you are using a Mac or Linux machine #os.system(./appragdolldemo ); When you run your Python code now, you should see the robot simulator start automatically and then terminate after 1000 time steps. There should be both a weights.dat and fit.dat file sitting in the directory. 23. We must now instruct the Python application to wait until the fit.dat file appears. First, make sure to delete fit.dat in the directory by hand and then add the code: import time; import sys; def Fitness3 os.system() while (exists( fit.dat ) == False): time.sleep(0.2); sys.exit(0); Now, when you run your Python application, you should see it stay open while the robot simulator runs. When the simulator terminates, so too should the Python application. 24. Remove the sys.exit(0) line, and follow it with code that reads in the single value stored in fit.dat. This value should be stored in the variable fitness, which is returned when Fitness3 Get terminates. 25. But before this function terminates it must delete the two files weights.dat and fit. You can accomplish this using the Python command os.remove(filename). 26. Now when you run your Python code you should should observe the following: (1) the Python application stays open while the simulator runs; (2) both applications terminate together; and (3) weights.dat and fit.dat are no longer in the directory. 27. You can now uncomment the remaining lines of Evolve, so that the hill climber runs. Remember to use the new Fitness3 Get function when evaluating the fitness of the child set of synaptic weights: 5

6 Figure 1: Visual display of a high-fitness robot and a running print out from the hill climber that produced it. parentfit = Fitness3_Get(parent); for g in range(0,numgenerations): childfit = Fitness3_Get(child); print g, parentfit, childfit; if ( childfit > parentfit ): 28. You should observe a series of robots now. Over a few minutes, you should observe robots evolving the ability to locomote increasingly further into the screen. Capture one such robot in a screenshot, alongside a print out of your hill climber running. The first value in each row should report the current generation; the second value reports the furthest any robot has yet travelled into the screen (parentfit); and the third value should report the distance travelled by the previous robot (childfit). Upload this single screenshot to the Wiki. 6