Artificial Intellegence

Transcription

1 Artificial Intellegence

2 Neural Net: Based on Nature

3 Perceptron Variations

4 Perceptrons: A Basic Neural Net In machine learning, the perceptron is an algorithm for supervised classification of an input into one of several possible non-binary outputs. It is a type of linear classifier, i.e. a classification algorithm that makes its predictions based on a linear predictor function combining a set of weights with the feature vector. The algorithm allows for online learning, in that it processes elements in the training set one at a time.(wikipedia)

5 Perceptron: Linear Discrimination Introduction to Machine Learning (Adaptive Computation and Machine Learning series) by Ethem Alpaydin ISBN-13: , ISBN-10: X

6 Perceptron: Linear Discrimination

7 Perceptron: Linear Discrimination

8 Perceptron

9 Perceptron

10 Training Examples 1.For each perceptron i, we can use prior data to choose the best W i =[W i,1,w i,2,,w i,d ] W to separate the data into two categories. This can be done statistically or with numerical method to determine the minimum possible error. 2. Base your training on survival of the fittest: Introduce random changes in the W i,j. If a change provides a better survival rate use the keep the changes other wise discard them. 3. Training data can be segmented so that part can be used to train the machine and the rest can be used to test the effectiveness of the training.

11 Inputs: Two inputs, which will be the X and Y components of the vector between the ship and the nearest asteroid. The speed that the two objects are moving together, which is determined by taking the moving velocity of each object and finding the component of velocity that lies along the direct path to the other object. The ship s moving direction, which gives the NN a frame of reference with which to make correlations between the other inputs. Outputs will provide boolean values for the ship's controls. They will determine whether or not the ship should thrust, turn left or right.

12 class NeuralNet public: NeuralNet(int nins,int nouts,int nhiddenlays,int nnodesinhiddenlays); void Init(); //access methods void Use(vector<float> &inputs,vector<float> &outputs); void Train(vector<float> &inputs,vector<float> &outputs); float GetError() return m_error; void WriteWeights(); void ReadWeights(); protected: //internal functions void AddLayer(int nneurons,int ninputs,int type); void SetInputs(vector<float>& inputs); void FindError(vector<float>& outputs); void Propagate(); void BackPropagate(); //data vector<nlayer> m_layers; NLayer* m_inputlayer; NLayer* m_outputlayer; float m_learningrate; float m_momentum; float m_error; int m_ninputs; int m_noutputs; int m_nlayers; int m_nhiddennodesperlayer; int m_acttype; int m_outputacttype; ;

13 The Init() function is the primary set-up function for the network. It builds the internal structure of the net, by iteratively calling AddLayer() to instantiate each layer s neurons. The system is set up to handle simple nets with only an input and output layer (perceptrons) as well as general, multilayer Nns. Propagate() takes the inputs to the net and spreads their influence forward through the network. BackPropagate() effectively reverses this operation by taking the error of the final outputs and finding the correct error gradients throughout the network, from the last layer backward to the first. Train() and Use() are the two main functions for actually using the NN. During training, you call the Train() method with the input-output pair you want to train. It then propagates the inputs through the NN, finds the error from the expected outputs, and backpropagates that error. Use() assumes a trained net. It just takes the inputs and returns the network s outputs. FindError() determines the output error of the network from given outputs during training. Using the derivative of the activation function, it determines the error gradient for each output neuron, which will then be used to back-propagate the necessary changes to the connection weights within the network, to close in on the optimal weights to perform well.

14

15 void NeuralNet::Init() m_inputlayer = NULL; m_outputlayer = NULL; m_acttype = ACT_BIPOLAR; m_outputacttype= ACT_LOGISTIC; m_momentum = 0.9f; m_learningrate = 0.1f; //error check if(m_nlayers<2) return; //clear out the layers, incase you re restarting the net m_layers.clear(); //input layer AddLayer(m_nInputs, 1, NLT_INPUT); if(m_nlayers > 2)//multilayer network //first hidden layer connect back to inputs AddLayer(m_nHiddenNodesperLayer, m_ninputs, NLT_HIDDEN); //any other hidden layers connect to other hidden outputs //-3 since the first layer was the inputs, //the second (connected to inputs) was initialized above, //and the last one (connect to outputs) will be initialized //below for (int i=0; i<m_nlayers-3; ++i) AddLayer(m_nHiddenNodesperLayer, m_nhiddennodesperlayer, NLT_HIDDEN); //the output layer also connects to hidden outputs AddLayer(m_nOutputs, m_nhiddennodesperlayer, NLT_OUTPUT); else//perceptron //output layer connects to inputs AddLayer(m_nOutputs, m_ninputs, NLT_OUTPUT); m_inputlayer = &m_layers[0]; m_outputlayer= &m_layers[m_nlayers-1];

16 void NeuralNet::Propagate() for (int i=0; i<m_nlayers-1; ++i) int type = (m_layers[i+1].m_type == NLT_OUTPUT)? m_outputacttype : m_acttype; m_layers[i].propagate(type,m_layers[i+1]); // void NeuralNet::BackPropagate() //backprop the error for (int i=m_nlayers-1; i>0; --i) m_layers[i].backpropagate (m_acttype,m_layers[i-1]); //adjust the weights for (i=1; i<m_nlayers; i++) m_layers[i].adjustweights(m_layers[i-1], m_learningrate,m_momentum); // void NeuralNet::Train(vector<float> &inputs,vector<float> &outputs) SetInputs(inputs); Propagate(); FindError(outputs); BackPropagate(); // void NeuralNet::Use(vector<float> &inputs,vector<float> &outputs) SetInputs(inputs); Propagate(); outputs.clear(); //return the net outputs for(int i =0;i< m_outputlayer->m_neurons.size();++i) outputs.push_back(m_outputlayer->m_neurons[i]->m_output); // void NeuralNet::SetInputs(vector<float>& inputs) int numneurons = m_inputlayer->m_neurons.size(); for (int i = 0; i<numneurons; ++i) m_inputlayer->m_neurons[i]->m_output = inputs[i];

17 void NeuralNet::FindError(vector<float>& outputs) m_error = 0; int numneurons = m_outputlayer->m_neurons.size(); for (int i=0; i<numneurons; ++i) float outputval = m_outputlayer->m_neurons[i]->m_output; float error = outputs[i]-outputval; switch(m_acttype) case ACT_TANH: m_outputlayer->m_neurons[i]->m_error = m_outputlayer-> InvTanh(outputVal)*error; case ACT_BIPOLAR: m_outputlayer->m_neurons[i]->m_error = m_outputlayer-> InvBipolarSigmoid(outputVal)*error; case ACT_LOGISTIC: default: m_outputlayer->m_neurons[i]->m_error = m_outputlayer-> InvLogistic(outputVal)*error; //error calculation for the entire net m_error += 0.5*error*error;

18 void NeuralNet::FindError(vector<float>& outputs) m_error = 0; int numneurons = m_outputlayer->m_neurons.size(); for (int i=0; i<numneurons; ++i) float outputval = m_outputlayer->m_neurons[i]->m_output; float error = outputs[i]-outputval; switch(m_acttype) case ACT_TANH: m_outputlayer->m_neurons[i]->m_error = m_outputlayer-> InvTanh(outputVal)*error; case ACT_BIPOLAR: m_outputlayer->m_neurons[i]->m_error = m_outputlayer-> InvBipolarSigmoid(outputVal)*error; case ACT_LOGISTIC: default: m_outputlayer->m_neurons[i]->m_error = m_outputlayer-> InvLogistic(outputVal)*error; //error calculation for the entire net m_error += 0.5*error*error;

19 The class houses the activation functions and their derivatives. Also, each layer has a list of its constituent neurons, as well as an m_type field (is this an input, hidden, or output layer?), and a threshold value (which is normally set to 1.0f, this value represents the output value the neuron must accumulate to fire if using a simple step activation function, or the gain of the sigmoid function being used, which corresponds to the smoothness of the s shape in the output graph: very small values approach a flat line, and very large values approach a step function shape). Propagate() is the layer extension to the function with the same name at the net level. It cycles through all the neurons in the level and performs the standard NN formula: sum all the inputs to the neuron, multiply by the corresponding connection weights, and then run it through the specified activation function. BackPropagate() is also the layer-specific continuation of this operation. It sums the total weight on each neuron, and then calculates the gradient by multiplying it with the output value, after having run the output through the derivative of the activation function. Several activation functions have been supplied. The standard logistic function gives values between 0 and 1. Both the tanh and bipolar sigmoid functions give values from 1 to 1. The linear function is the equivalent of no activation function, meaning that the output isn t scaled at all. AdjustWeights() performs the steepest-descent adjustment method on the weights because we ve computed a gradient of the delta we re looking for. Steepest descent is a greedy algorithm, meaning that it gets stuck in local minima very easily, so care must be taken with this method. Hence, we re using momentum within our weight adjustment, which means that adjustments have to come more frequently to make large changes because earlier changes have a much larger priority associated with them. This helps guard against the steepest descent method getting stuck, but it does make training slower, so you will want to adjust the momentum value.

20 void NLayer::Propagate(int type, NLayer& nextlayer) int weightindex; int numneurons = nextlayer.m_neurons.size(); for (int i=0; i<numneurons; ++i) weightindex = 0; float value = 0.0f; int numweights = m_neurons.size(); for (int j=0; j<numweights; ++j) //sum the (weights * inputs), the inputs //are the outputs of the prop layer value += nextlayer.m_neurons[i]->m_weights[j] * m_neurons[j]->m_output; //add in the bias (always has an input of -1) value+=nextlayer.m_neurons[i]->m_weights[numweights]*-1. 0f; //store the outputs, but run activation first switch(type) case ACT_STEP: nextlayer.m_neurons[i]->m_output = ActStep(value); case ACT_TANH: nextlayer.m_neurons[i]->m_output = ActTanh(value); case ACT_LOGISTIC: nextlayer.m_neurons[i]->m_output = ActLogistic(value); case ACT_BIPOLAR: nextlayer.m_neurons[i]->m_output = ActBipolarSigmoid(value); case ACT_LINEAR: default: nextlayer.m_neurons[i]->m_output = value; //if you wanted to run the Softmax activation function, you //would do it here, since it needs all the output values //if you pushed all the outputs into a vector, you could //uncomment the following line to use SoftMax activation //outputs = ActSoftmax(outputs); //and then put the outputs back into the correct spots Return;

21 void NLayer::BackPropagate(int type, NLayer &nextlayer) float outputval, error; int numneurons = nextlayer.m_neurons.size(); for (int i=0; i<numneurons; ++i) outputval = nextlayer.m_neurons[i]->m_output; error = 0; for (int j=0; j<m_neurons.size(); ++j) error+=m_neurons[j]->m_weights[i]*m_neurons[j]->m_error; switch(type) case ACT_TANH: nextlayer.m_neurons[i]->m_error = DerTanh(outputVal)*error; case ACT_LOGISTIC: nextlayer.m_neurons[i]->m_error = DerLogistic(outputVal)*error; case ACT_BIPOLAR: nextlayer.m_neurons[i]->m_error = DerBipolarSigmoid(outputVal)*error; case ACT_LINEAR: default: nextlayer.m_neurons[i]->m_error = outputval*error; void NLayer::AdjustWeights(NLayer& inputs,float lrate, float momentum) for (int i=0; i<m_neurons.size(); ++i) int numweights = m_neurons[i]->m_weights.size(); for (int j=0; j<numweights; ++j) //bias weight always uses -1 output value float output = (j==numweights-1)? -1 : inputs.m_neurons[j]->m_output; float error = m_neurons[i]->m_error; float delta = momentum*m_neurons[i]->m_lastdelta[j] + (1-momentum)*lrate * error * output; m_neurons[i]->m_weights[j] += delta; m_neurons[i]->m_lastdelta[j] = delta;

22 The NNAIControl class will serve as the AI controller for the neural network technique. This class houses the network itself and the technique-specific usage code that links it to the AIsteroids game proper. As you can see in the header, this class stores all the usual controller information (perception data and update methods, as well as being inherited from the FSMAIControl class so that it can also deal with the states of the AI ship), but also contains all the data and functionality for training and using the NN.

23 class NNAIControl: public FSMAIControl public: //constructor/functions NNAIControl(Ship* ship = NULL); ~NNAIControl(); void Update(float dt); void UpdatePerceptions(float dt); void Init(); void Reset(); void GetNetOutput(); void TrainNetAndSave(); void ReTrainNetAndSave(); //perception data float m_powerupscandist; //network output variables bool m_shouldthrust; bool m_shouldturnleft; bool m_shouldturnright; private: int m_numiterationstotrain; int m_numsavedtrainingsets; float m_maximumallowederror; //network input variables float m_speedmovingtogether; Point3f m_nearestasteroiddelta; float m_shipmovingdirection; //net, used for training and for actual usage in game NeuralNet* m_net; vector<float> m_inputs; vector<float> m_outputs; int m_numinputs; int m_numoutputs; int m_numhiddenlayers; int m_numhiddennodes; int m_netmode; ;

24 The constructor for this class sets itself up to do what needs to be done based on whether we re instantiating the controller in training mode, retraining mode, or the regular use mode. During the training modes, the network is instantiated by the training functions themselves and closed down after execution. The regular game-use mode instantiates the network right away because the game will potentially be using it to avoid obstacles. In regular training mode, there is no real AI running because the training uses real input from a human player. As you can see in the Update() function, the NNAIControl structure stores what will be the network input and output variables whenever the m_willcollide perception is true. When thousands of sets of data are collected, the Update() method then instantiates and trains a network using the data, and finally saves off the network weights so they can be reused later. Retrain mode works by loading the saved input and output training data from a file and training the network, then exiting from the game. Retraining is useful when you want to try different network designs (such as adjusting the number of hidden layers or nodes, changing to different activation functions, using more or less training iterations, etc.). Of course, if you decide to change the number of inputs or outputs, you ll need to recapture new training data using the regular NM_TRAIN mode.

25 void NNAIControl::TrainNetAndSave() m_net = new NeuralNet(m_numInputs, m_numoutputs, m_numhiddenlayers, m_numhiddennodes); vector<float> tempins; vector<float> tempouts; for(int i =0;i< m_numiterationstotrain;++i) for(int j = 0;j< m_numsavedtrainingsets; ++j) tempins.clear(); tempouts.clear(); //get training set inputs for(int k = 0;k<numInputs;++k) tempins.push_back(m_inputs[k+j*numinputs]); //get training set outputs for(k = 0;k<numOutputs;++k) tempouts.push_back(m_outputs[k+j*numoutputs]); m_net->train(tempins,tempouts); float totalerror = m_net->geterror(); if(totalerror < m_maximumallowederror) //save out net and exit m_net->writeweights(); return; void NNAIControl::ReTrainNetAndSave() FILE* pfile; if ((pfile = fopen("nntrainingdata.txt","r")) == NULL) return; m_net = new NeuralNet(m_numInputs,m_numOutputs, m_numhiddenlayers,m_numhiddennodes); vector<float> tempins; vector<float> tempouts; for(int i =0;i< m_numiterationstotrain;++i) for(int j = 0;j< m_numsavedtrainingsets; ++j) tempins.clear(); tempouts.clear(); //get training set inputs for(int k = 0;k<m_numInputs;++k) float temp; fscanf(pfile,"%f ",&temp); tempins.push_back(temp); //get training set outputs for(k = 0;k<m_numOutputs;++k) float temp; fscanf(pfile,"%f ",&temp); tempouts.push_back(temp); m_net->train(tempins,tempouts); float totalerror = m_net->geterror(); if(i> 100 && totalerror < m_maximumallowederror) //save out net and exit m_net->writeweights(); return;

26 NNAIControl::NNAIControl(Ship* ship): FSMAIControl(ship) m_net = NULL; Init(); if(m_netmode == NM_USE) m_net = new NeuralNet(m_numInputs,m_numOutputs, m_numhiddenlayers,m_numhiddennodes); m_net->readweights(); else if (m_netmode == NM_RETRAIN) m_numsavedtrainingsets = 1000; ReTrainNetAndSave(); // void NNAIControl::Update(float dt) Ship* ship = Game.m_mainShip; if(!ship) m_machine->reset(); return; switch(m_netmode) case NM_TRAIN: UpdatePerceptions(dt); if(m_willcollide) //write test data to file FILE* pfile; if ((pfile =fopen("nntrainingdata.txt","a"))== NULL) return; fprintf(pfile,"%f %f %f %f ", m_nearestasteroiddelta.x(), m_nearestasteroiddelta.y(), m_speedmovingtogether, m_shipmovingdirection); fprintf(pfile,"%d %d %d ",ship->isthruston(), ship->isturningright(),ship->isturningleft()); m_numsavedtrainingsets++; m_inputs.push_back(m_nearestasteroiddelta.x()); m_inputs.push_back(m_nearestasteroiddelta.y()); m_inputs.push_back(m_speedmovingtogether); m_inputs.push_back(m_shipmovingdirection); m_outputs.push_back(ship->isthruston()); m_outputs.push_back(ship->isturningright()); m_outputs.push_back(ship->isturningleft()); fclose(pfile); if(m_numsavedtrainingsets==num_training_sets_to_aquire) TrainNetAndSave(); Game.GameOver(); case NM_RETRAIN: Game.GameOver(); case NM_USE: default: UpdatePerceptions(dt); if(m_willcollide) GetNetOutput(); m_machine->updatemachine(dt);

27 void NNAIControl::GetNetOutput() //clear out temp storage m_inputs.clear(); m_outputs.clear(); //set up inputs // void StateNNEvade::Update(float dt) NNAIControl* parent = (NNAIControl*)m_parent; Ship* ship = parent->m_ship; m_inputs.push_back(m_nearestasteroiddelta.x()); m_inputs.push_back(m_nearestasteroiddelta.y()); m_inputs.push_back(m_speedmovingtogether); m_inputs.push_back(m_shipmovingdirection); //get output values m_net->use(m_inputs,m_outputs); m_shouldthrust = m_outputs[0] > BOOL_THRESHOLD; m_shouldturnright = m_outputs[1] > BOOL_THRESHOLD; m_shouldturnleft = m_outputs[2] > BOOL_THRESHOLD; if(parent->m_shouldthrust)//thrust ship->thruston(); else ship->thrustoff(); if(parent->m_shouldturnright) ship->turnright(); else if(parent->m_shouldturnleft) ship->turnleft(); else ship->stopturn(); parent->m_debugtxt = "Evade";