Instructor: Dr. Benjamin Thompson Lecture 15: 3 March 2009

Transcription

1 Instructor: Dr. Benjamin Thompson Lecture 15: 3 March 2009

2 Announcements Homework 5 due right now No homework assignment over Spring Break, because your instructor is such an awesome guy. In other words, try to spend sometime over the break thinking about your project! I will be available all next week for questions/office hours/coffee and a nice scone, perhaps. Maybe pumpkin, maybe blueberry. Black drip, please; none of that froo-froo latte stuff.

3 Characterized Analogously As a Constant Tone With Apparent Decreasing Frequency Decision Boundaries in Classification Problems Matlab Demonstration: Two Moons revisited Matlab Demonstration: Five-class problem Super-Awesome Really Fun and Amazing Term Project Assignment! Yay!

4 Characterized Analogously As a Constant Tone With Apparent Increasing Frequency Neural Network Training as an Optimization Problem Drawbacks of Gradient Descent Other Optimization Approaches Random Search Particle Swarm Optimization Now with minty-fresh MATLAB Demo! Genetic Algorithms Simulated Annealing

5 Because I know you re all craving more abstraction!

6 To Put It Another Way What are some other parameter estimation problems we ve seen? The Multi-Layered Perceptron Neural Network can be viewed as a parameter estimation problem: That is, the MLP may be viewed as a function o=f nn (x,w), where x is some vector of inputs, o is the output of the function, and W is the set of all the weights that parameterize the input/output relationship Then, given some (possibly unknown) function d=f(x), we seek to estimate the set of parameters W that makes F nn (x,w) come as closely as possible to equaling F(x) In technical terms, we seek to find the W that minimizes the approximation error ( ) (, ) 2 F x Fnn x W dx x

7 Still Puttin It Of course, since we don t necessarily know the underlying function d=f(x), we must rely on the training sample to tell us how we re doing So, for a set of inputs X={x p } and corresponding desired outputs D={d p }, we seek the set of parameters Wthat minimizes* the training error: P 1 F F x 2 p= 1 ( ) (, ) 2 p nn xp W which we ve seen in its more familiar form: 1 E = 2 P M ( d ) 2 pi, opi, p= 1 i= 1 *This isn t technically the same thing as on the previous slide. Remember our discussion on Generalization?

8 Long Story Slightly Longer In other words, we want to optimize the approximation error by finding the best Wfor the task! Backpropagation was just one proposed method for doing this, and was based on the gradient descent technique for iterative optimization

9 What s Wrong With Gradient Descent? Gradient descent is only guaranteed to work when the error surface is more-or-less bowl-like That is, all downward paths lead to the global minimum Gradient descent is highly prone to getting stuck in local minima In fact, if there s a local minimum between its initialization point and the global minimum, it s pretty much guaranteed to get stuck That being said, the performance of Gradient Descent is highly dependent on its initial position in weightspace

10 You know, like sneaking up from behind

11 Motivation For some error surfaces, Gradient Descent may not be up to the task of finding the best set of weights for a neural network Some other approaches may be warrantedea page from We once again take a page from nature in developing several alternatives

12 Goal The goal of all these approaches is, again, optimization: Specifically for the case of neural networks, we seek to optimize (minimize) the error or cost function defined as the error equation on the previous slide, as a function of the parameter set W In general, we seek to optimize some fitness function, which is a measure of how good a particular guess at the parameter set is Optimization may be thought of as a searchthrough weight-spaceto find the point that provides the optimal cost/error/fitness

13 As bad as it gets

14 Naïve Approach We ve already mentioned brute force search: That is, test every possible set of weights and pick the best one Practically impossible for almost all real problems Another approach: random search For each epoch k, do: Randomly select a guess at the weights, W[k] Calculate the fitness/cost/error of this guess, E k If E k is better than E best, save W[k] as W best set E best = E k Continue until E best reaches some acceptable level

15 More Naiveté This approach is memoryless That is, the guess on one iteration is completely unrelated to the guess on any other iteration Given enough time, this method will find the best answer Of course, enough might be infinite, or at least long enough for computers to evolve sentience and take over the entire world, thus rendering whatever optimization problem you were trying to solve completely moot Nobody ever, ever, ever uses this approach.

16 Better Options Particle Swarm Optimization Genetic Algorithms Simulated Annealing

17 Ever see Alfred Hitchcock s The Birds? It s like that, with less screaming.

18 Motivation Scientists and observers long noticed that a flock of birds behaved in an apparently choreographedmanner as it converged on a food source R. Eberhartand J. Kennedy (1995) proposed a simple set of rules that individual birds may follow to converge on a food source in such a manner Birds : a particular solution of some optimization problem i.e., a guess at W Food source : the location of the global optimum in the search space alternately, the food source may be interpreted the valueof the fitness/error/cost function when it is at its best

19 Particles? In order to abstract things, we call each bird a particle We have many, simultaneous guesses or particles, so it is a swarmof particles --hence, Particle Swarm Optimization(hereafter, PSO) Each particle moves about the search space according to some velocity That is, some directionand speed

20 Particles Illustrated w 2 Remember, each of these points represents one possible solution of W w 1

21 What defines the velocity? Each bird has a memory of two things: The best solution it has found so far personal best or p best The best solution any bird has found global best, or g best After each iteration, a bird adjusts its velocity using three components: Its current velocity (the momentum component) Its distance from its personal best (the cognitive component) Its distance from the global best (the social component)

22 Picture This! Personal best, so far Personal best, so far Current Velocity Current Velocity Global best, so far

23 So How Does This Work? Only two equations we need to know: position (weight) update: p = p + η v new old old velocity update: v = ω v + c r p p + c r g p ( ) ( ) new old 1 1 best new 2 2 best new Momentum term Cognitive term Social term ω, c1, and c 2 are tunable weights to adjust the emphasis of each of the three terms ωis typically chosen to be slightly less than one c1 and c 2 are typically chosen to be approx. 2 r1 and r2 are random numbers, typically taken from a U[0,1] distribution The addition of these terms enable a random component to the search

24 The PSO Algorithm Defined for minimization problems; how would you define this for maximization problems? Initialize N birds positions and velocities randomly Randomly initialize p best and g best Set E gbest to Inf Set E pbest to Inf FOR each iteration, do: FOR each bird, do: Update the bird s position using p = p + η v Evaluate the error of p new, E pnew If E pnew < E pbest Set E pbest = E pnew Set p best = p new If E pnew < E gbest Set E gbest = E pnew Set g best = p new Update velocities using End FOR End FOR new old old For a neural net, this means using pnew as the input, calculating the nnoutput, and comparing that to the desiredoutput. This also implies that you must have an oracleto determine the desired response. ( ) c r ( ) v = ω v + c r p p + g p new old 1 1 best new 2 2 best new

25 Some PSO Points Notice that nothing in the inner loop really depends on previous/following iterations of that loop That is, notice that, apart from the global best information, one bird doesn t care about what any other bird is doing at the time Thus, if the calculation of the error is computationally intensive, you could parallelize PSO by running each bird (or a subset of birds) on a separate computer, and have them communicate global-best information after each epoch

26 More PSO Points Extremely efficient, computationally speaking I ve coded it up in 6 lines of code in MATLAB, for example Only requires simple (scalar) multiplies and (vector) addition Computational footprint scales linearlywith the number of birds Easily extensible I ll mention some heuristic improvements in later slides Best of all: it works! (Demo after Spring Break!)

27 PSO Extensions Fully Informed PSO: Additional component in velocity update: That is, each bird also moves toward the weighted average of each other bird s personal best Repulsive PSO: Normal PSO tends to converge into a clump To counteract this, add an another component to the velocity N 1 update α r i= 1 In other words, move away from the centroid of all the other birds Random Jitter (, ) i i bird i bird Add an additive random vector to the velocity update And many, many, many more p p N 1 i= 1 α r ( p, p ) i i best i bird

28 PSO Demo My cost function is simply the familiar Rosenbrock function:

29 Instead of a corny quip, I present to you, reprinted with permission from xkcd.org:

30 Motivation Attempts to model the actual process of evolution on a small scale to evolvebetter solutions to some optimization problem given a previous generation of solutions In reality: Two parents in a given generation contribute chromosomes to an overall genetic code of a child via genetic crossover random mutation of the genetic code occurs with some small probability Each generation reproduces with some probability based on the fitness of the parents in that generation

31 The Genome In humans, of course, our genetic code determines our physical makeup Similarly, the genome in a Genetic Algorithm (GA) fully determines a single potential solution to the optimization problem For neural networks, the genome is just all the weights! Traditionally, GAs use bit-string genomes: That is, the genetic pattern is simply a string of ones and zeros So the continuously-valued weight matrices must be converted into binary! Non-binary methods also work

32 The Genome Each of these cells is a single gene Each block, then, may be thought of as a chromosome

33 A Short Primer on How To Convert To Binary 1. Pick a dynamic range (w min, w max ) for each parameter and number of bits N b to represent each parameter e.g., +/-10.0, and 10 bits Dynamic range impacts the min/max searchable values Number of bits impacts the refinement of your solution These can be different for each parameter, or the same. 2. Normalize your parameter wk by knorm, w w w w k min = 3. In MATLAB, use dec2bin(wk) to convert this to a binary (char) string w max min N 2 b

34 A Short Primer on How to Convert From Binary Given a binary string bcorresponding to a single number: 1) First, convert the binary string to an integer using, in MATLAB, w k,norm =bin2dec(b) 2) Un-normalize this integer by performing: w = w 2 Nb w w + w k ( ) knorm, max min min

35 The Algorithm, Conceptually Initialize a population of possible solutions Evaluate the fitness of each solution Each member of the population has a probability to reproduce based on its fitness More fit solutions are more likely to reproduce and pass on their fit genes Simple approach: normalize the fitness so that the most fit member will breed with probability 1, and the least fit with probability 0 Select pair-wise mates for each breeding member That is, the top two breeders mate, the third and fourth-best mate, and so on

36 More Conceptual Algorithm For a given mating pair: Ma Baby crossoveroccurs: each chromosome is randomly (coin flip) selected from one or the other parent to form a new offspring Pa

37 Still More Conceptual Algorithm Each gene of this new child has a very small probability to flip its bit this is mutation The child patterns form the guesses for the parameter vector on the following iteration That s it!

38 Genetic Algorithm Details Initialize a set of Nrandom guesses at the solution For each iteration, do: Evaluate the fitness of each member of the population For each member, do: If that member s fitness is in the top 50% of fitness, mark that member for reproduction end FOR From the set of marked for reproduction members, create Npairs of mates (cont. on next slide) I m not going into detail here because there are many, many ways to accomplish this

39 GA Details (cont.) For each mating pair, do: For each chromosome in a genome, do: generate a uniform[0,1] random number If that number is greater than or equal to 0.5, copy the chromosome from the father, else, copy the chromosome from the mother End FOR For each gene in the resulting child genome, do: Generate a uniform[0,1] number If this number is less than µ, the mutation rate parameter (a very small number), flip that gene s bit value (o->1 or 1->0) End FOR End FOR Repeat until a solution whose fitness is sufficiently low exists

40 GA Remarks A high mutation rate makes this approach random search A low mutation rate causes this to converge very slowly and be very prone to local minima GAs generally take a long time to converge The mutation of a single bit means that a single guess can jump a lot (for an MSB mutation) or very little (for an LSB mutation) Just like with PSO, many, many heuristics exist for improving the algorithm