Monte Carlo for Optimization

Transcription

1 Monte Carlo for Optimization Art Owen 1, Lingyu Chen 2, Jorge Picazo 1 1 Stanford University 2 Intel Corporation 1

2 Overview 1. Survey MC ideas for optimization: (a) Multistart (b) Stochastic approximation (c) Simulated annealing (d) Tabu search (e) Genetic algorithms (f) Ant colony methods 2. Describe theses of Chen and Picazo 2

3 Optimization Find best value of some x Best means: minimize or maximize f (x) f a computable cost or utility x is a (possibly long) list of variables under our control f Predicted profit Lift/Drag Investment return Travel distance x credit policy variables describing wing portfolio order in which cities visited 3

4 Easy optimization problems Easy to optimize smooth differentiable unimodal functions. Repeated quadratic approximations (Newton and variants) Extends to high dimensional (easy) problems 4

5 Newton s results on easy function x f (x) f 0 (x) f 00 (x) e e e e e e e e e f (x) = x + 0:05 log(x + 0:05) xλ = 0:95 5

6 Hard problems 1. f multimodal 2. f not quite smooth 3. f measured with noise 4. f is f (x; y) where for missing random y 5. x combinatorial, e.g. all 50! = 3: circuits through 50 cities 6

7 A harder optimization Several or many local optima Minima can be twisty passages in high dimensions End point depends on start point 7

8 Another hard optimization Derivatives may be unreliable Maybe f contains: ffl a nearest neighbor selection, ffl a secondary iteration, ffl an adaptive grid, etc. 8

9 Multistart Repeat Newton-like algorithm R times: x 0;r! x 1;r! x 2;r! x n(r);r ; Keep best f (x n(r);r ) 1. Start with random x, or, 2. a grid if x low dimensional Domains of attraction r = 1;:::;R Many x 0;r converge to same x n(r);r Need a good f (x n(r);r ) with a large domain of attraction Pr(Seen best) = 1 (1 pλ) R pλ is prob of starting in best domain of attraction 9

10 Pure random search Multistart, with no Newton-like optimization Pick x r at random, record f (x r ), keep the best one Can beat multistart on rough functions Non Monte Carlo: Hooke and Jeeves Nelder-Mead simplex Low discrepancy search Pattern search 10

11 Another hard optimization f might be measurable only with noise Eg: controlling temp with noisy sensors, or missing random y with criterion E(f (x; y)). 11

12 Optimizing expectations Often we can simulate Y (make our own noise) ef n (x) = ^E(f (x; Y )) = 1 n nx i=1 f (x; Y i ) Optimize e fn for very large n (by e.g. Newton s) x 0! x 1! x 2! Wasteful: too much work at suboptimal x s Alternatives: use more iterations, smaller n 12

13 ' Kiefer-Wolfowitz (1952) For x 2 R p and random Y minimize E(f (x; Y )) Suppose p = 1: Estimate slope at x n for random Y ^g n (x) = f (x n + c n ;Y + n ) f (x n c n ;Y n ) 2c n Better yet ::: use common Y ^g n (x) = f (x n + c n ;Y n ) f (x n c n ;Y n ) 2c n Step in downhill direction x n+1 = x n a n^g n (x) Note: search distance c n move distance / a n 13

14 Kiefer-Wolfowitz, ctd. Search distance: c n Move distance: a n^g n a n > 0 a n! 0 c n > 0 c n! 0 P n a n = 1 P n (a n=c n ) 2 < 1 P an c n < 1 Eg c n = c=n 1=6 a n = a=n x n! Local optimum xλ jx n xλj = O(n 1=3 ) for examples above 14

15 Emulating nature Some natural systems approximately solve hard optimizations: 1. Cooling metals approach minimum energy 2. Plants and animals evolve 3. Ants communicate to find food These provide fruitful paradigms, Starting as proposed panaceas, Evolving to niches 15

16 Simulated annealing Slowly cooling metals anneal, finding near minimum energy arrangement of many atoms, eg atoms Boltzman distribution Pr(x) / exp H(x) kt H(x) = Energy of system (Hamiltonian) T = Temperature k = Boltzmann s constant Low T =) small H(x) strongly favored High T =) small H(x) weakly favored 16

17 Simulated annealing 1. Pick an energy H(x) to match f (x) (a) H(x) = ±f (x), or, (b) H(x) = log(f (x)), or etc. 2. Sample x n ο exp( H(x)=(kT n )) 3. Where T n decreases slowly to zero (a) e.g. cooling every samples (b) or continously like T n / 1= log(n) At x n propose random y n x n+1 = At step 2: Metropolis-Hastings 8 < : y n with prob. A(x n! y n ) x n with prob. 1 A(x n! y n ) A(x n! y n ) = min(1; exp((h(x n ) H(y n ))=kt )) Accept energy reductions, and some energy increases (to get out of local minima) 17

18 Travelling salesman problem Find the shortest (or a short) round trip x passing through each of n cities. Skill is in proposing a good y n for x n Often y n is a neighbor of x n If x n is A 1! A 2!! A n! A 1 Then y n may be: 1. A 1, A 2,, A r 1, then 2. A s, A s 1,, A r (running backwards), then 3. A s+1, A s+2,, A n, A 1 18

19 Tabu search Some random searches tend to cycle. Tabu list contains forbidden moves to inhibit cycling. 1. Pick x 0, set k = 0, start tabu list T k = ; 2. Randomly pick m neighbors of x k : y k;1 ;:::;y k;m 3. Disqualify any neighbors in tabu list T k 4. Find best non-tabu neighbor y kλ 5. If f (y k;λ) < f (x k ) then x k+1 = y k;λ 6. k = k + 1, update T k, goto 2 Customization details: Termination rules, picking neighbors, describing T k 19

20 ' Tabu ctd Suppose x is Extend credit if variable x 1 has been x 2 for the last x 3 days, and f is predicted value using x x 1 x 2 x 3 x Bank Balance over y 1 Bank Balance over y 2 Card debt under y 3 Bank Balance over y m Bank Balance over Tabu lists 1. Recently tried x s are tabu (e.g. last 7), or, 2. Recent changes can t be undone (e.g forbid: 14! 21! 14), or, 3. Recently changed variables can t be changed, or, ::: 4. :::unless tabu violation yields really good f (x k+1 ) 20

21 Genetic algorithms 1. Encode x in binary x = (3; 7; 4)! (011; 111; 010)! Define fitness F (x) related to f (x) 3. Initial random population x 0;1 ;:::;x 0;m 4. Generation k + 1: (a) Sample x k+1;i = x k;j with prob / F (x k;j ) (b) Randomly marry pairs of x k+1;i (c) Randomly crossover j j1000! j j0101 (d) Randomly mutate each bit (with very small probability) 21

22 Genetic variations 1. Non-binary encodings 2. Replace single strand by: (a) paired strands, with dominant and recessive traits (b) multiple (paired) strands, analogs of chromosomes 3. Replace pairing by analogs of natural and agricultural practices 4. Dynamically vary the fitness function Strength of GAs Works in parallel Population evolves to higher fitness Survivors have solved some problem Offspring may inherit solutions from both parents 22

23 Ant colony heuristic Model from nature: Ants work together to find food source communicate through pheremone left on paths For TSP: 1. Place m ants at random in graph 2. Each ant chooses a random next city higher probability on nearby cities 3. Ants add to pheremone level on each arc they use 4. At end of tour they add more pheremone to the path they took, inversely proportional to the length of their tour Features Still very new Parallel, like GA s, but communicating a few parameters to tweak artificial ants can use lookahead, memory, calculus competitive results reported for TSP 23

24 Starting points Stochastic approximation: Kushner and Yin Stochastic Approximation Algorithms and Applications, 1997 Simulated Annealing: Numerical Recipes, Laarhoven and Aarts: Simulated Annealing: Theory and Applications Tabu Search: F. Glover Annals of Operations Research vol 41, 1993 Genetic algorithms: Goldberg Genetic Algorithms in Search, Optimization, and Machine Learning, 1989 Falkenauer Genetic Algorithms and Grouping Problems,1998 Ant colony heuristic: Dorigo, Caro, Sambardella: Ant Algorithms for Discrete Optimization, Artificial Life, Vol 5, No 3,

25 Lingyu Chen s work ffl Optimize E(f (X; Y )) for X with random Y ffl By stochastic optimization ffl Applied to portfolio allocation ffl allowing for fund loads, taxes ffl Citing: Spall, Kushner, Yin, Breiman, Cover Jorge Picazo s work ffl Simulate multivariable American options ffl Like Longstaff and Schwartz ffl But using classification instead of regression ffl By boosted stumps ffl Also citing: Freund, Shapire, Friedman, Hastie, Tibshirani, Broadie, Glasserman, Fu, Laprise, Madan, Su, Wu 25

26 Contact Art Owen Lingyu Chen Jorge Picazo 26