Optimization in MATLAB Seth DeLand 4 The MathWorks, Inc.
Topics Intro Using gradient-based solvers Optimization in Comp. Finance toolboxes Global optimization Speeding up your optimizations
Optimization workflow Modify Design Variables No Initial Design Variables Model Objectives met? Yes Optimal Design
Optimization Problems Maximize Fuel Efficiency Minimize Risk Maximize Profits 4
Demo: Solving an optimization problem Simple objective function of variables: f x = log + x x x + x 4 Bound constraints:.5 x.5.5 x.5 6 4 - - - - 5
Portfolio Optimization Quadratic Constraints Maximize returns, constraint is risk minimize f T x subject to xt Cx Cmax Use fmincon interior-point algorithm Specify analytic gradients of objective function and constraints g = f (gradient of objective function) gc = Cx (gradient of constraint) Hessian (H) of Lagrangian L = f T x + λ( xt Cx Cmax) H = L = λc This is a second order conic programming problem 6
Demo: Cash-flow matching 5 x 5 Idea: Buy bonds to cover pension fund obligations Variables: How many of each bond to buy? Payments ($) 5 4 5 6 7 8 Time Period Constraints: Payments from bonds must be greater than or equal to pension fund obligations Objective: Minimize the size of the investment you make Cash Flow ($) 8 6 4 4 5 6 7 8 Time Period 7
Extending the problem x: n n 5 n = y n 5 = y 5 n n 5 y y 5 min x f T x s.t. Ax b A eq x = b eq lb x ub x(intcon) must be integers n n 5 y y 5 = cashflows n n 5 obligations cashflows zeros(8,5) n n 5 y y 5 obligations 8
Optimization Toolbox solvers Linear and Mixed- Integer LINPROG INTLINPROG Least Squares LSQLIN LSQNONNEG LSQCURVEFIT LSQNONLIN Quadratic QUADPROG Nonlinear FMINCON FMINUNC FMINBND FMINSEARCH FSEMINF Nonlinear Equation Solving FSOLVE FZERO Multiobjective FGOALATTAIN FMINIMAX 9
Key optimization problems addressed by financial toolboxes Financial Toolbox Mean-variance portfolio optimization Conditional Value at Risk (CVaR) portfolio optimization Econometrics Toolbox Parameter estimation of conditional mean/variance models Financial Instruments Toolbox Hedging portfolios Fitting interest rate curves Bootstrapping
Global optimization algorithms MultiStart Peaks GlobalSearch Patternsearch 8 6 4 - -4 Genetic Algorithm Simulated Annealing -6 - y Local minima - - - Global minima x
What is MultiStart? Run a local solver from each set of start points Option to filter starting points based on feasibility Supports parallel computing
MultiStart demo nonlinear regression lsqcurvefit solution MultiStart solution
What is Pattern Search? An approach that uses a pattern of search directions around the existing points Expands/contracts around the current point when a solution is not found Does not rely on gradients: works on smooth and nonsmooth problems 4
Pattern Search overview Iteration Run from specified x y - - - - - - x 5
Pattern Search overview Iteration Apply pattern vector, poll new points for improvement Mesh size = Pattern vectors = [,], [,], [-,], [,-] Pnew mesh _ size* pattern _ vector x *[,] x.8 4.6.6 First poll successful y.4 - Complete Poll (not default) - - - - - x 6
Pattern Search overview Iteration Mesh size = Pattern vectors = [,], [,], [-,], [,-] 4.6.8.6 y -.8.4. - - - - - - x -4 Complete Poll 7
Pattern Search overview Iteration Mesh size = 4 Pattern vectors = [,], [,], [-,], [,-] 4.6.8.6 y -.8.4. - - -4 - - - - x 8
Pattern Search overview Iteration 4 Mesh size = 4*.5 = Pattern vectors = [,], [,], [-,], [,-] 4.6.8.6 y -.8.4. - - -4 - - - - x 9
Pattern Search overview Iteration N Continue expansion/contraction until convergence 4.6.8.6 y -.8.4. - -6.5 - -4 - - - - x
Patternsearch demo stochastic function Stochastic objective function
What is a Genetic Algorithm? Uses concepts from evolutionary biology Start with an initial generation of candidate solutions that are tested against the objective function Subsequent generations evolve from the st through selection, crossover and mutation
How evolution works binary case Selection Retain the best performing bit strings from one generation to the next. Favor these for reproduction parent = [ ] parent = [ ] Crossover parent = [ ] parent = [ ] child = [ ] Mutation parent = [ ] child = [ ]
Genetic Algorithm Iteration Evaluate initial population y - - - - - - x 4
Genetic Algorithm Iteration Select a few good solutions for reproduction y - - - - - - x 5
Genetic Algorithm Iteration Generate new population and evaluate y - - - - - - x 6
Genetic Algorithm Iteration y - - - - - - x 7
Genetic Algorithm Iteration y - - - - - - x 8
Genetic Algorithm Iteration y - - - - - - x 9
Genetic Algorithm Iteration N Continue process until stopping criteria are met y - Solution found - - - - - x
Genetic Algorithm demo - Multiobjective Use gamultiobj to find Pareto front Two competing objectives: Second objective has sinusoidal component results in discontinuous front
Global Optimization Solvers designed to explore the solution space and find global solutions Problems can be stochastic, or nonsmooth Solvers in Global Optimization Toolbox MultiStart GlobalSearch simulannealbnd patternsearch ga (single- and multi-objective) Parallel Computing with ga, patternsearch, MultiStart
Speeding up with Parallel Computing Toolbox Global Optimization Toolbox: ga: Members of population evaluated in parallel at each generation patternsearch: Pattern evaluated in parallel (CompletePoll == on) MultiStart: Start points evaluated in parallel Optimization Toolbox: fmincon: parallel evaluation of objective function for finite differences fminimax, fgoalattain: same as fmincon In the objective function parfor
Key takeaways Solvers for a wide variety of problems New Mixed-Integer Linear Programming Solver Supply gradient and Hessian if possible Global solvers for multiple minimum and nonsmooth problems Speed up with parallel computing 4
Learn more about optimization with MATLAB MATLAB Digest: Using Symbolic Gradients for Optimization MATLAB Digest: Improving Optimization Performance with Parallel Computing Recorded webinar: Tips and Tricks Getting Started Using Optimization with MATLAB Recorded webinar: Global Optimization with MATLAB Products 5