User s Guide to Climb High. (A Numerical Optimization Routine) Introduction

Transcription

1 User s Guide to Climb High (A Numerical Optimization Routine) Hang Qian Iowa State University Introduction Welcome to Climb High, a numerical routine for unconstrained non-linear optimization problems on the basis of the BFGS algorithm. It is especially ideal for problems in applied econometrics such as maximum likelihood or simulated likelihood estimation. It also works well in solving non-linear equations and non-linear least squares. This is a gradient-based algorithm, so local information is used to infer the global shape of the objective function. When the author wrote this program, he imagined himself as an alpinist equipped with an altimeter and clinometer, and he tried to climb as high as possible, and as efficiently as possible. All he knew is the shape of the earth under his feet, so the essential task is to decide what direction to proceed and when to recalibrate his direction. The landscape can be complicated and he might encounter rugged terrain and pits, rivers and other obstacles, and even cliffs. As a result, scientists cannot provide him with a complete recipe on how to climb. He must learn by doing, and rise from tumble. In the same vein, no prescribed algorithm can always find out the optimum. A good computer routine to implement the algorithm must be adaptive and resilient. When the author wrote this program, he found that numerical optimization is more like an art than a science. The altitude of climbing is the touchstone of the program. Climb High blends a variety techniques to flexibly adjust the step size in the line search of the maximum / minimum. Climb High is written in the MATLAB language. Compared with the canned optimization routines in MATLAB, the Climb High has the following features: Less picky on the starting values More tolerant of poorly scaled problems Stable and seldom produce error messages and stop halfway in the optimization Able to find an optimum at least as good as the canned routines in most cases Reasonable in speed Just in case -- suppose you find the canned routines in MATLAB does not work satisfactorily for your optimization problem at hand, may I suggest you give Climb High a try?

2 Usage Climb High optimization package contains two major MATLAB functions, one is MAXIMIZE.m and the other is MINIMIZE.m. In addition, it has a bonus function EQUATION.m to solve non-linear equation systems. Users can call the functions in the same way as the MATLAB canned optimization routines such as fminunc() and fminsearch(). The grammar looks like: [x, fval, exitflag, Gradient, Hessian] = MAXIMIZE(fun, x0, options) [x, fval, exitflag, Gradient, Hessian] = MINIMIZE(fun, x0, options) [x, fval, exitflag, Gradient, Hessian] = EQUATION(fun, x0, options) where fun is the user supplied objective function, which can be a function handle, anonymous function, or a character string specifying a function (see below). x0 is the starting value. If x0 is not specified, the program will try to generate one automatically. options is a structure containing optimization config. If options is not specified, the default values will be loaded. options consist of the following: MethodGradient: 0 = user s supplied analytic gradients (fun must return in the second output argument) 1 = numerical gradients with forward difference 2 = numerical gradients with central difference (default) 3 = numerical gradients with Richardson extrapolation of order 1 4 = numerical gradients with Richardson extrapolation of order 2 5 = numerical gradients with Richardson extrapolation of order 3 6 = numerical gradients with Richardson extrapolation on central difference DiffDelta: The small increment to compute numerical gradients ( default = 1e-6) Display: 1 = display iteration results on screen (default), 0 = write into a plain text file MaxIter: Maximum number of iterations (default = 200) TolFun: Convergence criterion for both parameters and function values (default = 1e-6) MaxAttempt: Maximum attempts to adjust line search window and darts shoot (default = 15) MaxDarts: In each attempt of darts shoot, the number of darts (default=5) NelderMead: If line search fails, whether to switch to simplex search mode (default = 1) Minutes: If x0 is not specified, the time allowing computer to search an initial value (default = 1, but not precise) FinalCheck: Post convergence random check of global maximum / minimum (default = 1) Rounding: If final parameters are very close to integers, whether to round results (default = 1) You may set options by options = struct('methodgradient',1,'display',0);fields are case sensitive. x is optimized parameters which maximize / minimize the objective function

3 fval is optimum function value that Climb High reaches exitflag is the marker of how the optimization routine terminates 1 = Function value, parameters, gradients all converge 2 = Both function value and parameters converge, but gradients remain non-zero 3 = Function value cannot further improve 4 = Maximum iteration reached, but not converged 0 = Error occurs somewhere Gradient is the gradients evaluated at optimum Hessian is the Hessian evaluated at optimum (It takes time to compute. Do not add this output argument if you do not need Hessian) The author tries to make the program as friendly as possible. The least requirement is that you just provide an objective function. For a complicated problem, you might restore your objective function in a stand-alone MATLAB file in the current folder. For example, you can create a function called myfun(): y = myfun(x) y = - (3 * x(1)^2 + 2 * x(1) * x(2) + x(2)^2) Then you can use MAXIMIZE(myfun) to find optimum, which are trivially two zeros analytically. If you do not provide a starting value, the program will try to generate one. However, the ability of automatic generation is limited. For complicated problems, it may or may not work. Since this problem is so easy, we can also use an anonymous function: MAXIMIZE(@(x) - (3 * x(1)^2 + 2 * x(1) * x(2) + x(2)^2)) Lastly, you can also use character strings to represent the optimization problem. For example: MAXIMIZE('- (3 * x(1)^2 + 2 * x(1) * x(2) + x(2)^2)') MAXIMIZE('- (3 * x1^2 + 2 * x1 * x2 + x2^2)') MAXIMIZE('- (3 * x^2 + 2 * x* y + y^2)') In addition, Climb High provides a graphic interface GUI.m to optimize or solve problems. It is convenience to use graphic interface when you have a small-scaled problem, say finding the root of a function, maximizing an objective function with one or two variables etc. GUI.m is a Matlab script, and once you open it, press F5 to run the script. (Alternatively, there are buttons on the taskbar and in the menu to run the script, or simply type GUI in the command window.)

4 Algorithm Starting Values For complicated problems, users are recommended to provide a decent starting value to ensure the fast convergence of the optimization problem. However, when Climb High is designed, it takes into account the fact that most users do not want to provide starting values. This is understandable, since users do not know the optimal values and feel afraid they may provide a bad guess at the beginning. They are forced to look for a good one manually only if the optimization routine gets stuck. Climb High is less picky on the starting values since its algorithm is adaptive and resilient. Even if the starting value is not specified, the program will try to generate one by brute force search for a while. This is nothing but a random and wild guess, and it is likely to work for small-scaled problems. However, for large-scaled problem, it may or may not work. During the specified search time (see options Minutes), the program will compare candidates, if any, and pick the one with best function value. If user specified initial value does not return a finite function value, the program will also search for an appropriate one with brute force. Gradients Climb High uses BFGS algorithm to locate the optimum, so gradients are necessary. Users are recommended to provide analytic gradients if it is feasible to do so, since analytic gradients will greatly improve the quality of the iteration. However, it is understood that most users feel reluctant to provide analytic gradients. So the Climb High provides an option of numerical gradients with several varieties, which differs in speed and accuracy. The first one is the forward difference. To fix ideas, suppose we are going to approximate the derivative of evaluated at some point. By Taylor expansion of, we have where ( ), is between. The reminder is of order. Put, we obtain the forward difference formula:

5 In complicated numerical optimization, function evaluations take up most of the computation time. Therefore, the speed of the codes is roughly proportional to how many times we evaluate the objective function. No matter whether gradients are computed, we nevertheless need to compute, thus it is a sunk cost. The variable cost of computing gradients by forward difference is one additional evaluation of for each dimension. This is the most parsimonious way to compute gradients, and the precision is also lowest. The theoretical error shrinks at the rate as. However, the fundamental problem of computer is that it has truncation errors in computation, since a double floating number can only keep 15 to 16 digits. To compute, we have to take a difference of the form. When is too small, many digits will be lost. If all digits are lost, it will return something absurd and undermine the numerical optimization procedure. For example, if and, then we do not have many digits left after taking the difference. However, if we increase, we might alleviate the truncation error, say. However, a large leads to more prominent theoretical errors since differs from by. In summary, using a small to compute gradient is accurate (smaller in theoretical error) but dangerous (larger in truncation error), while using a larger is less precise (larger in theoretical error) but safer (smaller in truncation error). Climb High has default choice of increment for forward difference. The second option of numerical gradients is central difference. Recall the Taylor expansion: Both and are of order. Put and take the difference. Note that both and terms will cancel. So we obtain the central difference formula:

6 Note that central difference does not take advantage of the free item, and it evaluates and instead. As a result, the variable cost of central difference is twice as high as forward difference, but the accuracy improves to as well. The third option of numerical gradients is Richardson extrapolation of order 1. Richardson extrapolation is a refinement of the gradients, and can be based on either forward difference or central difference. Again, we start from the Taylor expansion and this time we put. It follows that Cut by an half, we obtain. /. / Since we intend to cancel term, we define Richardson extrapolation of order 1 as. / Since is, and. / is, it follows that. Richardson extrapolation of order 1 takes advantage of the freebie, and evaluate two more points. / and, and the precision improves to. It is interesting to compare the central difference and Richardson extrapolation of order 1. Both of them will have two more points to evaluate, and both have an error of order. For the central difference, we can rewrite the reminder terms as extrapolation has a reminder term. The Richardson. However, that does not implies Richardson extrapolation is twice as accurate as central difference. Recall that central difference evaluates, so the span on the horizontal axis is, while in Richardson formula, evaluation. / has the span and evaluation has the span. So the mirage of smaller reminder in Richardson formula is merely caused by a small span. Actually, if we fix the span in the two approaches, the central difference should have a smaller reminder terms. However, Richardson formula has the

7 advantage of evaluating points on one side of instead of two sides, thus useful for evaluating points in the corner. In fact, central difference and Richardson extrapolation of order 1 are the same thing. Both can be interpreted quadratic approximate to and use the slope of quadratic function at to substitute. Central difference uses three points and evaluate at, and Richardson extrapolation of order 1 uses and evaluate at. The fourth option of numerical gradients is Richardson extrapolation of order 2. Note that Cut by an half, and take the difference with suitable multiplication, we obtain Richardson extrapolation of order 2:. / Richardson extrapolation of order 2 takes advantage of the freebie, and evaluate three more points. /,. /,, and the precision improves to. The fifth option of numerical gradients is Richardson extrapolation of order 3, which is another level of refinement.. / Richardson extrapolation of order 3 takes advantage of the freebie, and evaluate four more points. /,. /,. /,, and the precision improves to. The sixth option of numerical gradients is Richardson extrapolation on the basis of central difference. Note that once we take the difference of and, all the derivatives of even order should cancel, so we have So we can refine by. /

8 Richardson extrapolation on the basis of central difference has no freebie, and evaluate four more points,. /,. /,, and the precision improves to. Method Precision Free item Cost Forward difference 1 1 Central difference 0 2 Richardson extrapolation Richardson extrapolation Richardson extrapolation Richardson extrapolation CD 0 4 As the table suggests, you will not find cash on the Wall Street, since every improvement comes with a price, without any arbitrage opportunity at all. The default choice of Climb High is central difference. Also, considering the fact that Richardson extrapolation is precise in general, the choice of should be larger so as to reduce the truncation error. So if you set the DiffDelta option, the program uses for forward and central difference, and for Richardson extrapolation. Newton Type Iteration Once the objective function and gradients can be evaluated, the alpinist has equipped himself with an altimeter and clinometer. The next step is to climb the mountain efficiently. Rather than scramble randomly, the alpinist had better to predict the ensemble landscape from the shape of the earth underneath his feet. However, the path may be complicated so that he has to update his beliefs and predictions from time to time. An obvious way to infer the global shape from the local is by quadratic approximation. It has the advantage that the maximum / minimum of a quadratic function can be computed analytically. Suppose our objective function is, and we are currently at a point. We Taylor-expand the objective function up to the second order. ( ) where for some, -. are gradients and Hessian evaluated at. It is possible to write down the reminder term ( ) explicitly. However, we are not going to analyze the errors here, so we simply treat the quadratic term

9 as an approximation to the objective function. Our goal is to find the maximum of, so we solve optimum analytically for the quadratic function. FOC: The solution is This is the Newton type formula. Note that is the optimum for the quadratic approximation to, not itself. Once we arrive at, there is a need to reevaluate the landscape ahead and iterate the quadratic approximation. The iteration formula looks like If the objective function is globally concave, it should converge to the maximum of eventually. Unfortunately, the above procedure cannot work well for many empirical problems. First, virtually no one is willing to provide analytic Hessian for their optimization problems. However, due to precision loss in finite difference, numerical Hessian is usually of poor quality and very computationally expensive. Second, it is just a wishful thinking that our is globally concave (Consider the pits, rivers, cliffs facing the alpinist). Thus there is no guarantee the dogmatic Newton type iteration will succeed with maximum. The second problem can be partially addressed by line search of step size, while the first problem can be well addressed by BFGS algorithm. BFGS Algorithm Proposed independently by Broyden, Fletcher, Goldfarb and Shanno around the year 1970, BFGS is a power algorithm to learn the Hessian from the gradients, when the analytical or numerical Hessian cannot be computed directly. During the iteration, we obtain gradients, either analytically or numerically. Suppose is one-dimensional, despite precision loss we may use to approximate the second derivative. However, for multi-dimensional problem, it is impossible to infer Hessian merely from,, since their difference reflects the change of all variables from to. However, a first order approximation of suggest that

10 If we propose a method to infer from the information of historical gradients * +, it should satisfy that equality. BFGS uses an iterative procedure to gradually learn the Hessian where,. When the iteration converges, should well approximzte. It is easy to verify that the converged does satisfy the above identity. Therefore, * + is used as a proxy for Hessian in the Newton-type iteration. Step Size The alpinist might face all kinds of difficulty in climbing. A rigid procedure is not likely to lead alpinist far. There is only one golden principle: try whatever path that goes towards a higher altitude. The common practice is to use the adapted BFGS formula where is the step size, to update the parameters. We centainly want to choose a we want to such that the objective function is as high as possible. Therefore, Since the analytic solution is usually unavailable, Climb High uses a variety of ways to choose a. Climb High first tries the exact line search. Setting an initial window such as { }, the program compares which can bring a highest function value. If that happens to be on the margin, the window will be resized. For example, if will test is the best one among the five candidates, the program as well. The MaxAttempt option is used to control the maximum attempt of resizing the window. Its trial process will also be displayed on the screen, if you do not manually turn it off by the Display option.

11 Since exact line search is costly, it is not appropriate to use it every time. If the exact line search successfully identifies an optimal, in the next iteration the program will switch to the inexact line search mode. Since we expect eventually will be a proper choice, this step size will be tested first. To decide whether to accept it or not, Climb High uses the Wolfe condition, which includes the Armijo test and the strong curvature test. i) Armijo test:, where is a small positive number, say 1e-4 ii) Curvature test:, where is a positive number, say 0.9 If passes the two tests (as is usually true in the epilogue of iterations), it is accepted. Otherwise, Climb High will search for other candidates. One method of inexact line search is the quadratic approximation. Let We use a quadratic function w.r.t. to approximate, say We need to pin down the coefficients first. What is the least costly way to do so? At this moment fails the tests, but at least we have already computed the objective function and its gradients at as well as the objective function evaluated at. That is, we have where. It follows that The quadratic function has the maximizer, which is the step size we will test next time. If quadratic approximation fails to pass the tests, we can further use cubic approximation, say

12 The readily available points,,, can be used to pin down the coefficients. To be concrete,. / [ ] The cubic function has the maximizer + If cubic approximation fails, Climb High will try cubic approximation again with,,,. The trial process will be displayed on the screen, showing Cubic I approximation, Cubic II approximation. If all of these inexact line search attempts fail, Climb High will go back to the exact line search mode. If for any reason even the exact line search fails to improve the objective function, Climb High will switch to the Nelder Mead simplex search mode, which is explained below. Simplex search Nelder Mead simplex search is a gradient-free optimization algorithm. It actually can work standalone to locate the optimum. Climb High uses this algorithm only if the BFGS line search fails to improve the function value, which might occur when data are poorly scaled or there is randomness in the objective function as in the simulated likelihood. Simplex search may help to resolve those complexities. The heuristic Nelder Mead algorithm relies on the patterns of objective function values to search for a better function value. Suppose the optimization problem is where is an n-dimensional vector. In an n-dimensional plane, n+1 points can form a simplex. For example, in a 2D plane, 3 points constitute the vertices of a triangle. Nelder Mead algorithm starts from an initial simplex, and replace the worst vertex with a new point so as to form a better simplex.

13 First of all, we pick n+1 points, evaluate their function values, and sort them from worst to best, say,, where. We are going to replace with a better point. To this end, we first define the centroid (center of mass) of the rest n points The new point will be chosen somewhere along the line connecting and. Let this new point be defines a reflection point, since and reflects each other with respect to the centroid. defines an extended reflection point, since the distance, - is twice as long as that of, -. defines an outward contraction point, since the distance, - is half of, -. defines an inward contraction point, since lies at the midpoint of and. Essentially, we compare these four candidates and pick the best one. However, that means four additional evaluations of the objective function. Nelder Mead algorithm actually computes them in order. If one point is desirable, we will accept it, without evaluating the rest. To be exact, If reflection point satisfies, we just accept and use it to form a new simplex. Since is better than, the overall quality of the simplex is improved. If reflection point satisfies, we doubt whether extended reflection point (, call it ) can do an even better job, so we have a test. If, we surely will use to form a new simplex. Otherwise, we still use to form the new simplex. No matter which one is larger, the quality of the simplex is greatly improved since we directly observe function value rises. If reflection point satisfies, which implies travels too far. So we cut the reflection by an half and test outward contraction point ( call it ). If, we then accept to form new simplex. Otherwise, this is a failed contraction and we have to shrink the simplex (see below). If reflection point is so poor such that, we completely discard and try the midpoint of and in the hope that it could be better than. This is the inward contraction point ( call it ). If, we then accept to form new simplex. Otherwise, this is a failed contraction and we have to shrink the simplex (see below). In the unfortunate case when we have to shrink the simplex, we just cut the length of each side of the simplex by an half towards. That is, Once we have a new simplex, we repeat the above process. Hopefully the simplex may converge eventually. Since we always pick the best vertex of the simplex, simplex iteration will never deteriorate.

14 Darts shoot If line search fails and simplex search also fails to improve the function value, the last resort is the darts-throwing mode. The program will randomly try the points nearby, less nearby, and relatively remote from the current point. If it happens to increase the objective function, that point will be used in the next iteration. The MaxDarts option is used to control the number of darts in each trial, MaxAttempt also controls the maximum rounds of dart shooting game. Of course, the results of darts-shooting depend on your luck. If all these attempts fail, Climb High is unable to climb higher and has to regretfully exit. In this case, the exitflag is 3. Convergence Hopefully, after N iterations we will welcome the convergence, which means both parameters and the function values level off. The MaxIter, TolFun options are used to control the maximum iterations and convergence criterion respectively. For an unconstrained problem, convergence should also imply gradients become zero. If everything converges, the exitflag is 1. However, in the presence of rivers, cliffs and other obstacles on the mountain, gradients can be non-zero. In that case, Climb High will report both function value and parameters converge, but gradients remain non-zero with exitflag equal to 2. Lastly, if the FinalCheck option is on, Climb High will randomly check the points nearby, less nearby, and relatively remote from the converged point to ensure this is the global maximum. If there is no negative evidence, Climb High will exit will success. Otherwise, the converged point is likely to be a local maximum, and the program will continue to search until new converge or maximum iteration, whichever comes first. Right before Climb High reports its final results to the commander, it checks the optimized parameters. If it is close enough to an integer (discrepancy small than TolFun), it will round the results. You can turn off this feature by resetting the option Rounding. Climb High reports on request optimized parameters, optimum function value, type of exit as well as gradients evaluated at the optimum at no additional cost. Also, if the Hessian evaluated at the optimum is need, Climb High will compute it using finite differencing in the same way as it computes gradients. However, please note that for large scale problems with many parameters, computing Hessian takes time.

15 Miscellaneous As a by-produce of the optimization routine, Climb High can also be used to solve non-linear equations. The bonus EQUATION.m recasts non-linear equations into a non-linear least square problem and minimize its function value. EQUATION.m can be called in the same way as Matlab s lsqnonlin.m. The users supplied objective function should be vector-valued, representing the values of each equation evaluated at current parameter values. If the user also provides analytic Jacobian, the Jacobian matrix (each row is for an equation, each column is for a parameter) should be returned in a second output argument of the user-supplied function. Admittedly, non-linear equation systems might be better solved by Newton method directly. Even if it is rewritten into a non-linear least squares, algorithms like Levenberg Marquardt would be more appropriate since Hessian can be well approximated by a matrix-matrix multiplication of Jacobian. Nevertheless, there is nothing wrong to solve non-linear equations as if it is an optimization problem. It actually works well with Climb High. During the iterations, users can watch the change of squared sum of function values. If it shrinks towards zero and eventually become negligible, the equation systems are successful solved. In the output arguments of EQUATION.m, x stands for optimized parameters, and fval is a vector of the residual of each equation. Note that sometimes non-linear equations do not have solutions, in that case function values will not drop to zero and optimized parameters are not solutions. Also, if equations have more than one theoretical solutions, Climb High can report at most one, unless multiple starting values are tested in the hope of converging to other solutions. Contact information Written by Hang Qian, Iowa State University Contact me: matlabist@gmail.com More econometrics routines and pedagogical economics software are available at