PACBB: A Projected Adaptive Cyclic Barzilai-Borwein Method for Box Constrained Optimization*

Transcription

1 PACBB: A Projected Adaptive Cyclic Barzilai-Borwein Method for Box Constrained Optimization* Hongchao Zhang and William W. Hager Department of Mathematics, University of Florida, Gainesville, FL 32611, USA. {hzhang,hager} ufl.edu Summary. The adaptive cyclic Barzilai-Borwein (BB) method [DZ05] for unconstrained optimization is extended to bound constrained optimization. Using test problems from the CUTE library [BCGT95], performance is compared with SPG2 (a BB method), GENCAN (a BB/conjugate gradient scheme), and L-BFGS-B (limited BFGS for bound constrained problems). Key words: box constrained optimization, cyclic Barzilai-Borwein stepsize method, nonmonotone line search 1 Introduction Recently, we developed an adaptive cyclic Barzilai-Borwein (ACBB) method [DZ05] for solving unconstrained optimization problems. In this paper, we explain how the line search can be modified so as to solve bound constrained optimization problems of the form: min f(x), (1) where / is a smooth function, B= {x G 5R" L < x < U}, and L and U are upper and lower bounds, possibly infinite. For the bound constrained problem, the ACBB search direction are projected onto the feasible set. Hence, the new algorithm is denoted PACBB [projected adaptive cyclic Barzilai-Borwein method). A step in the BB method [BB88] is given by 4-iSk-i.. Xk+\=Xk-ak9k, ctk =-y, (2) Sk-iVk-i * This material is based upon work supported by the National Science Foundation under Grant No

2 388 Hongchao Zhang and William W. Hager Fig. 1. The projected line search. where g^ = Vf{xk) is the gradient, viewed as a column vector, Sfc_i = Xk Xk-i, and yu-i = Qk Qk-i' Advantages of BB type methods are their low memory requirements and their simphcity - in a neighborhood of a local minimizer, no Une search is needed since the convergence is hnear (see [DZ05]) when the Hessian is strictly convex at the solution. In the cyclic BB method, the same stepsize au is used repeatedly for several iterations - we observe in [DZ05] that by reusing a step for several iterations, convergence can be accelerated. In the adaptive cyclic BB method, we adaptively adjust the cycle length as the iteration progress. In the projected cyclic Barzilai-Borwein method, we project the ACBB iterates onto the feasible set B and perform a nonmonotone hne search between the current iterate and the projection point using the scheme in [DZ05]. 2 Algorithm The line search is illustrated in Figure 1. We first take a step along the negative gradient to a point Xk = Xk ctkgk, where the initial stepsize ak is a safeguarded version of either the previous stepsize or the newly computed stepsize if the cycle length has been reached. If the point Xk lies outside B, we compute

3 PACBB for Box Constrained Optimization 389 the projection Ps^Xk) of Xk onto B. The search direction dk = Pei^k) ^ ^k is a descent direction since B is convex. A nonmonotone line search is performed along the line segment connecting Xk and PB(xfc). If possible, we accept the point Ptsixk). Otherwise, we backtrack towards Xk- When Xk lies outside of B, the next initial stepsize a^+i is given by the BB formula found in (2). In a forthcoming paper, we prove that when B is replaced by a closed, convex set n, we have liminf Pr2(a;fc ~ gk) -XfcHoo = 0 fc >oo for a gradient projection/nonmonotone hne search scheme with the structure depicted in Figure 1 3 The numerical results In this section, we compare the performance of the projected adaptive cychc BB algorithm (PACBB) to the SPG2 algorithm developed in [BMROO, BMROl], to the GENCAN algorithm developed in [BM02], and to the L- BFGS-B version of the limited BFGS method for box constrained optimization developed in [BLN95, ZBN97]. All codes were written in Fortran and compiled with f77 (default compiler settings) on a Sun workstation. The GENCAN codes were obtained from Jose Martinez's web page: and the L-BFGS-B codes were obtained from Jorge Nocedal's web page: or For each code, we stopped the iterations if either \\PB{Xk-gk)-Xk\\oo<W~'^ (3) \fixk)-fixk.^)\/{l + \fixk)\)<10-''. (4) We also terminated a code if the number of function evaluations was more than 10^. The test set consisted of all bound constrained problems from the (2002) CUTE hbrary [BCGT95] with more than 50 variables. For all problems where more than one choice of the dimension is given, we use the largest dimension. The numerical results are posted at the following web page: Relative to the CPU time, the numerical comparison of PACBB with the other three routines can be summerized as follows: PACBB is faster than SPG2 in 36 problems, while SPG2 is faster in 6 problems.

4 390 Hongchao Zhang and William W. Hager PACBB is faster than GENCAN in 33 problems, while GENCAN is faster in 9 problems. PACBB is faster than L-BFGS-B in 34 problems, while L-BFGS-B is faster in 11 problems. Excluding the problems where the difference in CPU time was less than 10%, the numerical results can be summerized as follows: PACBB is faster than SPG2 in 33 problems, while SPG2 is faster in 4 problems. PACBB is faster than GENCAN in 30 problems, while GENCAN is faster in 8 problems. PACBB is faster than L-BFGS-B in 32 problems, while L-BFGS-B is faster in 9 problems. Figure 2 shows the performance profiles, proposed by Dolan and More [DM02], for the four codes. That is, for the methods analyzed, we plot the Fig. 2. Performance profiles fraction P of problems for which any given method is within a factor r of the best time. In a performance profile plot, the top curve is the method that solved the most problems in a time that was within a factor r of the best time. The percentage of the test problems for which a method is the fastest is given on the left axis of the plot. The right side of the plot gives the percentage of the test problems that were successfully solved by each of the methods. In essence, the right side is a measure of an algorithm's robustness.

5 PACBB for Box Constrained Optimization 391 Since the top curve in Figure 2 corresponds to PACBB, this method yielded the best CPU time performance for this set of 48 test problems with dimensions ranging from 50 to 15,625. Similar to SPG2, the algorithm PACBB is suitable for large dimensional problems due to its low memory requirements. It is pointed out in [BMROO] that for ill-conditioned problems, SPG2 may converge slowly. Although PACBB seems to deal with ill-conditioning better than SPG2, both GENCAN and L-BFGS-B are more efficient for the very ill-conditioned problems. Finally, the PACBB algorithm is very easy to implement and it has many promising applications (see [BCM99, GHR93]). References [BB88] J. Barzilai and J. M. Borwein, Two point step size gradient methods, IMA J. Numer. Anal, 8 (1988), pp [BCM99] E. G. Birgin, I. Chambouleyron, and J. M. Martinez, Estimation of the optical constants and the thickness of thin films using unconstrained optimization, J. Comput. Phys., 151 (1999), pp [BM02] E. G. Birgin and J. M. Martinez, Large-scale active-set box-constrained optimization method with spectral projected gradients, Comput. Optim. Appl., 23 (2002), pp [BMROO] E. G. Birgin, J. M. Martinez, and M. Raydan, Nonmonotone Spectral Projected Gradient Methods for convex sets, SIAM J. Optim., 10 (2000), pp [BMROl] E. G. Birgin, J. M. Martinez and M. Raydan, Algorithm 813: SPG - software for convex-constrained optimization, ACM Trans. Math. Software, 27 (2001), pp [BLN95] R. H. Byrd, P. Lu and J. Nocedal, A Limited Memory Algorithm for Bound Constrained Optimization, SIAM J. Sci. Comput., 16, (1995), pp [BCGT95] I. Bongartz, A. R. Conn, N. I. M. Gould, and P. L. Toint, CUTE: constrained and unconstrained testing environments, ACM Trans. Math. Software, 21 (1995), pp [DZOl] Y. H. Dai and H. Zhang, An Adaptive Two-point Stepsize Gradient Algorithm, Numer. Algorithms, 27 (2001), pp [DZ05] Y. H. Dai, W. W. Hager, K. Schittkowski and H. Zhang, Cyclic Barzilai- Borwein Stepsize Method for Unconstrained Optimization, March, 2005 (see [DM02] E. D. Dolan and J. J. More, Benchmarking optimization software with performance profiles. Math. Prog., 91 (2002), pp [GHR93] W. Glunt, T. L. Hayden, and M. Raydan, Molecular conformations from distance matrices, J. Comput. Chem., 14 (1993), pp [GLL86] L. Grippo, F. Lampariello, and S. Lucidi, A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal., 23 (1986), pp [RayOl] M. Raydan, Nonmonotone spectral methods for large-scale nonlinear systems Report in the International Workshop on "Optimization and Control with Applications", Erice, Italy, July 9-17, 2001

6 392 Hongchao Zhang and William W. Hager [Toi97] Ph. L. Toint, A non-monotone trust region algorithm for nonlinear optimization subject to convex constraints, Math. Prog., 77 (1997), pp [ZBN97] C. Zhu, R. H. Byrd and J. Nocedal, L-BFGS-B: Algorithm 778: L-BFGS-B, FORTRAN routines for large scale bound constrained optimization, ACM Trans. Math. Software, 23 (1997), pp