3 Interior Point Method

Transcription

1 3 Interior Point Method Linear programming (LP) is one of the most useful mathematical techniques. Recent advances in computer technology and algorithms have improved computational speed by several orders of magnitude. Therefore, many problems that could not be addressed previously can now be successfully solved. One of the most well-nown LP methods is Simplex Method (SM), developed by Dantzig 34]. This method has remained competitive due to continuous improvements and research, and is considered very robust, general purpose, and efficient. It has been used to solve problems in operational research, business, and engineering. However, its polynomial complexity, with respect to the size of the problems, has led the scientific community to search for new algorithms with better guaranteed performance. In 1979, Khachiyan 35] developed the ellipsoid method and proved that LP problems belong to the class of polynomially solvable problems. His method reaches a solution in a time that is at worst polynomial in the problem size. However, although ideal for theoretical grounds, the algorithm failed to compete with the Simplex method. In 1984 Karmarar 36] proposed the projective algorithm which has better complexity bounds than Khachiyan s. Karmarar claimed that his method presents excellent performance for large problems when compared to SM. He did not show any results, but his wor originated a great deal of research activity. From , more than 1300 papers were published 37] in respect of this Interior Point Method (IPM) for LP. This method presents not only theoretical good performance but it is also very well suited for practical applications. SM and IPM wor in different ways. SM visits the vertices of the feasible region until it reaches the optimal, whereas IPM starts from the interior (or the exterior) of the feasible region and reaches the optimal through successive steps. Each SM iteration is computationally inexpensive, but the method usually requires many iterations. On the other hand, each IPM iteration is computationally expensive, but requires only a few iterations to converge.

2 Chapter 3. Interior Point Method 29 IPM is available in many commercial codes, such as CPLEX, LIPSOL or MATLAB. One of the most successful IPMs is the Primal-Dual IPM which is briefly discussed in the next section. 3.1 Optimality Conditions The LP problem in its standard form, also called the primal problem (PP) 38], can be expressed as: min x s.t. c T x Ax = b x 0, (3-1) where c, x R n ; b R m ; and A is an m n matrix with full row ran. Associated with the PP is the dual problem (DP): max y s.t. b T y A T y + z = c z 0, (3-2) where y R m, and z R n. The duality measure: µ = xt z n = ct x b T z n (3-3) is nonnegative and is zero only at optimal values (x, y ). It is commonly used as the test parameter to stop iteration. The Karush-Kuhn-Tucer conditions are: A T y + z Ax = c = b XZe = 0 (3-4) (x, z) 0, where X is a diagonal matrix with diagonal x, similarly for Z, and e is a vector of all ones with suitable dimension depending on the context. The vector (x, y, z ) is called a primal dual solution Primal-Dual Path Following The primal-dual methods find solutions (x, y, z ) by applying variants of Newton s method to the first three equations of Equation (3-4), so that x, z 0 are satisfied at every iteration. We can restate the optimality

3 Chapter 3. Interior Point Method 30 conditions in terms of the mapping F : R 2n+m R 2n+m : A T y + z c F (x, y, z) = Ax b = 0 XZe (3-5) (x, z) 0. Newton s method is used to obtain the next iteration. It consists of obtaining the search direction ( x, y, z) by solving: x J (x, y, z) y = F (x, y, z), (3-6) z where J is the Jacobian of F. If r b = Ax b and r c = A T y + z c, then: 0 A T I x r c A 0 0 y = r b. (3-7) Z 0 X z XZe The next feasible iteration is: (x, y, z) + α ( x, y, z), (3-8) where parameter α (0, 1] is chosen such that the next iteration is ept strictly feasible, i.e., (x, z) > 0. Usually, we tae a step toward a point such that x i s i = σµ, where µ is the current duality measure and α, σ 0, 1] a. Figure 3.1 shows the steps from an initial point to the optimized one. The modified step equation is then: 0 A T I x A 0 0 y = Z 0 X z r c r b XZe + σµe. (3-9) The Central Path The central path is the arc of points C parameterized by a scalar τ > 0. Each point of (x τ, y τ, z τ ) C satisfies: a Further details on σ will be provided later in this chapter.

4 Chapter 3. Interior Point Method 31 Figure 3.1: Steps towards the optimum in the x space. A T y + z = c Ax = b XZe = τe (x, z) 0. (3-10) Equations (3-10) correspond to the optimality conditions for a logarithmic barrier version of the primal problem. This is obtained by adding a barrier term which penalizes non-positive values. The barrier problem is defined as: min c T x τ n i=1 ln x i s.t. Ax = b. (3-11) Note that equations (3-11) approximate the original system (3-4) as τ 0. Then the central path gives a route to reach the optimum. Figure 3.2 shows contour lines for the logarithmic barrier problem for different values of τ, where the dots indicate optimum values of τ. The σ parameter is called the centering parameter and values close to 1 give directions closer to the central path C. A zero value gives the standard Newton step. Practical algorithms tend to search an equilibrium between centrality and taing larger steps to the optimum by carefully choosing σ and α Long-Step Path Following In this algorithm the α parameter in each iteration is chosen to be the largest value to satisfy x + α x 0, for primal variables (α pri ), and z + α z 0, for dual variables (α dual ).

5 Chapter 3. Interior Point Method 32 Figure 3.2: Central path and contours. (a) τ = 100. (b) τ = 3. (c) τ = 1. (d) τ = Matlab code for maing this graphic is showed in Appendix A. The η (η (0, 1)) parameter is usually set very close to 1 to prevent zero values for x and z, while at the same time maximizing the step lengths Predictor-Corrector Algorithm As discussed above, the centering parameter, σ, has the effect of producing iterations closer to the central path (if σ 0) with slow convergence or far from the central path (if σ 1) with faster convergence, but close to the boundaries of the feasible region. The predictor-corrector algorithm attempts to balance these behaviors to accelerate the convergence. The predictor-corrector finds σ in an adaptive way. First, the Newton s search direction is found by solving Equation (3-15): the predictor step. The maximum α values are found from Equation (3-16) and σ is calculated according to the heuristic σ = (µ aff /µ) 3. The centering parameter, σ, is included in Equation (3-17) for the corrector step, and the search direction is also obtained. The predictor-corrector needs to solve two linear systems

6 Chapter 3. Interior Point Method 33 Algorithm 1 Long-Step IPM 1: Initialization 2: Given σ 0, 1] and (x 0, y 0, z 0 ) 3: for = 0, 1, 2,... do 4: Set µ = (x ) T z /n 5: Solve to obtain ( x, y, z ) 6: Choose α pri 7: Set 8: end for 0 AT I A 0 0 Z 0 X α pri,max = α pri and α dual min x y = z such that x i i: x i <0 x i = min(1, η α pri,max ),, α dual,max = αdual r c r b XZe + σµe min z i i: z i <0 z i (3-12) = min(1, η α dual,max ) (3-13) ( x +1 = x + α pri x y +1, z +1) = ( y, z ) ( + α dual y, z ) (3-14) with the same coefficient matrix but different right hand terms. Direct solvers deal with this by factorizing once and maing two bac substitutions at low cost Solving the linear systems As discussed above, Equations (3-12), (3-15) and (3-17) only differ on the right hand side and they may be expressed more generally as: 0 A T I x r c A 0 0 y = r b. (3-20) Z 0 X z r xz Since x and z are positive, X and Z are invertible, and Equations (3-20) can be rewritten as: D 2 xz A 0 A T ] ] x y ] r c + X 1 r xz = r b z = X 1 r xz X 1 Z x, (3-21) where D xz = Z 1/2 X 1/2. This is nown as the augmented system and can be expressed as:

7 Chapter 3. Interior Point Method 34 Algorithm 2 Predictor-Corrector IPM 1: Initialization 2: Given (x 0, y 0, z 0 ) 3: for = 0, 1, 2,... do 4: Solve to obtain ( x aff, y aff, z aff) 5: Choose α pri aff, αdual aff α pri aff α dual aff 0 AT I A 0 0 Z 0 X x aff y aff = z aff and µ aff such that ( = min 1, min ( = min i: x aff i <0 1, min i: z aff i <0 x i x aff i z i z aff i µ aff = (x + α pri aff xaff ) T (z + α dual ) ) r c r b (3-15) XZe aff zaff )/n (3-16) 6: Set σ = (µ aff /µ) 3 7: Solve to obtain ( x, y, z) 0 AT I x r c A 0 0 y = r b (3-17) Z 0 X z XZe X aff Z aff + σµe 8: Calculate α pri, αdual 9: Set 10: end for α pri = min(1, η α pri,max ), αdual = min(1, η α dual,max ) (3-18) ( x +1 = x + α pri x y +1, z +1) = ( y, z ) ( + α dual y, z ) (3-19) AD 2 xza T y = r b AXZ 1 r c + AZ 1 r xz z = r c A T y (3-22) x = Z 1 r xz XZ 1 z. Equations 3-22 are the normal equations. The three systems of equations are equivalent, but have different characteristics. Usually, system (3-20) is not solved. The augmented system (Equations 3-21) is more suitable to solve. The normal equations (Equations 3-22) are attractive because of their compact form and also because the matrix AD 2 xza T is positive definite, allowing the use of nown direct and iterative solvers. If a direct solver is used, the term AD 2 xza T can be factorized by the Cholesy algorithm and then a bac substitution is applied to obtain y. However, D xz contains many zeros and also large numbers, which can lead to AD 2 xza T being highly ill-conditioned or singular

8 Chapter 3. Interior Point Method 35 at the final iterations. The Cholesy algorithm needs to be modified to address this problem. If the truss optimization problem of the previous chapter (see Equation 2-5) is solved using IPM, the following assignments can be made: ] A = B B b = f c = l σ Tl ] (3-23) where the term B B x = σ C s + s ], ] is replaced by AD 2 xza T, and the diagonal matrix D 2 xz is partitioned into four submatrices of equal size: ] ] ] AD 2 xza T D xz,1 0 B T = B B 0 D xz,2 B T. The product term in the right hand side can be reduced to: (3-24) M = BD ipm B T, (3-25) where D ipm = D 2 xz,1 + D 2 xz,2. This is the form that will be used in the solvers described in the next chapters. Note that B is sparse and constant, but D ipm varies in every IPM iteration. As the iterations proceed, this term has both small and large numbers and BD ipm B T may become ill-conditioned. This causes numerical problems for direct and iterative solvers, although iterative solvers are more affected. When iterative solvers, such as PCG, are used it is not necessary to explicitly assemble BD ipm B T because these methods rely on matrix vector products of the form BD ipm B T v, where v is a given vector. An important aspect of the implementation presented here is assembling B and BD ipm B T, which is the subject of the next chapter.