The Affine Scaling Method

Transcription

1 MA33 Linear Programming W. J. Martin October 9, 8 The Affine Scaling Method Overview Given a linear programming problem in equality form with full rank constraint matrix and a strictly positive feasible solution x, we transform the problem to a new one with feasible solution = [,,,. The original problem and scaled problem Original: Scaled: max c x max ĉ x s. t. Ax = b s. t. Âx = b x x are related as follows: X is the diagonal matrix with the entries of x down the diagonal; Â = AX ; and ĉ = X c. In the scaled problem, we project the gradient ĉ onto the subspace Âx = to obtain a search direction d; find the largest scalar β such that + βd is still non-negative (the other feasibility conditions are already guaranteed by choice of d); obtain new (scaled) solution ˆx = + rβd where < r < is a global constant; update our solution x to the next solution x = X ˆx. This process is repeated, as needed, until we are sufficiently close to an optimal solution (e.g., as one may check by solving for a dual vector using complementary slackness conditions).

2 One iteration of the affine scaling method We assume we have an m n matrix A with full row rank, an m-vector b and an n-vector c. (If A does not have full row rank, first row reduce, and if Ax = b is feasible, then replace A by a smaller matrix.) We also have a fixed step size r, < r <. Given a strictly positive feasible solution x to the problem (so Ax = b), we obtain a better strictly positive feasible solution x as follows: Step : (Scale) Let  = AX and ĉ = X c where X is the diagonal matrix with entries from x down the diagonal. (Now we have a scaled problem with as a feasible solution.) Step : (Find projection) Compute the projection matrix onto the nullspace of Â: P = I  ( )  Step 3: (Find search direction) The search direction (in the scaled problem) is then d = P ĉ. { } Step 4: (Compute ratios) Let β = min d j : d j <. (If this set is empty, the problem is unbounded.) Step 5: (Scaled update) The improved solution in the scaled problem is ˆx = + rβd where r is the prescribed step size ( < r < ). Step 6: (Next iterate) Now scale back to the original problem and take x = X ˆx. Note that the above algorithm brings us from x to x. To obtain further improved solutions, we iterate this with x replaced by the current solution. The general step is described with only a slight change of notation and is given at the end of these notes, just before the exercises.

3 Some explanations In Step, we simply write down the data (Â, ĉ) for the scaled problem. right-hand side vector does not change. But that s okay because Note that the  = (AX ) = A(X ) = Ax = b so is indeed a feasible solution to the scaled problem and it is far away from the boundaries x j =. Moreover, X is a diagonal matrix, so it is symmetric: (X ) = X. Thus ĉ = (X c) = (c (X ) ) = c (X ) = c x. More generally, the objective value of any solution ˆx to the scaled problem is equal to the objective value of the corresponding solution x = X ˆx in the original problem. In Step, we do the most work. We need to project the gradient ĉ onto the subspace {x : Âx = } in order to have a feasible search direction. As we worked out in class, the projection matrix P does just the trick. In Step 3, we take d = P ĉ so that Âd = and, for any scalar β, the vector ˆx = + βd satisfies ˆx = Â( + βd) =  + βâd =  = b as established above. So, no matter how far we travel in this direction, the equality constraints will still be satisfied. From this, we know in Step 4 that any vector of the form ˆx = +βd satisfies the equality constraints. So in order to stay feasible, we simply need +βd. This requires +βd j for all j. Since we are going to take β positive (to improve the objective value), the only coordinates j that concern us are those with d j <, and each of these gives us an upper limit β /d j. In Step 5, we finally use our global step size r. We know that +βd and is otherwise feasible. But we need a strictly positive feasible solution. So we only move a fraction of the maximum possible distance. For example, with r =.9, we only go 9% of the way to the wall. Thus our update is + r(βd). The transformed problem has now served its purpose of pushing the search direction away from the boundaries. We discard it and transform the solution ˆx back to the original LP to get the next iterate x. As explained in Step, this is simply achieved through scaling by X. 3

4 Example : Consider the problem max 3 x + x s.t. 5 x + x + x 3 = x + x + x 4 = x, x, x 3, x 4 with initial feasible solution x = (,,, ). So our current objective value is ζ = We have A = [ 5 Step : Scale the problem: /5 X = /5 /5 Step : Compute P : [, b =,  = AX =  = [ 5, c = 3, x = [ 4/5 /5 /5 4/5 ) P = I (  Â. [, ( ) = 5 63 We eliminate fractions where possible to write 5 ) [  ( 5 = so that  ( )  = Let s call this last matrix Q. Then we have P I Q = 63 [ = 5 63 = /5 /5 /5., ĉ = X c = /5 /5.

5 Step 3: Find search direction d: d = P ĉ = which simplifies to d = ( 3,, 7, ). 63 3/5 /5 = Step 4: Ratios: So β = 9. { } { 63 β = min : d j < = min d j 7, 63 } = 9. Step 5: Scaled update (using step size r = /): 4 ˆx = + rβd = using rβ 63 = 9 63 = 4. Step 6: Next iterate: Now we go back to the original problem using X : /5 x = X ˆx = /5 / /4 /4 7/4 5/4 = 4 = 7 with objective value ζ = 5/4. As you can see, the numbers will get worse after this, so we use a computer if we want to go further Example : Consider the problem max x x x + x = 4 x, x with initial feasible solution x = (, ) and step size r = /. We have A = [, b = [ 4 [, c =, x = [. 5

6 Step : Scale the problem: X = [ Step : Compute P = I Â (ÂÂ ) Â: ÂÂ = [ [, Â = AX = [ [, ĉ = X c = = [ 8, P = [ P = Step 3: Find search direction d: d = P ĉ = [ [ [ [ / / / / = [. [ /8 [ = [ 3 3 [ = 3/ 3/ [ Step 4: Ratios: { } β = min : d j < = d j 3. Step 5: Scaled update: Step 6: Next iterate: ˆx = + rβd = [ x = X ˆx = [ + 3 [ [ 3/ / 3 3 = = [ 3/ [ 3/ / In this iteration, our objective value has improved from to +/. Pictorial Example The advantage of interior point methods is that they tend to avoid the combinatorial complexity of basic feasible solutions. One can imagine a high-dimensional polyhedron having large clusters of nearby vertices all of which are suboptimal. The simplex method can only move from one vertex to a neighbor, possibly contributing to inefficiency.. 6

7 To illustrate how an interior point approach can smooth out all these corners, let s look at a simple two-dimensional example with three nearby basic feasible solutions far away from optimality: maximize x subject to x x x x 9 x x 8 x 8 x, x We will skip the conversion to equality form. After introducing slack variables x 3,..., x 6, the initial strictly feasible solution (4, ) gives us x = (4,, 5,,, 5 ). The objective direction c is horizontal, pointing directly to the right. But the optimal solution is at (4, 8). Here we give the first eight iterates x,..., x 8 of the affine scaling method with two choices of step size, r = / and r = 3/4. top red curve r = bottom green curve r = 3 4 x c 7

8 Initially, we showed how to move from initial solution x to a better solution x. generic iteration looks like this: A Step : (Scale) Let  = AXk and ĉ = X k c where X k is the diagonal matrix with entries from the most recent solution x k down the diagonal. (Now we have a scaled problem with as a feasible solution.) Step : (Find projection) Compute the projection matrix onto the nullspace of Â: P = I  ( )  Step 3: (Find search direction) The search direction (in the scaled problem) is then d = P ĉ. { } Step 4: (Compute ratios) Let β = min d j : d j <. (If this set is empty, the problem is unbounded.) Step 5: (Scaled update) The improved solution in the scaled problem is ˆx = + rβd where r is the prescribed step size ( < r < ). Step 6: (Next iterate) Now scale back to the original problem to obtain the next iterate x k+ = X kˆx. Exercises.) Consider the linear programming problem max x x subject to x + x, x, x. (i) Starting with x = (/, /4), apply two iterations of the affine scaling method using r = /. (First, convert to equality form.) For each iteration, give the constraint matrix  and objective vector ĉ for the scaled problem; the projection matrix P and search direction d for this iteration; the ratio computation from Step 4; 8

9 the next iterate, both in scaled form ˆx and as a solution x k+ to the original problem above. (So you will be finding x and x.) (ii) On a sheet of graph paper, make a careful (and large!) drawing of the feasible region in R. For x and each of the next two iterates, x, and x, plot both the gradient of the objective function (namely c = [ ) at that point as well as the scaled step direction x k+ x k..) Repeat the steps of Exercise (again with r = /) for the linear programming problem max x + x subject to x, x. starting with x = (3/4, /4) (a column vector). (You may wish to use MAPLE to check your computations and to follow the trajectory further.) 3.) Perform two iterations of the affine scaling method for the problem maximize x + x + x 3 subject to x x = x + x + x 3 = 4 x, x, x 3 with step size r = / and initial solution x = (,, ). 4.) Perform two iterations of the affine scaling method for the problem max x subject to x + x = 7, x, x with step size r = 3 4 and initial solution x = (3, 4). 5.) Perform two iterations of the affine scaling method for the problem max x x + 3x 3 subject to x + x + x 3 = 5 (all variables nonnegative) with step size r = and initial solution x = (,, ). 6.) Perform two iterations of the affine scaling method for the problem max c x s.t. Ax = b, x with step size r = / and initial solution x = (5, 3, ) where [ [ 5 A =, b =, c = (,, ). 7 7.) Perform two iterations of the affine scaling method for the problem maximize x x 3 subject to x + 4x 4x 3 = x + x + x 3 = 8 x, x, x 3 with step size r = / and initial solution x = (,, ). 9

10 8.) Perform two iterations of the affine scaling method for the problem max x + x s.t. x + 5x = 3 x, x with step size r =.75 and initial solution x = (,.). 9.) Perform two iterations of the affine scaling method for the problem maximize x x 3 subject to x x = x + x 3 = x, x, x 3 with step size r = / and initial solution x = (,, )..) Perform two iterations of the affine scaling method for the problem min x x s.t. 3x + 5x = 3, x, x with step size r = / and initial solution x = (, )..) Perform two iterations of the affine scaling method for the problem maximize x + x + x 3 subject to x x = x + x + x 3 = 4 x, x, x 3 with step size r = / and initial solution x = (,, )..) More generically, consider the n-dimensional optimization problem with one equality constraint n j= x j =, objective function max c x, step size r = /, and initial solution x with positive entries summing to one. For convenience, also assume that n l= c l(x l ) = x and that the variables are ordered in such a way that c c c n. Compare the next iterate x produced by the affine scaling method against the corresponding unscaled update x + β c where c is the projection of the objective gradient c onto the affine subspace containing the feasible region and β is computed so as to prevent any entry of x + β c from going negative.