DEGENERACY AND THE FUNDAMENTAL THEOREM

Transcription

1 DEGENERACY AND THE FUNDAMENTAL THEOREM The Standard Simplex Method in Matrix Notation: we start with the standard form of the linear program in matrix notation: (SLP) m n we assume (SLP) is feasible, and A is of full row rank. (This implies, for example, that m n.) By solving a Phase I problem we can obtain a basis, B, that leads to a feasible canonical form. Specifically, by renumbering columns we can partition A into the m m form A = [ B N ] where B is non-singular and: with. We use the notation (the latter may be j interpreted as the column A written as a linear combination of the columns of B, with coefficients ). That is,. An immediate basic feasible solution is Finally, where and, (note: ). In summary our canonical form is: (CLP) Three Steps of the Simplex Method: 1. Column Choice: Choose a non-basic variable, x s, to become basic. We can select any s for which If there is no such column, we have an Degeneracy/Fundamental Theorem Revised 11/10/04 R. Van Slyke p. 1

2 optimal basic solution. (Data needed: ) There are various schemes for choosing from among the various negative values. The classic or Dantzig method, is the original scheme proposed by Dantzig, where that is, the most negative. Probably the most popular now is the steepest edge choice where Another popular approach, called partial pricing applies the classic method to only subset of the non-basic columns each iteration. The steepest edge method takes the most time per iteration, but usually decreases the number of iterations. Partial pricing decreases the time per iteration, but usually increases the number of iterations. We increase 2. Row Choice: Choose a basic variable, x r, to leave the basis (become nonbasic). (Data needed: ), while keeping all the other non-basic variables at 0. In the ith row we have (excluding the non-basics fixed at 0). If then x i hits 0 at The first basic variable to try to go negative is given by: (*) If there are ties we have degeneracy and we need a special tie breaking procedure to resolve the ties. We address this shortly. If all the the solution is unbounded. 3. Pivot: We replace x in the basis by x. (No additional data is needed). r In the next canonical form is a unit vector with a one in the rth row. The pivot matrix, P, that accomplishes this is: s Degeneracy/Fundamental Theorem Revised 11/10/04 R. Van Slyke p. 2

3 Note that it takes only n real numbers plus the index r, to represent a pivot matrix. Then: for all the non-basic variables, and takes the sth r place in the canonical form where U is the rth column unit vector. x is now in the basis. Similarly, If the aspect ration (n/m) is large, the updating of the non-basic variables can be computationally expensive. More importantly, if the original matrix, A, was sparse, the updated matrix, rapidly fills in. This motivates: The Revised Simplex Method: The general strategy is to keep the original data, A, b, c and only enough additional information to perform simplex iterations. Reviewing the above, we see that we need the following: for the column choice, and for the row test and pivoting. The original version of the revised method was to keep the inverse of the -1 basis, B, current, either explicitly or as a product of pivot matrices, or more likely, as a combination of both a relatively new basis inverse plus some recent s Degeneracy/Fundamental Theorem Revised 11/10/04 R. Van Slyke p. 3

4 pivot matrices. Modern, high performance solvers use other schemes (e.g., LU decomposition) but we will stick to the explicit inverse for simplicity. See Chapters 6 and 7 of Chvátal s book [Vašek Chvátal, Linear Programming, Freeman, 1983], for a good introduction to these more advanced schemes. -1 Suppose at the current iteration, we have the inverse B of the current basis, and the updated right hand side,, as well as the original data A, b, and c. We see how we can do the three steps of the simplex method using this data: 1. Column Choice: Calculate. 2 The first calculation takes order m work, and the second, order m (n-m). This provides the data for the classical column choice rule; moreover using partial pricing can reduce the cost (but, also, to some extent, the effectiveness) of 2). For very large aspect ratio, 2) dominates, but since N is often sparse, sparse matrix techniques can be used to gain efficiency. Steepest edge column selection gets more involved for the revised method; see, [Forrest, John and Donald Goldfarb, Steepest-edge simplex algorithms for linear programming. Mathematical Programming, vol. 57, pp , 1992]. 2. Row Choice: Calculate 2, which takes order m work. Choose the pivot row as before. 3. Pivot: Calculate the pivot matrix, P, as before; then 2 These take order m, and m work, respectively. Revised Simplex Method Takes better advantage of sparsity in problems Can effectively use partial pricing Standard Simplex Method Is more effective for dense problems Cannot effectively use partial pricing Degeneracy/Fundamental Theorem Revised 11/10/04 R. Van Slyke p. 4

5 Needs special, somewhat expensive, techniques to use steepest edge pricing. Is difficult to perform efficiently in parallel, especially, in loosely coupled systems. Frequently, the representation of the basis inverse is recomputed both for numerical stability and for efficiency (e.g., maintaining sparsity). The work is modest. Can easily use steepest edge pricing in addition to the classic choice rule. Very easy to convert to a distributed version with a loosely coupled system. Rarely, the dictionary has to be recomputed from the original data to maintain numerical stability (but not for efficiency). The work is substantial. Table 1: Comparison of Revised and Standard Forms of the Simplex Method Breaking Ties (Dealing with Degeneracy): Let D be any linearly independent set of m columns of A; e.g., the basis at the end of Phase I. Let D = [b D ], and pretend we replace by on the right hand side (we don t have to actually do this, because the columns of already available, where Let and be two vectors, each with m + 1 components, e.g., rows of D. Then is lexicographically less than if i = i for i = 0, 1,...,t-1, and t < t (or 0 < 0). Lexicographically greater is defined in a similar way. Then we replace the row test (* ) by (** ), where the arg min calculated using lexicographic ordering: (**). Now we can show (more accurately, you will show) that no canonical form repeats itself, and that, therefore, the simplex method is finite. Homework Problem: Complete the following proof: 1. Show that it can never be the case that a row of is all zeros (i.e., are Degeneracy/Fundamental Theorem Revised 11/10/04 R. Van Slyke p. 5

6 lexicographically 0) or that that no two rows of equal lexicographically). can be the same (i.e., 2. If D, originally, has all its rows lexicographically positive then will have all its rows lexicographically positive. 3. We update which we now interpret as a m+1 vector, by where D r is the row of D corresponding to the pivot row. Then, lexicographically. 4. Therefore, if the original D is lexicographically positive, we cannot repeat a canonical form using the simplex method with (**) for row choice, and the simplex method is finite. Generally speaking, you use the intial basis, either at the beginning of Phase I, or at the beginning of Phase II, as D. For the revised simplex method with explicit -1 inverse, the inverse B is, for the standard method, is made up of the columns in the current canonical form, corresponding to the columns of the original basis. Now that we have established the finite convergence of the simplex method, we can now state: Fundamental Theorem of Linear Programming: Every linear program has the following properties: (i) If it has no optimal solution, then it is either infeasible or unbounded. (ii) If it has a feasible solution, then it has a basic feasible solution. (iii) If it has an optimal solution, then it has a basic optimal solution. Using the revised form of the simplex method we can also prove a form of the: Strong Duality Theorem: If a linear program in standard form (SLP) has an optimal solution there exists a row vector, y*, such that: Degeneracy/Fundamental Theorem Revised 11/10/04 R. Van Slyke p. 6

7 where cx* is the value of the optimal solution obtained from the revised simplex method. y* is nothing more than at the end of the algorithm. We will put this theorem in a more general context in our discussion of duality. The role of degeneracy in theory and practice: The simplex method is important from two perspectives. First, it is an effective method for solving large, practical linear programs in practice. But the method also plays an important theoretical role. It offers constructive proofs of the fundamental theorem, and of duality theory. In contrast, the supporting and separating hyperplanes that we discussed in Notes can be used to develop duality in a non-constructive way (existence proofs). Degeneracy plays a different role in these two perspectives. Its theoretical role, is to show that the simplex method is a finite algorithm when the appropriate tie breaking procedure is used. The lexicographic method we have just sketched, can be used for this in either the standard, or revised method with explicit inverse basis. Lacking such a tie breaking feature, the simplex method can go into an infinite repeating cycle of basic solutions, and never terminate. In practice, cycling is rarely encountered, if for no other reason than roundoff error prevents a problem that would cycle with ideal arithmetic from cycling in practice. Moreover, the most efficient forms of the revised simplex method make implementation of lexicographic methods computationally expensive. In practice, the problem is stalling, where the value of the objective function doesn t change, or changes very little, for many iterations before it eventually starts to make substantive progress again. So in practice, somewhat heuristic and complicated schemes are used, primarily to prevent stalling, and only secondarily cycling. For example, in retrolp we used the EXPAND methodology [Gill, P.E., W. Murray, M.A. Saunders, M.H. Wright, A practical anti-cycling procedure for linearly constrained optimization, Mathematical Programming, vol. 45, no. 3, pp , 1989]. These schemes are also computationally expensive, so some just ignore the problem and hope for the best! Richard Van Slyke, 11/10/04 Degeneracy/Fundamental Theorem Revised 11/10/04 R. Van Slyke p. 7