1. Introduction. Consider the linear programming problem in the standard form [5, 14] minimize subject to Ax = b, x 0,

Transcription

1 EFFICIENT NESTED PRICING IN THE SIMPLEX ALGORITHM PING-QI PAN Abstract. We report a remarkable success of nested pricing rules over major pivot rules commonly used in practice, such as Dantzig s original rule as well as the steepest-edge rule and Devex rule. Key words. Dantzig s rule, steepest-edge rule, Devex rule, partial pricing, nested pricing. AMS subject classifications. 65K05, 90C05 1. Introduction. Consider the linear programming problem in the standard form [5, 14] minimize c T x subject to Ax = b, x 0, where A R m n (m < n) and rank(a) = m. It will be a simple matter to extend results of this paper to more general LP problems with bounds and ranges. We will use a i to denote the i-indexed column of A, and e i the unit (n m)-vector with its i-th component one, and the 2-norm of a vector. Let B be basis and N the associated nonbasis at current iteration of the simplex algorithm. Without confusion, denote basic and nonbasic index sets again by B and N, respectively. The nonbasic reduced costs may be obtained by a so-called pricing operation: Optimality is achieved if the index set c N = c N N T π, B T π = c B. J = {j c j < 0, j N} is empty. In the other case, any index in J can be taken to enter the basis to improve the current feasible basic solution. Goodness of entering indices is crucial to algorithm s efficiency, however, as it essentially determines the number of iterations required for solving LP problems. Recently, Pan, Li and Cao presented a scheme [23] that might be viewed as a variant of multiple pricing, but better be termed nested pricing. Using a nested version of Dantzig s criterion, it was implemented with a dense data structure within the deficient basis simplex algorithm [19, 20, 21]. The associated computational results were preliminary but encouraging. This paper considers more nested rules, and reports their computational performance against major commonly used rules on large-scale problems. Over 80 problems from Netlib, Kennington and BPMPD test sets, the nested Dantzig rule outperformed Devex rule with iterations ratio 3.48 and time ratio Over 77 problems of the test sets, it defeated the steepest-edge rule even with time ratio despite requiring more iterations (with iterations ratio 0.34). There are basically three types of commonly used pivot rules: full pricing rules, finite rules and partial pricing rules. In the remaining part of this section, we review some of them briefly. In Section 2, we describe pivot rules with nested pricing. In Section 3, we report computational results and make final remarks. Department of Mathematics, Southeast University, Nanjing, , People s Republic of China (panpq@seu.edu.cn). Project supported by National Natural Science Foundation of China. 1

2 2 Ping-Qi Pan 1.1. Full pricing. A rule involving all nonbasic reduced costs at each iteration is called full pricing rule, such as Dantzig s original rule [6, 8], the largest decrease rule [6] (see also [5, 13]), the steepest-edge rule [10] and the Devex rule [11]. These are standard commonly used rules in practice Dantzig s original rule. If J is nonempty, Dantzig s original rule selects an entering index q such that q = arg min { c j j J}. As a result, the objective value will decrease the most (by amount c q ) with the value of the associated variable increasing by a unit. Dantzig s rule had been used in practice for several decades since the simplex algorithm came out Largest decrease rule. If some index j J is chosen to enter the basis, a pivot row index, say i, has to be determined such that α j = min { b l /ā l,j ā l,j > 0, l = 1,..., m}, where b l is the l-th entry of B 1 b and ā l,j the l-th entry of B 1 a j. The resulting decrease in the objective value will be c j α j. The rule selects a nonbasic index q to enter the basis such that q = arg min { c j α j j J}, consequently achieving the largest possible decrease in the objective value after that iteration. This criterion was also mentioned in Dantzig s original paper. But since then this rule has been abandoned because of the overelaborate computation involved Steepest-edge rule. For simplicity, assume that basis B consists of the first m columns of the constraint matrix. Then the set of edge directions emanating from the current vertex can be written [ ] B d j = 1 N e I j m, j N. where I R (n m) (n m). It is noted that c j = c T d j. When J is nonempty, the steepest-edge rule chooses an index q to enter the basis such that c q / d q = min { c j / d j j N} < 0. Thus, edge direction d q is the most downhill with respect to the objective gradient c among the edges. If implemented explicitly, computation of the edge direction norms is too expensive to be practical. To expedite calculations, Goldfarb and Reid [10] derive recurrence formulas as follows. Assume that q replaces p in the basis. Introduce notation α q = π T p a q ; α j = π T p a j, j N, j q, where B T π p = e p. Then recurrence formulas of the edge directions are easily derived: from which it follows that d p = (1/α q )d q ; dj = d j (α j /α q )d q, j N, j q, d p 2 = (1/α 2 q) d q 2 = 1 + ā q 2, d j 2 = d j 2 2(α j /α q )a T j v + (α j /α q ) 2 d q 2, j N, j q.

3 Efficient Nested Pricing in the Simplex Algorithm 3 where Bā q = a q, B T v = ā q. Only the second equation of the preceding should be solved, since ā q is computed independent of updating d j 2 in the simplex algorithm. The steepest-edge rule involves more computational effort per iteration than Dantzig s rule. But, it usually requires much less iterations to solve large-scale LP problems [9]. Unfortunately, however, it has to compute d j 2 s associated with all nonbasic indices from scratch initially and periodically. The related costs are very high, especially if no advantage of computer architecture is taken to solve systems of equations with the same coefficient matrix. As a remedy, Devex rule eliminates such shortcoming to a large extent, as reviewed as follows Devex rule. Harris Devex Code [11] approximates the steepest-edge rule, in which the norms d j of the edge directions are replaced by weights w j. Initially and periodically, a so called reference framework is set to the current set of nonbasic indices, and w j are set to one for all j in the set. At other iterations, it uses w j to approximate the norms of the subvectors ˆd j consisting of only those components of the edge directions d j associated with the reference framework. The weights w j are updated by w p = max{1, ˆd q /α q }; w j = max{w j, (α j /α q ) ˆd q }, j N, j q. which results from using the larger of the norms of the two vectors ˆd j and (α j /α q ) ˆd q to approximate the norm of their sum. Note that ˆd q can be easily computed since d q is calculated at each iteration independent of the updating of weights. Accepted as the best for simplex algorithms, steepest-edge/devex rules are now commonly used in commercial packages, such as CPLEX [2, 12] Finite rules. The preceding rules are infinite in the sense that in the presence of degeneracy, the rule could yield zero-length steps, and hence even cycling. Although it turns out that cycling rarely occurs in practice, the solution process could be stuck at a degenerate vertex for too long a time before exiting it. Therefore, a number of finite rules were proposed in the past, such as Bland s rule and lexicographic rule [4, 7, 3, 24]; for a survey, see [25]). Although Pan s finite rule [18] seems to be more practical, such rules are not competitors of the standard rules, and therefore have never been used in practice Partial pricing. Important variants of the standard rules result from taking only a part of nonbasic costs into account Sectional pricing. MINOS [15] uses a sectional variant of Dantzig s rule as follows. With option p, all indices are partitioned to p roughly equal segments. The pricing operation begins on the segment that follows the one from which the previous entering index was selected. If a reduced cost is found that is less than some dynamic negative tolerance, the associated index is selected to enter the basis; otherwise, the same is repeated on the next segment, and so on. It turns out that such a strategy reduces not only the computational work per iteration but also the number of iterations required, in general (see, e.g., [11]) Multiple pricing. So-called multiple pricing rules are a special type of partial pricing ones (see, for example, [1, 5, 14, 16, 17]). Beginning with determining a small set of nonbasic indices with negative reduced costs, such rules selects from this set by some criterion an entering index but retain the other indices in the set as candidates for the next iteration. They continue in this way until the reduced costs associated with all the indices in the set are nonnegative. Then, it identifies a new set of indices with negative reduced costs, and repeats the process. 2. The nested pricing. At each iteration, one is faced with a subset J of N, indices in which is given priority to become basic. Pricing is conducted on J to determine a reduced cost by some criterion. If one is found significant negative, the associated index is selected to

4 4 Ping-Qi Pan enter B. If it is not, the same is done with the remaining set to determine an entering index; if no such one is found, then optimality is declared. In particular, we describe the nested version of Dantzig s original rule. Rule 1 (Nested-Dantzig). Let ε > 0 be optimality tolerance. Initially, set J = N. (i) If index set Ĵ = {j c j < ε, j J} is nonempty, select an entering index q such that or else (ii) if redefined index set q = arg min { c j j Ĵ}; (2.1) Ĵ = {j c j < ε, j N\J} is nonempty, select q by (2.1); or else (iii) declare optimality. If optimality is not declared, set J = Ĵ\q for the next iteration. A basic idea of the preceding rule can be interpreted as follows. It is evident that the rule becomes Dantzig s original rule at the first iteration and at iterations where N\J is touched. Such a full pricing iteration is followed by a series of nested pricing iterations, together might be termed a circle. Since each J is a proper subset of its predecessor, computational work for pricing is reduced iteration by iteration in a circle. It is more than that. At the k-th iteration of a circle, in fact, the indices in J are all those for which the associated reduced costs have remained significantly negative throughout the forgoing k iterations. It is reasonable to focus on such indices. It is clear that any full pricing rule can be recast to its nested version. In particular, nested versions of the Devex and the steepest-edge rules can be formulated as follows. Rule 2 (Nested-Steepest-Edge). This rule is the same as Rule 1 except for c j replaced by c j / d j, where d j is the edge vectors, defined in Section Rule 3 (Nested-Devex). This rule is the same as Rule 1 except for c j replaced by c j /w j, where w j is the weights, defined in Section Complexity. It is difficult to make a comparison precisely between the numbers of arithmetic operations involved in each iteration of the preceding three nested pricing rules or to distinguish between them and their standard counterparts with this respect, since this depends on J 1 that is of uncertainty in nature. However, a qualitative discussion is easy and still useful. First of all, it is clear that Nested-Dantzig rule is the simplest among the nested pricing rules, and should also be considerably cheaper than Dantzig s rule, since the former need not calculate the reduced costs associated with J 2 at all while the latter calculates all nonbasic reduced costs in each iteration. Unfortunately, in contrast, one could not gain as much from Nested-Steepest-Edge rule. Like its standard counterpart, it has to compute d j 2 associated with all nonbasic indices from scratch, initially and periodically. This work is very expensive (see Section 1.1.3). In each iteration, moreover, such quantities should be updated to be kept available, because, otherwise, they are calculated from scratch whenever J 2 is touched, consequently resulting in a huge amount of computation. Approximating Nested-Steepest-Edge rule, Nested-Devex rule is favorable in this respect. It has no such degrading computations involved it need not update w j associated with J 2 because its initialization of these weights is cheap.

5 Efficient Nested Pricing in the Simplex Algorithm 5 Table 3.1 A ratio Summary to N-Dantzig Problem Dantzig Devex P-Dantzig N-Devex Iters Time Iters Time Iters Time Iters Time Netlib(47) Kennington(16) BPMPD(17) Average(80) Certainly, the number of iterations required by nested pricing is crucial to its efficiency. At this stage, we are even not able to rule out the possibility of cycling. It is likely that a nested pricing rule requires infinitely many iterations to solve some LP problem. It is expected however that cycling rarely occurs in practice with these nested pricing rules, like with their standard counterparts. To a large extent at least, complexity evaluation of nested pricing is a computational issue, closely related to its performance in solving large-scale LP problems. Somewhat surprisingly, it turned out that nested pricing usually requires less iterations than full pricing, perhaps except for the steepest-edge rule. Despite this rule requires less iterations than other rules, however, it consumes much more total run time than Nested- Dantzig and Neste-Devex rules, as reported in the next Section. 3. Computational experiments. Computational experiments with the nested pricing were carried out extensively. For the sake of limited space, we only outline numerical results obtained and make final remarks in this Section. For more details, the reader is referred to [22]. To have comparisons easy and fair, in our tests each rule is implemented within MINOS 5.51, the latest version of MINOS. Actually, all resulting codes are the same as MINOS 5.51, except for its pivot rule replaced by relevant ones. Our first test set of problems consists of the 47 largest Netlib problems, the second includes all of the 16 Kennington problems, and the third the 17 largest BPMPD problems. We first outline results associated with the following five codes: 1) Dantzig: MINOS 5.51 with the full pricing option. 2) Devex: based on Devex rule (Section 1.1.4). 3) P-Dantzig: MINOS 5.51 with the default partial pricing option. 4) N-Devex: based on the Nested-Devex Rule 3. 5) N-Dantzig: based on Nested-Dantzig Rule 1. Table 3.1 offers iterations and time ratios of the first four codes to N-Dantzig for each test set and for all. Overall, the codes performed consistently with the three test sets. N-Dantzig and N- Devex defeated the standard codes in terms of both total iterations and run time. N-Dantzig even outperformed Devex rule with total iterations ratio 3.48 and average time ratio It can be asserted that N-Dantzig performed best while code Dantzig did worst with iterations ratio 6.78 and time ratio As for a comparison against other rules, N-Dantizig outperformed Steepest-Edge with average time ratio as high as despite requiring more iterations over 77 problems of the test sets (with iterations ratio 0.34) [22]. Nested-Steepest-Edge was inferior to Steepest- Edge. These results are consistent with discussions made previously in Section 2.1. The largest decrease rule is out of the line of competitors. In summary, both the nested Dantzig rule and nested Devex rule unambiguously outperformed major commonly used rules, including the steepest-edge rule and Devex rule. As the nested Dantzig rule is superior to the nested Devex rule both in efficiency and in simplicity, we strongly recommend the former to be used in simplex codes.

6 6 Ping-Qi Pan Acknowledgment. The author would like thank Professor Michael A. Saunders for his very useful comments and for kindly providing us the MINOS 5.51 package. The author is also grateful to an anonymous referee for his/her valuable comments, which pointed out that any full pricing rule can be recast to its nested version, and suggested comparing the nested pricing rules with the steepest-edge rule and Devex rule. This work would have not been possible without their assistance. REFERENCES [1] M. J. Benichou, J. Cautier, G. Hentges and G. Ribiere, The efficient solution of large scale linear programming problems, Mathematical Programming, 13 (1977), [2] R. E. Bixby, Solving real-world linear programs: A decade and more of progress, Operations Research, 50 (2002) No. 1, [3] R. G. Bland, New finite pivoting rules for the simplex method, Mathematics of Operations Research, 2 (1977), [4] A. Charnes, Optimality and degeneracy in linear programming, Econometrica, 20 (1952), [5] V. Chvatal, Linear Programming, W. H. Freeman and Company, New York, [6] G. B. Dantzig, Maximization of a linear function of variables subject to linear inequalities, in: Activity Analysis of Production and Allocation, T.C. Koopmans, ed., Cowles Commission Monograph 13, Wiley, New York, [7] G.B. Dantzig, A. Orden and P. Wolfe, The generalized simplex method for minimizing a linear form under linear inequality restraints, Pacific Journal of Mathematics, 5 (1955), [8] G. B. Dantzig, Linear Programming and Extensions, Princeton University Press, Princeton, NJ, [9] J. J. H. Forrest and D. Goldfarb, Steepest-edge simplex algorithms for linear programming, Mathematical Programming, 57 (1992), [10] D. Goldfarb and J. Reid, A practicable steepest edge simplex algorithm, Mathematical Programming, 12 (1977), [11] P. M. J. Harris, Pivot selection methods of the Devex LP code, Mathematical Programming, 5 (1973), [12] ILOG CPLEX: High Performance Software of Mathematical Programming, /products/cplex/. [13] H.W. Kuhn and R.E. Quandt, An experimental study of the simplex method, in Eperimental Arithmetic, High-Speed Computing and Mathematics (Proceedings of Symposia in Applied Mathematics XV; N.C. Metropolis, et al., eds.), American Mathematical Society, Providence, R.I., 1963, [14] I. Maros, Computational Techniques of the Simplex Method, International Series in Operations Research and Management, Vol. 61, Kluwer Academic Publishers, Boston, [15] B. A. Murtagh and M. A. Saunders, MINOS 5.5 User s Guide, Technical Report SOL 83-20R, Dept. of Operations Research, Stanford University, Stanford, [16] J. Nocedal and S.J. Wright, Numerical Optimization, Springer Science+ Business Media, Inc., Berlin, [17] W. Orchard-Hays, Advanced Linear Programming Computing Techniques, McGraw-Hill Book Company, New Yord, [18] P.-Q. Pan, Practical finite pivoting rules for the simplex method, OR Spektrum, 12 (1990), [19] P.-Q. Pan, A basis-deficiency-allowing variation of the simplex method, Computers and Mathematics with Applications, 36 (1998b) No. 3, [20] P.-Q. Pan, A projective simplex method for linear programming, Linear Algebra and Its Applications, 292 (1999), [21] P.-Q. Pan, A projective simplex algorithm using LU factorization, Computers and Mathematics with Applications, 39 (2000), [22] P.-Q. Pan, Computational results with Nested Pricing in the Simplex Algorithm, FILE/2007/03/1602.pdf [23] P.-Q. Pan, W. Li and J. Cao, Partial pricing rule simplex method with deficient basis, Numerical Mathematics, A Journal of Chinese Universities (English Series), 15(2006) No. 1, [24] T. Terlaky, A convergent criss-cross method, Math. Oper. und Stat. ser. Optimization, 16 (1985) No. 5, [25] T. Terlaky and S. Zhang, Pivot rules for linear programming: A survey on recent theoretical developments, Annals of Operations Research,46 (1993),