Combining Lift-and-Project and Reduce-and-Split

Transcription

1 Combining Lift-and-Project and Reduce-and-Split Egon Balas Tepper School of Business, Carnegie Mellon University, PA Gérard Cornuéjols Tepper School of Business, Carnegie Mellon University, PA Tamás Kis Computer and Automation Research Institute, Hungarian Academy of Sciences, Hungary Giacomo Nannicini Tepper School of Business, Carnegie Mellon University, PA November 7, 2010 Abstract Split cuts constitute a class of cutting planes that has been successfully employed by the majority of Branch-and-Cut solvers for Mixed Integer Linear Programs. Given a basis of the LP relaxation and a split disjunction, the corresponding split cut can be computed with a closed form expression. In this paper, we use the Lift-and-Project framework [10] to provide the basis, and the Reduce-and- Split algorithm [17] to compute the split disjunction. We propose a cut generation algorithm that starts from a Gomory Mixed Integer cut and alternates between Lift-and-Project and Reduce-and-Split in order to strengthen it. This paper has two main contributions. First, we extend the Balas and Perregaard procedure for strengthening cuts arising from split disjunctions involving one variable, to split disjunctions on multiple variables. Second, we apply the Reduce-and-Split algorithm to non-optimal bases of the LP relaxation. We provide detailed computational testing of the proposed methods. Keywords: Integer Programming, Computational Analysis, Branch-and-Cut, Liftand-Project. 1 Introduction Mixed Integer Linear Programs (MILPs), i.e. mathematical programs with linear objective and constraints and both continuous and integer variables, arise in a number of real-world applications, and their solution is therefore of great practical interest. The most successful softwares for solving general MILPs utilize a Branch-and-Cut algorithm, which combines cutting planes and Branch-and-Bound. Several classes of cutting planes 1

2 1 INTRODUCTION 2 used by these softwares, such as Gomory Mixed Integer (GMI) cuts [18], Mixed Integer Rounding (MIR) cuts [21] and Lift-and-Project cuts [8], fall into the category of split cuts [16], that is, disjunctive cuts derived from two parallel hyperplanes. It was shown in [4] that every split cut can be generated as an intersection cut [5] from an appropriate choice of a basis of the LP relaxation and a split disjunction. The advantage of generating split cuts as intersection cuts is that we can use closed form expressions, without having to resort to disjunctive programming [6]. In this paper, we propose a split cut generation procedure that is based on Lift-and-Project [8, 10] and Reduce-and-Split [3, 17]. In particular, we use the former to select a basis of the LP relaxation, and the latter to compute a split disjunction. Lift-and-Project(L&P) cuts have been successfully used in the Branch-and-Cut framework since the 90s [8]. A significant improvement in their practical performance came a few years later, when a procedure to generate L&P cuts without solving the higherdimensional Cut Generating Linear Program (CGLP) was introduced by Balas and Perregaard [10]. This procedure starts with a split cut arising from a violated two-term disjunction involving a single variable and the optimal basis of the LP relaxation (in other words, a GMI cut), and mimicks the solution of the CGLP by performing pivots in the original simplex tableau. The procedure yields a new (possibly infeasible) basis, from which a stronger cut than the initial GMI cut can be generated. This procedure has been incorporated into commercial solvers like Xpress-MP [22], MOPS [23], and several versions of it have been implemented in the open source project COIN-OR Cgl [14]. One of the main contributions of this paper consists in an extension of this procedure to split cuts arising from general split disjunctions, i.e. any violated two-term disjunction involving an integral linear combination of integer variables. This yields a procedure that, given any split disjunction and any basis, produces a different basis that gives rise to a stronger cut. In order to apply this extended L&P procedure, we need a method for generating an initial split disjunction. We use the Reduce-and-Split (RS) algorithm for this purpose. RS, first introduced in [3] and then revisited in [17], is a cut generation algorithm that starts from an optimal LP basis and a split disjunction on one variable (i.e. a GMI cut), and computes a split disjunction involving several variables that (heuristically) yields a better cut. Therefore, we have an algorithm to produce split disjunctions, which can be used to initialize the L&P procedure. Another contribution of this paper is that we apply RS on non-optimal, possibly infeasible, tableaux. As a consequence, we have a procedure that, given any split disjunction and any basis, produces a new, often better split disjunction for cut generation. Thus, we can alternate between the two procedures introduced in this paper, and iteratively change both the basis and the split disjunction from which a split cut is generated. We perform extensive computational experiments on a set of benchmark MILPs to assess the effectiveness of our ideas. Our computational results show that, within a Cut-and-Branch framework, the combination of the two cut generation algorithms yields stronger cutting planes than L&P or RS alone. We obtain the best results by alternating between the two more than once. The rest of this paper is organized as follows. In Section 2 we introduce our notation and provide the necessary theoretical background. In Section 3 we review in more detail the Lift-and-Project procedure introduced in [10], and extend it to general split disjunctions. Section 4 reviews the Reduce-and-Split method, and discusses its application on non-optimal bases of the LP relaxation. In Section 5 we describe our cut generation algorithm, which alternates between the Lift-and-Project and the Reduce-and-Split procedures. Section 6 presents an extensive computational evaluation. Section 7 concludes

3 2 NOTATION AND PRELIMINARIES 3 the paper. Detailed tables of results can be found in Appendix A. Appendix B contains a brief computational evaluation on pure integer instances that are not part of our main set of test instances. 2 Notation and preliminaries We are considering a MILP of the form: min c x Ax b x 0 j N I x j Z, (MILP) where A R m n, b R m, c R n, N := {1,...,n}, N I := {1,...,p} with p n, and where upper bounding constraints are subsumed by Ax b. In the sequel, Ãx b will denote Ax b, x 0, and (LP) will stand for the linear programming relaxation of (MILP). A split cut for (MILP) is a valid inequality derived from a disjunction of the form πx π 0 πx π 0 +1, (1) where π j is integer for j N I, π j = 0 for j N \N I, and π 0 is an integer whose value depends on the fractional point we want to cut off. For a given fractional point x, π 0 is chosen so as to have π 0 < π x < π 0 +1, (2) which yields π 0 = π x. If x is a basic solution to (LP) such that x k is fractional, then the disjunction x k x k x k x k +1, (3) which is a special case of (1), gives rise to a GMI cut. Alternatively, a GMI cut from (3) can be derived as follows. Rewrite (LP) in standard form: min c x (A, I)x = b x 0 (LP) s where x R n+m and the last m components are surplus variables. Let B( x) be the set of indices of basic variables in the basis associated with x, and J( x) = N \B( x) the set of nonbasic variables. Then, the corresponding simplex tableau can be written as: x i = x i ā ij x j i B( x). (4) j J( x) Let B I ( x) = B( x) N I, J I ( x) = J( x) N I, J C ( x) = J( x) \ N I be the sets of integer basic variables, integer nonbasic variables and continuous nonbasic variables, respectively. Consider a linear combination with integer coefficients π i of those rows of (4) where i B I ( x): π i x i = ˆx â j x j, (5) i B I( x) j J( x)

4 3 LIFT-AND-PROJECT ON GENERAL DISJUNCTIONS 4 where ˆx = i B π I( x) i x i = i B π I( x) iā ij for j J( x). â j Let π 0 = ˆx, and define f 0 = ˆx π 0, f j = â j â j for all j J. If ˆx Z, we can derive from (5) the following valid inequality for (MILP): f j 1 f j x j + x j + f 0 1 f 0 j J I( x):f j f 0 j J I( x):f j>f 0 j J C( x):â j 0 â j f 0 x j j J C( x):â j<0 â j 1 f 0 x j 1. This inequality is the GMI cut associated with the equation obtained through the row multipliers π i ; its validity is shown in [18]. Choosing π k = 1, π i = 0 i k yields the GMI cut from (3); note that in this case, (5) coincides with a single row of (4). 3 Lift-and-Project on general disjunctions In order to apply the Lift-and-Project procedure to a disjunction of the form (1), one could simply formulate the (CGLP) corresponding to ( Ãx b πx π 0 ) ( Ãx b πx π 0 +1 (6) (7) ). (8) However, in order to take full advantage of the correspondence between the (CGLP) and the (LP) established in [10], it will be preferable to introduce a new integer variable x n+m+1 to represent the difference between πx and π 0 : πx x n+m+1 = π 0 (9) Notice that, since πx is integer in any feasible solution of (MILP) and π 0 is integer, x n+m+1 has to be integer as well. Moreover, if x n+m+1 0, then πx π 0, while if x n+m+1 1, then πx π 0 +1, as desired. Using the new variable, we can rewrite (8): Ãx b πx x n+m+1 = π 0 x n+m+1 0 Ãx b πx x n+m+1 = π 0 x n+m+1 1. (10) The important difference from the previous applications of the Lift-and-Project procedure to single rows (4) of the simplex tableau is the following. The equation (9) is constructed in order to derive a cut from it. Once the cut is derived, the equation is no longer needed and therefore it is discarded, along with the variable x n+m+1. On the other hand, the variable x n+m+1, and its expression in terms of the current nonbasic variables, is needed throughout the pivoting process carried out in order to (implicitly) optimize the CGLP. Thus, we have to add a new row to the optimal (LP) tableau and keep it until the cut is optimized. This could be done by simply adding the equation πx x n+m+1 = π 0 to the constraint set of (MILP), and then computing the amended simplex tableau corresponding to the current basis. Instead, one can derive the new row as a closed form expression.

5 3 LIFT-AND-PROJECT ON GENERAL DISJUNCTIONS Proposition Let (A B,A J ) be the partition of (A, I) into basic and nonbasic columns. Then the expression for x n+m+1 = πx π 0 in terms of the nonbasic variables is x n+m+1 +(π B A 1 B A J π J )x J = (π B A 1 B )b π 0 (11) as Proof. The simplex tableau corresponding to the basis indexed by B is x B +A 1 B A Jx J = A 1 B b. If π = (π B,π J ) and π j = 0 for all j N \N I, then πx x n+m+1 = π 0 can be written x n+m+1 +π B x B +π J x J = π 0. Appending this equation to (A B,A J )x = b gives A B x B + A J x J = b π B x B x n+m+1 + π J x J = π 0. ( ) ( AB 0 A 1 The inverse of the (m+1) (m+1) matrix is π B 1 π B A 1 B Multiplying (12) with this augmented basis inverse gives B 0 1 ). (12) x B + (A 1 B A J)x J x n+m+1 + (π B A 1 B A J π J )x J = A 1 B b = π B A 1 B b π 0 The new source row (11) could of course be used to directly generate a generalized GMIcut; instead, weapplytoitthel&pprocedureof[10], oroneofitsvariantsdiscussed in [7] in order to obtain a stronger cut. Such a cut will be valid throughout the search tree in case of a mixed 0-1 program, but only at the descendants of the current search tree node for a general mixed integer program (see [9]). The L&P procedure is illustrated in Fig. 1, where the basic solution x is to be cut off, but the corresponding cut αx α 0 is weaker than the one that can be derived after pivoting to x and deriving the cut α x α 0 from that (infeasible) basic solution. 3.1 Implementation of the generalized Lift-and-project procedure It is not too difficult to modify any implementation of the L&P cut generation procedure that works on the simplex tableau and strengthens cuts derived from a disjunction (3), so that it can strengthen split cuts derived from a more general split disjunction(1). Namely, the L&P procedure must have a subroutine to extract the source row from the simplex tableau before pivoting and after each pivot. This is usually done by using the basis inverse, which is typically readily available: most Branch-and-Cut (or Cut-and-Branch) solvers use the revised dual simplex method, which maintains the basis inverse rather than the full simplex tableau. It suffices to modify this subroutine so that it computes the source row using (11). This can be implemented rather efficiently using the standard

6 4 REDUCE-AND-SPLIT FROM NON-OPTIMAL BASES 6 x x x k x k x k x k +1 αx = α 0 α x = α 0 r 1 r 3 r 2 Figure 1: Illustration of the Lift-and-Project procedure. Ftran or Btran subroutines, available in many commercial and free state-of-the-art LP solvers. 4 Reduce-and-Split from non-optimal bases The idea of looking for a linear combination (5) of rows of the simplex tableau (4) to generate strong cutting planes is not new in the integer programming literature: see e.g. [12, 3, 17]. As discussed in Section 2, every equation (5) such that i B I( x) π i x is fractional yields a valid GMI cut. Here, x need not be an optimal solution to (LP); however, this is the only case that is typically studied in the literature. In this section we consider the case where x is basic but not optimal for (LP). In particular, our discussion focuses on the case where x is a basic solution for a L&P tableau, i.e. a (possibly primal infeasible) tableau obtained by pivoting following the L&P procedure. We start with a brief description of the RS algorithm, as given in [17]. Let x be the optimal solution to (LP). RS first determines a working set of continuous nonbasic columns J W J C ( x), then generates an integral combination (5) of the rows of the simplex tableau corresponding to the basic variables in B I ( x) by minimizing: min (â j ) j JW, (13) π Z B I ( x) where â j is defined as in (6). This minimization problem yields row multipliers π i from which we derive (7). As can be seen from (7), small â j on continuous nonbasic columns should yield good (i.e. small) cut coefficients on the corresponding variables. Note that our aim is to improve the cut coefficients on continuous variables only. (13) is solved by relaxing integrality on π, determining the optimal continuous multipliers (imposing an additional normalization constraint), then rounding the fractional components π to the nearest integer. In [17] it is experimentally shown that variables with small reduced cost are good candidates for the set J W, as they yield cuts which close a larger integrality gap in practice. This is the basic variant of the Reduce-and-Split algorithm: we refer to [17]

7 4 REDUCE-AND-SPLIT FROM NON-OPTIMAL BASES 7 x k x k +1 x x k x k πx π 0 αx = α 0 πx π 0 +1 α x = α 0 r1 r 2 Figure 2: Illustration of the Reduce-and-Split procedure. for a thorough discussion. The geometric interpretation is as follows. RS keeps the basis B( x) fixed, and tries to modify the split disjunction (1) in order to obtain a cut with stronger coefficients. This is exemplified in Figure 2: the elementary disjunction x k x k x k x k +1, which yields the cut αx α 0, is modified to obtain a stronger cut α x α 0 (from disjunction πx π 0 πx π 0 + 1). In Section 3 we proposed a method to start with any split disjunction, and modify the basis via L&P to obtain a stronger cut. What we want to do now is to use the basis computed by L&P, and modify ( tilt ) the split disjunction to derive a better cut. A problem arises: a cut derived from a non-optimal basis of (LP) will certainly be valid, but how do we make sure that it will be violated by the point that we want to cut off? To show why such a cut might not be violated, we need to introduce some notation. Let x be the optimal solution to (LP), where the corresponding optimal tableau Ā has elements ā ij. Let x be the basic solution associated with the tableau Ā (with elements ā ij ) obtained by applying L&P starting from x. A GMI obtained from tableau Ā has the form α j x j α 0, j J( x) with α j 0,α 0 > 0. Since x j = 0 j J( x), this cut is violated by x. On the other hand, a split cut obtained as a GMI cut from tableau Ā has the form: α jx j α 0, j J( x ) and cuts off x but is not necessarily violated by x. Indeed, x j = 0 j J( x ) J( x) but x j 0 j J( x ) B( x), therefore the left hand side may be > 0 at x. The cut will be violated if and only if j J( x ) B( x) α j x j < α 0. This suggests that we should aim for small (hopefully zero) cut coefficients on the columns with indices in J( x ) B( x). In Figure 3, we picture an example of a non violated cut: the L&P cut obtained from the new basic solution x and the initial disjunction x k x k x k x k +1 cuts off x by construction, but as soon as the disjunction is modified, we are only guaranteed to cut

8 4 REDUCE-AND-SPLIT FROM NON-OPTIMAL BASES 8 α x = α 0 πx π 0 x πx π 0 +1 x r 1 r 3 r 2 x k x k x k x k +1 Figure 3: A Reduce-and-Split cut from the disjunction πx π 0 πx π 0 +1, obtained from the tableau associated with the basic solution x, that does not cut off the point x. off x (as shown by the cut α x α 0). In order to generate cuts from Ā that are likely to cut off x, we modify the RS algorithm as follows. Let B = J( x ) B( x) be the set of variables which are basic in the optimal LP tableau but are nonbasic in the L&P tableau on which we apply RS. Given J W J( x ) (e.g. using one of the techniques described in [17]) and scalars σ j > 0 j J W B, we compute: min π Z B I ( x ) i B I( x ) π i d i, (14) where d i = (σ jā ij ) j J W B ; in other words, d i are rows of a submatrix of Ā (corresponding to the set of columns J W ), where each column is rescaled with multipliers σ j. The effect of these multipliers is to modify the importance of the columns when determining π that minimizes the norm in (14), by increasing it (if σ j is large) or decreasing it (if σ j is small). Observe from (14) that we try to reduce the coefficients of (7) on all columns with indices in B : for continuous variables in B, this should yield a reduction on the resulting cut coefficient; for integer variables, the end result is not so clear because of the integer modularization, but â j = 0 always results in a zero cut coefficient in the corresponding column. Since we want to reduce the coefficients relative to B as much as possible, we set σ j = 2 j {i B : x i > 0}, and σ j = 1 otherwise. This prioritizes the reduction of the source row coefficients on the variables with indices B such that the corresponding component in x is nonzero. We experimentally tried other strategies to choose σ j, but this simple idea turned out to work well in practice. A discussion is given in Section 6. The rest of the RS algorithm is unmodified. Note that this method offers no guarantee of finding a violated cut, nor does it guarantee to increase the cut violation with respect to the cut associated with the original source row. However, RS has proven to generate strong cuts in practice, therefore we are interested in testing whether it is equally effective if applied to non-optimal bases of

9 5 COMBINING LIFT-AND-PROJECT AND REDUCE-AND-SPLIT 9 (LP), and in particular those generated by L&P. 5 Combining Lift-and-Project and Reduce-and-Split WecombinedthemethodsdescribedinSection3andSection4intoasinglecutgeneration algorithm, that alternates between the L&P and the RS cut improvement procedures. Our cut generation algorithm always starts with determining the set of basic integer variables that have a fractional value (at least 10 2 away from an integer) in the current solution to the LP relaxation; the corresponding (elementary) disjunctions are processed by nonincreasing violation (i.e. those with a violation closest to 0.5 are processed first), until a given maximum number of cuts M is generated, or there are no more violated disjunctions available. In this paper, we always use M = 50. This method for processing elementary disjunction is taken from [7]. Recall that these disjunctions give rise to the traditional GMI cuts. Then, we iteratively modify each GMI cut, changing either the underlying disjunction (through RS), or the underlying basis of the LP relaxation (through L&P). One parameter of our cut generation algorithm is the maximum number η of cut improvement steps that we want to perform, i.e. the number of times that we alternate between L&P and RS. When η = 0, we use the initial GMI cuts. Another parameter start is whether to apply L&P or RS at the first cut modification step. For instance, if start = L&P and η = 3, the GMI cuts are strengthened by L&P, then the underlying disjunction is modified by RS (using the simplex tableau computed by L&P), finally we change the basis again (using the new disjunction) with L&P. After each cut improvement step, we check the outcome of the routine (L&P or RS). If the routine fails, either because it could not improve the cut (i.e. L&P could not perform improving pivots, or RS could not find a disjunction that improves the cut coefficients) or because numerical problems were detected, then the improvement procedure for that particular cut is stopped, and we generate the cut computed at the previous iteration. For instance, for start = L&P and η = 3, if the RS algorithm at step 2 fails, we generate the L&P cut obtained at step 1. If the cut computed at the previous iteration does not satisfy the numerical requirements, then the cut is discarded and we restart the process with another elementary disjunction. Note that if the first improvement step fails, then we simply generate the initial GMI cut. This method is designed to be balanced between L&P and RS: since we always start with the same M elementary disjunction, we can compare the effects of starting with L&P or with RS. Note that this method is based on simple GMI cuts, which have proved to be one of the most effective and reliable general-purpose classes of cutting planes: our method tries to improve on the GMI cuts, but in case of failure, we revert back to the GMI cuts. 6 Computational experiments The cut generation algorithms presented in this paper were implemented in C++ within the COIN-OR Cgl [14] framework. Our L&P generator is a modification of the existing CglLandP generator [7]; likewise, the RS implementation is based on the existing CglRedSplit2 generator [17]. The CglLandP generator employs advanced simplex algorithm functions, and for this reason it only works with the COIN-OR Clp [15] LP solver. Traditional GMI cuts were generated using the CglLandP generator, setting the maximum number of pivots to zero. We used Cplex 12.1[19] to perform instance preprocessing

10 6 COMPUTATIONAL EXPERIMENTS 10 and Branch-and-Bound. More details on the interaction between Clp and Cplex are given in Section 6.1. Our set of test instances is a subset of the mixed-integer instances in the union of MIPLIB3 [11], MIPLIB2003 [2] and the set of test instances of the University of Bologna available from We selected all mixedinteger instances such that the LP has fewer than nonzero elements, and such that we were able to generate 10 rounds of cutting planes with the original CglLandP generator in less than 20 minutes. The instance bell5 was not selected because of its poor numerical properties, which made computational experiments give erratic results, thus producing noise in the data instead of useful information. We divide the instances in three difficulty classes, depending on the performance of our Cut-and-Branch algorithm (see Section 6.1) with cutting planes generated by the original CglLandP. Instances are labeled Easy if they can be solved requiring less than one minute of CPU time and 1000 nodes; they are labeled Medium if they are not Easy but can be solved in less than 2 hours; they are Hard if they cannot be solved in 2 hours of total CPU time. A list of instances is given in Table 1. In all tests reported in this section, the value of the optimal solution is given as a cutoff value so that the time of discovery of integer solutions does not affect the size of the enumeration tree. Easy Medium Hard 10teams aflow30a a1c1s1 blend2 arki001 aflow40b dcmulti bell3a b1c1s1 dsbmip gesa2 b2c1s1 egout gesa2 o bg fiber glass4 dano3mip fixnet6 mas74 danoint flugpl mas76 dg gen misc07 mkc gesa3 mod011 momentum1 gesa3 o modglob momentum2 khb05250 noswot nsrand-ipx misc06 pk1 opt1217 qnet1 pp08acuts roll3000 qnet1 o pp08a set1ch rentacar qiu swath rgn timtab1 rout timtab2 vpm1 tr12-30 vpm2 Table 1: List of test instances. 6.1 Cut-and-Branch To assess the effectiveness of our cut generation procedure, and compare our cut generator to the traditional L&P and RS cuts, we implemented a Cut-and-Branch algorithm on top of Cplex [19] and Clp [15]. Recall that the L&P cut generator requires a simplex tableau in Clp format. Therefore, we proceed as follows: each problem instance is read

11 6 COMPUTATIONAL EXPERIMENTS 11 and preprocessed by Cplex with default settings. The presolved reduced problem is then loaded with Clp, and cutting planes are generated for a maximum of 10 rounds or 20 minutes of CPU time. At each round of cut generation, we perform this sequence of operations. First, we generate at most 50 cuts, and add all of them to the LP formulation. Then, we check if any of the cutting planes generated at previous rounds (and subsequently removed from the LP) is violated by the current fractional point; if so, we add all such cuts to the LP. Finally, the LP is reoptimized, and all inactive cutting planes are removed. The LP formulation obtained after 10 rounds is loaded into Cplex, where another pass of presolve is executed before switching to Branch-and-Bound. To simulate a bare Branch-and-Bound algorithm within the Cplex environment, we apply the following settings: Cutting planes are disabled (cutsfactor = 0, and all cut generation algorithms manually disabled); Emphasis on proving optimality (mipemphasis = bestbound); Heuristics are disabled (heurfreq = -1, and all heuristics manually disabled); Absolute and relative integrality gap for optimality set to zero (epgap = 0, epagap = 0). Constraint and integrality precision were set to All other parameters are left to their default value. 6.2 Parameters for cut generation Our cut generation algorithm is described in Section 5, and has two main parameters. Additionally, both the L&P and the RS cut generators require some parameters to perform each improvement step. For L&P, the pivot selection rule is set to most negative reduced cost, the maximum number of pivots is set to 10, and we do not apply the iterative modularization technique discussed in [7]. For RS, the maximum support of the disjunction is set to 5, the maximum 1-norm of the disjunction is set to 10, the column selection strategy (i.e. the choice of the set J W ) is set to first 1/3 of the columns with smallest reduced cost, and the row selection strategy is set to rows with smallest angle with respect to the source row in the space J W J I (the latter two parameters correspond to the strategies CS1, RS8 in [17]). These parameters were chosen for their performance based on the computational experiences reported in [7, 17]. Even though other values for the L&P and RS cut generators were tested, for space reason we only report results with this set of values. For the combined cut generation algorithm, we tested up to 6 cut improvement iterations, starting either with L&P or with RS. Each combination of parameters yields a different cut generator, which we label as L&P-η if L&P is applied first and we perform up to η improvement steps alternating between RS and L&P, or as RS-η if RS is applied first and we perform up to η improvement steps alternating between L&P and RS. Note that L&P-1 and RS-1 correspond to simple L&P and RS cuts respectively. 6.3 Results with Cut-and-Branch We now report and discuss the results obtained within the framework presented in this section, for several cut generators. For each cut generator and each instance, we report: the amount of integrality gap closed at the root after 10 rounds of cut generation, the

12 6 COMPUTATIONAL EXPERIMENTS 12 CPU time required for cut generation, the number of generated cuts, the amount of integrality gap closed at the end of the Cut-and-Branch algorithm (100% if optimality is proven within the time limit, < 100% if the two hours limit is hit), the number of enumerated nodes, and the total CPU time required by Cut-and-Branch. Detailed results can be found in Tables 5 through 10, whereas averages are given in Table 2. The average integrality gap and number of cuts are computed as arithmetic averages; the average CPU time and number of nodes are geometric averages (to deal with zero values, we added one to each value before computing the average, and subtracted one from the result). For comparison, we also report, in Tables 3 and 11, results obtained within the same framework using traditional GMI cuts from the optimal tableau. In the integer programming community, it is known that comparing the strength of different cut generators is a difficult task, especially when we are interested in the performance in Cut-and-Branch, and average values alone can be misleading. [20] proposes a framework for statistical tests. In this paper, in addition to reporting average values, we opted for a simple pairwise comparison between the 12 tested cut generators; in each comparison, we count the number of instances on which the first method is clearly superior to the second one. Our comparision criteria are as follows: method A is superior to method B on a given instance if: A solves the instance in no more than ρ% the number of nodes required by B and the difference in the number of nodes is at least 100, or A closes at least γ% more integrality gap than B in the two hours. In this paper, we chose ρ = 50 and γ = 5. Observe that in the first criterion, we require the difference in the number of nodes to be at least 100, to deal with easy instances which are solved in few nodes by both methods. If no method is superior to the other, than both methods have comparable strength on that instance. In Table 4 we report detailed results of this pairwise comparison. We empirically verified that, as long as reasonable values for ρ and γ are used, changing ρ or γ does not affect the ranking of the algorithms. First, we observe that all proposed methods appear to be stronger than simple GMI cuts on average: in terms of gap closed at the root node, and in terms of number of nodes for Easy and Medium instances. On Hard instances, the gap closed by GMI cuts after Branch-and-Bound is comparable to some of the other tested method, even though GMI cuts appear to be weaker at the root. However, the number of nodes processed in two hours is larger for GMI cuts, which explains why the amount of gap closed after Branch-and-Bound is similar. GMI cuts are also the fastest method for Easy instances on average, but not for Medium instances. Table 4 confirms that GMI are the weakest of the cut generation algorithms tested in this paper: they are inferior the largest number of times (180), although the number of instance on which they are superior is not the lowest one. However, a more detailed analysis of the results show that the GMI cut generator tends to be superior mostly on Easy and Medium instances: on Hard instances, the other cut generators typically show better performance. We also observe that RS-1 performs very well on average on Medium and especially Hard instances, and appears to be stronger than L&P-1 by looking at Table 2 only; a more detailed analysis of the results reveals that its good average behaviour depends on some Hard and Medium instances on which RS-1 is considerably stronger than other cut generators (examples are vpm1, danoint, dg012412, opt1217), but on several other instances RS-1 is clearly weaker. This is well indicated by Table 4: RS-1 is the method (except GMI) that is inferior the largest number of times, while being consistently superior toothermethodson 7instances. TheweaknessofRScutsonsomeinstances

13 6 COMPUTATIONAL EXPERIMENTS 13 is mitigated by performing L&P on the RS cuts; we can see that already for η = 2: RS-2 is inferior to other methods only 70 times, compared to the 132 of RS-1. RS-η for η > 1 do not perform equally well as RS-1 on the few instances where RS-1 really dominates. Still, on average η > 1 yields better results on the Easy and Medium instances (significant reduction in the number of nodes), while being comparable on the Hard problems; the gap closed at the root node increases significantly on all problem classes. Therefore, iterating between RS and L&P seems to pay off. For η > 4, we hardly see any improvement. Overall, RS-η with η > 1 is a more reliable cut generation algorithm than RS-1. The strength of RS-1 on some problems, unmatched by any other cut generator, suggests that employing RS-1 in combination with other generators could perform well in practice. Similarly, L&P cuts benefit from their combination with the RS algorithm. Looking at Table 2, L&P-η with η > 1 is superior to L&P-1 in almost all respects: gap closed at the root (except on Hard instances for some values of η, for which we observe a slight decrease), number of nodes on the Easy and Medium instances, and gap closed after Branch-and-Bound on the Hard instances. More importantly, Table 4 shows a clear trend when moving from η = 1 to η = 4: the cut generators are superior a larger number of times, and inferior a smaller number of times. The peak is reached for L&P-4: overall, this generator is, at the same time, the one which is inferior the smallest number of times, and superior the largest number of times. It solves the Medium instances in the smallest average number of nodes, and performs well in terms of gap closed on Hard instances. Additionally, L&P-3 and L&P-4 are the only methods to solve one Hard instance (aflow40b) within the 2 hours time limit; L&P-4 requires 5240 seconds only for this task. These results suggest that combining the L&P and RS algorithms is indeed very effective, and that alternating between them 4 times yields the best result; after 4 iterations, we did not observe any improvement. Moreover, applying L&P as the first GMI cuts strengthening step seems a better choice than starting with RS: this is because RS cuts are not as consistently strong as L&P cuts, being very strong on some problems, but weak on others. Finally, we observe that the average cut generation time increases by 50% from η = 1 to η = 2: the second step is computationally expensive, but not as expensive as the first one. This is because at each step, we can reuse some of the data computed at previous iterations; in particular, we do not have to recompute the LP basis inverse from scratch. For each iteration η > 2, the CPU time required for cut generation increases by less than 10%, since the number of cuts that are modified decreases. On Easy instances, GMI cuts are the best choice in terms of CPU time, because all other methods spend too much time cutting at the root more than the time needed to solve the instance with GMI cuts only. On Medium problems, the reduction in number of nodes using our proposed methods can be significant, but the reduction in CPU time is not as large; this is because there are still some Medium instances for which our cut generation algorithms take too much time at the root, compared to the total solution time. For instance, if we compare L&P-4 (which yields the smallest average number of nodes) to GMI on Medium problems, we see that L&P-4 is only 0.5 seconds faster despite processing less than half as many nodes. However, on the two gesa2 problems, L&P-4 spends more than 90% of the total solution time in cut generation, slowing down the process. If we exclude those two instances, the difference increases in L&P-4 favor: seconds on average for L&P-4, for GMI. On Hard instances, which are not solved to optimality (except aflow40b by L&P-3 and L&P-4), it is difficult to compare CPU times. To conclude, our cuts can be effective in reducing total solution time, provided that we have a mechanism to detect easy instances for which excessive cut generation time is detrimental. This issue is beyond the scope of this paper.

14 6 COMPUTATIONAL EXPERIMENTS 14 Table 2: Average values for Tables 5 through 10. L&P RS root cut #cuts final #nodes total root cut #cuts final #nodes total η instances gap % time gap % time gap % time gap % time Easy Medium Hard Easy Medium Hard Easy Medium Hard Easy Medium Hard Easy Medium Hard Easy Medium Hard Table 3: Average values for Table 11. GMI root cut #cuts final #nodes total instances gap % time gap % time Easy Medium Hard RS cuts on non-optimal bases: cut violation We provide here a brief analysis of the cut violation when RS cuts are generated from non-optimal bases of the LP relaxation. Recall from Section 4 that in this case, we could generate cuts which are not violated, which would have to be discarded. It is natural to ask how often does this happen; this is the question we try to answer in this section. Thus, we recorded the number of non violated cuts that are computed while applying 10 rounds of cuts at the root on our test set, with the L&P-4 generator (which turned out to be the strongest one, see Section 6.3). We gathered the same statistics for other generators as well, and obtained very similar results, therefore here we only present data for L&P-4. As discussed in Section 4, if we generate large cut coefficients on the variables j B which are basic in the optimal LP basis, but are nonbasic in the L&P basis from which RS cutsaregenerated, thenthecutmaynotbeviolated. ThisexplainswhywemodifytheRS algorithm to try and reduce those cut coefficients as much as possible. How often do we generate non-violated cuts if we employ the RS algorithm unmodified? It turns out that, evenifwedonotconsiderthesetb whenapplyingtherscoefficientreductionalgorithm (i.e. we apply the RS algorithm directly as described in [17]), only 11 cuts are discarded because they are not violated. This is an extremely small number: for comparison, the total number of generated cuts is We give two possible explanations for this behavior. First, the L&P cut given in input to the RS procedure cuts off the optimal basic solution x by a larger amount than the initial GMI cut; hence, changing the split disjunction to obtain a stronger cut is likely to still cut off x. Second, sparsity plays in our favor: if the LP tableau on which RS is applied is sufficiently sparse, it is likely that computing a linear combination of its rows will not deteriorate the coefficients on the columns j B by a large amount.

15 6 COMPUTATIONAL EXPERIMENTS 15 Table 4: Pairwise comparison of cut generators: number of instances on which the cut generation algorithm on the row is superior to the one on the column. L&P-1 L&P-2 L&P-3 L&P-4 L&P-5 L&P-6 RS-1 RS-2 RS-3 RS-4 RS-5 RS-6 GMI Sum Sup. L&P L&P L&P L&P L&P L&P RS RS RS RS RS RS GMI Sum Inf Thus, there is not a big margin of improvement for the modification of the RS algorithm proposed in Section 4: the number of non-violated cuts is already negligible. Indeed, it turns out that with the modified RS algorithm, we still generate 11 nonviolated cuts (in total, we generate cuts in this case). However, an interesting side effect of the modification is that we close more integrality gap: the average integrality gap closed at the root over all instances after 10 rounds increases from 47.39% to 49.01%. Hence, the proposed modification seems to have a positive effect. Our intuition is that the modified RS algorithm is likely to increase the cut violation, yielding deeper cuts. This can be seen by looking at the expression for the distance cut off (first used as a measure of cut quality in [8]). Suppose the cut is αx α 0 ; then the normal vector of the hyperplane represented by this cut is α. Therefore, the distance d of the basic solution x from the hyperplane αx = α 0 satisfies α( x+dα) = α 0. From this we get the expression: d = (α 0 α x)/ α 2 2. (15) By giving more priority to reducing cut coefficients on the columns j B such that x j > 0, the modified RS algorithm acts on both the numerator and the denominator of (15), as opposed to only trying to reduce the denominator. 6.5 Cut density We conclude our computational study with an analysis of the density of the cutting planes generated by the methods proposed in this paper. The density is recorded on all cutting planes generated during the 10 rounds applied at the root node of all instances in our test set, and for each cut it is computed as a percentage with respect to the maximum density allowed, i.e.: number of nonzeroes over the maximum number of nonzeroes allowed. The maximum number of nonzeroes allowed is equal to min{n, n/5}, where n is the number of columns; similar strategies to select the maximum density are used in the

16 7 CONCLUSION 16 Branch-and-Cut solvers COIN-OR Cbc [13] and SCIP [1]. In Figure 4 we report the average density values for all cut generators L&P-η and RS-η with η = 1,...,6, for each round of cut generation applied at the root. For comparison, we additionally report the same curve for the traditional GMI cuts. density GMI LaP-1 LaP-2 LaP-3 LaP-4 LaP-5 LaP-6 RS-1 RS-2 RS-3 RS-4 RS-5 RS-6 cut density round Figure 4: Average cut density for the first 10 rounds. We can draw some conclusions from the graph. Surprisingly, GMI cuts are the densest cut on average, and they are also denser than most other cuts through the 10 rounds, with the exception of RS-1. RS-1 is close to GMI in most rounds; therefore, even if it aims at reducing cut coefficients (in the extended (n+m)-space, i.e. when the tableau is expressed with equality constraints), it does not reduce density (in the original n-space) by a large amount: 2% on average. L&P cuts, on the other hand, appear to be consistently sparser than GMI cuts through all 10 rounds. The same beneficial effect is observed when L&P and RS are combined. An important observation is that there does not seem to be an increase in cut density when η moves from 1 to 6: our combined L&P + RS cut generation algorithm is very stable in this respect, regardless of the number of iterations and whether we start with L&P or RS. Finally, density grows steadily with the number of applied rounds, and the distance between GMI and other cut generators becomes smaller: at the tenth round, all cut generators yield similarly dense cuts, and the density is more than double that of the first round. 7 Conclusion In this paper we presented a combination of two existing algorithms for generating split cuts: Lift-and-Project and Reduce-and-Split. In doing so, we introduced an extension of the Lift-and-Project procedure on the original simplex tableau that can be employed

17 REFERENCES 17 on general split disjunctions (instead of elementary disjunctions), and we analyzed the application of Reduce-and-Split on non-optimal bases of the LP relaxation. We obtained a cut generation algorithm that iteratively modifies both the LP basis and the split disjunction from which a split cut is generated. Computational experiments on a set of benchmark instances showed that this combination is effective on mixed-integer instances, solving problems in a smaller number of nodes and closing more integrality gap on the unsolved problems on average. In particular, iterating more than once between L&P and RS proved to be a good choice: in our experiments, applying L&P first and then iterating 4 times between the two algorithms yielded the best results. Our cut generation algorithm is not significantly slower than the original L&P and RS algorithms, but generates stronger cutting planes that should be useful in practice for the solution of difficult MILPs. References [1] T. Achterberg. Scip: Solving constraint integer programs. Mathematical Programming Computation, 1(1):1 41, [2] T. Achterberg, T. Koch, and A. Martin. MIPLIB Operations Research Letters, 34(4): , [3] K. Andersen, G. Cornuéjols, and Y. Li. Reduce-and-split cuts: Improving the performance of mixed integer Gomory cuts. Management Science, 51(11): , [4] K. Andersen, G. Cornuéjols, and Y. Li. Split closure and intersection cuts. Mathematical Programming A, 102(3): , [5] E. Balas. Intersection cuts - a new type of cutting planes for integer programming. Operations Research, 19(1):19 39, [6] E. Balas. Disjunctive programming. Annals of Discrete Mathematics, 5:3 51, [7] E. Balas and P. Bonami. Generating lift-and-project cuts from the lp simplex tableau: open source implementation and testing of new variants. Mathematical Programming Computation, 1: , [8] E. Balas, S. Ceria, and G. Cornuéjols. Mixed 0-1 programming by lift-and-project in a branch-and-cut framework. Management Science, 42(9): , [9] E. Balas, S. Ceria, G. Cornuéjols, and N. Natraj. Gomory cuts revisited. Operations Research Letters, 19(1):1 9, [10] E. Balas and M. Perregaard. A precise correspondence between lift-and-project cuts, simple disjunctive cuts, and mixed integer gomory cuts for 0-1 programming. Mathematical Programming, 94(2-3): , [11] R. E. Bixby, S. Ceria, C. M. McZeal, and M. W. P. Savelsbergh. An updated mixed integer programming library: MIPLIB 3.0. Optima, 58:12 15, [12] S. Ceria, G. Cornuéjols, and M. Dawande. Combining and strengthening Gomory cuts. In E. Balas and J. Clausen, editors, Integer Programming and Combinatorial Optimization, volume 920 of Lecture Notes in Computer Science, pages Springer Berlin / Heidelberg, 1995.

18 A DETAILED TABLES OF RESULTS 18 [13] COIN-OR Branch-and-Cut. [14] COIN-OR Cut Generation Library. [15] COIN-OR Linear Programming. [16] W. Cook, R. Kannan, and A. Schrijver. Chvátal closures for mixed integer programming problems. Mathematical Programming, 47: , [17] G. Cornuéjols and G. Nannicini. Reduce-and-split revisited: efficient generation of split cuts for mixed-integer linear programs. Technical report, Tepper School of Business, Carnegie Mellon University, April [18] R. E. Gomory. An algorithm for the mixed-integer problem. Technical Report RM-2597, RAND Corporation, [19] I. ILOG. IBM ILOG CPLEX 12.1 User s Manual. IBM ILOG, Gentilly, France, [20] F. Margot. Testing cut generators for mixed-integer linear programming. Mathematical Programming Computation, 1(1):69 95, [21] G. Nemhauser and L. Wolsey. A recursive procedure for generating all cuts for 0-1 mixed integer programs. Mathematical Programming, 46: , [22] M. Perregaard. A practical implementation of lift-and-project cuts. In International Symposium on Mathematical Programming, Copenhagen, [23] F. Wesselmann. Strengthening Gomory mixed-integer cuts: a computational study. Technical report, University of Paderborn, A Detailed tables of results

19 A DETAILED TABLES OF RESULTS 19 Table 5: Detailed results for L&P-1 and RS-1 cuts. L&P-1 RS-1 root cut gen. #cuts final #nodes total root cut gen. #cuts final #nodes total instance gap % time gap % time gap % time gap % time 10teams blend dcmulti dsbmip egout fiber fixnet flugpl gen gesa gesa3 o khb misc qnet qnet1 o rentacar aflow30a arki bell3a gesa gesa2 o glass mas mas misc mod modglob noswot pk pp08acuts pp08a qiu rgn rout vpm vpm a1c1s aflow40b b1c1s b2c1s bg dano3mip danoint dg mkc momentum momentum nsrand-ipx opt roll set1ch swath timtab timtab tr