AN EXPERIMENTAL INVESTIGATION OF A PRIMAL- DUAL EXTERIOR POINT SIMPLEX ALGORITHM

Transcription

1 AN EXPERIMENTAL INVESTIGATION OF A PRIMAL- DUAL EXTERIOR POINT SIMPLEX ALGORITHM Glavelis Themistoklis Samaras Nikolaos Paparrizos Konstantinos PhD Candidate Assistant Professor Professor Department of Applied Informatics, University of Macedonia, 156 Egnatia Str., Thessaloniki, Greece Department of Applied Informatics, University of Macedonia, 156 Egnatia Str., Thessaloniki, Greece Department of Applied Informatics, University of Macedonia, 156 Egnatia Str., Thessaloniki, Greece Abstract The aim of this paper is to present an experimental investigation of a Primal-Dual Exterior Point Simplex Algorithm (PDEPSA) for Linear Programming problems (LPs). There was a huge gap between the theoretical worst case complexity and practical performance of simplex type algorithms. The algorithm bases on interior points to move from one basic solution to another. The main advantage of PDEPSA is its computational performance. This performance stems from the fact that PDEPSA can deal much better with the problem of stalling and cycling. Moreover, the use of interior points is responsible for the reduction of iterations and the CPU time and especially in linear degenerate problems. A computational study is also presented with experiments on randomly generated sparse optimal linear problems, which increases the reliability of the conclusions. The algorithm is very encouraging and promising due to the fact that it proved its superiority to the exterior point algorithm and the primal revised simplex algorithm on the computational study. KEYWORDS Linear programming, Primal Simplex algorithms, Exterior Point Algorithms, Computational results. 1. INTRODUCTION One of the most significant and well-studied optimization problems is the Linear Programming problem (LP). Linear programming consists of optimizing, (minimizing or maximizing) a linear function over a certain domain. The domain is given by a set of linear constraints. The simplex algorithm is the oldest and most well-known solution method for the LP. George B. Dantzig is regarded as the founder of linear programming and the person who set the fundamental principles of optimization. The simplex starts from a feasible solution and moves from one vertex to an adjacent one until an optimum solution is computed (Dantzig, 1963). The two main drawbacks of the simplex algorithm are the stalling and/or the cycling problem. Moreover, the phenomenon of cycling is very often in many practical problems for the simplex algorithm. In order to avoid the problem of cycling, researchers have introduced many anti-cycling pivoting rules (Terlaky & Zhang, 1993). The simplex algorithm performs effectively in practice only under specific circumstances and especially on small or medium size LPs. It has been proved that the average number of iterations in simplex algorithm can be estimated with a polynomial bound (Borgwardt, 1982). Despite that the worst case complexity of the simplex method has exponential behavior (Klee & Minty, 1992). Until the decade of 1980 simplex algorithm and barrier methods were known as the only solution approach for the LP. This situation changed in 1984 when the first try of interior point methods for linear programming appeared (Karmarkar, 1984). Next years researchers focused their attention on how the simplex algorithm and the Interior Point Methods (IPMs) can be combined in order to improve the computational behavior of software packages (Erling et al., 1996). In contrast to the simplex method, IPMs compute an optimal solution by moving inside the feasible region. Moreover, the research adopted primal-dual algorithms based on IPMs and simplex method which were regarded as the most significant and useful algorithms. The primal-dual methods for linear programming have interesting theoretical properties. On the

2 computational side, Mehrotra s predictor corrector algorithm was the main idea for the most interior-point software (Wright, 1997). Primal-dual methods have good computational performance and they can be extended to wider classes of problems in mathematical programming (Gondzio, 1996). Apart from the IPMs, another approach to solve LPs is to modify the main idea of simplex algorithm to move from one basic vertex to another in the exterior of the feasible region. The algorithm can construct basic infeasible solutions instead of feasible. The algorithm which includes these kinds of features are called exterior point simplex algorithm (EPSA). The first attempt for an exterior point algorithm was introduced in 1991 for the assignment problem (Paparrizos, 1991). Since then, many papers and articles have been presented in order to enhance the capabilities of the exterior point algorithms. A significant improvement is the primal-dual versions of the algorithm which reveal its superiority to simplex algorithm. The main idea of exterior point algorithm is that it relies on two paths for estimating the optimal solution. The first past refers to a number of feasible solutions until the optimal is found and the other path includes basic points which lead to the exterior path. The exterior point algorithm has two main disadvantages (Paparrizos et al., 2003B). The first refers to the construction of moving directions which should lead the algorithm close to the optimal solution. The creation of a direction with these features is a difficult process. The second disadvantage is the fact that there is no known method which can reveal the path that leads into the interior of the feasible region, something which would make easier the search of a computational good direction. These advantages can be avoided if the exterior path is replaced with a dual feasible simplex path. This method has been introduced by Paparrizos et al. (Paparrizos et al., 2003A). The main idea of the Revised Primal Dual Simplex Algorithm (RPDSA) is based on the process of moving from any interior point to an optimal basic solution. The advantage of this algorithm is that the optimal solution is not computed by an interior point method but it can be used only in the first stage. After the first few iterations IPMs do not result in great enhancements of the objective function s value. RPDSA can be applied in the second stage and computes the optimal solution in a few iterations, much faster from an interior point method. Although this algorithm is better from the exterior point algorithm and it deals very well with the two disadvantages which were described above. The algorithm of our paper introduces a new technique which deals with the problems of stalling and cycling and improves the performance of the RPDSA. This algorithm has been developed by Samaras (Samaras, 2001). This algorithm is primal-dual, meaning that it simultaneously solves both the primal and dual problem. RPDSA begins with a boundary point of the feasible region. This boundary point is used in order to compute the leaving variable. It has been observed that in this stage the problem of stalling can arise very often. This weakness can be overcome if the boundary point is replaced by an interior point. The transfer into the interior of feasible region disappear the problem of cycling. In order to gain an insight into the practical behavior of the proposed algorithm, we have performed some computational experiments. Preliminary results on randomly generated LPs show that the reduction in the computational effort is promising. The paper is organized as follows. Following introduction, in section 2 we briefly describe the general framework of the proposed algorithm. In section 3 the computational study is presented. Finally, in section 4 there are the conclusions and possible enhancements of the proposed algorithm. 2. ALGORITHM DESCRIPTION In this section we briefly describe the Primal-Dual Exterior Point Simplex Algorithm (PDEPSA). For solving general LPs see (Paparrizos et al., 2003B) and for a full description of PDEPSA see (Samaras, 2001). Consider now the following linear program in the standard form. min c T x s. t. Ax = b (LP) x 0 where A R mxn, c,x R n, b R m and T denotes transpose. Assume that A has full rank, rank(a)=m (m<n). The dual problem associated with (LP) is max b T x s. t. A T w+s = c (DP) s 0

3 where w R n and s R n. Step 0 (Initialization): A) Start with a dual feasible basic partition (B, N) and an interior point y of (LP). Set: P = N, Q = and compute 1 Τ T 1 T T xb = ( AB) b, w = ( cb ) ( AB),( sn) = ( cn) w AN B) Compute the direction d B from the relation: db = yb xb Step 1 (Test of optimality and choice of the leaving variable): If x 0, STOP. The (LP) is optimal. Else, choose the leaving variable xk = xb[ r] from the relation: xbr [ ] xbr [ ] a = l max{ : db[] i 0 [] 0} d = d > x < B i Br [ ] Br [ ] Step 2 (Computation of the next interior point): Set: Compute the interior point: yb = xb + adb a a = Step 3 (Choice of the entering variable): 1 Set: H rn = ( B ) r. A. N Choose the entering variable x l from the relation: s s l j = min{ : H rj j N} H H 1 Compute the pivoting column: hl = B A. l If l P, P P\{l} Else Q Q\{} l rn Step 4 (Pivoting): Set: Br [ ] = land Q Q {k} Using the new partition (B, N) where N = (P, Q), compute the new basis inverse B -1 and the variables: 1 Τ T 1 T T xb = ( AB) b, w = ( cb ) ( AB),( sn) = ( cn) w AN Go to step 0B. rj 3. COMPUTATIONAL STUDY In order to check and test the performance and practical effectiveness of our algorithm, a computational study is presented below. The problems which were included in the computational study were small and medium scale sparse instances. The computational study includes the revised simplex algorithm, the Exterior Point Simplex Algorithm (EPSA) and the Primal Dual Exterior Point Simplex Algorithm (PDEPSA). All implemented algorithms are running in MATLAB Professional R2010b. MATLAB (MATrix LABoratory) is a powerful programming environment and as its name suggests, is especially designed for

4 matrix computations like, solving systems of linear equations or factorizing matrices. Also, it is an interactive environment and programming language for numeric scientific computing. The computing environment includes an Intel(R) Core i GHz (2 processors) and MB RAM. The operating system was Microsoft Windows 7 Professional SP1 edition. All times in the following tables are measured in seconds with the cputime built-in function of MATLAB, including the time spent on scaling. All runs were made as a batch job. Two techniques are used in order to improve the performance of memory-bound code in MATLAB. These techniques are: (i) Pre-allocate arrays, and (ii) Store and access data in columns. Three different density classes of LPs are used in the computational study: 5%, 10% and 20%. In each dimension 10 different instances are tested. These LPs include only randomly generated matrices. The initial basis consists of only the slack variables. Tables 1-3 present the average execution time in seconds (CPU time) and the average number of iterations (niter) that each of the competitive spent to solve the LPs in request. Table 1. Results for randomly generated sparse LPs with dimension nxn and density 5% PDEPSA EPSA Simplex Density 5% CPU_time (sec) Niter CPU_time (sec) Niter CPU_time (sec) Niter 500 x 500 0, , , x 600 1, , , x 700 2, , , x 800 3, , , x 900 5, , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , Total 468, , , As it is obvious from the first table, PDEPSA is clearly superior to the other two algorithms. The execution time of PDEPSA is much lower than EPSA and simplex algorithm. For example, PDEPSA needs almost 8 seconds to solve a 1000 x 1000 problem when EPSA demands almost 40 seconds and even worse Simplex Algorithm needs 83 seconds to complete the computations. Likewise, the number of iterations is much greater for Simplex Algorithm and much less from PDEPSA. In addition, in a 2000 x 2000 problem the simplex algorithm requires in average iterations when PDEPSA claims for iterations and EPSA algorithms. It is clear that PDEPSA can lead to significant reductions of the execution time and the number of iterations. In the same problem, PDEPSA can solve the linear problem almost in 90 seconds when Simplex algorithm needs seconds. One may infer a growth in the relative speed of PDEPSA with respect to simplex and EPSA as problem sizes increase. Table 2. Results for randomly generated sparse LPs with dimension nxn and density 10% PDEPSA EPSA Simplex Density 10% CPU_time (sec) Niter CPU_time (sec) Niter CPU_time (sec) Niter 500 x 500 0, , ,

5 600 x 600 1, , , x 700 2, , , x 800 4, , , x 900 6, , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , Total 543, , , In less sparse problems the PDEPSA continues to present its surprisingly good performance. Its superiority is clear and with no doubt it is an effective and promising algorithm. In all dimensions PDEPSA has the shortest execution time and the smallest number of iterations. Table 3. Results for randomly generated sparse LPs with dimension nxn and density 20% Density 20% PDEPSA EPSA Simplex CPU_time (sec) Niter CPU_time (sec) Niter CPU_time (sec) Niter 500 x 500 1, , , x 600 2, , , x 700 3, , , x 800 5, , , x 900 8, , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , x , , , Total 678, , , , , ,635 In addition, in the last group of problems with bigger density PDEPSA keeps to be the algorithm with the best performance by far. A characteristic example, with a 1500 x 1500 problem PDEPSA requires in average 52 seconds when EPSA needs 130 seconds and the simplex algorithm 600 seconds. Likewise, the number of iterations is much less in PDEPSA which is able to estimate the optimal solution only within iterations when EPSA needs to make and the simplex algorithm almost iterations.

6 Furthermore, below figures can show more clearly the superiority of PDEPSA over EPSA and the Simplex Algorithm. Figures 1-3 present the ratios (execution time of the EPSA)/(execution time of the PDEPSA), (iterations of the EPSA)/(iterations of the PDEPSA), (execution time of the Simplex Algorithm)/(execution time of the PDEPSA) and (iterations of the Simplex Algorithm)/(iterations of the PDEPSA) for the corresponding densities and dimensions. The above ratios indicate how many times PDEPSA is better than EPSA and the Simplex Algorithm. Figure 1. Speed-up ratios for nxn LPs and density 5% 25,00 cpu time ratio EPSA over PDEPSA cpu time ratio Simplex over PDEPSA niter ratio EPSA over PDEPSA niter ratio Simplex over PDEPSA 20,00 15,00 10,00 5,00 0,00 From the above figure, it is clear that as the problem dimensions increases the superiority of PDEPSA over EPSA does not vary a lot and we can claim that PDEPSA is five times faster than EPSA and it requires five times less iterations to complete its computations. On the other hand, the superiority of PDEPSA over simplex algorithm increases proportionally with the dimension of the problems. With small size LPs PDEPSA is almost five times faster and requires five times less iterations than simplex method. In contrast, in bigger LPs, like 2000 x 2000 LPs PDEPSA is almost 23 times faster and it needs 13 times less iterations. Figure 2. Speed-up ratios for nxn LPs and density 10% 25,00 cpu time ratio of EPSA over PDEPSA niter ratio of EPSA over PDEPSA cpu time ratio of Simplex over PDEPSA niter ratio of Simplex over PDEPSA 20,00 15,00 10,00 5,00 0,00 As the density of problems increases, PDEPSA continues its superiority and the results are as satisfactory as in more sparse problems. Comparatively to EPSA, PDEPSA is 4 times faster and it demands 4 times less iterations in small size LPs. However, in bigger scale LPs the difference decreases to 3 times for the execution time and the number of iterations. In regard to the simplex method, the results are very similar with the previous group of problems (density 5%), the superiority of PDEPSA over simplex algorithm increases analogically with the dimension of problems.

7 Figure 3. Speed-up ratios for nxn LPs and density 20% 18,00 16,00 14,00 12,00 10,00 8,00 6,00 4,00 2,00 0,00 cpu time ratio of EPSA over PDEPSA cpu time ratio of Simplex over PDEPSA niter ratio of EPSA over PDEPSA niter ratio of EPSA Simplex PDEPSA In the last group of problems with the density at the level of 20%, the results do not differ a lot from the previous. PDEPSA is almost 3 times better than EPSA both in the execution time and the number of iterations. On the other hand, PDEPSA is almost 6 times faster and it requires 6 times less iterations in small size LPs like 500 x 500 and 600 x 600 dimension problems. In bigger scale LPs, the difference increases and it reaches 16 times faster referring to execution time and the 10 times less iterations for PDEPSA. The results of the computational study clearly proved that the PDEPSA can perform faster and with a minor number of iterations in perspective to the performance of the simplex algorithm. Nevertheless, it is significant to mention the fact that as the density increases the superiority of PDEPSA decreases. More specific, in 2000 x 2000 dimension the PDEPSA is 23 times faster than the simplex algorithm in the 5 % density when in 20% density PDEPSA is almost 16 times faster. Despite the fact that this looks like a drawback because the performance of PDEPSA decreases comparing to simplex algorithm, in fact is not. In real world, the vast majority of problems are extremely sparse problems, and their density does not reach the level of 5%. Consequently, this computational performance of PDEPSA is a remarkable advantage and clearly shows its worth. 4. CONCLUSION The current paper investigates the practical behavior of the PDEPSA. The PDEPSA which is presented in this paper refers to the attempt to avoid the problem of stalling and/or cycling. The elimination of these disadvantages can lead to a more effective algorithm with better computational performance. Apart from the description of the algorithm, we presented an extended comparative computational study between the Revised Primal Simplex Algorithm, the Exterior Point Simplex Algorithm and the Primal-Dual Exterior Point Simplex Algorithm. In this computational study randomly generated sparse optimal linear problems are used and all the implementations were accomplished with the help of MATLAB programming environment. Regarding to the results, PDEPSA has proved its superiority to EPSA and the simplex method in all densities and sizes. PDEPSA indicates the same behavior comparing to EPSA in all sizes of the LPs. Their difference is not affected by the dimensions of the linear problems. In contrast to this, in all densities the performance of PDEPSA is getting better comparing to the simplex algorithm while the size of the LPs increases. Moreover, the computational performance of PDEPSA is much better in very sparse problems. As the results clarify, PDEPSA can perform little better in 5% density than in 20% density. The difference between simplex algorithm and PDEPSA is greater in sparser problems. This is a strong and significant advantage of PDEPSA because in real problems the level of density is very low.

8 REFERENCES Borgwardt H.K., The average number of pivot steps required by the simplex method is polynomial. Zeitschrift fur Operational Research, Series A: Theory Vo. 26, No. 5, pp Dantzig, G.B., Linear Programming and Extensions, Princeton, University Press, Princeton, NJ. Erling, D., Andersen, and Yinyu, Ye, Combining interior-point and pivoting algorithms for Linear Programming, Management Science, Vol. 42, No. 12, pp Gondzio, J., Multiple centrality corrections in a primal-dual method for linear programming, Computational Optimization and Applications, Vol. 6, No. 2, pp Karmarkar N.K., A polynomial-time algorithm for linear programming, Combinatorica, Vol. 4, pp Klee V, Minty GJ How good is the simplex algorithm?, Inequalities III. New York: Academic Press, pp Paparrizos K., An infeasible exterior point simplex algorithm for assignment problems. Mathematical Programming, Vol, 51, pp Paparrizos K., Samaras, N., and Stephanides, G., 2003A. A new efficient primal dual simplex algorithm, Computers and Operations Research, vol. 30, pp Paparrizos, K., Samaras, N., Stephanides, G., 2003B. An efficient simplex type algorithm for sparse and dense linear programs, European Journal of Operational Research, Vol. 148, No. 2, pp Samaras N., Computational improvements and efficient implementation of two path pivoting algorithms, PhD Thesis, Department of Applied Informatics, University of Macedonia. Terlaky T., Zhang S., Pivot rules for linear programming A survey, Annals of Operations Research, Vol. 3, No. 1, pp Wright St., 1997, Primal dual interior point methods, Society for Industrial and Applied Mathemaics, Philadelphia, United States of America.