Experiments on Exact Crossing Minimization using Column Generation

Transcription

1 Experiments on Exact Crossing Minimization using Column Generation Markus Chimani, Carsten Gutwenger, and Petra Mutzel Department of Computer Science, University of Dortmund, Germany Abstract. The crossing number of a graph G is the smallest number of edge crossings in any drawing of G into the plane. Recently, the first branch-and-cut approach for solving the crossing number problem has been presented in [3]. Its major drawback was the huge number of variables out of which only very few were actually used in the optimal solution. This restricted the algorithm to rather small graphs with low crossing number. In this paper we discuss two column generation schemes; the first is based on traditional algebraic pricing, and the second uses combinatorial arguments to decide whether and which variables need to be added. The main focus of this paper is the experimental comparison between the original approach, and these two schemes. We also compare these new results to the solutions of the best known crossing number heuristic. 1 Introduction Crossing minimization is among the oldest and most fundamental problems arising in the area of automatic graph drawing and VLSI design and has been studied in graph theory for many decades. A drawing of a graph G = (V, E) is a mapping of each node v V to a distinct point and each edge e = (v, w) E to a curve connecting the incident nodes v and w without passing through any other node. Common points of two edges that are not incident nodes are called crossings. The crossing number, denoted by cr(g), is the minimum number of crossings among all drawings of G. The crossing minimization problem is a central problem in the area of automatic graph drawing, since a low crossing number is among the most important design criteria for drawings [12]. Yet its roots are much older: P. Turán described in his Note of Welcome in the first issue of Journal of Graph Theory [13] that he started to examine this topic during the second world war. Even though much research has been conducted see, e.g., [14] for an overview even the crossing numbers for complete and complete bipartite graphs can only be conjectured. In 2005, Buchheim et al. [3] introduced the first algorithm to compute the exact crossing number, based on a branch-and-cut approach, which we outline in Section 2. The problem with this approach was the huge amount of variables required. This restricted the algorithm to relatively small graphs with few crossings, since its running time heavily depends on the latter parameter.

2 To overcome this drawback, we devised two different column generation schemes, i.e., methods starting with a small subset of variables and dynamically adding variables during the computation; see Section 3. The first approach, called algebraic pricing, is based on the standard pricing technique first introduced by Dantzig and Wolfe [4]. Starting from a small subset of the variables, we can compute reduced costs for the variables not yet in the ILP and decide on their addition based on these values. The second scheme is called combinatorial column generation. Its main idea is to interpret the absence of certain variables in a combinatorial way, such that we can quite intuitively decide, whether these variables have to be added or not. Section 4.1 presents the results of our experimental study, comparing the original method to the two column generation schemes. Note that even the original method is already an improved version of the one presented in [3], due to the use of primal heuristics during the cutting phase, as well as the recently developed preprocessing method called non-planar core [7]. The dataset used for the study is the well known Rome library [5], which has, e.g., also been used for a comparison between crossing minimization heuristics [8]. Section 4.2 gives an experimental analysis of the quality achieved by the currently best known heuristics compared to these values. 2 The Branch-and-Cut Approach ILP and Cutting Planes. In order to solve the crossing minimization problem for a graph G = (V, E), it is necessary to determine which edges cross in an optimal solution. However, it is even NP-complete to decide if there exists a drawing realizing a given set of edge crossings [10]. Therefore, the ILP solution needs to contain information on the order of these crossings along an edge. This is achieved by restraining the solution to crossing restricted drawings (or CR-drawings), i.e., drawings in which each edge is involved in at most one crossing. To compute an optimal solution for G, we represent each edge of G by a sufficiently long chain of segments. An optimal crossing restricted solution for this modified graph clearly induces a crossing minimal solution for G. Obviously, E deg(v) deg(w) 1 segments are sufficient for an edge (v, w), but also any upper bound for cr(g) bounds the number of required segments. This expansion leads to the problematically high number of variables. If D is a set of crossings and we place a new node on every crossing (e, f) D, i.e., we split both e and f and identify the created dummy nodes, we obtain a (partial) planarization G D of G with respect to D. Then, D is realizable if and only if G D is planar. The ILP-formulation for finding a crossing minimal CR-drawing has a 0/1 variable x[e, f] for each unordered pair (e, f) of segments, which is 1 if e and f cross and 0 otherwise. The optimization goal is to minimize the sum of all the x[e, f] according to the following constraints please refer to [3,2] for details: CR-constraints assuring that each segment crosses at most one other edge.

3 Kuratowski constraints that guarantee the realizability of the set of segment crossings given by the variables that are set to 1. Our formulation contains a Kuratowski constraint for each subdivision of K 5 or K 3,3 in every planarization with respect to a crossing restricted set of crossings. It is clearly impractical to generate all these constraints in advance, hence they are incorporated into a branch-and-cut framework, and Kuratowski constraints are only generated when necessary. For integer solutions, the separation problem for Kuratowski constraints can be solved in linear time by finding a Kuratowski subdivision in the partial planarization of G [15], for fractional solutions the separation problem is solved heuristically using rounding techniques. Primal Heuristic. The state-of-the-art crossing minimization heuristic is the planarization approach which has been proposed by Batini et al. [1]. It first computes a planar subgraph of G, and then inserts the remaining edges one after the other. During the edge insertion step, crossings are replaced by dummy vertices, such that the final result is a planarization of G. Gutwenger and Mutzel compared various variants of the planarization approach in an experimental study [8], including further permutation and postprocessing strategies. In our implementation, we use the heuristic which achieved the best quality in this study. Preprocessing. The traditional method of preprocessing for crossing minimization is to run the algorithm on each block, i.e., 2-connected component, of G separately, and to merge chains of edges, i.e., edges connected by nodes with degree two, into single edges. In [7], Gutwenger and Chimani presented a linear-time reduction scheme based on SPQR-trees [6], which can further simplify these blocks. The resulting non-planar core may contain edges with integer weights. It is easy to see, that we can extend the ILP given above to support these weights by simply modifying the edges coefficients in the objective function. This reduction method is known to roughly halve the graphs size on average for the Rome library which we use in our experimental study. This is an improvement of about 2/3 over the traditional preprocessing and was therefore used in all experiments given below. 3 Column Generation Schemes The main idea of column generation is to start solving a problem only with a relatively small subset of the ILP s variables. During the branch-and-cut computation we can decide which variables might have to be added in order to improve the solution. This idea is based on the observation, that in many problems there are a lot of unused variables which are 0 during the whole computation. Identifying and excluding superfluous variables leads to smaller ILPs and LP relaxations, thus improving the overall running time. Furthermore, the branch tree will be smaller, since there are less variables to branch on.

4 When analyzing the ILP presented above, we can clearly see that the full expansion of the graph, in order to guarantee the existence of a crossing restricted drawing that realizes the crossing number, will be unnecessary in most cases. Hence we can try to start with variables only for the first segment of each chain, adding additional segment-variables later when necessary. 3.1 Algebraic Pricing The standard idea of column generation is called pricing. Thereby we test all variables not yet in the ILP by computing their reduced costs. Let c be the vector of coefficients the variable v would have in the ILP s constraints (i.e., rows), and let d be the vector of the dual variables of the current solution. The reduced cost of v is simply defined as the scalar product of c and d. Based on the sign of this value we can decide whether it could be necessary to add this variable to the ILP or if the addition of this variable cannot lead to better solutions. This operation may give false positives, hence we try to add the variables with the smallest reduced costs first. Their addition can lead to rejecting other variables which had a negative reduced cost before. Due to the structure of our specific problem, we actually do not have to test all variables not introduced yet. It suffices to test variables which extend the currently introduced edge segments. More technically, this means that we start with a quadratic number of variables: for all pairwise disjoint and non-adjacent edges e and f, we label one of their expanded segments with e 0 and f 0, and add a variable representing a crossing x[e 0, f 0 ] between these segments. In the pricing step we have only to consider variables that are a convex extension of the existing variables. We can add variables for a segment left or right to the segment for which a variable already exists, leading to positive and negative indices. In the first iteration we will have to test the variables x[e ±1, f 0 ] and x[e 0, f ±1 ]. Figure 1 shows the situation at some later iteration step: The indices of segments of e are plotted on the horizontal axis, the indices of the segments of f on the vertical. The gray rectangles mark included variables; the circles denote potential new variables, whereby the crosses mark variables which, although adjacent to the current set of realized variables, are not tested, since they would extend the variable set in a way that is not orthogonally convex. Note that the orthogonally convex extensions are sufficient, and crossed out variables can be realized by first extending via the variables in between. 3.2 Combinatorial Column Generation Our second column generation approach, which turns out to be much more effective, works as follows: As for algebraic pricing, we start with a variable set of quadratic size, representing possible crossings between the first segments of the edges. Based on the fact that the graph extension was necessary only for efficient realizability testing, we will drop the CR-constraints for all these variables. For all additional segments associated to variables added throughout the calculation, the CR-constraints will be included into the ILP.

5 Fig. 1. Algebraic Pricing. Convex extension of the variable set included in the ILP Now we will modify the standard framework of calculation in a node in the branch-and-bound tree: the traditional order is to first solve the LP relaxation of the current (sub)problem, apply cutting planes afterwards, and finally perform the pricing step. In our scheme, we flip the cutting-plane and the column generation steps for reasons explained in the following see [2] for details. When solving the initial ILP, we may find solutions that would violate the CR-constraints which we did not add. Such solutions cannot be interpreted as a (partial) planarization, hence our cutting plane approach would fail. Our column generation scheme simply checks all these CR-constraints not in the ILP. Let e 0 be a segment whose CR-constraint would be violated by the variables x[e 0, f (1) i 1 ], x[e 0, f (2) i 2 ],..., x[e 0, f (k) i k ]. Intuitively this means that the segment e 0 is crossed by k different segments, namely f (1) i 1,..., f (k) i k, but the order of these crossings in unknown. For all y = 1,..., k, let j y be the largest index for which the variable x[e jy, f (y) i y ] is already in the ILP. We will then simply add the variables x[e jy+1, f (y) i y ], for all y. The trick is to reduce the objective function coefficient by some small ɛ for each variable which does not represent a crossing between two segments with index 0. Thereby we ensure that instead of putting all these edges on the first segment, at least one of these segments will be routed via the newly added segment since it is cheaper. If necessary, the LP relaxation is then also able to shift the already existing crossings on the other segment of e 0 s edge to higher indices. If, at some point, j y + 1 s with s being the maximum number of segments of the corresponding edge we will not introduce a new variable but add the CR-constraint corresponding to e 0 to the ILP instead. Note that this situation can happen, due to the fact that the ILP only tries to find a solution strictly better then the one given by the upper bound, and hence the maximum expansion of an edge might be one segment less than actually necessary for the realization of a crossing minimal drawing. See Figure 2 for an example of the presented column generation scheme: consider an integer solution by the LP relaxation to be interpreted as shown on the left: There are two segments crossing the first segment of the horizontal

6 f (1) 2 f (1) 2 f (1) 1 f (2) 2 f (1) 1 f (2) 2 e e 1 e 2 f (1) 0 f (2) 1 u e 0 e 1 e 2 0 (2) f 0 f (2) 0 Fig. 2. Combinatorial Column Generation. Left: implicit CR-constraint is violated; Right: by adding additional variables and reducing the cost on the additional segment, the situation is resolved. edge e. Since this violates the CR-constraint of that segment, we add the variables x[e 0, f (1) 0 ] and x[e 0, f (2) 1 ], such that a solution as given on the right can be computed. While the first solution had an objective value of = 2, the new solution is ɛ = 2 ɛ, and will therefore be preferred by the LP solver. Whenever the column generation terminates without adding new variables and therefore not forcing us to resolve the LP relaxation we can interpret the solution for our traditional cutting scheme. Note that whenever no implicit CR-constraint is violated by the LP-relaxation, it is obvious that the variables currently not in the ILP are not necessary for the optimal solution. Hence the optimal objective value on the restricted variable set is at least as small as on the fully expanded ILP, which suffices to proof the optimality of the induced solutions. 4 Experimental Comparison We implemented the described algorithms as part of the open-source C++ library Open Graph Drawing Framework OGDF [11]. We use the free branchand-cut-and-price framework ABACUS [9], in conjunction with the commercial optimization library CPLEX (version 9). The tests were performed on a single AMD Opteron CPU with 2.4 GHz and 32 GB RAM shared between 4 CPUs. As it turned out, the memory consumption seldomly exceeded 1 GB. For the following experiments, we used the Rome library, introduced in [5], as a benchmark set of graphs. It has already been used for various experimental studies, including [8] where different crossing heuristics have been compared. The set contains 11, 389 graphs that consist of 10 to 100 nodes and 9 to 158 edges. These graphs were generated from a core set of 112 real life graphs used in database design and software engineering applications. Due to the complexity of the crossing minimization problem, we restricted ourselves to the graphs with up to 75 nodes for the column generation comparison; combinatorial column generation (CCG) was then used to compute the graphs with up to 90 nodes. For each computation scheme, we applied a 30

7 Fig. 3. Percentage of graphs solved. minute time limit per instance. In the following diagrams, we will use AP as a shorthand for algebraic pricing, and F as a shorthand for the full ILP, without any column generation scheme. 4.1 Comparison between Column Generation Schemes Figure 3 shows the percentage of graphs for which the exact crossing number was computed. The size of the circles denotes the number of graphs per node count. While the full ILP could only solve a third of the large graphs, CCG could still solve about 50%. Note that even after only 5 minutes, CCG could already solve more graphs than F and AP after 30. Furthermore, there was no instance which could be solved by either F or AP, but remained unsolved by CCG. As it turns out, the number of edges in the non-planar core is much more influential than the number of nodes in the original graph. Figure 4 shows the relationship between those two properties: while we can clearly see a correlation, the variance is quite high. When we look at Figure 5 we can see the clear dependence of a successful computation on the number of core edges. Based on the primal heuristic, we know that the Rome library consists of many graphs with a crossing number of at most 1; we call these graphs trivial; in fact, nearly all graphs with up to 35 nodes are trivial. Since they are of no interest for our branch-and-cut algorithm, we restrict ourselves to the non-trivial graphs. Also note that for the following observations on the number of required variables, we only consider the graphs which were solved to provable optimality. Since CCG was able solve many more instances than AP, we will also consider the common set, which is the set of instances solved by AP and therefore also by CCG. Figure 6 shows the number of variables used by AP and CCG, relative to the full (potential) variable set. While CCG needed more variables on average for all the graphs it could solve, it used less variables than AP, when compared on the

8 Fig. 4. Correlation between number of nodes and number of core edges. Fig. 5. Percentage of graphs solved. common set. This shows that CCG was able to solve graphs which required a larger variable set, whereby AP was too slow to tackle such graphs successfully. Figure 7 shows the actual numbers of generated variables (compared on the common set). Note that the number of variables generated by CCG stays very close to the number of the initially generated variables. While AP could only solve graphs with roughly potential variables on average, a similar statistic for all the graphs solved by CCG would show an average of between and potential variables. According to the above statistics, it is obvious that the running time of CCG is superior to AP s, and that both are more efficient than no column generation at all. Yet, it is impressive that over the set of instances solved by F even the maximum running time of CCG is always far below F s average. When comparing AP to CCG on the common set, we see that AP s average is about equal to CCG s

9 Fig.6. Number of variables generated, relative to the full variable set. Fig. 7. Number of variables generated. maximum time. The average running time of CCG over all successfully solved graphs is under 5 minutes. 4.2 Comparison with Heuristics In order to evaluate the quality of crossing minimization heuristics, we compared the heuristic to the final upper bound of CCG after the 30 minutes time limit for each graph in the benchmark set. Figure 8 evaluates the quality of this upper bound by comparing it to the corresponding lower bound. The size of the circles reflects the number of graphs at the data point. The main diagonal shows the graphs solved to proven optimality; virtually all graphs with crossing number up to 7 are solved optimally, and even 5 graphs with crossing number 12 could be solved. In the following, we compare the heuristic solution to this upper bound, which we denote by BUP.

10 Fig. 8. Final lower bound compared to final upper bound. Fig. 9. Average number of crossings achieved by heuristic and CCG. Figure 9 shows the average number of crossings with respect to the number of nodes for both the heuristic and BUP. It turns out that the heuristic solution is very close to BUP. For graphs with up to 45 nodes, we know that CCG could solve almost all instances to optimality, so we can conclude that the heuristic performs very well on these instances. The dashed line shows the relative improvement achieved by CCG; for graphs with up to 45 nodes, this is about 2%. The small circles in the lower part of the diagram give the maximal absolute improvement over the heuristic; the largest improvement was 8 crossings achieved for a graph with 79 nodes. We further observed that the heuristic solution is optimal for virtually all instances with up to 3 crossings, and that the absolute deviation from the upper bound of CCG grows linear for graphs with up to 12 crossings by about 1/12 per crossings. We consider the absolute improvement achieved by CCG in more detail. Figure 10 shows the percentage of graphs that deviate by 1,2,... crossings from

11 Fig. 10. Absolute improvement of CCG by number of nodes. Fig. 11. Absolute improvement of CCG by number of crossings. the heuristic solution with respect to the number of nodes. It is interesting to consider the same statistic with respect to the value of the heuristic solution (see Figure 11). For graphs with up to 3 crossings, the heuristic solves nearly all instances to optimality. Between 10 and 16 crossings, the heuristic solution can be improved for about half of the graphs. For larger crossing numbers, the statistic is not useful, since CCG timed out early. 5 Conclusion As our experiments show, the algebraic pricing approach is clearly inferior to the combinatorial column generation. The former suffers from the high degree of redundancy in the full ILP and needs to add much too many variables in order to proof optimality. Combinatorial column generation on the other hand, gains a vast improvement compared to solving the ILP over the full variable set.

12 The second main finding is that the currently best known heuristics are very good in practice and solve many instances to their optimal value. Yet we can see that there is still room for improvement when the crossing numbers get higher. Acknowledgement We want to thank Michael Jünger and Christoph Buchheim for many useful discussions on branch-and-cut-and-price theory. References 1. C. Batini, M. Talamo, and R. Tamassia. Computer aided layout of entityrelationship diagrams. Journal of Systems and Software, 4: , C. Buchheim, M. Chimani, D. Ebner, C. Gutwenger, M. Jünger, G. W. Klau, P. Mutzel, and R. Weiskircher. A branch-and-cut approach to the crossing number problem. Discrete Optimization. Submitted for publication. 3. C. Buchheim, D. Ebner, M. Jünger, G. W. Klau, P. Mutzel, and R. Weiskircher. Exact crossing minimization. In P. Eades and P. Healy, editors, Graph Drawing 2005, volume 3843 of LNCS. Springer-Verlag. 4. G. B. Dantzig and P. Wolfe. Decomposition principle for linear programs. Operations Research, 8: , G. Di Battista, A. Garg, G. Liotta, R. Tamassia, E. Tassinari, and F. Vargiu. An experimental comparison of four graph drawing algorithms. Computational Geometry: Theory and Applications, 7(5-6): , G. Di Battista and R. Tamassia. On-line maintanance of triconnected components with SPQR-trees. Algorithmica, 15: , C. Gutwenger and M. Chimani. Non-planar core reduction of graphs. In P. Eades and P. Healy, editors, Graph Drawing 2005, volume 3843 of LNCS. Springer-Verlag. 8. C. Gutwenger and P. Mutzel. An experimental study of crossing minimization heuristics. In G. Liotta, editor, Graph Drawing 2003, volume 2912 of LNCS, pages Springer-Verlag, M. Jünger and S. Thienel. The ABACUS system for branch-and-cut-and-pricealgorithms in integer programming and combinatorial optimization. Software: Practice& Experience, 30(11): , J. Kratochvíl. String graphs. II.: Recognizing string graphs is NP-hard. Journal of Combinatorial Theory, Series B, 52(1):67 78, OGDF Open Graph Drawing Framework. University of Dortmund, Chair of Algorithm Engineering and Systems Analysis. Web site under construction. 12. H. C. Purchase. Which aesthetic has the greatest effect on human understanding? In G. Di Battista, editor, Graph Drawing 97, volume 1353 of LNCS, pages Springer-Verlag, P. Turán. A note of welcome. Journal of Graph Theory, 1:7 9, Imrich Vrto. Crossing numbers of graphs: A bibliography. sk/~imrich/crobib.pdf. 15. S. G. Williamson. Depth-first search and Kuratowski subgraphs. Journal of the ACM, 31(4): , 1984.