Combinatorial Optimization Top Ten List

Transcription

1 Combinatorial Optimization Top Ten List Discrete Mathematics 2000 as selected by William R. Pulleyblank T. J. Watson Research Center IBM Corporation Yorktown Heights, NY

2 Euler's Theorem 1736 Theorem: A graph has a Euler tour if and only if there are zero or two nodes of odd degree

3 L. Euler, Solutio Problematis ad Geometriam Situs Pertinentis, Commentarii Academiae Scientiarum Imperalis Petropolitanae 8 (1736) " As far as the problem of the seven bridges of Koenigsberg is concerned, it can be solved by making an exhaustive list of all possible routes, and then finding whether or not any route satisfies the conditions of the problem. Because of the number of possibilities, this method of solution would be too difficult and laborious, and in other problems with more bridges, it would be impossible..."

4 " So whatever arrangement may be proposed, one can easily determine whether or not a journey can be made, crossing each bridge once, by the following rules: If there are more than two areas to which an odd number of bridges lead, then such a journey is impossible. If, however, the number of bridges is odd for exactly two areas, then the journey is possible if it starts in either of these areas. If, finally, there are no areas to which an odd number of bridges leads, then the required journey can be accomplished starting from any area. "

5 Algorithm first published 137 years later C. Hierholzer, Ueber die Moeglichkeit, einen Linienzug ohne Wiederholung und ohne Unterbrechnung zu umfahren, Mathematische Annalen 6 (1873), start at an odd degree node; - walk until you are stuck * what's left has no odd degree nodes - solve the remaining problem and splice in the answer. see N.L. Biggs, E.K.Lloyd, R.J.Wilson, Graph Theory , Oxford University Press, 1976.

6 Chinese Postman problem Guan Meigu, Graphic Programming using odd and even points, Chinese Math. 1 (1962) The streets were given lengths, and any (even) number of odd degree intersections was permitted. The problem now became: Find the shortest tour traversing each street at least once. Solved polynomially in J. Edmonds and E.L. Johnson, Matching, Euler Tours and the Chinese Postman, Mathematical Programming 5 (1973) Many extensions - one in particular - P.D. Seymour, The matroids with the max-flow min-cut property, J. Combin. Theory Ser. B 23 (1977)

7 Herbert J. Ryser, Combinatorial Mathematics, The Mathematical Association of America, " Combinatorial mathematics cuts across the many subdivisions of mathematics, and this makes a formal definition difficult.... Two general types of problems appear throughout the literature. In the first the existence of the prescribed configuration is in doubt, and the study attempts to settle this issue. These we call existence problems. In the second the existence of the configuration is known, and the study attempts to determine the number of configurations or the classification of these configurations according to types. These we call enumeration problems. " March Second Louisiana Conference on Combinatorics, Graph Theory and Computing, LSU, Baton Rouge, Ryser gave an invited address and included a third type of problem: How do you efficiently find or construct the configurations?

8 Max-flow Min-cut Theorem 1956 Given a directed graph G = (V, E),with finite arc capacities ( u j : j? E), and designated source and sink nodes s and z, the maximum value of an s - z flow is the minimum capacity of an s - z cut. L.R. Ford Jr. and D.R. Fulkerson, Maximal flow through a network, Canadian J. of Math. 8 (1956) A. Kotzig, "Suvislost' a Pravidelina Suvislost Konecnych Grafor", Bratislava: Vysoka Skola Economicka (1956). P. Elias, A. Feinstein and C.E. Shannon, A note on the maximum flow through a network, IRE Transactions on Information Theory IT2 (1956)

9 Equivalent Theorems: Menger's Theorem (1927) (edge or node disjoint paths) Dilworth's Theorem (1950) (antichains in partially ordered sets) P. Hall's Theorem (1935) (Systems of Discrete Representatives) Koenig's Theorem (1931) (maximum matching and minimum cover in a bipartite graph) Hoffman's Circulation Theorem (1960) Gale's Supply Demand Flow (1957)... These were all linked by total unimodularity - Hoffman and Kruskal (1956) Write down a "natural" linear programming formulation, then there will be an integral optimal solution...

10 and some extensions... Flow augmenting paths Dinit (1970) and Edmonds-Karp (1972) - shortest flow augmenting paths give polynomial performance Preflow-push algorithms Generalized flows Multicommodity flows... Seymour's Theorem (1980): [A good characterization of total unimodularity] - a matrix is totally unimodular if and only if it can be composed from network matrices, their transposes, and two exceptions. P.D. Seymour, Decomposition of regular matroids, J. of Combin. Theory Ser. B 28 (1980)

11 Nonbipartite matching algorithm and polyhedral characterization Jack Edmonds, Paths, trees and flowers, Canadian J. of Maths. 17 (1965) There exists a polynomially bounded algorithm which finds a maximum matching in a nonbipartite graph. Generalized the Hungarian method of Kuhn 1955 and Munkres 1957 for bipartite graphs Alternating paths not enough You shrink the odd sets!

12 Paths, trees and flowers (Edmonds 1965) - 2. Digression... I am claiming, as a mathematical result, the existence of a good algorithm for finding a maximum cardinality matching in a graph.... an algorithm whose difficulty grows only algebraically with the size of the graph......for practical purposes the difference between algebraic and exponential order is often more crucial than the difference between finite and non-finite. also, Cobham (1965)

13 Foundation of Polyhedral Combinatorics The convex hull of the incidence vectors x of the matchings of a nonbipartite graph is defined by x j 0 for all j? E, 5 (x : j?@(v)) j 1 for all v?v, 5 (x : j?e(s)) j ( S -1)/2 for all odd cardinality S <V. J. Edmonds, Maximum matching and a polyhedron with 0-1 vertices, J. Res. Nat. Bureau of Standards B 69 (1965)

14 Extensions: bidirected capacitated weighted matchings - includes network flows, edge covers and natchings parity constraints - includes T-joins, cuts and postman problems stable sets in claw free graphs path systems

15 Matroid Intersection Theorem 1970 Matroids had been introduced by Hassler Whitney in 1935 as an abstraction of linear independence. A family F of (so called independent) subsets of a set E forms a matroid if 1. all subsets of an independent set are independent; 2. for any A < E, all maximal independent subsets of A have the same cardinality - denoted by r(a). (If 1. is satisfied, then F is called an independence system.) Examples - edge sets of forests in a graph, linearly independent sets of columns of a matrix, systems of distinct representatives of a set family,...

16 Matroid Optimization Problem: Let each element e? E have a weight w e. Find an independent set I for which 5 (w e : e? I) is maximized. Theorem (Rado 1957, Edmonds 1970): An independence system is a matroid if and only if the greedy algorithm solves the Matroid Optimization Problem, for any vector w of weights. Theorem (Edmonds 1970): The incidence vectors x of the independent sets of a matroid are the vertices of the polyhedron defined by x e 0 for all e? E, 5(x e : e? A) r(a) for all A < E.

17 Matroid Intersection Theorem (Edmonds(1970): Let M 1 = (E, F 1 ) and M 2 = (E, F 2 ) be matroids on a set E. Then maximum I = minimum {r 1 (S)+r 2 (E\S)}. I?F 1!F 2 S < E

18 Weighted Intersection Theorem (Edmonds 1970): Let M 1 = (E, F 1 ) and M 2 = (E, F 2 ) be matroids on a set E. Let each element e? E have weight w e. Then x is the incidence vector of a maximum weight set independent in both matroids if and only if x is a basic optimal solution to the linear program maximize wx subject to x e 0 for all e? E, 5(x e : e? A) r 1 (A) for all A < E, 5(x e : e? A) r 2 (A) for all A < E.

19 Consequences: Koenig's Theorem for maximum matchings in bipartite graphs; Edmonds' Matroid Partition Theorem Perfect's Theorem on Systems of Common Representatives Extensions: Polymatroids and polymatroid intersection Submodular and supermodular functions Unification with nonbipartite matching: Matroid parity problem, or matroid matching - cardinality case for linear matroids (Lovasz 1980)

20 Cook's Theorem 1971 If there exists a polynomially bounded algorithm for satisfiability, then if a class of decision problems has a polynomial length certificate for a "YES" answer, then there is a polynomial time algorithm to solve any decision problem in the class. S.A. Cook, The complexity of theorem-proving procedures, Proc. 3rd Ann. ACM Symp. on Theory of Computing, Association for Computing Machinery, New York (1971). NP - problems having short "YES" certificates - polynomially solvable on nondeterministic Turing machines P - problems solvable in polynomial time Cook's Theorem: SATISFIABILITY is NP-complete - a polynomial algorithm for SATISFIABILITY implies P = NP is P = NP?

21 R. M. Karp 1972 Reducibility among combinatorial problems in R.E. Miller and J.W. Thatcher (eds.), Complexity of Computer Computations, Plenum Press, New York, Showed that many well known problems were NP-complete - Hamiltonian circuit; large independent set of nodes; scheduling problems; 3-d matching... M.R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, Freeman, San Francisco, 1979 * More than 300 NP-complete problems

22 Conventional Wisdom: P CNP NP. Garey and Johnson - 12 open problems: 1. Graph Isomorphism open 2. Subgraph homeomorphism for a fixed graph P 3. Graph genus NP-complete 4. Chordal graph completion NP-complete 5. Chromatic Index NP-complete 6. Spanning tree parity P 7. Partial Order Dimension NP-complete 8. Precedence constrained 3-processor sched. open 9. Linear Programming P 10. Total Unimodularity P 11. Composite number open 12. Minimum length triangulation open

23 Dantzig, Fulkerson and Johnson - solution of the 49 city problem G. Dantzig, R. Fulkerson and S. Johnson, "Solution of a large-scale traveling salesman problem", Operations Research 2, (1954), Optimal solution of a 49 city traveling salesman problem using cutting planes... and the race was on

24 Largest solved TSP problems Size Researchers Year 49 Nodes Dantzig, Fulkerson, and Johnson Nodes Held and Karp Nodes Helbig, Hansen, and Krarup Nodes Camerini, Fratta, and Maffiol Nodes Groetschel Nodes Padberg and Hong Nodes Crowder and Padberg Nodes Padberg and Rinaldi Nodes Groetschel and Holland Nodes Padberg and Rinaldi Nodes Padberg and Rinaldi Nodes Applegate, Bixby, Chvatal, and Cook Nodes Applegate, Bixby, Chvatal, and Cook Nodes Applegate, Bixby, Chvatal, and Cook Nodes Applegate, Bixby, Chvatal, and Cook 1998

25 number of cities Size of largest TSP solved to provable optimality

26 Held and Karp's Subgradient relaxation of the TSP , 71 Start with a minimal integer programming formulation of the TSP: minimize 5 d ij x ij subject to x ij? {0,1} for all i, j 5 xij = 2 for all j i 5 x ij S - 1 i, j?s for all LCS V.

27 - remove the degree constraints! minimize 5 d ij x ij 5 xij - 5p j (2 - ) j i subject to x ij? {0,1} for all i, j 5 xij S - 1 i, j?s for all LCS V. M. Held and R.M. Karp, The traveling salesman problem and minimum spanning trees, Operations Research 18 (1970) , The traveling salesman problem and minimum spanning trees: Part II, Mathematical Programming 1 (1971) 6-25.

28 The Volume Algorithm Anbil and Barahona (1998) showed how subgradient methods could be extended to yield a solution to the linear program which is "almost" feasible and "almost" optimal. Essentially, they add a weighted average of the (infeasible) primal solutions produced by a subgradient method. this followed earlier work on approximate "almost feasible" by Plotkin, Shmoys, and Tardos; Grigoriadis and Khachiyan F. Barahona and R. Anbil, The volume algorithm: producing primal solutions with a subgradient method, Mathematical Programming, Ser. A 87, 2000,

29 The Volume Algorithm - producing good primal solutions.

30 Lagrangian Methods and The Volume Algorithm (continued) # Rows # Columns # Elements Primal- Dual Interior Dual Simplex Volume plus Dual Simplex

31 Lin Kernighan Algorithm 1973 Given a feasible solution to a Traveling Salesman problem, improve it by local exchanges Local search: alternatively removes and adds edges to a tour until a better tour is found or it gives up. Start from a random tour Chained Lin Kernighan (Martin and Otto 1996) - random 4 - Interchange o o o o o o o o

32 D.S. Johnson, J.L. Bentley, L.A. McGeoch and E.E. Rothberg 1997 L-K produces tours 2% over opt. Chained L-K produces tours 1% over opt. Can be implemented to run very quickly 10,000 node Euclidean TSP on average workstation seconds L-K developed into local search activity - Simulated Annealing, Tabu Search, Genetic Algorithms,...

33 Optimization = Separation 1980, 1981 The Ellipsoid Algorithm (L.G. Khachiyan 1979, 1980) operated by testing the centre x of an ellipsoid for membership in a polyhedron P. If x is not in P, this is shown by producing a violated defining inequality. Then the ellipsoid is updated. The entire process runs in polynomial time. Theorem (Karp and Papadimitriou 1980, Padberg and Rao 1981, Groetschel, Lovasz, Schrijver 1981): Optimization and Separation are polynomially equivalent

34 Applications: 1. Submodular function minimization (Groetschel, Lovasz, Schrijver 1981) 2. Maximum weight stable sets in perfect graphs (Groetschel, Lovasz, Schrijver 1981) ** 3. Separation for odd cuts (Padberg, Rao 1982) 4. Separation for matroid polyhedra (Cunningham 1984)

35 The Shannon capacity of the _ pentagon is L. Lovasz, On the Shannon capacity of a graph, IEEE Transactions on Information Theory IT-25 (1979) 1-7. =(G) is the stability number - the maximum number of pairwise nonadjacent nodes. Let G be a graph whose nodes are letters in an alphabet and where adjacency means that letters can be confused. The maximum number of 1 letter messages that can be sent without confusion is =(G). Let =(G k ) denote the maximum number of k-letter messages which can be sent without danger of confusion. (Confusion occurs if two words are identical or confusable in all positions.)

36 C 5 - the pentagon a =(C 5 ) = 2 e b =(C 52 ) = 5 e d d a c c b aa bc ce db ed

37 The Shannon Capacity, 3(G),is defined by k k 3(G) = sup =(G k ) = lim =(G k ). J 8kk _ _ Lovasz(1979) introduced the concept of an orthonormal representation of a graph G=(V,E) : a set of V vectors u i such that u i =1 for all i and u it u j = 0 for all pairs i,j of nonadjacent vertices.

38 Orthonormal representation constraint: Let u be an orthonormal representation of G=(V,E). For any c? IR n with c = 1, the incidence vector x of any stable set S satisfies IR V 5 (c T u i ) 2 x i 1. i? V TH(G) = { x? : x 0 and x satisfies the orthonormal representation constraints for all c? }. IR n Note: TH(G) is not polyhedral (G) = max{1 T x : x? TH(G)}.

39 Theorem: (Lovasz 1979): 3(G) (G) plus, properties of (G) and how to compute and estimate for a general graph G. _ Theorem (Lovasz 1979): (C 5 ) (C 5 ) (C 5 ) 5. and where did it lead?

40 Groetschel, Lovasz, Schrijver (1981, 1988): (G, w) = max{w T x : x? TH(G)}. Theorem: TH(G) is a polytope if and only if G is a perfect graph. Theorem: TH(G) = STAB(G) if and only if G is perfect. Theorem: The weak optimization problem for TH(G) can be solved in polynolmial time. Corollary: Both the weighted stable set problem and weighted clique problem for perfect graphs can be solved in polynomial time. but we still cannot recognize a perfect graph in polynomial time.

41 .878 Approximation Algorithm for the max cut problem 1994 Given an undirected graph G=(V,E) with edge weights w e : e? E, partition V into sets V 1 and V 2 so that the weights of the edges between the two sets is maximized. NP-hard, Sahni and Gonzalez (1976) gave a 1/2 approximation algorithm.

42 M.X. Goemans and D.P. Williamson, approximation algorithm for MAX-CUT and MAX-2SAT, Proceedings of the 26th Annual ACM Symposium on the Theory of Computing (1994) Formulated the problem as a quadratic {+1, -1} problem; Relaxed it to a semidefinite programming problem; Solved polynomially, using GLS 81, 87. Used randomized rounding to get good expected solution, Derandomized

43 Top Ten List 1. Euler's Theorem Max-flow Min-cut Theorem Edmonds' matching algorithm & polyhedron Edmonds' matroid intersection Cook's Theorem Dantzig Fulkerson and Johnson 49 cities 1954 Held & Karp relaxation of the TSP 1970, Lin Kernighan local search for the TSP Optimization = Separation Lovasz's Shannon Capacity of pentagon Goemans Williamson.878 approx for MAX CUT 1994

44 a reference...