CONSTRAINT AND INTEGER PROGRAMMING

Transcription

1 CONSTRAINT AND INTEGER PROGRAMMING

2 OPERATIONS RESEARCH/COMPUTER SCIENCE INTERFACES SERIES Professor Ramesh Sharda Oklahoma State University Series Editors Prof. Dr. Stefan VoG Universitdt Hamburg Other published titles in the series: Greenberg, Harvey 1.1 A Computer-Assisted Analysis System for Mathematical Programming Models and Solutions: A User 's Guidefor ANALYZE Greenberg, Harvey 1.1 Modeling by Object-Driven Linear Elemental Relations: A Users Guide for MODLER Brown, Donald/Scherer, William T. / Intelligent Scheduling Systems Nash, Stephen G.lSofer, Ariela I The Impact ofemerging Technologies on Computer Science & Operations Research Barth, Peter I Logic-Based 0-1 Constraint Programming Jones, Christopher V. I Visualization and Optimization Barr, Richard S./ Helgason, Richard V./ Kennington, Jeffery L. I Interfaces in Computer Science & Operations Research: Advan ces in Metaheuristics, Optimization, & Stochastic Modeling Technologies Ellacott, Stephen W.I Mason, John C.I Anderson, lain J.I Mathematics ofneural Networks : Models, Algorithms & Applications Woodruff, David L. I Advances in Computational & Stocha stic Optimization, Logic Programming, and Heuristi c Search Klein, Robert I Scheduling ofresource-constrained Projects Bierwirth, Christian I Adaptive Search and the Management oflogistics Systems Laguna, Manuel I Gonzalez-Velarde, Jose Luis I Computing Tools for Modeling, Optimization and Simulation Stilman, Boris I Linguistic Geometry: From Search to Construction Sakawa, Masatoshi I Genetic Algorithms and Fuzzy Multiobjective Optimization Ribeiro, Celso C./ Hansen, Pierre I Essays and Surveys in Metaheuristics Holsapple. Clydel Jacob, Varghese I Rao, H. R. I BUSINESS MODELLING: Multidisciplinary Approaches - Economics. Operational and Information Systems Perspectives Sleezer, Catherine M.I Wentling, Tim L.I Cude, Roger L. I HUMAN RESOURCE DEVELOPMENTAND INFORMATION TECHNOLOGY: Making Global Connections Voll, Stefan, Woodruff, David I Optimi zation Software Class Libraries Upadhyaya et all MOBILE COMPUTING: Implementing Pervasive information and Communications Technologies Reeves, Colin & Rowe, Jonathan! GENETIC ALGORITHMS-Principles and Perspectives: A Guide to GA Theory Bhargava, Hemant K. & Ye, Nong / COMPUTATIONAL MODELING AND PROBLEM SOLVING IN THE NETWORKED WORLD: Interfaces in Computer Science & Operations Research Woodruff, David L./ NETWORK INTERDICTIONAND STOCHASTIC INTEGER PRO GRAMMING Anandalingam, G. & Raghavan, S. Raghul TELECOMMUNICATIONS NETWORK DESIGN AND MANAGEMENT Laguna, Manuel & Marte, Rafaell SCATTER SEARCH: Methodology and Implementations in C Gosavi, Abhijit/SIMULATION BASED OPTIMIZATION: Parametric Optimization Techniques and Reinforcement Learning Koutsoukis, Nikitas-Spiros & Mitra, Gautaml DECiSION MODELLING AND INFORMATiON SYSTEMS: The Information Value Chain

3 CONSTRAINT AND INTEGER PROGRAMMING Toward a Unified Methodology Edited by MICHELA MILANO University of Bologna " ~. Springer Science+Business Media, LLC

4 Library of Congress Cataloging-in-Publication Data A C.I.P. Catalogue record for this book is available from the Library of Congress. CONSTRAINT AND INTEGER PROGRAMMING: Toward a Unified Methodology, edited by Michela Milano ISBN ISBN (ebook) DOI / Copyright 2004 by Springer Science+Business Media New York Originally published by Kluwer Academic Publishers in 2004 Softcover reprint of the hardcover 1 st edition 2004 AII rights reserved. No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording, or otherwise, without the written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work.

5 Contents List of Figures List of Tables Preface Contributing Authors Foreword John Hooker 1 Constraint and Integer Programming MichelaMilano and Michael Trick Introduction CP(FD) Basic Concepts 2.1 Modeling 2.2 Structure of a CP program 2.3 Solving Constraint Propagation Search Branch and Bound in CP 2.4 An example : the car sequencing problem Integer Linear Programming Basic Concepts 3.1 Modeling Logical Constraints Resource Constraints Routing Constraints Alternative Formulations 3.2 Solving Relaxations Branch and Bound Improving the Effectiveness 3.3 An example: the car sequencing problem revisited Incomplete search strategies Conclusion xiii xvii xix xxiii xxvii

6 vi References CONSTRAINT AND INTEGER PROGRAMMING 29 2 Two Generic Schemes for Efficient and Robust CooperativeAlgorithms 33 Emilie Dannaand Claude Le Pape 1 Introduction 34 2 Operations Research Algorithms and Constraint Programming 37 3 Operations Research Algorithms and Mixed Integer Programming 39 4 Constraint Programming and Mixed Integer Programming 40 5 Operations Research Algorithms and Local Search 44 6 Mixed Integer Programming and Local Search 45 7 Constraint Programming and Local Search 48 References 53 3 Branch-and-Infer: A Framework for Combining CP and IP 59 AlexanderBockmayr and Thomas Kasper Introduction Modeling in CP and IP An illustrating example: discrete tomography 3.1 Integer programming models 3.2 Constraint programming models Branch and Infer 4.1 Primitive and non-primitive constraints 4.2 Inferring primitive from non-primitive constraints 4.3 Search 4.4 Pruning the search tree Symbolic constraints in IP 5.1 Symbolic constraint abstractions 5.2 Extending expressiveness 5.3 Improving efficiency 5.4 Compositionality Example: Symbolic constraints for supply chain planning 6.1 Stock-resource constraint 6.2 Finite capacity resource 6.3 State resource 6.4 Combined example Summary References 4 Global Constraints and Filtering Algorithms Jean-Charles Regin 1 Introduction 2 Global Constraints

7 Contents Preliminaries 2.2 Definition and Advantages 2.3 Examples Filtering Algorithms 3.1 Algorithms Based on Constraints Addition 3.2 General Arc Consistency Filtering Algorithm Preliminaries A First Algorithm A better general algorithm Discussion and Example 3.3 Dedicated Filtering Algorithms Two Successful Filtering Algorithms 4.1 Preliminaries 4.2 The Alldifferent Constraint Consistency and Arc Consistency Complexity Some Results 4.3 The Global Cardinality Constraint Consistency and Arc Consistency Complexity Some results Global Constraints and Over-constrained Problems 5.1 Satisfiability Sum Constraint 5.2 Global Soft Constraints General Definitions of Cost Soft Alldifferent Constraint Quality offiltering Algorithms Discussion 7.1 Incomplete Algorithms and Fixed-Point Property 7.2 Closure 7.3 Power of a Filtering Algorithm Conclusion vii III References 5 Exploiting relaxations in CP Filippo Focacci, AndreaLodi and Michela Milano I Introduction and Motivation 2 Integer Linear Programming and Relaxations 2.1 Continuous Linear Relaxation 2.2 Structured Relaxations Surrogate Relaxation Lagrangean Relaxation 3 Integrating Relaxations in CP 3.1 Which relaxation to use 3.2 Which part ofthe problem Relaxation of global constraints 4 Relax to propagate

8 viii CONSTRAINTAND INTEGER PROGRAMMING 4.1 Mapping The Cost-Based Propagation Triggering Events Relax to guide the search Synchronous update ofthe oracle Select a branching object Select promising values Asynchronous update of the oracle Generation ofstatic heuristics Defining sub-problems and incomplete methods A case study: global optimization constraints for a Path constraint Global constraint architecture The Path constraint case An ILP model of the Path constraint Continuous (linear) relaxation The linear Assignment Problem The Minimum Spanning r-arborescence Adding cutting planes Lagrangean relaxation of cuts 162 References Hybrid Problem Solving in ECLiPSe 169 Farid Ajili and Mark G. Wallace Introduction Modelling Formalisms for Constraints and Mathematical Programming Requirements on Hybrid Modelling Formalisms ECLiPSe and Piecewise Linear Probing Integration ofconstraints and Operations Research Constraint Programming Operations Research Hybridization context Language Ingredients for Hybrid Solvers Conceptual Models and Design Models Context Creating a Design Model - Introduction and Example Information Passed between Solvers Combining Solvers and Search Engine Information Passed from Search Engine to Solvers Information Passed from Solvers to Search Engine Requirements on the Language used for Design Models Variable Attributes Control Requirements Infeasibility Detection ECLiPSe as a Platform for Building Hybrid Algorithms Modelling in ECLiPSe The Domain Solver: ic 183

9 Contents The linear solver interface: eplex 4.4 The repair solver 4.5 Attributed Variables and Demons in ECLiPSe Programming a Hybrid Search in ECLiPSe 5.1 An Illustrative Example 5.2 Problem Modelling 5.3 Hybrid Probe-based Algorithm The Algorithm Design Model Inference phase Probing phase Resource feasibility 5.4 Probing Strategies 5.5 Mixed Integer Programming based Probing 5.6 Linear Relaxation based Probing 5.7 Evaluation and Performance Analysis Setting up the benchmark instances Computational results Conclusion ix References 7 CP Based Branch-and-Price Kelly Easton, George Nemhauser and Michael Trick References Introduction Three Illustrative Examples 2.1 The Generalized Assignment Problem 2.2 The Traveling Tournament Problem 2.3 The Social Golfers Problem 2.4 Other Applications Implementation Issues 3.1 Set Partitioning Versus Set Covering 3.2 Initial Solution 3.3 Column Management 3.4 LP Relaxation 3.5 Branching 3.6 CP as a Subproblem Solver 3.7 Column Generation Heuristics 3.8 Combining Column and Row Generation 3.9 Parallel Implementation Issues Future Directions for CP Based Branch-and-Price 8 Randomized Backtrack Search Carla P. Gomes 1 Introduction

10 x CONSTRAINT AND INTEGER PROGRAMMING Randomization of Backtrack Search Methods Formal Models of Heavy-Tailed Behavior 3.1 Random Walk 3.2 Tree Search Model Heavy and Fat-Tailed Distributions 4.1 Heavy-Tailed Distributions Definition Visualization of Heavy-Tailed Behavior Estimation of Index of Stability (a) 4.2 Fat vs. Heavy-Tailed Distributions Heavy and Fat-Tailed Distributions in Backtrack Search 5.1 CSP Formulations 5.2 Mixed Integer Programming Formulations 5.3 Boolean Satisfiability Formulations 5.4 Graph Coloring Formulations 5.5 Discussion Restart Strategies 6.1 Elimination of Heavy-Tails 6.2 CutoffValue for Restart Strategy 6.3 Formal Results on Restarts 6.4 Restart Results on a Range of Problem Instances 6.5 Learning Dynamic Restart Strategies 6.6 Variants of Restart Strategies Portfolio Strategies 7.1 Portfolio Design 7.2 Portfolio Results 7.3 Variants of Portfolios Conclusions References 9 Local Search and Constraint Programming LS and CP illustrated on a transportation Problem Filippo Focacci, FrancoisLaburthe, Andrea Lodi Introduction A didactic transportation problem A CP approach for dtp 3.1 A CP model for dtp Basic model 3.2 Propagation Disjunctive Relations Linking trucks and bins Propagating costs 3.3 A redundant routing model Propagation Constructive Algorithms 4.1 Insertion algorithms 4.2 Greedy insertion

11 Contents Restricted Candidate Lists and GRASP 4.4 Discrepancy-based search LS as Post-Optimization 5.1 LS +constraint checks 5.2 Constraint checks within the neighborhood iteration 5.3 Freezing Fragments 5.4 CP models for the Neighborhoods Metaheuristics 6.1 Control strategies from metaheuristics 6.2 Managing several neighborhoods LS during construction 7.1 Single Route Optimization 7.2 Ejection Chains Conclusions xi References Open Perspectives 331 Mark Wallace, Yves Caseau and Jean-Francais Puget Motivations, Challenges and Applications Overview Challenges Focus on Hard to Solve Problems Global Constraints Scalability Limitations of Tree Search Limitations of Global "Soft" Constraints Problem Analysis vs. Resolution Multi-criteria problems Uncertain Problems Probabilistic Data Supporting the Problem Solving Process Software Engineering Issues Software Engineering in Support of Combinatorial Problem Solving Combinatorial Problem Solving in Support of Software Engineering Transforming Models to Algorithms Conceptual and Design Models Decompositions Transformations Separating Modeling for Performance Improvements Motivations and Examples Transforming Conceptual Models into Design Models Search Local and Constructive Search Combining Different Search Methods Concurrent Search Inference Global and Local Control of Inference 349

12 xii CONSTRAINT AND INTEGER PROGRAMMING Controlling the Local Inference Procedure Controlling Communication of Inferences 2.6 Symmetries New Techniques 3.1 Stochastic Optimisation 3.2 Overconstrained problems and robustness 3.3 User Support A Simple Environment for Solving LSICO Problems Automating Algorithm Development and Testing 3.4 Packaging New Application Areas 4.1 Computer-Aided decision analysis based on simulation 4.2 Cooperative Problem Solving 4.3 Interleaved Planning and Execution References Index

13 List of Figures 1.1 Interaction amongconstraints Cumulative constraintin discrete tomography Two- and three-dimensional diffn constraintin discrete tomography Primitive constraints in CP Primitive constraints in IP Architecture for constraint-based solving Inference in IP and CP Supply Chain Example An Assignment Timetable An example of a global cardinality constraint Rules of the ad-hoc filtering algorithm for the n-queens problem Linearization of all constraints in a single store Optimization component in each constraints Merging of two constraints Global constraintarchitecture Addition of cuts in optimization constraints The input scheduleand the PL cost function </> of an activity Results of running H(relax) and H(1ambda) on the four test classes Two different executions of a backtracksearch methods looking for an assignment to the variables A, B, C that satisfies the formula (A v -,B v -,C) 1\ (B v -'C) 1\ (A v C) Toppanel: symmetric randomwalk(to,ooo steps). Bottom panel: the heavy-tailed nature of a random walk contrasted with the exponential decay of the normaldistribution. 243

14 xiv CONSTRAINT AND INTEGER PROGRAMMING 8.3 Top panel: imbalanced tree model; P is the probability that the branching heuristic will make a wrong decision, and b is the branching factor (in this case, b = 2). This tree model has a finite mean and an infinite variance when Ilb 2 < P < lib; both the mean and the variance are infinite when p ~ lib. Bottom panel: example distributions the imbalanced and bounded imbalanced models, contrasted with the balanced model. Parameters: b = 2, n = 20, p = 0.5 and 0.75 (n is the number of variables) Quasigroup Completion Problem (QCP) Erratic behavior of mean cost of completing a quasi group (order 11, 30% pre-assignment) vs. stabilized behavior of mean for a standard distribution (gamma) The phenomenon of long tails with randomized complete search procedures (QCP instance, order 11, 30% preassignment) Left panel: log-log plot ofheavy-tailed behavior (QCP instance, order 11, 30% pre-assignment; uncensoreddata). Right panel: no heavy-tailed behavior Log-log plot of heavy-tailed behavior Standardized normal, Cauchy, and Levy densities Comparison of the fatness of tails: normal (1,1), lognormal(1,1), exponential(1), and paretotl,1) Comparison of the tail fatness: normal (l.i), lognormal(1,1), exponential(1), and pareto(1, 1) Log-log plot of the complement to one of the cumulative distribution of the runtime for the round-robin sports scheduling (12 teams; CSP formulation); (a) full distribution; (b) tail (X > 10,000) (a) Cost profiles for a logistics planning problems for depth-first and best-bound search strategies. (b) Hevaytailed behavior of depth-first search Log-log plot of heavy-tailed behavior for SAT formulations of (a) logistics planning, and (b) register allocation Log-log plot of the tail of the runtime distributions of an exact backtrack search coloring algorithm on three differentgraph topologies: random(lo!]2(p) = 0); smallworld (1092(P) = -4); and more structured (1092(P) = -8). P is the probability of randomly rewiring an edge of a regular graph. Size of the regular graph: 100 nodes and degree 8 (Walsh, 1999). 262

15 List offigures xv 8.16 Log-logplot of heavy-tailed behavior for the timetabling problem Left panel: heavy tail for quasi group completion with extensive propagation (CSP formulation, order 30, with 55% pre-asssignment; uncensored data). Right panel: comparison of Satz to Relsat on logistics planning (SAT formulations ) Survival function (1 - F(x» of the number of backtracks needed to solve different instances of model E with 20 variables and a domain size of 10. The parameter p captures the constrainedness of the instances; it is the probability of choosing one constraint (out of the total number of possible constraints). Two different regimes in the decay of the survival function can be identified: a heavy-tailed regime (curves with linear behavior) and a non-heavy-tailed regime Absence of heavy-tailed behavior for unsolvable QCP instances (CSP formulation) Left panel: comparison of runtime profiles for proving optimality of misc07 from MIPLIB. Right panel: absence of heavy-tailed behavior in proving optimality of misc Restarts : (a) Tail (of(1- F(x) as a function of the total number of backtracks for a QCP instance (CSP formulation), log-log scale; (b) same as (a), but for a logistics instance (SAT formulation). In (b) the left-most curve is for a cutoff value of 16, the middle curve is for a cutoff of 250, and the right-most curve is without restarts The effect of random restarts on solution cost for the logistics.d planning problem (SAT formulation) Portfolio results for logistics planning: left panel, 2 processors; right panel, 20 processors A range of portfolios for the MIP formulation of logistics planning (expected runtime) Expected runtime (a) and expected total cost (b) of optimal portfolio for different numbers of processors Example of dtp with N = 10, M = 3, G = Example of 3-opt move Example of node-transfer move Example of ejection chain. 319

16 List of Tables 3.1 Complexity of the polyomino reconstruction problem Primitive and non-primitive constraints Inference in IP and CP Definition of the four classes of experiments Estimates of the index of stability (0). k is the sample size, and u is the number of runs that failed to terminate before reaching the cutoff in the number of backtracks. The values within parentheses are the estimated asymptotic standard deviations Comparison of tail probabilities, Pr {X > c}, for standard normal, Cauchy, and Levy distributions Estimates of the index of stability (0). k is the sample size. The values within parentheses are the estimated asymptotic standard deviations (SAT encoding) ,4 Estimate of the index of stability (0) for the schooltimetabling problem. k is the sample size. The value within parentheses is the estimated asymptotic standard deviation Index of stability: impact of look-back strategies on heavy-tailed behavior (SAT encodings) Solving (a) a 16-team round-robin scheduling problem (CSP formulation) and (b) the logistics.d instance (SAT formulation) for a range of cutoff values Restarts using a hybrid CSPILP randomized backtrack search on a balanced QCP instance of order 50, at the phase transition Randomized rapid restarts (RRR) versus deterministic versions of backtrack search procedures (Satz solver used on SAT encodings; Hog solver on CSP encodings). 275

17 Preface During his invited talk at the first Workshop on Integration ofartificial Intelligence and Operations Research in Constraint Programming for Combinatorial Optimization, CP-AI-OR99, Jean-Francois Puget claimed that "Constraint Programming (CP) can be seen as Software Engineering applied to Operations Research (OR)". This statement is very strong, but indeed "almost" true. To my mind, CP is both something more and something less than this. It is something more since CP has its roots not only in Mathematical Programming, but also in Logic Programming and in Constraint Solving. From Logic Programming, CP inherits the declarative semantics, flexibility, the relational form which enables the definition of constraints without specifying input and output parameters, and the non-determinism which leads to the exploration of a search tree with a backtracking scheme. From Constraint Solving and Artificial Intelligence, CP has inherited powerful filtering algorithms, sophisticated search techniques and nogood recording. However, CP is something less than this: OR is so wide, and well studied since the fifties that so far CP could only cover a small, yet effective, portion of OR. The integration of areas of OR such as game theory, decision theory and stochastic programming into a CP framework has yet to be addressed. However, experience with current implementations has already shown that CP is a powerful framework where Mathematical Programming concepts, and also local search techniques can be smoothly integrated and easily used. The OR community has strengths both in breadth and in depth. The techniques of Integer and Linear programming and Local search are applicable to a huge swathe of (combinatorial) optimisation problems. Yet OR researchers have explored certain problem classes in great depth. These problems (the Travelling Salesman Problem, Set Partitioning and Covering, the Knapsack Problem, to name a few) are pure problems. No side constraints are considered. This simplification has an enormous advantage: pure problems are structured. The geometrical structure of the problem can help to reveal important problem properties. Geometric and algebraic structures enable practitioners to define ad hoc super-efficient (often polynomial) solving algorithms. Although pure problems rarely appear in real life applications, their investigation leads to the

18 xx CONSTRAINTAND INTEGER PROGRAMMING development of algorithms that can be effectively exploited to solve variants of these problems. These algorithms can be considered as software components providing results exploited in a large variety of applications. The exploitation of bounds, reduced costs, optimal solutions of subparts of the original problems, heuristic suggestions coming from relaxations or problem decompositions dramatically enhance CP solver performance. Linear programming solvers are nowadays integrated in all commercial CP solvers. When it was first introduced, Constraint Logic Programming was a declarative, general framework for solving Constraint Satisfaction Problems. It was more efficient than Logic Programming, but for combinatorial optimization problems, it was not powerful enough to compete with OR methods. Now, things have changed. CP is closer to software engineering applied to OR. Firstly global constraints have been introduced into CP solvers, embedding complex pruning techniques. They are a powerful, fundamental tool for modelling and solving constraint (optimization) problems. Secondly, the optimization side, that was naively treated at the beginning, is now becoming an important aspect of CP solvers exploiting OR techniques. Many steps need to be taken. CP researchers need to study, learn and think to discover which is the most general way to incorporate OR components in CP solvers. It is a big challenge. This book was conceived during the CP-AI-OR School on Optimization I organized in Le Croisic (France) in The School was a successful event with more than fifty participants, and the speakers covered almost all aspects of the integration in an exhaustive and stimulating way. After the school, I was solicited to collect papers concerning the talks into a book. Therefore, the book is mainly devoted to students, researchers and practitioners who are entering this evolving research field. The book covers a wide range of perspectives on the field. Beside chapters based on talks from the school, the book contains some additional papers covering aspects which were not treated during the school for reasons of time. The book is organized as follows: Chapter 1 is devoted to an introduction that provides a high level overview of the main concepts of Constraint Programming (CP) and Integer Programming (lp) used in the book. Chapter 2 informally introduces integration methods describing and classifying the main works in the field. Chapter 3 presents a unifying framework that presents under a uniform perspective the main concepts of CP and IP, underlining similarities and differences and stating the basis for possible integrations. In Chapter 4 global constraints are described as a vehicle for integrating IP concepts in CP in a transparent way for the user. Chapter 5 presents various ways to integrate relaxations in Constraint Programming focussing on global constraints. Then, Chapter6 describes hybrid solvers and Chapter 7 concerns Column Generation

19 PREFACE xxi and its integration in Constraint Programming. Chapter 8 concerns randomization and problem structure as a basis for understanding the intrinsic difficulty of the combinatorial problems. Many incomplete methods have been proposed mixing incomplete search (like local search and metaheuristics) in CPo Thus, Chapter 9 is devoted to a survey on the subject. Finally, the last chapter is devoted to open perspectives and future directions. The authors are eminent and well known researchers and have significantly contributed to the field. They come from Universities, Research Centers and industries. Finally, I would like to thank all the people who have helped me in the realization of this book. Andrea Lodi and Jean-Francois Puget suggested me to write this book. KrzysztofApt spared his precious time to help me at the beginning of the process with invaluable advice. Mark Wallace who kindly proofread some parts of this book. John Hooker introduced me to Kluwer and pointed me out the CS/OR series. Gary Folven provided me with constant assistance. Francois Laburthe and Narenda Jussien helped me in the organization of the School of Optimization in Le Croisic. Warm thanks go to the authors of the papers presented here for their work and their constant commitment in respecting the numerous requests I have asked of them, among which were requests to review other chapters in the book. Besides the authors, I would like to thank external reviewers: Torsten Fahle, Marco Gavanelli, Willem Jan van Hoeve, Ulrich Junker, Meinolf Sellman and Louis Martin Rousseau. I would also like to thank my colleagues Luca Benini and Andrea Roli for their help in the editing of several parts of the manuscript. MICHELA MILANO

20 Contributing Authors Farid Ajili IC-Parc, Imperial College London LondonSW7 2AZ, United Kingdom. Alexander Bockmayr Universite Henri Poincare, LORIA, B.P Vandamvre-les-Nancy, France Yves Caseau BOUYGUES TELECOM 20, quai du Point du Jour Boulogne Billancourt, France Emilie Danna ILOGand LlA, Universite d'avignon, CNRS-FRE2487 Kelly King Easton Department ofindustrial and Manufacturing Engineering Kansas State University 237 DurlandHall - Manhattan, KS keaston@isye.gatech.edu Filippo Focacci IWGS.A.

21 xxiv CONSTRAINTAND INTEGER PROGRAMMING 9, rue de Verdun BP 85, Gentilly, France Carla P. Gomes Dept. ofcomputer Science Cornell University Ithaca /4853 Thomas Kasper GBU Supply Chain Management, IBD3 Optimization SAP AG, Neurottstr. 16,69190 Walldoif, Germany Francois Laburthe BOUYGUES S.A. I, avenue E. Freyssinet St-Quentin-en-Yvelines Cedex, France Claude Le Pape IWGS.A. 9 rue de Verdun F Gentllly Cedex clepape@i1ogjr Andrea Lodi DEIS, University ofbologna Wale Risorgimento 2, Bologna, Italy alodi@deis.unibo.it Michela Milano DEIS University ofbologna V.leRisorgimento 2, Bologna, Italy mmilano@deis.unibo.it George L. Nemhauser School ofindustrial and Systems Engineering

22 Contributing Authors xxv Georgia Institute oftechnology Atlanta. GA USA Jean-Francois Puget IWGS.A. 9, rue de Verdun BP Gentiliy, France Jean-Charles Regln ILOG 1681, route des Dolines, Sophia Antipolis, Valbonne, France Michael Trick Graduate School 0/Industrial Admin istration Carnegie Mellon University Pittsburgh, PA USA trick@cmu.edu Mark G. Wallace IC-Pare, Imperial College London London SW7 2AZ, United Kingdom. mgw@imperial.ac.uk

23 Foreword Constraint Programming and Mathematical Programming are beginning to merge, despite their very different origins. Constraint programming grew out of the logic programming community as part of an effort to embed constraints in a programming language. Mathematical programming is a much older field, rooted in the mathematics of optimization. Yet recent years have seen a series of efforts to integrate the two. Why should these two technologies be combined? The most obvious reason is that they have complementary strengths. Constraint programming solvers are adept at logical reasoning, particularly constraint propagation, while mathematical programming solvers use sophisticated relaxation techniques. Constraint programming may therefore excel at combinatorial problems with weak continuous relaxations, such as sequencing and scheduling problems, or problems with disjunctive and other logical conditions. Mathematical programming may be the method of choice for resource allocation and other planning problems. Constraint programming can focus on highly structured portions of a constraint set, while mathematical programmingcan take a global view, particularly in its continuous relaxations. When side constraints complicate a problem, constraint programming can accommodate them gracefully and actually use them to advantage. When the problem structure is pure, without side constraints, mathematical programming's analysis of polyhedral structure can yield powerful methods. Constraint programming may be effective when constraints contain only a few variables, since such constraints tend to "propagate well." Mathematical programming is more suited for constraints (or objective functions) with many variables, such as constraints that measure cost, since they tend to "relax well." Constraint programming brings to the table a large modeling vocabulary that can result in succinct and natural problem formulations. Mathematical programming provides the ability to translate a wide variety of models into an economical language of inequalities, which may yield a useful continuous relaxation. While some problems seem suited for constraint programming and some for mathematical programming, many others have elements amenable to both. For

24 xxviii CONSTRAINTAND INTEGER PROGRAMMING instance, some constraints in the problem may propagate well, while others relax well. It seems only reasonable to call on the resources of both fields, as needed, in a "hybrid" or integrated solver. Integration can be accomplished in a fairly natural way, if one recognizes that constraint programming and mathematical programming use basically the same overall solution strategy. Focusing for the moment on exact solution, they have four points of commonality-similar search methods, their use of inference, their use of relaxation, and the analogous ways that inference and relaxation interact. More specifically: (a) Both constraintprogrammingand mathematical programming typically rely on branching methods to search the solution space exhaustively. (b) Inference in constraint programming takes the form of domain reduction, constraint propagation and constraint learning (otherwise known as nogood generation). Mathematical programming systems, on the other hand, infer cutting planes, separating cuts, and Benders cuts, the last two being forms of nogood generation. (c) Relaxation appears in constraint programming in the form of a "constraint store," which keeps track of the domain (set of feasible values) of each variable. Continuous relaxation is of course the work horse of mathematical programming. (d) In both technologies, inference is often used to strengthen the relaxation. Inference in constraint programming strengthens the constraint store by reducing its variable domains, and inference in mathematical programming strengthens the continuous relaxation by adding cutting planes. Thus constraint programming and mathematical programming are special cases of the same search-infer-and-relax solution strategy. They combine naturally in this common framework. At the inference stage of the algorithm, for example, one can apply inference methods from eitheror both fields, depending on the characteristics of the problem. One can do the same at the relaxation stage. There can be mutual reinforcement, as when reduced domains inferred for the constraintstore help tighten the continuous relaxation, and cutting planes inferred for the continuous relaxation propagate to reduce domains further. The respective strengths of constraint programming and mathematical programming complementeach other to accelerate the search. The inference technology of constraint programming strives to maintain various types of "consistency," which in the simplest case amounts to reducing the variable domains and thereby the number of values on which one must branch. The relaxation technology of mathematical programming accelerates the search by providing bounds on the optimal value, which can be used to prune the search tree as part of a branch-and-bound strategy (known as "branch and relax" in the constraint programming community). We are arriving at a deeper and more satisfying answer to the question posed above: why should constraint programming and mathematical programming be combined? They should be combined because they are already the same. They are instances of the same general method.

25 xxix The modeling style of this general method is likely to favor that of constraint programming rather than mathematical programming, but only because it already represents a certain kind of integration. The declarative modeling style of mathematical programming helped to inspire constraint programming's research goal of embedding constraints in a programming langauge. Yet constraint programmers, rooted as they are in a computerscience tradition, continue to regard constraints as procedures-specifically, as procedures that operate on the solution space by removing solutions that cannot satisfy the associated constraints. Constraint programming formulations are therefore a blend of declarative and procedural modeling. The simple idea of viewing constraints as procedures opens the door to exploiting substructure in a problem. Constraint programmers typically write a group of constraints with a specific overall structure as a single "global constraint," such as the alldifferent constraint, which requires a set of variables to take distinct values, or the cumulative constraint, which is used for resourceconstrained scheduling. In so doing the modeler communicates substructure in the formulation to the solver, which contains domain reduction algorithms tailored to each type of global constraint. Thus constraint programming exploits problem structure primarily through its inference methods, which tum out to be the key to its success. Mathematical programming, by contrast, exploits problem structure primarily through its relaxations, which are likewise essential to its success. The idea of global constraints can of course be used in a hybrid setting, where they can invoke specialized relaxations as well as domain reduction procedures. For instance, constraints that require a hamiltonian path (used in the traveling salesman problem) can be written as a single global constraint (the "cycle"constraint), which can activate specialized domain reduction procedures developed by constraint programmers and generate cutting planes developed by integer programmers. Viewing constraint and integer programming as special cases of a single search-infer-and-relax method poses the interesting intellectual challenge of defining just what this single method is, and how much it can be usefully generalized. A good deal of generalization turns out to be useful, particularly when taken far enough to encompass decomposition methods. For instance, the search can be viewed as the enumeration of Benders subproblems, and the relaxation as the Benders master problem. If the master problem is solved by mixed integer programming and the subproblems by constraint programming, the result is an integration scheme based on decomposition. (Benders cuts can be obtained from the constraint programming solution by extending classical concepts of duality.) This scheme turns out to be intuitive and useful in manufacturing applications, in which the master problem typically assigns jobs to machines, and the subproblem schedules the jobs that are assigned to

26 xxx CONSTRAINTAND INTEGER PROGRAMMING each machine-thus exploiting the mathematical programming advantage in resource allocation, and the constraint programming advantage in scheduling. In fact, decomposition approaches have so far delivered the most impressive computational speedups among hybrid methods. In general, a search-infer-and-relax method moves from one problem restriction to another, while applying various types of inference and relaxation to each problem restriction. In integer and constraint programming, the problem restrictions are obtained at the nodes of the search tree, and the relaxations are the constraint store and the continuous relaxation. In a Benders approach, the restrictions are the successive subproblems, and the relaxation is the master problem. In each case, the search direction is suggested by the solution of a relaxation. For instance, in integer programming one branches on a variable with a fractional value in the solution of the continuous relaxation, and in Benders decomposition the next subproblem is obtained by solving the master problem. Local search methods also fit into this scheme, since they proceed by searching a sequence of neighborhoods, each of which can be viewed as the feasible set of a problem restriction. Local search has long been studied in Operations Research, but the constraint programming community has recently provided a fresh perspective. Ithas experimented with "large neighborhood" search, where constraint programming methods are used to search neighborhoods, as well as quasi-declarative languages for specifying local search algorithms. There have also been efforts to incorporate randomized search and other heuristic ideas into an exhaustive enumeration procedure. At this writing there is no consensus on the best architecture for an integrated method. Some commercial software products support a certain amount of integration, such as the ECLiPSe system, ILOG's OPL Studio, and Dash Optimization's Mosel language, but no system comes close to exploiting the full advantage of integration. Nonetheless, the situation will quickly change. Research proceeds apace, computational successes accumulate, and more powerful software is under development. Integrated methods have already proved themselves in industrial applications. Hybrid solvers and modeling systems may become the norm in a very few years. The very term "hybrid" may become obsolete, as we increasingly see today's technologies as parts of a seamless whole. It is therefore an ideal time to familiarize oneself with the basic ideas of what are now known as hybrid methods. The present volume serves this end. Michela Milano has performed a great service by collecting a set of tutorial essays written by some of the top researchers in the field. They present some of the basic ideas of constraint programming and mathematical programming, explore approaches to integration, bring us up to date on heuristic methods, and attempt to discern future directions in a fast-moving field. JOHN HOOKER

27 To my parents