2. Modeling AEA 2018/2019. Based on Algorithm Engineering: Bridging the Gap Between Algorithm Theory and Practice - ch. 2

Transcription

1 2. Modeling AEA 2018/2019 Based on Algorithm Engineering: Bridging the Gap Between Algorithm Theory and Practice - ch. 2

2 Content Introduction Modeling phases Modeling Frameworks Graph Based Models Mixed Integer Programming Constraint Programming 2 / 49

3 Modeling: the first step in Algorithm Engineering First step in AE: understand the problem and construct a formal model that capture the requirements (applied to a wider range of situations) Modeling: the procedure of abstracting from problem instances to problem classes that still contain the essential details of the problem structure Find an appropriate level of abstraction that captures the essential real structure and omits less important aspects 3 / 49

4 Modeling Classical algorithm theory: tends to consider rather artificial problems and manufactured models assumes an existing formal model and omits a discussion on the model s development process Advantages of models: generalization, faster explanation of problems to others and (possibly) the availability of well analyzed solutions Standard (theoretical) models often do not reflect properties which are inherent to practical applications Not possible to establish a sharp border between modeling and designing 4 / 49

5 Content Introduction Modeling phases Modeling Frameworks Graph Based Models Mixed Integer Programming Constraint Programming 5 / 49

6 The modeling phases The modeling process can be subdivided in: 1. Understand and formalize the problem 2. Reformulate the problem towards one or more candidates of approaches (solution approach models) 3. Evaluate and verify the results. Start the procedure again with the feedback gained when unsatisfactory results are obtained. 6 / 49

7 1. Fundamentals The properties of a model: abstraction from the original application the purpose of the model: specify a problem, describe an approach, or both specify the requirements of the application; for a problem class (decision-, construction-, counting-, optimization problems), what a valid solution looks like; for optimization problems, define the objective function A model can be specified using the following formalisms: informal: the first step in practice semi-formal: colloquial, but uses some formal concepts (graph, sets); the first step towards abstraction formal: uses mathematical concepts to describe requirements, valid solutions, and objective functions 7 / 49

8 Fundamentals Use the following concepts, when specifying a model: variable: decision possibilities parameter: a value/property of a problem instance, used as an abstraction constraint: abstract description of a requirement Example: Two workers have to produce an item on a certain machine. Bill needs 30 minutes to produce an item, John 15 minutes. How many items can be produced in an eight hour shift by each worker? x: the number of items that can be created, parameter w: the working time needed to complete an item, the constraint w x = 8. 8 / 49

9 Problem Analysis A good comprehension of the problem 1. What does the input consist of? 2. What are the desired results or output? 3. Is it possible to divide constraints into hard and soft ones? Hard constraints have to be fulfilled, soft constraints should be fulfilled (are goals). 4. Can variables or parameters of the problem be divided into important and unimportant ones? 5. Is the problem of a certain problem class, or does it imply a certain solution approach? Impact how the problem is modeled and might suggest solution approaches. 9 / 49

10 Problem Analysis The imprecision of data discrepancy between the model of computation used in classical algorithm theory and the reality of actual hardware and operating systems imperfect information affects the problem (imperfect input data/constraints, the parameters change fast) How to handle vague and uncertain data: obtain new sharpened versions; use the theory of fuzzy sets to model and process It s possible to reduce the complexity of some problems by using vague information and avoiding highly detailed specifications 10 / 49

11 Problem Specification: Examples Sudoku N = {1,..., 9}, N = {1,..., 3}. An instance of a Sudoku puzzle: a set S of (i, j, n) N 3, n is at position (i, j). A solution: a function f : N 2 N provides for each pair the value at this position. Requirements: all cells in row i have distinct values: f satisfy {f (i, j) j N} = N; similarly for columns; for subsquares: C ij = {(3(i 1) + i, 3(j 1) + j ) (i, j ) N xn }. The model: f (i, j) = n, (i, j, n) S {f (i, j) j N} = N, i N {f (i, j) i N} = N, j N f (C ij ) = N, (i, j) N xn 11 / 49

12 Problem Specification: Examples Traveling Salesman Problem (TSP) Consider n cities. A round trip through all cities: a permutation π : {1,..., n} {1,..., n}. The cost for traveling between two cities c ij ; C = (c i,j ) 1 i,j n ; the cost c(π) for a round trip π: n 1 c(π) = c π(i),π(i+1) + c π(n),π(1) i=1 The task: find a permutation π with minimum cost. Board Assembly Problem The problem of printed circuit board assembly can be interpreted as a TSP: the mounting holes: the cities to be visited the robot arm: the salesman the cost matrix: the time needed for the arm to be moved between holes. 12 / 49

13 Problem Specification: Examples Car manufacturing problem M a set of car models built by the car company, P a set of plants, R a set of retailers. For each plant p P, M p M a set of car models produced at this plant, and for each retailer r R, a vector d r Z M specify the retailer s demand for each car model. Look for a matrix X = (x p,m ) p P,m M, x p,m the no. of cars for model m produced at p, and Y = (y p,m,r ) p P,m M,r R, y p,m,r the no. of cars of model m that will go from plant p to r. X and Y must satisfy: the demand of each retailer has to be met the no. of cars of model m leaving p must be exactly x p,m The goal is to fulfill the requirements of the retailers at min. cost. 13 / 49

14 Problem Specification: Examples Car manufacturing problem (cont.) Given a matrix A = (a p,m ) p P,m M describing the cost for producing one unit of car model m at plant p, and a matrix B = (b p,r ) p P,r R providing the cost for transporting a car from plant p to retailer r, look for matrices X and Y that minimize the total cost c(x, Y ) := p P,m M a p,mx p,m + m M p P,r R b p,r y p,m,r. 14 / 49

15 Remember: The modeling phases The modeling process can be subdivided in: 1. Understand and formalize the problem 2. Reformulate the problem towards one or more candidates of approaches (solution approach models) 3. Evaluate and verify the results. Start the procedure again with the feedback gained when unsatisfactory results are obtained. 15 / 49

16 2. Modeling a solution approach The next step: transform/use the problem model to gain a solution approach model Identify the constraints What constraints have to be considered to solve the problem? How do these constraints affect the solution? Can all necessary constraints be formulated with the approach? Analyze the problem again: find properties a solution approach may exploit Can the problem be decomposed into subproblems? How does accuracy impact the application? Is the optimal answer needed or would an approximation be satisfactory? How important is efficiency? What is the time frame in which an instance should be solved? How large are typical problem instances? How much time and effort can be invested in implementation? Which model to choose: simplicity, existing solutions, patents, complexity of implementations, time, costs 16 / 49

17 3. Model Assessment After developing, the model is verified and rated; continue with the next step of AE, or restart and improve the model Have all necessary aspects of the problem been considered? Are all modeled aspects really necessary for the problem? Is the model consistent? Test the model with small instances or use a standard solver Difficulties within the modeling process: formalize the criteria for assessing the quality of a solution (objective function); ex: sequence alignments of proteins 17 / 49

18 Content Introduction Modeling phases Modeling Frameworks Graph Based Models Mixed Integer Programming Constraint Programming 18 / 49

19 Content Introduction Modeling phases Modeling Frameworks Graph Based Models Mixed Integer Programming Constraint Programming 19 / 49

20 Graph-Based Models Used for combinatorial problems Graphs: combinatorial structures that can be used to model a set of objects and the pairwise relations between them. Graph G := (V, E), V a set of vertices, E ( V 2) a set of edges. Advantages: intuitive and easy to grasp design direct and, in many cases, efficient algorithms theory that might help in modeling or designing algorithms At the heart of more sophisticated models, like LP/MIP Cannot cope with more complicated constraints No formalism for specifying graph-based models 20 / 49

21 Graph-based Models: examples TSP: model an instance as a weighted complete graph. Vertices: the cities, the weights c ij the distances between the cities (no connection between cities i and j: a large weight c ij ). A solution: a cycle that visits all the vertices (Hamiltonian cycle); the cost of a cycle C = (v 1,..., v n ) is: n 1 c(c) = c vi,v i+1 + c vn,v1 i=1 21 / 49

22 Graph-based Models: Sudoku Use edges to represent conflicts. A model using vertex coloring: use a set of 81 vertices (one for each square); insert edges s.t. they reflect the rules of Sudoku: connect all vertices corresponding to a row, column and a subsquare (express that they should be numbered differently); the assignment of numbers to the squares as a vertex coloring. Figure: A graph-based model for Sudoku of size 2 22 / 49

23 Graph-based Models: Sudoku as a stable set problem Create 9 vertices for each square (each vertex corresponds to assigning the square the number of the vertex); insert an edge between every pair of vertices belonging to the same square to model that only one of these vertices can be selected; pairwise connect all vertices i in a row, column, and subsquare by an edge, for each i N. Encode a solution as a stable set: an instance has a solution if and only if there is a stable set of size 81 that contains the vertices corresponding to the prescribed numbers 23 / 49

24 Content Introduction Modeling phases Modeling Frameworks Graph Based Models Mixed Integer Programming Constraint Programming 24 / 49

25 Mixed Integer Programming Used in decision making and optimization The set of feasible solutions is described by linear constraints (linear equations or inequalities). The objective is a linear function of the variables (real or integer) used to model the solutions. Formally, min x,y c T 1 x + c T 2 y s.t. Ax + By b x R n 1 y Z n 2 x denotes the n 1 real variables, y the n 2 integer variables, c 1, c 2, b, A, B arbitrary real numbers (parameters). 25 / 49

26 Mixed Integer Programming Types of MIPs: Linear programs (LP): real variables; solvable in polynomial time Integer programs (IP): integer variables; NP-hard Binary integer programs (BIP): binary integer variables; NP-hard When constructing a MIP model: identify variables that encode the decisions and use them to express the problems requirements and the objective MIP solvers that performs well in practice: ILOG CPLEX (commercial), SCIP (open-source); based on LP-relaxation (omit the integrality constraints for obtaining lower bounds) 26 / 49

27 MIP example: Sudoku It is not possible to translate the requirements for f to IP constraints by replacing f (i, j) by an integer variable x ij N (the condition that each number should appear exactly once). Use binary variables x ijn, x ijn = 1: n is put in cell (i, j). min 0 s.t. x ijn = 1, (i, j, n) S, x ijn = 1, i, j N, n N x ijn = 1, i, n N, j N x ijn = 1, j, n N, i N (fixed) (nums) (rows) (columns) (i,j ) C i,j x i j n = 1, (i, j) N xn, n N, (squares) x ijn {0, 1}, i, j, n N 27 / 49

28 MIP example: Sudoku (cont.) The model uses the sets C ij containing the indices for the cells in one subsquare. The constraint (nums): select exactly one number for cell (i, j). The constraint (rows): each row contains each value exactly once. The objective is 0, i.e. Sudoku is a pure feasibility problem. 28 / 49

29 Further modeling aspects A large number of constraints or variables does not indicate a bad model Facility location problem: C := {1,..., m} customers, W := {1,..., n} possible locations of warehouses; decide at which of the possible locations to build a warehouse and which warehouse serves each customer - binary variables x ij = 1: customer j is assigned to warehouse i, and y i = 1: warehouse i is available; f i : the cost for building warehouse i; c ij : the costs incurred if customer j is assigned to warehouse i. Warehouses should be opened s.t. the total cost is minimized. min c ij x ij + f i y i i W,j C i W s.t. x ij = 1, j C i W x ij y i, i W, j C(single) x ij, y i {0, 1}, i W, j C 29 / 49

30 Further modeling aspects Facility location problem (cont.) min s.t. c ij x ij + f i y i i W,j C i W x ij = 1, j C i W x ij my i, i W (agg) j C x ij, y i {0, 1}, i W, j C Difference: the inequalities (single) have been aggregated by summing over all customers. Is not a good decision to prefer the second model: the LP relaxation is tighter for the 1st relaxation (the reason: {x ij, y i [0, 1] x ij y i, i W, j C} is contained in {x ij, y i [0, 1] j C x ij my i, i W }). 30 / 49

31 Branch and Bound Main idea: solve MILP problems by solving a sequence of linear relaxations to provide bounds Remove the integrality restrictions (LP relaxation) and solve the LP. If the result satisfy the integrality restrictions, stop. If not, pick some variable x that is restricted to be integer, but whose value in the LP relaxation is fractional. Exclude this value by imposing two restrictions. P 0 the original MIP, x the branching variable; branching on x, produces two sub-mips P 1, P 2 ; if optimal solutions for P 1 and P 2 can be computed, then take the better of these solutions (it will be optimal to P 0 ); in this way we replaced P 0 by two simpler (more-restricted) MIPs. Apply the same idea to these two MIPs, solving the corresponding LP relaxations and, if necessary, selecting branching variables. 31 / 49

32 Branch and Bound A search tree is generated: nodes: the MIPs generated by the search procedure (P 0 the root node) the leaves: the nodes from which we have not yet branched; Figure: Each node in B&B is a new MIP 32 / 49

33 Branch and Bound Fathomed and incumbent nodes Suppose that we solved the LP relaxation of a node in the search tree. If the integrality restrictions in the original MIP are satisfied, then we found a feasible solution to the original MIP. Designate this node as fathomed; not necessary to branch on this node - a leaf. Analyze the information provided by the feasible solution; incumbent: the best integer solution found at any point in the search; at the start of the search - no incumbent if the integer feasible solution just found has a better objective value than the current incumbent, then record the solution as the new incumbent, and its objective value. 33 / 49

34 Branch and Bound Other possibilities that can lead to a fathomed node: The branch that led to the current node added a restriction that made the LP relaxation infeasible; if this node contains no feasible solution to the LP relaxation, then it contains no integer feasible solution. An optimal relaxation solution is found, but its objective value is bigger than that of the current incumbent; this node cannot yield a better integral solution and can be fathomed. 34 / 49

35 Best Bound and Gap The objective value for the incumbent is an upper bound on the optimal solution of the MIP (minimization problem): never accept an integer solution of value higher than this value. A lower bound (best bound): take the minimum of the optimal objective values of all of the current leaf nodes. Gap: the difference between the current upper and lower bounds. When the gap is zero we demonstrated optimality. 35 / 49

36 Further modeling aspects Cutting planes: solve the linear relaxation; if the obtained optimum is not an integer solution, there is a linear inequality (cut) that separates the optimum from the convex hull of the true feasible set (separation problem); add the cut to the relaxed linear program; 36 / 49

37 Further modeling aspects Presolving: convert the model to an equivalent one, better suited for the subsequent solution process; obtain models that are easier to write down - reduce the size of the problem and tighten the formulation; Ex: variable substitution x 1 + x 2 + x 3 15 x 1 7 x 2 3 x 3 5 Substitute out the variables by x 1 = 7, x 2 = 3, x 3 = 5 and remove the constraints. 37 / 49

38 Content Introduction Modeling phases Modeling Frameworks Graph Based Models Mixed Integer Programming Constraint Programming 38 / 49

39 Constraint Programming (CP) A declarative modeling formalism similar to MIP A constraint programming model consists of: a set of variables X = {x 1,..., x n }, for each x i, a set D i of possible values (domain), a set of constraints restricting the values variables can take simultaneously. For a set of variables {x i1,..., x ik }, a constraint is an arbitrary subset of D i1 x...xd ik. A solution is an assignment of values to the variables from their domains s.t. every constraint is satisfied. A CP model is very general: variable domains can be chosen arbitrarily (intervals of real, integer or natural numbers), and are not restricted to numbers; constraints can be any relation on the domain; CP is more powerful regarding expressiveness than MIP 39 / 49

40 Constraint programming: examples Sudoku. Variables x ij {1,..., 9} =: N (the values in the corresponding cells); use pairwise inequality constraints to express that all values in a row, column, subsquare are different: x ij x ik x ij x kj x i1 j 1 x i2 j 2 x ij N k i, j N, (rows) k i, j N, (columns) (i 1, j 1 ) (i 2, j 2 ) C ij, i, j N, (subsquares) i, j N Use the global alldifferent constraint for a more powerful formulation: alldifferent(x ij j N) i N, (rows) alldifferent(x ij i N) j N, (columns) alldifferent(c ij ) i, j N, (subsquares) x i,j N i, j N More compact model, better search-space pruning. 40 / 49

41 Constraint programming: examples TSP. A solution is a permutation of cities; x i : the number of the city at position i. Use alldifferent constraint to ensure that the variables constitute a permutation. Use variables as indices to other variables (variable indexing) to express the objective function: min n 1 c xi,x i+1 + c xn,x1 i=1 s.t. alldiferent({x 1,..., x n }), x i {1,..., n} 41 / 49

42 Constraint programming Solve CSPs by: backtracking (ILOG Solver, ECL i PS e ) consistency techniques or domain reduction: reduce the search space by detecting partial assignments that cannot be extended to feasible ones; local consistency (node-,arc-, path-consistency), directional consistency (ex: variable elimination) Use CP techniques to solve optimization problems: additional constraints that forbids solutions inferior to the currently best one 42 / 49

43 Consistency techniques Given a constraint network R = (X, D, C), R ij C, a variable x i is arc-consistent relative to x j if and only if for every value a i D i there is a value a j D j s.t. (a i, a j ) R ij. A network is arc-consistent if all all of its arcs are arc-consistent. input: a subnetwork defined by two variables X = {x i, x j }, domains D i, D j, constraint R ij output: D i s.t. x i arc-consistent relative to x j for each a i D i do if there is no a j D j s.t. (a i, a j ) R ij then delete a i from D i Algorithm 1: Revise for arc-consistency Complexity: O(k 2 ), k values 43 / 49

44 Arc-consistency example 44 / 49

45 AC-3 input: a network of constraints R = (X, D, C) output: R the largest arc-consistent network equivalent to R for every pair {x i, x j } that participates in a constraint R ij R do queue queue {(x i, x j ), (x j, x i )} while queue {} do select and delete (x i, x j ) from queue Revise((x i ), x j ) if Revise((x i ), x j ) causes a change in D i then queue queue {(x k, x i ), i k} Algorithm 2: Arc-consistency-3(AC-3) Complexity: O(ek 3 ), e no. of arcs, k values 45 / 49

46 Consistency techniques Figure: Arc-consistency example Arc examined V 1 V 2 V 1 V 3 V 2 V 3 V 1 V 2 V 1 V 3 V 2 V 3 Value deleted none V 1 (G) V 2 (G) V 1 (R) none none 46 / 49

47 Combine backtracking and constraint propagation Use constraint propagation (arc-consistency) to propagate the effect of the current partial assignment (i.e. eliminate values inconsistent with the current ones). Backtracking with Forward Checking When examining V i = d k, remove any values inconsistent with the assignment from neighboring domains. 47 / 49

48 Mathematical programming vs. Constraint programming A CP optimization model has the same structure as a mathematical programming (MP) model: a set of decision variables, a set of constraints, an objective function. Modeling differences: A CP model supports logical constraints and a full range of arithmetic expressions A CP model can use specialized constraints (alldifferent) that can accelerate search A CP model has no limitation on the arithmetic constraints that can be set on decision variables, while MP considers problems whose formulation satisfies certain mathematical properties A CP model has only discrete decision variables (integer or boolean), while a MP model supports discrete or continuous decision variables 48 / 49

49 Mathematical programming vs. Constraint programming Optimization engine differences: A CP engine after each decision on variables and values, perform a set of logical inferences to reduce the remaining variables domains. A MP engine, uses a combination of relaxations (strengthened by cutting-planes) and B&B. A CP engine proves optimality by showing that no better solution than the current one can be found, while a MP engine uses a lower bound proof provided by cuts and linear relaxation. A CP engine doesn t make assumptions on the mathematical properties of the solution space (convexity, linearity etc.), while a MP engine requires that the model falls in a well-defined mathematical category. 49 / 49