BACK TO MATH LET S START EASY.

Transcription

1 BACK TO MATH LET S START EASY. Of all rectangles of the same perimeter, which one has the largest area? (Yes, you have all solved this problem before, in Calc I. Apologies if this insults your intelligence. ) Objective function? Constraints, if any? How do we solve it? This is a simple nonlinear optimization problem. 15

2 ANOTHER EASY ONE. Sandwiches We want to make two kinds of sandwiches for a party from the ingredients below. At most how many can we make in total? Type 1 Type 2 Inventory cheese slices ham slices salami slices cucumbers 1-35 slices capers pieces 16

3 LINEAR OPTIMIZATION This is an example of a linear optimization problem, aka linear program, aka LP. The objective function is linear. The constraints are linear equations/inequalities. (The inequalities are not strict.) 17

4 18 LINEAR OPTIMIZATION This is an example of a linear optimization problem, aka linear program, aka LP. The objective function is linear. The constraints are linear equations/inequalities. How can we solve it? Don t try to set derivatives equal to zero. Remember the endpoints of the interval? More generally, the optimal solution may be at the boundary of the feasible region. In linear optimization, the boundary is where all the action is! Let s draw a picture

5 19 LINEAR OPTIMIZATION This is an example of a linear optimization problem, aka linear program, aka LP. The objective function is linear. The constraints are linear equations/inequalities. How can we solve it? Don t try to set derivatives equal to zero. Discussion Name as many things as you can that our model ignored or missed. If we add all of these, would we still have a linear optimization model?

6 20 LINEAR OPTIMIZATION MODELS: MORE EXAMPLES Multi-day sandwich production with or without expiration dates (inventory) Further problems (handout)

7 21 DEMO: SOLVING LINEAR OPTIMIZATION PROBLEMS Using Excel: Using Matlab: There are much more sophisticated tools, too for modeling and solution; we won t talk about them. (See 493 next semester.)

8 22 HOW DO COMPUTERS SOLVE THESE PROBLEMS?

9 23 EXISTENCE OF SOLUTIONS Is min f(x) x S well-defined? Does it exist? Is it unique? Do we care (and why)? inf/sup versus min/max Inf and sup always exist, although might be infinite (What is inf? And sup?) If the inf or sup is not attained, there is no optimal solution. Infeasible vs unbounded problems

10 INFEASIBLE AND UNBOUNDED PROBLEMS The brilliant Cerebron, attacking the problem analytically, discovered three distinct kinds of dragon: the mythical, the chimerical, and the purely hypothetical. They were all, one might say, nonexistent, but each nonexisted in an entirely different way. Infeasible Feasible region is empty. The objective function does not matter. Practically very common(!), especially in early stages of modeling Unbounded Feasible region is unbounded Objective function can take arbitrarily small (for minimization) or large (for maximization) values optimal value is ± Uncommon: in practical problems, it can be the sign of missing essential constraints (useless model) 24

11 INFEASIBLE AND UNBOUNDED PROBLEMS Three outcomes In linear optimization, there are only three possible outcomes: 1. There is an optimal solution (finite min/max) 2. The problem is infeasible 3. The problem is unbounded In nonlinear optimization, there is a fourth one: 4. The inf/sup is finite but not attained. Example: minimize e x. 25

12 26 WHAT IS A SOLUTION? Suppose we are trying to solve the equation: x 1 + 2x 2 = 7 2x 1 3x 2 = 0 We are told that the solution is x 1 = 3, x 2 = 2. Is that a satisfactory answer? Why?

13 27 WHAT IS A SOLUTION? Suppose we are trying to solve the equation: x 1 + 2x 2 = 7 2x 1 3x 2 = 0 We are told that the solution is x 1 = 3, x 2 = 2. Is that a satisfactory answer? Why? Now let s return to the sandwich example. We are told that the optimal solution is x 1 = 22, x 2 = 21. Is that a satisfactory answer? Why?

14 CERTIFICATES OF (IN)FEASBILITY AND OPTIMALITY It is usually easy to prove and verify feasibility Certificate of feasibility: a feasible solution How do we check it? Plug it back into the problem. Proving and verifying infeasibility is usually much trickier What is a certificate of infeasibility?? Stating that a solution is optimal always involves two statements: one feasibility, and one infeasibility The optimal solution is feasible No feasible solution has a better objective function value than the optimal solution. Ideally, we want a certificate of optimality with our optimal solutions. Or a certificate of infeasibility. Or a certificate of unboundedness. 28

15 29 CERTIFYING INFEASIBILITY IN LINEAR OPTIMIZATION Suppose a system of linear equations Ax b has no solution. Is there (and how do we get) a certificate of infeasibility? Farkas Lemma [pron: Farkash]: Theorem (Farkas). For every A R m n and b R m, precisely one of the following two statements holds: 1. Ax b has a solution x R n. 2. y 0, A T y = 0, and b T y < 0 has a solution y R m. Note: there are many syntactically different, but equivalent forms. (If you google it, you might find a different theorem.) Intuitively, this means

16 CERTIFYING OPTIMALITY Recall: a statement of optimality = a statement of feasibility and a statement of infeasibility. Feasibility is easy to certify We just saw how to certify infeasibility One can derive optimality certificates from this The details are somewhat complicated, we ll skip it this time. 30

17 CERTIFYING OPTIMALITY Again, why do we care? This is what all algorithms are based on. (They essentially have to be!) Generic optimization algorithm: start with a feasible solution. Certify that it s optimal; if not, find a better one, and recurse. Sounds familiar? (Compare to steepest descent from calculus.) 31

18 32 THE SIMPLEX METHOD The most commonly used algorithm for the solution of linear programs. The precise (linear algebraic) description of the method is fairly involved, but the geometric idea is simple. Relies on the fact that if the feasible region has a corner and there exists an optimal solution, then there is an optimal corner. The simplex method (sketch): Find a corner of the feasible region, or a certificate of infeasibility. (In the latter case, stop.) Find the adjacent corners; move to a better adjacent corner if there is one. If all adjacent corners are worse than the current one, stop. At the current corner, find an optimality certificate, or a certificate of unboundedness.

19 33 LINEAR OPTIMIZATION SOFTWARE Excel Solver. Not for serious work, but fantastically convenient. Demo: the sandwich example. Matlab s linprog function Provides interfaces similar to the hardcore optimization software. The simplest interface is a simple command: Objective coefficients lb x ub [x,fval] = linprog(f,aineq,bineq,aeq,beq,lb,ub,x0,opts) Inequality constraints, equality constraints, variable bounds Demo: the sandwich example. Military-grade (export controlled!) software: CPLEX, Gurobi. (Very expensive, though free student licenses exist for US students.) Open source libraries for C/C++, Python, Java, etc: SCIP, LPsolve, CLP, etc.

20 34 INTEGER AND MIXED- INTEGER OPTIMIZATION Conceptually, it s the same as linear optimization: Continue formulating everything using linear functions: Minimize/Maximize a linear function Subject to linear equality and inequality constraints New: some or all variables may be required to be integers. This is a real game changer as far as the expressivity of the models is concerned. Examples from the handout. No free lunch: these problems are way harder to solve than linear optimization problems.