LINEAR PROGRAMMING
Introduction Development of linear programming was among the most important scientific advances of mid-20th cent. Most common type of applications: allocate limited resources to competing activities in an optimal way. Very large and wide range of applications from local to very global ones. Linear programming uses a mathematical model. Linear because it requires linear functions. Programming as synonymous of planning. 31
Introduction Besides allocate resources to activities, any problem described by a mathematical model can be solved. Very efficient solution procedure: simplex method. This method will be described next as well as the interior-point algorithm for larger problems. Besides these algorithms, we will also discuss sensitivity analysis. 32
Prototype example Wyndor Glass Co. produces glass products, including windows and glass doors. Plant 1 produces aluminum frames, Plant 2 produces wood frames and Plant 3 produces glass and assembles. Two new products: Glass door with aluminum framing (Plant 1 and 3) New wood-framed glass window (Plant 2 and 3) As the products compete for Plant 3, the most profitable mix of the two products is needed. 33
Data for Wyndor Glass Co. Wyndor Glass Co. Product-Mix Problem Doors Windows Profit Per Batch $3.000 $5.000 (Batches of 20) Hours available per week Hours Used Per Batch Produced Plant 1 1 0 4 Plant 2 0 2 12 Plant 3 3 2 18 34
Formulation of the LP problem x 1 = number of batches of product 1 produced per week x 2 = number of batches of product 2 produced per week Z = total profit per week (in thousands) maximize Z 3x 5x 1 2 subject to x 4 2 1 2 1 2x 12 3x 2x 18 and x 0, x 0. 1 2 35
Graphical solution Trial-and-error: Z = 10, 20, 36 Slope of the line is 3/5: 3 1 x2 x1 Z 5 5 Feasible region: satisfies all constraints and x 0, x 0. 1 2 36
Solution (using Excel) Wyndor Glass Co. Product-Mix Problem Doors Windows Range Name Cells Profit Per Batch $3.000 $5.000 BatchesProduced C12:D12 Hours Hours HoursAvailable G7:G9 Hours Used Per Batch Produced Used Available HoursUsed E7:E9 Plant 1 1 0 2 <= 4 HoursUsedPerBatchProduceC7:D9 Plant 2 0 2 12 <= 12 ProfitPerBatch C4:D4 Plant 3 3 2 18 <= 18 TotalProfit G12 Doors Windows Total Profit Batches Produced 2 6 $36.000 37
Generalizing example Prototype example Production capacities of plants General problem Resources 3 plants m resources Production of products Activities 2 products n activities Production rate of product j, x j Profit Z Level of activity j, x j Overall measure of performance Z The most common type of LP application involves allocating resources to activities. 38
Linear programming model Z = value of overall performance measure x j = level of activity j, for j = 1, 2,, n. c j = parameters of Z related to x j. b j = amount of resource i that is available for allocation of activities, for i = 1, 2,, m. a ij = amount of resource i consumed by each unity of activity j. 39
Standard form maximize Zcx 1 1cx 2 2 cx n n subject to a x a x a x b 11 1 12 2 1n n 1 a x a x a x b 21 1 22 2 2n n 2 a x a x a x b m1 1 m2 2 mn n m and 0, 0,, 0. x1 x2 x n 40
Other possible forms Minimizing rather then maximizing minimize Zcxcx cx n n 1 1 2 2 Some functional constraints are greater-than-or-equal ai1x1ai2x2 ainxn bi, for some values of i Some functional constraints in equation form ai1x1ai2x2 ainxn bi, for some values of i Deleting the non-negativity constraints for some vars x j unestricted in sign for some values of i 41
Allocation of resources to activities Resource Resource Usage per Unit of Activity Activity 1 2 n Amount of Resource available 1 a 11 a 12 a 1n b 1 2 a 21 a 22 a 2n b 2 m a m1 a m2 a mn b m Contribution to Z per unit of activity c 1 c 2 c n 42
Terminology for solutions Solution: any specification of values for the decision variables (x 1, x 2,, x n ). Feasible solution: solution for which all the constraints are satisfied. Infeasible solution: solution for which at least one constraint is violated. Feasible region No feasible solution: no solution satisfies all constraints. 43
Example: no feasible solution If in the Wyndor Glass Co. problem the profit should be above $50000 per week: 44
Terminology for solutions Optimal solution: solution with the most favorable value. Most favorable value: largest value if maximizing or smallest value if minimizing. No optimal solutions: (1) no feasible solutions or, (2) unbounded Z (see next slide). More than one solution: see next slide. 45
Examples of solutions More than one solution Unbounded cost function 46
Corner-point feasible solution Lies in a corner. Corners must have at least one optimal solution. 47
Assumptions of linear programming Proportionality: value of objective function Z is proportional to level of activity x j. Violations: 1) Setup costs 48
Violations of proportionality 2) Increase of marginal return due to economy of scale (longer production runs, quantity discounts, learning-curve effect, etc.) 49
Violations of proportionality 3) Decrease of marginal return due to e.g. marketing costs: to sell more the advertisement grows exponentially. 50
Assumptions of linear programming Additivity: every function in a linear programming model is the sum of the individual contributions. Divisibility: decision variables can have any values. If values are limited to integer values, the problem needs an integer programming model. Certainty: parameters of the model c j, a ij and b i are assumed to be known constants. In real applications parameters have always some degree of uncertainty, and sensitivity analysis must be conducted. 51
Conclusions Linear programming models can be solved using: Spreadsheet as Excel Solver for relatively simple problems Problems with thousands (or even millions!) of decision variables and/or functional constraints must use modeling languages such as: CPLEX/MDL or LINGO/LINDO Matlab Optimization Toolbox Linear programming solves many problems. However, when one or more assumptions are violated seriously, other programming techniques are needed. 52
SIMPLEX METHOD
Simplex method Algebraic procedure with underlying geometric concepts. Definitions: Constraint boundary: line of boundary of the constraint. Corner-point solution Corner-point feasible solution (CFP solution) CFP solutions are adjacent if they share n 1 constraint boundaries. The line segment between two CFP solutions is an edge. 54
Example revisited Wyndor Glass Co. 55
Optimality test Optimality test: consider LP with at least one optimal solution. If a CFP solution has no adjacent CFP solutions that are better, it must be an optimal solution. Adjacent CFP solutions for each CFP solution of Wyndor Glass Co. problem: CFP solution Its adjacent CFP solutions (0, 0) (0, 6) and (4, 0) (0, 6) (2, 6) and (0, 0) (2, 6) (4, 3) and (0, 6) (4, 3) (4, 0) and (2, 6) (4, 0) (0, 0) and (4, 3) 56
Solving the example Initialization: at CPF solution (0, 0). Optimality test: conclude that (0, 0) is not optimal Iteration 1: 1. Choose to move along the edge with faster rate (x 2 ) 2. Stop at new constraint boundary (2x 2 = 12) 3. Determine point of intersection of new set boundaries (0,6) Optimality test: conclude that (0, 6) is not optimal Iteration 2: repeat steps of iteration 1. CFP sol. (2,6) Optimality test: conclude that (2, 6) is optimal. Stop. 57
Steps of simplex algorithm Sequence of CFP solutions: 58
Setting up the simplex method Original form maximize Z 3x 5x 2 1 2 1 2 subject to x 4 1 2x 12 3x 2x 18 and x 0, x 0. 1 2 Augmented form maximize Z 3x 5x subject to 1 2 1 2 1 2 (1) x x 4 (2) 2x x 12 (3) 3x 2x x 18 and x 0, j 1,2,3,4,5. j 3 4 5 Convert functional inequality constraints to equivalent equality constraints using slack variables. 59
Setting up the simplex method Augmented solution: solution with original variables (decision variables) and slack variables. Example: (3, 2, 1, 8, 5) Basic solution: augmented corner-point solution. Example (infeasible solution): (4, 6, 0, 0, 6) Basic feasible (BF) solution: augmented corner-point feasible solution. 60
Basic definitions Example problem has 5 variables and 3 non redundant equations, and 2 (5 3) degrees of freedom. Two variables have an arbitrary value. Simplex uses the zero value for these nonbasic variables. The solution for the other variables (basic variables) is a basic solution. Basic solutions have thus a number of properties. 61
Properties of a basic solution Each variable is basic or nonbasic. Number of basic variables is equal to number of functional constraints (equations). The nonbasic variables are set to zero. Values of basic variables are solving system of equations (functional constraints in augmented form). Set of basic variables are called the basis. If basic variables satisfy the nonnegativity constraints basic solution is a BF solution. 62
Final form maximize Z subject to (0) Z 3x 5x 0 1 2 (1) x x 4 1 3 (2) 2x x 12 2 4 (3) 3x 2x x 18 1 2 5 and x 0, j 1,,5. j 63
Tabular form (simplex tableau) (a) Algebraic form (b) Tabular Form Basic variable Eq. Coefficient of: Z x 1 x 2 x 3 x 4 x 5 Right side (0) Z 3x 1 5x 2 = 0 Z (0) 1-3 -5 0 0 0 0 (1) x 1 + x 3 = 4 x 3 (1) 0 1 0 1 0 0 4 (2) 2x 2 + x 4 = 12 x 4 (2) 0 0 2 0 1 0 12 (3) 3x 1 + 2x 2 + x 5 = 18 x 5 (3) 0 3 2 0 0 1 18 64
Simplex method in tabular form Initialization. Introduce slack variables. Select decision variables to be initial nonbasic variables and slack variables to be initial basic variables. Initial solution of example: (0, 0, 4, 12, 18) Optimality test. Current BF solution is optimal if every coefficient in row zero is nonnegative. If yes, stop. Example: coefficients of x 1 and x 2 are 3 and 5. 65
Simplex method in tabular form Iteration. Step 1: Determine entering basic variable, which is the nonbasic variable with the largest negative coefficient. pivot column entering basic variable Basic variable Eq. Coefficient of: Z x 1 x 2 x 3 x 4 x 5 Right side Z (0) 1-3 -5 0 0 0 0 x 3 (1) 0 1 0 1 0 0 4 x 4 (2) 0 0 2 0 1 0 12 x 5 (3) 0 3 2 0 0 1 18 66
Iteration Step 2: Determine leaving basic variable, by applying minimum ratio test: Basic variable Eq. Coefficient of: Z x 1 x 2 x 3 x 4 x 5 Right side Ratio Z (0) 1-3 -5 0 0 0 0 x 3 (1) 0 1 0 1 0 0 4 x 4 (2) 0 0 2 0 1 0 12 12/2 = 6 minimum x 5 (3) 0 3 2 0 0 1 18 18/2 = 9 pivot number pivot row 67
Iteration Step 3: solve for the new BF solution by using elementary row operations. Iteration Basic variable Eq. Coefficient of: Z x 1 x 2 x 3 x 4 x 5 Right side Z (0) 1-3 -5 0 0 0 0 0 x 3 (1) 0 1 0 1 0 0 4 x 4 (2) 0 0 2 0 1 0 12 x 5 (3) 0 3 2 0 0 1 18 Z (0) 1-3 0 0 2.5 0 30 1 x 3 (1) 0 1 0 1 0 0 4 x 2 (2) 0 0 1 0 0.5 0 6 x 5 (3) 0 3 0 0-1 1 6 68
Iteration 2: Steps 1 and 2 Iteration Basic variable Eq. Coefficient of: Z x 1 x 2 x 3 x 4 x 5 Right side Ratio Z (0) 1-3 0 0 2.5 0 30 1 x 3 (1) 0 1 0 1 0 0 4 4/1 = 4 x 2 (2) 0 0 1 0 0.5 0 6 x 5 (3) 0 3 0 0-1 1 6 6/3 = 2 minimum 69
Complete set of simplex tableau Iteration Basic variable Eq. Coefficient of: Z x 1 x 2 x 3 x 4 x 5 Right side Z (0) 1-3 -5 0 0 0 0 0 x 3 (1) 0 1 0 1 0 0 4 x 4 (2) 0 0 2 0 1 0 12 x 5 (3) 0 3 2 0 0 1 18 Z (0) 1-3 0 0 2.5 0 30 1 x 3 (1) 0 1 0 1 0 0 4 x 2 (2) 0 0 1 0 0.5 0 6 x 5 (3) 0 3 0 0-1 1 6 Z (0) 1 0 0 0 1.5 1 36 2 x 3 (1) 0 0 0 1 1/3-1/3 2 x 2 (2) 0 0 1 0 0.5 0 6 x 1 (3) 0 1 0 0-1/3 1/3 2 70
Other model forms Equality constraints In this case, artificial variables are used. The Big M method is used. Functional constraints in form Surplus variables are used. Minimization Maximize Z. Big M method can have two phases: 1) all artificial variables driven to zero; 2) compute BF solutions. 71
Postoptimality analysis Reoptimization: when the problem changes slightly, new solution is derived from the final simplex tableau. Shadow price for resource i (denoted y i* ) is the marginal value of i. i.e. the rate at which Z can be increased by slightly increasing the amount of this resource b i. Example: recall b i values of Wyndor Glass problem. The final tableau gives: y 1* = 0, y 2* = 1.5, y 3* = 1. 72
Graphical check Increasing b 2 in 1 unit, the new profit is 37.5 73
Shadow prices Increasing b 2 in 1 unit increases the profit in $1500. Should this be done? Depends on the marginal profitability of other products using that hour time. Shadow prices are part of the sensitivity analysis. Constraint on resource 1 is not binding the optimal solution. Resources as this are free goods. Constraints on resources 2 and 3 are binding constraints. Resources are scarce goods. 74
Sensitivity analysis Identify sensitive parameters (those that cannot be changed without changing optimal solution). Parameters b i can be analyzed using shadow prices. Parameters c i for Wyndor example (see figure in next slide). Parameters a ij can also be analyzed graphically. For problems with a large number of variables usually only b i and c i are analyzed, because a ij are determined by the technology being used. 75
Sensitivity analysis of parameters c i Graphical analysis of Wyndor example: c 1 = 3 can have values in [0, 7.5] without changing optimal c 2 = 5 can have any value greater than 2 without changing optimal 76
Interior-point approach Discovery made in 1984 by Narendra Karmarkar. Used in huge linear programming problems, beyond the scope of the simplex method. Starts by identifying a feasible trial solution, moving a better one until it reaches the optimal trial solution. Trial solutions are interior points. Interior-point or barrier algorithms (each constraint boundary is treated as a barrier). 77
Example The Wyndor Glass Co. example reaches the solution in 15 iterations for changes smaller than 0.0001 78
Theory of simplex method Recall the standard form: maximize Zcx 1 1cx 2 2 cx n n subject to a x a x a x b 11 1 12 2 1n n 1 a x a x a x b 21 1 22 2 2n n 2 a x a x a x b m1 1 m2 2 mn n m x1 x2 x n and 0, 0,, 0. 79
Augmented form Z c x c x c x c x maximize 1 1 n n n1 n1 nm nm subject to a x a x a x x b 11 1 12 2 1n n n1 1 a x a x a x x b 21 1 22 2 2n n n2 2 a x a x a x x b m1 1 m2 2 mn n nm m and x 0, j 1,2,, m. j 80
Matrix form (standard form) maximize Z cx subject to Axb and x0 c [,,, ], c1 c2 c n x x x x 1 2 n, b b b b 1 2 m, 0 0 0, 0 A a a a a a a a a a 11 12 1n 21 22 2n m1 m2 mn. 81
Matrix augmented form Introduce the column vector of slack variables: x s x x x n1 n2 nm The constraints are (with I (mm) and 0 (1(n+m)) ): x x [ A, I] b and 0 xs xs 82
Solving for a BF solution One iteration of simplex: The n nonbasic variables are set to zero. The m basic variables is a column vector denoted as x B. The basic matrix B is obtained from [A, I], by eliminating the columns with coefficients of nonbasic variables. The problem becomes: Bx B = b. The solution for basic variables and the value of the objective function for this basic solution are: x B b and Z c x c B b 1 1 B B B B 83
Revised simplex method 1. Initialization: (as original) Introduce slack variables x s. Decision variables are initial nonbasic variables. 2. Iteration: Step 1: (as original) Determine entering basic variable (nonbasic variable with the largest negative coefficient). Step 2: Determine leaving basic variable. As original, but computations are simplified. Step 3: Determine new BF solution: set x B = B 1 b. 3. Optimality test: solution is optimal if every coefficient in Z is 0. Computations are simplified. 84
DUALITY THEORY AND SENSITIVITY ANALYSIS
Duality theory Every linear programming problem (primal) has a dual problem. Most problem parameters are estimates, others (as resource amounts) represent managerial decisions. The choice of parameter values is made based on a sensitivity analysis. Interpretation and implementation of sensitivity analysis are key uses of duality theory. 86
Primal and dual problems Primal problem Dual problem maximize Z subject to n j1 n cx j j1 a x b, for i 1,2,, m ij j i and x 0 for j 1,2,, n. j j minimize W subject to m i1 m by i i1 a y c, for j 1,2,, n ij i j and y 0 for i 1,2,, m. i i 87
Duality theory Relations between parameters: The coefficients of the objective functions of the primal are the right-hand sides of the functional constraints in the dual. The right-hand side of the functional constraints in the primal are coefficients of the objective functions in the dual. The coefficients of a variable in the functional constraints of the primal are the coefficients in the functional constraints of the dual. 88
Primal-dual table Primal problem Coefficient of: x 1 x 2 x n Right side Coefficient of: y 1 a 11 a 12 a 1n b 1 y 2 a 21 a 22 a 2n b 2 y m a n1 a n2 a mn b m Right side Coefficients of objective function (minimize) Dual problem c 1 c 2 c n Coefficients of objective function (maximize) 89
Example: Wyndor Glass Co. Primal problem Maximize Z subject to x x 2 1 3 5, 1 0 4 x1 0 2 12 x 2 3 2 18 x 0 x 2 0 1 and. Dual problem 4 Minimize Z y 1 y2 y 3 12, subject to 18 y y y 0 2 3 5 1 2 3 10 32 y y y and 0 0 0. 1 2 3 90
Example: Wyndor Glass Co. Tableau representation x 1 x 2 y 1 1 0 4 y 2 0 2 12 y 3 3 2 18 3 5 91
Primal-dual relationships Weak duality property: If x is a feasible solution for the primal problem and y is a feasible solution for the dual problem, then cx yb Strong duality property: If x * is an optimal solution for the primal problem and y * is an optimal solution for the dual problem, then cx * = y * b 92
Primal-dual relationships Complementary solutions property: At each iteration, the simplex method identifies a CFP solution x for the primal problem and a complementary solution for the dual problem, where cx = yb If x is not optimal for the primal problem, then y is not feasible for the dual problem. 93
Primal-dual relationships Complementary optimal solutions property: At the final iteration, the simplex method identifies an optimal solution x* for the primal problem and a complementary optimal solution y* for the dual problem, where cx* = y*b The y i * are the shadow prices for the primal problem. 94
Primal-dual relationships Symmetry property: For any primal problem and its dual problem, all relationships between them must be symmetric because the dual of this dual problem is this primal problem. Duality theorem: The following are the only possible relationships between the primal and dual problems: 95
Primal-dual relationships 1. If one problem has feasible solutions and a bounded objective function, then so does the other problem, so both the weak and strong duality properties are applicable. 2. If one problem has feasible solutions and an unbounded objective function, then the other problem has no feasible solution. 3. If one problem has no feasible solutions, then the other problem has either no feasible solutions or an unbounded objective function. 96
Primal-dual relationships Relationships between complementary basic solutions Primal basic solution Complementary dual basic solution Both basic solutions Primal feasible? Dual feasible? Suboptimal Superoptimal Yes No Optimal Optimal Yes Yes Superoptimal Suboptimal No Yes Neither feasible nor superoptimal Neither feasible nor superoptimal No No 97
Primal-dual relationships 98
Applications of duality theory If the number of functional constraints m is bigger than the number of variables n, applying simplex directly to the dual problem will achieve a substantial reduction in computational effort. Evaluating a proposed solution for the primal problem. i) if cx = yb, x and y must be optimal even without applying simplex; ii) if cx < yb, yb provides an upper bound on the optimal value of Z. Economic interpretation as before 99
Duality theory in sensitivity analysis Sensitivity analysis investigate the effect of changing parameters a ij, b i, c j in the optimal solution. Can be easier to study these effects in the dual problem Optimal solution of the dual problem are the shadow prices of the primal problem. Changes in coefficients of a nonbasic variable. Only changes one constraint in the dual problem. If this constraint is satisfied the solution is still optimal. Introducing a new variable in the objective function. Only introduces a new constraint in the dual problem. If the dual problem is still feasible, solution is still optimal. 100
Essence of sensitivity analysis LP assumes that all the parameters of the model (a ij, b i and c j ) are known constants. Actually: The parameters values used are just estimates based on a prediction of future conditions; The data may represent deliberate overestimates or underestimates to protect the interests of the estimators. An optimal solution is optimal only with respect to the specific model being used to represent the real problem fi sensitivity analysis is crucial! 101
Simplex tableau with parameter changes What changes in the simplex tableau if changes are made in the model parameters, namely bfi b, cfi c, Afi A? New initial tableau Coefficient of: Eq. Z Original variables Slack variables Right side (0) 1 c 0 0 (1,2,,m) 0 A I b Revised final tableau (0) 1 z* cy* Ac y* (1,2,,m) 0 A* S* A S* Z* y* b b* S* b 102
Applying sensitivity analysis Changes in b: Allowable range: range of values for which the current optimal BF solution remains feasible (find range of b i such that b* S* b0, assuming this is the only change in the model). The shadow price for b i remains valid if b i stays within this interval. The 100% rule for simultaneous changes in right-hand side: the shadow prices remain valid as long as the changes are not too large. If the sum of the % changes does not exceed 100%, the shadow prices will still be valid. 103
Applying sensitivity analysis Changes in coefficients of nonbasic variable: Allowable range to stay optimal: range of values over which the current optimal solution remains optimal (find c j y* A j, assuming this is the only change in the model). The 100% rule for simultaneous changes in objective function coefficients: if the sum of the % changes does not exceed 100%, the original optimal solution will still be valid. Introduction of a new variable: Same as above. 104
Other simplex algorithms Dual simplex method Modification useful for sensitivity analysis Parametric linear programming Extension for systematic sensitivity analysis Upper bound technique A simplex version for dealing with decision variables having upper bounds: x j u j, where u j is the maximum feasible value of x j. 105