Linear Programming Terminology The carpenter problem is an example of a linear program. T and B (the number of tables and bookcases to produce weekly) are decision variables. The profit function is an objective function. The side conditions are called constraints. The feasible region is the region containing points satisfying all of the constraints. An extreme point is a vertex of the feasible region. 1
Question: What makes the carpenter problem a linear program? Answer: Whenever a decision variable appears in the objective function or a constraint, it must appear linearly, that is, it must appear only as a power term with an exponent of 1, possibly multiplied by a constant (for example, 25T + 30B is OK, but 25T 2 and 25BT are not). The objective function is unique (there is only one quantity to optimize). The decision variables are permitted to assume real values (they are not restricted to integer values). 2
Observations about the carpenter problem The optimal solution occurred at one of the extreme points (vertices) of the feasible region. The feasible region was a convex set (a set is not convex if you can draw a line segment joining two points in the set which does not lie entirely within the set). These observations hold in general, and have been exploited to develop methods (for example, the simplex method) to solve linear programs efficiently. 3
Comment on the number of solutions for a linear program For the carpenter problem, we found a unique solution. It is possible for a linear program to have no solutions, or an infinite number of solutions. A linear program has no solutions when the feasible region is empty (in that case, the constraints are inconsistent) or when the feasible region is unbounded. Exercise: Tweak the numbers in the carpenter problem so that it has an infinite number of solutions instead of a unique solution. 4
In general (that is, for linear programs with a non-empty and bounded feasible region), the optimal solution for a linear program occurs at one of the extreme points of the feasible region. This suggests the following solution procedure: 1. Find all intersection points of all constraints. 2. Reject all intersection points that are not feasible. 3. For all remaining intersection points, namely those that are extreme points of the feasible region, evaluate the objective function. 4. Choose the extreme point(s) with the largest/smallest value for the objective function. 5
The above procedure is easy to follow when the number of decision variables is 2. In that case, such as for the carpenter problem, we can graph the feasible region, and visually decide whether a particular point of intersection is feasible or not. However, it is not immediately obvious how to do this when there are more than 2 decision variables. In this case, such as for the horse-feeding problem, the feasible region is in R n, where n is the number of decision variables. What is needed is an algebraic procedure to decide whether a point of intersection is feasible or not... 6
Slack variables The idea of a slack variable is to turn an inequality constraint into an equality constraint. Let y i be the slack variable for the i th constraint (not including the non-negativity constraints). Add/Subtract the slack variable to/from the constraint in such a way (and replace the inequality to an equality) that the value of y i means the following: If y i > 0, the i th constraint is satisfied with room to spare (with slack). If y i = 0, the i th constraint is just satisfied. If y i < 0, the i th constraint is violated. That is, y i will measure the extent to which the i th constraint is satisfied. 7
Slack variables & number of intersection points Let the original linear program have m decision variables and n constraints. Then the number of intersection points to be considered is ( ) n n! = m m!(n m)!. For the carpenter problem, m = 2 and n = 4, so the number of intersection points to be considered was 4! 2!2! = 24 4 = 6. For the horse-feeding problem, m = 4 and n = 7, giving 35 points of intersection to be considered. Using slack variables, this can be done systematically, but it s tedious! A more efficient method is needed... 8
The Simplex Method for Linear Programs Efficient method of solution that does not require examination of all points of intersection. Algorithm for moving from one feasible extreme point to another in such a way that the objective function is always improved (never worse at the least). The algorithm will find optimal solutions whenever they exist. 9
10
Logic of the Simplex Method 1. As soon as you arrive at a new corner of the feasible region, adjust the dictionary. That is, rewrite the constraints so that the non-corner variables and the objective function are expressed in terms of the corner variables. 2. Examine the objective function as expressed in the current dictionary. Suppose that V 1,..., V n are the current corner variables, and that the objective function now reads as P = P 0 + k 1 V 1 +... + k n V n. Then: (a) If all the k i s are strictly less than zero, terminate the algorithm. 11
(b) If none of the k i s are strictly positive, but some k i = 0, then we are at an optimal corner, but there may exist other optimal solutions. Terminate the algorithm. (c) If some of the k i s are positive, then increase any variable V i whose coefficient k i is the largest. Increase V i to the maximal allowable extent (that is, V i loses status as corner variable; it is referred to as the entering variable). In doing so, some non-corner variable will necessarily fall to zero (that is, it will gain status as corner variable; it is referred to as the exiting variable). In this way, you are moving to a new corner of the feasible region. 3. Go back to step 1, until the algorithm terminates.
Tableau Notation for the Simplex Method The tableau for corner 1 of the carpenter problem is as follows: t b y 1 y 2 RHS 20 30 1 0 60 5 4 0 1 12 25 30 0 0 P-160 Objective row: Last row, representing the objective function. Basic rows: All other rows, representing a constraint each. Corner variables: Those with non-elementary columns (here, t and b). Non-corner variables: Those with elementary columns (here, y 1 and y 2 ). 12
The Simplex Method using Tableaus To move from one feasible corner to another, we proceed as follows: Pick a corner of the feasible region to start at (usually the trivial corner), and make sure that the objective function is expressed solely in terms of the corresponding corner variables. Adjust the objective row if necessary. 1. Examine the objective row, and select the largest positive coefficient (assuming we re maximizing the objective function). Place a vertical arrow under this coefficient. The corresponding column is called the pivot column, and the 13
corresponding variable is the entering variable (this variable will lose corner-variable status). If there are no positive coefficients, you re done. 2. Determine the maximal possible increase in the entering variables as follows. Consider the entries in the basic rows of the pivot column. Divide each such entry into the right-hand entry in the same row, and record the coefficients obtained. Choose the row determined by the smallest positive coefficient and place a horizontal arrow next to this row. This row is called the pivot row. The corresponding variable (the one with entry 1) is called the exiting variable (it will gain corner-variable status). 3. Determine the new corner variables (old corner variables - entering variable + exiting variable).
4. Pivot. That is, perform elementary row operations (cf. linear algebra) to transform the dictionary at the old corner to the dictionary at the new corner. The pivot entry (the one entry that is in both the pivot column and the pivot row) must become 1, and all other entries in the pivot column must become 0. 5. Go back to step 1...