Linear Programming What s Linear Programming? Often your try is to maximize or minimize an objective within given constraints A linear programming problem can be expressed as a linear function of certain variables and the constraints can be expressed as equalities or inequalities on those variables Linear programming could be a minimization linear program or a maximization linear program 2
Election Spending Problem You re running for office 100K urban voters, 200K suburban, and 50K rural voters are your target You want to win over at least 50% of voters in each area There s a catch. These voters may be persuaded differently by advertisements on different issues 3 Election Spending Problem The effect of policies (and hence advertising on those policies) on thousands of voters who can be won over (or lost) by spending $1,000 on particular issue. Policy urban suburban rural building roads -2 5 3 gun control 8 2-5 farm subsidies 0 0 10 gasoline tax 10 0-2 4
Evolving the Linear Program Rather than trial and error, you can formulate the problem Let x 1 be the number of thousands of dollars spent on advertising on building roads, x 2... on gun control, x 3... on farm subsidies, and x 4... on gasoline tax (for public transit improvements) You can formulate the requirement to win half urban voters (50k out of 100k voters) as -2x 1 + 8x 2 + 0x 3 + 10x 4 >= 50...(similarly for other areas) You have to ensure x i >= 0 (as spending can t be negative) and minimize x 1 + x 2 + x 3 + x 4 5 Linear Program for Election $ Minimize x 1 + x 2 + x 3 + x 4 subject to -2x 1 + 8x 2 + 0x 3 + 10x 4 >= 50 5x 1 + 2x 2 + 0x 3 + 0x 4 >= 100 3x 1-5x 2 + 10x 3-2x 4 >= 25 x 1, x 2, x 3, x 4 >= 0 Solution to the linear program yields the optimal strategy. 6
General Linear Programs Optimize a linear function subject to a set of linear inequalities Given a set of real numbers a 1, a 2,..., a n and a set of variables x 1, x 2,..., x n, we define a linear function f on those variables: f(x 1, x 2,..., x n ) = a 1 x 1 + a 2 x 2 +... + a n x n =! 1<=i<=n a i x i plus linear constraints: either a linear equality f(x 1, x 2,..., x n ) = b or linear inequalities f(x 1, x 2,..., x n ) <= b or f(x 1, x 2,..., x n ) >= b. 7 Canonical Forms of LP Standard Maximization of a linear function subject to linear inequalities Slack Maximization of a linear function subject to linear equalities 8
Feasible Solution A linear program has n variables subject to a set of m linear inequalities A feasible solution is any values for variables that satisfy all the constraints Feasible region is set of all feasible solutions The maximization function is called the objective function and the value of this function at particular point in the feasible region an objective value There may be infinite number of points in a feasible region, so we need a way to practically find the optimal solution for the objective value 9 Solving Two Variables You can plot the constraints on a cartesian coordinate and find an optimal solution for the objective value Optimal Solution f 1 f 2 Objective function f 3 The optimal solution is the vertex of the feasible region If it is a line, you can use end points of the line segment 10
Solving Multiple Variables Each constrain forms a half-space in n-th dimensional space Feasible region formed by the intersection of these half-spaces is called simplex The objective function is a hyperplane and the optimal solution occurs at the vertex of the simplex The simplex algorithm moves through the exterior of the simplex, determining the vertex with maximum value that satisfy the objective function 11 The Simplex Algorithm Iterative algorithm Set each nonbasic variable to 0 and compute the values of the basic variables from the equality constraints In each iteration you convert from one slack form to another slack form You improve the objective value in each iteration We increase values (subject to constraints) of nonbasic variables from 0 in order to maximize the objective value You convert the equations into equivalent forms until the objective function has all variables with negative coefficient 12
Example of Simplex Alg Given the linear program in standard form maximize 3x 1 + x 2 + 2x 3 subject to x 1 + x 2 + 3x 3 <= 30 2x 1 + 2x 2 + 5x 3 <= 24 4x 1 + x 2 + 2x 3 <= 36 x 1, x 2, x 3 >= 0 13 Convert to Slack Form z = 3x 1 + x 2 + 2x 3 x 4 = 30 - x 1 - x 2-3x 3 x 5 = 24-2x 1-2x 2-5x 3 x 6 = 36-4x 1 - x 2-2x 3 Nonbasic variables on the right, basic variables on the left 14
z = 3x 1 + x 2 + 2x 3 Giving Values x 4 = 30 - x 1 - x 2-3x 3 x 5 = 24-2x 1-2x 2-5x 3 x 6 = 36-4x 1 - x 2-2x 3 Giving values of 0 to nonbasic variables, we have (0, 0, 0, 30, 24, 36) and z gets a value of 0. Now we want to reformulate so the basic solution has a greater objective value. 15 z = 3x 1 + x 2 + 2x 3 Reformulating x 4 = 30 - x 1 - x 2-3x 3 x 5 = 24-2x 1-2x 2-5x 3 x 6 = 36-4x 1 - x 2-2x 3 Take a nonbasic variable whose coefficient in the objective function is positive and increase the value as much as possible without violating constraints. This variable becomes basic and some other variable becomes nonbasic. 16
z = 3x 1 + x 2 + 2x 3 x 4 = 30 - x 1 - x 2-3x 3 x 5 = 24-2x 1-2x 2-5x 3 x 6 = 36-4x 1 - x 2-2x 3 Reformulating Increasing x 1 will result in decreasing x 4, x 5, and x 6. However, non of them can become negative due to constraints. x 4, x 5, and x 6, restrict x 1 to 30, 12, and 9, respectively. The third constraint is the most limiting. We switch roles of x 1 and x 6. 17 Pivoting z = 3x 1 + x 2 + 2x 3 x 4 = 30 - x 1 - x 2-3x 3 x 5 = 24-2x 1-2x 2-5x 3 x 1 = 9 - x 2 /4 - x 3 /2 - x 6 /4 You pivot that is you chose an entering variable (x 1 ) and a leaving variable (x 6 ) and exchange their roles. You now replace all x 1 18
Pivoting z = 27 + x 2 /4 + x 3 /2-3x 6 /4 x 1 = 9 - x 2 /4 - x 3 /2 - x 6 /4 x 4 = 21-3x 2 /4-5x 3 /2 + x 6 /4 x 5 = 6-3x 2 /2-4x 3 + x 6 /2 You re ready to pick the next pivot, an entering variable with positive coefficient in the objective function and a corresponding leaving variable. Let s chose x 3 and that will lead to the tightest function and hence the leaving variable x 5. 19 Pivoting z = 111/4 + x 2 /16 - x 5 /8-11x 6 /16 x 1 = 33/4 - x 2 /16 + x 5 /8-5x 6 /16 x 3 = 3/2-3x 2 /8 - x 5 /4 + x 6 /8 x 4 = 69/4 + 3x 2 /16 + 5x 5 /8 - x 6 /16 The lone standing positive coefficient nonbasic variable in objective function is x 2 Using that as a pivot,... 20
Pivoting z = 28 - x 3 /6 - x 5 /6-2x 6 /3 x 1 = 8 + x 3 /6 + x 5 /6 - x 6 /3 x 2 = 4-8x 3 /3-2x 5 /3 + x 6 /3 x 4 = 18 - x 3 /2 + x 5 /2 Giving zero values to the nonbasic variables, we get x 1 = 8, x 2 = 4, x 3 = 0, and the optimal solution z = 28. 21 Slack Variables maximize 3x 1 + x 2 + 2x 3 x 1 + x 2 + 3x 3 <= 30 12 <= 30, slack is 18 which is x 4 2x 1 + 2x 2 + 5x 3 <= 24 24 <= 24, slack is 0 which is x 5 4x 1 + x 2 + 2x 3 <= 36 36 <= 36, slack is 0 which is x 6 x 1, x 2, x 3 >= 0 Solution x 1 = 8, x 2 = 4, x 3 = 0, and the optimal solution z = 28. x 4, x 5, and x 6 are the slack variable. The slacks are 18, 0, and 0. If you plug in values for x 1, x 2, and x 3 into original inequalities, you can see the slack. 22
Pivot 23 Simplex Algorigthm 24
Issues to consider It is possible that the feasible region is unbounded It is possible that the iteration leaves the objective value unchanged known as degeneracy You need to look for cycles where the slack forms simply cycle through transformations Since Simplex algorithm is deterministic, it will cycle over the same sequence You can determine it cycles if it does not terminate in at most ( ) iterations n+m m 25