Linear Programming and its Applications

Linear Programming and its Applications

Outline for Today What is linear programming (LP)? Examples Formal definition Geometric intuition Why is LP useful? A first look at LP algorithms Duality

Linear Programming Example Maximize x1 + 6x2 Subject to x1 200 x2 300 x1 + x2 400 x1 0 x2 0

Linear Programming: General Formulation

Linear Programming: General Formulation Notes Inputs: a m x n real-valued matrix A, and vectors c in R n and b in R m Output: x We adopt the convention that vectors are columns. The number of variables is n, and the number of constraints is m There is one constraint per row of A

LP Variants There are many variants, all of which can easily be converted to canonical form: 1. The objective function can be maximized or minimize 2. Constraints can be equalities or inequalities 3. Variables need not be negative

Geometry of the Feasible Solution Space Two dimensions example p p p Feasible region The feasible region for a three-dimensional problem is a polygon Dasgupta et al.

Geometry of the Feasible Solution Space Two dimensions example p p p Feasible region Dotted lines show contour lines of the objective function Dasgupta et al.

Geometry of the Feasible Solution Space Two dimensions Erickson

Geometry of the Feasible Solution Space Two dimensions example p p p Feasible region Dotted lines show contour lines of the objective function Dasgupta et al.

Geometry of the Feasible Solution Space Three dimensions example The feasible region for a threedimensional problem is a polyhedron Dasgupta et al.

Geometry of the Feasible Solution Space Three dimensions example Each constraint corresponds to a plane that divides the polyhedron onto half-spaces The feasible region is the intersection of half-spaces (seven here) Exercise: Which inequality corresponds to each face of the polyhedron? Dasgupta et al.

Geometry of the Feasible Solution Space Three dimensions Moore and Mertens

Geometry of the Feasible Solution Space Three dimensions Moore and Mertens

Geometry of the Feasible Solution Space Observations With n variables, the feasible region is an n- dimensional convex polytope The set of values satisfying a given constraint corresponds to an (n-1)-dimensional hyperplane, or facet The vertices of the polytope are points at the intersection of at least n facets An edge connects two vertices, and is the intersection of n-1 facets The objective function is also an (n-1)-dimensional hyperplane

Geometry of the Feasible Solution Space Observations An optimum solution to a linear program always lies at a vertex of the feasible region, except possibly when the program has no solution (the feasible region is empty) the program has an infinite number of solutions (the feasible region is unbounded)

Geometry of the Feasible Solution Space Observations There may be more than one optimum solution The number of vertices of the feasible region (polytope) can grow exponentially with n

Why is Linear Programming Useful? It provides a natural and very general way to formulate optimization problems Exercise: formulate Max Flow as a linear program

Algorithms for LP: First try

Algorithms for LP: First try There are m constraints, so the feasible polytope has at most m facets Since each vertex is the intersection of (at least) n facets, there are at most (m choose n) vertices. Algorithm: Find each vertex v by taking the intersection of the hyperplanes corresponding to n constraints Check if v is feasible: does it satisfy all constraints? Compute c T v for each feasible v, and return the maximum

Algorithms for LP: First try There are m constraints, so the feasible polytope has at most m facets Since each vertex is the intersection of (at least) n facets, there are at most (m choose n) vertices. Algorithm: Find each vertex v by taking the intersection of the hyperplanes corresponding to n constraints Check if v is feasible: does it satisfy all constraints? Compute c T v for each feasible v, and return the maximum

Algorithms for LP: Simplex (informal)

Algorithms for LP: Simplex (informal)

Algorithms for LP: Simplex Let v be any vertex of the feasible region While there is a neighbor vʹ of v with better objective value: Set v = vʹ

History 1939: Russian mathematician Leonid Kantorovich first described the idea of formulating planning problems as linear programs Late 1950s: Linear programming was established as a major field of mathematical economics 1975: Kantorovich shared the Nobel Prize in economics with T. C. Koopmans, who coined the term linear programming. George B. Dantzig, who independently discovered linear programming in 1947 invented the simplex algorithm 2000: The simplex method was listed among the ten algorithms with the greatest influence on the development and practice of science and engineering in the 20 th century Moore and Mertens

Duality A way to verify that a feasible LP solution is optimal Dasgupta et al.

Duality Exercise: Can you find a certificate of optimality for our first example? Maximize x1 + 6x2 Subject to x1 200 x2 300 x1 + x2 400 x1 0 x2 0 Dasgupta et al.

Duality Dasgupta et al.

Duality Duality theorem If a linear program has a bounded optimum, then so does its dual, and the two optimum values coincide. Dasgupta et al.