LP Geometry: outline. A general LP. minimize x c T x s.t. a T i. x b i, i 2 M 1 a T i x = b i, i 2 M 3 x j 0, j 2 N 1. where

Transcription

1 LP Geometry: outline I Polyhedra I Extreme points, vertices, basic feasible solutions I Degeneracy I Existence of extreme points I Optimality of extreme points IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 46 A general LP where minimize x c T x s.t. x b i, i 2 M 1 x apple b i, i 2 M 2 x = b i, i 2 M 3 x j 0, j 2 N 1 x j apple 0, j 2 N 2 I x 2< n decision/optimization variables, and c 2< n I N 1, N 2 {1,...,n} and M 1, M 2, M 3 finite index sets I a i 2< n, b 2 i 2<for each i 2 M 1 [ M 2 [ M 3 a T Notation: A = For analysis, usually consider (wolog) LPs a T m in the form minimize c T x s.t. Ax b or in standard form minimize c T x s.t. Ax = b, x 0 IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 47

2 Recall: A few terms of the trade minimize c T x s.t. Ax b I x 1,...,x n decision variables I Vector x =(x 1,...,x n ) T satisfying all the constraints feasible solution, orfeasible vector, orfeasible point I Set of all feasible solutions feasible region, or feasible set I If no feasible vectors exist, we are facing an infeasible problem I The function c T x objective, orcost, function I A feasible solution x? that minimizes the objective function over the feasible region optimal solution I The value c T x? optimal cost I If for every K 2<there exists a feasible x such that c T x < K, the optimal cost is 1, orunbounded below (or the problem is unbounded). IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 48 Possible solution outcomes for an LP I The LP is infeasible (feasible region is empty) I There exists a unique optimal solution I There exist multiple optimal solutions; in this case, the set of optimal solutions can be either bounded or unbounded 4 I The optimal cost is 1, and no feasible solution is optimal Another situation, possible in other optimization problems: the optimal cost is bounded, but an optimal solution does not exist, does not arise in linear programming. 4 AsetS 2< n is bounded if 9K such that the absolute value of every component of every element of S is less than or equal to K. IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 49

3 Hyperplanes, halfspaces, and polyhedra Definitions: I Let 0 6= a 2< n and let b be a scalar. I The set {x 2< n : a T x = b} is called a hyperplane I The set {x 2< n : a T x b} is called a halfspace I A polyhedron is a set that can be described in the form {x 2< n : Ax b}, wherea is an m n matrix and b is a vector in < m. I Geometrically, a polyhedron is an intersection of a finite number of halfspaces IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 50 Convex sets Definitions: I AsetS < n is convex if for any x, y 2 S, andany 2 [0, 1], we have x +(1 )y 2 S. I Let x 1,...,x k be vectors in < n. I Let 1,..., k be nonnegative scalars whose sum is 1. The vector P k i=1 ix i is a convex combination of the vectors x 1,...,x k. I The convex hull of the vectors x 1,...,x k is the set of all convex combinations of these vectors. Theorem 2.1 I The intersection of convex sets is convex I Every polyhedron is a convex set I A convex combination of a finite number of elements of a convex set also belongs to that set I The convex hull of a finite number of vectors is a convex set IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 51

4 Extreme points and vertices Definitions: I Let P < n be a polyhedron. A vector x 2 P is an extreme point of P if we cannot find two vectors y, z 2 P, both di erent from x, andascalar 2 [0, 1], such that x = y +(1 )z. I Let P be a polyhedron. A vector x 2 P is a vertex of P if there exists some c 2< n such that c T x < c T y for all y 2 P and y 6= x. The above are geometric definitions, hard to work with from an algorithmic point of view. To develop an equivalent algebraic definition, need to recall a few linear algebra tools. IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 52 Linear algebra tools: Linear dependence/independence I Linear combination of vectors x 1,...,x K :avector y = P K k=1 kx k for some numbers k 2<, k =1,...,K I Vectors x 1,...,x K 2< n are linearly dependent if some linear combination of them (with some of k s non-zero) evaluates to 0. I.e.,vectors are linearly independent if system of equations P K k=1 kx k = 0 has a unique solution: = 0 Theorem 1.2 Let D be a square matrix. The following statements are equivalent: (a) The matrix D is invertible (b) The matrix D T is invertible (c) The determinant of D is nonzero (d) The rows of D are linearly independent (e) The columns of D are linearly independent (f) 8d, thelinearsystemdx = d has a unique solution (g) 9d such that the linear system Dx = d has a unique solution IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 53

5 Towards algebraic representation of a corner point Polyhedron P 2< n can be represented by x b i, i 2 M 1 x apple b i, i 2 M 2 x = b i, i 2 M 3, where M 1, M 2, M 3 are finite index sets. Definition: I If x satisfies x = b i for some i 2 M 1 [ M 2 [ M 3,the corresponding constraint is active, or binding, at x. IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 54 Basic solutions P = {x : x b i, i 2 M 1, x apple b i, i 2 M 2, x = b i, i 2 M 3 } Definition Consider a polyhedron P defined by linear equality and inequality constraints, and let x 2< n. I x is a basic solution if: I All equality constraints are active (i.e., satisfied) I Among the constraints that are active at x, thereexistn that are linearly independent. I If x is a basic solution that satisfies all of the constraints, we say it is a basic feasible solution. IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 55

6 Basic Solutions: Representation-dependence and Degeneracy I Notion of a Basic Solution is representation-dependent I E.g., standard vs. general form: vs. P = {x : Ax = b, x 0} P = {x : Ax b, Ax b, x 0} I Definition: A basic solution x 2< n is said to be degenerate if more than n of the constraints are active at x. I Degeneracy is not a purely geometric property: I Consider point (0, 0, 1) in {x : x 1 x 2 =0, x 1 + x 2 + x 3 =2, x 1, x 2, x 3 0} vs. {x : x 1 x 2 =0, x 1 + x 2 + x 3 =2, x 1, x 3 0} IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 56 Equivalence Theorem 2.3 Let P be a nonempty polyhedron and let x 2 P. Then,the following are equivalent: I x is a vertex I x is an extreme point I x is a basic feasible solution. IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 57

7 Linear algebra tools: Subspaces and bases Definitions: I ;6= S < n is a subspace of < n if for all x, y 2 S,, 2<, x + y 2 S. (Must contain 0!) I y = P K k=1 kx k,where k s are real numbers, is a linear combination of x 1,...,x K. Set of all linear combinations of these vectors is their span it is a subspace I Given a subspace {0} 6= S < n,abasis of S is a collection of linearly independent vectors whose span is equal to S. I Every basis of S has the same size, which is called the dimension of S (dimension of < n is n, dimension of {0} is 0) Theorem 1.3 Suppose that the span S of x 1,...,x K has dimension m. Then: I 9 a basis of S consisting of m of the vectors x 1,...,x K, I If k apple m and x 1,...,x k are linearly independent, we can form a basis of S by starting with x 1,...,x k and choosing m k of the remaining vectors. IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 58 Understanding basic solutions Recall: in a polyhedron defined by linear equality and inequality constraints, x is a basic solution if all equality constraints are active and, out of the constraints that are active at x, therearen of them that are linearly independent. Theorem 2.2 Let x 2< n,andleti = {i : x = b i }. Then the following are equivalent: 1. There exist n vectors in the set {a i : i 2 I } which are linearly independent (slang: constraints are linearly independent ) 2. The span of the vectors a i, i 2 I is all of < n,thatis,every vector in < n can be expressed as a linear combination of the vectors a i, i 2 I 3. The system of equations x = b i, i 2 I,hasaunique solution. Proof: 1 () 2, 2 () 3 IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 59

8 Equivalence Theorem 2.3 Let P be a nonempty polyhedron and let x 2 P. Then,the following are equivalent: I x is a vertex I x is an extreme point I x is a basic feasible solution. Proof: Vertex ) Extreme point ) Basic feasible solution ) Vertex Corollary 2.1 Given a finite number of linear inequality constraints, there can only be a finite number of basic or basic feasible solutions. IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 60 Existence of extreme points Definition 2.12 I A polyhedron P 2< n contains a line if 9x 2 P and 9d 2< n, d 6= 0 such that x + d 2 P 8 2<. Theorem 2.6 Suppose P = {x : Ax b} 6= ;. Then the following are equivalent: (a) P has at least one extreme point (b) P does not contain a line (c) There exist n linearly independent vectors out of the family a 1,...,a m Proof: (b))(a))(c))(b) Terminology: x 2 P has rank k if there are k and only k linearly independent constraints active at x. Corollary 2.2 Every nonempty bounded polyhedron, and every polyhedron in standard form (P = {x 0, Ax = b}) has at least one BFS. IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 61

9 Optimality of extreme points Theorem 2.7 Consider the LP minimize c T x s.t. x 2 P. Suppose that P has at least one extreme point and that there exists an optimal solution. Then, there exists an optimal solution which is an extreme point of P. Theorem 2.8 Consider the LP minimize c T x s.t. x 2 P. Suppose that P has at least one extreme point. Then, either the optimal cost is 1, or there exists an extreme point which is optimal. Corollary 2.3 Consider the LP minimize c T x s.t. x 2 P, P 6= ;. Then,eitherthe optimal cost is 1, or there exists an optimal solution. IOE 610: LP II, Fall 2013 Geometry of Linear Programming Page 62