DM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini

Transcription

1 DM545 Linear and Integer Programming Lecture 2 The Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark

2 Outline Standard Form Basic Feasible Solutions Algorithm 2

3 Mathematical Model Graphical Representation: Machines/Materials A and B Products 1 and 2 x 2 max 6x 1 + 8x 2 5x x x 1 + 4x 2 40 x 1 0 x 2 0 6x 1 + 8x 2 = 16 5x x 2 60 x 1 4x 1 + 4x

4 Outline Standard Form Basic Feasible Solutions Algorithm 4

5 Linear Programming Abstract mathematical model: Decision Variables Criterion Constraints objective func. max / min c T x c R n constraints A x b A R m n, b R m x 0 x R n, 0 R n Any vector x R n satisfying all constraints is a feasible solution. Each x R n that gives the best possible value for c T x among all feasible x is an optimal solution or optimum The value c T x is the optimum value 5

6 In Matrix Form max c 1 x 1 + c 2 x 2 + c 3 x c n x n = z s.t. a 11 x 1 + a 12 x 2 + a 13 x a 1n x n b 1 a 21 x 1 + a 22 x 2 + a 23 x a 2n x n b 2... a m1 x 1 + a m2 x 2 + a m3 x a mn x n b m x 1, x 2,..., x n 0 c T = [ c 1 c 2... c n ] a 11 a a 1n x 1 b 1 a 21 a a 2n A =., x = x 2., b = b 2. a 31 a a mn x n b m max z = c T x Ax = b x 0 6

7 Definitions N natural numbers, Z integer numbers, Q rational numbers, R real numbers column vector and matrices scalar product: y T x = n i=1 y ix i linear combination moreover: x R k x 1,..., x k R λ = (λ 1,..., λ k ) T R k x = k i=1 λ ix i λ 0 conic combination λ T 1 = 1 ( k i=1 λ i = 1) affine combination λ 0 and λ T 1 = 1 convex combination 7

8 Definitions set S is linear independent if no element of it can be expressed as combination of the others Eg: S R = max n lin. indep. rank of a matrix for columns (= for rows) if (m, n)-matrix has rank = min{m, n} then the matrix is full rank if (n, n)-matrix is full rank then it is regular and admits an inverse G R n is an hyperplane if a R n \ {0} and α R: G = {x R n a T x = α} H R n is an halfspace if a R n \ {0} and α R: H = {x R n a T x α} (a T x = α is a supporting hyperplane of H) 8

9 Definitions a set S R is a polyhedron if m Z +, A R m n, b R m : P = {x R Ax b} = m i=1{x R n A i x b i } a polyhedron P is a polytope if it is bounded: B R, B > 0: p {x R n x B} Theorem: every polyhedron P R n is determined by finitely many halfspaces 9

10 Definitions General optimization problem: max{ϕ(x) x F }, F is feasible region for x If A and b are rational numbers, P = {x R n Ax b} is a rational polyhedron convex set: if x, y P and 0 λ 1 then λx + (1 λ)y P convex function if its epigraph {(x, y) R 2 : y f (x)} is a convex set or f : X R, if x, y X, λ [0, 1] it holds that f (λx + (1 λ)y) λf (x) + (1 λ)f (y) 10

11 Definitions Given a set of points X R n the convex hull conv(x ) is the convex linear combination of the points conv(x ) = {λ 1 x 1+λ 2x λ nx n x i X ; λ 1, λ 2,..., λ n 0 and i λ i = 1} 11

12 Definitions A face of P is F = {x P ax = α}. Hence F is either P itself or the intersection of P with a supporting hyperplane. It is said to be proper if F and F P. A point x for which {x} is a face is called a vertex of P and also a basic solution of Ax b A facet is a maximal face distinct from P cx d is facet defining if cx = d is a supporting hyperplane of P 12

13 Linear Programming Problem Input: a matrix A R m n and column vectors b R m, c R n Task: 1. decide that {x R n ; Ax b} is empty (prob. infeasible), or 2. find a column vector x R n such that Ax b and c T x is max, or 3. decide that for all α R there is an x R n with Ax b and c T x > α (prob. unbounded) 1. F = 2. F and solution 1. one solution 2. infinite solution 3. F and solution 13

14 Linear Programming and Linear Algebra Linear algebra: linear equations (Gaussian elimination) Integer linear algebra: linear diophantine equations Linear programming: linear inequalities (simplex method) Integer linear programming: linear diophantine inequalities 14

15 Outline Standard Form Basic Feasible Solutions Algorithm 15

16 Theorem (Fundamental Theorem of Linear Programming) Given: min{c T x x P} where P = {x R n Ax b} If P is a bounded polyhedron and not empty and x is an optimal solution to the problem, then: x is an extreme point (vertex) of P, or x lies on a face F P of optimal solution Proof: assume x not a vertex of P then a ball around it still in P. Show that a point in the ball has better cost if x is not a vertex then it is a convex combination of vertices. Show that all points are also optimal. 16

17 Implications: the optimal solution is at the intersection of hyperplanes supporting halfspaces. hence finitely many possibilities Solution method: write all inequalities as equalities and solve all ( ) n m systems of linear equalities (n # variables, m # constraints) for each point we need then to check if feasible and if best in cost. each system is solved by Gaussian elimination 17

18 1. find a solution that is at the intersection of some n hyperplanes 2. try systematically to produce the other points by exchanging one hyperplane with another 3. check optimality, proof provided by duality theory 18

19 Demo 19

20 Outline Standard Form Basic Feasible Solutions Algorithm 20

21 1. Forward elimination reduces the system to triangular (row echelon) form (or degenerate) elementary row operations (or LU decomposition) 2. back substitution Example: 2x + y z = 8 (I ) 3x y + 2z = 11 (II ) 2x + y + 2z = 3 (III ) 21

22 /2 I+II 0 1/2 1/2 1 I+III /2 1/2 1-4 II+III /2 1/ => x= => y= => z= x + y z = 8 (I ) y z = 1 (II ) + 2y + 1z = 5 (III ) 2x + y z = 8 (I ) y z = 1 (II ) z = 1 (III ) 2x + y z = 8 (I ) y z = 1 (II ) z = 1 (III ) x = 2 (I ) y = 3 (II ) z = 1 (III ) Polynomial time O(n 2 m) but needs to guarantee that all the numbers during the run can be represented by polynomially bounded bits 22

23 Outline Standard Form Basic Feasible Solutions Algorithm 23

24 A Numerical Example max n c j x j j=1 n a ij x j b i, i = 1,..., m j=1 x j 0, j = 1,..., n max 6x 1 + 8x 2 5x x x 1 + 4x 2 40 x 1, x 2 0 max c T x Ax b x 0 x R n, c R n, A R m n, b R m max [ ] [ ] x x 2 [ ] [ ] 5 10 x1 4 4 x 2 x 1, x 2 0 [ ]

25 Outline Standard Form Basic Feasible Solutions Algorithm 25

26 Standard Form Each linear program can be converted in the form: max c T x Ax b x R n c R n, A R m n, b R m if equations, then put two constraints, ax b and ax b if ax b then ax b if min c T x then max( c T x) and then be put in standard (or equational) form max c T x Ax = b x 0 x R n, c R n, A R m n, b R m 1. = constraints 2. x 0 nonnegativity constraints 3. (b 0) 4. max 26

27 Transformation into Std Form Every LP can be transformed in std. form 1. introduce slack variables (or surplus) 5x x 2 + x 3 = 60 4x 1 + 4x 2 + x 4 = if x 1 0 then x 1 = x 1 x 1 x 1 0 x (b 0) 4. min c T x max( c T x) LP in n m converted into LP with at most (m + 2n) variables and m equations (n # original variables, m # constraints) 27

28 Geometry 28

29 Ax = b is a system of equations that we can solve by Gaussian elimination Elementary row operations of [ A b ] do not affect set of feasible solutions multiplying all entries in some row of [ A b ] by a nonzero real number λ replacing the ith row of [ A b ] by the sum of the ith row and jth row for some i j We assume rank( [ A b ] ) = rank(a) = m, ie, rows of A are linearly independent otherwise, remove linear dependent rows 29

30 Outline Standard Form Basic Feasible Solutions Algorithm 30

31 Basic Feasible Solutions Basic feasible solutions are the vertices of the feasible region: More formally: Let B = {1... m}, N = {m n + m} be subsets of columns A B is made of columns of A indexed by B: Definition x R n is a basic feasible solution of the linear program max{c T x Ax = b, x 0} for an index set B if: x j = 0 j B the square matrix A B is nonsingular, ie, all columns indexed by B are lin. indep. x B = A 1 B b is nonnegative, ie, x B 0 (feasibility) 31

32 We call x j, j B basic variables and remaining variables nonbasic variables. Theorem A basic feasible solution is uniquely determined by the set B. Proof: Ax =A B x B + A N x N = b x B + A 1 B A Nx N = A 1 B b x B = A 1 B b A B is singular hence one solution Note: we call B a (feasible) basis 32

33 Extreme points and basic feasible solutions are geometric and algebraic manifestations of the same concept: Theorem Let P be a (convex) polyhedron from LP in std. form. For a point v P the following are equivalent: (i) v is an extreme point (vertex) of P (ii) v is a basic feasible solution of LP Proof: by recognizing that vertices of P are linear independent and such are the columns in A B Theorem Let LP = max{c T x Ax = b, x 0} be feasible and bounded, then the optimal solution is a basic feasible solution. Proof. consequence of previous theorem and fundamental theorem of linear programming 33

34 Idea for solution method: examine all basic solutions. There are finitely many: ( ) m+n m. However, if n = m then ( ) 2m m 4 m. 34

35 Outline Standard Form Basic Feasible Solutions Algorithm 35

36 max z = [ 6 8 ] [ ] x 1 x 2 [ ] x 1 x 2 x 3 x 4 [ ] 60 = 40 x 1, x 2, x 3, x 4 0 Canonical std. form: one decision variable is isolated in each constraint and does not appear in the other constraints or in the obj. func. and b terms are positive It gives immediately a feasible solution: x 1 = 0, x 2 = 0, x 3 = 60, x 4 = 40 Is it optimal? Look at signs in z if positive then an increase would improve. 36

37 Let s try to increase a promising variable, ie, x 1, one with positive coefficient in z (is the best choice?) 5x 1 + x 3 = 60 x 1 = 60 5 x3 5 x 3 = 60 5x 1 0 x 3 If x 1 > 12 then x 3 < 0 x 1 5x 1 + x 3 = 60 4x 1 + x 4 = 40 x 1 = 40 4 x4 4 x 4 = 40 4x 1 0 x 4 If x 1 > 10 then x 4 < 0 4x 1 + x 4 = 40 x 1 we can take the minimum of the two x 1 increased to 10 x 4 exits the basis and x 1 enters

38 Simplex Tableau First simplex tableau: x 1 x 2 x 3 x 4 z b x x we want to reach this new tableau x 1 x 2 x 3 x 4 z b x 3 0? 1? 0? x 1 1? 0? 0? 0? 0? 1? Pivot operation: 1. Choose pivot: column: one s with positive coefficient in obj. func. (to discuss later) row: ratio between coefficient b and pivot column: choose the one with smallest ratio: θ = min i { bi a is : a is > 0 }, θ increase value of entering var. 2. elementary row operations to update the tableau 38

39 x 4 leaves the basis, x 1 enters the basis Divide row pivot by pivot Send to zero the coefficient in the pivot column of the first row Send to zero the coefficient of the pivot column in the third (cost) row x1 x2 x3 x4 -z b I =I-5II / II =II/ / III =III-6II / From the last row we read: 2x 2 3/2x 4 z = 60, that is: z = x 2 3/2x 4. Since x 2 and x 4 are nonbasic we have z = 60 and x 1 = 10, x 2 = 0, x 3 = 10, x 4 = 0. Done? No! Let x 2 enter the basis x1 x2 x3 x4 -z b I =I/ /5-1/4 0 2 II =II-I 1 0-1/5 1/ III =III-2I 0 0-2/

40 Reduced costs: the coefficients in the objective function of the nonbasic variables, c N Optimality: The basic solution is optimal when the reduced costs in the corresponding simplex tableau are nonpositive, ie, such that: c N 0 40

41 Graphical Representation x 2 x 2?? x 1?? x 1 41