The Simplex Algorithm. Chapter 5. Decision Procedures. An Algorithmic Point of View. Revision 1.0

Transcription

1 The Simplex Algorithm Chapter 5 Decision Procedures An Algorithmic Point of View D.Kroening O.Strichman Revision 1.0

2 Outline 1 Gaussian Elimination 2 Satisfiability with Simplex 3 General Simplex Form 4 Simplex Basics 5 The General Simplex Algorithm Decision Procedures The Simplex Algorithm 2

3 Gaussian Elimination Given a linear system Ax = b a 11 a a 1k a 21 a a 2k x 1 x 2. = b 1 b 2. a k1 a k2... a kk x k b k Manipulate A b to obtain an upper-triangular form a 11 a a 1k b 0 a a 1 2k b a kk b k Decision Procedures The Simplex Algorithm 3

4 Gaussian Elimination Then, solve backwards from k s row according to: x i = 1 a (b i ii k j=i+1 a ijx j ) Decision Procedures The Simplex Algorithm 4

5 Example x 1 x 2 x 3 = Decision Procedures The Simplex Algorithm 5

6 Example x 1 x 2 x 3 = R3 = ( 4, 1, 8 9 ) 4R1 = ( 4, 8, 4 24 ) R3 + = 4R Decision Procedures The Simplex Algorithm 5

7 Example x 1 x 2 x 3 = R3 = ( 4, 1, 8 9 ) 4R1 = ( 4, 8, 4 24 ) R3 + = 4R R2 = ( 2, 3, 4 3 ) 2R1 = ( 2, 4, 2 12 ) R2 + = 2R Decision Procedures The Simplex Algorithm 5

8 Example x 1 x 2 x 3 = R3 = ( 4, 1, 8 9 ) 4R1 = ( 4, 8, 4 24 ) R3 + = 4R R2 = ( 2, 3, 4 3 ) 2R1 = ( 2, 4, 2 12 ) R2 + = 2R R3 = ( 0, 9, ) 9 7 R2 = ( 0, 9, ) R3 + = 9 7 R Decision Procedures The Simplex Algorithm 5

9 Example x 1 x 2 x 3 = R3 = ( 4, 1, 8 9 ) 4R1 = ( 4, 8, 4 24 ) R3 + = 4R R2 = ( 2, 3, 4 3 ) 2R1 = ( 2, 4, 2 12 ) R2 + = 2R R3 = ( 0, 9, ) 9 7 R2 = ( 0, 9, ) R3 + = 9 7 R2 Now: x 3 = 1, x 2 = 3, x 1 = 1. Problem solved! Decision Procedures The Simplex Algorithm

10 Satisfiability with Simplex Simplex was originally designed for solving the optimization problem: max c x s.t. A x b, x 0 We are only interested in the feasibility problem = satisfiability problem. Decision Procedures The Simplex Algorithm 6

11 General Simplex We will learn a variant called general simplex. Very suitable for solving the satisfiability problem fast. Decision Procedures The Simplex Algorithm 7

12 General Simplex We will learn a variant called general simplex. Very suitable for solving the satisfiability problem fast. The input: A x b A is a m n coefficient matrix The problem variables are x = x 1,..., x n First step: convert the input to general form Decision Procedures The Simplex Algorithm 7

13 General Form Definition (General Form) A x = 0 and m l i s i u i i=1 A combination of Linear equalities of the form i a ix i = 0 Lower and upper bounds on variables Decision Procedures The Simplex Algorithm 8

14 Transformation to General Form Replace i a ix i b j (where {=,, }) with i a ix i s j = 0 and s j b j. s 1,..., s m are called the additional variables Decision Procedures The Simplex Algorithm 9

15 Example 1 Convert x + y 2! Decision Procedures The Simplex Algorithm 10

16 Example 1 Convert x + y 2! Result: x + y s 1 = 0 s 1 2 It is common to keep the conjunctions implicit Decision Procedures The Simplex Algorithm 10

17 Example 2 Convert x +y 2 2x y 0 x +2y 1 Decision Procedures The Simplex Algorithm 11

18 Example 2 Convert x +y 2 2x y 0 x +2y 1 Result: x +y s 1 = 0 2x y s 2 = 0 x +2y s 3 = 0 s 1 2 s 2 0 s 3 1 Decision Procedures The Simplex Algorithm 11

19 Geometrical Interpretation Linear inequality constraints, geometrically, define a convex polyhedron. c Wikipedia Decision Procedures The Simplex Algorithm 12

20 Geometrical Interpretation Our example from before: y 2x y 0 x +y 2 2x y 0 x +2y x + 2y 0 (C) x + y 2 (A) (B) x Decision Procedures The Simplex Algorithm 13

21 Matrix Form Recall the general form: A x = 0 and m i=1 l i s i u i A is now an m (n + m) matrix due to the additional variables. x +y s 1 = 0 2x y s 2 = 0 x +2y s 3 = 0 s 1 2 s 2 0 s 3 1 x y s 1 s 2 s Decision Procedures The Simplex Algorithm 14

22 The Tableau The diagonal part is inherent to the general form: x y s 1 s 2 s Instead, we can write: s 1 s 2 s 3 x y Decision Procedures The Simplex Algorithm 15

23 The Tableau The tableaux changes throughout the algorithm, but maintains its m n structure Distinguish basic and nonbasic variables Basic variables s 1 s 2 s 3 x y Nonbasic variables Initially, basic variables = the additional variables Decision Procedures The Simplex Algorithm 16

24 The Tableau Notation: B N the basic variables the nonbasic variables The tableaux is simply a different notation for the system (x i = ) a ij x j x i B x j N The basic variables are also called the dependent variables. Decision Procedures The Simplex Algorithm 17

25 Data Structures Simplex maintains: The tableau, an assignment α to all variables, an assignment to the bounds. Initially, B = additional variables, N = problem variables, α(xi) = 0 for i {1,..., n + m} Decision Procedures The Simplex Algorithm 18

26 Invariants Two invariants are maintained throughout: 1 A x = 0 2 All nonbasic variables satisfy their bounds The basic variables need not satisy their bounds Decision Procedures The Simplex Algorithm 19

27 Invariants Two invariants are maintained throughout: 1 A x = 0 2 All nonbasic variables satisfy their bounds The basic variables need not satisy their bounds Can you see why these invariants are maintained initially? We should check that they are indeed maintained Decision Procedures The Simplex Algorithm 19

28 Invariants The initial assignment satisfies A x = 0 If the bounds of all basic variables are satisfied by α, return Satisfiable Otherwise... pivot. Decision Procedures The Simplex Algorithm 20

29 Pivoting 1 Find a basic variable x i that violates its bounds. Suppose that α(x i ) < l i. 2 Find a nonbasic variable x j such that a ij > 0 and α(x j ) < u j, or a ij < 0 and α(x j ) > l j. Why? Decision Procedures The Simplex Algorithm 21

30 Pivoting 1 Find a basic variable x i that violates its bounds. Suppose that α(x i ) < l i. 2 Find a nonbasic variable x j such that a ij > 0 and α(x j ) < u j, or a ij < 0 and α(x j ) > l j. Why? Such a variable is called suitable. Decision Procedures The Simplex Algorithm 21

31 Pivoting 1 Find a basic variable x i that violates its bounds. Suppose that α(x i ) < l i. 2 Find a nonbasic variable x j such that a ij > 0 and α(x j ) < u j, or a ij < 0 and α(x j ) > l j. Why? Such a variable is called suitable. 3 If there is no suitable variable, return Unsatisfiable Why? Decision Procedures The Simplex Algorithm 21

32 Pivoting x i and x j (1) 1 Solve equation i for x j : From: x i = a ij x j + k j a ik x k To: x j = x i a ik x k a ij a ij k j Decision Procedures The Simplex Algorithm 22

33 Pivoting x i and x j (1) 1 Solve equation i for x j : From: x i = a ij x j + k j a ik x k To: x j = x i a ik x k a ij a ij k j 2 Swap x i and x j, and update the i-th row accordingly From: a i1... a ij... a in To: a i1 a ij... 1 a ij... a in a ij Decision Procedures The Simplex Algorithm 22

34 Pivoting x i and x j (2) 3 Update all other rows: Replace x j with its equivalent obtained from row i: x j = x i a ik x k a ij a ij k j Decision Procedures The Simplex Algorithm 23

35 Pivoting x i and x j (2) 3 Update all other rows: Replace x j with its equivalent obtained from row i: x j = x i a ik x k a ij a ij k j 4 Update α as follows: Increase α(x j ) by θ = l i α(x i ) a ij Now x j is a basic variable: it may violate its bounds Update α(x i ) accordingly Q: What is α(x i ) now? Update α for all other basic (dependent) variables Decision Procedures The Simplex Algorithm 23

36 Pivoting: Example (1) Recall the tableau and constraints in our example: x y s s s s 1 0 s 2 1 s 3 Initially, α assigns 0 to all variables = The bounds of s 1 and s 3 are violated Decision Procedures The Simplex Algorithm 24

37 Pivoting: Example (1) Recall the tableau and constraints in our example: x y s s s s 1 0 s 2 1 s 3 Initially, α assigns 0 to all variables = The bounds of s 1 and s 3 are violated We will fix s 1. x is a suitable nonbasic variable for pivoting. It has no upper bound! So now we pivot s 1 with x Decision Procedures The Simplex Algorithm 24

38 Pivoting: Example (2) x y s s s s 1 0 s 2 1 s 3 Decision Procedures The Simplex Algorithm 25

39 Pivoting: Example (2) x y s s s s 1 0 s 2 1 s 3 Solve 1 st row for x: s 1 = x + y x = s 1 y Decision Procedures The Simplex Algorithm 25

40 Pivoting: Example (2) x y s s s s 1 0 s 2 1 s 3 Solve 1 st row for x: s 1 = x + y x = s 1 y Replace x in other rows: s 2 = 2(s 1 y) y s 2 = 2s 1 3y s 3 = (s 1 y) + 2y s 3 = s 1 + 3y Decision Procedures The Simplex Algorithm 25

41 Pivoting: Example (3) x = s 1 y s 2 = 2s 1 3y s 3 = s 1 + 3y Decision Procedures The Simplex Algorithm 26

42 Pivoting: Example (3) x = s 1 y s 2 = 2s 1 3y s 3 = s 1 + 3y This results in the following new tableau: s1 y x 1 1 s s s 1 0 s 2 1 s 3 Decision Procedures The Simplex Algorithm 26

43 Pivoting: Example (3) x = s 1 y s 2 = 2s 1 3y s 3 = s 1 + 3y This results in the following new tableau: s1 y x 1 1 s s s 1 0 s 2 1 s 3 What about the assignment? We should increase x by θ = Hence, α(x) = = 2 = 2 Now s 1 is equal to its lower bound: α(s1) = 2 Update all the others Decision Procedures The Simplex Algorithm 26

44 Pivoting: Example (4) The new state: s 1 y x 1 1 s s α(x) = 2 α(y) = 0 α(s 1 ) = 2 α(s 2 ) = 4 α(s 3 ) = 2 2 s 1 0 s 2 1 s 3 Decision Procedures The Simplex Algorithm 27

45 Pivoting: Example (4) The new state: s 1 y x 1 1 s s α(x) = 2 α(y) = 0 α(s 1 ) = 2 α(s 2 ) = 4 α(s 3 ) = 2 2 s 1 0 s 2 1 s 3 Now s 3 violates its lower bound Which nonbasic variable is suitable for pivoting? Decision Procedures The Simplex Algorithm 27

46 Pivoting: Example (4) The new state: s 1 y x 1 1 s s α(x) = 2 α(y) = 0 α(s 1 ) = 2 α(s 2 ) = 4 α(s 3 ) = 2 2 s 1 0 s 2 1 s 3 Now s 3 violates its lower bound Which nonbasic variable is suitable for pivoting? That s right... y Decision Procedures The Simplex Algorithm 27

47 Pivoting: Example (4) The new state: s 1 y x 1 1 s s α(x) = 2 α(y) = 0 α(s 1 ) = 2 α(s 2 ) = 4 α(s 3 ) = 2 2 s 1 0 s 2 1 s 3 Now s 3 violates its lower bound Which nonbasic variable is suitable for pivoting? That s right... y We should increase y by θ = 1 ( 2) 3 = 1 Decision Procedures The Simplex Algorithm 27

48 Pivoting: Example (5) The final state: s 1 s 3 x 2/3 1/3 s y 1/3 1/3 α(x) = 1 α(y) = 1 α(s 1 ) = 2 α(s 2 ) = 1 α(s 3 ) = 1 2 s 1 0 s 2 1 s 3 All constraints are satisfied. Decision Procedures The Simplex Algorithm 28

49 Observations I The additional variables: Only additional variables have bounds. These bounds are permanent. Additional variables enter the base only on extreme points (their lower or upper bounds). When entering the base, they shift towards the other bound and possibly cross it (violate it). Decision Procedures The Simplex Algorithm 29

50 Observations II Q: Can it be that we pivot x i, x j and then pivot x j, x i and thus enter a (local) cycle? Decision Procedures The Simplex Algorithm 30

51 Observations II Q: Can it be that we pivot x i, x j and then pivot x j, x i and thus enter a (local) cycle? A: No. For example, suppose that a ij > 0. We increased α(x j ) so now α(x i ) = l i. After pivoting, possibly α(x j ) > u j, but a ij = 1/a ij > 0, hence the coefficient of x i is not suitable Decision Procedures The Simplex Algorithm 30

52 Termination Is termination guaranteed? Decision Procedures The Simplex Algorithm 31

53 Termination Is termination guaranteed? Not obvious. Perhaps there are bigger cycles. In order to avoid circles, we use Bland s rule: 1 Determine a total order on the variables 2 Choose the first basic variable that violates its bounds, and the first nonbasic suitable variable for pivoting. 3 It can be shown that this guarantees that no base is repeated, which implies termination. Decision Procedures The Simplex Algorithm 31

54 General simplex With Bland s Rule 1 Transform the system into the general form A x = 0 and m l i s i u i. i=1 2 Set B to be the set of additional variables s 1,..., s m. 3 Construct the tableau for A. 4 Determine a fixed order on the variables. 5 If there is no basic variable that violates its bounds, return Satisfiable. Otherwise, let x i be the first basic variable in the order that violates its bounds. 6 Search for the first suitable nonbasic variable x j in the order for pivoting with x i. If there is no such variable, return Unsatisfiable. 7 Perform the pivot operation on x i and x j. 8 Go to step 5. Decision Procedures The Simplex Algorithm 32