CS675: Convex and Combinatorial Optimization Spring 2018 Consequences of the Ellipsoid Algorithm. Instructor: Shaddin Dughmi
|
|
- Ann Shelton
- 5 years ago
- Views:
Transcription
1 CS675: Convex and Combinatorial Optimization Spring 2018 Consequences of the Ellipsoid Algorithm Instructor: Shaddin Dughmi
2 Outline 1 Recapping the Ellipsoid Method 2 Complexity of Convex Optimization 3 Complexity of Linear Programming 4 Equivalence of Separation and Optimization
3 Recall: Feasibility Problem The ellipsoid method solves the following problem. Convex Feasibility Problem Given as input the following A description of a compact convex set K R n An ellipsoid E(c, Q) (typically a ball) containing K A rational number R > 0 satisfying vol(e) R. A rational number r > 0 such that if K is nonempty, then vol(k) r. Find a point x K or declare that K is empty. Equivalent variant: drop the requirement on volume vol(k), and either find a point x K or an ellipsoid E K with vol(e) < r. Recapping the Ellipsoid Method 1/27
4 All the ellipsoid method needed was the following subroutine Recapping the Ellipsoid Method 2/27 Separation oracle An algorithm that takes as input x R n, and either certifies x K or outputs a hyperplane separting x from K. i.e. a vector h R n with h x h y for all y K. Equivalently, K is contained in the halfspace with x at its boundary. H(h, x) = {y : h y h x}
5 All the ellipsoid method needed was the following subroutine Recapping the Ellipsoid Method 2/27 Separation oracle An algorithm that takes as input x R n, and either certifies x K or outputs a hyperplane separting x from K. i.e. a vector h R n with h x h y for all y K. Equivalently, K is contained in the halfspace with x at its boundary. Examples: H(h, x) = {y : h y h x}
6 All the ellipsoid method needed was the following subroutine Recapping the Ellipsoid Method 2/27 Separation oracle An algorithm that takes as input x R n, and either certifies x K or outputs a hyperplane separting x from K. i.e. a vector h R n with h x h y for all y K. Equivalently, K is contained in the halfspace with x at its boundary. H(h, x) = {y : h y h x} Examples: Explicitly written polytope Ay b: take h = a i to the row of A corresponding to a constraint violated by x.
7 All the ellipsoid method needed was the following subroutine Recapping the Ellipsoid Method 2/27 Separation oracle An algorithm that takes as input x R n, and either certifies x K or outputs a hyperplane separting x from K. i.e. a vector h R n with h x h y for all y K. Equivalently, K is contained in the halfspace with x at its boundary. H(h, x) = {y : h y h x} Examples: Explicitly written polytope Ay b: take h = a i to the row of A corresponding to a constraint violated by x. Convex set given by a family of convex inequalities f i (y) 0: Let h = f i (x) for some violated constraint.
8 All the ellipsoid method needed was the following subroutine Examples: Explicitly written polytope Ay b: take h = a i to the row of A corresponding to a constraint violated by x. Convex set given by a family of convex inequalities f i (y) 0: Let h = f i (x) for some violated constraint. The positive semi-definite cone S + n : Let H be the outer product vv of an eigenvector v of X corresponding to a negative eigenvalue. Recapping the Ellipsoid Method 2/27 Separation oracle An algorithm that takes as input x R n, and either certifies x K or outputs a hyperplane separting x from K. i.e. a vector h R n with h x h y for all y K. Equivalently, K is contained in the halfspace with x at its boundary. H(h, x) = {y : h y h x}
9 Recapping the Ellipsoid Method 3/27 Ellipsoid Method 1 Start with initial ellipsoid E = E(c, Q) K 2 Using the separation oracle, check if the center c K. If so, terminate and output c. Otherwise, we get a separating hyperplane h such that K is contained in the half-ellipsoid E {y : h y h c} 3 Let E = E(c, Q ) be the minimum volume ellipsoid containing the half ellipsoid above. 4 If vol(e ) r then set E = E and repeat (step 2), otherwise stop and return empty.
10 Recapping the Ellipsoid Method 3/27 Ellipsoid Method 1 Start with initial ellipsoid E = E(c, Q) K 2 Using the separation oracle, check if the center c K. If so, terminate and output c. Otherwise, we get a separating hyperplane h such that K is contained in the half-ellipsoid E {y : h y h c} 3 Let E = E(c, Q ) be the minimum volume ellipsoid containing the half ellipsoid above. 4 If vol(e ) r then set E = E and repeat (step 2), otherwise stop and return empty.
11 Recapping the Ellipsoid Method 3/27 Ellipsoid Method 1 Start with initial ellipsoid E = E(c, Q) K 2 Using the separation oracle, check if the center c K. If so, terminate and output c. Otherwise, we get a separating hyperplane h such that K is contained in the half-ellipsoid E {y : h y h c} 3 Let E = E(c, Q ) be the minimum volume ellipsoid containing the half ellipsoid above. 4 If vol(e ) r then set E = E and repeat (step 2), otherwise stop and return empty.
12 Recapping the Ellipsoid Method 3/27 Ellipsoid Method 1 Start with initial ellipsoid E = E(c, Q) K 2 Using the separation oracle, check if the center c K. If so, terminate and output c. Otherwise, we get a separating hyperplane h such that K is contained in the half-ellipsoid E {y : h y h c} 3 Let E = E(c, Q ) be the minimum volume ellipsoid containing the half ellipsoid above. 4 If vol(e ) r then set E = E and repeat (step 2), otherwise stop and return empty.
13 Properties Recapping the Ellipsoid Method 4/27 Using T to denote the runtime of the separation oracle Theorem The ellipsoid algorithm terminates in time polynomial n, ln R r, and T, and either outputes x K or correctly declares that K is empty. We proved most of this (modulo the ellipsoid updating Lemma which we cited and briefly discussed).
14 Properties Using T to denote the runtime of the separation oracle Theorem The ellipsoid algorithm terminates in time polynomial n, ln R r, and T, and either outputes x K or correctly declares that K is empty. We proved most of this (modulo the ellipsoid updating Lemma which we cited and briefly discussed). Note For runtime polynomial in input size we need T polynomial in input size R r exponential in input size Recapping the Ellipsoid Method 4/27
15 Outline 1 Recapping the Ellipsoid Method 2 Complexity of Convex Optimization 3 Complexity of Linear Programming 4 Equivalence of Separation and Optimization
16 Complexity of Convex Optimization 5/27 Recall: Convex Optimization Problem A problem of minimizing a convex function (or maximizing a concave function) over a convex set. minimize subject to f(x) x X Where X R n is convex and closed, and f : R n R is convex
17 Complexity of Convex Optimization 5/27 Recall: Convex Optimization Problem A problem of minimizing a convex function (or maximizing a concave function) over a convex set. minimize subject to f(x) x X Where X R n is convex and closed, and f : R n R is convex Recall: A problem Π is a family of instances I = (f, X ) When represented explicitly, often given in standard form minimize f(x) subject to g i (x) 0, for i C 1. a i x = b i, for i C 2. The functions f,{g i } i are given in some parametric form allowing evaluation of each function and its derivatives.
18 Complexity of Convex Optimization 5/27 Recall: Convex Optimization Problem A problem of minimizing a convex function (or maximizing a concave function) over a convex set. minimize subject to f(x) x X Where X R n is convex and closed, and f : R n R is convex We will abstract away details of how instances of a problem are represented, but denote the length of the description by I Require polynomial time (in I and n) implementation of separation oracle, and other subroutines.
19 Complexity of Convex Optimization 6/27 Solvability of Convex Optimization There are many subtly different solvability statements. This one is the most useful, yet simple to describe, IMO. Requirements We say an algorithm weakly solves a convex optimization problem in polynomial time if it: Takes an approximation parameter ɛ > 0 Terminates in time poly( I, n, log( 1 ɛ )) Returns an ɛ-optimal x X : f(x) min y X f(y) + ɛ[max f(y) min f(y)] y X y X
20 Solvability of Convex Optimization Complexity of Convex Optimization 6/27 Theorem (Polynomial Solvability of CP) Consider a family Π of convex optimization problems I = (f, X ) admitting the following operations in polynomial time (in I and n): A separation oracle for the feasible set X R n A first order oracle for f: evaluates f(x) and f(x). An algorithm which computes a starting ellipsoid E X with = O(exp( I, n)). vol(e) vol(x ) Then there is a polynomial time algorithm which weakly solves Π.
21 Solvability of Convex Optimization Complexity of Convex Optimization 6/27 Theorem (Polynomial Solvability of CP) Consider a family Π of convex optimization problems I = (f, X ) admitting the following operations in polynomial time (in I and n): A separation oracle for the feasible set X R n A first order oracle for f: evaluates f(x) and f(x). An algorithm which computes a starting ellipsoid E X with = O(exp( I, n)). vol(e) vol(x ) Then there is a polynomial time algorithm which weakly solves Π. Let s now prove this, by reducing to the ellipsoid method
22 Proof (Simplified) Complexity of Convex Optimization 7/27 Simplifying Assumption Assume we are given min y X f(y) and max y X f(y). Without loss of generality assume they are [0, 1].
23 Proof (Simplified) Complexity of Convex Optimization 7/27 Simplifying Assumption Assume we are given min y X f(y) and max y X f(y). Without loss of generality assume they are [0, 1]. Our task reduces to the following convex feasibility problem: find subject to x x X f(x) ɛ
24 Proof (Simplified) Complexity of Convex Optimization 7/27 Simplifying Assumption Assume we are given min y X f(y) and max y X f(y). Without loss of generality assume they are [0, 1]. Our task reduces to the following convex feasibility problem: find x subject to x X f(x) ɛ We can feed this into the Ellipsoid method! Needed Ingredients 1 Separation oracle for new feasible set K: 2 Ellipsoid E containing K: 3 Guarantee that vol(e) vol(k) exp(n, I, log 1 ɛ ):
25 Proof (Simplified) Complexity of Convex Optimization 7/27 Simplifying Assumption Assume we are given min y X f(y) and max y X f(y). Without loss of generality assume they are [0, 1]. Our task reduces to the following convex feasibility problem: find x subject to x X f(x) ɛ We can feed this into the Ellipsoid method! Needed Ingredients 1 Separation oracle for new feasible set K: Use the separation oracle for X and first order oracle for f 2 Ellipsoid E containing K: 3 Guarantee that vol(e) vol(k) exp(n, I, log 1 ɛ ):
26 Proof (Simplified) Complexity of Convex Optimization 7/27 Simplifying Assumption Assume we are given min y X f(y) and max y X f(y). Without loss of generality assume they are [0, 1]. Our task reduces to the following convex feasibility problem: find x subject to x X f(x) ɛ We can feed this into the Ellipsoid method! Needed Ingredients 1 Separation oracle for new feasible set K: Use the separation oracle for X and first order oracle for f 2 Ellipsoid E containing K: Use the ellipsoid containing X 3 Guarantee that vol(e) vol(k) exp(n, I, log 1 ɛ ):
27 Proof (Simplified) Complexity of Convex Optimization 7/27 Simplifying Assumption Assume we are given min y X f(y) and max y X f(y). Without loss of generality assume they are [0, 1]. Our task reduces to the following convex feasibility problem: find x subject to x X f(x) ɛ We can feed this into the Ellipsoid method! Needed Ingredients 1 Separation oracle for new feasible set K: Use the separation oracle for X and first order oracle for f 2 Ellipsoid E containing K: Use the ellipsoid containing X 3 Guarantee that vol(e) vol(k) exp(n, I, log 1 ɛ ): Not obvious, but true!
28 Proof (Simplified) K = {x X : f(x) ɛ} Lemma vol(k) ɛ n vol(x). This shows that vol(k) is only exponentially smaller (in n and log 1 ɛ ) than vol(x ), and therefore also vol(e), so it suffices. Complexity of Convex Optimization 8/27
29 Proof (Simplified) K = {x X : f(x) ɛ} Lemma vol(k) ɛ n vol(x). This shows that vol(k) is only exponentially smaller (in n and log 1 ɛ ) than vol(x ), and therefore also vol(e), so it suffices. Assume wlog 0 X and f(0) = min x X f(x) = 0. Complexity of Convex Optimization 8/27
30 Proof (Simplified) K = {x X : f(x) ɛ} Lemma vol(k) ɛ n vol(x). This shows that vol(k) is only exponentially smaller (in n and log 1 ɛ ) than vol(x ), and therefore also vol(e), so it suffices. Assume wlog 0 X and f(0) = min x X f(x) = 0. Consider scaling X by ɛ to get ɛx. vol(ɛx ) = ɛ n vol(x). Complexity of Convex Optimization 8/27
31 Proof (Simplified) K = {x X : f(x) ɛ} Lemma vol(k) ɛ n vol(x). This shows that vol(k) is only exponentially smaller (in n and log 1 ɛ ) than vol(x ), and therefore also vol(e), so it suffices. Assume wlog 0 X and f(0) = min x X f(x) = 0. Consider scaling X by ɛ to get ɛx. vol(ɛx ) = ɛ n vol(x). We show that ɛx K by showing f(y) ɛ for all y ɛx. Complexity of Convex Optimization 8/27
32 Proof (Simplified) K = {x X : f(x) ɛ} Lemma vol(k) ɛ n vol(x). This shows that vol(k) is only exponentially smaller (in n and log 1 ɛ ) than vol(x ), and therefore also vol(e), so it suffices. Assume wlog 0 X and f(0) = min x X f(x) = 0. Consider scaling X by ɛ to get ɛx. vol(ɛx ) = ɛ n vol(x). We show that ɛx K by showing f(y) ɛ for all y ɛx. Let y = ɛx for x X, and invoke Jensen s inequality f(y) = f(ɛx + (1 ɛ)0) ɛf(x) + (1 ɛ)f(0) ɛ Complexity of Convex Optimization 8/27
33 Complexity of Convex Optimization 9/27 Proof (General) Denote L = min y X f(y) and H = max y X f(y) If we knew the target T = L + ɛ[h L], we can reduce to solving the feasibility problem over K = {x X : f(x) T }.
34 Complexity of Convex Optimization 9/27 Proof (General) Denote L = min y X f(y) and H = max y X f(y) If we knew the target T = L + ɛ[h L], we can reduce to solving the feasibility problem over K = {x X : f(x) T }. If we knew it lied in a sufficiently narrow range, we could binary search for T
35 Complexity of Convex Optimization 9/27 Proof (General) Denote L = min y X f(y) and H = max y X f(y) If we knew the target T = L + ɛ[h L], we can reduce to solving the feasibility problem over K = {x X : f(x) T }. If we knew it lied in a sufficiently narrow range, we could binary search for T We don t need to know anything about T! Key Observation We don t really need to know T, H, or L to simulate the same execution of the ellipsoid method on K!!
36 Proof (General) Complexity of Convex Optimization 10/27 find subject to x x X f(x) T = L + ɛ[h L] Simulate the execution of the ellipsoid method on K Polynomial number of iterations, terminating with point in K
37 Proof (General) Complexity of Convex Optimization 10/27 find subject to x x X f(x) T = L + ɛ[h L] Simulate the execution of the ellipsoid method on K Polynomial number of iterations, terminating with point in K Require separation oracle for K to use f only as a last resort This is allowed. Tries to get feasibility whenever possible.
38 Proof (General) Complexity of Convex Optimization 10/27 find subject to x x X f(x) T = L + ɛ[h L] Simulate the execution of the ellipsoid method on K Polynomial number of iterations, terminating with point in K Require separation oracle for K to use f only as a last resort This is allowed. Tries to get feasibility whenever possible. Action of algorithm in each iteration other than the last can be described independently of T If ellipsoid center c / X, use separating hyperplane with X. Else use f(c)
39 Proof (General) Complexity of Convex Optimization 10/27 find subject to x x X f(x) T = L + ɛ[h L] Simulate the execution of the ellipsoid method on K Polynomial number of iterations, terminating with point in K Require separation oracle for K to use f only as a last resort This is allowed. Tries to get feasibility whenever possible. Action of algorithm in each iteration other than the last can be described independently of T If ellipsoid center c / X, use separating hyperplane with X. Else use f(c) Run this simulation until enough iterations have passed, and take the best feasible point encountered. This must be in K.
40 Outline 1 Recapping the Ellipsoid Method 2 Complexity of Convex Optimization 3 Complexity of Linear Programming 4 Equivalence of Separation and Optimization
41 Recall: Linear Programming Complexity of Linear Programming 11/27 Recall: Linear Programming Problem A problem of maximizing a linear function over a polyhedron. maximize subject to c x Ax b
42 Recall: Linear Programming Complexity of Linear Programming 11/27 Recall: Linear Programming Problem A problem of maximizing a linear function over a polyhedron. maximize subject to c x Ax b When stated in standard form, optimal solution occurs at a vertex.
43 Recall: Linear Programming Complexity of Linear Programming 11/27 Recall: Linear Programming Problem A problem of maximizing a linear function over a polyhedron. maximize subject to c x Ax b When stated in standard form, optimal solution occurs at a vertex. We will consider both explicitly and implicit LPs Explicit: given by A, b and c Implicit: Given by c and a separation oracle for Ax b.
44 Recall: Linear Programming Complexity of Linear Programming 11/27 Recall: Linear Programming Problem A problem of maximizing a linear function over a polyhedron. maximize subject to c x Ax b When stated in standard form, optimal solution occurs at a vertex. We will consider both explicitly and implicit LPs Explicit: given by A, b and c Implicit: Given by c and a separation oracle for Ax b. In both cases, we require all numbers to be rational
45 Recall: Linear Programming Complexity of Linear Programming 11/27 Recall: Linear Programming Problem A problem of maximizing a linear function over a polyhedron. maximize subject to c x Ax b When stated in standard form, optimal solution occurs at a vertex. We will consider both explicitly and implicit LPs Explicit: given by A, b and c Implicit: Given by c and a separation oracle for Ax b. In both cases, we require all numbers to be rational In the explicit case, we require polynomial time in A, b, and c, the number of bits used to represent the parameters of the LP.
46 Recall: Linear Programming Complexity of Linear Programming 11/27 Recall: Linear Programming Problem A problem of maximizing a linear function over a polyhedron. maximize subject to c x Ax b When stated in standard form, optimal solution occurs at a vertex. We will consider both explicitly and implicit LPs Explicit: given by A, b and c Implicit: Given by c and a separation oracle for Ax b. In both cases, we require all numbers to be rational In the explicit case, we require polynomial time in A, b, and c, the number of bits used to represent the parameters of the LP. In the implicit case, we require polynomial time in the bit complexity of individual entries of A, b, c.
47 Complexity of Linear Programming 12/27 Theorem (Polynomial Solvability of Explicit LP) There is a polynomial time algorithm for linear programming, when the linear program is represented explicitly. Proof Sketch (Informal) Using result for weakly solving convex programs, we need 4 things: A separation oracle for Ax b: trivial when explicitly represented A first order oracle for c x: also trivial A bounding ellipsoid of volume at most an exponential times the volume of the feasible polyhedron: tricky A way of rounding an ɛ-optimal solution to an optimal vertex solution: tricky
48 Complexity of Linear Programming 12/27 Theorem (Polynomial Solvability of Explicit LP) There is a polynomial time algorithm for linear programming, when the linear program is represented explicitly. Proof Sketch (Informal) Using result for weakly solving convex programs, we need 4 things: A separation oracle for Ax b: trivial when explicitly represented A first order oracle for c x: also trivial A bounding ellipsoid of volume at most an exponential times the volume of the feasible polyhedron: tricky A way of rounding an ɛ-optimal solution to an optimal vertex solution: tricky Solution to both issues involves tedious accounting of numerical issues
49 Ellipsoid and Volume Bound (Informal) Key to tackling both difficulties is the following observation: Lemma Let v be vertex of the polyhedron Ax b. It is the case that v has polynomial bit complexity, i.e. v M, where M = O(poly( A, b )). Specifically, the solution of a system of linear equations has bit complexity polynomially related to that of the equations. Complexity of Linear Programming 13/27
50 Ellipsoid and Volume Bound (Informal) Key to tackling both difficulties is the following observation: Lemma Let v be vertex of the polyhedron Ax b. It is the case that v has polynomial bit complexity, i.e. v M, where M = O(poly( A, b )). Specifically, the solution of a system of linear equations has bit complexity polynomially related to that of the equations. Bounding ellipsoid: all vertices contained in the box 2 M x 2 M, which in turn is contained in an ellipsoid of volume exponential in M and n. Complexity of Linear Programming 13/27
51 Ellipsoid and Volume Bound (Informal) Key to tackling both difficulties is the following observation: Lemma Let v be vertex of the polyhedron Ax b. It is the case that v has polynomial bit complexity, i.e. v M, where M = O(poly( A, b )). Specifically, the solution of a system of linear equations has bit complexity polynomially related to that of the equations. To guarantee volume lowerbound, need to instead solve a relaxed problem. Specifically, relaxing to Ax b + ɛ, for sufficiently small ɛ with ɛ = poly(m). Gives volume exponentially small in M, but no smaller. Still close enough to original polyhedron so solution to relaxed problem can be rounded to solution of the original problem. Complexity of Linear Programming 13/27
52 Ellipsoid and Volume Bound (Informal) Key to tackling both difficulties is the following observation: Lemma Let v be vertex of the polyhedron Ax b. It is the case that v has polynomial bit complexity, i.e. v M, where M = O(poly( A, b )). Specifically, the solution of a system of linear equations has bit complexity polynomially related to that of the equations. Rounding to a vertex: If a point y is ɛ-optimal for the ɛ-relaxed problem, for sufficiently small ɛ chosen carefully to polynomial in description of input, then rounding to the nearest x with M bits recovers the vertex. Complexity of Linear Programming 13/27
53 Complexity of Linear Programming 14/27 Theorem (Polynomial Solvability of Implicit LP) Consider a family Π of linear programming problems I = (A, b, c) admitting the following operations in polynomial time (in I and n): A separation oracle for the polyhedron Ax b Explicit access to c Moreover, assume that every a ij, b i, c j are at most poly( I, n). Then there is a polynomial time algorithm for Π (both primal and dual*). Informal Proof Sketch (Primal)
54 Complexity of Linear Programming 14/27 Theorem (Polynomial Solvability of Implicit LP) Consider a family Π of linear programming problems I = (A, b, c) admitting the following operations in polynomial time (in I and n): A separation oracle for the polyhedron Ax b Explicit access to c Moreover, assume that every a ij, b i, c j are at most poly( I, n). Then there is a polynomial time algorithm for Π (both primal and dual*). Informal Proof Sketch (Primal) Separation oracle and first order oracle are given
55 Complexity of Linear Programming 14/27 Theorem (Polynomial Solvability of Implicit LP) Consider a family Π of linear programming problems I = (A, b, c) admitting the following operations in polynomial time (in I and n): A separation oracle for the polyhedron Ax b Explicit access to c Moreover, assume that every a ij, b i, c j are at most poly( I, n). Then there is a polynomial time algorithm for Π (both primal and dual*). Informal Proof Sketch (Primal) Separation oracle and first order oracle are given Rounding to a vertex exactly as in the explicit case. Every vertex v still has polynomial bit complexity M
56 Complexity of Linear Programming 14/27 Theorem (Polynomial Solvability of Implicit LP) Consider a family Π of linear programming problems I = (A, b, c) admitting the following operations in polynomial time (in I and n): A separation oracle for the polyhedron Ax b Explicit access to c Moreover, assume that every a ij, b i, c j are at most poly( I, n). Then there is a polynomial time algorithm for Π (both primal and dual*). Informal Proof Sketch (Primal) Separation oracle and first order oracle are given Rounding to a vertex exactly as in the explicit case. Every vertex v still has polynomial bit complexity M Bounding ellipsoid: Still true that we get a bounding ellipsoid of volume exponential in I and n
57 Complexity of Linear Programming 14/27 Theorem (Polynomial Solvability of Implicit LP) Consider a family Π of linear programming problems I = (A, b, c) admitting the following operations in polynomial time (in I and n): A separation oracle for the polyhedron Ax b Explicit access to c Moreover, assume that every a ij, b i, c j are at most poly( I, n). Then there is a polynomial time algorithm for Π (both primal and dual*). Informal Proof Sketch (Primal) Separation oracle and first order oracle are given Rounding to a vertex exactly as in the explicit case. Every vertex v still has polynomial bit complexity M Bounding ellipsoid: Still true that we get a bounding ellipsoid of volume exponential in I and n However, no lowerbound on the volume of Ax b, and can t relax to Ax b + ɛ as in the explicit case. (In fact, volume may be zero!)
58 Complexity of Linear Programming 14/27 Theorem (Polynomial Solvability of Implicit LP) Consider a family Π of linear programming problems I = (A, b, c) admitting the following operations in polynomial time (in I and n): A separation oracle for the polyhedron Ax b Explicit access to c Moreover, assume that every a ij, b i, c j are at most poly( I, n). Then there is a polynomial time algorithm for Π (both primal and dual*). Informal Proof Sketch (Primal) Separation oracle and first order oracle are given Rounding to a vertex exactly as in the explicit case. Every vertex v still has polynomial bit complexity M Bounding ellipsoid: Still true that we get a bounding ellipsoid of volume exponential in I and n However, no lowerbound on the volume of Ax b, and can t relax to Ax b + ɛ as in the explicit case. (In fact, volume may be zero!) It turns out this is still OK, but takes a lot of work.
59 Complexity of Linear Programming 14/27 Theorem (Polynomial Solvability of Implicit LP) Consider a family Π of linear programming problems I = (A, b, c) admitting the following operations in polynomial time (in I and n): A separation oracle for the polyhedron Ax b Explicit access to c Moreover, assume that every a ij, b i, c j are at most poly( I, n). Then there is a polynomial time algorithm for Π (both primal and dual*). For the dual, we need equivalence of separation and optimization. Also, we necessarily get a solution to a normalized version of the dual. (HW)
60 Outline 1 Recapping the Ellipsoid Method 2 Complexity of Convex Optimization 3 Complexity of Linear Programming 4 Equivalence of Separation and Optimization
61 Equivalence of Separation and Optimization 15/27 Separation and Optimization One interpretation of the previous theorem is that optimization of linear functions over a polytope of polynomial bit complexity reduces to implementing a separation oracle As it turns out, the two tasks are polynomial-time equivalent.
62 Separation and Optimization One interpretation of the previous theorem is that optimization of linear functions over a polytope of polynomial bit complexity reduces to implementing a separation oracle As it turns out, the two tasks are polynomial-time equivalent. Lets formalize the two questions, parametrized by a polytope P. Linear Optimization Problem Input: Linear objective c R n. Output: argmax x P c x. Separation Problem Input: y R n Output: Decide that y P, or else find h R n s.t. h x < h y for all x P. Equivalence of Separation and Optimization 15/27
63 Recall: Minimum Cost Spanning Tree Equivalence of Separation and Optimization 16/27 Given a connected undirected graph G = (V, E), and costs c e on edges e, find a minimum cost spanning tree of G.
64 Recall: Minimum Cost Spanning Tree Equivalence of Separation and Optimization 16/27 Given a connected undirected graph G = (V, E), and costs c e on edges e, find a minimum cost spanning tree of G. Spanning Tree Polytope x e X 1, for X V. e X x e = n 1 e E x e 0, for e E.
65 Recall: Minimum Cost Spanning Tree Equivalence of Separation and Optimization 16/27 Given a connected undirected graph G = (V, E), and costs c e on edges e, find a minimum cost spanning tree of G. Spanning Tree Polytope x e X 1, for X V. e X x e = n 1 e E x e 0, for e E. Optimization: Find the minimum/maximum weight spanning tree Separation: Find X V with e X x e > X 1, if one exists i.e. When edge weights are x, find a dense subgraph
66 Equivalence of Separation and Optimization 17/27 Theorem (Equivalence of Separation and Optimization for Polytopes) Consider a family P of polytopes P = {x : Ax b} described implicitly using P bits, and satisfying a ij, b i poly( P, n). Then the separation problem is solvable in poly( P, n, y ) time for P P if and only if the linear optimization problem is solvable in poly( P, n, c ) time. Colloquially, we say such polytope families are solvable.
67 Equivalence of Separation and Optimization 17/27 Theorem (Equivalence of Separation and Optimization for Polytopes) Consider a family P of polytopes P = {x : Ax b} described implicitly using P bits, and satisfying a ij, b i poly( P, n). Then the separation problem is solvable in poly( P, n, y ) time for P P if and only if the linear optimization problem is solvable in poly( P, n, c ) time. Colloquially, we say such polytope families are solvable. E.g. Spanning tree polytopes, represented by graphs, are solvable.
68 Equivalence of Separation and Optimization 17/27 Theorem (Equivalence of Separation and Optimization for Polytopes) Consider a family P of polytopes P = {x : Ax b} described implicitly using P bits, and satisfying a ij, b i poly( P, n). Then the separation problem is solvable in poly( P, n, y ) time for P P if and only if the linear optimization problem is solvable in poly( P, n, c ) time. Colloquially, we say such polytope families are solvable. E.g. Spanning tree polytopes, represented by graphs, are solvable. We already sketched the proof of the forward direction Separation optimization
69 Equivalence of Separation and Optimization 17/27 Theorem (Equivalence of Separation and Optimization for Polytopes) Consider a family P of polytopes P = {x : Ax b} described implicitly using P bits, and satisfying a ij, b i poly( P, n). Then the separation problem is solvable in poly( P, n, y ) time for P P if and only if the linear optimization problem is solvable in poly( P, n, c ) time. Colloquially, we say such polytope families are solvable. E.g. Spanning tree polytopes, represented by graphs, are solvable. We already sketched the proof of the forward direction Separation optimization For the other direction, we need polars
70 Recall: Polar Duality of Convex Sets One way of representing the all halfspaces containing a convex set. Polar Let S R n be a closed convex set containing the origin. The polar of S is defined as follows: S = {y : x y 1 for all x S} Note Every halfspace a x b with b 0 can be written as a normalized inequality y x 1, by dividing by b. S can be thought of as the normalized representations of halfspaces containing S. Equivalence of Separation and Optimization 18/27
71 Equivalence of Separation and Optimization 19/27 Properties of the Polar 1 If S is bounded and 0 interior(s), then the same holds for S. 2 S = S S = {x : y x 1 for all y S } S = {y : x y 1 for all x S}
72 Equivalence of Separation and Optimization 20/27 Polarity of Polytopes Polytopes Given a polytope P represented as Ax 1, the polar P is the convex hull of the rows of A. Facets of P correspond to vertices of P. Dually, vertices of P correspond to facets of P.
73 Proof Outline: Optimization Separation Equivalence of Separation and Optimization 21/27
74 Proof Outline: Optimization Separation Equivalence of Separation and Optimization 21/27
75 Proof Outline: Optimization Separation Equivalence of Separation and Optimization 21/27
76 S = {x : y x 1 for all y S } S = {y : x y 1 for all x S} Equivalence of Separation and Optimization 22/27
77 S = {x : y x 1 for all y S } S = {y : x y 1 for all x S} Lemma Separation over S reduces in constant time to optimization over S, and vice versa since S = S. Equivalence of Separation and Optimization 22/27
78 S = {x : y x 1 for all y S } S = {y : x y 1 for all x S} Lemma Separation over S reduces in constant time to optimization over S, and vice versa since S = S. Proof We are given vector x, and must check whether x S, and if not output separating hyperplane. Equivalence of Separation and Optimization 22/27
79 S = {x : y x 1 for all y S } S = {y : x y 1 for all x S} Lemma Separation over S reduces in constant time to optimization over S, and vice versa since S = S. Proof We are given vector x, and must check whether x S, and if not output separating hyperplane. x S iff y x 1 for all y S Equivalence of Separation and Optimization 22/27
80 S = {x : y x 1 for all y S } S = {y : x y 1 for all x S} Lemma Separation over S reduces in constant time to optimization over S, and vice versa since S = S. Proof We are given vector x, and must check whether x S, and if not output separating hyperplane. x S iff y x 1 for all y S equivalently, iff max y S y x 1. Equivalence of Separation and Optimization 22/27
81 S = {x : y x 1 for all y S } S = {y : x y 1 for all x S} Lemma Separation over S reduces in constant time to optimization over S, and vice versa since S = S. Proof We are given vector x, and must check whether x S, and if not output separating hyperplane. x S iff y x 1 for all y S equivalently, iff max y S y x 1. If we find y S s.t. y x > 1, then y is the separating hyperplane y z 1 < y x for every z S. Equivalence of Separation and Optimization 22/27
82 Optimization Separation Equivalence of Separation and Optimization 23/27
83 Beyond Polytopes Equivalence of Separation and Optimization 24/27 Essentially everything we proved about equivalence of separation and optimization for polytopes extends (approximately) to arbitrary convex sets.
84 Beyond Polytopes Essentially everything we proved about equivalence of separation and optimization for polytopes extends (approximately) to arbitrary convex sets. Problems parametrized by P, a closed convex set. Weak Optimization Problem Input: Linear objective c R n. Output: x P +ɛ, and c x max x P c x ɛ Weak Separation Problem Input: y R n Output: Decide that y P ɛ, or else find h R n with h = 1 s.t. h x < h y + ɛ for all x P. Equivalence of Separation and Optimization 24/27
85 Beyond Polytopes Essentially everything we proved about equivalence of separation and optimization for polytopes extends (approximately) to arbitrary convex sets. Problems parametrized by P, a closed convex set. Weak Optimization Problem Input: Linear objective c R n. Output: x P +ɛ, and c x max x P c x ɛ Weak Separation Problem Input: y R n Output: Decide that y P ɛ, or else find h R n with h = 1 s.t. h x < h y + ɛ for all x P. I could have equivalently stated the weak optimization problem for convex functions instead of linear. Equivalence of Separation and Optimization 24/27
86 Equivalence of Separation and Optimization 25/27 Theorem (Equivalence of Separation and Optimization for Convex Sets) Consider a family P of convex sets described implicitly using P bits. Then the weak separation problem is solvable in poly( P, n, y, log(1/ɛ)) time for P P if and only if the weak optimization problem is also solvable in poly( P, n, c, log(1/ɛ)) time.
87 Equivalence of Separation and Optimization 25/27 Theorem (Equivalence of Separation and Optimization for Convex Sets) Consider a family P of convex sets described implicitly using P bits. Then the weak separation problem is solvable in poly( P, n, y, log(1/ɛ)) time for P P if and only if the weak optimization problem is also solvable in poly( P, n, c, log(1/ɛ)) time. The approximation in this statement is necessary, since we can t solve convex optimization problems exactly. Weak separation suffices for ellipsoid, which is only approximately optimal anyways By polarity, weak optimization is equivalent to weak separation
88 Equivalence of Separation and Optimization 25/27 Theorem (Equivalence of Separation and Optimization for Convex Sets) Consider a family P of convex sets described implicitly using P bits. Then the weak separation problem is solvable in poly( P, n, y, log(1/ɛ)) time for P P if and only if the weak optimization problem is also solvable in poly( P, n, c, log(1/ɛ)) time. The approximation in this statement is necessary, since we can t solve convex optimization problems exactly. Weak separation suffices for ellipsoid, which is only approximately optimal anyways By polarity, weak optimization is equivalent to weak separation For proof / details, see the GLS book.
89 Implication: Operations preserving solvability Assume you can efficiently optimize over two convex sets P and Q Question What about P Q and P Q? Equivalence of Separation and Optimization 26/27
90 Implication: Operations preserving solvability Assume you can efficiently optimize over two convex sets P and Q Question What about P Q and P Q? P Q Yes! Simply optimize over each separately, and take the better of the two outcomes. Equivalent to optimizing over the convex hull of P Q. Implication of Separation/optimization equivalence: there is a separation oracle for convexhull(p Q). Equivalence of Separation and Optimization 26/27
91 Implication: Operations preserving solvability Assume you can efficiently optimize over two convex sets P and Q Question What about P Q and P Q? P Q Yes! Follows from equivalence of separation and optimization. Specifically, can separate over P and Q individually, therefore can separate over P Q, and then can optimize over P Q. Applications: colorful spanning tree, cardinality-constrained matching,... Equivalence of Separation and Optimization 26/27
92 Implication: Constructive Caratheodory Problem Given a point x P, where P R n is a solvable polytope, write x as a convex combination of n + 1 vertices of P, and do so in polynomial time. Existence: Caratheodory s theorem. E.g. Birkhoff Von-Neumann, fractional spanning trees, fractional matchings,... Follows from equivalence of separation and optimization. See HW. Equivalence of Separation and Optimization 27/27
Lecture 2 - Introduction to Polytopes
Lecture 2 - Introduction to Polytopes Optimization and Approximation - ENS M1 Nicolas Bousquet 1 Reminder of Linear Algebra definitions Let x 1,..., x m be points in R n and λ 1,..., λ m be real numbers.
More informationCS675: Convex and Combinatorial Optimization Spring 2018 The Simplex Algorithm. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Spring 2018 The Simplex Algorithm Instructor: Shaddin Dughmi Algorithms for Convex Optimization We will look at 2 algorithms in detail: Simplex and Ellipsoid.
More informationInteger Programming Theory
Integer Programming Theory Laura Galli October 24, 2016 In the following we assume all functions are linear, hence we often drop the term linear. In discrete optimization, we seek to find a solution x
More informationMathematical and Algorithmic Foundations Linear Programming and Matchings
Adavnced Algorithms Lectures Mathematical and Algorithmic Foundations Linear Programming and Matchings Paul G. Spirakis Department of Computer Science University of Patras and Liverpool Paul G. Spirakis
More informationCS599: Convex and Combinatorial Optimization Fall 2013 Lecture 14: Combinatorial Problems as Linear Programs I. Instructor: Shaddin Dughmi
CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 14: Combinatorial Problems as Linear Programs I Instructor: Shaddin Dughmi Announcements Posted solutions to HW1 Today: Combinatorial problems
More informationLinear Programming Duality and Algorithms
COMPSCI 330: Design and Analysis of Algorithms 4/5/2016 and 4/7/2016 Linear Programming Duality and Algorithms Lecturer: Debmalya Panigrahi Scribe: Tianqi Song 1 Overview In this lecture, we will cover
More informationCS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets Instructor: Shaddin Dughmi Outline 1 Convex sets, Affine sets, and Cones 2 Examples of Convex Sets 3 Convexity-Preserving Operations
More informationCS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 29
CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring 2018 1 / 29 CS 473: Algorithms, Spring 2018 Simplex and LP Duality Lecture 19 March 29, 2018
More informationCS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 36
CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring 2018 1 / 36 CS 473: Algorithms, Spring 2018 LP Duality Lecture 20 April 3, 2018 Some of the
More information/ Approximation Algorithms Lecturer: Michael Dinitz Topic: Linear Programming Date: 2/24/15 Scribe: Runze Tang
600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Linear Programming Date: 2/24/15 Scribe: Runze Tang 9.1 Linear Programming Suppose we are trying to approximate a minimization
More information1 Linear programming relaxation
Cornell University, Fall 2010 CS 6820: Algorithms Lecture notes: Primal-dual min-cost bipartite matching August 27 30 1 Linear programming relaxation Recall that in the bipartite minimum-cost perfect matching
More informationLinear programming and duality theory
Linear programming and duality theory Complements of Operations Research Giovanni Righini Linear Programming (LP) A linear program is defined by linear constraints, a linear objective function. Its variables
More informationMath 5593 Linear Programming Lecture Notes
Math 5593 Linear Programming Lecture Notes Unit II: Theory & Foundations (Convex Analysis) University of Colorado Denver, Fall 2013 Topics 1 Convex Sets 1 1.1 Basic Properties (Luenberger-Ye Appendix B.1).........................
More informationAdvanced Operations Research Techniques IE316. Quiz 2 Review. Dr. Ted Ralphs
Advanced Operations Research Techniques IE316 Quiz 2 Review Dr. Ted Ralphs IE316 Quiz 2 Review 1 Reading for The Quiz Material covered in detail in lecture Bertsimas 4.1-4.5, 4.8, 5.1-5.5, 6.1-6.3 Material
More informationEllipsoid II: Grötschel-Lovász-Schrijver theorems
Lecture 9 Ellipsoid II: Grötschel-Lovász-Schrijver theorems I László Lovász. Ryan O Donnell We saw in the last lecture that the Ellipsoid Algorithm can solve the optimization problem max s.t. c x Ax b
More informationWeek 5. Convex Optimization
Week 5. Convex Optimization Lecturer: Prof. Santosh Vempala Scribe: Xin Wang, Zihao Li Feb. 9 and, 206 Week 5. Convex Optimization. The convex optimization formulation A general optimization problem is
More informationSolving LP in polynomial time
Combinatorial Optimization 1 Solving LP in polynomial time Guy Kortsarz Combinatorial Optimization 2 Sketch of the ideas in the Ellipsoid algorithm If we use LP we need to show how to solve it in polynomial
More informationLinear Programming in Small Dimensions
Linear Programming in Small Dimensions Lekcija 7 sergio.cabello@fmf.uni-lj.si FMF Univerza v Ljubljani Edited from slides by Antoine Vigneron Outline linear programming, motivation and definition one dimensional
More informationCS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets. Instructor: Shaddin Dughmi
CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets Instructor: Shaddin Dughmi Announcements New room: KAP 158 Today: Convex Sets Mostly from Boyd and Vandenberghe. Read all of
More informationCS599: Convex and Combinatorial Optimization Fall 2013 Lecture 1: Introduction to Optimization. Instructor: Shaddin Dughmi
CS599: Convex and Combinatorial Optimization Fall 013 Lecture 1: Introduction to Optimization Instructor: Shaddin Dughmi Outline 1 Course Overview Administrivia 3 Linear Programming Outline 1 Course Overview
More informationLecture notes on the simplex method September We will present an algorithm to solve linear programs of the form. maximize.
Cornell University, Fall 2017 CS 6820: Algorithms Lecture notes on the simplex method September 2017 1 The Simplex Method We will present an algorithm to solve linear programs of the form maximize subject
More informationCS 435, 2018 Lecture 2, Date: 1 March 2018 Instructor: Nisheeth Vishnoi. Convex Programming and Efficiency
CS 435, 2018 Lecture 2, Date: 1 March 2018 Instructor: Nisheeth Vishnoi Convex Programming and Efficiency In this lecture, we formalize convex programming problem, discuss what it means to solve it efficiently
More informationSimplex Algorithm in 1 Slide
Administrivia 1 Canonical form: Simplex Algorithm in 1 Slide If we do pivot in A r,s >0, where c s
More informationLecture 4: Convexity
10-725: Convex Optimization Fall 2013 Lecture 4: Convexity Lecturer: Barnabás Póczos Scribes: Jessica Chemali, David Fouhey, Yuxiong Wang Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationLecture 6: Faces, Facets
IE 511: Integer Programming, Spring 2019 31 Jan, 2019 Lecturer: Karthik Chandrasekaran Lecture 6: Faces, Facets Scribe: Setareh Taki Disclaimer: These notes have not been subjected to the usual scrutiny
More informationMathematical Programming and Research Methods (Part II)
Mathematical Programming and Research Methods (Part II) 4. Convexity and Optimization Massimiliano Pontil (based on previous lecture by Andreas Argyriou) 1 Today s Plan Convex sets and functions Types
More informationConvexity: an introduction
Convexity: an introduction Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 74 1. Introduction 1. Introduction what is convexity where does it arise main concepts and
More informationEC 521 MATHEMATICAL METHODS FOR ECONOMICS. Lecture 2: Convex Sets
EC 51 MATHEMATICAL METHODS FOR ECONOMICS Lecture : Convex Sets Murat YILMAZ Boğaziçi University In this section, we focus on convex sets, separating hyperplane theorems and Farkas Lemma. And as an application
More informationLecture 5: Properties of convex sets
Lecture 5: Properties of convex sets Rajat Mittal IIT Kanpur This week we will see properties of convex sets. These properties make convex sets special and are the reason why convex optimization problems
More informationCS522: Advanced Algorithms
Lecture 1 CS5: Advanced Algorithms October 4, 004 Lecturer: Kamal Jain Notes: Chris Re 1.1 Plan for the week Figure 1.1: Plan for the week The underlined tools, weak duality theorem and complimentary slackness,
More informationConvex Optimization Lecture 2
Convex Optimization Lecture 2 Today: Convex Analysis Center-of-mass Algorithm 1 Convex Analysis Convex Sets Definition: A set C R n is convex if for all x, y C and all 0 λ 1, λx + (1 λ)y C Operations that
More information1. Lecture notes on bipartite matching February 4th,
1. Lecture notes on bipartite matching February 4th, 2015 6 1.1.1 Hall s Theorem Hall s theorem gives a necessary and sufficient condition for a bipartite graph to have a matching which saturates (or matches)
More informationLecture 5: Duality Theory
Lecture 5: Duality Theory Rajat Mittal IIT Kanpur The objective of this lecture note will be to learn duality theory of linear programming. We are planning to answer following questions. What are hyperplane
More informationIntroduction to Mathematical Programming IE496. Final Review. Dr. Ted Ralphs
Introduction to Mathematical Programming IE496 Final Review Dr. Ted Ralphs IE496 Final Review 1 Course Wrap-up: Chapter 2 In the introduction, we discussed the general framework of mathematical modeling
More informationPolytopes Course Notes
Polytopes Course Notes Carl W. Lee Department of Mathematics University of Kentucky Lexington, KY 40506 lee@ms.uky.edu Fall 2013 i Contents 1 Polytopes 1 1.1 Convex Combinations and V-Polytopes.....................
More informationConvex Geometry arising in Optimization
Convex Geometry arising in Optimization Jesús A. De Loera University of California, Davis Berlin Mathematical School Summer 2015 WHAT IS THIS COURSE ABOUT? Combinatorial Convexity and Optimization PLAN
More informationMATH 890 HOMEWORK 2 DAVID MEREDITH
MATH 890 HOMEWORK 2 DAVID MEREDITH (1) Suppose P and Q are polyhedra. Then P Q is a polyhedron. Moreover if P and Q are polytopes then P Q is a polytope. The facets of P Q are either F Q where F is a facet
More informationLecture 2 September 3
EE 381V: Large Scale Optimization Fall 2012 Lecture 2 September 3 Lecturer: Caramanis & Sanghavi Scribe: Hongbo Si, Qiaoyang Ye 2.1 Overview of the last Lecture The focus of the last lecture was to give
More informationMA4254: Discrete Optimization. Defeng Sun. Department of Mathematics National University of Singapore Office: S Telephone:
MA4254: Discrete Optimization Defeng Sun Department of Mathematics National University of Singapore Office: S14-04-25 Telephone: 6516 3343 Aims/Objectives: Discrete optimization deals with problems of
More informationLecture 16 October 23, 2014
CS 224: Advanced Algorithms Fall 2014 Prof. Jelani Nelson Lecture 16 October 23, 2014 Scribe: Colin Lu 1 Overview In the last lecture we explored the simplex algorithm for solving linear programs. While
More information6. Lecture notes on matroid intersection
Massachusetts Institute of Technology 18.453: Combinatorial Optimization Michel X. Goemans May 2, 2017 6. Lecture notes on matroid intersection One nice feature about matroids is that a simple greedy algorithm
More informationSome Advanced Topics in Linear Programming
Some Advanced Topics in Linear Programming Matthew J. Saltzman July 2, 995 Connections with Algebra and Geometry In this section, we will explore how some of the ideas in linear programming, duality theory,
More information12.1 Formulation of General Perfect Matching
CSC5160: Combinatorial Optimization and Approximation Algorithms Topic: Perfect Matching Polytope Date: 22/02/2008 Lecturer: Lap Chi Lau Scribe: Yuk Hei Chan, Ling Ding and Xiaobing Wu In this lecture,
More informationConvex Sets (cont.) Convex Functions
Convex Sets (cont.) Convex Functions Optimization - 10725 Carlos Guestrin Carnegie Mellon University February 27 th, 2008 1 Definitions of convex sets Convex v. Non-convex sets Line segment definition:
More informationOutline. CS38 Introduction to Algorithms. Linear programming 5/21/2014. Linear programming. Lecture 15 May 20, 2014
5/2/24 Outline CS38 Introduction to Algorithms Lecture 5 May 2, 24 Linear programming simplex algorithm LP duality ellipsoid algorithm * slides from Kevin Wayne May 2, 24 CS38 Lecture 5 May 2, 24 CS38
More informationApproximation Algorithms
Approximation Algorithms Prof. Tapio Elomaa tapio.elomaa@tut.fi Course Basics A 4 credit unit course Part of Theoretical Computer Science courses at the Laboratory of Mathematics There will be 4 hours
More informationACO Comprehensive Exam October 12 and 13, Computability, Complexity and Algorithms
1. Computability, Complexity and Algorithms Given a simple directed graph G = (V, E), a cycle cover is a set of vertex-disjoint directed cycles that cover all vertices of the graph. 1. Show that there
More informationDM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini
DM545 Linear and Integer Programming Lecture 2 The Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. 2. 3. 4. Standard Form Basic Feasible Solutions
More informationLinear and Integer Programming :Algorithms in the Real World. Related Optimization Problems. How important is optimization?
Linear and Integer Programming 15-853:Algorithms in the Real World Linear and Integer Programming I Introduction Geometric Interpretation Simplex Method Linear or Integer programming maximize z = c T x
More information6.854 Advanced Algorithms. Scribes: Jay Kumar Sundararajan. Duality
6.854 Advanced Algorithms Scribes: Jay Kumar Sundararajan Lecturer: David Karger Duality This lecture covers weak and strong duality, and also explains the rules for finding the dual of a linear program,
More informationChapter 4 Concepts from Geometry
Chapter 4 Concepts from Geometry An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Line Segments The line segment between two points and in R n is the set of points on the straight line joining
More informationmaximize c, x subject to Ax b,
Lecture 8 Linear programming is about problems of the form maximize c, x subject to Ax b, where A R m n, x R n, c R n, and b R m, and the inequality sign means inequality in each row. The feasible set
More informationAdvanced Operations Research Techniques IE316. Quiz 1 Review. Dr. Ted Ralphs
Advanced Operations Research Techniques IE316 Quiz 1 Review Dr. Ted Ralphs IE316 Quiz 1 Review 1 Reading for The Quiz Material covered in detail in lecture. 1.1, 1.4, 2.1-2.6, 3.1-3.3, 3.5 Background material
More informationInteger Programming ISE 418. Lecture 1. Dr. Ted Ralphs
Integer Programming ISE 418 Lecture 1 Dr. Ted Ralphs ISE 418 Lecture 1 1 Reading for This Lecture N&W Sections I.1.1-I.1.4 Wolsey Chapter 1 CCZ Chapter 2 ISE 418 Lecture 1 2 Mathematical Optimization Problems
More informationApproximation Algorithms
Approximation Algorithms Prof. Tapio Elomaa tapio.elomaa@tut.fi Course Basics A new 4 credit unit course Part of Theoretical Computer Science courses at the Department of Mathematics There will be 4 hours
More information11 Linear Programming
11 Linear Programming 11.1 Definition and Importance The final topic in this course is Linear Programming. We say that a problem is an instance of linear programming when it can be effectively expressed
More informationLecture The Ellipsoid Algorithm
8.433 Combinatorial Optimization November 4,9 Lecture The Ellipsoid Algorithm November 4,9 Lecturer: Santosh Vempala The Algorithm for Linear rograms roblem. Given a polyhedron, written as Ax b, find a
More informationLecture 3. Corner Polyhedron, Intersection Cuts, Maximal Lattice-Free Convex Sets. Tepper School of Business Carnegie Mellon University, Pittsburgh
Lecture 3 Corner Polyhedron, Intersection Cuts, Maximal Lattice-Free Convex Sets Gérard Cornuéjols Tepper School of Business Carnegie Mellon University, Pittsburgh January 2016 Mixed Integer Linear Programming
More information4 Integer Linear Programming (ILP)
TDA6/DIT37 DISCRETE OPTIMIZATION 17 PERIOD 3 WEEK III 4 Integer Linear Programg (ILP) 14 An integer linear program, ILP for short, has the same form as a linear program (LP). The only difference is that
More informationThe Simplex Algorithm
The Simplex Algorithm Uri Feige November 2011 1 The simplex algorithm The simplex algorithm was designed by Danzig in 1947. This write-up presents the main ideas involved. It is a slight update (mostly
More informationConvex Optimization CMU-10725
Convex Optimization CMU-10725 Ellipsoid Methods Barnabás Póczos & Ryan Tibshirani Outline Linear programs Simplex algorithm Running time: Polynomial or Exponential? Cutting planes & Ellipsoid methods for
More informationCMPSCI611: The Simplex Algorithm Lecture 24
CMPSCI611: The Simplex Algorithm Lecture 24 Let s first review the general situation for linear programming problems. Our problem in standard form is to choose a vector x R n, such that x 0 and Ax = b,
More informationProblem set 2. Problem 1. Problem 2. Problem 3. CS261, Winter Instructor: Ashish Goel.
CS261, Winter 2017. Instructor: Ashish Goel. Problem set 2 Electronic submission to Gradescope due 11:59pm Thursday 2/16. Form a group of 2-3 students that is, submit one homework with all of your names.
More informationPOLYHEDRAL GEOMETRY. Convex functions and sets. Mathematical Programming Niels Lauritzen Recall that a subset C R n is convex if
POLYHEDRAL GEOMETRY Mathematical Programming Niels Lauritzen 7.9.2007 Convex functions and sets Recall that a subset C R n is convex if {λx + (1 λ)y 0 λ 1} C for every x, y C and 0 λ 1. A function f :
More informationSubmodularity Reading Group. Matroid Polytopes, Polymatroid. M. Pawan Kumar
Submodularity Reading Group Matroid Polytopes, Polymatroid M. Pawan Kumar http://www.robots.ox.ac.uk/~oval/ Outline Linear Programming Matroid Polytopes Polymatroid Polyhedron Ax b A : m x n matrix b:
More information2. Convex sets. affine and convex sets. some important examples. operations that preserve convexity. generalized inequalities
2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationNumerical Optimization
Convex Sets Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Let x 1, x 2 R n, x 1 x 2. Line and line segment Line passing through x 1 and x 2 : {y
More information2. Convex sets. x 1. x 2. affine set: contains the line through any two distinct points in the set
2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationPolar Duality and Farkas Lemma
Lecture 3 Polar Duality and Farkas Lemma October 8th, 2004 Lecturer: Kamal Jain Notes: Daniel Lowd 3.1 Polytope = bounded polyhedron Last lecture, we were attempting to prove the Minkowsky-Weyl Theorem:
More informationLecture 2: August 29, 2018
10-725/36-725: Convex Optimization Fall 2018 Lecturer: Ryan Tibshirani Lecture 2: August 29, 2018 Scribes: Adam Harley Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have
More informationConvex Optimization. Convex Sets. ENSAE: Optimisation 1/24
Convex Optimization Convex Sets ENSAE: Optimisation 1/24 Today affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes
More informationApproximation Algorithms: The Primal-Dual Method. My T. Thai
Approximation Algorithms: The Primal-Dual Method My T. Thai 1 Overview of the Primal-Dual Method Consider the following primal program, called P: min st n c j x j j=1 n a ij x j b i j=1 x j 0 Then the
More information3 No-Wait Job Shops with Variable Processing Times
3 No-Wait Job Shops with Variable Processing Times In this chapter we assume that, on top of the classical no-wait job shop setting, we are given a set of processing times for each operation. We may select
More informationCS 372: Computational Geometry Lecture 10 Linear Programming in Fixed Dimension
CS 372: Computational Geometry Lecture 10 Linear Programming in Fixed Dimension Antoine Vigneron King Abdullah University of Science and Technology November 7, 2012 Antoine Vigneron (KAUST) CS 372 Lecture
More informationLecture 2: August 29, 2018
10-725/36-725: Convex Optimization Fall 2018 Lecturer: Ryan Tibshirani Lecture 2: August 29, 2018 Scribes: Yingjing Lu, Adam Harley, Ruosong Wang Note: LaTeX template courtesy of UC Berkeley EECS dept.
More information15-451/651: Design & Analysis of Algorithms October 11, 2018 Lecture #13: Linear Programming I last changed: October 9, 2018
15-451/651: Design & Analysis of Algorithms October 11, 2018 Lecture #13: Linear Programming I last changed: October 9, 2018 In this lecture, we describe a very general problem called linear programming
More informationIntroduction to Modern Control Systems
Introduction to Modern Control Systems Convex Optimization, Duality and Linear Matrix Inequalities Kostas Margellos University of Oxford AIMS CDT 2016-17 Introduction to Modern Control Systems November
More informationCombinatorial Optimization
Combinatorial Optimization Frank de Zeeuw EPFL 2012 Today Introduction Graph problems - What combinatorial things will we be optimizing? Algorithms - What kind of solution are we looking for? Linear Programming
More informationLinear Programming. Larry Blume. Cornell University & The Santa Fe Institute & IHS
Linear Programming Larry Blume Cornell University & The Santa Fe Institute & IHS Linear Programs The general linear program is a constrained optimization problem where objectives and constraints are all
More informationbe a polytope. has such a representation iff it contains the origin in its interior. For a generic, sort the inequalities so that
( Shelling (Bruggesser-Mani 1971) and Ranking Let be a polytope. has such a representation iff it contains the origin in its interior. For a generic, sort the inequalities so that. a ranking of vertices
More informationLinear Programming and its Applications
Linear Programming and its Applications Outline for Today What is linear programming (LP)? Examples Formal definition Geometric intuition Why is LP useful? A first look at LP algorithms Duality Linear
More informationTHEORY OF LINEAR AND INTEGER PROGRAMMING
THEORY OF LINEAR AND INTEGER PROGRAMMING ALEXANDER SCHRIJVER Centrum voor Wiskunde en Informatica, Amsterdam A Wiley-Inter science Publication JOHN WILEY & SONS^ Chichester New York Weinheim Brisbane Singapore
More information15. Cutting plane and ellipsoid methods
EE 546, Univ of Washington, Spring 2012 15. Cutting plane and ellipsoid methods localization methods cutting-plane oracle examples of cutting plane methods ellipsoid method convergence proof inequality
More informationLinear Programming and Integer Linear Programming
Linear Programming and Integer Linear Programming Kurt Mehlhorn May 26, 2013 revised, May 20, 2014 1 Introduction 2 2 History 3 3 Expressiveness 3 4 Duality 4 5 Algorithms 8 5.1 Fourier-Motzkin Elimination............................
More informationCombinatorial Geometry & Topology arising in Game Theory and Optimization
Combinatorial Geometry & Topology arising in Game Theory and Optimization Jesús A. De Loera University of California, Davis LAST EPISODE... We discuss the content of the course... Convex Sets A set is
More informationAMS : Combinatorial Optimization Homework Problems - Week V
AMS 553.766: Combinatorial Optimization Homework Problems - Week V For the following problems, A R m n will be m n matrices, and b R m. An affine subspace is the set of solutions to a a system of linear
More informationProbabilistic Graphical Models
School of Computer Science Probabilistic Graphical Models Theory of Variational Inference: Inner and Outer Approximation Eric Xing Lecture 14, February 29, 2016 Reading: W & J Book Chapters Eric Xing @
More informationLecture 10,11: General Matching Polytope, Maximum Flow. 1 Perfect Matching and Matching Polytope on General Graphs
CMPUT 675: Topics in Algorithms and Combinatorial Optimization (Fall 2009) Lecture 10,11: General Matching Polytope, Maximum Flow Lecturer: Mohammad R Salavatipour Date: Oct 6 and 8, 2009 Scriber: Mohammad
More information1 Bipartite maximum matching
Cornell University, Fall 2017 Lecture notes: Matchings CS 6820: Algorithms 23 Aug 1 Sep These notes analyze algorithms for optimization problems involving matchings in bipartite graphs. Matching algorithms
More informationLecture 7. s.t. e = (u,v) E x u + x v 1 (2) v V x v 0 (3)
COMPSCI 632: Approximation Algorithms September 18, 2017 Lecturer: Debmalya Panigrahi Lecture 7 Scribe: Xiang Wang 1 Overview In this lecture, we will use Primal-Dual method to design approximation algorithms
More information1 Unweighted Set Cover
Comp 60: Advanced Algorithms Tufts University, Spring 018 Prof. Lenore Cowen Scribe: Yuelin Liu Lecture 7: Approximation Algorithms: Set Cover and Max Cut 1 Unweighted Set Cover 1.1 Formulations There
More informationIntroduction to Mathematical Programming IE406. Lecture 20. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 20 Dr. Ted Ralphs IE406 Lecture 20 1 Reading for This Lecture Bertsimas Sections 10.1, 11.4 IE406 Lecture 20 2 Integer Linear Programming An integer
More information5.3 Cutting plane methods and Gomory fractional cuts
5.3 Cutting plane methods and Gomory fractional cuts (ILP) min c T x s.t. Ax b x 0integer feasible region X Assumption: a ij, c j and b i integer. Observation: The feasible region of an ILP can be described
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 6
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 6 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 19, 2012 Andre Tkacenko
More information16.410/413 Principles of Autonomy and Decision Making
16.410/413 Principles of Autonomy and Decision Making Lecture 17: The Simplex Method Emilio Frazzoli Aeronautics and Astronautics Massachusetts Institute of Technology November 10, 2010 Frazzoli (MIT)
More informationLecture 2: August 31
10-725/36-725: Convex Optimization Fall 2016 Lecture 2: August 31 Lecturer: Lecturer: Ryan Tibshirani Scribes: Scribes: Lidan Mu, Simon Du, Binxuan Huang 2.1 Review A convex optimization problem is of
More informationApproximation Algorithms
Approximation Algorithms Group Members: 1. Geng Xue (A0095628R) 2. Cai Jingli (A0095623B) 3. Xing Zhe (A0095644W) 4. Zhu Xiaolu (A0109657W) 5. Wang Zixiao (A0095670X) 6. Jiao Qing (A0095637R) 7. Zhang
More informationApproximate Nearest Neighbor Problem: Improving Query Time CS468, 10/9/2006
Approximate Nearest Neighbor Problem: Improving Query Time CS468, 10/9/2006 Outline Reducing the constant from O ( ɛ d) to O ( ɛ (d 1)/2) in uery time Need to know ɛ ahead of time Preprocessing time and
More information6 Randomized rounding of semidefinite programs
6 Randomized rounding of semidefinite programs We now turn to a new tool which gives substantially improved performance guarantees for some problems We now show how nonlinear programming relaxations can
More informationAlgorithms for Integer Programming
Algorithms for Integer Programming Laura Galli November 9, 2016 Unlike linear programming problems, integer programming problems are very difficult to solve. In fact, no efficient general algorithm is
More information