CS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets. Instructor: Shaddin Dughmi

Transcription

1 CS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets Instructor: Shaddin Dughmi

2 Outline 1 Convex sets, Affine sets, and Cones 2 Examples of Convex Sets 3 Convexity-Preserving Operations 4 Separation Theorems

3 Convex Sets A set S R n is convex if the line segment between any two points in S lies in S. i.e. if x, y S and θ [0, 1], then θx + (1 θ)y S. Convex sets, Affine sets, and Cones 0/20

4 Convex Sets A set S R n is convex if the line segment between any two points in S lies in S. i.e. if x, y S and θ [0, 1], then θx + (1 θ)y S. Equivalent Definition S is convex if every convex combination of points in S lies in S. Convex Combination Finite: y is a convex combination of x 1,..., x k if y = θ 1 x θ k x k, where θ i 0 and i θ i = 1. General: expectation of probability measure on S. Convex sets, Affine sets, and Cones 0/20

5 Convex Sets Convex Hull The convex hull of S R n is the smallest convex set containing S. Intersection of all convex sets containing S The set of all convex combinations of points in S Convex sets, Affine sets, and Cones 1/20

6 Convex Sets Convex Hull The convex hull of S R n is the smallest convex set containing S. Intersection of all convex sets containing S The set of all convex combinations of points in S A set S is convex if and only if convexhull(s) = S. Convex sets, Affine sets, and Cones 1/20

7 Affine Set A set S R n is affine if the line passing through any two points in S lies in S. i.e. if x, y S and θ R, then θx + (1 θ)y S. Obviously, affine sets are convex. Convex sets, Affine sets, and Cones 2/20

8 Affine Set A set S R n is affine if the line passing through any two points in S lies in S. i.e. if x, y S and θ R, then θx + (1 θ)y S. Obviously, affine sets are convex. Equivalent Definition S is affine if every affine combination of points in S lies in S. Affine Combination y is an affine combination of x 1,..., x k if y = θ 1 x θ k x k, and i θ i = 1. Generalizes convex combinations Convex sets, Affine sets, and Cones 2/20

9 Convex sets, Affine sets, and Cones 3/20 Affine Sets Equivalent Definition II S is affine if and only if it is a shifted subspace i.e. S = x 0 + V, where V is a linear subspace of R n. Any x 0 S will do, and yields the same V. The dimension of S is the dimension of subspace V.

10 Convex sets, Affine sets, and Cones 3/20 Affine Sets Equivalent Definition II S is affine if and only if it is a shifted subspace i.e. S = x 0 + V, where V is a linear subspace of R n. Any x 0 S will do, and yields the same V. The dimension of S is the dimension of subspace V. Equivalent Definition III S is affine if and only if it is the solution of a set of linear equations (i.e. the intersection of hyperplanes). i.e. S = {x : Ax = b} for some matrix A R m n and b R m.

11 Affine Sets Affine Hull The affine hull of S R n is the smallest affine set containing S. Intersection of all affine sets containing S The set of all affine combinations of points in S Convex sets, Affine sets, and Cones 4/20

12 Affine Sets Affine Hull The affine hull of S R n is the smallest affine set containing S. Intersection of all affine sets containing S The set of all affine combinations of points in S A set S is affine if and only if affinehull(s) = S. Convex sets, Affine sets, and Cones 4/20

13 Affine Sets Affine Hull The affine hull of S R n is the smallest affine set containing S. Intersection of all affine sets containing S The set of all affine combinations of points in S A set S is affine if and only if affinehull(s) = S. Affine Dimension The affine dimension of a set is the dimension of its affine hull Convex sets, Affine sets, and Cones 4/20

14 Cones A set K R n is a cone if the ray from the origin through every point in K is in K i.e. if x K and θ 0, then θx K. Note: every cone contains 0. Convex sets, Affine sets, and Cones 5/20

15 Cones A set K R n is a cone if the ray from the origin through every point in K is in K i.e. if x K and θ 0, then θx K. Note: every cone contains 0. Special Cones A convex cone is a cone that is convex A cone is pointed if whenever x K and x 0, then x K. We will mostly mention proper cones: convex, pointed, closed, and of full affine dimension. Convex sets, Affine sets, and Cones 5/20

16 Cones Equivalent Definition K is a convex cone if every conic combination of points in K lies in K. Conic Combination y is a conic combination of x 1,..., x k if y = θ 1 x θ k x k, where θ i 0. Convex sets, Affine sets, and Cones 6/20

17 Cones Conic Hull The conic hull of K R n is the smallest convex cone containing K Intersection of all convex cones containing K The set of all conic combinations of points in K Convex sets, Affine sets, and Cones 7/20

18 Cones Conic Hull The conic hull of K R n is the smallest convex cone containing K Intersection of all convex cones containing K The set of all conic combinations of points in K A set K is a convex cone if and only if conichull(k) = K. Convex sets, Affine sets, and Cones 7/20

19 Convex sets, Affine sets, and Cones 8/20 Cones Polyhedral Cone A cone is polyhedral if it is the set of solutions to a finite set of homogeneous linear inequalities Ax 0.

20 Outline 1 Convex sets, Affine sets, and Cones 2 Examples of Convex Sets 3 Convexity-Preserving Operations 4 Separation Theorems

21 Examples of Convex Sets 9/20 Linear Subspace: Affine, Cone Hyperplane: Affine, cone if includes 0 Halfspace: Cone if origin on boundary Line: Affine, cone if includes 0 Ray: Cone if endpoint at 0 Line segment

22 Examples of Convex Sets 10/20 Polyhedron: finite intersection of halfspaces Polytope: Bounded polyhedron

23 Nonnegative Orthant R n +: Polyhedral cone Simplex: convex hull of affinely independent points Unit simplex: x 0, i x i 1 Probability simplex: x 0, i x i = 1. Examples of Convex Sets 11/20

24 Euclidean ball: {x : x x c 2 r} for center x c and radius r Ellipsoid: { x : (x x c ) T P 1 (x x c ) 1 } for symmetric P 0 Equivalently: {x c + Au : u 2 1} for some linear map A Examples of Convex Sets 12/20

25 Norm ball: {x : x c r} for any norm. Examples of Convex Sets 13/20

26 Examples of Convex Sets 13/20 Norm ball: {x : x c r} for any norm. Norm cone: {(x, r) : x r} Cone of symmetric positive semi-definite matrices M Symmetric matrix A 0 iff x T Ax 0 for all x

27 Outline 1 Convex sets, Affine sets, and Cones 2 Examples of Convex Sets 3 Convexity-Preserving Operations 4 Separation Theorems

28 Intersection The intersection of two convex sets is convex. This holds for the intersection of an infinite number of sets. Examples Polyhedron: intersection of halfspaces PSD cone: intersection of linear inequalities z T Az 0, for all z R n. Convexity-Preserving Operations 14/20

29 Intersection The intersection of two convex sets is convex. This holds for the intersection of an infinite number of sets. Examples Polyhedron: intersection of halfspaces PSD cone: intersection of linear inequalities z T Az 0, for all z R n. In fact, we will see that every closed convex set is the intersection of a (possibly infinite) set of halfspaces. Convexity-Preserving Operations 14/20

30 Convexity-Preserving Operations 15/20 Affine Maps If f : R n R m is an affine function (i.e. f(x) = Ax + b), then f(s) is convex whenever S R n is convex f 1 (T ) is convex whenever T R m is convex f(θx + (1 θ)y) = A(θx + (1 θ)y) + b = θ(ax + b) + (1 θ)(ay + b)) = θf(x) + (1 θ)f(y)

31 Convexity-Preserving Operations 16/20 Examples An ellipsoid is image of a unit ball after an affine map A polyhedron Ax b is inverse image of nonnegative orthant under f(x) = b Ax

32 Convexity-Preserving Operations 17/20 Perspective Function Let P : R n+1 R n be P (x, t) = x/t. P (S) is convex whenever S R n+1 is convex P 1 (T ) is convex whenever T R n is convex

33 Convexity-Preserving Operations 17/20 Perspective Function Let P : R n+1 R n be P (x, t) = x/t. P (S) is convex whenever S R n+1 is convex P 1 (T ) is convex whenever T R n is convex Generalizes to linear fractional functions f(x) = Ax+b c T x+d Composition of perspective with affine.

34 Outline 1 Convex sets, Affine sets, and Cones 2 Examples of Convex Sets 3 Convexity-Preserving Operations 4 Separation Theorems

35 Separation Theorems 18/20 Separating Hyperplane Theorem If A, B R n are disjoint convex sets, then there is a hyperplane weakly separating them. That is, there is a R n and b R such that a x b for every x A and a y b for every y B.

36 Separation Theorems 18/20 Separating Hyperplane Theorem (Strict Version) If A, B R n are disjoint closed convex sets, and at least one of them is compact, then there is a hyperplane strictly separating them. That is, there is a R n and b R such that a x < b for every x A and a y > b for every y B.

37 Farkas Lemma Let K be a closed convex cone and let w K. There is z R n such that z x 0 for all x K, and z w < 0. Separation Theorems 19/20

38 Separation Theorems 20/20 Supporting Hyperplane Supporting Hyperplane Theorem. If S R n is a closed convex set and y is on the boundary of S, then there is a hyperplane supporting S at y. That is, there is a R n and b R such that a x b for every x S and a y = b.