/24 Lecture: Convex Sets http://bicmr.pku.edu.cn/~wenzw/opt-27-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes
Introduction 2/24 affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual cones and generalized inequalities
Affine set through x, x 2 : all points line through x, x 2 : all points x = θx + ( θ)x 2 (θ R) x = θx + ( θ)x 2 (θ R) θ =.2 x θ = θ =.6 x 2 θ = θ =.2 neaffine set: set: contains contains the the line line through any two distinct points points thein set the s example: solution set of linear equations {x Ax = b} mple: solution set of linear equations {x Ax = b} (conversely, every affine set can be expressed as solution set of system of linear equations) versely, every affine set can be expressed as solution set of syste 3/24
4/24 Convex set θ x = θx + ( θ)x 2 line segment between x and x 2 : all points with θ x = θx + ( θ)x 2 et: ontains contains line segment line segment between contains line segment between between any any any two two two points points points the the in convex set: contains line segment between any two points in the set x 2,, xx 2 C, 2 C, C, θ θ θ = = = θx θx + θx+ ( ( + ( θ)x θ)x 2 θ)x 2 C 2 x, x 2 C, θ = θx + ( θ)x 2 C e examples (one convex, two (one convex, convex, two two nonconvex two nonconvex sets) sets) sets)
5/24 vex combination of x,..., x k : any point x of the form Convex combination x = θ x + and θ 2 xconvex 2 + + hull θ k x k x = θ x + θ 2 x 2 + + θ k x k convex combination of x,..., x k : any point x of the form + θ k =, θ i θ + + θ k =, θ i x = θ x + θ 2 x 2 +... + θ k x k with θ +... + θ k =, θ i ull conv S: set of all convex combinations of points in S vex hull conv S: set of all convex combinations of points in convex hull convs: set of all convex combinations of points in S
Convex (nonnegative) cone combination of x and x 2 : any point of the fo x = θ x + θ 2 x 2 conic (nonnegative) combination of x and x 2 : any point of the form θ, θ 2 with θ, θ 2 x = θ x + θ 2 x 2 x x 2 convex cone: set that contains all conic combinations of points in the set ex cone: set that contains all conic combinations of points in th 6/24
Hyperplanes and halfspaces x a hyperplane: set of the form halfspace: set of the form T x = b perplanes and halfspaces {x a T x = b}(a halfspace: ) set of the form {x a T {x x ab}(a T x ) b} (a ) e form {x a T x = b} (a ) a x a x a T x = b form {x a T x a b} is the (a normal ) vector a is the normal vector hyperplanes a are affine and convex; halfspaces are convex hyperplanes are affine and convex; halfspaces are convex x Convex sets a T x b x x a T x b a T x b T 7/24
8/24 lidean) Euclidean ball with ballscenter and ellipsoids x c and radius r: (Euclidean) ball with center x c and radius r: B(x c, r) = {x x x c 2 r} = {x c + ru u 2 } oid: set of the form B(x c, r) = {x x x c 2 r} = {x c + ru u 2 } ellipsoid: set of the form {x (x x c ) T P (x x c ) } {x (x x c ) T P (x x c ) } with P S n ++ (i.e., P symmetric positive definite) P S n ++ (i.e., P symmetric positive definite) other representation: {x c + Au u 2 } with A square and nonsingular x c
Norm balls and norm cones norm: a function that satisfies x ; x = if and only if x = tx = t x for t R norm; symb is particular norm x + y x + y {x x x c r} notation: is general (unspecified) norm; symb is particular norm norm ball with center x c and radius r: {x x x c r} t.5 x 2 x norm cone: {(x, t) x t} Euclidean norm cone is called second-order cone norm balls and cones are convex 9/24
/24 Polyhedra solution set of finitely many linear inequalities and equalities solution set of finitely many linear inequalities and equalities Ax b, Cx = d Ax b, Cx = d (A (A R R m n m n,, C R p n p n,, is is componentwise inequality) a a2 a 5 P a 3 a 4 polyhedron is intersection of finite number of halfspaces and hyperplanes polyhedron is intersection of finite number of halfspaces and hyperplanes
Positive semidefinite Positive conesemidefinite cone notation: notation: S n S n is set of symmetric n n matrices is set of symmetric n n matrices S n S n + = {X + = {X S n S n X }: positive semidefinite n n matrices X }: positive semidefinite n n matrices X X S n S n + + z T Xz z T Xz for all forz all z SS n + n is a convex + is a convex cone SS n ++ n = {X S n X }: positive definite n matrices ++ = {X S n X }: positive definite n n matrices [[ ] ] x y x y example: S y z 2 + S 2 y z + z.5 y x.5 /24
Operations that preserve convexity 2/24 practical methods for establishing convexity of a set C apply definition x, x 2 C, θ = θx + ( θ)x 2 C 2 show that C is obtained from simple convex sets (hyperplanes, halfspaces, norm balls,... ) by operations that preserve convexity intersection affine functions perspective function linear-fractional functions
the Intersection intersection of (any of number (any number of) convex of) convex sets isets convex is convex 3/24 the intersection of (any number of ) convex sets is convex example: example: S = {x S = {x R m R p(t) m p(t) for t for t π/3} π/3} S = {x R m p(t) for t π/3} ewhere p(t) = p(t) x cos = x t + cos xt 2 t + cos + x 2 xcos 2t 2 cos + 2t + 2t... + x m + cos xmt m mt cos mt for = for2: m = 2: 2 2 p(t) p(t) x2 x2 S S π/3 π/32π/3 2π/3 π π t t 2 2 2 2 2 2 x x
Affine function 4/24 suppose f : R n R m is affine (f (x) = Ax + b with A R m n, b R m ) the image of a convex set under f is convex S R n convex = f (S) = {f (x) x S} convex the inverse image f (C) of a convex set under f is convex C R m convex = f (C) = {x R n f (x) C} convex examples scaling, translation, projection solution set of linear matrix inequality {x x A +... + x m A m B} (with A i, B S p ) hyperbolic cone {x x T Px (c T x) 2, c T x } (with P S n + )
Perspective and linear-fractional function 5/24 perspective function P : R n+ R n : P(x, t) = x/t, dom P = {(x, t) t > } images and inverse images of convex sets under perspective are convex linear-fractional function f : R n R m : f (x) = Ax + b c T x + d, dom f = {x ct x + d > } images and inverse images of convex sets under linear-fractional functions are convex
6/24 example of a linear-fractional f(x) f(x) = function = x x + x 2 + x + x 2 + x f (x) = x + x 2 + x x2 x2 C C x2 x2 f(c) f(c) x x x x
Separating Separating hyperplane hyperplane theorem theorem If C and D are disjoint convex sets, then there exists a, b such that and D are disjoint convex sets, then there exists a, b such that a T x b for x C, a T x b for x D where C and a T x D are b for thex closure C, of Ca T and x D. b for x D a T x b a T x b D C a hyperplane the hyperplane {x a T {x ax = T x b} = separates b} separates C and C and D D t separation strict separation requires requires additional additional assumptions assumptions (e.g., C (e.g., is closed, C is closed, D is a D leton) is a singleton) 7/24
Supporting hyperplane theorem upporting hyperplane to set C at boundary point x : supporting hyperplane to set C at boundary point x : {x{x a a T x = a T ax T x} } here where a a and and a T ax T x a T a T x for all x C C x a upporting supporting hyperplane hyperplane theorem: theorem: if C if C is is convex, a nonempty then convex there exists set, a then there exists a supporting hyperplane at every boundary point of upporting hyperplane at every boundary point of C C 8/24
Generalized inequalities 9/24 a convex cone K R n is a proper cone if K is closed (contains its boundary) K is solid (has nonempty interior) K is pointed (contains no line) examples nonnegative orthant K = R n + = {x R n x i, i =,..., n} positive semidefinite cone K = S n + nonnegative polynomials on [, ]: K = {x R n x + x 2 t + x 3 t 2 +... + x n t n for t [, ]}
2/24 generalized inequality defined by a proper cone K : x K y y x K, x K y y x int K examples componentwise inequality (K = R n +) x R n + y x i y i, i =,..., n matrix inequality (K = S n + ) X S n + Y Y X positive semidefinite these two types are so common that we drop the subscript in K properties: many properties of K are similar to on R, e.g., x K y, u K v = x + u K y + v
Dual cones and generalized inequalities 2/24 dual cone of a cone K : examples K = R n + : K = R n + K = S n + : K = S n + K = {y y T x for all x K} K = {(x, t) x 2 t} : K = {(x, t) x 2 t} K = {(x, t) x t} : K = {(x, t) x t} first three examples are self-dual cones dual cones of proper cones are proper, hence define generalized inequalities: y K y T x for all x K
22/24 d minimal elements Minimum and minimal elements ering: we can have x K y and y K x K is not in general a linear ordering : we can have x K y and y K x t of S with respect to K if x S is the minimum element of S with respect to K if = x K y y S = x K y S xwith S respect is a minimal to K element if of S with respect to K if K x = y = x y S, y K x = y = x x S S 2 x 2 x is the minimum element of S example (K = R2 +) x 2 is a minimal element of S 2
Minimum and minimal elements via dual inequalities Minimum and minimal elements via du minimum element w.r.t. Minimum and minimal K elements via dual inequalities minimum element w.r.t. K minimum element w.r.t. K x is minimum x is minimum of S iff for element all of S iff for all x is minimum element of S iff for all λ λ K, K x, isxthe isλ the K unique, xminimizer is the unique minimizer of λof T S zλover T z over S Sof λ T z over S x x minimal element w.r.t. K minimal element w.r.t. K minimal if xelement minimizes w.r.t. λ T z over K S for some λ K, then x is minimal if x minimizes λ λ T z over S for some λ K, then λ if x minimizes λ T z over S for some λ K, then x is minimal x S if x is a minimal element of a convex x λ 2 x 2 S set S, then there exists a nonzero λ K such that x minimizes λ T z λ 2 x 2 over ifsx is a minimal element of a convex set S, then there exists a nonzero λ such that x minimizes λ T z over S 23/24
optimal production frontier 24/24 different production methods use different amounts of resources erent amounts x R n of resources x R n for all production possible production set P : resource methodsvectors x for all possible production methods orrespond to resource vectors x efficient (Pareto optimal) methods correspond to resource vectors x that are minimal w.r.t. R n + fuel x x x5 2 x 4 λ P x 3 labor example (n = 2) x, x 2, x 3 are efficient; x 4, x 5 are not