Lecture 2 Convex Sets

Transcription

1 Optimization Theory and Applications Lecture 2 Convex Sets Prof. Chun-Hung Liu Dept. of Electrical and Computer Engineering National Chiao Tung University Fall /9/29 Lecture 2: Convex Sets 1

2 Outline Affine and convex sets Some important examples Operations that preserve convexity Generalized inequalities Separating and supporting hyperplanes Dual cones and generalized inequalities 2016/9/29 Lecture 2: Convex Sets 2

3 Affine Set 2016/9/29 Lecture 2: Convex Sets 3

4 Affine Set A set C R n is affine if the line through any two distinct points in C lies in C, i.e., if for any x 1,x 2 2 C and 2 R, we have x 1 +(1 )x 2 2 C. In other words, C contains the linear combination of any two points in C, provided the coefficients in the linear combination sum to one. Generalization: If C is an affine set, x 1,...,x k 2 C, and k =1, then the point 1 x k x k also belongs to C. 2016/9/29 Lecture 2: Convex Sets 4

5 Affine Hull The set of all affine combinations of points in some set C R n is called the affine hull of C, and denoted a C : The affine hull is the smallest affine set that contains C, in the following sense: if S is any affine set with C S, then a C S. 2016/9/29 Lecture 2: Convex Sets 5

6 Convex Set 2016/9/29 Lecture 2: Convex Sets 6

7 Convex Combination and Convex Hull conv C C 2016/9/29 Lecture 2: Convex Sets 7

8 Convex Combination and Convex Hull Generalization: Suppose 1, 2,... satisfy and x 1,x 2,...2 C, where C R n is convex. Then if the series converges. More R generally, suppose p : R n! Rsatisfies p(x) 0 for all x 2 C and p(x)dx =1 C, where C R n is convex. Then if the integral exists. 2016/9/29 Lecture 2: Convex Sets 8

9 Convex Cone A set C is called a cone if for every and we have. x 2 C 0 x 2 C Conic (nonnegative) combination Convex cone: 2016/9/29 Lecture 2: Convex Sets 9

10 Conic Hull The conic hulls (shown shaded) of two sets 2016/9/29 Lecture 2: Convex Sets 10

11 Hyperplanes and Halfspaces Boundary: a T x = b Open halfspace: {x a T x<b} 2016/9/29 Lecture 2: Convex Sets 11

12 Hyperplanes: Geometrical Interpretation Geometrically, the hyperplane {x a T x = b} can be interpreted as the set of points with a constant inner product to a given vector a, or as a hyperplane with normal vector a; the constant b 2 R determines the offset of the hyperplane from the origin. This geometric interpretation can be understood by expressing the hyperplane in the form where x 0 is any point in the hyperplane (i.e., any point that satisfies a T x = b ). This representation can in turn be expressed as where a? denotes the orthogonal complement of a, i.e., the set of all vectors orthogonal to it: 2016/9/29 Lecture 2: Convex Sets 12

13 Euclidean Balls and Ellipsoids A ball is an ellipsoid with P = r 2 I 2016/9/29 Lecture 2: Convex Sets 13

14 Norm balls and Norm Cones Norm balls and cones are convex! 2016/9/29 Lecture 2: Convex Sets 14

15 Polyhedra A polyhedron is defined as the solution set of a finite number of linear equalities and inequalities: A polyhedron is thus the intersection of a finite number of halfspaces and hyperplanes. 2016/9/29 Lecture 2: Convex Sets 15

16 Polyhedra 2016/9/29 Lecture 2: Convex Sets 16

17 Notation: Positive Semidefinite Cone Positive semidefinite cone 2016/9/29 Lecture 2: Convex Sets 17

18 Operations that Preserve Convexity Practical methods for establishing convexity of a set C 2016/9/29 Lecture 2: Convex Sets 18

19 Intersection Convexity is preserved under intersection: if S 1 and S 2 are convex, then S 1 \ S 2 is convex. This property extends to the intersection T of an infinite number of sets: if S is convex for every 2 A, then 2A S is convex. A polyhedron is the intersection of halfspaces and hyperplanes (which are convex), and therefore is convex. 2016/9/29 Lecture 2: Convex Sets 19

20 Intersection Fig Fig /9/29 Lecture 2: Convex Sets 20

21 Affine Function 2016/9/29 Lecture 2: Convex Sets 21

22 Affine Function 2016/9/29 Lecture 2: Convex Sets 22

23 Perspective and Linear-fractional Function The perspective function scales or normalizes vectors so the last component is one, and then drops the last component. Images and inverse images of convex sets under perspective are convex. A linear-fractional function is formed by composing the perspective function with an affine function Images and inverse images of convex sets under linear-fractional functions are convex. 2016/9/29 Lecture 2: Convex Sets 23

24 Perspective and Linear-fractional Function We can interpret the perspective function as the action of a pin-hole camera. A pin-hole camera (in R 3 ) consists of an opaque horizontal plane x 3 =0, with a single pin-hole at the origin, through which light can pass, and a horizontal image plane x 3 = 1. An object at x, above the camera (i.e., with x 3 > 0 ), forms an image at the point (x 1 /x 3,x 2 /x 3, 1) on the image plane. Dropping the last component of the image point (since it is always -1), the image of a point at x appears at y = (x 1 /x 3,x 2 /x 3 )= P (x) on the image plane. Fig Pin-hole camera interpretation of perspective function. 2016/9/29 Lecture 2: Convex Sets 24

25 Perspective and Linear-fractional Function 2016/9/29 Lecture 2: Convex Sets 25

26 Proper Cone l K is convex 2016/9/29 Lecture 2: Convex Sets 26

27 Generalized Inequalities 2016/9/29 Lecture 2: Convex Sets 27

28 Generalized Inequalities 2016/9/29 Lecture 2: Convex Sets 28

29 Generalized Inequalities 2016/9/29 Lecture 2: Convex Sets 29

30 Minimum and Minimal Elements Fig /9/29 Lecture 2: Convex Sets 30

31 Minimum and Minimal Elements 2016/9/29 Lecture 2: Convex Sets 31

32 Separating Hyperplane Theorem 2016/9/29 Lecture 2: Convex Sets 32

33 Proof of Separating Hyperplane Theorem We assume that the (Euclidean) distance between C and D, defined as is positive, and that there exist points c 2 C and d 2 D that achieve the minimum distance, i.e., kc dk 2 = dist(c, D) (These conditions are satisfied, for example, when C and D are closed and one set is bounded.) Define We will show that the affine function is nonpositive on C and nonnegative on D, i.e., that the hyperplane {x a T x = b}separates C and D. 2016/9/29 Lecture 2: Convex Sets 33

34 Proof of Separating Hyperplane Theorem This hyperplane is perpendicular to the line segment between c and d, and passes through its midpoint, as shown in Fig Fig /9/29 Lecture 2: Convex Sets 34

35 Proof of Separating Hyperplane Theorem Here we only show that f is nonnegative on D. Suppose there were a point u 2 D for which We can express f(u) as Thus, (this term is negative) so for some small, with, we have t>0 t apple /9/29 Lecture 2: Convex Sets 35

36 Proof of Separating Hyperplane Theorem i.e., the point d + t(u d) is closer to c than d is. Since D is convex and contains d and u, we have d + t(u d) 2 D. But this is impossible, since d is assumed to be the point in D that is closest to C. 2016/9/29 Lecture 2: Convex Sets 36

37 Supporting Hyperplane Theorem The geometric interpretation is that the hyperplane is tangent to C at x 0, and the halfspace {x a T x apple a T x 0 } contains C. {x a T x = a T x 0 } 2016/9/29 Lecture 2: Convex Sets 37

38 Dual Cones and Generalized Inequalities Dual cone of a cone K: As the name suggests, K is a cone, and is always convex, even when the original cone K is not. Some examples are: Dual cones of proper cones are proper, hence define generalized inequalities: 2016/9/29 Lecture 2: Convex Sets 38

39 Dual Cones and Generalized Inequalities 2016/9/29 Lecture 2: Convex Sets 39

40 Dual Cones and Generalized Inequalities Now suppose that the convex cone K is proper, so it induces a generalized inequality K. Then its dual cone K is also proper, and therefore induces a generalized inequality. We refer to the generalized inequality K as the dual of the generalized inequality. K Some important properties relating a generalized inequality and its dual are: Since K = K, the dual generalized inequality associated with is K, so these properties hold if the generalized inequality and its dual are swapped. K 2016/9/29 Lecture 2: Convex Sets 40

41 Minimum and Minimal Elements via Dual Inequalities 2016/9/29 Lecture 2: Convex Sets 41

42 Minimum and Minimal Elements via Dual Inequalities 2016/9/29 Lecture 2: Convex Sets 42

43 Minimum and Minimal Elements via Dual Inequalities Fig The production set P, for a product that requires labor and fuel to produce, is shown shaded. The two dark curves show the efficient production frontier. The points x 1,x 2,x 3 are efficient. The points and are not. x 4 x /9/29 Lecture 2: Convex Sets 43