COM Optimization for Communications Summary: Convex Sets and Convex Functions
|
|
- Ashlynn Carroll
- 6 years ago
- Views:
Transcription
1 1 Convex Sets Affine Sets COM Optimization for Communications Summary: Convex Sets and Convex Functions A set C R n is said to be affine if A point x 1, x 2 C = θx 1 + (1 θ)x 2 C, θ R (1) y = k θ i x i, (2) where θ 1 + θ θ k = 1, is an affine combination of the points x 1,..., x k. An affine set can always be expressed as where x o C, and V is a subspace. C = V + x o (3) The affine hull of a set C (not necessarily affine) is affc = {θ 1 x θ k x k x 1,..., x k C, θ i R, i = 1,..., k, θ θ k = 1} (4) The affine hull is the smallest affine set that contains C. Convex Sets A set C R n is said to be convex if x 1, x 2 C = θx 1 + (1 θ)x 2 C, θ [0, 1] (5) A point k y = θ i x i, (6) where θ 1,..., θ k 0, θ 1 + θ θ k = 1, is a convex combination of the points x 1,..., x k. The convex hull of a set C (not necessarily convex) is convc = {θ 1 x θ k x k x 1,..., x k C, θ i 0, i = 1,..., k, θ θ k = 1} (7) The convex hull is the smallest convex set that contains C. Convex Cones A set C R n is said to be a convex cone if x 1, x 2 C = θ 1 x 1 + θ 2 x 2 C, θ 1, θ 2 0 (8) 1
2 A point k y = θ i x i, (9) where θ 1,..., θ k 0, is a conic combination of the points x 1,..., x k. The conic hull of a set C (not necessarily convex) is conicc = {θ 1 x θ k x k x 1,..., x k C, θ i 0, i = 1,..., k} (10) Some Examples of Convex Sets Hyperplane: {x a T x = b}. Halfspace: {x a T x b}. Norm ball associated with norm. : B(x c, r) = {x x x c r} (11) where x c is the center and r is the radius. When. is the 2-norm it is known as the Euclidean norm. Ellipsoid: where P 0. E = {x (x x c ) T P 1 (x x c ) 1} (12) Norm cone associated with. : K = {(x, t) x t} (13) When. is the 2-norm K is called the 2nd-order cone or the ice cream cone. A norm cone is not only convex but also a convex cone. Polyhedron: P = {x Ax b, Cx = d} A bounded polyhedron is called a polytope. = {x a T j x b j, j = 1,..., m, c T j x = d j, j = 1,..., p} (14) Simplex: Given a set of vectors v 0,... v k that are affine independent, a simplex is A simplex is a polyhedron. C = conv{v 0,..., v k } = {θ 0 v θ k v k θ 0, 1 T θ = 0} (15) PSD cone: S n + = {X S n X 0} is a convex cone. (recall that S n is the set of all real n n symmetric matrices.) The empty set is convex. A singleton {x o } is convex. 2
3 Convexity Preserving Operations Intersection: S 1, S 2 convex = S 1 S 2 convex (16) S α convex for every α A = S α convex (17) Affine mapping: If S R n is convex and f : R n R m is affine, then the image of S under f α A f(s) = {f(x) x S} (18) is convex. Similarly, if C R m is convex and f : R n R m is affine, then the inverse image of C under f f 1 (C) = {x f(x) C} (19) is convex. Image under perspective function: The perspective function P : R n+1 R n, with domain domp = R n R ++ is given by P (z, t) = z/t (20) If C domp, then P (C) is convex. Proper Cones and Generalized Inequalities A cone K R n is proper if K is convex K is closed K is solid; i.e., intk K is pointed; i.e., x K, x K = x = 0 Generalized inequality associated with a proper cone K: Properties of generalized inequalities x K y, u K v = x + u K y + v x K y, y K z = x K z x K y, α 0 = αx K αy x K y, y K x = y = x Some examples: K = R n +. Then, x K y x i y i for all i. K = S n +. Then, X K Y means that Y X is PSD. x K y y x K (21) x K y y x intk (22) Minimum and minimal elements: A point x S is the minimum element of S if y S = x K y (23) provided that such an x exists. The minimum element, if it exists, is unique. A point x S is a minimal element of S if y S, y K x = y = x (24) 3
4 Separating Hyperplane Theorem Suppose that C, D R n are convex and that C D =. Then there exist a 0 and b such that a T x b = x C (25) a T x b = x D (26) The hyperplane {x a T x = b} is called a separating hyperplane for C and D. Supporting Hyperplanes Suppose that x o bdc (a boundary point of C). If there exist a 0 such that a T x a T x o for all x C, then {x a T x = b} is called a supporting hyperplane to C at point x o. For any nonempty convex set C, and for any x o bdc, there exists a supporting hyperplane to C at x o. Dual Cones The dual cone of a cone K is A cone K is called self-dual if K = K. If K is proper then K is also proper. K = {y y T x 0 x K} (27) Some examples: R n + and S n + are self-dual. The dual cone of a norm cone K = {(x, t) x t} is where. is the dual norm of.. K = {(x, t) x t} (28) 4
5 2 Convex Functions Definition A function f : R n R is convex if domf is convex and for all x, y domf, 0 θ 1, f(θx + (1 θ)y) θf(x) + (1 θ)f(y) (29) A function f : R n R is strictly convex if domf is convex and for all x, y domf, x y, 0 < θ < 1, A function f : R n R is concave if f is convex. Fundamental Properties f(θx + (1 θ)y) < θf(x) + (1 θ)f(y) (30) f is convex if and only if it is convex when restricted to any line that intersects its domain; i.e., for all x domf and ν, g(t) = f(x + tν) (31) is convex over {t x + tν domf}. First order condition: Suppose that f is differentiable. A function f with a convex domain domf is convex if and only if f(y) f(x) + f(x) T (y x) (32) for all x, y domf. Second order condition: Suppose that f is twice differentiable. A function f with a convex domain domf is convex if and only if its Hessian 2 f(x) 0 (33) for all x domf. A function f with a convex domain domf is stricly convex if for all x domf (the converse is not true). Sublevel sets: The sublevel set of f is 2 f(x) 0 (34) C α = {x domf f(x) α} (35) If f is convex, then C α is convex for every α (the converse is not true). Epigraph: The epigraph of f is f is convex if and only if epif is convex. epif = {(x, t) x domf, f(x) t} (36) 5
6 Examples Examples on R: e ax is convex on R log x is concave on R ++ x log x is convex on R ++ log x e t2 /2 dt is concave on R Examples on R n : A linear function a T x + b is convex and concave. A quadratic function x T P x + 2q T x + r is convex if and only if P 0, and is strictly convex if P 0. Every norm x is convex. max{x 1,..., x n } is convex. The geometric mean ( n x i) 1 n is concave on R n ++. Examples on R n m tr(ax) is linear on R n m, and hence is convex and concave. The negative logarithmetic determinant function log det X is convex on S n ++. tr(x 1 ) is convex on S n ++. Jensen Inequality For a convex f, holds for any x, y domf and 0 θ 1. f(θx + (1 θ)y) θf(x) + (1 θ)f(y) (37) Extension: For a convex f, for any random variable z. f(ez) Ef(z) (38) Jensen inequality can be used to derive certain inequalities; e.g., the arithmetic-geometric mean inequality: a + b ab, a, b 0 (39) 2 and ( n x i ) 1 n 1 n n x i, x i 0, i = 1,..., n (40) 6
7 Convexity Preserving Operations Nonnegative weighted sums: f 1,..., f m convex, w 1,..., w m 0 = f(x, y) convex in x for each y A, w(y) 0 for each y A = Example: f(x) = x i log x i is convex on R n ++. Composition with an affine mapping: is convex if f is convex. Pointwise maximum and supremum: Examples: m w i f i convex (41) A w(y)f(x, y)dy convex (42) g(x) = f(ax + b) (43) f 1, f 2 convex = g(x) = max{f 1 (x), f 2 (x)} convex (44) f(x, y) convex in x for each y A = g(x) = sup f(x, y) convex (45) y A A piecewise linear function f(x) = max,...,l a T i x + b i is convex. f(x) = sup y C x y is convex for any set C. The largest eigenvalue of X is convex on S n. The 2-norm of X f(x) = λ max (X) = sup y T Xy = sup tr(xyy T ) (46) y 2 =1 y 2 =1 f(x) = X 2 = sup y 2=1 Xy 2 (47) is convex on R n m. Composition: Let f(x) = h(g(x)), where h : R R, and g : R n R. Let h(x) = { h(x), x domh, otherwise (48) Then, f is convex if h is convex and nondecreasing, and g is convex. f is convex if h is convex and nonincreasing, and g is concave. Minimization: f(x, y) convex in (x, y), C convex nonempty = g(x) = inf f(x, y) convex (49) y C provided that g(x) > for some x. Examples: 7
8 dist(x, S) = inf y S x y is convex for convex S. The Schur complement [ ] A B B T 0 C C 0, A BC B T 0 (50) may be proven by the convex minimization property. Perspective: The perspective of a function f is a function g : R n+1 R g(x, t) = tf(x/t), domg = {(x, t) x/t domf, t > 0} (51) If f is convex then g is convex. Conjugate Function f (y) = f is convex irrespective of the convexity of f. sup (y T x f(x)) (52) x domf domf consists of y for which the supremum is finite; i.e., domf = {y f (y) < }. Examples: The conjugate function of f(x) = 1 2 xt Qx, Q 0, is f (y) = 1 2 yt Q 1 y. The conjugate function of f(x) = log det X, domf = S n ++ is f (Y ) = log det( Y 1 ) n, domf = S n ++. Quasiconvex Functions Definition: A function f : R n R is quasiconvex (or unimodal) if domf is convex and the sublevel set is convex for every α. A function f is quasiconcave if f is quasiconvex. A function f is quasilinear if f is quasiconvex and quasiconcave. Examples: log x is quasilinear on R ++. A linear fractional function is quasilinear. S α = {x domf f(x) α} (53) f(x) = at x + b c T x + d, domf = {x ct x + d > 0} (54) rankx is quasiconcave on S n + (proven using the modified Jensen inequality). Modified Jensen inequality: f is quasiconvex if and only if for any x, y domf, and 0 θ 1, f(θx + (1 θ)y) max{f(x), f(y)} (55) 8
9 First-order condition: Suppose that f is differentiable. f is quasiconvex if and only if domf is convex and for all x, y domf f(y) f(x) = f(x) T (y x) 0 (56) Second-order condition: Suppose f is differentiable. If f is quasiconvex then for all x domf, y R n, Convexity with respect to Generalized Inequality y T f(x) = 0 = y T 2 f(x)y 0 (57) Let K be a proper cone. A function f : R n R m is K-convex if for all x, y domf and 0 θ 1, f(θx + (1 θ)y) K θf(x) + (1 θ)f(y) (58) For K = R n +, a K-convex function is a function for which each component function f i is convex. Consider K = S n +. f(x) = X T X is K-convex on R n m. f(x) = X 1 is K-convex on S n ++. 9
Shiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 2. Convex Optimization
Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 2 Convex Optimization Shiqian Ma, MAT-258A: Numerical Optimization 2 2.1. Convex Optimization General optimization problem: min f 0 (x) s.t., f i
More informationConvex sets and convex functions
Convex sets and convex functions Convex optimization problems Convex sets and their examples Separating and supporting hyperplanes Projections on convex sets Convex functions, conjugate functions ECE 602,
More informationConvex sets and convex functions
Convex sets and convex functions Convex optimization problems Convex sets and their examples Separating and supporting hyperplanes Projections on convex sets Convex functions, conjugate functions ECE 602,
More information2. Convex sets. affine and convex sets. some important examples. operations that preserve convexity. generalized inequalities
2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationLecture 4: Convexity
10-725: Convex Optimization Fall 2013 Lecture 4: Convexity Lecturer: Barnabás Póczos Scribes: Jessica Chemali, David Fouhey, Yuxiong Wang Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationTutorial on Convex Optimization for Engineers
Tutorial on Convex Optimization for Engineers M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D-98684 Ilmenau, Germany jens.steinwandt@tu-ilmenau.de
More informationCS675: Convex and Combinatorial Optimization Fall 2014 Convex Functions. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Fall 2014 Convex Functions Instructor: Shaddin Dughmi Outline 1 Convex Functions 2 Examples of Convex and Concave Functions 3 Convexity-Preserving Operations
More informationCS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets Instructor: Shaddin Dughmi Outline 1 Convex sets, Affine sets, and Cones 2 Examples of Convex Sets 3 Convexity-Preserving Operations
More informationAffine function. suppose f : R n R m is affine (f(x) =Ax + b with A R m n, b R m ) the image of a convex set under f is convex
Affine function suppose f : R n R m is affine (f(x) =Ax + b with A R m n, b R m ) the image of a convex set under f is convex S R n convex = f(s) ={f(x) x S} convex the inverse image f 1 (C) of a convex
More informationConvex Optimization. 2. Convex Sets. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University. SJTU Ying Cui 1 / 33
Convex Optimization 2. Convex Sets Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2018 SJTU Ying Cui 1 / 33 Outline Affine and convex sets Some important examples Operations
More informationLecture 2: August 29, 2018
10-725/36-725: Convex Optimization Fall 2018 Lecturer: Ryan Tibshirani Lecture 2: August 29, 2018 Scribes: Adam Harley Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have
More informationConvexity Theory and Gradient Methods
Convexity Theory and Gradient Methods Angelia Nedić angelia@illinois.edu ISE Department and Coordinated Science Laboratory University of Illinois at Urbana-Champaign Outline Convex Functions Optimality
More informationConvex Sets. CSCI5254: Convex Optimization & Its Applications. subspaces, affine sets, and convex sets. operations that preserve convexity
CSCI5254: Convex Optimization & Its Applications Convex Sets subspaces, affine sets, and convex sets operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationCS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets. Instructor: Shaddin Dughmi
CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets Instructor: Shaddin Dughmi Announcements New room: KAP 158 Today: Convex Sets Mostly from Boyd and Vandenberghe. Read all of
More informationConvexity I: Sets and Functions
Convexity I: Sets and Functions Lecturer: Aarti Singh Co-instructor: Pradeep Ravikumar Convex Optimization 10-725/36-725 See supplements for reviews of basic real analysis basic multivariate calculus basic
More information2. Convex sets. x 1. x 2. affine set: contains the line through any two distinct points in the set
2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationConvex Optimization. Convex Sets. ENSAE: Optimisation 1/24
Convex Optimization Convex Sets ENSAE: Optimisation 1/24 Today affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes
More informationLecture 2: August 31
10-725/36-725: Convex Optimization Fall 2016 Lecture 2: August 31 Lecturer: Lecturer: Ryan Tibshirani Scribes: Scribes: Lidan Mu, Simon Du, Binxuan Huang 2.1 Review A convex optimization problem is of
More informationLecture 2: August 29, 2018
10-725/36-725: Convex Optimization Fall 2018 Lecturer: Ryan Tibshirani Lecture 2: August 29, 2018 Scribes: Yingjing Lu, Adam Harley, Ruosong Wang Note: LaTeX template courtesy of UC Berkeley EECS dept.
More information2. Convex sets. affine and convex sets. some important examples. operations that preserve convexity. generalized inequalities
2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationConvex Sets (cont.) Convex Functions
Convex Sets (cont.) Convex Functions Optimization - 10725 Carlos Guestrin Carnegie Mellon University February 27 th, 2008 1 Definitions of convex sets Convex v. Non-convex sets Line segment definition:
More informationConvex Optimization - Chapter 1-2. Xiangru Lian August 28, 2015
Convex Optimization - Chapter 1-2 Xiangru Lian August 28, 2015 1 Mathematical optimization minimize f 0 (x) s.t. f j (x) 0, j=1,,m, (1) x S x. (x 1,,x n ). optimization variable. f 0. R n R. objective
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 6
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 6 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 19, 2012 Andre Tkacenko
More informationIntroduction to Modern Control Systems
Introduction to Modern Control Systems Convex Optimization, Duality and Linear Matrix Inequalities Kostas Margellos University of Oxford AIMS CDT 2016-17 Introduction to Modern Control Systems November
More informationLecture 5: Properties of convex sets
Lecture 5: Properties of convex sets Rajat Mittal IIT Kanpur This week we will see properties of convex sets. These properties make convex sets special and are the reason why convex optimization problems
More informationSimplex Algorithm in 1 Slide
Administrivia 1 Canonical form: Simplex Algorithm in 1 Slide If we do pivot in A r,s >0, where c s
More information60 2 Convex sets. {x a T x b} {x ã T x b}
60 2 Convex sets Exercises Definition of convexity 21 Let C R n be a convex set, with x 1,, x k C, and let θ 1,, θ k R satisfy θ i 0, θ 1 + + θ k = 1 Show that θ 1x 1 + + θ k x k C (The definition of convexity
More informationLecture: Convex Sets
/24 Lecture: Convex Sets http://bicmr.pku.edu.cn/~wenzw/opt-27-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/24 affine and convex sets some
More informationIntroduction to Convex Optimization. Prof. Daniel P. Palomar
Introduction to Convex Optimization Prof. Daniel P. Palomar The Hong Kong University of Science and Technology (HKUST) MAFS6010R- Portfolio Optimization with R MSc in Financial Mathematics Fall 2018-19,
More informationMathematical Programming and Research Methods (Part II)
Mathematical Programming and Research Methods (Part II) 4. Convexity and Optimization Massimiliano Pontil (based on previous lecture by Andreas Argyriou) 1 Today s Plan Convex sets and functions Types
More informationLecture 2 Convex Sets
Optimization Theory and Applications Lecture 2 Convex Sets Prof. Chun-Hung Liu Dept. of Electrical and Computer Engineering National Chiao Tung University Fall 2016 2016/9/29 Lecture 2: Convex Sets 1 Outline
More informationLecture 2 - Introduction to Polytopes
Lecture 2 - Introduction to Polytopes Optimization and Approximation - ENS M1 Nicolas Bousquet 1 Reminder of Linear Algebra definitions Let x 1,..., x m be points in R n and λ 1,..., λ m be real numbers.
More informationChapter 4 Convex Optimization Problems
Chapter 4 Convex Optimization Problems Shupeng Gui Computer Science, UR October 16, 2015 hupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 1 / 58 Outline 1 Optimization problems
More informationNumerical Optimization
Convex Sets Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Let x 1, x 2 R n, x 1 x 2. Line and line segment Line passing through x 1 and x 2 : {y
More informationConvex Sets. Pontus Giselsson
Convex Sets Pontus Giselsson 1 Today s lecture convex sets convex, affine, conical hulls closure, interior, relative interior, boundary, relative boundary separating and supporting hyperplane theorems
More informationConvex Optimization M2
Convex Optimization M2 Lecture 1 A. d Aspremont. Convex Optimization M2. 1/49 Today Convex optimization: introduction Course organization and other gory details... Convex sets, basic definitions. A. d
More informationCombinatorial Geometry & Topology arising in Game Theory and Optimization
Combinatorial Geometry & Topology arising in Game Theory and Optimization Jesús A. De Loera University of California, Davis LAST EPISODE... We discuss the content of the course... Convex Sets A set is
More informationConvex Optimization Lecture 2
Convex Optimization Lecture 2 Today: Convex Analysis Center-of-mass Algorithm 1 Convex Analysis Convex Sets Definition: A set C R n is convex if for all x, y C and all 0 λ 1, λx + (1 λ)y C Operations that
More informationConvexity: an introduction
Convexity: an introduction Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 74 1. Introduction 1. Introduction what is convexity where does it arise main concepts and
More informationLecture 2. Topology of Sets in R n. August 27, 2008
Lecture 2 Topology of Sets in R n August 27, 2008 Outline Vectors, Matrices, Norms, Convergence Open and Closed Sets Special Sets: Subspace, Affine Set, Cone, Convex Set Special Convex Sets: Hyperplane,
More informationIE 521 Convex Optimization
Lecture 4: 5th February 2019 Outline 1 / 23 Which function is different from others? Figure: Functions 2 / 23 Definition of Convex Function Definition. A function f (x) : R n R is convex if (i) dom(f )
More informationAspects of Convex, Nonconvex, and Geometric Optimization (Lecture 1) Suvrit Sra Massachusetts Institute of Technology
Aspects of Convex, Nonconvex, and Geometric Optimization (Lecture 1) Suvrit Sra Massachusetts Institute of Technology Hausdorff Institute for Mathematics (HIM) Trimester: Mathematics of Signal Processing
More informationConvex Geometry arising in Optimization
Convex Geometry arising in Optimization Jesús A. De Loera University of California, Davis Berlin Mathematical School Summer 2015 WHAT IS THIS COURSE ABOUT? Combinatorial Convexity and Optimization PLAN
More informationIntroduction to Convex Optimization
Introduction to Convex Optimization Lieven Vandenberghe Electrical Engineering Department, UCLA IPAM Graduate Summer School: Computer Vision August 5, 2013 Convex optimization problem minimize f 0 (x)
More informationCMU-Q Lecture 9: Optimization II: Constrained,Unconstrained Optimization Convex optimization. Teacher: Gianni A. Di Caro
CMU-Q 15-381 Lecture 9: Optimization II: Constrained,Unconstrained Optimization Convex optimization Teacher: Gianni A. Di Caro GLOBAL FUNCTION OPTIMIZATION Find the global maximum of the function f x (and
More informationConic Duality. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 1 Conic Duality Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/
More informationFACES OF CONVEX SETS
FACES OF CONVEX SETS VERA ROSHCHINA Abstract. We remind the basic definitions of faces of convex sets and their basic properties. For more details see the classic references [1, 2] and [4] for polytopes.
More informationLECTURE 7 LECTURE OUTLINE. Review of hyperplane separation Nonvertical hyperplanes Convex conjugate functions Conjugacy theorem Examples
LECTURE 7 LECTURE OUTLINE Review of hyperplane separation Nonvertical hyperplanes Convex conjugate functions Conjugacy theorem Examples Reading: Section 1.5, 1.6 All figures are courtesy of Athena Scientific,
More informationThis lecture: Convex optimization Convex sets Convex functions Convex optimization problems Why convex optimization? Why so early in the course?
Lec4 Page 1 Lec4p1, ORF363/COS323 This lecture: Convex optimization Convex sets Convex functions Convex optimization problems Why convex optimization? Why so early in the course? Instructor: Amir Ali Ahmadi
More informationConvexity and Optimization
Convexity and Optimization Richard Lusby DTU Management Engineering Class Exercises From Last Time 2 DTU Management Engineering 42111: Static and Dynamic Optimization (3) 18/09/2017 Today s Material Extrema
More informationDM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini
DM545 Linear and Integer Programming Lecture 2 The Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. 2. 3. 4. Standard Form Basic Feasible Solutions
More informationConvexity and Optimization
Convexity and Optimization Richard Lusby Department of Management Engineering Technical University of Denmark Today s Material Extrema Convex Function Convex Sets Other Convexity Concepts Unconstrained
More informationLecture 3: Convex sets
Lecture 3: Convex sets Rajat Mittal IIT Kanpur We denote the set of real numbers as R. Most of the time we will be working with space R n and its elements will be called vectors. Remember that a subspace
More informationEC 521 MATHEMATICAL METHODS FOR ECONOMICS. Lecture 2: Convex Sets
EC 51 MATHEMATICAL METHODS FOR ECONOMICS Lecture : Convex Sets Murat YILMAZ Boğaziçi University In this section, we focus on convex sets, separating hyperplane theorems and Farkas Lemma. And as an application
More informationLECTURE 10 LECTURE OUTLINE
We now introduce a new concept with important theoretical and algorithmic implications: polyhedral convexity, extreme points, and related issues. LECTURE 1 LECTURE OUTLINE Polar cones and polar cone theorem
More informationLocally convex topological vector spaces
Chapter 4 Locally convex topological vector spaces 4.1 Definition by neighbourhoods Let us start this section by briefly recalling some basic properties of convex subsets of a vector space over K (where
More informationChapter 4 Concepts from Geometry
Chapter 4 Concepts from Geometry An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Line Segments The line segment between two points and in R n is the set of points on the straight line joining
More informationMTAEA Convexity and Quasiconvexity
School of Economics, Australian National University February 19, 2010 Convex Combinations and Convex Sets. Definition. Given any finite collection of points x 1,..., x m R n, a point z R n is said to be
More informationLecture 1: Introduction
CSE 599: Interplay between Convex Optimization and Geometry Winter 218 Lecturer: Yin Tat Lee Lecture 1: Introduction Disclaimer: Please tell me any mistake you noticed. 1.1 Course Information Objective:
More informationof Convex Analysis Fundamentals Jean-Baptiste Hiriart-Urruty Claude Lemarechal Springer With 66 Figures
2008 AGI-Information Management Consultants May be used for personal purporses only or by libraries associated to dandelon.com network. Jean-Baptiste Hiriart-Urruty Claude Lemarechal Fundamentals of Convex
More informationLinear programming and duality theory
Linear programming and duality theory Complements of Operations Research Giovanni Righini Linear Programming (LP) A linear program is defined by linear constraints, a linear objective function. Its variables
More informationConvex Optimization. Erick Delage, and Ashutosh Saxena. October 20, (a) (b) (c)
Convex Optimization (for CS229) Erick Delage, and Ashutosh Saxena October 20, 2006 1 Convex Sets Definition: A set G R n is convex if every pair of point (x, y) G, the segment beteen x and y is in A. More
More informationMath 5593 Linear Programming Lecture Notes
Math 5593 Linear Programming Lecture Notes Unit II: Theory & Foundations (Convex Analysis) University of Colorado Denver, Fall 2013 Topics 1 Convex Sets 1 1.1 Basic Properties (Luenberger-Ye Appendix B.1).........................
More informationRevisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization. Author: Martin Jaggi Presenter: Zhongxing Peng
Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization Author: Martin Jaggi Presenter: Zhongxing Peng Outline 1. Theoretical Results 2. Applications Outline 1. Theoretical Results 2. Applications
More informationCharacterizing Improving Directions Unconstrained Optimization
Final Review IE417 In the Beginning... In the beginning, Weierstrass's theorem said that a continuous function achieves a minimum on a compact set. Using this, we showed that for a convex set S and y not
More informationCS 435, 2018 Lecture 2, Date: 1 March 2018 Instructor: Nisheeth Vishnoi. Convex Programming and Efficiency
CS 435, 2018 Lecture 2, Date: 1 March 2018 Instructor: Nisheeth Vishnoi Convex Programming and Efficiency In this lecture, we formalize convex programming problem, discuss what it means to solve it efficiently
More informationConvexity. 1 X i is convex. = b is a hyperplane in R n, and is denoted H(p, b) i.e.,
Convexity We ll assume throughout, without always saying so, that we re in the finite-dimensional Euclidean vector space R n, although sometimes, for statements that hold in any vector space, we ll say
More information13. Cones and semidefinite constraints
CS/ECE/ISyE 524 Introduction to Optimization Spring 2017 18 13. Cones and semidefinite constraints ˆ Geometry of cones ˆ Second order cone programs ˆ Example: robust linear program ˆ Semidefinite constraints
More informationHowever, this is not always true! For example, this fails if both A and B are closed and unbounded (find an example).
98 CHAPTER 3. PROPERTIES OF CONVEX SETS: A GLIMPSE 3.2 Separation Theorems It seems intuitively rather obvious that if A and B are two nonempty disjoint convex sets in A 2, then there is a line, H, separating
More informationGeneralized Nash Equilibrium Problem: existence, uniqueness and
Generalized Nash Equilibrium Problem: existence, uniqueness and reformulations Univ. de Perpignan, France CIMPA-UNESCO school, Delhi November 25 - December 6, 2013 Outline of the 7 lectures Generalized
More informationM3P1/M4P1 (2005) Dr M Ruzhansky Metric and Topological Spaces Summary of the course: definitions, examples, statements.
M3P1/M4P1 (2005) Dr M Ruzhansky Metric and Topological Spaces Summary of the course: definitions, examples, statements. Chapter 1: Metric spaces and convergence. (1.1) Recall the standard distance function
More informationLecture 19: Convex Non-Smooth Optimization. April 2, 2007
: Convex Non-Smooth Optimization April 2, 2007 Outline Lecture 19 Convex non-smooth problems Examples Subgradients and subdifferentials Subgradient properties Operations with subgradients and subdifferentials
More informationWeek 5. Convex Optimization
Week 5. Convex Optimization Lecturer: Prof. Santosh Vempala Scribe: Xin Wang, Zihao Li Feb. 9 and, 206 Week 5. Convex Optimization. The convex optimization formulation A general optimization problem is
More informationAdvanced Operations Research Techniques IE316. Quiz 2 Review. Dr. Ted Ralphs
Advanced Operations Research Techniques IE316 Quiz 2 Review Dr. Ted Ralphs IE316 Quiz 2 Review 1 Reading for The Quiz Material covered in detail in lecture Bertsimas 4.1-4.5, 4.8, 5.1-5.5, 6.1-6.3 Material
More informationLecture 5: Duality Theory
Lecture 5: Duality Theory Rajat Mittal IIT Kanpur The objective of this lecture note will be to learn duality theory of linear programming. We are planning to answer following questions. What are hyperplane
More informationApplied Integer Programming
Applied Integer Programming D.S. Chen; R.G. Batson; Y. Dang Fahimeh 8.2 8.7 April 21, 2015 Context 8.2. Convex sets 8.3. Describing a bounded polyhedron 8.4. Describing unbounded polyhedron 8.5. Faces,
More informationAdvanced Operations Research Techniques IE316. Quiz 1 Review. Dr. Ted Ralphs
Advanced Operations Research Techniques IE316 Quiz 1 Review Dr. Ted Ralphs IE316 Quiz 1 Review 1 Reading for The Quiz Material covered in detail in lecture. 1.1, 1.4, 2.1-2.6, 3.1-3.3, 3.5 Background material
More informationConvex Optimization Euclidean Distance Geometry 2ε
Convex Optimization Euclidean Distance Geometry 2ε In my career, I found that the best people are the ones that really understand the content, and they re a pain in the butt to manage. But you put up with
More informationLecture 2 Optimization with equality constraints
Lecture 2 Optimization with equality constraints Constrained optimization The idea of constrained optimisation is that the choice of one variable often affects the amount of another variable that can be
More information3. The Simplex algorithmn The Simplex algorithmn 3.1 Forms of linear programs
11 3.1 Forms of linear programs... 12 3.2 Basic feasible solutions... 13 3.3 The geometry of linear programs... 14 3.4 Local search among basic feasible solutions... 15 3.5 Organization in tableaus...
More informationAMS : Combinatorial Optimization Homework Problems - Week V
AMS 553.766: Combinatorial Optimization Homework Problems - Week V For the following problems, A R m n will be m n matrices, and b R m. An affine subspace is the set of solutions to a a system of linear
More informationSome Advanced Topics in Linear Programming
Some Advanced Topics in Linear Programming Matthew J. Saltzman July 2, 995 Connections with Algebra and Geometry In this section, we will explore how some of the ideas in linear programming, duality theory,
More informationIntroduction to optimization
Introduction to optimization G. Ferrari Trecate Dipartimento di Ingegneria Industriale e dell Informazione Università degli Studi di Pavia Industrial Automation Ferrari Trecate (DIS) Optimization Industrial
More informationAlternating Projections
Alternating Projections Stephen Boyd and Jon Dattorro EE392o, Stanford University Autumn, 2003 1 Alternating projection algorithm Alternating projections is a very simple algorithm for computing a point
More informationConvex Optimization Euclidean Distance Geometry 2ε
Convex Optimization Euclidean Distance Geometry 2ε 1 Overview 19 2 Convex geometry 31 2.1 Convex set.................................... 31 2.2 Vectorized-matrix inner product........................ 42
More informationLinear and Integer Programming :Algorithms in the Real World. Related Optimization Problems. How important is optimization?
Linear and Integer Programming 15-853:Algorithms in the Real World Linear and Integer Programming I Introduction Geometric Interpretation Simplex Method Linear or Integer programming maximize z = c T x
More informationPolytopes Course Notes
Polytopes Course Notes Carl W. Lee Department of Mathematics University of Kentucky Lexington, KY 40506 lee@ms.uky.edu Fall 2013 i Contents 1 Polytopes 1 1.1 Convex Combinations and V-Polytopes.....................
More informationELEG Compressive Sensing and Sparse Signal Representations
ELEG 867 - Compressive Sensing and Sparse Signal Representations Introduction to Matrix Completion and Robust PCA Gonzalo Garateguy Depart. of Electrical and Computer Engineering University of Delaware
More informationMath 414 Lecture 2 Everyone have a laptop?
Math 44 Lecture 2 Everyone have a laptop? THEOREM. Let v,...,v k be k vectors in an n-dimensional space and A = [v ;...; v k ] v,..., v k independent v,..., v k span the space v,..., v k a basis v,...,
More informationmaximize c, x subject to Ax b,
Lecture 8 Linear programming is about problems of the form maximize c, x subject to Ax b, where A R m n, x R n, c R n, and b R m, and the inequality sign means inequality in each row. The feasible set
More informationLecture 2 September 3
EE 381V: Large Scale Optimization Fall 2012 Lecture 2 September 3 Lecturer: Caramanis & Sanghavi Scribe: Hongbo Si, Qiaoyang Ye 2.1 Overview of the last Lecture The focus of the last lecture was to give
More informationOPERATIONS RESEARCH. Linear Programming Problem
OPERATIONS RESEARCH Chapter 1 Linear Programming Problem Prof. Bibhas C. Giri Department of Mathematics Jadavpur University Kolkata, India Email: bcgiri.jumath@gmail.com 1.0 Introduction Linear programming
More informationMATH 890 HOMEWORK 2 DAVID MEREDITH
MATH 890 HOMEWORK 2 DAVID MEREDITH (1) Suppose P and Q are polyhedra. Then P Q is a polyhedron. Moreover if P and Q are polytopes then P Q is a polytope. The facets of P Q are either F Q where F is a facet
More informationLecture 19 Subgradient Methods. November 5, 2008
Subgradient Methods November 5, 2008 Outline Lecture 19 Subgradients and Level Sets Subgradient Method Convergence and Convergence Rate Convex Optimization 1 Subgradients and Level Sets A vector s is a
More informationCOMP331/557. Chapter 2: The Geometry of Linear Programming. (Bertsimas & Tsitsiklis, Chapter 2)
COMP331/557 Chapter 2: The Geometry of Linear Programming (Bertsimas & Tsitsiklis, Chapter 2) 49 Polyhedra and Polytopes Definition 2.1. Let A 2 R m n and b 2 R m. a set {x 2 R n A x b} is called polyhedron
More informationConvex Optimization. Chapter 1 - chapter 2.2
Convex Optimization Chapter 1 - chapter 2.2 Introduction In optimization literatures, one will frequently encounter terms like linear programming, convex set convex cone, convex hull, semidefinite cone
More information11 Linear Programming
11 Linear Programming 11.1 Definition and Importance The final topic in this course is Linear Programming. We say that a problem is an instance of linear programming when it can be effectively expressed
More informationSection 5 Convex Optimisation 1. W. Dai (IC) EE4.66 Data Proc. Convex Optimisation page 5-1
Section 5 Convex Optimisation 1 W. Dai (IC) EE4.66 Data Proc. Convex Optimisation 1 2018 page 5-1 Convex Combination Denition 5.1 A convex combination is a linear combination of points where all coecients
More informationOptimization under uncertainty: modeling and solution methods
Optimization under uncertainty: modeling and solution methods Paolo Brandimarte Dipartimento di Scienze Matematiche Politecnico di Torino e-mail: paolo.brandimarte@polito.it URL: http://staff.polito.it/paolo.brandimarte
More informationMinima, Maxima, Saddle points
Minima, Maxima, Saddle points Levent Kandiller Industrial Engineering Department Çankaya University, Turkey Minima, Maxima, Saddle points p./9 Scalar Functions Let us remember the properties for maxima,
More information