Tutorial on Convex Optimization for Engineers
|
|
- Henry Evans
- 5 years ago
- Views:
Transcription
1 Tutorial on Convex Optimization for Engineers M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box D Ilmenau, Germany January 2012
2 Course mechanics strongly based on the advanced course Convex Optimization I by Prof. Stephen Boyd at Stanford University, CA info, slides, video lectures, textbook, software on web page: mandatory assignment (30 % of the final grade) Prerequisites working knowledge of linear algebra
3 Course objectives to provide you with overview of convex optimization working knowledge on convex optimization in particular, to provide you with skills about how to recognize convex problems model nonconvex problems as convex get a feel for easy and difficult problems efficiently solve the problems using optimization tools
4 Outline 1. Introduction 2. Convex sets 3. Convex functions 4. Convex optimization problems 5. Lagrangian duality theory 6. Disciplined convex programming and CVX
5 1. Introduction
6 Mathematical optimization (mathematical) optimization problem minimize f 0 (x) subject to f i (x) b i, i = 1,...,m x = (x 1,...,x n ): optimization variables f 0 : R n R: objective function f i : R n R, i = 1,...,m: constraint functions optimal solution x has smallest value of f 0 among all vectors that satisfy the constraints Introduction 1 2
7 Solving optimization problems general optimization problem very difficult to solve methods involve some compromise, e.g., very long computation time, or not always finding the solution exceptions: certain problem classes can be solved efficiently and reliably least-squares problems linear programming problems convex optimization problems Introduction 1 4
8 Least-squares minimize Ax b 2 2 solving least-squares problems analytical solution: x = (A T A) 1 A T b reliable and efficient algorithms and software computation time proportional to n 2 k (A R k n ); less if structured a mature technology using least-squares least-squares problems are easy to recognize a few standard techniques increase flexibility (e.g., including weights, adding regularization terms) Introduction 1 5
9 Linear programming solving linear programs no analytical formula for solution minimize c T x subject to a T i x b i, i = 1,...,m reliable and efficient algorithms and software computation time proportional to n 2 m if m n; less with structure a mature technology using linear programming not as easy to recognize as least-squares problems a few standard tricks used to convert problems into linear programs (e.g., problems involving l 1 - or l -norms, piecewise-linear functions) Introduction 1 6
10 Convex optimization problem minimize f 0 (x) subject to f i (x) b i, i = 1,...,m objective and constraint functions are convex: f i (αx+βy) αf i (x)+βf i (y) if α+β = 1, α 0, β 0 includes least-squares problems and linear programs as special cases Introduction 1 7
11 Why is convex optimization so essential? convex optimization formulation always achieves global minimum, no local traps certificate for infeasibility can be solved by polynomial time complexity algorithms (e.g., interior point methods) highly efficient software available (e.g., SeDuMi, CVX) the dividing line between easy and difficult problems (compare with solving linear equations) = whenever possible, always go for convex formulation Introduction
12 Brief history of convex optimization theory (convex analysis): ca algorithms 1947: simplex algorithm for linear programming (Dantzig) 1960s: early interior-point methods (Fiacco & McCormick, Dikin,... ) 1970s: ellipsoid method and other subgradient methods 1980s: polynomial-time interior-point methods for linear programming (Karmarkar 1984) late 1980s now: polynomial-time interior-point methods for nonlinear convex optimization (Nesterov & Nemirovski 1994) applications before 1990: mostly in operations research; few in engineering since 1990: many new applications in engineering (control, signal processing, communications, circuit design,... ); new problem classes (semidefinite and second-order cone programming, robust optimization) Introduction 1 15
13 2. Convex sets
14 Subspaces a space S R n is a subspace if x 1, x 2 S, θ 1, θ 2 R = θ 1x 1 + θ 2x 2 S geometrically: plane through x 1, x 2 S and the origin representation: range(a) = {Aw w R q } (A = [a 1,... a q]) = {w 1a 1 + w qa q w i R} = span {a 1, a 2,..., a q} null(a) = {x Bx = 0} (B = [b 1,... b p] T ) = { x b T 1 x = 0,..., b T p x = 0 } Convex sets
15 Affine set line through x 1, x 2 : all points x = θx 1 +(1 θ)x 2 (θ R) θ = 1.2 x 1 θ = 1 θ = 0.6 x 2 θ = 0 θ = 0.2 affine set: contains the line through any two distinct points in the set example: solution set of linear equations {x Ax = b} (conversely, every affine set can be expressed as solution set of system of linear equations) Convex sets 2 2
16 Convex set line segment between x 1 and x 2 : all points with 0 θ 1 x = θx 1 +(1 θ)x 2 convex set: contains line segment between any two points in the set x 1,x 2 C, 0 θ 1 = θx 1 +(1 θ)x 2 C examples (one convex, two nonconvex sets) Convex sets 2 3
17 Convex combination and convex hull convex combination of x 1,..., x k : any point x of the form with θ 1 + +θ k = 1, θ i 0 x = θ 1 x 1 +θ 2 x 2 + +θ k x k convex hull convs: set of all convex combinations of points in S Convex sets 2 4
18 Convex cone conic (nonnegative) combination of x 1 and x 2 : any point of the form with θ 1 0, θ 2 0 x = θ 1 x 1 +θ 2 x 2 x 1 0 x 2 convex cone: set that contains all conic combinations of points in the set Convex sets 2 5
19 Hyperplanes and halfspaces hyperplane: set of the form {x a T x = b} (a 0) a x 0 x a T x = b halfspace: set of the form {x a T x b} (a 0) a x 0 a T x b a T x b a is the normal vector hyperplanes are affine and convex; halfspaces are convex Convex sets 2 6
20 Euclidean balls and ellipsoids (Euclidean) ball with center x c and radius r: B(x c,r) = {x x x c 2 r} = {x c +ru u 2 1} ellipsoid: set of the form {x (x x c ) T P 1 (x x c ) 1} with P S n ++ (i.e., P symmetric positive definite) x c other representation: {x c +Au u 2 1} with A square and nonsingular Convex sets 2 7
21 Norm balls and norm cones norm: a function that satisfies x 0; x = 0 if and only if x = 0 tx = t x for t R x+y x + y notation: is general (unspecified) norm; symb is particular norm norm ball with center x c and radius r: {x x x c r} 1 norm cone: {(x,t) x t} Euclidean norm cone is called secondorder cone norm balls and cones are convex t x x Convex sets 2 8
22 Polyhedra solution set of finitely many linear inequalities and equalities Ax b, Cx = d (A R m n, C R p n, is componentwise inequality) a 1 a2 a 5 P a 3 a 4 polyhedron is intersection of finite number of halfspaces and hyperplanes Convex sets 2 9
23 Positive semidefinite cone notation: S n is set of symmetric n n matrices S n + = {X S n X 0}: positive semidefinite n n matrices X S n + z T Xz 0 for all z S n + is a convex cone S n ++ = {X S n X 0}: positive definite n n matrices 1 example: [ x y y z ] S 2 + z y 1 0 x Convex sets 2 10
24 Operations that preserve convexity practical methods for establishing convexity of a set C 1. apply definition x 1,x 2 C, 0 θ 1 = θx 1 +(1 θ)x 2 C 2. show that C is obtained from simple convex sets (hyperplanes, halfspaces, norm balls,... ) by operations that preserve convexity intersection affine functions perspective function linear-fractional functions Convex sets 2 11
25 Examples of convex sets linear subspace: S = {x Ax = 0} is a convex cone affine subspace: S = {x Ax = b} is a convex set polyhedral set: S = {x Ax b} is a convex set PSD matrix cone: S = {A A is symmetric, A 0} is convex second order cone: S = {(t, x) t x } is convex intersection intersection of linear subspaces affine subspaces convex cones convex sets is also a linear subspace affine subspace convex cone convex set example: a polyhedron is intersection of a finite number of halfspaces Convex sets
26 3. Convex functions
27 Definition f : R n R is convex if domf is a convex set and f(θx+(1 θ)y) θf(x)+(1 θ)f(y) for all x,y domf, 0 θ 1 (x,f(x)) (y,f(y)) f is concave if f is convex f is strictly convex if domf is convex and f(θx+(1 θ)y) < θf(x)+(1 θ)f(y) for x,y domf, x y, 0 < θ < 1 Convex functions 3 2
28 Examples on R convex: affine: ax+b on R, for any a,b R exponential: e ax, for any a R powers: x α on R ++, for α 1 or α 0 powers of absolute value: x p on R, for p 1 negative entropy: xlogx on R ++ concave: affine: ax+b on R, for any a,b R powers: x α on R ++, for 0 α 1 logarithm: logx on R ++ Convex functions 3 3
29 Examples on R n and R m n affine functions are convex and concave; all norms are convex examples on R n affine function f(x) = a T x+b norms: x p = ( n i=1 x i p ) 1/p for p 1; x = max k x k examples on R m n (m n matrices) affine function f(x) = tr(a T X)+b = m i=1 n A ij X ij +b j=1 spectral (maximum singular value) norm f(x) = X 2 = σ max (X) = (λ max (X T X)) 1/2 Convex functions 3 4
30 First-order condition f is differentiable if domf is open and the gradient f(x) = exists at each x domf ( f(x), f(x),..., f(x) ) x 1 x 2 x n 1st-order condition: differentiable f with convex domain is convex iff f(y) f(x)+ f(x) T (y x) for all x,y domf f(y) f(x) + f(x) T (y x) (x,f(x)) first-order approximation of f is global underestimator Convex functions 3 7
31 Second-order conditions f is twice differentiable if domf is open and the Hessian 2 f(x) S n, exists at each x domf 2 f(x) ij = 2 f(x) x i x j, i,j = 1,...,n, 2nd-order conditions: for twice differentiable f with convex domain f is convex if and only if 2 f(x) 0 for all x domf if 2 f(x) 0 for all x domf, then f is strictly convex Convex functions 3 8
32 Examples quadratic function: f(x) = (1/2)x T Px+q T x+r (with P S n ) f(x) = Px+q, 2 f(x) = P convex if P 0 least-squares objective: f(x) = Ax b 2 2 f(x) = 2A T (Ax b), 2 f(x) = 2A T A convex (for any A) quadratic-over-linear: f(x,y) = x 2 /y 2 2 f(x,y) = 2 y 3 [ y x ][ y x ] T 0 f(x,y) convex for y > 0 1 y 0 2 x 0 Convex functions 3 9
33 Epigraph and sublevel set α-sublevel set of f : R n R: C α = {x domf f(x) α} sublevel sets of convex functions are convex (converse is false) epigraph of f : R n R: epif = {(x,t) R n+1 x domf, f(x) t} epif f f is convex if and only if epif is a convex set Convex functions 3 11
34 Properties of convex functions convexity over all lines: f (x) is convex f (x 0 + th) is convex in t for all x 0, h positive multiple: f (x) is convex = αf (x) is convex for all α 0 sum of convex functions: f 1(x), f 2(x) convex = f 1(x) + f 2(x) is convex pointwise maximum: f 1(x), f 2(x) convex = max{f 1(x), f 2(x)} is convex affine transformation of domain: f (x) is convex = f (Ax + b) is convex Convex functions
35 Operations that preserve convexity practical methods for establishing convexity of a function 1. verify definition (often simplified by restricting to a line) 2. for twice differentiable functions, show 2 f(x) 0 3. show that f is obtained from simple convex functions by operations that preserve convexity nonnegative weighted sum composition with affine function pointwise maximum and supremum composition minimization perspective Convex functions 3 13
36 Quasiconvex functions f : R n R is quasiconvex if domf is convex and the sublevel sets are convex for all α S α = {x domf f(x) α} β α a b c f is quasiconcave if f is quasiconvex f is quasilinear if it is quasiconvex and quasiconcave Convex functions 3 23
2. Convex sets. affine and convex sets. some important examples. operations that preserve convexity. generalized inequalities
2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More information2. Convex sets. x 1. x 2. affine set: contains the line through any two distinct points in the set
2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationConvex Sets. CSCI5254: Convex Optimization & Its Applications. subspaces, affine sets, and convex sets. operations that preserve convexity
CSCI5254: Convex Optimization & Its Applications Convex Sets subspaces, affine sets, and convex sets operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationCOM Optimization for Communications Summary: Convex Sets and Convex Functions
1 Convex Sets Affine Sets COM524500 Optimization for Communications Summary: Convex Sets and Convex Functions A set C R n is said to be affine if A point x 1, x 2 C = θx 1 + (1 θ)x 2 C, θ R (1) y = k θ
More informationShiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 2. Convex Optimization
Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 2 Convex Optimization Shiqian Ma, MAT-258A: Numerical Optimization 2 2.1. Convex Optimization General optimization problem: min f 0 (x) s.t., f i
More informationConvex sets and convex functions
Convex sets and convex functions Convex optimization problems Convex sets and their examples Separating and supporting hyperplanes Projections on convex sets Convex functions, conjugate functions ECE 602,
More information2. Convex sets. affine and convex sets. some important examples. operations that preserve convexity. generalized inequalities
2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual
More informationConvex Optimization. Convex Sets. ENSAE: Optimisation 1/24
Convex Optimization Convex Sets ENSAE: Optimisation 1/24 Today affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes
More informationConvex sets and convex functions
Convex sets and convex functions Convex optimization problems Convex sets and their examples Separating and supporting hyperplanes Projections on convex sets Convex functions, conjugate functions ECE 602,
More informationAffine function. suppose f : R n R m is affine (f(x) =Ax + b with A R m n, b R m ) the image of a convex set under f is convex
Affine function suppose f : R n R m is affine (f(x) =Ax + b with A R m n, b R m ) the image of a convex set under f is convex S R n convex = f(s) ={f(x) x S} convex the inverse image f 1 (C) of a convex
More informationCS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets Instructor: Shaddin Dughmi Outline 1 Convex sets, Affine sets, and Cones 2 Examples of Convex Sets 3 Convexity-Preserving Operations
More informationConvex Optimization M2
Convex Optimization M2 Lecture 1 A. d Aspremont. Convex Optimization M2. 1/49 Today Convex optimization: introduction Course organization and other gory details... Convex sets, basic definitions. A. d
More informationIntroduction to Convex Optimization. Prof. Daniel P. Palomar
Introduction to Convex Optimization Prof. Daniel P. Palomar The Hong Kong University of Science and Technology (HKUST) MAFS6010R- Portfolio Optimization with R MSc in Financial Mathematics Fall 2018-19,
More informationCS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets. Instructor: Shaddin Dughmi
CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets Instructor: Shaddin Dughmi Announcements New room: KAP 158 Today: Convex Sets Mostly from Boyd and Vandenberghe. Read all of
More informationIntroduction to Modern Control Systems
Introduction to Modern Control Systems Convex Optimization, Duality and Linear Matrix Inequalities Kostas Margellos University of Oxford AIMS CDT 2016-17 Introduction to Modern Control Systems November
More informationLecture 4: Convexity
10-725: Convex Optimization Fall 2013 Lecture 4: Convexity Lecturer: Barnabás Póczos Scribes: Jessica Chemali, David Fouhey, Yuxiong Wang Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationCS675: Convex and Combinatorial Optimization Fall 2014 Convex Functions. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Fall 2014 Convex Functions Instructor: Shaddin Dughmi Outline 1 Convex Functions 2 Examples of Convex and Concave Functions 3 Convexity-Preserving Operations
More informationLecture 2: August 31
10-725/36-725: Convex Optimization Fall 2016 Lecture 2: August 31 Lecturer: Lecturer: Ryan Tibshirani Scribes: Scribes: Lidan Mu, Simon Du, Binxuan Huang 2.1 Review A convex optimization problem is of
More informationConvexity I: Sets and Functions
Convexity I: Sets and Functions Lecturer: Aarti Singh Co-instructor: Pradeep Ravikumar Convex Optimization 10-725/36-725 See supplements for reviews of basic real analysis basic multivariate calculus basic
More informationLecture: Convex Sets
/24 Lecture: Convex Sets http://bicmr.pku.edu.cn/~wenzw/opt-27-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/24 affine and convex sets some
More informationConvexity Theory and Gradient Methods
Convexity Theory and Gradient Methods Angelia Nedić angelia@illinois.edu ISE Department and Coordinated Science Laboratory University of Illinois at Urbana-Champaign Outline Convex Functions Optimality
More informationConvex Optimization - Chapter 1-2. Xiangru Lian August 28, 2015
Convex Optimization - Chapter 1-2 Xiangru Lian August 28, 2015 1 Mathematical optimization minimize f 0 (x) s.t. f j (x) 0, j=1,,m, (1) x S x. (x 1,,x n ). optimization variable. f 0. R n R. objective
More informationLecture 2: August 29, 2018
10-725/36-725: Convex Optimization Fall 2018 Lecturer: Ryan Tibshirani Lecture 2: August 29, 2018 Scribes: Adam Harley Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have
More informationConvex Sets (cont.) Convex Functions
Convex Sets (cont.) Convex Functions Optimization - 10725 Carlos Guestrin Carnegie Mellon University February 27 th, 2008 1 Definitions of convex sets Convex v. Non-convex sets Line segment definition:
More informationConvex Optimization. 2. Convex Sets. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University. SJTU Ying Cui 1 / 33
Convex Optimization 2. Convex Sets Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2018 SJTU Ying Cui 1 / 33 Outline Affine and convex sets Some important examples Operations
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 6
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 6 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 19, 2012 Andre Tkacenko
More informationLecture 2: August 29, 2018
10-725/36-725: Convex Optimization Fall 2018 Lecturer: Ryan Tibshirani Lecture 2: August 29, 2018 Scribes: Yingjing Lu, Adam Harley, Ruosong Wang Note: LaTeX template courtesy of UC Berkeley EECS dept.
More informationConvex Optimization. Chapter 1 - chapter 2.2
Convex Optimization Chapter 1 - chapter 2.2 Introduction In optimization literatures, one will frequently encounter terms like linear programming, convex set convex cone, convex hull, semidefinite cone
More informationMathematical Programming and Research Methods (Part II)
Mathematical Programming and Research Methods (Part II) 4. Convexity and Optimization Massimiliano Pontil (based on previous lecture by Andreas Argyriou) 1 Today s Plan Convex sets and functions Types
More informationSimplex Algorithm in 1 Slide
Administrivia 1 Canonical form: Simplex Algorithm in 1 Slide If we do pivot in A r,s >0, where c s
More informationLecture 5: Properties of convex sets
Lecture 5: Properties of convex sets Rajat Mittal IIT Kanpur This week we will see properties of convex sets. These properties make convex sets special and are the reason why convex optimization problems
More informationEE/AA 578: Convex Optimization
EE/AA 578: Convex Optimization Instructor: Maryam Fazel University of Washington Fall 2016 1. Introduction EE/AA 578, Univ of Washington, Fall 2016 course logistics mathematical optimization least-squares;
More information60 2 Convex sets. {x a T x b} {x ã T x b}
60 2 Convex sets Exercises Definition of convexity 21 Let C R n be a convex set, with x 1,, x k C, and let θ 1,, θ k R satisfy θ i 0, θ 1 + + θ k = 1 Show that θ 1x 1 + + θ k x k C (The definition of convexity
More informationThis lecture: Convex optimization Convex sets Convex functions Convex optimization problems Why convex optimization? Why so early in the course?
Lec4 Page 1 Lec4p1, ORF363/COS323 This lecture: Convex optimization Convex sets Convex functions Convex optimization problems Why convex optimization? Why so early in the course? Instructor: Amir Ali Ahmadi
More informationLecture 2 Convex Sets
Optimization Theory and Applications Lecture 2 Convex Sets Prof. Chun-Hung Liu Dept. of Electrical and Computer Engineering National Chiao Tung University Fall 2016 2016/9/29 Lecture 2: Convex Sets 1 Outline
More informationAspects of Convex, Nonconvex, and Geometric Optimization (Lecture 1) Suvrit Sra Massachusetts Institute of Technology
Aspects of Convex, Nonconvex, and Geometric Optimization (Lecture 1) Suvrit Sra Massachusetts Institute of Technology Hausdorff Institute for Mathematics (HIM) Trimester: Mathematics of Signal Processing
More informationCMU-Q Lecture 9: Optimization II: Constrained,Unconstrained Optimization Convex optimization. Teacher: Gianni A. Di Caro
CMU-Q 15-381 Lecture 9: Optimization II: Constrained,Unconstrained Optimization Convex optimization Teacher: Gianni A. Di Caro GLOBAL FUNCTION OPTIMIZATION Find the global maximum of the function f x (and
More informationConvexity: an introduction
Convexity: an introduction Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 74 1. Introduction 1. Introduction what is convexity where does it arise main concepts and
More informationIntroduction to Convex Optimization
Introduction to Convex Optimization Lieven Vandenberghe Electrical Engineering Department, UCLA IPAM Graduate Summer School: Computer Vision August 5, 2013 Convex optimization problem minimize f 0 (x)
More informationChapter 4 Convex Optimization Problems
Chapter 4 Convex Optimization Problems Shupeng Gui Computer Science, UR October 16, 2015 hupeng Gui (Computer Science, UR) Convex Optimization Problems October 16, 2015 1 / 58 Outline 1 Optimization problems
More informationLecture 2. Topology of Sets in R n. August 27, 2008
Lecture 2 Topology of Sets in R n August 27, 2008 Outline Vectors, Matrices, Norms, Convergence Open and Closed Sets Special Sets: Subspace, Affine Set, Cone, Convex Set Special Convex Sets: Hyperplane,
More informationDM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini
DM545 Linear and Integer Programming Lecture 2 The Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. 2. 3. 4. Standard Form Basic Feasible Solutions
More informationConic Duality. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 1 Conic Duality Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/
More informationConvex Optimization Lecture 2
Convex Optimization Lecture 2 Today: Convex Analysis Center-of-mass Algorithm 1 Convex Analysis Convex Sets Definition: A set C R n is convex if for all x, y C and all 0 λ 1, λx + (1 λ)y C Operations that
More informationConvex Optimization. Erick Delage, and Ashutosh Saxena. October 20, (a) (b) (c)
Convex Optimization (for CS229) Erick Delage, and Ashutosh Saxena October 20, 2006 1 Convex Sets Definition: A set G R n is convex if every pair of point (x, y) G, the segment beteen x and y is in A. More
More informationLecture 1: Introduction
CSE 599: Interplay between Convex Optimization and Geometry Winter 218 Lecturer: Yin Tat Lee Lecture 1: Introduction Disclaimer: Please tell me any mistake you noticed. 1.1 Course Information Objective:
More informationConvex Optimization. August 26, 2008
Convex Optimization Instructor: Angelia Nedich August 26, 2008 Outline Lecture 1 What is the Course About Who Cares and Why Course Objective Convex Optimization History New Interest in the Topic Formal
More informationConvex Optimization MLSS 2015
Convex Optimization MLSS 2015 Constantine Caramanis The University of Texas at Austin The Optimization Problem minimize : f (x) subject to : x X. The Optimization Problem minimize : f (x) subject to :
More informationIE 521 Convex Optimization
Lecture 4: 5th February 2019 Outline 1 / 23 Which function is different from others? Figure: Functions 2 / 23 Definition of Convex Function Definition. A function f (x) : R n R is convex if (i) dom(f )
More informationAlternating Projections
Alternating Projections Stephen Boyd and Jon Dattorro EE392o, Stanford University Autumn, 2003 1 Alternating projection algorithm Alternating projections is a very simple algorithm for computing a point
More informationCS599: Convex and Combinatorial Optimization Fall 2013 Lecture 1: Introduction to Optimization. Instructor: Shaddin Dughmi
CS599: Convex and Combinatorial Optimization Fall 013 Lecture 1: Introduction to Optimization Instructor: Shaddin Dughmi Outline 1 Course Overview Administrivia 3 Linear Programming Outline 1 Course Overview
More informationCS 435, 2018 Lecture 2, Date: 1 March 2018 Instructor: Nisheeth Vishnoi. Convex Programming and Efficiency
CS 435, 2018 Lecture 2, Date: 1 March 2018 Instructor: Nisheeth Vishnoi Convex Programming and Efficiency In this lecture, we formalize convex programming problem, discuss what it means to solve it efficiently
More information13. Cones and semidefinite constraints
CS/ECE/ISyE 524 Introduction to Optimization Spring 2017 18 13. Cones and semidefinite constraints ˆ Geometry of cones ˆ Second order cone programs ˆ Example: robust linear program ˆ Semidefinite constraints
More informationLecture 2 - Introduction to Polytopes
Lecture 2 - Introduction to Polytopes Optimization and Approximation - ENS M1 Nicolas Bousquet 1 Reminder of Linear Algebra definitions Let x 1,..., x m be points in R n and λ 1,..., λ m be real numbers.
More informationAdvanced Operations Research Techniques IE316. Quiz 1 Review. Dr. Ted Ralphs
Advanced Operations Research Techniques IE316 Quiz 1 Review Dr. Ted Ralphs IE316 Quiz 1 Review 1 Reading for The Quiz Material covered in detail in lecture. 1.1, 1.4, 2.1-2.6, 3.1-3.3, 3.5 Background material
More informationCO367/CM442 Nonlinear Optimization Lecture 1
CO367/CM442 Nonlinear Optimization Lecture 1 Instructor: Henry Wolkowicz hwolkowi@uwaterloo.ca TA: Vris Cheung, yl2cheun@math.uwaterloo.ca orion.math.uwaterloo.ca/ hwolkowi/henry/teaching/w10/367.w10/index.shtml
More informationApplied Integer Programming
Applied Integer Programming D.S. Chen; R.G. Batson; Y. Dang Fahimeh 8.2 8.7 April 21, 2015 Context 8.2. Convex sets 8.3. Describing a bounded polyhedron 8.4. Describing unbounded polyhedron 8.5. Faces,
More informationMA4254: Discrete Optimization. Defeng Sun. Department of Mathematics National University of Singapore Office: S Telephone:
MA4254: Discrete Optimization Defeng Sun Department of Mathematics National University of Singapore Office: S14-04-25 Telephone: 6516 3343 Aims/Objectives: Discrete optimization deals with problems of
More informationLinear and Integer Programming :Algorithms in the Real World. Related Optimization Problems. How important is optimization?
Linear and Integer Programming 15-853:Algorithms in the Real World Linear and Integer Programming I Introduction Geometric Interpretation Simplex Method Linear or Integer programming maximize z = c T x
More informationIE521 Convex Optimization Introduction
IE521 Convex Optimization Introduction Instructor: Niao He Jan 18, 2017 1 About Me Assistant Professor, UIUC, 2016 Ph.D. in Operations Research, M.S. in Computational Sci. & Eng. Georgia Tech, 2010 2015
More informationAlgorithms for convex optimization
Algorithms for convex optimization Michal Kočvara Institute of Information Theory and Automation Academy of Sciences of the Czech Republic and Czech Technical University kocvara@utia.cas.cz http://www.utia.cas.cz/kocvara
More information15. Cutting plane and ellipsoid methods
EE 546, Univ of Washington, Spring 2012 15. Cutting plane and ellipsoid methods localization methods cutting-plane oracle examples of cutting plane methods ellipsoid method convergence proof inequality
More informationOptimization under uncertainty: modeling and solution methods
Optimization under uncertainty: modeling and solution methods Paolo Brandimarte Dipartimento di Scienze Matematiche Politecnico di Torino e-mail: paolo.brandimarte@polito.it URL: http://staff.polito.it/paolo.brandimarte
More informationLECTURE 10 LECTURE OUTLINE
We now introduce a new concept with important theoretical and algorithmic implications: polyhedral convexity, extreme points, and related issues. LECTURE 1 LECTURE OUTLINE Polar cones and polar cone theorem
More informationMTAEA Convexity and Quasiconvexity
School of Economics, Australian National University February 19, 2010 Convex Combinations and Convex Sets. Definition. Given any finite collection of points x 1,..., x m R n, a point z R n is said to be
More informationWeek 5. Convex Optimization
Week 5. Convex Optimization Lecturer: Prof. Santosh Vempala Scribe: Xin Wang, Zihao Li Feb. 9 and, 206 Week 5. Convex Optimization. The convex optimization formulation A general optimization problem is
More informationCME307/MS&E311 Theory Summary
CME307/MS&E311 Theory Summary Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/~yyye http://www.stanford.edu/class/msande311/
More informationLecture 3: Convex sets
Lecture 3: Convex sets Rajat Mittal IIT Kanpur We denote the set of real numbers as R. Most of the time we will be working with space R n and its elements will be called vectors. Remember that a subspace
More informationConvex Optimization CMU-10725
Convex Optimization CMU-10725 Ellipsoid Methods Barnabás Póczos & Ryan Tibshirani Outline Linear programs Simplex algorithm Running time: Polynomial or Exponential? Cutting planes & Ellipsoid methods for
More informationConvex Optimization Euclidean Distance Geometry 2ε
Convex Optimization Euclidean Distance Geometry 2ε In my career, I found that the best people are the ones that really understand the content, and they re a pain in the butt to manage. But you put up with
More informationConvex Optimization. Lijun Zhang Modification of
Convex Optimization Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Modification of http://stanford.edu/~boyd/cvxbook/bv_cvxslides.pdf Outline Introduction Convex Sets & Functions Convex Optimization
More information1. Introduction. performance of numerical methods. complexity bounds. structural convex optimization. course goals and topics
1. Introduction EE 546, Univ of Washington, Spring 2016 performance of numerical methods complexity bounds structural convex optimization course goals and topics 1 1 Some course info Welcome to EE 546!
More informationof Convex Analysis Fundamentals Jean-Baptiste Hiriart-Urruty Claude Lemarechal Springer With 66 Figures
2008 AGI-Information Management Consultants May be used for personal purporses only or by libraries associated to dandelon.com network. Jean-Baptiste Hiriart-Urruty Claude Lemarechal Fundamentals of Convex
More informationDomain Specific Languages for Convex Optimization
Domain Specific Languages for Convex Optimization Stephen Boyd joint work with E. Chu, J. Mattingley, M. Grant Electrical Engineering Department, Stanford University ROKS 2013, Leuven, 9 July 2013 1 Outline
More informationA Course in Convexity
A Course in Convexity Alexander Barvinok Graduate Studies in Mathematics Volume 54 American Mathematical Society Providence, Rhode Island Preface vii Chapter I. Convex Sets at Large 1 1. Convex Sets. Main
More informationA mini-introduction to convexity
A mini-introduction to convexity Geir Dahl March 14, 2017 1 Introduction Convexity, or convex analysis, is an area of mathematics where one studies questions related to two basic objects, namely convex
More informationLecture 19: Convex Non-Smooth Optimization. April 2, 2007
: Convex Non-Smooth Optimization April 2, 2007 Outline Lecture 19 Convex non-smooth problems Examples Subgradients and subdifferentials Subgradient properties Operations with subgradients and subdifferentials
More informationNumerical Optimization
Convex Sets Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Let x 1, x 2 R n, x 1 x 2. Line and line segment Line passing through x 1 and x 2 : {y
More informationConvex Sets. Pontus Giselsson
Convex Sets Pontus Giselsson 1 Today s lecture convex sets convex, affine, conical hulls closure, interior, relative interior, boundary, relative boundary separating and supporting hyperplane theorems
More informationRevisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization. Author: Martin Jaggi Presenter: Zhongxing Peng
Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization Author: Martin Jaggi Presenter: Zhongxing Peng Outline 1. Theoretical Results 2. Applications Outline 1. Theoretical Results 2. Applications
More informationCME307/MS&E311 Optimization Theory Summary
CME307/MS&E311 Optimization Theory Summary Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/~yyye http://www.stanford.edu/class/msande311/
More informationLecture 5: Duality Theory
Lecture 5: Duality Theory Rajat Mittal IIT Kanpur The objective of this lecture note will be to learn duality theory of linear programming. We are planning to answer following questions. What are hyperplane
More informationConvex Optimization Euclidean Distance Geometry 2ε
Convex Optimization Euclidean Distance Geometry 2ε 1 Overview 19 2 Convex geometry 31 2.1 Convex set.................................... 31 2.2 Vectorized-matrix inner product........................ 42
More informationCS 372: Computational Geometry Lecture 10 Linear Programming in Fixed Dimension
CS 372: Computational Geometry Lecture 10 Linear Programming in Fixed Dimension Antoine Vigneron King Abdullah University of Science and Technology November 7, 2012 Antoine Vigneron (KAUST) CS 372 Lecture
More informationConvex Geometry arising in Optimization
Convex Geometry arising in Optimization Jesús A. De Loera University of California, Davis Berlin Mathematical School Summer 2015 WHAT IS THIS COURSE ABOUT? Combinatorial Convexity and Optimization PLAN
More informationCombinatorial Geometry & Topology arising in Game Theory and Optimization
Combinatorial Geometry & Topology arising in Game Theory and Optimization Jesús A. De Loera University of California, Davis LAST EPISODE... We discuss the content of the course... Convex Sets A set is
More information4 LINEAR PROGRAMMING (LP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1
4 LINEAR PROGRAMMING (LP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 Mathematical programming (optimization) problem: min f (x) s.t. x X R n set of feasible solutions with linear objective function
More informationProgramming, numerics and optimization
Programming, numerics and optimization Lecture C-4: Constrained optimization Łukasz Jankowski ljank@ippt.pan.pl Institute of Fundamental Technological Research Room 4.32, Phone +22.8261281 ext. 428 June
More informationLec13p1, ORF363/COS323
Lec13 Page 1 Lec13p1, ORF363/COS323 This lecture: Semidefinite programming (SDP) Definition and basic properties Review of positive semidefinite matrices SDP duality SDP relaxations for nonconvex optimization
More informationLocally convex topological vector spaces
Chapter 4 Locally convex topological vector spaces 4.1 Definition by neighbourhoods Let us start this section by briefly recalling some basic properties of convex subsets of a vector space over K (where
More informationCS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 50
CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring 2018 1 / 50 CS 473: Algorithms, Spring 2018 Introduction to Linear Programming Lecture 18 March
More informationLinear Programming in Small Dimensions
Linear Programming in Small Dimensions Lekcija 7 sergio.cabello@fmf.uni-lj.si FMF Univerza v Ljubljani Edited from slides by Antoine Vigneron Outline linear programming, motivation and definition one dimensional
More informationELEG Compressive Sensing and Sparse Signal Representations
ELEG 867 - Compressive Sensing and Sparse Signal Representations Introduction to Matrix Completion and Robust PCA Gonzalo Garateguy Depart. of Electrical and Computer Engineering University of Delaware
More informationNonlinear Programming
Nonlinear Programming SECOND EDITION Dimitri P. Bertsekas Massachusetts Institute of Technology WWW site for book Information and Orders http://world.std.com/~athenasc/index.html Athena Scientific, Belmont,
More informationCS675: Convex and Combinatorial Optimization Spring 2018 The Simplex Algorithm. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Spring 2018 The Simplex Algorithm Instructor: Shaddin Dughmi Algorithms for Convex Optimization We will look at 2 algorithms in detail: Simplex and Ellipsoid.
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 3: Linear Programming, Continued Prof. John Gunnar Carlsson September 15, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, 2010
More informationConvex optimization algorithms for sparse and low-rank representations
Convex optimization algorithms for sparse and low-rank representations Lieven Vandenberghe, Hsiao-Han Chao (UCLA) ECC 2013 Tutorial Session Sparse and low-rank representation methods in control, estimation,
More informationMinima, Maxima, Saddle points
Minima, Maxima, Saddle points Levent Kandiller Industrial Engineering Department Çankaya University, Turkey Minima, Maxima, Saddle points p./9 Scalar Functions Let us remember the properties for maxima,
More informationEC 521 MATHEMATICAL METHODS FOR ECONOMICS. Lecture 2: Convex Sets
EC 51 MATHEMATICAL METHODS FOR ECONOMICS Lecture : Convex Sets Murat YILMAZ Boğaziçi University In this section, we focus on convex sets, separating hyperplane theorems and Farkas Lemma. And as an application
More informationConstrained optimization
Constrained optimization Problem in standard form minimize f(x) subject to a i (x) = 0, for i =1, 2, p c j (x) 0 for j =1, 2,,q f : R n R a i : R n R c j : R n R f(x )=, if problem is infeasible f(x )=,
More information