Conic Duality. yyye

Size: px
Start display at page:

Download "Conic Duality. yyye"

Transcription

1 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 1 Conic Duality Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. yyye

2 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 2 Vectors and Norms Real numbers: R, R +, int R + n-dimensional Euclidean space R n, R n +, int R n + Component-wise: x y means x j y j for j =1, 2,..., n 0: vector of all zeros; and e: vector of all ones Inner-product of two vectors: x y := x T y = n x j y j j=1 Euclidean norm: x 2 = x T x, Infinity-norm: x =max{ x 1, x 2,..., x n }, ( n ) 1/p p-norm: x p = j=1 x j p

3 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 3 The dual of the p norm, denoted by., is the q norm, where 1 p + 1 q =1 Column vector: Row vector: x =(x 1 ; x 2 ;...; x n ) x =(x 1,x 2,...,x n ) Transpose operation: A T A set of vectors a 1,..., a m is said to be linearly dependent if there are scalars λ 1,..., λ m, not all zero, such that the linear combination m λ i a i = 0 i=1 A linearly independent set of vectors that span R n is a basis.

4 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 4 Hyper plane and Half-spaces H = {x : ax = n a j x j = b} j=1 H + = {x : ax = H = {x : ax = n a j x j b} j=1 n a j x j b} j=1

5 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 5 y (0,3) 3x+5y>15 3x+5y=15 3x+5y<15 0 (5,0) x Figure 1: Plane and Half-Spaces

6 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 6 Matrices and Norms Matrix: R m n, ith row: a i., jth column: a.j, ijth element: a ij A I denotes the submatrix of A whose rows belong to index set I, A J denotes the submatrix whose columns belong to index set J, A IJ denotes the submatrix whose rows belong to index set I and columns belong to index set J. Determinant: det(a), Trace: tr(a) The operator norm of A, A 2 := Ax 2 max 0 x R n x 2 All-zero matrix: 0, and identity matrix: I Symmetric matrix: Q = Q T

7 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 7 Positive Definite: Q 0 iff x T Qx > 0, for all x 0 Positive Semidefinite: Q 0 iff x T Qx 0, for all x Null space and Row space of matrix A: Theorem 1 The null space and row space of a matrix are perpenticular to each other, that is, x T s =0, Ax = 0 and s = A T y.

8 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 8 Symmetric Matrix Space n-dimensional symmetric matrix space: S n Inner Product: X Y = trx T Y = i,j X i,j Y i,j Frobenius norm: X f = trx T X Positive semidefinite matrix set: S n +, Positive definite matrix set: int S n +

9 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 9 Decomposition of Symmetric Positive Semidefinite Matrices: X = r x i x T i where r is the rank of X, and x T i x j =0for i j. Let X be a positive semidefinite matrix of rank r, A be any given symmetric i=1 matrix. Then, there is a decomposition of X X = r x j x T j, j=1 such that for all j, x T j Ax j = A (x j x T j )=A X/r.

10 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 10 Affine and Convex Combination S R n is affine if [ x, y S and α R ]= αx +(1 α)y S. When x and y are two distinct points in R n and α runs over R, {z : z = αx +(1 α)y} is the line set determined by x and y. When 0 α 1, it is called the convex combination of x and y and it is the line segment between x and y.

11 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 11 Convex Sets Set notations: x Ω, y Ω S T, S T Ω is said to be a convex set if for every x 1, x 2 Ω and every real number α [0, 1], the point αx 1 +(1 α)x 2 Ω. Intersection of convex sets is convex; the convex hull of a set Ω is the intersection of all convex sets containing Ω A point in a set is called an extreme point of the set if it cannot be represented as the convex combination of two distinct points of the set. A set is a polyhedral set if it has finitely many extreme points.

12 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 12 Set Combinations Let C 1 and C 2 be convex sets in a same space. Then, C 1 C 2 is convex. C 1 + C 2 is convex, where C 1 + C 2 = {b 1 + b 2 : b 1 C 1 and b 2 C 2 }. C 1 C 2 is convex, where C 1 C 2 = {(b 1 ; b 2 ): b 1 C 1 and b 2 C 2 }.

13 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 13 Cones A set K is a cone if x K implies αx K for all α>0 A convex cone is cone and it s also a convex-set. Dual cone: K := {y : y x 0 for all x K} K is also called the polar of K. The dual of a cone is always a closed convex cone.

14 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 14 Cone Examples Example 2.1: The n-dimensional non-negative orthant, R n + = {x R n : x 0}, is a convex cone; and it s self dual. Example 2.2: The set of all positive semi-definite symmetric matrices in S n, S+ n, is a convex cone, called the positive semi-definite matrix cone; and it s self dual. Example 2.3: The set {x R n : x 1 x 1 }, N n 2 R n called the second-order (norm) cone; and it s self dual., is a convex cone in Example 2.4: The set {x R n : x 1 x 1 p }, Np n, is a convex cone in R n for p 1 called the p-order (norm) cone; and its dual is the q-order cone where 1 p + 1 q =1.

15 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 15 Cone and Dual Facts Let K 1 and K 2 be both closed convex cones. Then i) (K1 ) = K 1. ii) K 1 K 2 = K2 K1. iii) (K 1 K 2 ) = K1 K2. iv) (K 1 + K 2 ) = K1 K2. v) (K 1 K 2 ) = K1 + K2.

16 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 16 Convex Polyhedral Cones I A cone K is (convex) polyhedral if its intersection with a hyperplane is a polyhedral set. A convex cone K is polyhedral if and only if K can be represented by K = {x : Ax 0} or {x : x = Ay, y 0} for some matrix A. In the latter case, K is generated by the columns of A. The nonnegative orthant is a polyhedral cone but the second-order cone is not polyhedral.

17 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 17 Polyhedral Cone Nonpolyhedral Cone Figure 2: Polyhedral and non-polyhedral cones.

18 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 18 Convex Polyhedral Cones II It has been proved that for cones the concepts of polyhedral and finitely generated are equivalent according to the following theorem. Theorem 2 (Minkowski and Weyl) A convex cone C is polyhedral if and only if it is finitely generated, that is, the cone is generated by a finite number of vectors b 1,...,b m : C = cone(b 1,..., b m ):= { m i=1 b i y i : y i 0, i=1,..., m }.

19 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 19 Carathéodory s theorem The following theorem states that a polyhedral cone can be generated by a set of basic directional vectors. Theorem 3 Let convex polyhedral cone C = cone(b 1,..., b m ) and x C. Then, x cone(b i1,..., b id ) for some linearly independent vectors b i1,...,b id chosen from b 1,...,b m. Some times we even have: x R2 + : x 0 = 1 2 y y 2 : y 1,y 2 0.

20 Conic Linear Optimization and Appl. MS&E314 Lecture Note # (1,2) 2 (2,1) 1 2 Figure 3: Representations of a polyhedral cone.

21 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 21 Separating hyperplane theorem The most important theorem about the convex set is the following separating hyperplane theorem (Figure 4). Theorem 4 (Separating hyperplane theorem) Let C E, where E is either R n or S n, be a closed convex set and let b be a point exterior to C. Then there is a vector a Esuch that a b > sup a x x C where a is the norm direction of the hyperplane.

22 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 22 C b -a Figure 4: Illustration of the separating hyperplane theorem; an exterior point b is separated by a hyperplane from a convex set C.

23 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 23 Examples Let C be a unit circle centered at point (1; 1). That is, C = {x R 2 : (x 1 1) 2 +(x 2 1) 2 1}. Ifb = (2; 0), a =( 1; 1) is a separating hyperplane vector. If b =(0; 1), a = (0; 1) is a separating hyperplane vector. It is worth noting that these separating hyperplanes are not unique.

24 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 24 Farkas Lemma for Polyhedral Cone Theorem 5 Let A R m n and b R m. Then, the system {x : Ax = b, x R n +} has a feasible solution x if and only if that {y : A T y R n +, b T y > 0, (b T y =1)} has no feasible solution. A vector y, with A T y 0 and b T y > 0, is called a infeasibility certificate for the system {x : Ax = b, x 0}. Example: Let A =(1, 1) and b = 1. Then, y = 1 is an infeasibility certificate for {x : Ax = b, x 0}.

25 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 25 Alternative Systems Farkas lemma is also called the alternative theorem, that is, exactly one of the two systems: and {x : Ax = b, x 0} {y : A T y 0, b T y > 0, (b T y =1)}, is feasible. Geometrically, Farkas lemma means that if a vector b R m does not belong to the cone generated by a.1,..., a.n, then there is a hyperplane separating b from cone(a.1,..., a.n ), that is, b C := {Ax : x 0}, which is a closed convex set(?).

26 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 26 Proof Let {x : Ax = b, x 0} have a feasible solution, say x. Then, {y : A T y 0, b T y > 0} is infeasible, since otherwise, since x 0 and A T y 0. 0 < b T y =(Ax) T y = x T (A T y) 0 Now let {x : Ax = b, x 0} have no feasible solution, that is, b C := {Ax : x 0}. Then, by the separating hyperplane theorem, there is y such that or y b > sup y c c C y b > sup y (Ax) =supa T y x. (1) x 0 x 0 Since 0 C we have y b > 0.

27 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 27 Furthermore, A T y 0. Since otherwise, say (A T y) 1 > 0, one can have a vector x 0 such that x 1 = α>0, x 2 =... = x n =0, from which sup A T y x A T y x =(A T y) 1 α x 0 and it tends to as α. This is a contradiction because sup x 0 A T y x is bounded from above by (1).

28 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 28 Farkas Lemma variant Theorem 6 Let A R m n and c R n. Then, the system {y : A T y c} has a solution y if and only if that Ax = 0, x 0, c T x < 0 has no feasible solution x. Again, a vector x 0, with Ax = 0 and c T x < 0, is called a infeasibility certificate for the system {y : A T y c}. Example: Let A =(1; 1) and c =(1; 2). Then, x = (1; 1) is an infeasibility certificate for {y : A T y c}.

29 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 29 Alternative Systems for General Cone? {x : Ax = b, x C} and {y : A T y C, b T y > 0}? Counterexample: A 1 = ,A 2 = and b = 0 2,C= S 2 +.

30 Conic Linear Optimization and Appl. MS&E314 Lecture Note #02 30 Farkas Lemma for General Convex Cone Theorem 7 Consider system {x : Ax = b, x K} for a (closed) convex cone K. Suppose that there exists vector ȳ such that A T ȳ int K. Then, Set C := {Ax : x K} is a closed convex set. The system {x : Ax = b, x K} has a feasible solution x if and only if that {y : A T y K, b T y > 0, (b T y =1)} has no feasible solution. Corollary 1 Consider system {(y, s) : A T y + s = c, s K} for a (closed) convex cone K. Suppose that there exists vector x such that A x = 0, x int K. Then, Set C := {A T y + s : s K} is a closed convex set. The system {x : Ax = 0, c x = 1, x K } has a feasible solution x if and only if that {(y, s) : A T y + s = c, s K} has no feasible solution.

Math 5593 Linear Programming Lecture Notes

Math 5593 Linear Programming Lecture Notes Math 5593 Linear Programming Lecture Notes Unit II: Theory & Foundations (Convex Analysis) University of Colorado Denver, Fall 2013 Topics 1 Convex Sets 1 1.1 Basic Properties (Luenberger-Ye Appendix B.1).........................

More information

Lecture 2. Topology of Sets in R n. August 27, 2008

Lecture 2. Topology of Sets in R n. August 27, 2008 Lecture 2 Topology of Sets in R n August 27, 2008 Outline Vectors, Matrices, Norms, Convergence Open and Closed Sets Special Sets: Subspace, Affine Set, Cone, Convex Set Special Convex Sets: Hyperplane,

More information

Convex Geometry arising in Optimization

Convex Geometry arising in Optimization Convex Geometry arising in Optimization Jesús A. De Loera University of California, Davis Berlin Mathematical School Summer 2015 WHAT IS THIS COURSE ABOUT? Combinatorial Convexity and Optimization PLAN

More information

Numerical Optimization

Numerical Optimization Convex Sets Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Let x 1, x 2 R n, x 1 x 2. Line and line segment Line passing through x 1 and x 2 : {y

More information

Lecture 5: Duality Theory

Lecture 5: Duality Theory Lecture 5: Duality Theory Rajat Mittal IIT Kanpur The objective of this lecture note will be to learn duality theory of linear programming. We are planning to answer following questions. What are hyperplane

More information

MA4254: Discrete Optimization. Defeng Sun. Department of Mathematics National University of Singapore Office: S Telephone:

MA4254: Discrete Optimization. Defeng Sun. Department of Mathematics National University of Singapore Office: S Telephone: MA4254: Discrete Optimization Defeng Sun Department of Mathematics National University of Singapore Office: S14-04-25 Telephone: 6516 3343 Aims/Objectives: Discrete optimization deals with problems of

More information

Combinatorial Geometry & Topology arising in Game Theory and Optimization

Combinatorial Geometry & Topology arising in Game Theory and Optimization Combinatorial Geometry & Topology arising in Game Theory and Optimization Jesús A. De Loera University of California, Davis LAST EPISODE... We discuss the content of the course... Convex Sets A set is

More information

Polytopes Course Notes

Polytopes Course Notes Polytopes Course Notes Carl W. Lee Department of Mathematics University of Kentucky Lexington, KY 40506 lee@ms.uky.edu Fall 2013 i Contents 1 Polytopes 1 1.1 Convex Combinations and V-Polytopes.....................

More information

Polar Duality and Farkas Lemma

Polar Duality and Farkas Lemma Lecture 3 Polar Duality and Farkas Lemma October 8th, 2004 Lecturer: Kamal Jain Notes: Daniel Lowd 3.1 Polytope = bounded polyhedron Last lecture, we were attempting to prove the Minkowsky-Weyl Theorem:

More information

2. Convex sets. x 1. x 2. affine set: contains the line through any two distinct points in the set

2. Convex sets. x 1. x 2. affine set: contains the line through any two distinct points in the set 2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual

More information

Convex Optimization. Convex Sets. ENSAE: Optimisation 1/24

Convex Optimization. Convex Sets. ENSAE: Optimisation 1/24 Convex Optimization Convex Sets ENSAE: Optimisation 1/24 Today affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes

More information

EC 521 MATHEMATICAL METHODS FOR ECONOMICS. Lecture 2: Convex Sets

EC 521 MATHEMATICAL METHODS FOR ECONOMICS. Lecture 2: Convex Sets EC 51 MATHEMATICAL METHODS FOR ECONOMICS Lecture : Convex Sets Murat YILMAZ Boğaziçi University In this section, we focus on convex sets, separating hyperplane theorems and Farkas Lemma. And as an application

More information

Lecture 2 Convex Sets

Lecture 2 Convex Sets Optimization Theory and Applications Lecture 2 Convex Sets Prof. Chun-Hung Liu Dept. of Electrical and Computer Engineering National Chiao Tung University Fall 2016 2016/9/29 Lecture 2: Convex Sets 1 Outline

More information

Lecture 2 September 3

Lecture 2 September 3 EE 381V: Large Scale Optimization Fall 2012 Lecture 2 September 3 Lecturer: Caramanis & Sanghavi Scribe: Hongbo Si, Qiaoyang Ye 2.1 Overview of the last Lecture The focus of the last lecture was to give

More information

Linear programming and duality theory

Linear programming and duality theory Linear programming and duality theory Complements of Operations Research Giovanni Righini Linear Programming (LP) A linear program is defined by linear constraints, a linear objective function. Its variables

More information

2. Convex sets. affine and convex sets. some important examples. operations that preserve convexity. generalized inequalities

2. Convex sets. affine and convex sets. some important examples. operations that preserve convexity. generalized inequalities 2. Convex sets Convex Optimization Boyd & Vandenberghe affine and convex sets some important examples operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual

More information

FACES OF CONVEX SETS

FACES OF CONVEX SETS FACES OF CONVEX SETS VERA ROSHCHINA Abstract. We remind the basic definitions of faces of convex sets and their basic properties. For more details see the classic references [1, 2] and [4] for polytopes.

More information

Convex Sets. CSCI5254: Convex Optimization & Its Applications. subspaces, affine sets, and convex sets. operations that preserve convexity

Convex Sets. CSCI5254: Convex Optimization & Its Applications. subspaces, affine sets, and convex sets. operations that preserve convexity CSCI5254: Convex Optimization & Its Applications Convex Sets subspaces, affine sets, and convex sets operations that preserve convexity generalized inequalities separating and supporting hyperplanes dual

More information

Lecture 2 - Introduction to Polytopes

Lecture 2 - Introduction to Polytopes Lecture 2 - Introduction to Polytopes Optimization and Approximation - ENS M1 Nicolas Bousquet 1 Reminder of Linear Algebra definitions Let x 1,..., x m be points in R n and λ 1,..., λ m be real numbers.

More information

DM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini

DM545 Linear and Integer Programming. Lecture 2. The Simplex Method. Marco Chiarandini DM545 Linear and Integer Programming Lecture 2 The Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. 2. 3. 4. Standard Form Basic Feasible Solutions

More information

Convex Optimization - Chapter 1-2. Xiangru Lian August 28, 2015

Convex Optimization - Chapter 1-2. Xiangru Lian August 28, 2015 Convex Optimization - Chapter 1-2 Xiangru Lian August 28, 2015 1 Mathematical optimization minimize f 0 (x) s.t. f j (x) 0, j=1,,m, (1) x S x. (x 1,,x n ). optimization variable. f 0. R n R. objective

More information

In this chapter we introduce some of the basic concepts that will be useful for the study of integer programming problems.

In this chapter we introduce some of the basic concepts that will be useful for the study of integer programming problems. 2 Basics In this chapter we introduce some of the basic concepts that will be useful for the study of integer programming problems. 2.1 Notation Let A R m n be a matrix with row index set M = {1,...,m}

More information

CS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets. Instructor: Shaddin Dughmi

CS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets. Instructor: Shaddin Dughmi CS675: Convex and Combinatorial Optimization Spring 2018 Convex Sets Instructor: Shaddin Dughmi Outline 1 Convex sets, Affine sets, and Cones 2 Examples of Convex Sets 3 Convexity-Preserving Operations

More information

Shiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 2. Convex Optimization

Shiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 2. Convex Optimization Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 2 Convex Optimization Shiqian Ma, MAT-258A: Numerical Optimization 2 2.1. Convex Optimization General optimization problem: min f 0 (x) s.t., f i

More information

Convexity: an introduction

Convexity: an introduction Convexity: an introduction Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 74 1. Introduction 1. Introduction what is convexity where does it arise main concepts and

More information

Lecture 4: Rational IPs, Polyhedron, Decomposition Theorem

Lecture 4: Rational IPs, Polyhedron, Decomposition Theorem IE 5: Integer Programming, Spring 29 24 Jan, 29 Lecture 4: Rational IPs, Polyhedron, Decomposition Theorem Lecturer: Karthik Chandrasekaran Scribe: Setareh Taki Disclaimer: These notes have not been subjected

More information

Mathematical and Algorithmic Foundations Linear Programming and Matchings

Mathematical and Algorithmic Foundations Linear Programming and Matchings Adavnced Algorithms Lectures Mathematical and Algorithmic Foundations Linear Programming and Matchings Paul G. Spirakis Department of Computer Science University of Patras and Liverpool Paul G. Spirakis

More information

Tutorial on Convex Optimization for Engineers

Tutorial on Convex Optimization for Engineers Tutorial on Convex Optimization for Engineers M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D-98684 Ilmenau, Germany jens.steinwandt@tu-ilmenau.de

More information

Lecture 4: Convexity

Lecture 4: Convexity 10-725: Convex Optimization Fall 2013 Lecture 4: Convexity Lecturer: Barnabás Póczos Scribes: Jessica Chemali, David Fouhey, Yuxiong Wang Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:

More information

Section Notes 5. Review of Linear Programming. Applied Math / Engineering Sciences 121. Week of October 15, 2017

Section Notes 5. Review of Linear Programming. Applied Math / Engineering Sciences 121. Week of October 15, 2017 Section Notes 5 Review of Linear Programming Applied Math / Engineering Sciences 121 Week of October 15, 2017 The following list of topics is an overview of the material that was covered in the lectures

More information

Advanced Operations Research Techniques IE316. Quiz 1 Review. Dr. Ted Ralphs

Advanced Operations Research Techniques IE316. Quiz 1 Review. Dr. Ted Ralphs Advanced Operations Research Techniques IE316 Quiz 1 Review Dr. Ted Ralphs IE316 Quiz 1 Review 1 Reading for The Quiz Material covered in detail in lecture. 1.1, 1.4, 2.1-2.6, 3.1-3.3, 3.5 Background material

More information

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets. Instructor: Shaddin Dughmi

CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets. Instructor: Shaddin Dughmi CS599: Convex and Combinatorial Optimization Fall 2013 Lecture 4: Convex Sets Instructor: Shaddin Dughmi Announcements New room: KAP 158 Today: Convex Sets Mostly from Boyd and Vandenberghe. Read all of

More information

A mini-introduction to convexity

A mini-introduction to convexity A mini-introduction to convexity Geir Dahl March 14, 2017 1 Introduction Convexity, or convex analysis, is an area of mathematics where one studies questions related to two basic objects, namely convex

More information

Convex Optimization. 2. Convex Sets. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University. SJTU Ying Cui 1 / 33

Convex Optimization. 2. Convex Sets. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University. SJTU Ying Cui 1 / 33 Convex Optimization 2. Convex Sets Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2018 SJTU Ying Cui 1 / 33 Outline Affine and convex sets Some important examples Operations

More information

LP Geometry: outline. A general LP. minimize x c T x s.t. a T i. x b i, i 2 M 1 a T i x = b i, i 2 M 3 x j 0, j 2 N 1. where

LP Geometry: outline. A general LP. minimize x c T x s.t. a T i. x b i, i 2 M 1 a T i x = b i, i 2 M 3 x j 0, j 2 N 1. where LP Geometry: outline I Polyhedra I Extreme points, vertices, basic feasible solutions I Degeneracy I Existence of extreme points I Optimality of extreme points IOE 610: LP II, Fall 2013 Geometry of Linear

More information

Lecture 6: Faces, Facets

Lecture 6: Faces, Facets IE 511: Integer Programming, Spring 2019 31 Jan, 2019 Lecturer: Karthik Chandrasekaran Lecture 6: Faces, Facets Scribe: Setareh Taki Disclaimer: These notes have not been subjected to the usual scrutiny

More information

MATH 423 Linear Algebra II Lecture 17: Reduced row echelon form (continued). Determinant of a matrix.

MATH 423 Linear Algebra II Lecture 17: Reduced row echelon form (continued). Determinant of a matrix. MATH 423 Linear Algebra II Lecture 17: Reduced row echelon form (continued). Determinant of a matrix. Row echelon form A matrix is said to be in the row echelon form if the leading entries shift to the

More information

Convex Sets. Pontus Giselsson

Convex Sets. Pontus Giselsson Convex Sets Pontus Giselsson 1 Today s lecture convex sets convex, affine, conical hulls closure, interior, relative interior, boundary, relative boundary separating and supporting hyperplane theorems

More information

LECTURE 10 LECTURE OUTLINE

LECTURE 10 LECTURE OUTLINE We now introduce a new concept with important theoretical and algorithmic implications: polyhedral convexity, extreme points, and related issues. LECTURE 1 LECTURE OUTLINE Polar cones and polar cone theorem

More information

ORIE 6300 Mathematical Programming I September 2, Lecture 3

ORIE 6300 Mathematical Programming I September 2, Lecture 3 ORIE 6300 Mathematical Programming I September 2, 2014 Lecturer: David P. Williamson Lecture 3 Scribe: Divya Singhvi Last time we discussed how to take dual of an LP in two different ways. Today we will

More information

CME307/MS&E311 Optimization Theory Summary

CME307/MS&E311 Optimization Theory Summary CME307/MS&E311 Optimization Theory Summary Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/~yyye http://www.stanford.edu/class/msande311/

More information

Multiple View Geometry in Computer Vision

Multiple View Geometry in Computer Vision Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 Projective 3D Geometry (Back to Chapter 2) Lecture 6 2 Singular Value Decomposition Given a

More information

Advanced Operations Research Techniques IE316. Quiz 2 Review. Dr. Ted Ralphs

Advanced Operations Research Techniques IE316. Quiz 2 Review. Dr. Ted Ralphs Advanced Operations Research Techniques IE316 Quiz 2 Review Dr. Ted Ralphs IE316 Quiz 2 Review 1 Reading for The Quiz Material covered in detail in lecture Bertsimas 4.1-4.5, 4.8, 5.1-5.5, 6.1-6.3 Material

More information

Convex Optimization. Chapter 1 - chapter 2.2

Convex Optimization. Chapter 1 - chapter 2.2 Convex Optimization Chapter 1 - chapter 2.2 Introduction In optimization literatures, one will frequently encounter terms like linear programming, convex set convex cone, convex hull, semidefinite cone

More information

Lecture 3: Convex sets

Lecture 3: Convex sets Lecture 3: Convex sets Rajat Mittal IIT Kanpur We denote the set of real numbers as R. Most of the time we will be working with space R n and its elements will be called vectors. Remember that a subspace

More information

Some Advanced Topics in Linear Programming

Some Advanced Topics in Linear Programming Some Advanced Topics in Linear Programming Matthew J. Saltzman July 2, 995 Connections with Algebra and Geometry In this section, we will explore how some of the ideas in linear programming, duality theory,

More information

of Convex Analysis Fundamentals Jean-Baptiste Hiriart-Urruty Claude Lemarechal Springer With 66 Figures

of Convex Analysis Fundamentals Jean-Baptiste Hiriart-Urruty Claude Lemarechal Springer With 66 Figures 2008 AGI-Information Management Consultants May be used for personal purporses only or by libraries associated to dandelon.com network. Jean-Baptiste Hiriart-Urruty Claude Lemarechal Fundamentals of Convex

More information

POLYHEDRAL GEOMETRY. Convex functions and sets. Mathematical Programming Niels Lauritzen Recall that a subset C R n is convex if

POLYHEDRAL GEOMETRY. Convex functions and sets. Mathematical Programming Niels Lauritzen Recall that a subset C R n is convex if POLYHEDRAL GEOMETRY Mathematical Programming Niels Lauritzen 7.9.2007 Convex functions and sets Recall that a subset C R n is convex if {λx + (1 λ)y 0 λ 1} C for every x, y C and 0 λ 1. A function f :

More information

Simplex Algorithm in 1 Slide

Simplex Algorithm in 1 Slide Administrivia 1 Canonical form: Simplex Algorithm in 1 Slide If we do pivot in A r,s >0, where c s

More information

Convex sets and convex functions

Convex sets and convex functions Convex sets and convex functions Convex optimization problems Convex sets and their examples Separating and supporting hyperplanes Projections on convex sets Convex functions, conjugate functions ECE 602,

More information

Alternating Projections

Alternating Projections Alternating Projections Stephen Boyd and Jon Dattorro EE392o, Stanford University Autumn, 2003 1 Alternating projection algorithm Alternating projections is a very simple algorithm for computing a point

More information

COM Optimization for Communications Summary: Convex Sets and Convex Functions

COM Optimization for Communications Summary: Convex Sets and Convex Functions 1 Convex Sets Affine Sets COM524500 Optimization for Communications Summary: Convex Sets and Convex Functions A set C R n is said to be affine if A point x 1, x 2 C = θx 1 + (1 θ)x 2 C, θ R (1) y = k θ

More information

Math 414 Lecture 2 Everyone have a laptop?

Math 414 Lecture 2 Everyone have a laptop? Math 44 Lecture 2 Everyone have a laptop? THEOREM. Let v,...,v k be k vectors in an n-dimensional space and A = [v ;...; v k ] v,..., v k independent v,..., v k span the space v,..., v k a basis v,...,

More information

Lecture: Convex Sets

Lecture: Convex Sets /24 Lecture: Convex Sets http://bicmr.pku.edu.cn/~wenzw/opt-27-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/24 affine and convex sets some

More information

CME307/MS&E311 Theory Summary

CME307/MS&E311 Theory Summary CME307/MS&E311 Theory Summary Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/~yyye http://www.stanford.edu/class/msande311/

More information

CS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 29

CS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 29 CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring 2018 1 / 29 CS 473: Algorithms, Spring 2018 Simplex and LP Duality Lecture 19 March 29, 2018

More information

AMS : Combinatorial Optimization Homework Problems - Week V

AMS : Combinatorial Optimization Homework Problems - Week V AMS 553.766: Combinatorial Optimization Homework Problems - Week V For the following problems, A R m n will be m n matrices, and b R m. An affine subspace is the set of solutions to a a system of linear

More information

C&O 355 Lecture 16. N. Harvey

C&O 355 Lecture 16. N. Harvey C&O 355 Lecture 16 N. Harvey Topics Review of Fourier-Motzkin Elimination Linear Transformations of Polyhedra Convex Combinations Convex Hulls Polytopes & Convex Hulls Fourier-Motzkin Elimination Joseph

More information

Linear Programming Duality and Algorithms

Linear Programming Duality and Algorithms COMPSCI 330: Design and Analysis of Algorithms 4/5/2016 and 4/7/2016 Linear Programming Duality and Algorithms Lecturer: Debmalya Panigrahi Scribe: Tianqi Song 1 Overview In this lecture, we will cover

More information

Introduction to Mathematical Programming IE496. Final Review. Dr. Ted Ralphs

Introduction to Mathematical Programming IE496. Final Review. Dr. Ted Ralphs Introduction to Mathematical Programming IE496 Final Review Dr. Ted Ralphs IE496 Final Review 1 Course Wrap-up: Chapter 2 In the introduction, we discussed the general framework of mathematical modeling

More information

Introductory Operations Research

Introductory Operations Research Introductory Operations Research Theory and Applications Bearbeitet von Harvir Singh Kasana, Krishna Dev Kumar 1. Auflage 2004. Buch. XI, 581 S. Hardcover ISBN 978 3 540 40138 4 Format (B x L): 15,5 x

More information

6 Randomized rounding of semidefinite programs

6 Randomized rounding of semidefinite programs 6 Randomized rounding of semidefinite programs We now turn to a new tool which gives substantially improved performance guarantees for some problems We now show how nonlinear programming relaxations can

More information

60 2 Convex sets. {x a T x b} {x ã T x b}

60 2 Convex sets. {x a T x b} {x ã T x b} 60 2 Convex sets Exercises Definition of convexity 21 Let C R n be a convex set, with x 1,, x k C, and let θ 1,, θ k R satisfy θ i 0, θ 1 + + θ k = 1 Show that θ 1x 1 + + θ k x k C (The definition of convexity

More information

Lecture 3. Corner Polyhedron, Intersection Cuts, Maximal Lattice-Free Convex Sets. Tepper School of Business Carnegie Mellon University, Pittsburgh

Lecture 3. Corner Polyhedron, Intersection Cuts, Maximal Lattice-Free Convex Sets. Tepper School of Business Carnegie Mellon University, Pittsburgh Lecture 3 Corner Polyhedron, Intersection Cuts, Maximal Lattice-Free Convex Sets Gérard Cornuéjols Tepper School of Business Carnegie Mellon University, Pittsburgh January 2016 Mixed Integer Linear Programming

More information

A Course in Convexity

A Course in Convexity A Course in Convexity Alexander Barvinok Graduate Studies in Mathematics Volume 54 American Mathematical Society Providence, Rhode Island Preface vii Chapter I. Convex Sets at Large 1 1. Convex Sets. Main

More information

On the positive semidenite polytope rank

On the positive semidenite polytope rank On the positive semidenite polytope rank Davíd Trieb Bachelor Thesis Betreuer: Tim Netzer Institut für Mathematik Universität Innsbruck February 16, 017 On the positive semidefinite polytope rank - Introduction

More information

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 6

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 6 EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 6 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 19, 2012 Andre Tkacenko

More information

Convexity Theory and Gradient Methods

Convexity Theory and Gradient Methods Convexity Theory and Gradient Methods Angelia Nedić angelia@illinois.edu ISE Department and Coordinated Science Laboratory University of Illinois at Urbana-Champaign Outline Convex Functions Optimality

More information

Linear Programming. Larry Blume. Cornell University & The Santa Fe Institute & IHS

Linear Programming. Larry Blume. Cornell University & The Santa Fe Institute & IHS Linear Programming Larry Blume Cornell University & The Santa Fe Institute & IHS Linear Programs The general linear program is a constrained optimization problem where objectives and constraints are all

More information

Polyhedral Computation and their Applications. Jesús A. De Loera Univ. of California, Davis

Polyhedral Computation and their Applications. Jesús A. De Loera Univ. of California, Davis Polyhedral Computation and their Applications Jesús A. De Loera Univ. of California, Davis 1 1 Introduction It is indeniable that convex polyhedral geometry is an important tool of modern mathematics.

More information

Linear Optimization. Andongwisye John. November 17, Linkoping University. Andongwisye John (Linkoping University) November 17, / 25

Linear Optimization. Andongwisye John. November 17, Linkoping University. Andongwisye John (Linkoping University) November 17, / 25 Linear Optimization Andongwisye John Linkoping University November 17, 2016 Andongwisye John (Linkoping University) November 17, 2016 1 / 25 Overview 1 Egdes, One-Dimensional Faces, Adjacency of Extreme

More information

However, this is not always true! For example, this fails if both A and B are closed and unbounded (find an example).

However, this is not always true! For example, this fails if both A and B are closed and unbounded (find an example). 98 CHAPTER 3. PROPERTIES OF CONVEX SETS: A GLIMPSE 3.2 Separation Theorems It seems intuitively rather obvious that if A and B are two nonempty disjoint convex sets in A 2, then there is a line, H, separating

More information

arxiv: v1 [math.co] 15 Dec 2009

arxiv: v1 [math.co] 15 Dec 2009 ANOTHER PROOF OF THE FACT THAT POLYHEDRAL CONES ARE FINITELY GENERATED arxiv:092.2927v [math.co] 5 Dec 2009 VOLKER KAIBEL Abstract. In this note, we work out a simple inductive proof showing that every

More information

maximize c, x subject to Ax b,

maximize c, x subject to Ax b, Lecture 8 Linear programming is about problems of the form maximize c, x subject to Ax b, where A R m n, x R n, c R n, and b R m, and the inequality sign means inequality in each row. The feasible set

More information

CS522: Advanced Algorithms

CS522: Advanced Algorithms Lecture 1 CS5: Advanced Algorithms October 4, 004 Lecturer: Kamal Jain Notes: Chris Re 1.1 Plan for the week Figure 1.1: Plan for the week The underlined tools, weak duality theorem and complimentary slackness,

More information

What is a cone? Anastasia Chavez. Field of Dreams Conference President s Postdoctoral Fellow NSF Postdoctoral Fellow UC Davis

What is a cone? Anastasia Chavez. Field of Dreams Conference President s Postdoctoral Fellow NSF Postdoctoral Fellow UC Davis What is a cone? Anastasia Chavez President s Postdoctoral Fellow NSF Postdoctoral Fellow UC Davis Field of Dreams Conference 2018 Roadmap for today 1 Cones 2 Vertex/Ray Description 3 Hyperplane Description

More information

Synthetic Geometry. 1.1 Foundations 1.2 The axioms of projective geometry

Synthetic Geometry. 1.1 Foundations 1.2 The axioms of projective geometry Synthetic Geometry 1.1 Foundations 1.2 The axioms of projective geometry Foundations Def: A geometry is a pair G = (Ω, I), where Ω is a set and I a relation on Ω that is symmetric and reflexive, i.e. 1.

More information

MATH 890 HOMEWORK 2 DAVID MEREDITH

MATH 890 HOMEWORK 2 DAVID MEREDITH MATH 890 HOMEWORK 2 DAVID MEREDITH (1) Suppose P and Q are polyhedra. Then P Q is a polyhedron. Moreover if P and Q are polytopes then P Q is a polytope. The facets of P Q are either F Q where F is a facet

More information

CS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 36

CS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 36 CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring 2018 1 / 36 CS 473: Algorithms, Spring 2018 LP Duality Lecture 20 April 3, 2018 Some of the

More information

Convex sets and convex functions

Convex sets and convex functions Convex sets and convex functions Convex optimization problems Convex sets and their examples Separating and supporting hyperplanes Projections on convex sets Convex functions, conjugate functions ECE 602,

More information

Convex Optimization M2

Convex Optimization M2 Convex Optimization M2 Lecture 1 A. d Aspremont. Convex Optimization M2. 1/49 Today Convex optimization: introduction Course organization and other gory details... Convex sets, basic definitions. A. d

More information

Search and Intersection. O Rourke, Chapter 7 de Berg et al., Chapter 11

Search and Intersection. O Rourke, Chapter 7 de Berg et al., Chapter 11 Search and Intersection O Rourke, Chapter 7 de Berg et al., Chapter 11 Announcements Assignment 3 web-page has been updated: Additional extra credit Hints for managing a dynamic half-edge representation

More information

4 Integer Linear Programming (ILP)

4 Integer Linear Programming (ILP) TDA6/DIT37 DISCRETE OPTIMIZATION 17 PERIOD 3 WEEK III 4 Integer Linear Programg (ILP) 14 An integer linear program, ILP for short, has the same form as a linear program (LP). The only difference is that

More information

Applied Integer Programming

Applied Integer Programming Applied Integer Programming D.S. Chen; R.G. Batson; Y. Dang Fahimeh 8.2 8.7 April 21, 2015 Context 8.2. Convex sets 8.3. Describing a bounded polyhedron 8.4. Describing unbounded polyhedron 8.5. Faces,

More information

Geometric transformations assign a point to a point, so it is a point valued function of points. Geometric transformation may destroy the equation

Geometric transformations assign a point to a point, so it is a point valued function of points. Geometric transformation may destroy the equation Geometric transformations assign a point to a point, so it is a point valued function of points. Geometric transformation may destroy the equation and the type of an object. Even simple scaling turns a

More information

Discrete Optimization 2010 Lecture 5 Min-Cost Flows & Total Unimodularity

Discrete Optimization 2010 Lecture 5 Min-Cost Flows & Total Unimodularity Discrete Optimization 2010 Lecture 5 Min-Cost Flows & Total Unimodularity Marc Uetz University of Twente m.uetz@utwente.nl Lecture 5: sheet 1 / 26 Marc Uetz Discrete Optimization Outline 1 Min-Cost Flows

More information

Lecture Notes 2: The Simplex Algorithm

Lecture Notes 2: The Simplex Algorithm Algorithmic Methods 25/10/2010 Lecture Notes 2: The Simplex Algorithm Professor: Yossi Azar Scribe:Kiril Solovey 1 Introduction In this lecture we will present the Simplex algorithm, finish some unresolved

More information

Decision Aid Methodologies In Transportation Lecture 1: Polyhedra and Simplex method

Decision Aid Methodologies In Transportation Lecture 1: Polyhedra and Simplex method Decision Aid Methodologies In Transportation Lecture 1: Polyhedra and Simplex method Chen Jiang Hang Transportation and Mobility Laboratory April 15, 2013 Chen Jiang Hang (Transportation and Mobility Decision

More information

Outline. Combinatorial Optimization 2. Finite Systems of Linear Inequalities. Finite Systems of Linear Inequalities. Theorem (Weyl s theorem :)

Outline. Combinatorial Optimization 2. Finite Systems of Linear Inequalities. Finite Systems of Linear Inequalities. Theorem (Weyl s theorem :) Outline Combinatorial Optimization 2 Rumen Andonov Irisa/Symbiose and University of Rennes 1 9 novembre 2009 Finite Systems of Linear Inequalities, variants of Farkas Lemma Duality theory in Linear Programming

More information

Division of the Humanities and Social Sciences. Convex Analysis and Economic Theory Winter Separation theorems

Division of the Humanities and Social Sciences. Convex Analysis and Economic Theory Winter Separation theorems Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Topic 8: Separation theorems 8.1 Hyperplanes and half spaces Recall that a hyperplane in

More information

The Matrix-Tree Theorem and Its Applications to Complete and Complete Bipartite Graphs

The Matrix-Tree Theorem and Its Applications to Complete and Complete Bipartite Graphs The Matrix-Tree Theorem and Its Applications to Complete and Complete Bipartite Graphs Frankie Smith Nebraska Wesleyan University fsmith@nebrwesleyan.edu May 11, 2015 Abstract We will look at how to represent

More information

CS-9645 Introduction to Computer Vision Techniques Winter 2019

CS-9645 Introduction to Computer Vision Techniques Winter 2019 Table of Contents Projective Geometry... 1 Definitions...1 Axioms of Projective Geometry... Ideal Points...3 Geometric Interpretation... 3 Fundamental Transformations of Projective Geometry... 4 The D

More information

College of Computer & Information Science Fall 2007 Northeastern University 14 September 2007

College of Computer & Information Science Fall 2007 Northeastern University 14 September 2007 College of Computer & Information Science Fall 2007 Northeastern University 14 September 2007 CS G399: Algorithmic Power Tools I Scribe: Eric Robinson Lecture Outline: Linear Programming: Vertex Definitions

More information

Lecture 3: Totally Unimodularity and Network Flows

Lecture 3: Totally Unimodularity and Network Flows Lecture 3: Totally Unimodularity and Network Flows (3 units) Outline Properties of Easy Problems Totally Unimodular Matrix Minimum Cost Network Flows Dijkstra Algorithm for Shortest Path Problem Ford-Fulkerson

More information

Lecture 2: August 29, 2018

Lecture 2: August 29, 2018 10-725/36-725: Convex Optimization Fall 2018 Lecturer: Ryan Tibshirani Lecture 2: August 29, 2018 Scribes: Adam Harley Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have

More information

Lecture 4: Linear Programming

Lecture 4: Linear Programming COMP36111: Advanced Algorithms I Lecture 4: Linear Programming Ian Pratt-Hartmann Room KB2.38: email: ipratt@cs.man.ac.uk 2017 18 Outline The Linear Programming Problem Geometrical analysis The Simplex

More information

Boundary Curves of Incompressible Surfaces

Boundary Curves of Incompressible Surfaces Boundary Curves of Incompressible Surfaces Allen Hatcher This is a Tex version, made in 2004, of a paper that appeared in Pac. J. Math. 99 (1982), 373-377, with some revisions in the exposition. Let M

More information

6.854 Advanced Algorithms. Scribes: Jay Kumar Sundararajan. Duality

6.854 Advanced Algorithms. Scribes: Jay Kumar Sundararajan. Duality 6.854 Advanced Algorithms Scribes: Jay Kumar Sundararajan Lecturer: David Karger Duality This lecture covers weak and strong duality, and also explains the rules for finding the dual of a linear program,

More information

Polyhedral Computation Today s Topic: The Double Description Algorithm. Komei Fukuda Swiss Federal Institute of Technology Zurich October 29, 2010

Polyhedral Computation Today s Topic: The Double Description Algorithm. Komei Fukuda Swiss Federal Institute of Technology Zurich October 29, 2010 Polyhedral Computation Today s Topic: The Double Description Algorithm Komei Fukuda Swiss Federal Institute of Technology Zurich October 29, 2010 1 Convexity Review: Farkas-Type Alternative Theorems Gale

More information

3D Computer Vision II. Reminder Projective Geometry, Transformations

3D Computer Vision II. Reminder Projective Geometry, Transformations 3D Computer Vision II Reminder Projective Geometry, Transformations Nassir Navab" based on a course given at UNC by Marc Pollefeys & the book Multiple View Geometry by Hartley & Zisserman" October 21,

More information